Flow Ingest: Support for Flow timestamps
Kentik now supports the collection of the NetFlow and IPFIX timestamp fields (starting kproxy v7.38.0). The additional 3 fields are available to be viewed in the Raw Flow Viewer:
Flow Start
- Timestamp of the flow startFlow End
- Timestamp of the flow endDuration
- Calculated duration of the flow asFlow Start - Flow End
NetFlow v5 and v9
In the case of the NetFlow v5 and v9, Start and End Flow timestamps are calculated using System Uptime
and Unix Seconds
fields from the NetFlow header and the following fields from Flow records:
Field Type | Value | Length (bytes) | Description |
---|---|---|---|
LAST_SWITCHED | 21 | 4 | System uptime at which the last packet of this flow was switched |
FIRST_SWITCHED | 22 | 4 | System uptime at which the first packet of this flow was switched |
IPFIX
In the case of the IPFIX, the timestamps are determined in the following two ways:
Using
flowStartSysUpTime
,flowEndSysUpTime
andsystemInitTimeMilliseconds
fields:ID Name Type Description Units 21 flowEndSysUpTime unsigned32 The relative timestamp of the last packet of this Flow. It indicates the number of milliseconds since the last (re-)initialization of the IPFIX Device (sysUpTime). sysUpTime can be calculated from systemInitTimeMilliseconds. milliseconds 22 flowStartSysUpTime unsigned32 The relative timestamp of the first packet of this Flow. It indicates the number of milliseconds since the last (re-)initialization of the IPFIX Device (sysUpTime). sysUpTime can be calculated from systemInitTimeMilliseconds. milliseconds Both fields in the description refer to the additional field which is equivalent to the SysUpTime:
ID Name Type Description Units 160 systemInitTimeMilliseconds dateTimeMilliseconds The absolute timestamp of the last (re-)initialization of the IPFIX Device. milliseconds Using some of the IPFIX-specific fields for the flow start and flow end timestamps. Not all device’s IPFIX implementation used these fields, but if they are present, they are preferred and used:
ID Name Type Description Units 150 flowStartSeconds dateTimeSeconds The absolute timestamp of the first packet of this Flow. seconds 151 flowEndSeconds dateTimeSeconds The absolute timestamp of the last packet of this Flow. seconds 152 flowStartMilliseconds dateTimeMilliseconds The absolute timestamp of the first packet of this Flow. milliseconds 153 flowEndMilliseconds dateTimeMilliseconds The absolute timestamp of the last packet of this Flow. milliseconds 154 flowStartMicroseconds dateTimeMicroseconds The absolute timestamp of the first packet of this Flow. microseconds 155 flowEndMicroseconds dateTimeMicroseconds The absolute timestamp of the last packet of this Flow. microseconds 156 flowStartNanoseconds dateTimeNanoseconds The absolute timestamp of the first packet of this Flow. nanoseconds 157 flowEndNanoseconds dateTimeNanoseconds The absolute timestamp of the last packet of this Flow. nanoseconds 158 flowStartDeltaMicroseconds unsigned32 This is a relative timestamp only valid within the scope of a single IPFIX Message. It contains the negative time offset of the first observed packet of this Flow relative to the export time specified in the IPFIX Message Header. microseconds 159 flowEndDeltaMicroseconds unsigned32 This is a relative timestamp only valid within the scope of a single IPFIX Message. It contains the negative time offset of the last observed packet of this Flow relative to the export time specified in the IPFIX Message Header. microseconds
sFlow
sFlow is packet sampling technology, which means that there is no flow, hence no flow start or flow end time.
sFlow also does not contain any timestamp, so the behavior is the following:
- Flow Start: save the time of arrival/processing of the sFlow packet
- Flow End - same as Flow Start
- Duration - 0
Raw Flow Viewer
In the Raw Flow Viewer, these additional flow timestamps need to be selected as Dimensions to be shown in the output, as shown below: