This may be a minor, but a widely requested improvement on the device screen: all SNMP community and SNMPv3 passphrases are now obfuscated in the UI.

This complies with widespread company policies to never display passwords in a web UI.

See screenshot below

Kentik now supports collection of the NetFlow and IPFIX fields for position 3 MPLS Label in the label stack , which we previously not collected from the received flows. The related fields is shown in tables below:

NetFlow v9 VLAN fields:

Resource: https://www.cisco.com/en/US/technologies/tk648/tk362/technologies_white_paper09186a00800a3db9.html

IPFIX VLAN fields:

Resource: https://www.iana.org/assignments/ipfix/ipfix.xhtml

This field is collected from NetFlow/IPFIX protocols and stored in the Kentik’s MPLS Label 3 and MPLS Label 3 EXP dimensions.

The support is available in kproxy starting from version v7.37.0. The example of the Data Explorer query is shown below:

When digging around in enriched Network Telemetry data, you'll find yourself noticing a bump, or a trough in any of the displayed time series and think to yourself: 

"Hmmm... is that peak an anomaly, or did it behave the same last day|week|... at the same time?"

Ask yourself no more, "Compare over previous period" is here!

In a nutshell, this new Data Explorer feature lets you look at the same time range you are currently looking at, but in the past, and help you visualize variations between each of your query result's time series within the same time range in the past.

This feature comes with a streamlined redesign of the Time Range query panel in Data Explorer.

Hitting the Compare over previous period switch will unveil options to be able to compare the current time range, to the same one at a configurable time in the past, such as: "this hour with the same hour yesterday, or a week ago..."

Upon selecting this option, your data-table will now include two new tabs:

• A Previous Period tab showing you the TopN time-series for the same time range in the past
• A Comparison Summary tab, outlining in an sortable fashion such useful insights such as: 
  • previous current rank for the time series vs previous rank,  
  • rank variation for the time series
  • and the percentage of change (sortable) in the selected metrics between Previous and Current periods

From any of these 3 tabs, you will then be able to select any time series and look at a combined line chart displaying both the current and previous period for the selected time series.

Switching between Current Period and Previous Period will display the full set of time series for either periods - always leveraging the same convention:

• Plain Line:        Current Period
• Dashed Line:   Previous Period

Now go ahead, play around with the feature and let us know what you think of it.
As always, you'll always find in Kentik Product Management a friendly ear to suggest improvements to this feature (and any other for that matter), do let us know!

Kentik’s Data Explorer is enriched with additional query options which influence how time series will be visualized and calculated. These options are located in the Advanced section of the query panel as shown on the screenshot below

The Query execution performed against the Kentik Data Engine consist of the two steps:

1. Initial flow aggregation into samples - Flow Aggregation which is performed on the configurable time Window. For all the query (filter) matching flow’s the summation of bytes or packets will be performed, grouped (broken down) by all selected Flow Dimension. The resulting flow aggregation samples will always represent an average Bit Rate (Packet Rate) across all the summed flows, because the sum of all bytes/packets is divided by the Flow Aggregation Window.
2. Construction of the Time Series - From all the Flow Aggregation samples returned from the initial aggregation, the Time Series will be constructed for selected Top N series by volume. Time Series Buckets are constructed by aggregating multiple Flow Aggregation samples using the selected Aggregation function.

Additional Data Explorer Time Series query options consist of the following 3 options:

• Flow Aggregation Window: This configures the time window which is used for the initial Flow Aggregation into samples. Available options are 1, 5, 10, 20 minutes for Full Dataseries and 1, 6, 12 hours for Fast data series. If the Auto is selected, the behavior will be the same as the usual Data Explorer behavior so far. current Further limitations apply:
  • 1 minute available for up to 3 day wide queries
  • 5 minutes available for up to 14 day wide queries
  • 10 minutes available for up to 30 day wide queries
  • 20 minutes available for up to 60 day wide queries
  • no limitation for the use of Fast dataseries.
• Time Series Bucket Width: Time Window used for the construction of the Time Series from Flow Aggregation Samples. The available options are 10 min, 20 min, 30 min, 1h, 6h, 12h, 1 day, 1 week.
  • This configuration option only takes effect if the Time Series Bucket Aggregation is not "None", 
  • The value always has to be larger than the selected Flow Aggregation Window.
  • The option “Auto” always maps to 1h buckets.
• Time Series Bucket Aggregation:This option defines the function which will be performed while aggregating multiple Flow Aggregation Samples into Time Series Buckets. The available options are:
  • None - does not perform any aggregation to the Time Series Buckets, so only Flow Aggregation Samples are shown
  • Average - Calculates the average value of the Flow Aggregation samples in each Time Series Bucket
  • Count - Counts number of Flow Aggregation sample occurrences within the Time Series Bucket
  • Sum - Calculates the sum of all the Flow Aggregation samples in each Time Series Bucket
  • Maximum - Selects the Maximum value of the Flow Aggregation samples in each Time Series Bucket
  • Minimum - Selects the Minimum value of the Flow Aggregation samples in each Time Series Bucket

Connectivity Costs is one of the workflows in the Edge module that seems to be the most successful with our users. It allows users to centralize the financial modeling of their Transit, Peering and IX contracts in one place, and track costs for each of these throughout the month, and every past month.
For a refresher on this workflow, take a minute to read this blog post from 2021 on Why and How to Track Connectivity Costs.

Today, we add quite a few highly anticipated features to it, read on!

Landing page calendar drawer now collapsible

The drawer with the invoice and contract expiry schedule is now collapsible, which was a common request from users whose contract dates are always the same: it can now be hidden, and the preference will stick for each user.

Dynamic Interface Selection

In the previous iteration of Connectivity Costs, interfaces has to be manually selected and added to the relevant Cost Group in order to bind them to a contract and financial model. For networks with a vast footprint, these external facing interfaces keep coming in and out of the picture, and users demanded to be able to manage this part of the configuration more efficiently. This is now done: from the already popular Capacity Planning workflow, we have ported the Dynamic and Static Interface Selector module into Connectivity Costs.

In addition, we've improved it significantly to make things even easier to maintain - amongst others, users can now assign interfaces in a cost group based on:

• Interface Name and Description regular expressions
• Device Names, including regular expressions (comes in handy if a portion of your device names is Site or Function related - think edge-01_ams01 for an Edge Router in Amsterdam)
• Device Labels, allowing you to exclusively focus on specifically tagged devices
• Last but not least: users can now choose between Logical Only interfaces, Physical Only interfaces, or both - for instance, depending on which of the physical or logical part of an Aggregate logical interface your SNMP accounting is going to come from

Interface or Group level additional costs expressed in any currency

Users can add Cost Group level (monthly or yearly) costs to their cost groups, but can add Interface Level costs as well. Until now, the currency of the parent Cost Group would be used to evaluate these additional costs into the daily computations.

With this release, users can now select the currency for both. Here's an example of where it comes in handy: optional interface charges are often used to document cross-connects. Our customers also tend to use local providers in whichever area they are sourcing bandwidth from. This often results in Bandwidth as documented in the Cost Group financial model being expressed in the local currency, and data-center cross connects (usually from a global data-center contract) expressed in another currency. Problem solved !

While at it, we made the Interface-Level UI for adding/editing costs more usable, see for yourself - in the example below, the cost group is in dollars and the interface charges for the cross connect in Euros. 

New Computation Methods

Based on feedback from our users, we also added a few new computation method flavors as to how our engine should compute monthly invoices. To the existing Peak of Sums and Sum of Peaks we added two new models:

• Peak of Sums (Multiple interface sets)
• Peak of Sums (Max of In and Out)

And to make sure our users select the right one when configuring a cost group, we also gave better in-UI explanations of how each computation method works in the back-scenes:

What comes next ?

In future quarters, we are going to both extend the workflow to deal with non-edge costs, and also create functionality for users who have documented their costs into the workflow to be able to price any slice of their traffic.

Stay tuned, and in the meantime do let us know what you'd like to see next in this workflow !

Kentik now supports SNMP collection of the CPU and Memory utilization for F5 BIG-IP devices. Some of the F5 BIG-IP devices do not support flow export features, which means that the collection of the SNMP metrics would require use of Kentik's kproxy with the “bootstrap_devices” option.

The general behavior of the Kentik’s SaaS ingest or Kentik’s Kproxy is that the SNMP metrics collection would only start once the flows are received from the certain device. This behavior can be limiting in some cases and can be changed by the use of the Kproxy’s Bootstrap devices feature. This feature is enable with the use of the-bootstrap_devices command line argument. This argument should provide the comma-separated list of devices’ IDs. For those devices, the Kproxy will start the SNMP metrics collection immediately, without waiting to receive devices’ flows. More information about the Kproxy CLI arguments can be found in our Knowledge Base article: https://kb.kentik.com/v0/Bd04.htm#Bd04-kproxy_CLI_Reference

The Kentik’s Kproxy will determine that the discovered device is F5 BIG-IP by looking for the “big-ip” string in the well-known SNMP sysDescr OID.

The Kproxy would monitor the following 4 Device “Components”, with the use of the SNMP OIDs described below:

• Name “Global”:
  • MemoryTotal [bytes] - OID Name: sysGlobalHostMemTotal, OID: .1.3.6.1.4.1.3375.2.1.1.2.20.2.0
  • MemoryUsed [bytes] - OID Name: sysGlobalHostMemUsed, OID: .1.3.6.1.4.1.3375.2.1.1.2.20.3.0
  • CPU [percentage] - OID Name: sysGlobalHostCpuUsageRatio5m, OID: .1.3.6.1.4.1.3375.2.1.1.2.20.37.0
• Name “TMM”:
  • MemoryTotal [bytes]- OID Name: sysStatMemoryTotal, OID: .1.3.6.1.4.1.3375.2.1.1.2.1.44.0
  • MemoryUsed [bytes] - OID Name: sysStatMemoryUsed, OID: .1.3.6.1.4.1.3375.2.1.1.2.1.45.0
  • CPU - value will be 0
• Name “Other”:
  • MemoryTotal [bytes] - OID Name: sysGlobalHostOtherMemoryTotal, OID: .1.3.6.1.4.1.3375.2.1.1.2.20.44.0
  • MemoryUsed [bytes] - OID Name: sysGlobalHostOtherMemoryUsed, OID: .1.3.6.1.4.1.3375.2.1.1.2.20.45.0
  • CPU - value will be 0
• Name “Swap”:
  • MemoryTotal [bytes]- OID Name: sysGlobalHostSwapTotal, OID: .1.3.6.1.4.1.3375.2.1.1.2.20.46.0
  • MemoryUsed [bytes] - OID Name: sysGlobalHostSwapUsed, OID: .1.3.6.1.4.1.3375.2.1.1.2.20.47.0
  • CPU - value will be 0

For each component  MemoryFree and MemoryUtilization are calculated from collected MemoryTotal and MemoryUsed metrics. Each component use standard Uptime which is collected at the device level from sysUpTime OID.

The support is available in kproxy starting from version v7.36.0. The example of the Data Explorer query is shown below:

The list of supported devices from which Kentik can collect CPU and Memory utilization is growing. Kentik now supports SNMP collection of the CPU and Memory utilization for Palo Alto Network devices. 

Palo Alto devices are using the standard HOST-RESOURCES-MIB, for CPU and Memory usage. The Kentik’s ingest/kproxy will determine that the discovered device is from Palo Alto by looking for the “palo alto” string in the well-known SNMP sysDescr OID.

For CPU:

• Component name is provided in the OID: hrDeviceDescr: .1.3.6.1.2.1.25.3.2.1.3.index
• CPU utilization is provided in the OID: hrProcessorLoad: .1.3.6.1.2.1.25.3.3.1.2.index

For Memory:

• Component name is provided in the OID: hrDeviceDescr: .1.3.6.1.2.1.25.3.2.1.3.index
• Memory utilization is provided from the SNMP table: hrStorageTable: .1.3.6.1.2.1.25.2.3

The support is available in kproxy starting from version v7.36.0. The example of the Data Explorer query is shown below:

Kentik now supports collection of the NetFlow and IPFIX fields for source/destination VLAN, which we previously not collected from the received flows. 

The related VLAN fields are shown in tables below:

NetFlow v9 VLAN fields

Resource: https://www.cisco.com/en/US/technologies/tk648/tk362/technologies_white_paper09186a00800a3db9.html

IPFIX VLAN fields

Resource: https://www.iana.org/assignments/ipfix/ipfix.xhtml

These two fields are collected from NetFlow/IPFIX protocols and stored in the Kentik’s Source VLAN and Destination VLAN dimensions.

The support is available in kproxy starting from version v7.36.0. The example of the Data Explorer query is shown below:

Data Explorer allows you to drill into your traffic flows for better understanding of what is going on in your network. Data Explorer uses different dimensions and filtering to show you the flows of interest. 

ENI Entity name and ENI Entity type were added to the list of supported dimensions on AWS. This allows users to see and filter traffic to and from AWS services such as Load Balancers and VPC endpoints. 

By popular demand, we've added a new dimension named Site Market in Kentik Portal. It allows our users to group sites within Markets and use this new dimension in all parts of the product to filter, or group by.

When is this feature useful?

We've heard many times the situation where having multiple PoPs in the same metropolitan area makes it difficult to pivot Guided Mode dashboards around.
For instance, it is not uncommon that large networks have multiple PoPs around Los Angeles, say LAX1, LAX2, ..., LAXN. Most of the time these networks want to consider all of these as a single Metro: enters Site Markets, slap a "Los Angeles Metro" Site Market Label on all of these and you can now use it as a Dashboard Guide Mode parameter !

With its Ultimate Exit Site Market, and its special "Site Market doesn't equal Ultimate Exit Site Market" large networks are going to be able to perform useful queries telling them when traffic form one of their peers isn't hot-potato'ed and carried at higher cost over long distances. 

How can I configure the feature?

You can navigate to Settings > Manage Sites where you'll notice the new Site Market attribute. You can create a new Site Market on the spot or pick an already existing one.

From the Manage Sites screen, you can even Bulk assign Site Market to sites, which obviously comes in handy the first time you discover the feature.

When first configuring your Site Markets, you can also leverage the filters on the Site screen to quickly select all those without a Site Market, which together with the aforementioned feature really helps moving faster.

The Manage Site Markets screen

Additionally, Site Markets now have their own Settings screen, which can be found here:

Custom Geos vs Site Markets

There is a misleading parallel between Custom Geos and Site Markets, so let's take a look at these:

• Custom Geos: map a Source Geo and Destination Geobased on
  • GeoIP data from Source IP and Destination IP
    and
    Custom Geo to Country mapping from the Custom Geo user configuration
• Site Markets: map a Site Market based on
  • Site to Site Market mapping from the Site Market user configuration

The main differences are:

• Site Market is a superset of Sites when Custom Geo is a superset of countries
• Site Market is not directional (green column in Data Explorer's dimension selector), while Custom Geo exists in Source and Destination variants

Using Site Markets

Site Market dimensions are available in both Data Explorer Dimensions and Filters, and come with their "Ultimate Exit" pendent: Ultimate Exit Site Market is mapped from Ultimate Exit Site using the users' Site Market groupings from the configuration screen.

Finally, the Site Market filter comes with a very useful operand saying "Site Market doesn't equal Ultimate Exit Site Market" which will help networks identify traffic that does not stay local (and is therefore more expensive to transit - we trust this feature will be very useful to large Service Provider networks.

Enjoy and let us know what you think about the feature!