In our November 2017 Product Update we mentioned that we have added the ability to start a manual mitigation as opposed to triggering one off an alert. We’ve now implemented this capability as an API and added it to the extensive list of REST APIs we’ve made available to programmatically manage Kentik Detect.

Our KB article contains more information and be sure to check out our API tester, which will help guide you on using this new method.

Custom Dimensions (covered in this KB article) is one of the more powerful features of Kentik Detect because it enables you to add custom columns to the traffic flow records in your Kentik database, and to populate those custom fields based on a match for a given value in the ingested flow fields. You can use those columns for any number of purposes, including overlaying business information on top of network information so you can run powerful analytics based on the two sets of data.

We recently added the ability to create ranges when defining populators that match on Port and ASN. To define a port range for a populator, first go to Admin » Custom Dimensions and click on a listed dimension to open the Edit Dimension dialog. Choose the Populators tab, then click the Add Populator button, which opens the Add Populator dialog. On the IP Matching tab, define a range in the Port field as shown below. When you add the populator a match will result from any number in the range, not just a single value.

Another new View Type feature we first announced last month was Geo HeatMaps. Initially available for queries whose group-by dimension was source or destination country or site, HeatMaps have now been extended to cover regions and cities as well.

As with countries, the mapping depends on the traffic attributable to the sites within a given region or city, which means that you must first use the Admin » Sites page to enter addresses for each of your sites (see instructions for Editing a Site in our KB).

Region Heatmap

A region HeatMap shows the total network traffic, inbound plus outbound, by each region (e.g. a state in the United States) based on the addresses specified specified for sites. In Data Explorer, choose the group-by dimensions Destination Region and Destination Country, and chose Geo HeatMap from the View Type selector at upper right of the display area. When you run the query, the resulting visualization should look similar to the below. For even more information, hover over a bubble (colored circle) on the map to open a pop-up, which states the location’s latitude, longitude, and total traffic.

City Heatmap

The city heatmap is very similar to the region heatmap, but it’s more granular because it drills down to individual cities. To get a visualization similar to the one below, add Destination City to the group-by dimensions we already had (Region and Country).

Like the other visualizations in Data Explorer, heatmaps can be turned into panels on a Dashboard so users can easily monitor traffic on a geographical basis.

Data Explorer’s new Gauge visualization displays the current value of the primary metric, which is set with the Metrics setting in sidebar’s Query pane. In the Gauge this number is set against a background that changes color depending on the metric’s current value in relation to user-defined “brackets” (ranges). Assigning colors to different value ranges allows a gauge to show at a glance whether the metric’s current value is fine or problematic.

Before configuring a gauge in Data Explorer, set your dimensions, filters, time, and devices as you would for a query of any other view type. Then choose Gauge from the View Type selector at the upper right of the display area. A new Bracketing pane will appear in the Sidebar; click it to open the Bracketing Options dialog.

In the dialog (shown below), you first specify the basis on which the brackets are defined (Bracketing Type), such as static ranges, percentages, or percentiles. Then you specify the current value that will be displayed and that will be evaluated to determine which bracket it’s in (and thus what the background color will be). The Bracketing Value switch determines whether “current” means the most recent datapoint of the primary metric or the primary metric’s value over query time-range (e.g. average Mbps for the last hour). Lastly you define the ranges. By default there are two, but you can have up to five ranges with different colors (e.g. orange for a range that’s slightly above normal and red for a critical peak). For more in-depth information on Bracketing, head on over to our Bracketing Pane Settings article in the KB.

Once you run the query you can add it to a dashboard as a panel. This is a really powerful tool, as you can see below where there’s a set of such Guage panels, each showing a different metric. With the dashboard set to Live Update you get really easy-to-grasp indicators that update as network traffic changes. Keep an eye on our Product Update, Knowledge Base, and blog sites for more upcoming features and enhancements around Dashboards and Bracketing.

Another new feature in the Data Explorer is the ability to customize the metrics that are shown in the table as well as those shown on the +Y and -Y axis. By default, the Data Explorer will show a single metric with table columns for the Average, 95th Percentile, Max, and Last Datapoint calculations for that metric. Querying with multi-metric enables you to look at things like ingress and egress on the same graph or, as we’ll see in the following example, Bits/s compared to Packets/s.

To enable the new multi-metric feature, click on the Customize Metrics button in the query section of the sidebar, which opens the Metrics dialog. The dialog (shown below) has the following sections that are important to note:

1. Metric Library – A list of available metrics by category that allows you to choose (using checkboxes) the metrics you’d like to see as a column in the table returned from the query.
2. Selected Metrics – A list of the metrics that that you’ve chosen from the library.
3. Display & Sort Metrics – A set of dropdown menus that determine the primary and secondary metric by which the results in the table will be sorted. Note that if only a single family of metrics it selected, you will only have a Primary Display & Sort Metric option.

A query run with the settings shown above would gives us a graph (shown below) with the three metrics chosen from the Bits/s category on the +Y (top) axis and the three metrics chosen from the Packets/s category on the -Y (bottom) axis. In the table (only the heading of which are shown in the image) we have columns for all six of the metrics. The bits/s tab is sorted by our primary metric (Average bits/s) while our packets/s tab will show the same data sorted by our secondary metric (Average packets/s).

Filter-based dimensions allow Data Explorer to represent — as plots on graphs and rows in tables — a number of time-series that are each user-defined with filters. For example, you might want to compare HTTP, HTTPs, DNS (TCP), and DNS (UDP) traffic.

If you queried for total traffic with those filter parameters, you’d return their total cumulative traffic plotted as one line with a corresponding single row in the table. But with filter-based dimensions, you can see each as a Series broken out into its own plot and row. Note that a query can’t mix these Series with regular (“preset”) group-by dimensions, and you can only have one filter-based dimension at a time. So any dimensions that you already have in the group-by selector will be overwritten when you save a filter-based dimension.

To use filter-based dimensions, click in the Group By Dimensions field in the Query pane of the sidebar, then click on the Filter-Based tab in the resulting dialog. Use the switch at top to enable filter-based dimensions, after which you’ll see a form similar to the below. Using our example, you’d define a series for HTTP, then add series for HTTPs, DNS (TCP), and DNS (UDP).

When you save the settings any dimensions that were specified in the dimension selector of the Query pane will be overwritten with the new filter-based dimension. Run the query in Explorer and you’ll get results that look something like the below, with each series plotted in the graph and represented as a row in the table.

In December 2017, we announced our new Alerting Scoreboard, which makes it easier to see at a glance the things that most need your attention. Now we’ve added the ability to include one or more scoreboards as panels on a dashboard. This type of panel is especially useful for Dashboards designed to get insight into attacks or changes in the network environment.

To add a scoreboard from an existing Dashboard, first, click Edit Mode, then click the Alert Policy Scoreboard button in the panel at the top (under “Select visualization type…”). In the resulting Add Dashboard Panel dialog, you’ll see the controls for configuring the scoreboard grid: Dimension (X axis), Policy (Y axis), etc. Make your grid settings, set your thresholds for inclusion of various levels of alarms, and give the panel a title before saving with the Add Dashboard Panel button.

If you’re a regular reader of these updates, you’ll recall that we introduced Guided Mode Dashboards back in our November Product update. We’ve recently added the ability to filter a panel on this type of dashboard using our BGP Ultimate Exit dimensions for Site and Device.

While this example made use of Sites, a very similar workflow is available for Devices.

Dashboard Navigation reduces the time spent drilling down to root causes by enabling users to navigate from a given dashboard panel directly to another dashboard that’s related to the same use case. Kentik will be rolling out a library of these ready-made dashboard workflows, but power users can go ahead and create workflows today to match their needs. Note that the general creation and editing of Dashboards is covered in the Dashboards article of our Knowledge Base, and that an upcoming post on our blog will provide more information on how to use these new dashboard features.

Creating a nested dashboard begins with the settings for the dashboard panel from which you will be navigating. In the following example, we’ll update an existing dashboard with existing panels. However, the process to create a new dashboard or panel would be very similar.

1. From the Dashboard page (Dashboards on the main portal navbar), open the Dashboard to which you want to add nesting.
2. 
   When the dashboard opens, click the Edit Mode button at upper right.
3. 
   A round blue Navigate To button will now appear on the right edge of each of the dashboard’s panels (shown below). Because no destination has been set yet, the tooltip for these buttons will say “No Dashboard Destination Set.” Click on the button for the panel to which you wish to add a nested dashboard.
4. The Edit Dashboard Panel dialog will open with the lower section on the Navigate To tab (shown above right). Set the controls as follows:
   – Turn on the Enable Dashboard Navigation switch.
   – In the resulting Destination Dashboard selector, choose the dashboard to which this panel should navigate.
   – The Destination Dashboard Settings controls will appear. Choose how the destination dashboard will be affected by the devices, time-range, and filters of the current dashboard.

December 2017 comes with a heavy delivery of new features, this month we're adding:

• the Network Classification concept and dimensions (tldr: what's in your network, what's not)
• additional Interface Classification connectivity types
• Heatmap visualizations
• New Host Metrics
• A brand new Alerting Scoreboard
• IPv6 compatibility for Radware mitigations

Network Classification

We are constantly looking for ways to make Kentik Detect easier to use, especially for users that aren’t highly network savvy. One such improvement is Network Classification, which enables the following capabilities for our users:

• Network Directionality: Group traffic based on the direction from which it enters the network and to which it leaves.
• Host Directionality: Group host traffic captured by kprobe based on the direction it is flowing.

Network Classification Dimensions

The capabilities listed above are supported by four new dimensions that can be applied to each flow:

• Traffic Origination: This dimension indicates whether the source for a given flow is inside or outside of your network.
• Traffic Termination: This dimension indicates whether the destination for a given flow record is inside or outside of the network.
• Host Direction: When the flow record has been generated on a host, this dimension indicates whether the direction of traffic is into or out of that host.
• Traffic Profile: Derived from Traffic Origination and Traffic Termination, this dimension categorizes traffic into one of the following directionalities, which are illustrated in the graphic below:
  – Through: Traffic coming from outside the network and terminating outside the network.
  – Ingress: Traffic coming from outside the network and terminating inside the network.
  – Internal: Traffic originating and terminating inside the network.
  – Egress: Traffic coming from inside the network and terminating outside the network.

The dimensions described above are available throughout Kentik Detect as:

• Group-by Dimensions
• Filter match criteria
• Alert keys (dimensions of an Alert Policy)
• Alert filter match criteria

Network Directionality Use Case

One interesting use case involves using Network Directionality to investigate spikes in traffic. For example, in the left graph below from the Kentik portal’s Data Explorer, we can see a big spike in flows to a customer called Pear, Inc.

In the corresponding Data Explorer table (below the graph; not shown), we can dig deeper into the data by clicking the Action menu at the right of the row for Pear, Inc. We choose Show By to open the Show By Dimensions dialog, then choose one of our new Network Classification dimensions, Traffic Origination (listed under Source). After closing the dialog by clicking the Show By Selected Dimensions button, we re-run the query. We can now see (right graph) that the spike is made up of traffic that originated outside of our network. If we wanted to continue digging further, we would use Show By again, this time looking at source ASN or IP address.

Note that these same dimensions can also be used in Alerting to monitor traffic that comes from outside the network separately from traffic that is internal to the network. For more information on creating alerts, check out the Policy Alerts Overview in our Knowledge Base.

Host Directionality Use Case

Another use case for Network Classification is specific to host traffic captured by kprobe (Kentik’s software host agent). Since most hosts have only a single interface through which traffic can pass, kprobe captures both inbound and outbound traffic. Until now, it was difficult for a Kentik Detect user to separate which traffic was coming in and which traffic was leaving. But now it’s possible to distinguish one from the other. As shown in the graph, with Host Direction used as the group-by dimension you can now see separately the flows in (black) and out (blue) of a host.

Configuring Network Classification

• Internal IPs: Enter a list of the IP CIDR blocks used inside the network. By default, the RFC1918 IP Space is included with the user defined list; this can be changed with the checkbox below the field.
• Internal ASNs: Enter a list of ASNs used inside the network. By default, the Private ASN range is included with the user defined list; this can be changed with the checkbox below the field.

Once the fields are filled and you’ve clicked the Save button, you’re ready to begin using Network Classification.

New Types for Interface Classification

• Datacenter Interconnect
• Aggregation Interconnect

If your network includes these types of interconnects, after you run Interface Classification you can now specify these interface types when you query, build dashboards, monitor with alerts, and more. For more information on using this feature, head to our Knowledge Base for the article on Interface Classification.

HeatMaps for Countries and Sites

One feature that’s been among the most common requests from customers is the ability to visualize network data on a map. We just added mapping capabilities in the form of a new Geo Heat Map view type (accessed via a drop down view type menu like you’ll find at the upper right of the display area in Data Explorer). This new type enables a couple of map visualizations that we think you’ll find quite useful:

• Country Heatmap
• Site Heatmap

Country Heatmap

A country heatmap shows the source or destination of traffic by country on a global map. This feature uses the GeoIP data that we add, based on Source and Destination IP address, to each flow record in the Kentik Data Engine. The visualization shows one direction (source or destination) at a time of traffic using Country geo-information. A color key indicates the volume of traffic for each country.

To generate a country heatmap in Data Explorer, first set the group-by dimension to either source or destination country. Then change the visualization type in the drop down in the upper right hand corner to Geo HeatMap. The resulting graph should look similar to the map below.

Site Heatmap

A site heatmap shows the total traffic, inbound plus outbound, by each site in your network. The larger the circle on the map, the more traffic at that site. For greater detail, hover over a circle to open a pop-up giving the site’s coordinates and the total amount of traffic.

Before you can use this feature you have to provide a street address for each of your sites, which you can do with the Edit Site dialog in Admin » Sites (you must be an Admin user; see Editing a Site in our Knowledge Base article) or with our Site API.

Once the site addresses have been specified, you can build a query in the Data Explorer using the group-by dimension Site (FULL) and the view type Geo HeatMap. The resulting visualization should look something like the map below.

Expanded Host Metrics

The savvy Kentik user is already aware of our software host agent, kprobe, which runs on a server and provides host performance metrics. For those not familiar with this functionality, check out our blog post on one intriguing application for this, which is using kprobe to monitor DNS infrastructure.

This month we’ve expanded kprobe functionality by adding the following metrics:

• Repeated Retransmits: The number of TCP packets that have been retransmitted more than once, likely due to packet loss along the path from sender to receiver.
• Zero-Window: The number of times the TCP receive window has been set to zero, indicating that the receiver cannot keep up with the flow of data from the sender.
• Receive Window Size: The size of the TCP receive window reported by the receiving host.
• Connection ID: The TCP or UDP Connection ID for the session that the reported flow belongs to.

To start leveraging the expanded metrics, install the latest version of kprobe (see kprobe download and install in our Knowledge Base).

Scoreboard for Alerting

Anomaly detection, alerting, and mitigation are core features of Kentik Detect, and we continue to devote a lot of effort toward improving workflow and usability. Recent changes to the Alerting section make this powerful functionality easier to set up and use. The first place you go when you click Alerting in the Kentik Detect navbar is the Active Alerts page, which we’ve just enhanced with an Alerting scoreboard. So it’s now easier to see at a glance the things that most need your attention.

The top part of the scoreboard is a set of summary tiles (shown above), one for each of three types of events:

• Mitigations: Shows a count of how many alerts are currently being mitigated, either automatically or manually. A button (+ sign) for manual mitigation is also included. The background color of the tile varies depending on the count:
  – Grey: No mitigations currently in progress.
  – Purple: 1 or more mitigations currently in progress.
• Alarms: Shows a count of alerts that are in ALARM state, meaning that the conditions defined in the alert policy have been met and notifications have been triggered. A count of the alarms at each severity is also included. The background color of the tile varies depending on the severity (minor, major, critical) of the most severe alarm:
  – Grey: No alarms currently active
  – Dark Red: The highest severity level is Critical.
  – Red: The highest severity level is Major.
  – Orange:  The highest severity level is Minor.
• Acknowledgements: Shows a count of alerts that are in ACK_REQ state, meaning that the conditions that resulted in an alarm are no longer present, but an acknowledgement is required from a user in your organization before the alert is removed from the active list. The background color of the tile varies depending on the count:
  – Grey: No acknowledgements pending.
  – Blue: 1 or more acknowledgements pending.

Below the summary tiles is a matrix whose rows represent either mitigations or alert policies that are in alarm. The columns represent the top values of a dimension chosen when the matrix was configured (click the gear button to edit the configuration). The matrix lets you quickly see what’s going on with the policies that are most in need of attention.

Key an eye on our Knowledge Base for coming topics related to the Alerting Scoreboard.

IPv6 Support for Radware Mitigation

Kentik Detect includes the ability to integrate with mitigation systems from third parties including Radware. Until recently, you could only define a Radware Mitigation Platform using an IPv4 address. We now have the ability to make an API call to a Radware appliance using either an IPv4 or an IPv6 address.