This December 2018, we revamped the export of chart and table information from Data Explorer. The labeling in the Export submenu (from the drop-down Options menu at the top right of the chart display area) is now more intuitive, offering the following export options:

• Chart + Legend: Export, as a single PDF, both the visualization and the results table.
• Chart Image: Export, as either bitmap (PNG) or vector (SVG), just the visualization.
• Data: Export, as CSV, the data for either the visualization or the results table.

If Data Explorer is currently displaying the results of a compound (multi-axis) query, then in addition to the options listed above the Export submenu will include a Series Data option (as shown below) from which you can choose to export either the visualization (as PNG) or the results table (as CSV) associated with each individual axis of the query results.

Kentik Detect lets you apply dimension-based filtering in many locations throughout the portal, including Library dashboards, Data Explorer, alert policies, the analytics pages, and even the user admin page (where you can filter the traffic that’s visible to Member users). Filtering is applied with the Filtering Options dialog, which we’ve now redesigned in several different ways - read on!

First, we’ve restructured the dialog so that there are no longer separate working areas for ad hoc filters (defined in the dialog itself) and saved filters (Kentik presets or previously saved “company” filters). Instead, as demonstrated by the initial dialog state shown in the screenshot below, there is a single Filter Groups pane where you configure filter groups containing both types of filters.

Consolidating these working areas allows us to make a more fundamental improvement, which is to change the logic used in compound filters (filters built from multiple filter groups). It used to be that all filter groups with saved filters were first ANDed together and then ANDed with the combination of all filter groups with ad hoc filters. But now you can mix and match saved and ad hoc filters in the same filter group, either at a single level or nested, and groups can be either ANDed or ORed together.

In the filter group below, for example, you can see four distinct filters: two single-condition filters at top (source country and destination URL), then a saved filter (MYNETWORK_IN), and then a nested group that excludes traffic from two source cities.

To implement these new capabilities we’ve made some changes to filtering controls, notably the addition of the Add Saved Filter button to each filter group. As shown below, we’ve also made it easy to check the individual components of a saved filter. Just click the expand icon (right-facing triangle) to the left of a filter’s name to reveal a list of its parts.

Another improvement is that you can now convert a saved filter to an ad hoc filter, allowing you to build new filters from saved filters rather than starting from scratch. To do so, click the saved filter’s Customize button, which you can see at the top right in the screenshot above.

The enhanced flexibility that we’ve built into our new filtering UI now enables you to zero in on the precise result you need. For a complete explanation of the new filtering controls, see the Filter Groups Interface article in our Knowledge Base, or ask the Kentik Customer Success team at support@kentik.com.

Raw flow refers to actual data fields (normalized from NetFlow, sFlow, etc.) that Kentik Detect stores for each ingested flow record. We’ve long had a Raw Flow viewer in the portal at Analytics » Raw Flow, allowing you to see flow data for specified devices over a specified time range. Now you can go directly from a query in Data Explorer to a view of that query in the Raw Flow viewer.

Start by clicking the Options menu icon (hamburger) at the upper right of the chart display area in Data Explorer. At the bottom of the resulting drop-down menu you’ll now find Explore Raw Flows (see screenshot at right). Choosing this item will take you to a Raw Flow viewer that has the sidebar controls set to display flow records for the traffic that you were viewing in Data Explorer.

Alerting is one of the core features of Kentik Detect, and we’ve now made it even more useful to customers who operate very large-scale networks:

• The maximum number of active policies per company has been increased from 60 to 100.
• The maximum number of keys evaluated per policy has been increased from 300 to 1,000.

The increase in these limits reflects additional work we’ve done to better define what our alerting processes can handle and to optimize the system accordingly. This will allow customers to track and detect even more potentially anomalous network behavior. Stay tuned for additional scaling improvements in this area moving forward.

As we work to make Kentik Detect more powerful we’re also aware that features are most valuable to users when they’re easy and straightforward to use, as well as to customize for individual needs. The improvements in the next couple of sections are designed with that in mind.

Redesigned Dimension Selector

Working with dimensions for both group-by and filtering is an integral part of defining the queries used throughout Kentik Detect. We’ve taken a fresh look at how customers access and select dimensions, and made the following improvements in the dimension selectors for both group-by and filtering:

• Dimension categories – Dimensions are now organized into more intuitive categories, such as Network & Traffic Topology, IP & BGP Routing, and Cloud.

• Directional columns – Additionally, dimensions are now organized into Source, Destination, and Non-Directional columns to make them easier to locate. This also means that the Source and Destination versions of related dimensions are always located next to each other in the same row.

• New dimensions for clouds – The newly-added categories include provider-specific categories for AWS and GCP dimensions.

• Bi-directional filtering – In the dimension selector for Ad-Hoc Filter Groups, we’ve also added a Source or Destination column to simplify filtering on the same value in two related dimensions.

Custom Color Palettes

Kentik customers have always recognized UI design as one of our major differentiators. The ability to customize aspects of that design, tailoring it to individual needs and preferences, makes it even more powerful. Our latest step in this direction is the new Visualizations tab, which you’ll find on the My Profile page (access from the drop-down menu at far right in the main portal navbar).

As shown below, you can tune five different color settings that determine how visualizations are rendered:

• Theme – Toggle between the Standard Theme with a light background or the Dark Theme.
• Labels – Customize the color of labels by choosing from a popup color palette or entering a hex number.
• Overlays – Change the color of the Total and Historical overlays that are used on time-series visualizations.
• Quantitative – Choose a color theme for all chart types that display quantitative data such as Stacked Area Chart, 100% Stacked Area Chart, Stacked Column Chart, and Bar Chart.
• Qualitative – Choose a color theme for all chart types that show qualitative data such as Line Chart, Pie Chart, and Sunburst.

When changing the Quantitative and Qualitative color palettes, you can preview the effect on different view types by choosing a view type from the drop-down menu at upper right of the preview visualization. Once you save the changes, the new palette will be applied to all of your existing (and future) visualizations of that type (Quantitative or Qualitative) throughout the portal.

Finally, for Qualitative and Qualitative settings, you can create custom color palettes by turning on the Use custom values switch.

For more information, please contact the Kentik Customer Success team at support@kentik.com.

Beyond new features, we’ve continued our ongoing work on refining the utility and performance of Kentik Detect. The following enhancements cover areas that you can see as well as areas that are under the hood.

Enhanced Mitigation States and Controls

Anomaly detection, alerting, and mitigation, which are among the core features of Kentik Detect, sometimes involve complex situations like multiple mitigation actions and overlapping alarms. To better handle these scenarios we’ve simplified our state machine model. Updates include:

• Take manual control – Users can now assert manual control over mitigations that were originally triggered automatically. To support this change, we’ve created a separate set of manual mitigation states that parallel the states used in automated mitigation.
• Easier mitigation deletion – When deleting a mitigation, users now don’t have to additionally clear the mitigation on the mitigation appliance or wait for state transition to occur.
• Mitigation escalation – When an alarm escalates (starts as Minor and becomes Major), mitigations will now escalate in parallel. That means users can now associate a particular mitigation method with the minor threshold and a different method with the major threshold.

In addition to these backend changes, the UI for mitigation actions in the Active Alarms table (Alerting » Active) has been changed to provide more flexible and granular control. Play and Stop icons have now been replaced with context dependent icons and tool tips that reflect the current mitigation state.

Selective Interface Classification

A number of customers requested that we allow Interface Classification rules to be applied to some devices and not others. As shown below, the IF settings in the Add Rule dialog now include two new controls that enable you to tailor sets of Included Devices and Excluded Devices that govern application of the rule.

These whitelists and blacklists will also be displayed, as shown below, in the Rules List on the main classification page (Admin » Interface Classification).

For more information, please see the Rule IF Settings topic in the Kentik Knowledge Base or contact the Kentik Customer Success team at support@kentik.com.

Query Engine Improvements

Kentik Data Engine is the backend where your network traffic data is collected and enriched, and from which it is pulled at query run-time. Recent enhancements enable Kentik Detect to support ad-hoc queries over longer time ranges with much higher cardinality. For example, we can see source/destination IP pairs as a time series over a time range of 90 or more days. As requested by some customers, Data Explorer’s Table view can also now display much deeper results — up to 50,000 rows — for queries on certain group-by dimensions when the metric type is Total. Additional changes include improved performance for queries that filter on long lists of IP addresses.

Autonomous systems (ASes) represent the networks that collectively make up the Internet and enable the use of BGP to compute routes for the delivery of traffic. With our new AS Groups feature we allow you to assign one or more AS to a group that you name so that you can view traffic from a set of ASes as if they were a single entity.

There are several scenarios in which this ability to control how ASes are represented in query results may be useful to the operator of a network:

• Logical grouping of ASes – ISPs that are part of a broader consortium may have multiple ASes spanning multiple countries. AS grouping gives you the ability to assess the traffic from a multi-AS organization as a single entity, which is particularly important for interconnection/capacity planners in the context of peering.
• Mapping of existing AS labels – Many organizations refer to ASes with labels that are distinct from the official name of the AS. An AS group may be used to associate an existing label with one or more ASes, so that the AS naming by which ASes are represented in Kentik Detect can be made consistent with the AS naming that you already use in other systems.
• Naming by internal topology – It’s now common in modern, large-scale data centers to deploy Clos architecture, which relies on BGP to the rack, with each rack corresponding to its own private ASN. AS groups enable you to reference and visualize groups of private ASes in a way that reflects this Clos-based datacenter topology.

As shown here, Kentik Detect users with Admin access can find the AS Groups page either from the Admin landing page, in the Enrich Your Data pane, or via the popout sidebar on all Admin pages.

Managing AS Groups

The AS Groups page allows users to configure/add groups of ASes, either private or public. If a group includes just one AS then the group’s name will simply override Kentik’s default name for that AS.

The use of AS groups has the following effect on Kentik Detect queries:

• Group-by - If at least one AS group has been configured in your organization then the Use AS Groups switch will be present in the Advanced Options section of the Query pane in the Data Explorer sidebar. When the switch is on (default) and the group-by dimensions include Destination AS or Source AS, the results from all ASes in a given group will be summed for top-X evaluation, graph plotting, and display in the results table (as shown below, a group icon will appear at the left of the group name). In this context, no additional dimension is needed to use AS groups.

• Filtering - Inclusion or exclusion of traffic from a given AS group may be achieved by setting that group as the value of a filter on the dimension AS Group, as shown below.

For more on the AS Groups feature, please see the AS Groups article in the Kentik Knowledge Base or contact the Kentik Customer Success team at support@kentik.com.

In concert with our cloud efforts, we’ve also taken an overall look at how best to support two important useability goals: improving ease of onboarding and making it easier for you to discover valuable features that you might not be aware of.

Redesigned Onboarding Workflow

To simplify Kentik Detect setup in the portal we’ve added a landing page to the Admin section, with setup tasks grouped into panes that each correspond to a stage of the setup workflow:

1. Add data sources – Access to dialogs and Admin pages for routers and switches, servers and hosts, AWS, and GCP.
2. Tell us about your network – Interface and network classification, sites, and device labels.
3. Enrich your data:
   - Custom dimensions: Add business context to your network data.
   - Flow tags: Categorize your network data.
   - Custom geo: Align geolocation with your business territories or regions.
   - Custom DNS: Specify alternate servers for DNS resolution.
4. Customize your Kentik experience:
   - Set up tenants for the My Kentik Portal.
   - Configure subscriptions for reporting.
   - Define saved filters.
5. Access and security:
   - Register and manage users.
   - Set up single sign-on and access control.
6. Plans – Check your utilization of Kentik Detect compared to the limits defined in your plans.

Collapsible Admin Sidebar

In addition to adding a landing page in Admin we’ve also made the sidebar collapsible, so it’s out of the way when you’re not navigating from page to page. To access the Admin menu, just move your mouse toward the left edge of any Admin page, which will expand the sidebar and show the familiar list of Admin pages.

Improved Feature Discoverability

Admin isn’t the only area that’s been rethought. The landing page of the portal Library has also been revamped to make it easier to discover features that will maximize the benefits you can gain from Kentik Detect. We’ll continue to make valuable features more apparent. At the moment we’re using the landing page to highlight the following options:

• Configure data sources now prompts you to add new devices and clouds to your existing fleet, which is typically the first action a new Kentik user will want to perform.
• Featured Content exposes noteworthy Kentik presets for Dashboards and Saved Views. This section will evolve to promote content that’s tailored to individual users based on role, type of network, and use cases.
• Resources list useful links that can help new Kentik users get up to speed and bootstrap their knowledge of Kentik Detect and the portal. Expect to see more personalization here over time as well.
• Product Updates include links to the latest entries in this Product Update blog, so you no longer have to check for them on the Kentik website.

Since Kentik’s founding, our core focus has been to provide a 360° view of the performance, composition, and paths of actual network traffic. We started with visibility for owned/on-prem infrastructure. As more and more organizations have incorporated cloud infrastructure, we’ve extended our traffic analytics to follow suit, first by integrating with GCP VPC Flow Logs this past summer, and now with support for AWS Flow Logs. By enabling a unified view across all of your infrastructure — whether it’s on-prem, hybrid-cloud, or multi-cloud — Kentik is now a complete infrastructure visibility solution.

Out-of-the-Box GCP and AWS Visibility

Need cloud visibility, but you’re not sure where to begin? No problem! To get you started we’ve now expanded our portal Library with a broad set of predefined dashboards that provide detailed views of cloud network traffic flows. You’ll find these preset dashboards under Cloud and Data Center in the Library sidebar. As shown below, there are eight dashboards for AWS and nine for GCP. Several are highlighted in the Library’s Featured Content (indicated by an orange check mark).

We’ve also incorporated some of these same cloud metrics into alert policy presets that we’ve added to our Alerting Library. These new policies can identify unexpected changes in the source, destination, or composition of traffic in key parts of your cloud infrastructure.

AWS VPC Flow Logs Integration

As mentioned above, Kentik now integrates with AWS. If you have workloads running in VPCs on AWS, Kentik can now ingest the AWS VPC Flow Logs as a primary data source for analytics.

The steps to connect AWS infrastructure and add AWS VPC Flow Logs as a data source are fairly straightforward:

1. Create an AWS Role - In the AWS IAM console, create a new AWS role with permissions that allow access by Kentik’s AWS services.
2. Create an S3 Bucket - In the AWS S3 console, create a bucket to which logs can be published and from which Kentik can pull them.
3. Configure Log Publishing - In the AWS VPC Dashboard, configure each VPC (or subnet or interface) to publish logs to a destination folder in the bucket.
4. Confirm Log Publishing - Back in the S3 console, confirm that logs for each VPC are being published to the destination folders in the bucket.
5. Register Cloud in portal - In the Kentik Detect portal, create a new Cloud pointing to the S3 bucket (as shown below), which results in a Kentik “cloud device” being automatically created for each destination log folder.

Once the new AWS Cloud is added it will appear in the Admin » Clouds page with details including name, provider, and status.

AWS Marketplace Integration

Last but not least, Kentik Detect is now available in the Amazon AWS Marketplace. Customers can simply pick a plan and subscribe to the Kentik analytics service on AWS.

For complete Cloud Visibility information, please see the Kentik for AWS and Kentik for GCP topics in the Kentik Knowledge Base or contact the Kentik Customer Success team at support@kentik.com.

Notification channels are assigned in an alert policy to determine who will be notified when there’s a change in the alert state, e.g. an alarm is triggered by one of the alert’s thresholds. Each channel represents a notification type (e.g. email) and, where applicable, one or more notification targets (e.g. a set of email addresses). Kentik Detect supports multiple notification types (e.g. email, JSON POST Webhook, PagerDuty, Slack Channel and system log) so that each customer organization can use its prefered methods of notification. As shown below,  we’ve now added a couple of additional options for notification types: OpsGenie and ServiceNow.

For further information on Notification Channels, please see the Alert Notification topic in the Kentik KB or contact the Kentik Customer Success team at support@kentik.com.