Last month, we announced VRF Awareness Phase 1, including new dimensions associated with VRFs (virtual routing and forwarding). We’ve now added support for VRF attributes in the Interface methods of our Device API, which you can experiment with in our API tester (EU customers will find the tester here). The new parameters for these methods give you programmatic control of VRF attributes associated with each interface.

If SNMP polling is enabled in your Kentik instance, the VRF attributes of each interface would normally be discovered automatically as Kentik polls the VRF MIB (refer to our KB topic on Kentik-polled SNMP OIDs to see the OIDs we poll). But with VRF support in our Interface methods you can now programmatically define or redefine mappings (including RT/RD related data) between physical interfaces and the VRFs to which they belong. This could be particularly useful in the following situations:

• If your Kentik instance doesn’t use SNMP polling.
• If you have SNMP polling enabled on your device, but either:
  • Your device doesn’t support the VRF MIB.
  • The device SNMP policy doesn’t allow polling the VRF OIDs.
• If you want your VRFs to appear in Kentik Detect with names that are more human-readable than the names retrieved from SNMP.

The new parameters in the Interface methods of the Device API support both of the following cases:

• Interface creation: VRF parameters have been added to the Interface Create method.

• Interface updating: VRF parameters have also been added to the Interface Update method.

For additional information, please contact our Customer Success team.

Another data source we’ve recently integrated with KDE using Universal Data Records is Istio, which is an open source “service mesh” that integrates tightly with Kubernetes. You can learn more in our recent Kubernetes Networking 101 blog post, but basically, service meshes were invented to allow operations teams to deploy monitoring, security, failover, etc. in a cloud-native environment without developers needing to change their code. Istio’s insight and control layer lets you secure, connect, and manage the applications that make up a distributed microservices architecture for hybrid and multi-cloud deployments. It’s currently one of the most popular service mesh architectures, and it enjoys strong backing from Google, Lyft and IBM.

With Kentik’s new integration, you can now ingest Istio metrics data into KDE. That allows you to view the connectivities and status of components and services within an application in the context of the underlying network infrastructure that delivers them, even if the application is spread across multiple physical, hybrid, or multi-cloud infrastructures. If one component fails, you can now observe the impact on the big picture.

One fundamental component of the Istio control plane is called “Mixer.” This layer provides operators with rich insights and control over service behavior without requiring changes to service binaries. Kentik integrates with Istio via a backend plugin that acts as a data receiver for Mixer. While metrics from Mixer can be exported to a number of different tools, Kentik is the only solution that fuses Mixer metrics with flow data, which is key to understanding how the application and its supporting infrastructure affect each other.

The process for using Istio metrics in queries is pretty much the same as described above for Cisco ASA:

• Add an Istio device via the Admin » Devices page.
• Select an Istio device in the Devices pane of Data Explorer.
• Select an Istio metric as a filtering or group-by dimension as shown below.

Among the various ways that you can use the Istio dimensions is to build a Sankey diagram like the one below, which maps service-to-service flow between the microservices that are using Istio within an application.

Descriptions of the Istio fields supported by Kentik can be found in our Knowledge Base topic on Istio Dimensions. For more information about using Istio with Kentik Detect, contact our Customer Success team.

In January’s Product Update, we announced our new Universal Data Records architecture, which enables the Kentik Data Engine (our distributed big data backend) to flexibly allocate columns to the flow fields of diverse devices. Our first integration using this new approach was to support the ingest of flow data from Palo Alto Networks firewalls for enhanced operational and security intelligence.

In February we added another integration, this time with Cisco Adaptive Security Appliances (ASA). This latest implementation includes:

• Support for the full ASA NetFlow template in group-by and filtering dimensions, Dashboards, and Data Explorer.
• Support for ASA fields in Alert Policy filters.
• Support for ingest of ASA flows both directly and via our Netflow Proxy Agent (chfagent).

Now let’s take a look at the workflow involved in putting ASA data to use in Kentik Detect.

Step 1.

In the Add Device dialog (accessed via the Admin » Devices page), add an ASA as a new device. Make sure to set the Type control to Cisco Adaptive Security Appliance.

Step 2.

In Data Explorer, use the device selector (accessed via the Devices pane in the sidebar) to include the ASA in the set of queried devices.

Step 3.

After you add an ASA in the Devices pane you can open the dimension selector — either for group-by in the Query pane or for ad hoc filters in the Filtering pane — to see and choose ASA-specific dimensions. These dimensions will help you to understand significant events in a flow, and track flow state changes over time.

Putting these new ASA dimensions to work in Kentik Detect will help you to better understand aspects of your traffic — top denied countries, applications, IPs, paths and so on — that are key for robust infrastructure security. One way to take advantage of these dimensions is to build a security-focused dashboard like the one shown below.

You can also use ASA dimensions in the filters that you use for alert policies (see the Data Funneling topic in the KB), which means that you can generate alerts based on firewall events such as unexpected denied traffic.

For more information, please refer to the Kentik Knowledge Base topic on Cisco ASA Dimensions, or contact our Customer Success team.

Like Remote Trigger Blackhole (RTBH), Flowspec is a means of using BGP to respond defensively to a DDoS attack. But while RTBH is a blunt instrument, dropping all traffic toward the victim's IP address, Flowspec lets you respond to attacks with surgical precision. It offers a greater range of possible mitigation actions and it’s also far more granular in terms of defining which traffic is affected by those actions.

Based on IETF RFC 5575, Flowspec works by distributing a “flow specification” that can be read by any routing system that supports MP-BGP. The Flowspec defines a filter for matching traffic based on certain packet properties (IP, ports, protocol, etc.) as well as an extended community value that specifies actions to take on matching packets (drop, forward, rate-limit, etc.).

Kentik’s support for Flowspec-based DDoS mitigation is integrated directly into our powerful anomaly detection and alerting system, enabling our customers to leverage the built-in traffic filtering capabilities of their existing network hardware. For our initial “preview” phase of Flowspec support, the following Flowspec capabilities are now implemented in Kentik Detect:

• Manual Flowspec mitigation
• Flowspec mitigation from alarms
• Programmable Flowspec mitigation via “Infer From Alarm”

Flowspec Setup

The workflow for Flowspec setup is detailed in our Knowledge Base topic on Configuring Flowspec Mitigation, but here’s an overview of the process:

• Configure devices for Flowspec
• Enable Flowspec in Kentik device admin
• Create a Flowspec mitigation method
• Specify Flowspec conditions and actions
• Create a Flowspec mitigation platform
• Link the method to the platform

Once we’ve created a Flowspec we can use it in a couple of different ways. The most common application would be to deploy an automated Flowspec mitigation, which means that we assign the mitigation to a threshold in an alert policy (as shown above), so the mitigation is triggered when the threshold’s conditions are met. Alternatively, you can deploy the Flowspec mitigation manually as described in Start a Manual Mitigation.

For more information, please see our blog post Kentik Takes a Leap Forward in DDoS Defense and the Flowspec Mitigation Knowledge Base topic. To discuss the benefits of Flowspec, or to enable Flowspec support for your organization, please contact the team at Kentik Customer Success.

Kentik Data Engine (KDE) ingests and enriches flow records from routers, switches, and firewalls, as well as flow logs from cloud providers. This unaggregated data on individual flows is the basis for our comprehensive analytics functionality, including charts, graphs, tables, and dashboards. While those views are extremely useful, we’ve also given customers the option to directly view, filter, and export the raw underlying data in the Raw Flow Viewer (Analytics » Raw Flow; see the Raw Flow article in our Knowledge Base). We’ve now extended this functionality with Raw Flow Views for dashboard panels.

A Raw Flow View allows raw flow output to be embedded into a dashboard. Panels based on Raw Flow Views can inherit the devices, filters, and time range specified for the dashboard on which they live. Such panels are particularly useful on dashboards that are linked to Kentik alert policies, providing more traffic detail when investigating an alarm.

You can add a raw flow panel to a dashboard with a few easy steps:

1. Create or Edit a dashboard. When you click the Edit Mode button, you’ll see Raw Flow View as one of the options available in the Add A Dashboard Panel pane.

2. Click the Add button in the Raw Flow View card, which will open a configuration window where you can customize your query and get a preview of raw flows.

3. Click the Add Dashboard Panel button to add the panel to the dashboard.

For further information about the Raw Flow View, please contact our Customer Success team.

Virtual routing and forwarding (VRF) is a technology that allows multiple routing table instances to co-exist within the same router at the same time. Because Internet service providers (ISPs) often take advantage of VRFs to create separate virtual private networks (VPNs) for customers, the technology is also referred to as VPN routing and forwarding. With VRF support in Kentik Detect, you no longer need to manually map interface names and descriptions to VRF names and IDs (which are hard to read, troubleshoot, and support). Instead, flow data is enriched with VRF identifiers as it’s ingested into the KDE, enabling the use of VRF attributes to filter or segment network traffic in your Kentik queries.

The first phase of our VRF implementation includes support for Cisco L3VPN, Cisco VRF-lite, and Juniper L3VPN. As shown in the screenshot below, there are eight new dimensions associated with VRF support: source and destination VRF Name, VRF Route Distinguisher, VRF Route Target, and VRF Extended Route Distinguisher.

Our new VRF functionality enables multiple use cases:

• An enterprise network can verify that VRF-lite network partitions are functioning correctly (e.g. to ensure there is no traffic leaking).
• An infrastructure/network planner can see inbound or outbound traffic at the Provider Edge (PE) segmented by VRFs.
• A network operator can see all traffic associated with a specific Route Distinguisher (RD) or verify the names of the VRFs that are associated with a specific RD.
• A network operator can get alerts for changes (e.g. increase/decrease) in traffic volume per customer using VRF IDs to distinguish customers at the PE

The screenshot below shows a Sankey graph and table with all of the details about how VRFs map to interfaces on network devices. With this view, network teams can accelerate troubleshooting and easily answer questions about how traffic maps to VRFs.

As shown below, the new VRF dimensions are also supported in Alert Policies.

As we extend our VRF capabilities going forward we’ll be able to provide an even richer set of insights for analytics and visibility, including deeper integration with per-VRF BGP routing data and Kentik’s existing Ultimate Exit feature. For more information, please see the listing of VRF dimensions in our Knowledge Base, or contact our Customer Success team.

Kentik now supports the full set of fields from the NetFlow Templates that are supported by Palo Alto Networks firewalls. This first phase of our PAN integration adds huge value for Kentik customers who use PAN firewalls, providing single-pane-of-glass network visibility that now includes firewall policies and events. Beyond standard flow-record fields such as IPs, protocols, and interfaces, you’ll also now have visibility into data including user IDs, application names, and more.

You can see this new functionality in Kentik Detect’s Data Explorer. First, make sure that your PAN firewall is included in the devices selected in the Devices pane in the Explorer sidebar (shown at right). Then click on the Group-by Dimensions selector in the Query pane to open the Group-by Dimensions dialog. Scrolling down, you’ll now find (as shown below) a dimension category for Palo Alto Networks Firewall. Available PAN dimensions include:

• Source Dimensions: Post-NAT Transport Port, Post-NAT Address
• Destination Dimensions: Post-NAT Transport Port, Post-NAT Address
• Non-Directional Dimensions: ICMP Type, Flow ID, Application ID, User ID, Firewall Event, Direction

Once you select the dimensions of interest, you can order them as you like and run the query. Moreover, you can combine other data (i.e. Geo) and filter on specific firewall events to answer questions like, “Is this a region-specific problem or a general problem?” The following screenshot, for example, shows a Geo HeatMap chart of traffic filtered by a Firewall Event value of Flow Denied (the filter setting is shown in the overlaid inset).

Bringing PAN firewall events into Kentik Detect flow records empowers enterprise network teams with application visibility and enables additional security use cases, such as:

• Forensic investigation of current and past threat activity.
• Real-time verification that firewall policies are not over-or under-blocking applications.
• Cross-correlation between source countries and firewall events with a map view.

We’ll continue executing additional phases of integration, aiming for the best user experience (e.g. UI improvement and alerting integration). For more information, please see the Kentik Knowledge Base topic on Palo Alto Networks Firewall dimensions, or contact our Customer Success team.

The secret of our speedy integration with Palo Alto Networks firewalls is our new Universal Data Records architecture. With Universal Data Records, we’ve made it even easier to take advantage of the Kentik Data Engine (KDE) data store’s ability to store, unify, and query disparate data types, mapping its flexible schema to an even wider set of traffic sources, and so to bring data integration (i.e. vendor, protocol, etc.) faster to the customers for actionable insights. This approach has many advantages, like storing vendor-specific flow fields, more capacity for Custom Dimensions, and even the ability to store non-flow records that don’t contain standard flow fields like IP addresses. That makes it much faster for us to expand the types of data sources ingested into KDE, enabling visibility into a wider range of customer networks and infrastructure.

Using Palo Alto Networks firewalls as an example, with Universal Data Records we can now accept all of the fields included in PAN NetFlow Templates. While most of those fields are IANA IPFIX standard, we also include two vendor-specific fields, App-ID and User-ID (see below), that we previously couldn’t have ingested or stored.

Flow data that identifies applications and users is extremely valuable, and with Universal Data Records our customers can now take full advantage of this data to get a complete end-to-end picture of network activity.

This December 2018, we revamped the export of chart and table information from Data Explorer. The labeling in the Export submenu (from the drop-down Options menu at the top right of the chart display area) is now more intuitive, offering the following export options:

• Chart + Legend: Export, as a single PDF, both the visualization and the results table.
• Chart Image: Export, as either bitmap (PNG) or vector (SVG), just the visualization.
• Data: Export, as CSV, the data for either the visualization or the results table.

If Data Explorer is currently displaying the results of a compound (multi-axis) query, then in addition to the options listed above the Export submenu will include a Series Data option (as shown below) from which you can choose to export either the visualization (as PNG) or the results table (as CSV) associated with each individual axis of the query results.

Kentik Detect lets you apply dimension-based filtering in many locations throughout the portal, including Library dashboards, Data Explorer, alert policies, the analytics pages, and even the user admin page (where you can filter the traffic that’s visible to Member users). Filtering is applied with the Filtering Options dialog, which we’ve now redesigned in several different ways - read on!

First, we’ve restructured the dialog so that there are no longer separate working areas for ad hoc filters (defined in the dialog itself) and saved filters (Kentik presets or previously saved “company” filters). Instead, as demonstrated by the initial dialog state shown in the screenshot below, there is a single Filter Groups pane where you configure filter groups containing both types of filters.

Consolidating these working areas allows us to make a more fundamental improvement, which is to change the logic used in compound filters (filters built from multiple filter groups). It used to be that all filter groups with saved filters were first ANDed together and then ANDed with the combination of all filter groups with ad hoc filters. But now you can mix and match saved and ad hoc filters in the same filter group, either at a single level or nested, and groups can be either ANDed or ORed together.

In the filter group below, for example, you can see four distinct filters: two single-condition filters at top (source country and destination URL), then a saved filter (MYNETWORK_IN), and then a nested group that excludes traffic from two source cities.

To implement these new capabilities we’ve made some changes to filtering controls, notably the addition of the Add Saved Filter button to each filter group. As shown below, we’ve also made it easy to check the individual components of a saved filter. Just click the expand icon (right-facing triangle) to the left of a filter’s name to reveal a list of its parts.

Another improvement is that you can now convert a saved filter to an ad hoc filter, allowing you to build new filters from saved filters rather than starting from scratch. To do so, click the saved filter’s Customize button, which you can see at the top right in the screenshot above.

The enhanced flexibility that we’ve built into our new filtering UI now enables you to zero in on the precise result you need. For a complete explanation of the new filtering controls, see the Filter Groups Interface article in our Knowledge Base, or ask the Kentik Customer Success team at support@kentik.com.