February 2022 comes with a broad set of Kentik Cloud improvements. Google VPC Flow Logs now include GKE (Google Kubernetes Engine) dimensions, users can now post their own AWS metadata using a newly created ingest API (instead of allowing Kentik to retrieve it via AMI assumption), dissociate the accounts used to collect Flow Logs vs Cloud Metadata, and last but not least, the Kappa agent for Kubernetes celebrated its 1.0 version !

Google Kubernetes Engine dimensions

Kentik Data Explorer now supports Google’s extended flow logs for Google Kubernetes Engine (GKE) environments. Google extended their VPC Flow Logs to include annotations that describe network traffic inside the Google Kubernetes Engine environment – i.e, the pods, services, nodes, etc. After hearing from a customer who needed this capability inside Kentik, our engineering team added support for these new dimensions.

AWS Agentless Ingest

Kentik Cloud users can now choose how they want to send their cloud data (VPC flow logs and AWS metadata) to Kentik. This helps solve a problem for some who couldn’t allow Kentik to reach into their AWS accounts via an IAM role assumption. We have exposed a REST API to which you can manually post AWS metadata and we can provide a Kentik-hosted S3 bucket to which VPC flow logs can be written or replicated.

Improved Metadata-only onboarding/settings

It’s now easier to configure an AWS cloud export to collect only metadata from a given account/region. Now that we’ve made this simpler, anyone should be able to understand how to configure such environments.

Enhanced kappa agent for Kubernetes

Kentik has released version 1.0 of the eBPF-based kappa agent for Kubernetes network performance and telemetry. Improvements in this version include easier deployment, critical performance telemetry (% Retransmit and % Out of Order Packets), and host metadata reporting.

KMI is a new service provider workflow that uses the global routing table to classify the peering and transit relationships between ASes and to identify the providers, peers, and customers for any AS in any geography. KMI estimates the volume of IP space transited by ASes in different geographies and produces rankings based on that volume, thereby enabling users to compare ASes in various markets.

This new workflow is available to all Kentik users with Premier Edition or with the service provider add-on available for the Kentik Pro Edition. This new workflow does not require any configuration and is immediately usable, as it relies on public routing data from a large number of BGP vantage points all around the world.

As routing data gets crunched on a daily basis, it can now be consumed via a simple interface allowing our users to decrypt how networks are connected to each other, what any network’s customer base looks like or what their providers and peers are.

KMI uses the global routing table to classify the peering and transit relationships between ASes and to identify the providers, peers, and customers for any AS in any geography.

Additionally, KMI scores and ranks any network against the size of their customer base in any subdivision of markets, as well as per customer base type such as retail, wholesale or backbone. KMI can now serve as a public, neutral and objective benchmark to score and rank all networks.

Here are a few pointers to get you started with KMI:

• Product page: Kentik Market Intelligence
• Blog post: Launching a labor of love, Kentik Market Intelligence
• Press release: Kentik Market Intelligence launches to benchmark the internet
• Knowledge Base article: Details about how the neutral, objective scoring and ranking algorithm works

A lot of new additions have surfaced in February 2022 in the Kentik Map UI. Amongst others, users will find: a health digest reporting function, new layers for AWS Regions and cloud backbone, as well as improvements in the map legends.

Kentik Map Health Digest

An indicator in the toolbar gives a count of issues we’ve discovered on your network and opens a popover that provides a high-level rundown. Click the View Problems button to see a list detailing the issues, then drill down to the map to see an impacted component in the context of its surrounding infrastructure.

Kentik Map Layer Selector

A new layer selector for the Kentik Map lets you choose which categories of overlays to display, including link traffic types, traffic layer types, cloud regions and backbones, traffic utilization, health, and clustering.

Kentik Map legends

The legends that identify the value ranges represented by link colors are now persistent, so it’s always easy to see what the colors mean.

Kentik Map AWS Regions and Backbones

A layer for AWS regions and backbones has been added to the Kentik Map.

Kentik continues to expand the agent fleet for synthetics, giving users more flexibility and locations from which to originate tests.

App agents, which until now were only available as global agents, can now be deployed as a private agent. App agents differ from network agents in that they support both network and web layer tests.

With private app agent support, customers can run network and web layer tests from within their environment. For example, customers can run Page Load tests to measure the performance of key websites from their branch or data center locations or any location they can install a private agent. In addition to web performance metrics, they can also collect network performance metrics — loss, latency, jitter, etc., to these destinations, with an app agent.

The app agents are initially targeted towards supporting new web layer test types (Page Load and transaction tests) but they are able to run all test types so users that want to install a single agent for all test types can use the app agent. The app agent can be deployed using Debian/Ubuntu, RPM or Docker packaging.

The agent will be marked beta as we continue to collect usage data/feedback from users. We recommend the existing network agent for non-web layer testing.

Read the Knowledge Base entry for Kentik App Agents.

This new feature is the beginning of a broader Kentik initiative to share the Network Observability love outside of the Kentik user crowd. As a lot of our users have asked for means to be able to reliably and systematically share Kentik visualization outside of their own silo, all the way to public audiences, Kentik now offers Public Link Sharing ! 

Sharing is caring

Publicly accessible visualization pages can now be created from Data Explorer visualizations and Synthetics test results and shared as links with people who aren’t registered Kentik users.

As of Friday, February 25th, users can use Public Link Shares. This feature allows users to share Data Explorer visualizations and synthetic monitoring tests with unauthenticated users. This feature has many useful applications, such as helping:

• Efficiently instrument troubleshooting sessions internally and externally
• Show evidence of outages and performance issues to vendors
• Augment support responses to customers with actual, interactive evidence
• Build brand confidence by transparently communicating on outages and their impact
• Empower your product and marketing teams with tools to showcase the performance of your infrastructure

Each public share created comes with a public URL in the form of: https://portal.kentik.com/share/<view-type>/<entropy_hash>/<title>

You will then be presented with a list of options including the ability to send an email to notify public users and get them to navigate to the share.

Native email notifications

A public share notification email will look like the one below:

Alternatively, Kentik Portal will copy the public URL for the new public share upon creation. Users accessing this URL will not be required to authenticate and will be able to visualize the same chart on a public interface.

Managing your Public Shares

Users will be able to modify, delete and audit existing shares via this new navigation menu entry:

Useful pointers

• For more information on using Public Link Sharing documentation is available in the Kentik Knowledge Base.
• Also see our blog post, Network observability, now publicly shareable that describes some use cases for the new Public Link Sharing feature.

The Capacity Planning workflow saw significant changes recently. You’ll find a completely refreshed user-interface with many new features to enhance your productivity.

You can read more about the changes in the blog post Network capacity planning made easy in 2022.

General UI overhaul

The Capacity Planning UI has been significantly improved.

• Each tracked capacity plan is shown in card format and displays an at-a-glance status. The colored status bar at the top of each card is a visual measure of the severity of the plan based on the status of each interface and its contribution to the overall capacity.
• You’ll find noise filters that allow you to disable unnecessary information on healthy capacity plans.
• Each plan card now displays the strategy and thresholds within the card, as well as counters for each interface in Warning or Critical status.
• An options menu for each plan allows you to export and access configuration directly from the landing page.
• And a brand-new, right-side panel now displays an ordered list of the critical plans to attend to for both Interface Runout and Utilization.

Configuration of Capacity Plans revamp

The configuration section of each capacity plan has been revised to more closely match the standard adopted in the Edge > Connectivity Costs workflow: Attributes and strategies for all plans are on one tab, while the interface selection is on another one.

Capacity Plan Details UI

The Capacity Plan details page has also been completely reworked and it now includes the following:

• A summary of the plan status and configuration mirroring that of the cards on the landing page for that plan
• The ability to filter out by level of severity as implemented on the landing page
• A new option to group the interface details within that plan by multiple levels: None (flat), Site or by Device/Site to accommodate users’ different ways of tackling capacity.
• Each plan can now be exported both as PDF (ad-hoc or email attachment), as well as CSV for further massaging (note that this export types are also available directly from the card on the landing page)
• Lastly, each interface in the details screen can be expanded to reveal a traffic chart for the interface with visual thresholds from the configuration on it.

The mitigations UI/UX experience has been updated this December, these few changes are the initial steps towards a much longer Q1 2022 Alerting and Mitigation push, stay tuned for more.

Mitigations

• Mitigation controls - integrations complete for policies, support in v4
• Added acknowledgment required for thresholds
• Multiple mitigations are supported per policy
• Mitigation apply/clear timers per policy

As usual, our Cloud Product team turned around a ton of great enhancements to the Kentik Cloud and Map products over the month of December 2021. More than ever, Kentik goes one step closer from being the only and most complete Hybrid Cloud observability solution out there. See for yourself.

Weather Map improvements

This month saw one of the largest updates to the Weather Map yet. In this release we’ve incorporated several new features worth discussing. We added layers that allow users to view utilization and/or health data into their map, along with a nifty layer selector:

Layer Selector

Utilization layer

A major use case for Weather Map in ISPs and large backbone networks is to assist users in performing capacity planning exercises. In order to accomplish this, we needed to color the map based on interface utilization rather than total bytes. This means that our new map breaks down the interface utilization of a single interface or an aggregated bundle and buckets these interfaces into 10 groups of increasing utilization.

We also needed to add in support for interface bundling as well as support visually aggregated interfaces when more than one link connects a site cluster to another site or another cluster. To support this, we use the backend attributes that we poll from our customer’s SNMP data to determine if an interface is configured as part of a bundle or is operating as a single interface.

If a link is drawn between two site clusters, we’ll aggregate the bandwidth over both links and calculate the total utilization on the fly:

When a user clicks on this link, the system allows them to choose which link they’d like to focus on:

Weather Map sidebar improvements

We’ve also added in sidebar improvements that make selections of Sites and Links easier. Consider the following example. A user has clicked on the 3 site cluster in the Chicago region on the Kentik map:

The sidebar now opens up with a running count of the sites and links interconnecting the sites.

Expanding these elements allows to interactively browse the map:

Health Layer

We have also started to resurface health on the Weather Map. We started by adding health into the cluster popovers as such:

Of course, we also show the site health on the canvas and sidebar as well. Here we see an unhealthy ORD1 site on the canvas:

And a list of all of the sites within a view color-coded by health in the sidebar:

You will notice that we also now color links by health. We currently have two link-health states — down and degraded:

Down indicates that the router that originates or terminates a link has reported an ifOperStatus of DOWN or administratively shut down, while Degraded state means that the system is reporting errors.

Mini-map site topology

A popular request we’ve heard is to show a user’s site topology without having to load the entire site view. We now show the site topology in the sidebar itself. This is useful for getting at-a-glance understandings of how sites are constructed and is a step closer to our next iteration which allows users to see the devices that connect to other sites directly on the Weather Map canvas.

This December, a few top demands make it to the synthetics product area, amongst others: 

• App Agents get the traceroute and ping data back-ported from the legacy Private Agents
• BGP Monitor tests are made easier to navigate by allowing you to switch between the prefixes being monitored
• All existing Notification Channels are now available to synthetic tests

Ping and trace in page load tests

You are now able to run ping and traceroute on the App Agents. As a first step in the rollout, we have enabled UI changes to be able to run this as part of the Page Load test (the only test that uses the App Agents today).

BGP Monitor aggregated prefixes and prefix selector filter

Based on feedback from certain customers we realized that the BGP test results were noisy and it was hard to get a high level view of the number of changes and prefixes involved before digging into the details. We have made three changes to address this:

• We are aggregating the result set by prefix and grouping them into collapsed groups.
• The header for each group indicates the total number of BGP events grouped by type (Announcements, Withdrawals and Unexpected Origin events).
• A prefix selector on the top right allows users to easily filter down to a specific prefix and doing so only shows events for that specific prefix over the selected time range.

Support for all notification channel types in Synthetics

When we added support for alerts and notifications in Synthetics about a year ago, we supported Slack, email and JSON/webhook. We did the work necessary to support and template all the other notification types and have exposed them in the UI.

This month the cloud product team delivered a ton of improvements: tightening up our maps, making our backend systems more resilient, and improving Azure onboarding. We released an alpha version of Kentik Kube, a product designed to help networkers awash in a sea of containers find solid ground, a variety of map improvements, support for VPC endpoint interfaces and gateways, and the ability to retain VPC Flow Logs inside of S3 buckets indefinitely. And we delivered the first of two projects designed to make agentless cloud ingest a reality. We are actively seeking design partners for Kentik Kube. Please contact us if you’d like to help guide our vision for this product early on. We welcome your input! Read on for all the details.

Kentik Kube: Network visibility for Kubernetes clusters

One thing we believe at Kentik is that everything that runs over the network should be observable — from containers to clouds, and everything in between. And with that in mind, whenever we become aware of a new visibility gap making operators’ lives more difficult, we aim to bridge it.

Enter Kubernetes. In 2020, we released an agent that could be used to export network flows from Kubernetes. And this month, we’ve released Kentik Kube to visualize this data.

Kentik Kube is a new module of the Kentik Network Observability Cloud that helps cloud and infrastructure engineers gain detailed network traffic and performance visibility both inside and among their Kubernetes clusters to quickly detect and solve network problems.

As the industry has adopted Kubernetes as a de facto standard for workload scheduling, teams that support these environments are discovering that their network and Kubernetes monitoring tools can’t help them answer critical questions because they:

• Are unaware of traffic patterns within Kubernetes or do not create traffic logs for these environments
• Do not take into account the network models implemented by Kubernetes
• Do not fuse Kubernetes metadata into traffic data.

We built Kentik Kube to provide this visibility for these teams. Our solution supports cloud-managed Kubernetes clusters (AKS, EKS, and GKS) and on-prem, self-managed clusters using the most widely used cloud implemented network models.

Kentik Kube helps by supporting the following use cases:

Network performance: Discover which services and pods are experiencing network delays so you can troubleshoot and fix problems faster. Identify service misconfigurations without capturing packets. Configure alert policies to proactively find high latency impacting nodes, pods, workloads or services.

Top talkers: Identify clients/requesters consuming your Kubernetes services so you can track down problematic connections. Discover oversubscribed microservices so you can adjust scaling, configure node affinity, etc. Know exactly who was talking to which pod, and when.

Policy validation: Ensure that your network reality matches your design. See which pods, namespaces and services are speaking with each other to ensure that your configured policy is working as expected.

Total infrastructure visualization: Know which pods are deployed on which nodes — even historically. See which pods and services are communicating with non-Kubernetes infrastructure or the internet. View your network from container to cloud.

AWS Flow Logs in-bucket file retention

Kentik has always allowed our AWS customers to retain their flow logs in their buckets. However, we ran into problems with customers who wanted to keep their flow logs around for long periods of time. We’ve made some significant changes and are happy to report that customers can now keep their flows around as long as they like — without any adverse effects on their Kentik subscriptions.

VPC endpoint support and ENI gateways

This month we’ve begun our journey towards full support of VPC endpoints in Kentik Cloud. Our first volley of product improvements includes giving customers the ability to group and filter flows based on these gateway IDs in the Data Explorer.

We support both Gateway-style VPC Endpoints as well as Interface VPC endpoints (AWS PrivateLink). However, despite their similar names, these are very different things under the hood. VPC Endpoint Gateways act as gateways to Amazon services, keeping this traffic inside your VPC and off the public internet. Using these endpoints can save money by keeping VPC network egress costs down — making identifying when traffic isn’t using these gateways a key use case. These gateways aren’t exposed as elastic network interfaces and thus can be difficult to track traffic using other network solutions. Kentik is the only solution on the market that gives you this capability.

AWS PrivateLink allows AWS and their customers to configure their services for consumption without forcing traffic to flow over the public internet. Over 128 different AWS services are available behind these so-called “interface endpoints” while just a small few are available behind VPC endpoint gateways. Interface Endpoints serve to reduce cost and can serve a security function in that sensitive information can be shared between AWS customers privately. These endpoints are exposed in a more standard fashion inside of VPCs using Elastic Network Interfaces as the gateways. The major difference in how this impacts Kentik users is that these interfaces are grouped in the Interface Type dimensions and specific interfaces are identified in the ENI ID dimension while the Gateway-style endpoints are grouped into the Gateway Type dimensions and the specifics are found under Gateway ID dimension.

S3 bucket ingest

We took a major step forward this month in our promise to offer agentless ingest to AWS customers. Instead of having Kentik reach into customer buckets to retrieve flow logs, we can now support customers who wish to write their flow logs directly into an S3 bucket that Kentik manages and maintains. This allows customers to write flow logs more flexibly (via direct VPC write, lambda copies, S3 actions, etc.) and also allows customers to avoid giving Kentik permissions into their S3 buckets. Next up on our agentless agenda is to provide a method for companies to post their metadata to a similar endpoint for enrichment and mapping visualizations. Stay tuned!

Map improvements

Following our September release of the Weather Map, we’ve continued to iterate on our Kentik Maps product. This month we’ve added a bunch of important fixes and improvements.

We’ve improved handling overlapping lines in AWS. Previous versions of Kentik Map would allow lines connecting close together objects to overlap. We’ve added some logic to detect this condition and give these objects a bit of breathing room, making it easier to visually determine the path connecting the objects and select them with your mouse.

We’ve improved visualizing site-to-site VPN connections. Prior versions of Kentik Map for AWS kept the customer gateways in the middle of the screen. We moved customer gateways into the “On Prem” box where they rightfully belonged. This helps clean up the interconnection section of the map considerably. We have more work coming here this quarter to make the map experience even easier to use for companies with lots of interconnections.