This month the cloud product team delivered a ton of improvements: tightening up our maps, making our backend systems more resilient, and improving Azure onboarding. We released an alpha version of Kentik Kube, a product designed to help networkers awash in a sea of containers find solid ground, a variety of map improvements, support for VPC endpoint interfaces and gateways, and the ability to retain VPC Flow Logs inside of S3 buckets indefinitely. And we delivered the first of two projects designed to make agentless cloud ingest a reality. We are actively seeking design partners for Kentik Kube. Please contact us if you’d like to help guide our vision for this product early on. We welcome your input! Read on for all the details.

Kentik Kube: Network visibility for Kubernetes clusters

One thing we believe at Kentik is that everything that runs over the network should be observable — from containers to clouds, and everything in between. And with that in mind, whenever we become aware of a new visibility gap making operators’ lives more difficult, we aim to bridge it.

Enter Kubernetes. In 2020, we released an agent that could be used to export network flows from Kubernetes. And this month, we’ve released Kentik Kube to visualize this data.

Kentik Kube is a new module of the Kentik Network Observability Cloud that helps cloud and infrastructure engineers gain detailed network traffic and performance visibility both inside and among their Kubernetes clusters to quickly detect and solve network problems.

As the industry has adopted Kubernetes as a de facto standard for workload scheduling, teams that support these environments are discovering that their network and Kubernetes monitoring tools can’t help them answer critical questions because they:

• Are unaware of traffic patterns within Kubernetes or do not create traffic logs for these environments
• Do not take into account the network models implemented by Kubernetes
• Do not fuse Kubernetes metadata into traffic data.

We built Kentik Kube to provide this visibility for these teams. Our solution supports cloud-managed Kubernetes clusters (AKS, EKS, and GKS) and on-prem, self-managed clusters using the most widely used cloud implemented network models.

Kentik Kube helps by supporting the following use cases:

Network performance: Discover which services and pods are experiencing network delays so you can troubleshoot and fix problems faster. Identify service misconfigurations without capturing packets. Configure alert policies to proactively find high latency impacting nodes, pods, workloads or services.

Top talkers: Identify clients/requesters consuming your Kubernetes services so you can track down problematic connections. Discover oversubscribed microservices so you can adjust scaling, configure node affinity, etc. Know exactly who was talking to which pod, and when.

Policy validation: Ensure that your network reality matches your design. See which pods, namespaces and services are speaking with each other to ensure that your configured policy is working as expected.

Total infrastructure visualization: Know which pods are deployed on which nodes — even historically. See which pods and services are communicating with non-Kubernetes infrastructure or the internet. View your network from container to cloud.

AWS Flow Logs in-bucket file retention

Kentik has always allowed our AWS customers to retain their flow logs in their buckets. However, we ran into problems with customers who wanted to keep their flow logs around for long periods of time. We’ve made some significant changes and are happy to report that customers can now keep their flows around as long as they like — without any adverse effects on their Kentik subscriptions.

VPC endpoint support and ENI gateways

This month we’ve begun our journey towards full support of VPC endpoints in Kentik Cloud. Our first volley of product improvements includes giving customers the ability to group and filter flows based on these gateway IDs in the Data Explorer.

We support both Gateway-style VPC Endpoints as well as Interface VPC endpoints (AWS PrivateLink). However, despite their similar names, these are very different things under the hood. VPC Endpoint Gateways act as gateways to Amazon services, keeping this traffic inside your VPC and off the public internet. Using these endpoints can save money by keeping VPC network egress costs down — making identifying when traffic isn’t using these gateways a key use case. These gateways aren’t exposed as elastic network interfaces and thus can be difficult to track traffic using other network solutions. Kentik is the only solution on the market that gives you this capability.

AWS PrivateLink allows AWS and their customers to configure their services for consumption without forcing traffic to flow over the public internet. Over 128 different AWS services are available behind these so-called “interface endpoints” while just a small few are available behind VPC endpoint gateways. Interface Endpoints serve to reduce cost and can serve a security function in that sensitive information can be shared between AWS customers privately. These endpoints are exposed in a more standard fashion inside of VPCs using Elastic Network Interfaces as the gateways. The major difference in how this impacts Kentik users is that these interfaces are grouped in the Interface Type dimensions and specific interfaces are identified in the ENI ID dimension while the Gateway-style endpoints are grouped into the Gateway Type dimensions and the specifics are found under Gateway ID dimension.

S3 bucket ingest

We took a major step forward this month in our promise to offer agentless ingest to AWS customers. Instead of having Kentik reach into customer buckets to retrieve flow logs, we can now support customers who wish to write their flow logs directly into an S3 bucket that Kentik manages and maintains. This allows customers to write flow logs more flexibly (via direct VPC write, lambda copies, S3 actions, etc.) and also allows customers to avoid giving Kentik permissions into their S3 buckets. Next up on our agentless agenda is to provide a method for companies to post their metadata to a similar endpoint for enrichment and mapping visualizations. Stay tuned!

Map improvements

Following our September release of the Weather Map, we’ve continued to iterate on our Kentik Maps product. This month we’ve added a bunch of important fixes and improvements.

We’ve improved handling overlapping lines in AWS. Previous versions of Kentik Map would allow lines connecting close together objects to overlap. We’ve added some logic to detect this condition and give these objects a bit of breathing room, making it easier to visually determine the path connecting the objects and select them with your mouse.

We’ve improved visualizing site-to-site VPN connections. Prior versions of Kentik Map for AWS kept the customer gateways in the middle of the screen. We moved customer gateways into the “On Prem” box where they rightfully belonged. This helps clean up the interconnection section of the map considerably. We have more work coming here this quarter to make the map experience even easier to use for companies with lots of interconnections.