We released a new configuration knob that allows customers to change the sampling rate for AWS and Azure on their own without contacting Kentik team.

That will allow customers to consume flow logs at the preferred rate fitting into the licensing strategy, assigning priority for certain types of traffic and being flexible by changing the sampling rate at any time and separately for each flow log exporter.

Licensing will be enforced after the sampling, so customers can use heavier sampling in some cases, and saving the licensed FPS for the another S3 buckets containing flow logs.

There is a slight difference in available options for AWS and Azure.

AWS flow log sampling

Historically Kentik was supporting a “legacy” mode of sampling where for the large files with flow logs we were randomly picking 10,000 flow records per file in S3 bucket and ingesting only those records into Kentik Data Engine. Since the number of the flow in a file can vary this was considered an “adaptive sampling” where larger files were getting more heavily sampled comparing to the smaller files. Another option was no sampling i.e.  all the records were consumed from the file.

Moving forward we now support 3 options for AWS:

• Legacy sampling - random 10,000 flow records per file.
• Sampling rate - where user can provide the sampling rate in 1:N format (meaning 1 out N records to be picked up for an ingest into Kentik Data Engine), where N should be between 2 and 2000.
• Unsampled - all the records in a flow log file will be taken into ingest. Effectively that is the same as sampling rate 1:1.

Sampling rate can be configured when new flow log file is added, or changed for the existing exporter.

Azure flow log sampling

Flow log exporters for Azure before this release were supporting only Unsampled mode, where all the flows from the flow log file were processed by the Kentik Data Engine.

Since for some situations full flow log visibility might be not required, we added sampling knob that allows users to configure sampling rate 1:N format (meaning 1 out N records to be picked up for an ingest into Kentik Data Engine), where N should be between 2 and 2000.

Customers who are using Azure tags within their cloud infrastructure can import those tags into Kentik and use them as a Custom Dimensions within Kentik Data Explorer.

This import can be configure as any other Custom Dimensions configuration, under Settings - Custom Dimensions - Add Custom Dimensions

After that user is able to pick up which Azure Entities will be used to generate the Custom Dimensions from:

After Kentik will populate custom dimensions that were chosen (can take up to 30 min), they became fully available in a Data Explorer, under the section “Custom”.

Note: other custom dimensions are shown on the picture as well. Dimension that was generated from Azure tag is highlighted.

We also wanted to use this opportunity and remind that AWS tags are supported as Custom Dimensions as well 🙂.

Happy flow hunting!

It’s possible now to check for a traffic flows that were denied by the NSG rules configured on a Subnet or VNET level.

There are two ways how you can see that traffic:

• It’s available on a Kentik Map as a sidebar “Details” widget (similar to existing AWS functionality)
  
• You can search for them in a Data Explorer using source and destination Firewall Action as a dimensions, and change the metric to the flow/s.
  
  This feature will be a significant aid in troubleshooting the NSG firewall issues and decrease mean time to resolution.
  
  
  
  

Kentik Data Explorer supported filtering based on instance tags for a while now, but now we are expanding the tag support for other cloud objects as well, such as: VPC, Subnets, ENI, VPC endpoints and Transit Gateway attachments.

No matter what object you assigned a tag on, you can use Data Explorer for filtering traffic flows related to those tags.

In order to start using the AWS tags you need to create custom dimension through the Settings - Custom Dimensions - Add Custom Dimension.

After that you can automatically populate your custom dimensions with AWS tags and pick what fields you want to use in your Data Explorer for filtering.

After Kentik will populate custom dimensions that were chosen (can take up to 30 min), they became fully available in a Data Explorer, under the section “Custom”.

This will allow you to filter the traffic flows using the tagging and naming convention of your organization, for instance to see the traffic of a particular team, business unit or division.

Happy flow hunting!

We’re excited to announce our beta launch of Kentik Kube, an industry-first solution that reveals how K8s traffic routes through an organization’s data center, cloud(s), and the internet.

With this launch, Kentik can observe the entire network — on prem, in the cloud, on physical hardware or virtual machines, and anywhere in between. Kentik Kube enables network, infrastructure, platform, and DevOps engineers to gain full visibility of network traffic within the context of their Kubernetes deployments — so they can quickly detect & solve network problems, and surface traffic volumes from pods to external services.

Kubernetes cluster running on AKS, displaying traffic and latency to the front end of an online shopping site.

Why we built Kentik Kube

Kubernetes has become the de facto standard for cloud-based applications. As companies migrate their workloads, ensuring the reliability, connectivity and performance from user applications and their clusters, to the entire infrastructure and internet is critical.

Very often, pods and services experience network delays that degrade a user’s experience. It is difficult to identify which Kubernetes services and pods are experiencing network delays. The complexity of microservices leaves engineers wondering if the network reality matches their design, who are the top requesters consuming Kubernetes services or which microservices are oversubscribed, and how the infrastructure is communicating both within itself or across the internet.

Kentik Kube use cases

We built Kentik Kube to provide visibility for cloud-managed Kubernetes clusters (AKS, EKS, and GKE) as well as on-prem, self-managed clusters using the most widely implemented network models. Teams responsible for complex networks can:

Improve network performance

• Discover which services and pods are experiencing network latency
• Identify service misconfigurations without capturing packets
• Configure alert policies to proactively find high latency impacting nodes, pods, workloads or services.

Gain end-to-end K8s visibility

• Identify all clients and requesters consuming your Kubernetes services
• Know exactly who was talking to which pod, and when.

Validate policies and security measures

• See which pods, namespaces, and services are speaking with each other to ensure configured policy is working as expected.
• Identify pods and services that are communicating with non-Kubernetes infrastructure or the internet — when they should not be.

How Kentik Kube works

Kentik Kube relies on data generated from a lightweight eBPF agent that is installed onto your Kubernetes cluster. It sends data back to the Kentik SaaS platform, allowing you to query, graph and alert on conditions in your data. This data coupled with our analytics engine, enables users to gain complete visibility and context for traffic performance inside and among Kubernetes clusters.

Mapping your network with Kentik Kube

Kentik Kube provides east-west and north-south traffic analytics inside and among Kubernetes clusters. 

Network map showing EKS clusters communicating between AWS regions.

Kentik Kube can display details so you can see if your route tables, NACLs, etc. are all configured correctly. You can drill down into a cluster to see if there are latency or other issues. Our eBPF telemetry agent deployed into these clusters lets you see the traffic between nodes and pods as well as any latency.

How to get started with Kentik Kube

Kentik Kube is now in beta. You can apply to trial the beta by clicking on the Kentik Kube section of the menu. Please share your feedback with us. We’d love to hear what you think.

We continue developing new features for the connectivity checker in order to ensure better user experience for our customers

• It’s now possible to run an ad-hoc test without creating a report first. This test can be saved as a report for later use.
• New source and destination types. You can run a test between subnets, network interfaces and instances. All of the source and destination types are searchable using their names.

• You can start a connectivity test directly from the object on the topology, with the source being automatically pre-filled.

• Direct link to the AWS console is added on under details of impacted objects for easier failed connectivity test troubleshooting.

Data Explorer allows you to drill into your traffic flows for better understanding of what is going on in your network. Data Explorer uses different dimensions and filtering to show you the flows of interest. 

ENI Entity name and ENI Entity type were added to the list of supported dimensions on AWS. This allows users to see and filter traffic to and from AWS services such as Load Balancers and VPC endpoints. 

Observation Deck is our vision for a new focal point of the Kentik platform. It brings together all of our products & modules (current and future), in one place, so customers can get a complete and tailored view of their network and infrastructure. Every new customer will get a curated, default Observation Deck at the end of their initial onboarding. Observation Deck can then be configured to meet their exact needs.

Now customers with Azure Cloud infrastructures can add widgets to their Observation Deck and the Kentik Cloud landing page has been extended to show Azure views as well.

Note the addition of “Cloud Provider Filters” at the top right. This allows to visualize either a multi-cloud environment (“All”), or a specific Cloud. Traffic/flow widgets will filter data by cloud provider if selected.

Azure regions are now shown on the Weathermap.

For a multi-cloud environment, here is how this looks, including the Cloud backbone traffic.

Setting up monitoring of Cloud environments with Kentik can be difficult, especially when users run large multi-account environments. Ensuring that each account has the proper roles with the necessary permissions for Kentik to retrieve metadata and flow logs can be challenging to maintain at scale. That’s where our Cloud Config status comes in. It quickly analyses complex environments to ensure that all accounts are configured as required for Kentik to monitor Cloud networks. This capability is now available for Azure in addition to AWS.