As cloud network engineers work towards optimizing their cloud infrastructures for costs, performance, and security, they need to be able to quickly and easily investigate traffic across various public cloud regions and availability zones in GCP, AWS, and Azure. 

This detailed traffic path between cloud components is central to Kentik's Cloud offering; however, up until now, filtering for this traffic in Data Explorer required users to manually list all of the Zone to Zone options – which was inconvenient at best. So we fixed that. Read on.

Introducing a new type of filter clauses

The magic here lies in this new filter, now available for a certain list of hybrid cloud related Data Explorer dimensions: 

Source != (or ==) Destination

Here is how these look in Data Explorer now:

...using these two filter operators from the above screenshot now allow our users to filter traffic that either goes across zones (Does not equal the value of) or stays within a zone (Equals the value of) 

What dimensions is this available for?

While we were at it, we identified several other dimensions that could benefit from these new operators. Here's the list:

Beyond the hybrid cloud use-cases mentioned above, these new operators are also quite useful to look at inter-site traffic based on source or destination IP addresses as defined in the Site attributes.

Important note on Alerting
At the moment, these comparators are not available in our traffic-based alerting system.

Take these for a spin and let us know what you think!

Cloud Network Management

• Azure

  • In March, dynamic path computing in Azure Map was introduced, allowing for easier cross-subscription searches.
  • April brought Azure VNET Peering path computing, improving route visualization and destination extraction for complex hub-and-spoke topologies, with enhanced path filters and dynamic subscription loading.
  • The Azure Entity Explorer widget now includes additional data for Azure Application Gateways and Load Balancers

• GCP

  • This spring saw the introduction of custom dimensions for GCP metadata labels, simplifying data organization.
  • In April, we added GCP NAT Gateway and GCP Metrics integration into the GCP topology map, allowing for detailed entity data and on-demand metrics.
  • In May, historical metrics collection for GCP Cloud Export was introduced, enabling users to analyze past GCP element metrics.
  • In July, support for Google Cloud Firewalls was added, providing easy troubleshooting of denied traffic under the "Details" sidebar for each VPC.

Oracle Cloud Infrastructure (OCI)

• This spring, we introduced directional and non-directional gateway configurations in OCI topology for improved network management.
• In May, OCI Dynamic Routing Gateways were visualized in the topology map, enhancing connectivity insights.
• When configuring an OCI export, you can now select multiple child Compartment IDs 

Hybrid Cloud Connectivity and Cloud Core Management

• Kentik Weather Map: In May, hybrid cloud connections from on-premise to CSPs were visualized, providing insights into traffic between on-premise and cloud environments.

Kentik Weather Map

• Cloud WAN: August introduced enhancements to the cloud core network, enabling customers to inspect details of entities in the cloud WAN map via the details drawer.

Cloud WAN enhancements

AWS:

• In May, AWS flow log collection was improved with the ability to specify subfolders within S3 buckets, allowing for better log organization and management.

Kentik's Cloud Performance Monitor is a great way to see at a glance the performance and accessibility of different services provided by AWS.

This month, we added support for Aurora and Lambda services. Customers may use the Cloud Performance Monitor to see the amount of traffic forwarded to and from these services. In addition to the traffic volume, customers can also see AWS service performance information, such as HTTP latency and status codes, if they deploy synthetic agents with tests targeting those service endpoints. Set up synthetic tests that target Aurora and Lambda service endpoints to see performance information similar to what is shown below:

Note: The Cloud Performance Monitor page is dynamic and shows only services actively communicating with your cloud deployment. If your account doesn’t show these new services or the services pictured in the screenshot above, that’s likely due to a lack of traffic towards those services in your environment.

Google Cloud allows customers to configure sampling rates at the flow records generation point on a per-subnet basis, which can significantly reduce the cost of flow log generation.

The Kentik portal now displays the average sampling rate for each exporter and the number of subnets configured with sampling. This is not a configuration knob, but a value configured on the Google Cloud dashboard. The inclusion of this data in Kentik Cloud provides a quick way for users working multi-cloud environments to consume sampling rate information.

We are happy to announce the general availability of Azure Firewall logs as a datasource following its initial introduction in May.

Customers can consolidate flow records generation by using Azure Firewall as the primary source of flow information sent to Kentik. The ingest process is identical to the one used for NSG flow logs and requires customers to store records in their storage accounts.

Customers can filter traffic flows traveling through a particular firewall in the Data Explorer by using the “Logging Resource Category” and “Logging Resource Name” dimensions and then excluding NetworkSecurityGroupFlowEvent (NSG flow logs):

Note: Azure Firewall flow logs don’t have traffic throughput information, so users must choose flows/s as the metrics instead of bit/s. This will create a visualization similar to the graph shown above, and allow Kentik to see flows going through the firewall. With these metrics, customers can determine the relative load of the Azure Firewall and attribute flows to the firewalls.

For customers seeking traffic throughput information from Azure Firewall Logs, the Azure team has advised using “Fat Flows”. However, at the time of publishing this announcement, the Fat Flows feature is in preview and unavailable for API ingestion. Once it is fully supported in the API, Kentik will add “Fat Flows” support.

To simplify troubleshooting workflows in Google Cloud, we've added several new details to the Kentik Map for Kentik Cloud. In the Cloud Network Details sidebar, we've added support for displaying the Firewall Policies that apply to each VPC. We've also added support for displaying details about the Load Balancers deployed in each VPC.

The new Firewall Policies area complements the Firewall Rules area added in July 2023, which shows the actions taken by those rules—denied and allowed traffic—in the selected VPC. 

For troubleshooting, the VPC details sidebar is now the single destination for getting all information related to firewall actions and rules:

In addition, the Cloud Network Details sidebar now also contains detailed metadata about the load-balancers deployed in the selected VPC:

We are happy to announce the support for the Google Cloud firewalls for Kentik Cloud. 

For customers looking for an easy and simple way to check and troubleshoot connectivity inside GCP and see what type of traffic is being denied by the configured firewall rules, we show that under the "Details" sidebar for every VPC.

We added a new dimension for the Data Explorer, allowing customers to use the Subscription name to identify traffic flows and create alerts and dashboards.

Previously only Subscription ids were used for that purpose, and that’s not ideal, as ids are not easily humanly readable and are machine-generated.

This quarter we started working on supporting the OCI cloud.

The first iteration of the work will include the cloud map showing all the basic OCI cloud networking objects, such as regions, VCN (OCI equivalent of VPC), Subnets, On-prem assets, and VPN links.