Connectivity Costs is one of the workflows in the Edge module that seems to be the most successful with our users. It allows users to centralize the financial modeling of their Transit, Peering and IX contracts in one place, and track costs for each of these throughout the month, and every past month.
For a refresher on this workflow, take a minute to read this blog post from 2021 on Why and How to Track Connectivity Costs.

Today, we add quite a few highly anticipated features to it, read on!

Landing page calendar drawer now collapsible

The drawer with the invoice and contract expiry schedule is now collapsible, which was a common request from users whose contract dates are always the same: it can now be hidden, and the preference will stick for each user.

Dynamic Interface Selection

In the previous iteration of Connectivity Costs, interfaces has to be manually selected and added to the relevant Cost Group in order to bind them to a contract and financial model. For networks with a vast footprint, these external facing interfaces keep coming in and out of the picture, and users demanded to be able to manage this part of the configuration more efficiently. This is now done: from the already popular Capacity Planning workflow, we have ported the Dynamic and Static Interface Selector module into Connectivity Costs.

In addition, we've improved it significantly to make things even easier to maintain - amongst others, users can now assign interfaces in a cost group based on:

• Interface Name and Description regular expressions
• Device Names, including regular expressions (comes in handy if a portion of your device names is Site or Function related - think edge-01_ams01 for an Edge Router in Amsterdam)
• Device Labels, allowing you to exclusively focus on specifically tagged devices
• Last but not least: users can now choose between Logical Only interfaces, Physical Only interfaces, or both - for instance, depending on which of the physical or logical part of an Aggregate logical interface your SNMP accounting is going to come from

Interface or Group level additional costs expressed in any currency

Users can add Cost Group level (monthly or yearly) costs to their cost groups, but can add Interface Level costs as well. Until now, the currency of the parent Cost Group would be used to evaluate these additional costs into the daily computations.

With this release, users can now select the currency for both. Here's an example of where it comes in handy: optional interface charges are often used to document cross-connects. Our customers also tend to use local providers in whichever area they are sourcing bandwidth from. This often results in Bandwidth as documented in the Cost Group financial model being expressed in the local currency, and data-center cross connects (usually from a global data-center contract) expressed in another currency. Problem solved !

While at it, we made the Interface-Level UI for adding/editing costs more usable, see for yourself - in the example below, the cost group is in dollars and the interface charges for the cross connect in Euros. 

New Computation Methods

Based on feedback from our users, we also added a few new computation method flavors as to how our engine should compute monthly invoices. To the existing Peak of Sums and Sum of Peaks we added two new models:

• Peak of Sums (Multiple interface sets)
• Peak of Sums (Max of In and Out)

And to make sure our users select the right one when configuring a cost group, we also gave better in-UI explanations of how each computation method works in the back-scenes:

What comes next ?

In future quarters, we are going to both extend the workflow to deal with non-edge costs, and also create functionality for users who have documented their costs into the workflow to be able to price any slice of their traffic.

Stay tuned, and in the meantime do let us know what you'd like to see next in this workflow !

Early this year we launched Kentik Market Intelligence to spell out the internet interconnection ecosystem for you. It crunches large amounts of public BGP routing data, and scores and ranks all networks in any market based on the size of their customer base.

With this new release, we've added a significant feature set: Kentik Market Intelligence Insights. 

KMI Insights? Tell me more...

Insights are the relevant news items around the networks and markets you care about. When crunching the billions of BGP dump entries multiple times a day, our platform now identifies interesting insights that get contextualized in the KMI UI, coloring them with the networks involved and the market they are observed in.

Sample insights can be: <This provider> added <this transit customer> within <this geography> for instance, or any network adding new providers, as well as rank changes and changes in routing announcements between two networks.

Each insight comes with a certain amount of mandatory attributes: 

• A Customer Network: as identified in the Provider/Customer ASN relation we crunch with every run
• A Provider Network: as identified in the Provider/Customer ASN relation we crunch with every run
• A Market: the market this insight applies to
• Insights type:  as explained before, our engine classifies insights based on their nature (see list in the screenshot above)
• Magnitude: an arbitrary 1-5 value that helps order these insights from most important to least important, which is most often correlated with the change in prefix score, either plus or minus, between two networks

All these attributes have two purposes: helping the Kentik Market Intelligence UI display them in context (see further down this announcement), and allowing users to slide and dice these insights and tailor them to any subset they'd like to keep track of.

Contextualized insights

A new "Top insights" panel now appears on the right side of the landing page, it can be collapsed/expanded using the top right [insights] toggle.

The landing page will display unfiltered insights, ordered by magnitude, and this list of insights will by default follow the market filter used on the landing ranking screen.

This side panel can be configured to include or exclude specific types of insights, or to extend the period covered by the insights.

When navigating to any Network Details screen, the insights right-side panel will be displayed on the "Overview" tab for this network, and filtered to focus on the investigated network.

Track Networks and Markets that matter to you !

A while back, we introduced Observation Deck, which is a place for every user to compose their landing page with the areas of network visibility that specifically matter to them, based on widgets from specific workflows and areas of the product.

Kentik Market Intelligence now has its first widget and it's powerful!

In the future, we will add more KMI-related widgets to the Observation Deck so that you can embed widgets showing rankings and visualizations from KMI.

We hope you'll enjoy this update as much as we do. Please send us your feedback on it, we'd love to hear what you think!

Kentik added support for 6PE technology and BGP IPv6 Labeled-Unicast address family to be used for Flow enrichment.

The IPv6 Provider Edge (6PE) is the technology that enables isolated IPv6 networks to communicate using MPLS LSPs over an IPv4 MPLS backbone network.

The diagram below demonstrates the typical scenario for the use of the 6PE technology. CE routers which are at the border of the IPv6 islands, advertise their IPv6 routes to the 6PE routers of the MPLS network. These PE routers are the only dual-stack routers, which support both IPv4 and IPv6. The 6PE router advertise the received IPv6 routes to other 6PE routers using MP-BGP session over IPv6 Labeled-unicast address family. These route have:

• Original IPv6 route received from CE
• Inner MPLS label value, which would be used in the 6PE router’s data-plane to encapsulate packets toward IPv6 island’s networks.
• Next-hop with the IPv6-mapped IPv4 address which is in the form of “::FFFF:<IPv4-address>”.
  • The mapped IPv4 address is the address of the advertising 6PE router.
  • It determines the Outer MPLS label which will encapsulate IPv6 packets inside the MPLS network

To perform enrichment based on the 6PE information, Kentik’s user should establish the BGP session with the IPv6 Labeled-unicast AF between their 6PE router and Kentik. Based on the received IPv6 LU routes the Kentik’s ingest layer would be able to enrich the Flow’s received from those 6PE routers. All “standard BGP” dimensions would be populated, but more specifically:

• The “Next-hop IP” dimension will be populated with the next-hop IPv4-mapped IPv6 address from the received route
• Based on the IPv4-mapped address, the Kentik ingest would identify the next-hop 6PE router and populate “Ultimate-Exit Device” dimension and based on that, the other “Ultimate-Exit” dimensions.

Additionally, to instruct the use of Labeled-unicast routes, a user needs to select additional configuration for 6PE routers in the Kentik.  At the Settings → Devices page, select the 6PE router device and “Edit device”, then at the BGP Tab, at the “BGP route selection” drop-down menu select the option  “VPN table, fallback to Labeled-Unicast table, fallback to Unicast table”.

The example of the Data Explorer output with the 6PE BGP next-hop dimensions is shown below:

Kentik has added a new configuration option, which determines how the BGP routes are selected for flow enrichment process. To make the whole process clear enough we should start with the basics.

BGP sessions

BGP session between customer’s router and Kentik can be established over:

• IPv4
• IPv6

Since these are “Multiprotocol BGP” sessions, for each of the sessions, it is possible to enable multiple Address Families, for example: Unicast, Multicast, Labeled-unicast, L3VPN, Flowspec, etc.

These Address Families are defined with AFI (Address Family Identifiers) and SAFI (Subsequent Address Family Identifiers) attributes. They are regulated by IANA and the exact values can be found on the following links:

• IANA AFI numbers: https://www.iana.org/assignments/address-family-numbers/address-family-numbers.xhtml
• IANA SAFI numbers: https://www.iana.org/assignments/safi-namespace/safi-namespace.xhtml

The Kentik side of the BGP peering with customer’s devices will be enabled with the Unicast, Labeled-Unicast and L3VPN families by default. For the BGP “IPvX” session from the Kentik side will have the following AFs enabled:

• “IPvX” unicast
• IPv4 and IPv6 labeled-unicast
• “IPvX” L3VPN - (IPv6 L3VPN address family is not used)

Received routes from each of these address families are stored in the separate route table, which is check during the Flow enrichment process.

NOTE: IPv6 VPN routes are received, but not used for the enrichment

The Flowspec address family will be enabled only if the customer explicitly enable it in the device configuration on the Kentik portal.

BGP attributes enrichment process

Assignment of the “Route Prefix/LEN” dimension

The assignment of the Src and Dst Route Prefix is the following:

• Src and Dst Route Prefix dimensions are first populated from the Flow information using Src and Dst Mask field from Flows - if applicable.
• Src and Dst Route Prefix will be overwritten further in the ingest processing if there is a matched BGP route.
• The way to know if the Src or Dst Route Prefix is coming from flow or BGP is by observing other BGP route attributes:
  • if the Route prefix originates from the flow information the dimension “Next-hop AS Number” will be “0 - -Reserved AS-,ZZ” and the dimension “AS Path” will be empty.
  • if the Route prefix is overwritten by the BGP information, the BGP related dimensions such as “Next-hop AS Number” and “AS path” will be populated

VRF metadata collection

As the part of the SNMP interface discovery process, Kentik SaaS or Kentik kproxy will perform the VRF discovery and interface association. This information about the VRFs is collected over SNMP using MPLS-L3VPN-STD-MIB, if the device supports it. The devices from Cisco and Juniper Networks support this MIB. We have also developed support for for Nokia’s proprietary MIBs.

For each VRF, Kentik collects:

• Name
• Description
• Route Distinguisher (RD)
• Route Target (RT)
• Interface association

BGP route matching process

The enrichment of the BGP/Route related Flow dimensions is performed as a result of matching the Flow’s IP address against the BGP route received from customer’s device over BGP sessions. The default behavior of the matching process is the following:

• Flow’s Src interface is checked if it is assigned to the VRF.
  • If the source interface is in the VRF, flow’s Dst IP address is looked-up against the BGP VPNv4 routes with the RD associated with the source interface’s VRF:
    • If there is a route match, the route will be assigned to the flow
    • If there is no match, or there is no BGP VPNv4 table at all, or even no L3VPN AF established as part of the BGP peering, the match will not be found and BGP route dimensions are not populated.
  • If the source interface is not in the VRF, flow’s Dst IP address is looked-up against the “global” BGP table containing Unicast IPv4/IPv6 AF routes.
• The same process is performed for flow’s source IP address route lookup, based on the destination interface association with the VRF.

BGP route selection configuration

To address some additional scenario’s that we have seen in the customer’s network, Kentik added the configurable option to influence the BGP route selection process related to which BGP routes will be used for matching process.

This configuration is available at the Settings → Devices → Edit Device dialog → BGP Tab.

At the dialog, there is a new drop down menu called “BGP Route Selection” with the following three options:

• VPN table for VRF interface, Unicast table for non-VRF interface (default option)
• VPN table, fallback to Unicast table
• VPN table, fallback to Labeled-Unicast table, fallback to Unicast table

The following table describes the behavior of each configuration option:

By popular demand, we've added a new dimension named Site Market in Kentik Portal. It allows our users to group sites within Markets and use this new dimension in all parts of the product to filter, or group by.

When is this feature useful?

We've heard many times the situation where having multiple PoPs in the same metropolitan area makes it difficult to pivot Guided Mode dashboards around.
For instance, it is not uncommon that large networks have multiple PoPs around Los Angeles, say LAX1, LAX2, ..., LAXN. Most of the time these networks want to consider all of these as a single Metro: enters Site Markets, slap a "Los Angeles Metro" Site Market Label on all of these and you can now use it as a Dashboard Guide Mode parameter !

With its Ultimate Exit Site Market, and its special "Site Market doesn't equal Ultimate Exit Site Market" large networks are going to be able to perform useful queries telling them when traffic form one of their peers isn't hot-potato'ed and carried at higher cost over long distances. 

How can I configure the feature?

You can navigate to Settings > Manage Sites where you'll notice the new Site Market attribute. You can create a new Site Market on the spot or pick an already existing one.

From the Manage Sites screen, you can even Bulk assign Site Market to sites, which obviously comes in handy the first time you discover the feature.

When first configuring your Site Markets, you can also leverage the filters on the Site screen to quickly select all those without a Site Market, which together with the aforementioned feature really helps moving faster.

The Manage Site Markets screen

Additionally, Site Markets now have their own Settings screen, which can be found here:

Custom Geos vs Site Markets

There is a misleading parallel between Custom Geos and Site Markets, so let's take a look at these:

• Custom Geos: map a Source Geo and Destination Geobased on
  • GeoIP data from Source IP and Destination IP
    and
    Custom Geo to Country mapping from the Custom Geo user configuration
• Site Markets: map a Site Market based on
  • Site to Site Market mapping from the Site Market user configuration

The main differences are:

• Site Market is a superset of Sites when Custom Geo is a superset of countries
• Site Market is not directional (green column in Data Explorer's dimension selector), while Custom Geo exists in Source and Destination variants

Using Site Markets

Site Market dimensions are available in both Data Explorer Dimensions and Filters, and come with their "Ultimate Exit" pendent: Ultimate Exit Site Market is mapped from Ultimate Exit Site using the users' Site Market groupings from the configuration screen.

Finally, the Site Market filter comes with a very useful operand saying "Site Market doesn't equal Ultimate Exit Site Market" which will help networks identify traffic that does not stay local (and is therefore more expensive to transit - we trust this feature will be very useful to large Service Provider networks.

Enjoy and let us know what you think about the feature!

Kentik Market Intelligence gets a couple useful improvements added.

KMI will now let you export rankings and customer/provider lists as a CSV file.
Additionally, we've added a "density chart" on top of the KMI landing page before the ranking table.

The  KMI landing page has been updated with what we call a Density Chart: the usual table-chart we all know and love did not really convey the idea of the actual distance between consecutive networks in the ranking. This new widget has been added on top of the ranking table and places Networks on the line based on how far they are from each other in terms of KMI score.
(check out this article out if you want to understand how KMI score are computed)

Based on popular demand from our KMI customers, we've added a functionality to download the CSV data from both the landing page, and any given network's detail page:

KMI is a new service provider workflow that uses the global routing table to classify the peering and transit relationships between ASes and to identify the providers, peers, and customers for any AS in any geography. KMI estimates the volume of IP space transited by ASes in different geographies and produces rankings based on that volume, thereby enabling users to compare ASes in various markets.

This new workflow is available to all Kentik users with Premier Edition or with the service provider add-on available for the Kentik Pro Edition. This new workflow does not require any configuration and is immediately usable, as it relies on public routing data from a large number of BGP vantage points all around the world.

As routing data gets crunched on a daily basis, it can now be consumed via a simple interface allowing our users to decrypt how networks are connected to each other, what any network’s customer base looks like or what their providers and peers are.

KMI uses the global routing table to classify the peering and transit relationships between ASes and to identify the providers, peers, and customers for any AS in any geography.

Additionally, KMI scores and ranks any network against the size of their customer base in any subdivision of markets, as well as per customer base type such as retail, wholesale or backbone. KMI can now serve as a public, neutral and objective benchmark to score and rank all networks.

Here are a few pointers to get you started with KMI:

• Product page: Kentik Market Intelligence
• Blog post: Launching a labor of love, Kentik Market Intelligence
• Press release: Kentik Market Intelligence launches to benchmark the internet
• Knowledge Base article: Details about how the neutral, objective scoring and ranking algorithm works

In the back scenes, Kentik's OTT Service Tracking workflow leverages an engine, aptly named True Origin. This engine correlates DNS Query/Response data with network telemetry to provide a scalable, affordable DPI-lite type of service.
You can learn more about it here.

While this is something that we rarely spend time talking about, our OTT detection engine, a broadband provider’s favorite, keeps on learning and evolving as time goes by, and the amount of DNS hostname patterns that it is able to detect the underlying service for keeps on increasing constantly.

Over the month of October 2021, the True Origin engine has learned to classify an additional ~1,000 new hostname patterns to match flow with DNS queries. This month, its improvements were centered around Vietnam, Denmark, Sweden and Germany.

Kentik has always supported metric options to count unique source and destination IPs in each row of a query response. We’ve now added a new metric that shows the average bit rate (Mbps) per unique IP. This metric is particularly useful to examine if and when an issue is arising. Here are a couple of examples:

• Find the Average Mbps per Destination towards subscribers in a specific last mile aggregate of their traffic. The aggregate can be easily described in a Custom Dimension matching traffic based on a set of CMTS or DSLAM local loop aggregators, or even more simply just a Site.
• To look at traffic towards subscribers for a given Internet Access Plan. The Plan aggregation level can also be a Custom Dimension based on CIDRs. In this case, users are assigned to specific CIDRs in the IPAM based on their plan.

This “Bitrate Per IP” can be found in the Data Explorer under both “Source IPs” and “Destination IPs”:

This feature works best when paired with Kentik’s ability to detect Over The Top (OTT) services, to display Average/Max/95-99p of the Bitrate for each individual OTT service. For video-based OTT services, we now have a scalable way to calculate Average Bitrate across subscriber sets. As a result, ISPs are now able to track subscriber experience for important content sources.

Let’s look at an example content provider (OTT Service) that has their own CDN and also embeds caches within the ISP network. We’ll compare performance (Mbps per subscriber) for traffic sourced from On-Net Caches vs the OTT service’s CDN.

First, we use filters to set these criteria:

Second, we’ll use a Filter-Based Dimension to compare two time series: Traffic from embedded local caches vs Traffic served from the Content Provider’s own Network and off-net caches (i.e., long-tail content):

Lastly, we’ll select the new “95th Bitrate by Destination IP” metric:

The resulting chart below clearly confirms the assumption that the video traffic coming from On-Net Embedded Caching Servers has a higher Bitrate than the long tail traffic coming from Off-Net Caching Servers in Content Provider’s CDN.

For more information, please contact our Customer Success team.