Following on to the the release of Role Based Access Control (RBAC) in Kentik Portal we initiated a long journey which will eventually lead to legacy User Levels ("Member", "Admin", "SuperAdmin") to disappear from the product. Throughout the quarters, we are migrating existing portal features under the RBAC umbrella, so that an increasing number of functionality becomes subject to granular permissions.
With this iteration, we are adding two changes:

• we have redesigned the RBAC Admin UI both in the RBAC section, and in the User Management Section
• we are adding label-based RBAC support for Dashboards and Saved Views

Read on !

Admin interface changes for RBAC Management

We have decided to move away from a Modal based format to a full screen based format, as we felt the information presented for RBAC configuration was too dense to comfortably fit in a small modal. See for yourself in the screenshots below:

On the "Manage Users" screen, beyond the design change of the User Edit modal into a full page, we have added new capabilities such as the ability to identify which role a specific permission on a user comes from:

Label-based RBAC

We also added a novel functionality allowing our users to manage RBAC at scale: Label-based RBAC. As you may have noticed, labels have been extended to a lot of additional areas within Kentik Portal, amongst others to the "Library".

With Label-based RBAC, you can assign permissions to a role that apply to an exclusive set of labels, instead as to all elements of the same kind. This functionality is at current available to Dashboards and Saved Views permissions and allows to restrict the permission to content marked with specific labels - this is displayed in the screenshot below from the edit screen of a role.

Here are a few important gotchas about Dashboards and Saved Views RBAC permissions:

• To be able to change labels (add, remove) on a Dashboard (resp. Saved View), the user needs to have the Dashboard (resp. Saved View) Update permissions (this avoids privilege escalations via RBAC).
• Labels configured in 'Label Access" type permissions are all OR'ed together, meaning said permission is valid for any view with any of the configured labels on it.
• Assigning label-based Dashboards (resp. Saved Views) permissions supersedes the "Sharing" settings of these: if a view is shared but a given user's RBAC permissions for accessing or editing dashboards are restricted to certain labels, they won't be able to view or edit them: i.e. "Label Access" is more restrictive than "Full Access" as the name indicates.
• if a user has both a "Full Access" permission to views in one role, and a "Label Access" permission to the same view type, the resulting user permission for that view time is the union of both, i.e. "Full Access"

What's next ?

As we progress on this RBAC journey, your user input as what your most wanted areas of Kentik Portal to be RBAC'ed matters. We are all ears on what you'd like us to offer next on that front !

Some quirks in our Interface Classification settings screen made it more tedious than it should be to create and test rules. For example, a new interface classification rule being created was automatically sent to the bottom of the stack. Therefore, when testing the rule at creation time, it was evaluated before all the already-present rules.
The only way to test the effects of a rule being created was to save it, re-locate it within the stack, open it again, and see if the rule placement change fixed it.

We are adding new capabilities that alleviate this inconvenience, read on !

Interface Classification Refresher

First off, let's go back to how Interface Classification works: upon SNMP polling (frequency varies based on your device settings), the classification engine goes through your updated interfaces and runs them through the Rules stack. To do so, it runs every interface one by one through the stack of rules. Each interface is run consecutively through the stack of rules in the order which they are defined in the Interface Classification screen – starting with Kentik-managed rules first, and then on to each rule from top to bottom.
Each interface exits the evaluation loop of the stack upon first match of the IF clause, and applies the THEN action defined by the rule to classify: Connectivity Type, Network Boundary (inherited from Connectivity Type) and Provider if defined. Once the THEN action is applied, the algorithm moves on to the next interface... and so on.
An interface that goes through the list without match stays categorized as "unclassified".

When creating a rule, users are able to test it from the creation modal to make sure the resulting classification is as expected, allowing the user to see the number of interface matches per device and zero-in on a specific device to inspect the results of the classification action.

The main usability issue came when users were creating a new rule: the rule gets automatically created at the bottom of the stack and hitting the "[Test Rule]" button doesn't offer any choice of locating the rule elsewhere in the stack – the only way to do so was until now to save the rule, go back to the stack, move it, open it and test again in with the new position. This is the part that we just improved.

Adding Rules Mid-Stack

When hovering on any of the rules in the stack, users will now see + signs above and below it. These icons allow the user to create a rule Before or After the one currently hovered. Additionally the Rule creation and test context will honor this desired position.

Hitting [Test Rule] will now proceed with testing this new rule given the position it now has in the stack, instead of the default "Bottom" position it used to have.

But wait, there's more

While we're at it, we added a few more niceties:

• When clicking the top right "Add Rule" button, the editor modal will also contain the same previously shown drop-down to select where in the stack to create and test the rule
• The kebab menu at the right of every rule now contains two new options to directly relocate a rule at the Top or Bottom – we all know how frustrating it can be to drag elements on a web page after or below the fold.
  
  

Kentik is excited to announce an enhancement to our telemetry ingest capabilities with the support for IPFIX Information Element (IE) 315. Let’s dive into how this Information Element is used and why users should care about it.

Understanding the Flow Collection and Export Process

The flow collection process of network devices consist of three stages:

1. sampling of the packets: selective capturing of the network packets to reduce data volume
2. aggregation and caching: flow metadata and counters stored in device’s flow cache table
3. export of the expired flows: sending flow data to a flow collector

During packet sampling, the information from the packet headers and device’s interfaces are extracted to form a unique identifier (key) for each flow in the flow cache table. New packets that match an existing flow key will increment flow’s byte and packet counters, while unmatched packets will trigger a creation of the new flow record. 

Flows are exported to the collector based on two conditions:

• the inactive-timeout period: no new packets have been detected for a flow, which means that the flow is inactive. 
• the active-timeout period: current state of flow counters is exported even though the flow is still active.

The Evolution of Flow Collection in Modern Networks

At the time when the flow collection technology was developed, traffic volumes were significantly lower and network devices could “afford” to capture and export data of all traffic flows. However, with today's networks where hundreds of devices are handling traffic at gigabits or terabits per second throughputs, complete flow capture has become impractical. To effectively manage these traffic volumes, the sampling has become a reality in most use cases with typical sampling rates ranging from one in a few hundred to one in several thousand packets.

For near real-time traffic analysis, users would generally set inactive-timeout at 15 seconds and active-timeout at 60 seconds. For DDoS detection use cases, these counters can be configured even with lower values. Given the high sampling rates and rapid flow expiration, the effectiveness of traditional flow caching on network devices is now questionable. 

According to the research performed by Juniper Networks, with the flow sampling in the range of 1:1000s and active-timeout of 60 seconds, it is expected that around 90% of the exported flows will have only 1 packet matched. In such environments, the flow caching process on network devices does not bring much benefits. An alternative approach of directly exporting packet headers to a collector seems more effective and this is where IPFIX IE 315 comes into play. 

Introducing IPFIX IE 315

IPFIX IE 315, known as dataLinkFrameSection, carries a sample of the network packet headers, starting from the L2 protocol header up to a maximum sample length determined by the network device capabilities. This capability varies by vendor and device models, but typically supports sampling of 64 to 160 bytes. 

The typical IPFIX flow data record includes: ingress and egress interface index, flow direction, and the length and data of the sampled frame. 

Vendor support

Juniper Networks

Juniper Networks offers support for IPFIX IE 315 through its IMON (Inline Monitoring) feature. It is supported on MPC10E and MPC11E linecards for MX-series devices, MX304, linecards LC480 and LC9600 for MX10K and certain line cards for PTX10K. The feature is implemented in hardware, so there is a minimal delay and no restrictions in the volume of exported data. It supports export of 64 to 126 bytes of packet’s header. More information about the implementation, configuration and support on devices can be found in Juniper’s technical documentation. 

The flow data record includes the following IPFIX IEs:

Cisco

Cisco supports IPFIX IE 315 on its NCS 5000 and ASR 9000 devices. It uses slightly different flow records with IPFIX IE 410 (sectionExportedOctets) to store the length of the sampled frame section and does not include flow direction field. The size of exported frame is up to and including L4 header, to the maximum of 160 bytes that can be exported.

The flow data record includes the following IPFIX IEs:

Benefits of using IPFIX IE 315

The approach of IPFIX IE 315 shifts the packet decoding and metadata extraction work from a network device to a flow collector. This reduces the processing requirements on network devices and eliminates the need for maintaining a flow cache table, leading to lower CPU and memory usage and potentially simpler hardware designs. Moreover, the immediate export of packet samples enhances the detection time of DDoS attacks.

Support in Kentik

Kentik Flow ingest on the SaaS supports use of the IPFIX IE 315, with the Kentik’s default Device Type “NetFlow-enabled device”. The feature is also available on Kentik kproxy version 7.43.0 and it supports both Juniper and Cisco implementations.

Some features are hidden jewels that are hard to spot: they don't come with a shiny UI that make them stand out, but they pack some heavy punch and you wouldn't know unless someone reveals their true power to you.
This is one of those features. More often than not, assessing the benefits (aka the amount of traffic that can be peeled off to the IX) in joining a certain exchange requires so much discouraging work that these decisions end up being made on a gut feel, almost as an act of faith.

Enter the Data Explorer PeeringDB filter dimensions!

The theory

Let's say we want to evaluate how much of our actual traffic could be peered off at a specific Internet Exchange. What do we do as a Network Engineer without access to Kentik ?

1. Collect source ASN or destination AS flow data inbound and outbound, then dump it in a spreadsheet
2. For each ASN in that breakdown, list all Internet Exchanges that they are present either via leveraging the PeeringDB public API, or manually by reviewing all ASNs' PeeringDB record – some ASNs are at tens if not hundreds of exchanges...At the same time, note what their peering policy is.
3. Run that ASN list through a sieve to only keep those at the Internet Exchange you are interested in, and that have a peering policy you can comply with.
4. Sum all the resulting traffic.

Having done this routinely in my career, I can safely say that:

Either you have coding skills and you can code your way out of it... or you don't and this will take multiple hours. This outlines the issue of inefficient peering decisions: analytical decisions are gated by an inordinate difficulty in collecting and correlating simple data sets together, and the common result is that hunches and arcane culture end up replacing the science.

So how does it work ?

Here's how we wanted this to work: add filters to Data Explorer, so that users can make these types of queries – let's think SQL for a second:

Show me my top Destination ASNs where said ASNs are registered in this specific IX, with an open policy

When the query is run, Kentik's Data Engine issues a two steps query: 

1. Pulls the list of top destination ASNs, 
2. Pulls the list of ASNs at this specific IX and filter it by the list returned in the first step. 

Remember, Kentik is mirroring the PeeringDB database daily, so this information is always fresh.

and Voila !

Cool story bro, now just show me a real life example kthx.

Let's try this with a concrete example: Kentik is present at Linx NoVA. I want to know how much inbound traffic we can peel off from our Transit upstreams at this exchange. We've got a few peering sessions there, but I have a hunch I can reclaim quite a bit of bandwidth through this exercise – let's see if I'm right.

I'm going to look at these dimensions, no surprise here

But when building my filter, I'm going to use this new PeeringDB section here, where I can now set whatever IX I want my source top ASNs to be part of:

Notice how you can leverage this same dimension with a "SOURCE or DESTINATION ASN is member of this IX"

You'll notice that this filter item sources its values directly from PeeringDB, so I'll enter "LINX NoVA" for "Source ASN is Member of IX" – note that I'm filtering on Source Network Boundary = External because I just want to see the traffic coming into my network.

My intuition gets confirmed: if I look at the resulting SanKey diagram, it appears that only a small part of traffic towards LINX NoVA member ASNs I receive traffic from is actually coming through my IX port at LINX NoVA, see for yourself.
According to this query, I could peel ~384Mbps from inbound Transit traffic to this exchange by establishing the right sessions with these ASNs on the right side of diagram I'm currently receiving traffic from.

Now If I really want to get a realistic estimate, I need to ensure that I'm only considering traffic from ASNs with an Open Peering Policy - no problem: PeeringDB also has that information, so let's just add it to the filter

And so the Transit part of the previous SanKey gets revised with a bit less traffic because the query engine is now discarding source ASNs without an Open Policy:

As always, there's more!

There are more use cases that this well-hidden gem empowers our users to do:

• Reclaim Transit traffic and convert it to IX peering traffic, and more importantly maximize the use of your IX ports
• Identify PNIs to secure with IX peering as a fallback, or additional peering locations you can meet your existing peers at for more resilience 
• More than anything: evaluate which IX to deploy to next based on how much assured traffic you can peer off and estimate the port capacity you will need.

All of these are focused around IXes, but the same applies to this other filter dimension set: "<Source | Destination | both> ASN member is available at <specific datacenter facility>" – which helps you answer the same use case, but from a data center standpoint for PNIs.

This is a pivotal step towards an even better outcome: what if every day, Kentik ran a report on all of your top Source or Destination ASNs and made informed, analytical suggestions about which of them you should go next?

Watch this space, because this is where we are headed.

If you remember back in May 2023, we released our initial version of a PeeringDB integration - more details here: https://new.kentik.com/we-now-integrate-peeringdb-data-akabK

The theory

PeeringDB contains a directory of most of the registered Internet Exchanges (IXes), as show in an extract below.
 For each one of them, their data contains the IP Ranges, v4 and v6.

 For each interface that we poll on your devices using SNMP or Streaming Telemetry, we get information such as the IP Addresses configured. Using the aforementioned IX IP Range data, we are then theoretically able to match any interface to an Internet Exchange from Peering DB based on its IP Address.

What does it look like in the product ?

We've added a somewhat magic rule, which we have decided to call a Kentik-managed Rule in our Interface Classification UI. This rule is a bit special compared to the other ones in the stack:

• It cannot be edited by users
• It can only be enabled/disabled by users
• It sits on top of the rules stack, which means that it will always be evaluated first to bypass any user defined rule (because for each interface, the algorithm exits on first match)
• This rule checks the IP Addresses, and looks the IP address up in all the IP Ranges from all the IXes in PeeringDB, if a match is found
  • it sets the interface to Connectivity Type = IX
  • it sets the interface to Network Boundary = External
  • it sets the Interface Provider to the name of the IX from PeeringDB
  • it establishes the mapping between this IX and the PeeringDB Exchange in the PeeringDB integration configuration so you don't have to

Here's an example of what Interface Classification Test UI for this rule will look like in case of matches

Existing Customers vs New Customers

Obviously a lot of you may already have their interfaces well classified, and most likely you used dynamic regex matching on their descriptions to get your IX classifications going.
As we didn't want to disrupt your existing setup, we have adopted what we think are reasonable defaults to roll the feature out:

• Existing customers will have the PeeringDB rule disabled by default
  -> up to them to enable it if they would like
• New customers will always have the PeeringDB rule enabled by default

But wait, there's more...

As an additional freebie, we've modified the Interface Provider field for any custom rule that sets an interface as IX: you can now choose to keep your own provider naming, or can select the actual PeeringDB name for the internet exchange - in this case the field will offer a typeahead that directly looks up values from the PeeringDB exchange names, as showcased below.

There are still a lot of features that we plan on implementing relying on the PeeringDB dataset, so watch this space, and don't hesitate to give us feedback !

Going back and forth between Data Explorer and our Library to cycle through Saved Views when doing your rounds has always been somewhat of a tax on user productivity.
We recently decided to tackle this workflow and streamline it: we have tested out this new feature with a few of our customers before launch and are now proud to present the Saved Views Drawer feature in Data Explorer.

An expert eye may already have noticed that a new button has sneaked into the Data Explorer actions bar: 

Clicking on it will unveil a new drawer in Data Explorer, allowing our users to directly browse, search and find a Saved View to load in Data Explorer, without having to go back to the Library module.

You can now entirely navigate the list of Saved Views with both mouse or keyboard, it is ordered in 3 sections (horizontal arrow keys):

• Favorites: any Saved View you have favorited either from the Library or on the View itself with the star icon next to its title
• Recent: we're also making Recent Views persistent, so you can see the TopN last displayed Saved Views
• Browser: any view in your Library that you have access to: this will be the default tab if you don't have any Favorite or Recent

Remember, the entire functionality is entirely keyboard friendly, as reminded at the bottom of this new drawer

Lastly, plenty of filtering options allow you to find the right Saved View you are looking for in a sea of user generated views:

...these filters can be cleared at any moment via the UI displayed below

We weren't entirely satisfied with our last revamp of the Library UI so we gave it a few more iterations to make it more legible, we hope we like these new tweaks.

Top Recent/Favorites Strip

In the initial iteration of the overhaul, we noticed that the top strip would sometimes get in the way for users with little vertical real estate (i.e. low vertical resolution). It would consume a large part of vertical real estate and a large part of the content of the library would be pushed below the fold so we went ahead and fixed that:

The top Strip is now toggle-able, and your last setting will be remembered by your browser as it is stored in your local session. Beyond that, there's now a keyboard shortcut to show/hide that Favorites/recent top-strip: [shift]+[f]

Views grouping

Users can now group the views in the data-table based on a few attributes: ungrouped, Owner (who created the view), Sharing (Shared, Private, or Kentik Preset), View Type (Dashboard or Saved View)

...these groupings will appear in the list table as group header rows with counts - as depicted below when grouping on "Sharing"

New labels visual format

Finally, the previous display of labels in the list using full-color labels made it very noisy, so we rethought the way we're displaying labels - a picture being worth a thousand words:

vs the previous format

• The title of the view was made bigger
• the labels made more subtle while still showing their colors (avoiding color overload)
• the labels are now sitting next to the title (a visual treatment we stole from GitHub's clean UX)
• the labels for presets also have had their format changed to this to better match the new visual format 
  

New Label selector filter UI

We changed the Label Selector to be more efficient, and in line with the new visual format both in the filter panel, but also in the bulk interface above the list of views.
Note how labels are now sorted by 1) your company's label (alphabetically) 2) Kentik Presets labels (alphabetically) one after the other and not together anymore like they used to be.

Why all these changes on labels ?

One of the reasons we're updating the visual aesthetics of labels is that as you may have noticed, labels will take a more and more central role throughout Kentik Portal, especially when it comes to RBAC (Role Based Access Control) - and to achieve that, we want a label UX throughout Kentik Portal as smooth and legible as can be.

Moving forward in 2024, expect other areas of Kentik Portal with labels involved to receive the same visual update to match their style with this new one.

Up until now, Description text for both Saved Views and Dashboards haven't been first class citizens when users navigate to either of them and want to know what a view they haven't created themselves is all about.
We just made it incrementally better, read on.

Where we come from

Displaying Description text for Saved Views and Dashboards used to require a little bit of user gymnastics, especially for Dashboards where the only ways to visualize this description would be

• look at the description summary in the Library screen (which may be truncated if too long)
• have edit privileges on the dashboard and proceed to Action > Edit > Dashboard properties

For Saved Views, descriptions were only displayed in a rather space constrained Edit field in the query panel - again, not entirely ideal.

We've noticed that some users are Content Creators (to use a common term these days), and they create the best dashboards for everyone in the company to use - and some other users are Consumers, meaning they rarely build out Saved Views and Dashboards, but consume them heavily for their operational tasks.
More often than not, the subtlety of the view crafted by the Content Creator can escape the Consumer User (instructions on how to use, limitations of the view, how to interpret the data...) - this is what we want to improve with this micro-feature.

Where we're going

Upon loading either a Dashboard or a saved view, you will now notice a speech bubble icon next to it:

Beyond this, we bound this action to a new shortcut, [shift]+[d]so that shortcut-savvy users can access it without needing any extra click - the modal can also be discarded with the [esc] key.

Remember, all the shortcuts that are active on any screen can be displayed by using another shortcut: [shift]+[?]

What the future holds

We are already considering rich, markdown-style panels for our future Dashboard revamp (roadmap milestone still tbd) - what do you think?

As some of our customers had noticed, entering a Site address as a one liner instead of multiple fields for city, region and country came with some occasional misfires.
We've fixed this in our UI, so that you can always override these issues manually when they happen.

What was the problem?

We chose for our users to enter their Site locations via one-liner addresses. Upon saving a site, the provided one-liner address is issued to the Google Maps API and we interpret the results to store the following attributes for the site:

• City
• Region
• Country
• Lat/Long 

Address formats vary from one country to another, and so does the data structure for an address returned by Google Map's API. Based on this returned data structure, we attribute City, Region and Country based on our deciphering of this data structure, which changes for every country, meaning that sometimes we get it wrong, here's an example

...where the resulting address in the Site object is as follows

...unfortunately, the city here should be Barueri, because Tamboré is the name of the neighborhood.

The solution

While this issue is a rare occurrence, it hits some geographies much more than others, with the cognitive load borne by our customers in these locations: a Data Explorer query returning a City they have never heard of...
This is why we are now allowing our users to override these values manually by selecting a City, Region and Country of their choice if they so desire - in continuation of the previous example:

We are excited to announce the preview of Kentik Journeys AI.

Kentik Journeys takes the concept of natural language query a step further by providing an AI-assisted conversational user experience purpose-built for the process of troubleshooting and network analysis.

When troubleshooting, engineers typically ask a question, analyze the answer, then ask a new, and hopefully better-informed question. Sometimes this process is simple. Often it is not. This process is repeated until the user comes up with the root cause and then applies a fix leading to a resolution.

Journeys provide a dedicated space to perform this iterative process, saving questions and answers as the user performs their troubleshooting journey. These journeys can be shared across all Kentik users of a company to encourage communication and knowledge sharing.

The real power of Journeys is its network context. Meaning it understands your network - the devices involved, your cloud regions, on-premises sites, etc. Anyone can converse with Journeys about their specific network, within a user experience that is purpose built for the journey of troubleshooting, hence making it much faster and more efficient.

Journeys respect your Security and Privacy. Under the hood we use a Large Language Model (LLM) service from the leader in GenAI. Customer data is never used for training any models and no customer data (like flows or metrics) are ever exposed to the LLM. Some set of partial metadata (site and provider names) are supplied to the LLM prompts. On top of this, our service provider privacy policy clearly defines that no data supplied to their models will be ever used for training of AI models.

How to get access to Journeys

We would like to highlight that our Journeys capability is currently in "Preview" mode, as our goal is to incrementally enhance the user experience. We plan to do it by carefully considering your feedback and suggestions, improving accuracy, expanding the range of supported questions, and integrating additional Kentik features. 

Due to this, we are taking a phased approach to granting access, ensuring a fulfilling experience and providing appropriate support for every customer. Access to Journeys will initially require a request, and we will progressively make it available to all of our users. We appreciate your patience during this process and promise to keep you updated on when access will be available for your company.

If you are interested to participate in the preview, please request access for your company on the Journeys AI page.

How to use Journeys

When you first enter into the Journeys AI page, a short tutorial will be started that demonstrates how to use it. Please follow the instructions to get familiar with the user interface and read the instructions on how to make good queries.

We hope you give it a try and let us know what you think.

Here are some example screenshots of Journeys.

CPU usage

Traffic analysis