GeoIP is one of the critical enrichments provided by our ingest layer to your Network or Cloud telemetry.

As our experience with GeoIP data has matured, we have realized that GeoIP datasets from commercial providers are most often strongly accurate for broadband/mobile providers; only moderately accurate for Content Providers, and not accurate at all for Backbone providers.

We have just rolled out an update to the Kentik platform that constantly augments our GeoIP records leveraging an extendable selection of datasets.

State of the GeoIP union

Originally, GeoIP was used to map source_IP and destination_IP from Netflow/sflow data in our unified, enriched flow record data structure, making City, Region and Country available as Flow Dimensions for querying.

Later in the life of the Kentik platform, these GeoIP mappings got used in the portal, under the hood, in a large number of areas, such as:

• In Synthetic testing; we compute the distance of a test based on the sum of the distances for all hops in the traceroute of a Network Test. If GeoIP is off for certain hops along the way, the distance will be incorrect, affecting our inference of the end-to-end latency when we compare it to the latency values obtained during the test.
• In Kentik Market Intelligence; we rank network providers in any market based on the amount of IP prefix space they get from their customer base (the more IPs, the higher the score). This means that the weight of a network can be overplayed or downplayed in the case of incorrect GeoIP data.

How did we fix it?

We initially relied on a simple system that takes in GeoIP data daily from our providers and reloads it into our ingest layer to constantly fetch the provider's updates, and apply them as soon as available.

When you as a customer would notice inaccuracies, we'd relay the evidence to our provider and they would surface them once blessed by their experts in a future update. We were not satisfied with the end-to-end time to satisfy customer requests for relocation so we built an override layer system to which we could feed both:

• Manually, by entering our own overrides
• Programmatically, by using additional external trusted datasets

We landed on a modular and layered architecture below, that does the following with each daily run:

1. fetch the provider GeoIP Dataset
2. overlay our own overrides based on their respective precedence/priority
3. generate a resulting GeoIP custom Dataset
4. swap the dataset on the fly on our ingest memory datastore as part of the daily job

Leveraging your SNMP data

Network Devices exporting flow data to Kentik can have SNMP enabled on them and fed to our ingest. One of the MIBs polled by our SNMP service is the interface MIB. With each poll, we get all interfaces for the network element and the configured IPs on these interfaces.

As part of our enrichments, our customers also declare their network elements in Sites, so they can later query telemetry data by site. The bonus is that users submit an address that we translate in Geocoded data so we can place sites on a map.

Every day, we scan the entire partitions of IP addresses learned on each device via SNMP, and for each public IP address, we associate it to the Geocoded data of the site, which contains the network element, which contains the public IP address.

This is a somewhat unique dataset that no GeoIP provider out there has at their disposal. We now leverage it daily as a "Layer" of overrides superseding this base GeoIP dataset from our provider. 

Benefits for our customers

As of now, if you ever notice an inaccurate GeoIP mapping when using Kentik, we have reduced the time to correction by as much as 15x: while the typical back and forth process with the GeoIP provider would potentially take 15 days (we vet, then the GeoIP provider vets, then slots it to their next synchronous release).

Today, we are able to add an override to our GeoIP system immediately after we have vetted the evidence you submit, cutting this turnaround time to one day or less.

For critical updates, we can even have the engine recompute a complete map on demand.

As of today, our public link sharing capabilities got extended to now include dashboards.

As displayed in this recent update, you can try the feature on a dashboard with the new Share console that appears when you click on the top-right share button:

The new Share console as displayed from a dashboard will look like so, where you can easily change the human-readable URL, rope in email recipients to share it with (including an explanatory message)

And here's a sample example of a publicly shared dashboard:

Why don't you play around with the feature and show us some of your most eye-candy public dashboards ?

Today, we just released a significant update to the way Kentik users will share the Kentik network observability experience, inside and outside of the company.

We have taken a set of horizontal features and have grouped them into consistent UX patterns for sharing Kentik content, whether these are reports, public snippets or even subscriptions. Read on!

What’s new?

With this update, we’ve centralized the Sharing functionality and streamlined the way it is presented on all screens of the UI/UX. As explained earlier, this creates a more holistic approach to sharing data from Kentik inside and outside.

This includes multiple areas of work, detailed in the sections below.

A redesigned, streamlined Share button on every screen

Extending the subscription capabilities to many more screens

Historically, subscriptions were only available for dashboards, we wanted to do more to meet Kentik users “where they live”.
We extended subscriptions to a discrete set of screens/workflows, with the intent of furthering this list in the future. Subscriptions are available immediately for the following items (we followed our users and their feature requests to make a determination on which to work on first):

• Dashboards
  We’ve added a few under the hood goodies such as the ability to treat carriage returns properly in subscription email messages, as well as offering another previously unavailable frequency: “Last Day of the Month” (useful to vet connectivity invoices at the end of the month)
  
  
• Saved Views
  Subscriptions were not available for Saved Views previously. They are now.
  
  
• Connectivity Costs
  We’re offering here a compelling subset of functionality, 4 types of subscriptions can be configured: Summary (corresponding to the summary screen), Provider(s), Site(s) and Connectivity Type(s).
  Each one of the non summary subscriptions allows Admins to configure which set of Providers/Sites/Connectivity Types to configure these subscriptions for to bundle in one delivery.
  
  
• Capacity Planning
  As for Connectivity costs, users can now subscribe to a summary (aka front page) or a subset of Capacity Plans.

Extending the CSV export functionalities to many more screens/workflows

A significant number of non-technical users have requested (via FRs) to be able to download CSV data for some of the most used workflows, and be able to massage it further into their own reporting.

Additional CSV exports available

First and foremost, CSV report download is now available in these workflows/screens

• Saved views

• Connectivity Costs

  • All connectivity costs
  • Per provider
  • Per site
  • Per connectivity type

• Capacity Planning

  • All plans details
  • Any number of plans together in a single CSV

• Kentik Market Intelligence

  • Rankings CSV
  • Customer and Provider CSV for any given network

To further streamline the experience, all reports download actions are now located in the exact same location, regardless to the screen the user is on, that is within the [Action] button as it initially was for Data Explorer.

CSV downloads now available for subscriptions

Not only have these CSV capabilities been extended, but they are now also available in subscriptions, which answers multiple Feature Requests we had had on Capacity Planning and Connectivity Costs.

Below is an example of a Connectivity Cost report being sent 

• for Transit and Paid Private Peering
• As a CSV spreadsheet
• The Last day of the month

Multiple convenience UX changes to make this feature set work better

Settings + relocating Public Link Shares

While initially in the Navigation panel, Public Shares have now been moved together to the Settings screen, in their own “Share” panel to the right, bundled together as a cohesive functional unit:

Subscriptions & Sharing features in Library

1. Library now gives a clear indication of when a given Saved View or Dashboard is involved in a subscription: the underlying goal is for admins to know before modifying a view that other users may be relying on it.
   
   
   
2. Users can also now both subscribe and unsubscribe directly to a Saved View or Dashboard from the library by using the contextual actions:
   

Once Public Sharing is finalized for Saved Views and Dashboards (currently ongoing), a similar marker will be displayed.

Redesigned Report Subscriptions management screen

As displayed in the previous screenshots, the Report Subscriptions UI can now be accessed directly from the Sharing Modal, as displayed below:

The Report Subscriptions management screen has been significantly updated and now offers all the convenience features that our usual list screens offer, such as the search panel on the left.

Amongst the very useful items, Users can now quickly identify using filters:

• Subscriptions that they are recipients of
• Subscriptions that have non-kentik account emails in them (aka Shared Externally)

These two situations being potential data exfiltration threats, the UI allows Kentik Admins to identify them easily.

Subscription Auto-Sense at share time

As users ideally want to limit the amount of email subscriptions being created around the same screen/workflow/report, the sharing modal performs some amount of magic:

If a subscription already exist when the user hits the [Share] button and goes to the Subscription tab, the Subscription Title entry will offer to either

• Modify an existing subscription
  OR
• To create a new one

Sample share screen displaying the choice between "Create New subscription" vs "Edit Existing Subscription"

This new feature is the beginning of a broader Kentik initiative to share the Network Observability love outside of the Kentik user crowd. As a lot of our users have asked for means to be able to reliably and systematically share Kentik visualization outside of their own silo, all the way to public audiences, Kentik now offers Public Link Sharing ! 

Sharing is caring

Publicly accessible visualization pages can now be created from Data Explorer visualizations and Synthetics test results and shared as links with people who aren’t registered Kentik users.

As of Friday, February 25th, users can use Public Link Shares. This feature allows users to share Data Explorer visualizations and synthetic monitoring tests with unauthenticated users. This feature has many useful applications, such as helping:

• Efficiently instrument troubleshooting sessions internally and externally
• Show evidence of outages and performance issues to vendors
• Augment support responses to customers with actual, interactive evidence
• Build brand confidence by transparently communicating on outages and their impact
• Empower your product and marketing teams with tools to showcase the performance of your infrastructure

Each public share created comes with a public URL in the form of: https://portal.kentik.com/share/<view-type>/<entropy_hash>/<title>

You will then be presented with a list of options including the ability to send an email to notify public users and get them to navigate to the share.

Native email notifications

A public share notification email will look like the one below:

Alternatively, Kentik Portal will copy the public URL for the new public share upon creation. Users accessing this URL will not be required to authenticate and will be able to visualize the same chart on a public interface.

Managing your Public Shares

Users will be able to modify, delete and audit existing shares via this new navigation menu entry:

Useful pointers

• For more information on using Public Link Sharing documentation is available in the Kentik Knowledge Base.
• Also see our blog post, Network observability, now publicly shareable that describes some use cases for the new Public Link Sharing feature.

The Actions menu in both Dashboards and Saved View now contains all the actions that Data Explorer would offer in this menu. This has been a long time requirement from our customers and we are happy to get this delivered.

Additional features include:

• The ability to copy a saved view into a dashboard
• The ability to generate a report subscription directly from Dashboards and Saved Views
• Show API call on saved views
• And many others

As one of our customers’ go-to modules in Kentik Portal, the Library was due for a facelift. Some of Kentik’s customers have as many as 500 dashboards and saved views. We decided to make improvements to handle large numbers of dashboards and saved views.

See the product update that details the highlights of this new version.

Some of these include:

• User manageable Categories to classify content
• Drag and drop Categories
• Guided Mode dashboard inputs straight from the Library
• Favorites and Recently Used are back!
• Multiple contextual actions to do more directly from the Library
• Ability to create report subscriptions directly from the Library context menu
• Keyboard shortcuts

We know that many of our users are fond of Kentik’s dashboarding capabilities. A quick survey of our user population shows that some users save as many as 500 dashboards or views. Taking this to heart, we’ve entirely redesigned and optimized the Library module. 

Read on for a summary of the great new things we’ve done.

Shortcuts, shortcuts, shortcuts

First, we wanted to make sure the Library is easier to access. For that, we have added keyboard shortcuts to quickly summon the Library from anywhere in the portal.

Try SHIFT+? to see a list of available shortcuts. There you’ll see you can bring up the Library by simply hitting SHIFT+L.

User set categories

A top request from users is to be able to create and manage categories in the Library. Now you can create a category on the spot! To do this, click on the top right Add Category icon on the Categorized Views screen.

You can also now rename categories using the triple-dot icon to the right of categories:

Content can be moved between categories by using drag-and-drop. You can auto-open categories using drag-hover, scrolling up and down when dragging a category above and below the fold.

Content that hasn’t been placed in a category will sit in the Uncategorized Views section of the main Library screen, waiting for you to find a new category home for them.

If shortcuts are your thing, we even added a very useful shortcut. When SHIFT-clicking on a category’s expand or collapse arrow, you will now be able to expand or collapse all categories in the Library screen.

Be more productive

Further study of some of our sample users’ portal behavior shows that workflows often gravitate around frequently eyeballing a discrete set of dashboards and saved views, so we’ve made improvements to help.

Favorites and recently viewed panes

The right side of the Library screen is now dedicated to two always-on Favorites and Recently Viewed panes.

You can favorite (or un-favorite via the star icon toggles) any dashboard or saved view from anywhere in Kentik Portal, whether from the Library or in the central content section. You can also favorite in the always-on panels and directly from dashboards and saved views.

Beyond that, you can also favorite Kentik Presets content from the second tab of the Library screen!

As you visualize more dashboards and saved views, your recently viewed items will stack up in the namesake Library panel.

Quicker access to Guided Mode dashboards

Guided Mode dashboards are a long-time favorite, and you can now directly input the pivot value from the Library screen.

The Guided Mode input will also be available directly from within both the Favorites and Recently Viewed panels.

Contextual actions galore

A handful of actions can now be taken directly from the Library entry of dashboards and saved views. These will reduce the number of clicks needed to visualize, and allow you to immediately select the necessary action:

If you are a My Kentik portal tenant, you can now directly export and add saved views to dashboards, clone, create subscriptions and even view directly. You too are one click away from the Library screen!

Our Connectivity Cost workflow is enhanced and now available!

We are excited to announce both a complete redesign and enhancements for our Connectivity Cost workflow. The new and advanced capabilities make managing and optimizing connectivity costs a cinch. All tedious work done manually to calculate connectivity cost extracting and munging data points from different silos into endless spreadsheets is now gone. You are most welcome.

You can now routinely and timely answer analytical interconnection decision-making questions, such as:

• How does our overall transit cost compare to our peering costs?
• Is it worth it to pay for this specific local provider vs. the current transit upstream?
• Cost-wise, how do these two transit providers compare? Should one be canceled? Or added?
• How have my costs evolved for this provider over the past X months?

Be ready to negotiate and optimize connectivity cost at any time with Kentik’s Connectivity Cost workflow.

What Makes Connectivity Cost Different?

Our Connectivity Cost workflow enables efficient management and planning of connectivity spending per Mbps across connectivity type, provider, and point of presence (POP)/sites. Kentik customers can now effortlessly observe cost evolution, in any currency, against strategic goals, and act timely to implement more cost-efficient connectivity agreements.

See the newly enhanced capabilities in the following sections.

Manage Connectivity Cost from Provider, Connectivity Type, and Site Perspectives

Get provider costs broken down by provider, connectivity type, or sites to track and optimize your cost per Mbps. Efficiently consolidate invoices and instrument contract negotiations with an understanding of current and historical data.

The below screengrab illustrates Kentik’s Connectivity Cost workflow providing total monthly spend and cost per Mbps benchmark metrics.

Automate Cost Calculation

Get provider costs applied to your network traffic volume automatically and monitor your monthly spending.

With the enhancements in our Connectivity Cost workflow, contract pricing data (typically kept in forms in the legal department and used in manual cost analysis) is now part of a streamlined process that automatically applies pricing to the interface traffic for cost calculations. All popular computation methods, including flat, committed, with or without tiers, and configurable percentile, are supported out-of-the-box. We understand that pricing agreements can have unique conditions that are hardwired in generic formulas, so the enhanced workflow also offers flexibility to accommodate custom pricing items, as “additional charges,” at contract or interface level (handy for cross-connect costs).

In Kentik style, calculations complexities are handled behind the scenes and your own unique cost items are accommodated as well. It is the end of time-consuming toil and out-of-band adjustments; all is done in one automated workflow.

See the image below:

Handle Global Connectivity and Currencies Fluctuations

The Connectivity Cost workflow also supports cost models of local contracts in different currencies and the dynamics of exchange rates. Bandwidth procured in bulk is sensitive to exchange-rate variations, adding another layer of complexity to cost tracking. Kentik makes it completely transparent by embedding exchange rate into daily calculations, ensuring that cost estimates for the ongoing month, as well as cost history for the past months, will always:

• Allow financial planning to consolidate all cost metrics in a single currency, and
• Keep both current month estimates and historical data truthful to the exchange rates when they are computed.

As shown below, Kentik’s Connectivity Costs workflow now natively addresses multi-currencies:

Optimize Connectivity Cost and Cross-functional Work

Connectivity costs are metrics consumed by teams other than network operations and data is rarely easy to ingest and parse for the non-technical crowd (whether the underlying cost models or the actual metrics).

With our workflow, multiple audiences and teams can now easily track and optimize your cost per Mbps, consolidate invoices, and instrument contract negotiations. Provider costs are automatically applied to network traffic volume and visualized in intuitive dashboards broken down by provider, connectivity type, or sites.

The Future of Connectivity Cost Management is Bright

Let’s say your teams are now reaping the benefits of the Connectivity Costs workflows. Connectivity costs are reviewed regularly, with goals set and tracked.

Think about this: These interconnection cost models are now available to the rest of Kentik workflows. The future is quite bright. All the necessary ingredients are now in place to be able to compute the cost of any slice of traffic coming in or out of your network: It can be different services that you run or offer; it can be other classes of users or customers that you identify with your Custom Dimensions logic; or even any CDN, OTT traffic, etc.

Be sure to check out our Connectivity Cost workflow and see how easy it is to stay on top of connectivity spending and capitalize on optimization opportunities.

This January, we introduce the 1st round towards a better way to manage your fleet of kproxy agents.
You can deploy kproxy agents to secure the network telemetry data between your own devices and Kentik's public Flow/SNMP/BGP ingest front-doors, configuring your routing gear to ship this data to a kproxy instance in your network, which will relay the resulting multiplex securely to Kentik's SaaS Clusters.
More information about kproxy here in our knowledge base.

Kproxy is now a first-class citizen in the portal, i.e. it comes with its own Settings screen. This screen comes with the following elements:

• Ability to name and locate your agents in sites that you've configured for other routing devices, and set flow listening IP (mostly useful for config snippets)
• 
• A new onboarding path that will let users deploy a kproxy instance in the first touch experience as a first step in the flow-generating device provisioning first touch onboarding flow.
  
  • The onboarding flow, as well as the modal from the Flow Proxy Agent screen in Settings now displays updated instructions to deploy kproxy both in an ad-hoc manner as well as using the industry-standard systemd service manager.
• The updated deployment method for kproxy now alleviates the need for user-specific API credentials to launch a kproxy instance. Company-global credentials are provided in the instructions, whether in onboarding or on the kproxy settings screen.
• Multiple screens in the existing Kentik UI have been updated to reflect a flow device’s dependency on kproxy as follows:
  • The Network Explorer Device Details page, in the “More Info” section
  • The Settings > Device page with its device details drawer
    
    

Note that existing kproxy-based users will see their kproxies show up in the UI without the site, name and IP metadata associated with them. These can freely be updated as suggested by the UI.

Interface Classification is one of the pillars behind an optimally efficient use of Kentik for Network Observability: it helps our underlying query engine understand what enters, leaves, goes through your network. Via Connectivity Type classifications, it brings business color into your flow data. While programmatic interface classification based on SNMP data is achieved by our Interface Classification engine, we realize that not all of your interfaces always come with consistent descriptions. To solve this problem we're introducing this January a whole lot of additional Interface Management capabilities to help you take classification to 100%.

---

Interface IP Address Overrides

We are now supporting the capability of overriding an interface’s IP address. This override works in the same way the other interface overrides work. Here are some example scenarios:

• The ability to set IP addresses for platforms that do not support obtaining the IP addresses of interfaces from SNMP
• The ability to remove the IP address that is obtained from SNMP
• The ability to set the IP address regardless of what SNMP provides

Static Interface Classification

Users are now able to statically classify interfaces from multiple areas in the product. This feature provides a good way for companies with loosely-enforced interface naming conventions to be able to correctly classify their traffic. This feature allows users to benefit from the full breadth of Network Explorer and all modules relying on Interface Classification.

Important: Interface Classification attributes (Network Boundary, Connectivity Type, Provider) defined statically per-interface will always supersede dynamic rules defined in the Interface Classification engine. As a result, the dynamic interface classification UI will also show when such static definitions are set.

The rules evaluation UI has also changed so that statically classified interfaces appear as such when they escape dynamic rules, as they always supersede them. An orange checkmark will indicate that an interface is statically classified, as displayed in the screenshot below:

From the Settings > Manage Interfaces Screen

Multiple interfaces can be selected and bulk-configured from the top-right Classify dropdown, and users can also access a more complete Interface Attributes config screen from the edit icon button at the end of each interface’s row.

The interface screen’s right-side filter section will assist the user in identifying unclassified interfaces, statically classified interfaces making the manual classification process easier.

Additionally, this panel will also assist users in identifying interfaces that have been statically classified, offering the selectors displayed in this screenshot, prefixed with the “Overridden” mention.

Lastly, users will also be able to reset an interface’s static configuration from the single interface configuration screen, as depicted below, by hitting the “Restore” link:

From Any Network Explorer > Interface Detail Screen

As shown in the screenshot below:

If you'd like to learn more about Interface Classification in Kentik, the best place to start is this knowledge base article.