Every network operations team has a "tribal knowledge" problem. When a critical alert fires at 2am, the outcome can depend too much on who responds — who has seen this problem before, who remembers that one strange configuration issue from six months ago, or who knows that when one signal looks normal, the next thing you should really be checking is somewhere else entirely.

Runbooks have always been the answer in theory, allowing organizations to codify that knowledge and those best practices into a repeatable checklist that can be applied consistently across the team irregardless of how good of network and infrastructure engineers they are - they used to be called NOC procedures. 

If only it were quite that simple.

In practice, runbooks tend to be partially maintained notes scattered across Confluence, Slack threads, old tickets, and sleeping brains. And even when a good runbook does exist, it is usually disconnected from the tools where the investigation is actually taking place — so you're context-switching between your observability platform and a document, manually translating "check the BGP neighbor state" into the right query, on the right device, for the right point in time.

Let's look at how AI Advisor natively includes the concept of Runbooks and how we cranked it up a notch in this release!

AI Advisor Runbooks in a nutshell

Runbooks: TL;DR

If you've been using AI Advisor for a while, you may already know that AI Advisor has a native Runbook capability designed to solve for this.
You can write, in natural (wiki-style markdown) language, specific step-by-step instructions for how you want AI Advisor to investigate a particular situation, including what to look at, what certain results mean, and what it should investigate next. 

You can either manage them in Org Settings > Kentik AI > Runbooks

...or ask AI Advisor directly

Composing with Runbooks

Here are a few cool things you can do with Runbooks - go ahead and try them

• Let AI Advisor select the right Runbook for you. Runbooks can be dynamically invoked by AI Advisor based on the context of a user's prompt, so you don't always need to explicitly call one by name. AI Advisor will also expose if it is selecting and using a rubook for an investigation.
• Run a Runbook with variable arguments. If the first lines of a Runbook define required variables, AI Advisor will ask the user to provide those inputs before execution.
  Example: Run the Peering Vetting Procedure for AS12322, where Peering Vetting Procedure is the runbook andAS12322 is passed in as the argument.
• Run Runbooks on a schedule. A Runbook can be executed at a defined frequency using Org Settings > Report Subscriptions with the AI Advisor report type, with the resulting output attached to an email as a PDF.
• Trigger Runbooks from alerts. In the near future, Runbooks will be able to map directly to Alert Policies and automatically execute as an Auto-Investigation whenever the associated alert fires. This capability is currently in Early Access with select customers and will be available more broadly soon.

Runbooks vs Custom Network Context

Now some of you may wonder what's the difference between Runbooks and the Custom Network Context that is also part of AI Advisor. To understand that, ket's look at the concept of AI Context and how it works in general. 

AI Advisor leverages incremental context within a session: this attached context is a consumable budget - it is handed over to the LLM with each prompt to help it answer the most recent question and each turn/response appends to the context, consuming budget: it is therefore a finite quantity and precious resource 

A fresh AI Advisor session launches with the default System Prompt that's common to every customer, to which we append the Custom Network Context (CNC) on the fly. This is specific information you or your organization has provided to AI Advisor that matters for understanding your network.

This is why CNC is best used for general directives and metadata that apply broadly to the entire infrastructure, because it's your default context spend.

If are familiar with Claude, Custom Network Context is basically your claude.md.

Realistically, you cannot attach every single task-specific procedure your NOC has in the CNC because of the risk of eating your entire context budget the second AI Advisor instantiates. For that, we we created Runbooks.

Runbooks leverage dynamic linking: AI Advisor fires a Runbook tool with every turn, which looks up similarities between the ongoing prompt and all of the Runbook's Title and Description (do make sure these are self-explanatory). When a match is found, the Runbook is appended to the session's AI Context, only consuming from the budget when a match is found.

Again, if you are fluent with Claude, a good way to reason about AI Runbooks in Kentik is the parallel with Claude Skills.
 
As explained, both technically and conceptually behave the same:

• Runbooks are task-specific, describe complex and multi-steps workflows (troubleshooting, analyzing, auditing, planning...)
• Runbooks are discovered via metadata (title, descriptions) and loaded only when Claude recognizes a relevant task.

Runbooks & AI Safety

Modifying Runbooks (also valid for CNC), may make AI Advisor start giving different answers overnight to users that were use to getting consistent answers from the same prompts day after day.

As a first principle, Kentik strives to provide the necessary guardrails for the entire company to use AI Safely and Transparently, which implies the following features that come for free with Runbooks:

• Any change (or creation, deletion...) on a Runbook is logged in our Audit Log facility
• Any time AI Advisor dynamically summons a Runbook, it will
  • explicitly mention it in its reasoning
  • let the user view the runbook as part of the process
• Any change in an AI Runbook populates a history, with a view to track changes across time and the ability to revert them

Building on Runbooks

Cranking it up a notch

Now that you know everything there is to know about Runbooks, a problem still stands: you can't foresee every situation in advance, and not every troubleshooting process is already documented well enough to turn into a runbook.

In fact, something we've heard from customers is that one of the useful parts of working with AI Advisor is the ability to prod and poke at an investigation — telling it to look somewhere else, providing additional context, correcting an assumption, or guiding it toward the signals you know matter in your network. 

Here's the beauty of it all: over the course of a fruitful investigation, you've effectively taught AI Advisor how you would troubleshoot that particular problem.
...and now you're telling me I need to open an extra tab, and tediously copy/past every single prompt I've entered during that session into a newly created Runbook ?  Color me disappointed!

Try a prompt like this and AI Advisor will have your back!
"- Turn this session into a Runbook
 - use this title: 'Peering Request Assessment Procedure'
 - use the following Description: "Standard procedure our Peering Managers use to evaluate peering requests from an ASN"

... and AI Advisor will create said runbook for you to keep hacking on.

You didn't give it a name or a description? AI Advisor will figure it out for you based on the content of your session.

Lastlt -- useful advice for the social Runbook writer

• Your NOC team most likely has existing wiki-based or google docs-based procedures.
  Why don't you just pick a couple of them, copy paste them into Runbooks and try them on for size? You'd be amazed how little tweaking these actually need!
• AI is a muscle: build habit around always turning a fruitful AI Advisor session into a runbook.