AI in the Stack

⚡ Byte Size Summary See why we shipped an OpenShift diagnostic MCP server as read-only by design, and the RBAC wall that made write access harder than it looks Walk through a real failed remediation test where an agent recommended a correct-looking fix built on stale, deprecated config — and what that failure mode actually is Get the maturity-gated approval architecture we designed for write access — and why it’s still sitting on paper, not in production The Story In Article 06 we wired an n8n workflow to MCP and RAG for automated incident triage. That article ended with a question: what happens when the agent gets a longer leash? ...

⚡ Byte Size Summary Wire n8n to an MCP (Model Context Protocol) server over Streamable HTTP and a RAG (Retrieval-Augmented Generation) pipeline to build an automated incident triage workflow — runbook lookup and live cluster diagnostics assembled before a human opens a single dashboard The moment you place an LLM agent node inside an n8n workflow, execution becomes non-deterministic — the model decides how many tool calls to make, so your timeout and call limits must assume worst case, not average case Start with hard execution limits (5-minute timeout, 2-3 max calls per run) and loosen as you validate — a runaway workflow with no bounds will exhaust API quotas and flood notification channels before you notice The Story I started using n8n because I watched a teammate build a workflow in twenty minutes that would have taken me a day to write in Python. The visual canvas, the drag-and-drop nodes, the instant execution feedback — it made automation feel fast. So I started experimenting. ...

⚡ Byte Size Summary Deploy a LiteLLM proxy on OpenShift to decouple your applications from individual LLM providers — switch or failover between models by changing a ConfigMap, not your code Inject cluster-specific context (API versions, cloud annotations, golden templates) before every prompt to prevent models from generating “plausible averages” that look correct but fail on your platform Validate every generated manifest with oc apply --dry-run=server before applying — the cluster’s API server catches structural errors that no amount of prompt engineering will prevent The Story I was building a GenAI data poisoning demo for a proof of concept on ROSA (Red Hat OpenShift on AWS). The demo needed OpenShift-native manifests — Routes with TLS termination, proper SecurityContextConstraints, current API versions. I built it using Claude, and it worked. The model understood OpenShift. The manifests deployed cleanly. The demo ran on the cluster. ...

⚡ Byte Size Summary Store prompts as versioned YAML manifests in Git and run them through a three-stage GitHub Actions gate — schema validation, secret scanning with gitleaks, and model policy enforcement — before any LLM call reaches your OpenShift environment A CI-gated prompt pipeline gives your enterprise auditors a traceable answer to “what prompt was active during the incident window” — without it, the forensic work is manual, billed, and slow Prompt versioning is necessary but not sufficient: you’re versioning one variable in a system with multiple unversioned dependencies, and this article shows you what to do about the rest of them The Story I was presenting a prototype at a conference. The demo was built over three weeks of late-night sessions — an AI-assisted operations assistant for OpenShift that could answer runbook-style questions against live cluster state. The architecture was solid. The underlying idea was good. ...

⚡ Byte Size Summary MCP (Model Context Protocol) is the standard that lets AI agents interact with external systems — your cluster, your observability stack, your ticketing system — without bespoke integration code for every tool. MCP directly addresses AI hallucination and 2AM incident response by grounding AI answers in live system state. It doesn’t solve tribal knowledge alone — that needs RAG alongside it. This article covers the production-grade architecture: what MCP servers are, how to design them for platform engineering use cases, and what you need to get right before running them anywhere near production. In Article 01 we built the evaluation framework. In Article 02 we built a RAG pipeline for static runbook knowledge. Now we give AI agents access to live infrastructure. ...

⚡ Byte Size Summary RAG inserts a retrieval layer between your existing runbooks and an LLM — answers come from your documentation, not generic training data, with source citations included. This article builds a complete FastAPI service with /ingest, /query, and /health endpoints, using OpenAI embeddings and Chroma as the vector store. Everything is cloneable from GitHub. The goal isn’t to replace your runbooks. It’s to make them queryable at the moment an incident is happening. In Article 01 we built a framework for evaluating where AI actually changes outcomes in platform engineering. This article implements the first concrete use case: making your existing runbooks semantically searchable. ...

Byte size summary After reading this article, you’ll have a framework for evaluating AI tools in platform engineering contexts — not by capability type, but by where in your workflow the tool actually changes the outcome. You’ll understand why the tools that sound most compelling are still hype, where genuine productivity gains exist today, and what governance infrastructure you need in place before any AI component gets near production. This article is the foundation for the series; subsequent articles implement each touch point against real OpenShift infrastructure. ...