Kitaru by ZenML (Part 1): Architecture & Durable Execution for AI Agents

This is Part 1 of our 3-part technical deep dive into Kitaru by ZenML. In this series, we cover durable execution architecture (Part 1), human-in-the-loop wait() gates and framework adapters (Part 2), and deterministic trace replay for enterprise CI/CD (Part 3).

Production AI agents fail in ways that traditional software and classical ML pipelines were never designed to handle.

In classical MLOps, you define a Directed Acyclic Graph (DAG) upfront: ingest data, train model, evaluate, register. Every edge is known before execution starts. In contrast, an autonomous AI agent is non-linear and stateful. It loops dynamically (while not done), branches on tool outputs, retries flaky external APIs, and burns expensive frontier model tokens at every step.

If your agent crashes on step 14 of a 20-step workflow because a downstream webhook timed out or a Kubernetes pod was evicted, a standard Python script loses all in-memory state. Restarting from step 1 wastes tokens, increases latency, and risks duplicate side effects (such as issuing a second refund or opening duplicate Jira tickets).

Kitaru (from the Japanese 来る, meaning “to arrive”), built by the ZenML team, solves this problem. It is an open-source, framework-agnostic durable execution runtime and evaluation platform engineered specifically for AI agents.

1. Architectural Foundations: Kitaru vs. Classic ZenML Pipelines

Engineers familiar with ZenML often ask: “Why did ZenML build Kitaru instead of just using @pipeline and @step?”

The answer lies in the structural mismatch between batch training pipelines and autonomous agent loops:

1. Plain Python Control Flow vs. Static DAGs: Classic ZenML pipelines compile an execution graph before running. Kitaru lets you write ordinary Python (if/else, while, try/except, dynamic recursion) without declaring a rigid graph DSL upfront.
2. Checkpoint Caching vs. Batch Stage Artifacts: In Kitaru, @checkpoint acts as a lightweight persistence and serialisation boundary. When a flow resumes after a crash, previously completed checkpoints are resolved directly from the artifact store without re-invoking the underlying LLM or external API.
3. Zero-Compute Suspension: Traditional ML pipelines run start-to-finish in batch mode. Kitaru natively supports suspending execution mid-flow (kitaru.wait()), releasing compute resources while awaiting human approval or external events.
4. Shared Core Engine: Under the hood, Kitaru leverages ZenML’s battle-tested metadata tracking and versioned artifact engine, giving you full lineage, execution history, and reproducibility without forcing you into a training-pipeline mental model.

2. Step-by-Step Local Setup

Let’s set up a clean local Kitaru environment using uv for fast, reproducible Python dependency management.

Step 1: Create a Virtual Environment and Install Kitaru

uv venv .venv
source .venv/bin/activate
uv pip install "kitaru[local]"

The [local] extra installs the local Kitaru server and dashboard components so you can inspect executions, checkpoint artifacts, and traces on your workstation before deploying to cloud infrastructure.

Step 2: Initialise Your Repository and Connect to the Local Server

Inside your agent repository, run:

# Initialise Kitaru project configuration in the current repo
kitaru init

# Start and connect to the local Kitaru server
kitaru login --local

# Verify server health and active connection
kitaru status

Once connected, every @flow execution is automatically tracked, versioned, and persisted in Kitaru’s metadata and artifact store.

3. Building Your First Durable Agent (@flow and @checkpoint)

Kitaru’s core execution model relies on two decorators:

• @flow: Marks the top-level Python function as a durable workflow boundary. Kitaru tracks its lifecycle, inputs, outputs, and state transitions.
• @checkpoint: Marks an expensive or non-deterministic sub-function (such as an LLM completion, embedding lookup, or external API tool call). Kitaru hashes the inputs, serialises the return value as a versioned artifact, and caches it for crash recovery.

Here is a step-by-step multi-stage research agent implemented with plain Python control flow:

from typing import Dict, List
from kitaru import flow, checkpoint


@checkpoint
def retrieve_enterprise_context(topic: str) -> List[str]:
    """
    Boundary 1: Expensive vector + keyword retrieval across internal docs.
    """
    print(f"[Checkpoint] Querying knowledge base for: {topic}")
    return [
        f"Policy A: All {topic} changes require RBAC audit.",
        f"Policy B: Production deployments for {topic} use OIDC.",
    ]


@checkpoint
def synthesise_technical_brief(topic: str, docs: List[str]) -> Dict[str, str]:
    """
    Boundary 2: Frontier LLM reasoning call.
    Cached automatically once completed so downstream crashes never re-bill tokens.
    """
    print(f"[Checkpoint] Calling LLM to synthesise brief for {topic}...")
    joined_context = "\n".join(docs)
    return {
        "topic": topic,
        "summary": f"Executive brief for {topic} grounded in {len(docs)} policies.",
        "evidence": joined_context,
    }


@checkpoint
def publish_to_internal_wiki(brief: Dict[str, str]) -> str:
    """
    Boundary 3: Side-effecting external API call.
    """
    print(f"[Checkpoint] Publishing brief for {brief['topic']}...")
    return f"wiki://architecture/{brief['topic'].lower().replace(' ', '-')}"


@flow
def architecture_brief_agent(topic: str, publish: bool = True) -> Dict[str, str]:
    """
    Standard Python control flow (if/else, loops) wrapped in @flow.
    """
    docs = retrieve_enterprise_context(topic)
    brief = synthesise_technical_brief(topic, docs)

    if publish and len(docs) > 0:
        brief["wiki_url"] = publish_to_internal_wiki(brief)

    return brief


if __name__ == "__main__":
    execution = architecture_brief_agent.run("Zero-Trust MCP Gateway")
    result = execution.wait()
    print(f"Completed execution: {result}")

What Happens Under the Hood

1. Hashing and Serialisation: When retrieve_enterprise_context returns, Kitaru serialises the list of strings as a versioned artifact linked to the flow execution.
2. Instant Crash Recovery: Suppose publish_to_internal_wiki raises a TimeoutError because the Confluence API is momentarily down. Neither retrieve_enterprise_context nor synthesise_technical_brief needs to run again. When the execution resumes, Kitaru serves their outputs directly from the artifact store and jumps straight to the failed checkpoint.
3. Asynchronous Execution Handle: Calling architecture_brief_agent.run(...) returns an execution handle immediately, and .wait() blocks until the durable flow finishes.

Next in This Series

In Part 2, we will move beyond simple crash recovery to examine zero-compute human-in-the-loop pauses with kitaru.wait(), checkpoint boundary design rules, and integrating Kitaru with PydanticAI, LangGraph, and the OpenAI Agents SDK.

Why Alps Agility?

At Alps Agility, we combine deep AI engineering expertise with production MLOps and cloud architecture to help enterprises build durable, observable, and cost-effective agentic systems.

Contact us today to architect your production AI agent runtime.

Struggling to move AI from prototype to production? We help enterprises build robust, scalable AI architectures. Book a Generative AI Readiness Assessment.