AI in production needs operations, just like any other software. But “operations” means different things depending on what you’re running. MLOps, LLMOps and AgentOps are three layers of the same idea, each adding controls for a more powerful kind of AI.
The short version
| MLOps | LLMOps | AgentOps | |
|---|---|---|---|
| You run | Models you train | Apps built on LLMs | AI agents that act |
| Main unit | A model version | A prompt + model + retrieval setup | A multi-step task run |
| Evaluate with | Accuracy on a test set | Golden sets, LLM judges, human review | Task success, safety, steps, cost |
| Main risks | Data drift, stale models | Hallucinations, cost, prompt injection | Harmful actions, runaway loops, spend |
| Инструментов: Key | Feature store, registry, pipelines | Prompt registry, evals, AI gateway, tracing | Step tracing, policy engine, approvals, budgets |
MLOps: running models you train
MLOps applies DevOps practices to machine learning. It matters when you train your own models, such as fraud scoring, demand forecasting or recommendations.
The lifecycle:
Data (versioned, validated) → Train (tracked) → Evaluate & register
↑ ↓
Retrain ← Monitor (drift, accuracy) ← Deploy (batch / API, canary)What makes it work:
- Versioned data and features, so any model can be reproduced.
- Experiment tracking and a model registry, so you always know which model is live and why.
- Automated pipelines (CI/CD) that retrain and only promote a model that beats the current one.
- Monitoring for drift: the world changes and models quietly get worse.
Deep dive: our MLOps roadmap.
LLMOps: running LLM applications
Most teams today don’t train models; they build on large language models through APIs. LLMOps is the operations layer for that.
The lifecycle:
Design → Build (prompts, RAG, tools) → Evaluate (golden set)
↑ ↓
Improve ← Observe (traces, cost, feedback) ← Deploy (gateway, canary)What’s different from MLOps:
- Prompts are code. Version them, review them, and log which version produced each answer.
- Evaluation is harder. Outputs are open-ended text, so you use golden test sets, rubric-based LLM judges and human spot checks, run on every change.
- Models change under you. Providers update models, so evals must catch behaviour changes.
- Cost and latency are daily concerns. Every token costs money; an AI gateway adds routing to cheaper models, fallbacks, caching and quotas.
- Safety needs guardrails on inputs and outputs, especially against prompt injection.
Deep dive: our LLMOps roadmap and RAG roadmap.
AgentOps: running AI agents
An AI agent doesn’t just answer; it plans, calls tools and acts, often over many steps. That changes the risks: a bad answer becomes a bad action, and a loop can quietly burn through your budget. AgentOps adds the controls agents need.
The lifecycle:
Design (goal, tools, risk level) → Build (agent loop, MCP tools) → Evaluate (task suite)
↑ ↓
Improve & govern ← Observe (step traces, cost) ← Deploy (policies, approvals, budgets)What AgentOps adds:
- Step-level tracing: every thought, tool call and result, so you can replay a run and find the first wrong step.
- Task-based evaluation: did the agent finish realistic tasks, safely, in a reasonable number of steps and cost?
- Permissions and approvals: a policy layer outside the model checks every tool call; risky actions (send, pay, delete) wait for a human.
- Budgets: hard limits on steps, tokens, money and time, enforced in code.
- A kill switch and incident playbook before launch.
Deep dive: our AgentOps roadmap, plus how to build an AI agent and how to build an MCP server.
Which one do you need?
- Training your own models? → MLOps.
- Building features on LLM APIs (chatbots, RAG, summarisation)? → LLMOps.
- Letting AI take actions with tools? → LLMOps plus AgentOps.
Many companies need all three: a fraud model (MLOps), a support assistant (LLMOps) and an operations agent (AgentOps), often sharing the same monitoring and governance.
Five practices that apply to all three
- Version everything that changes behaviour: data, code, prompts, models, tools.
- Evaluate before every release with a fixed test set, and add real failures to it.
- Observe in production: quality, latency, errors and cost, in one place.
- Roll out gradually (shadow, canary) and keep rollback one click away.
- Govern access: least privilege, audit logs and clear ownership.
Learn them step by step
Each roadmap has a step-by-step path with lifecycle diagrams, key concepts, interview questions and a cheat sheet:
Частые вопросы
Что такое the difference between MLOps, LLMOps and AgentOps?
MLOps runs machine learning models you train yourself. LLMOps runs applications built on large language models, focusing on prompts, evaluation, cost and safety. AgentOps runs AI agents that take many steps and use tools, adding step-level tracing, task-based evaluation, permissions, approvals and budgets.
Do I need MLOps if I only use LLM APIs?
Usually not full MLOps. If you don't train models, LLMOps practices (prompt versioning, evals, gateways, tracing and cost control) are what you need. Add AgentOps if your app uses agents that take actions.
Is AgentOps part of LLMOps?
It builds on LLMOps. Everything in LLMOps still applies, plus extra controls because agents take multiple steps and act in the real world.
