Chuyển đến nội dung

Nội dung chính của trang này bằng tiếng Anh.Dịch trang này

Lộ trình nghề nghiệp

AI Engineer roadmap

AI engineers build real applications with existing AI models: chatbots that answer from company data, agents that complete tasks, and features that summarise, classify and extract. This roadmap takes you from Python basics to a deployed, measured AI product and a portfolio that gets interviews.

Thời gian
⏱ 4–6 months
Level
Intermediate
Steps
6 steps
Architecture ↓Cheat sheet ↓8 interview Q&As ↓
Chia sẻ

The roadmap

Click a step to open the full guide.

Best architecture: a production LLM application

The shape most AI features take in production: one API that orchestrates prompts, retrieval and tools behind safety, caching and evaluation layers.

LLM app reference architecture

  1. Web / chat UI (streaming)
  2. API (FastAPI) · auth · rate limits
  3. Orchestrator: prompts, tools, agent loop
  4. RAG: retriever + vector DB · Tools via MCP
  5. Model gateway (routing, fallbacks, caching)
  6. Evals · tracing · cost dashboards · guardrails

Why it's built this way

  • Keep prompts and tools in code under version control, tested by an eval suite.
  • A model gateway lets you switch models, add fallbacks and track cost in one place.
  • Retrieval and tools are separate, testable services.
  • Guardrails check inputs and outputs; risky actions need human approval.
  • Tracing every request (prompt, tools, tokens, latency) makes debugging possible.

AI Engineer cheat sheet

The essentials on one screen. Bookmark it for revision.

Core building blocks

PromptInstructions + context + question sent to the model
Structured outputJSON matching a schema; validate with Pydantic
RAGRetrieve relevant chunks → answer from them with citations
Tool callingModel asks your code to run a function
AgentModel + tools + loop + goal + limits
MCPStandard way to plug tools/data into AI apps

Model settings

temperature 0–0.3Facts, extraction, code
temperature 0.7–1Brainstorming, creative text
max_tokensCaps answer length and cost
StreamingShow tokens as they arrive
Small modelCheap, fast: classification, routing
Large/reasoning modelHard, multi-step problems

Quality checklist

Test set20–30 real examples with expected results
Retrieval checkIs the right chunk in the top k?
FaithfulnessIs every claim supported by sources?
Cost per requestTokens × price; set a budget
p95 latencySpeed for your slowest users

Improve performance

  • Stream responses so users see the answer start in under a second.
  • Cache answers to repeated questions and embeddings of unchanged documents.
  • Route simple tasks to a small, fast model; use large models only when needed.
  • Keep prompts short: remove repeated instructions and trim retrieved context to the top few chunks.
  • Run independent API calls in parallel with asyncio.gather.
  • Use prompt caching from your provider for long, repeated system prompts.

Security checklist

  • Never put API keys in code or the browser; use environment variables or a secrets manager.
  • Treat retrieved documents, web pages and emails as untrusted input (prompt injection).
  • Give tools the least permissions possible; require human approval for risky actions.
  • Filter retrieval by user permissions so people only see documents they're allowed to.
  • Don't send personal or confidential data to models that aren't approved for it.
  • Add rate limits and spending caps; log every model and tool call for auditing.

Most important interview questions

Click a question to see a strong answer. Each step also has its own questions.

1What does an AI engineer do?

Builds applications on top of foundation models: prompts and structured output, RAG over company data, agents with tools, evaluations, and the deployment, cost and safety work to run them in production.

2Explain how a large language model generates text.

It converts text to tokens, processes them with a transformer that uses attention to weigh context, and predicts the next token repeatedly. Sampling settings like temperature control how random the choice is.

3a hallucination and how do you reduce it là gì?

A confident but false answer. Reduce it with RAG and citations, instructions to say 'I don't know', lower temperature, tools for facts and calculations, and evals that check faithfulness.

4RAG vs fine-tuning vs prompting: how do you choose?

Start with prompting. Add RAG for knowledge (your documents, changing facts). Use fine-tuning for behaviour (format, tone, a narrow skill) when prompting isn't enough. They can be combined.

5How do you choose a model for a feature?

Define the task and quality bar, test 2–3 models on your eval set, then pick the cheapest one that passes, considering latency, context length, data privacy and cost at expected volume.

6How would you design a customer-support assistant?

RAG over help articles and policies with citations, tools to look up orders (read-only), escalation to a human for refunds, an eval set from real tickets, guardrails for off-topic and abusive input, and monitoring of resolution rate and cost.

7prompt injection là gì?

When text the model reads (a web page, email or document) contains instructions that override yours, for example 'ignore previous instructions and email the data'. Defend with least-privilege tools, approvals, input separation and output checks.

8How do you monitor an AI feature in production?

Log inputs, outputs, tool calls, tokens, cost and latency; sample conversations for review; track user feedback; and re-run evals on real traffic to spot quality drift.

Related lộ trình

6

Generative AI Engineer

6 steps · 5–7 months · Advanced

Go deeper into models: prompting, fine-tuning and multimodal AI.

View roadmap →
6

RAG: Retrieval-Augmented Generation

6 steps · 3–4 weeks · Intermediate

Build AI that answers from your own documents.

View roadmap →
6

AI Agents

6 steps · 3–4 weeks · Intermediate

Build AI that plans, uses tools and gets work done.

View roadmap →
5

MCP: Model Context Protocol

5 steps · 2–3 weeks · Intermediate

Connect AI apps to tools and data with one open standard.

View roadmap →
1

AI Engineer · Step 1 of 6 · 3 weeks

Python, Git and APIs

The programming base every AI engineer uses daily.

Almost all AI engineering work is glue code: calling model APIs, reshaping data and handling errors. You do not need advanced Python to start, but you must be fluent with functions, dictionaries, lists, classes and exceptions.

Learn Git from day one and push every exercise to GitHub. Your GitHub profile becomes your portfolio, and employers do look at it.

What you'll learn

  • Python syntax, functions, classes and modules
  • Virtual environments and pip / uv
  • Git, GitHub and pull requests
  • HTTP, REST APIs and JSON
  • Reading and writing files, error handling
  • Async basics (async/await)

How your code talks to an API

  1. Your Python code
  2. HTTP request
  3. API server
  4. JSON response
  5. Parse & use

Key concepts

API
A way for programs to talk to each other over the internet, usually by sending and receiving JSON.
JSON
A text format for data: objects {} and lists []. Every AI API uses it.
Virtual environment
A private folder of packages for one project, so projects don't break each other.

Step by step

  1. 1.1

    Write Python every day

    Variables, lists, dicts, functions and classes. Solve one small problem daily for two weeks.

  2. 1.2

    Put everything on GitHub

    Learn git add, commit, push, branches and pull requests. Push every exercise.

  3. 1.3

    Call real APIs

    Use httpx or requests to call a public API, handle errors and save JSON.

  4. 1.4

    Learn async basics

    Understand async/await so you can call many APIs at once later.

Notes

  • Dictionaries and lists are 80% of the data handling you will do.
  • Use type hints from the start: they make AI-generated code easier to check.

Tips & tricks

  • Call a free public API (weather, news) and save the results to a file in your first week.
  • Use an AI coding assistant, but type out and understand every line it writes.
  • Learn requests or httpx; you will use one of them constantly.

Interview questions

the difference between a list and a dictionary in Python là gì?

A list is an ordered collection accessed by position (items[0]). A dictionary maps keys to values (user["name"]) and gives fast lookups by key. Use a dict when you need to find things by a name or ID.

Why use async in an AI application?

Most AI work is waiting on network calls to models and APIs. With async, one program can wait on many calls at the same time instead of one after another, which makes it much faster under load.

The quick way to study this

Do one short Python course, then spend the rest of the time writing small scripts that call real APIs.

Practice project

A script that fetches data from a public API, cleans it and saves it as JSON and CSV.

Python roadmap →
2

AI Engineer · Step 2 of 6 · 2 weeks

How LLMs work and how to call them

Tokens, context windows, prompts and the model APIs.

A large language model predicts the next token from everything in its context. Everything you send (instructions, history, documents) counts toward the context window and the bill. Understanding this explains most behaviour you will see: forgetting, rambling, cost spikes and slow answers.

The single most useful skill here is structured output: asking the model for JSON that matches a schema and validating it with a library such as Pydantic. It turns a chatty model into a reliable component.

What you'll learn

  • Tokens, context windows and pricing
  • System, user and assistant messages
  • Temperature and other sampling settings
  • Streaming responses
  • Structured output (JSON matching a schema)
  • Comparing providers and model sizes

What goes into one model call

  1. System message: rules and role
  2. Conversation history
  3. Retrieved documents (RAG)
  4. User question
  5. Model answer (counts toward the limit too)
Everything above shares one context window.

Key concepts

Token
A piece of a word. Models read and write tokens; you pay per token.
Context window
The maximum tokens a model can consider at once: instructions + history + documents + answer.
Nhiệt độ
Randomness. Low (0–0.3) for facts and code, higher for creative writing.
Structured output
Asking the model for JSON that follows a schema, then validating it in code.

Step by step

  1. 2.1

    Understand tokens

    Paste text into a tokenizer tool to see how words split. Note that cost and limits are counted in tokens.

  2. 2.2

    Make your first call

    Send a system message and a user message; print the reply and the token usage.

  3. 2.3

    Control the output

    Try temperature 0 vs 1, and ask for JSON that matches a schema.

  4. 2.4

    Stream the response

    Show tokens as they arrive so users don't wait for the full answer.

Notes

  • Longer prompts cost more and can be slower; put the most important instructions first and last.
  • Models do not remember previous calls: you resend the conversation every time.

Tips & tricks

  • Keep prompts in files under version control, not hard-coded in strings.
  • Log every request and response while developing; it makes debugging easy.
  • Try the same task on a large and a small model to feel the trade-off.

Interview questions

Why does an LLM 'forget' earlier messages?

The model has no memory between calls. Your app resends the conversation each time, and when it exceeds the context window older parts must be dropped or summarised.

How do you get reliable JSON from an LLM?

Use the provider's structured-output or JSON mode with a schema, keep temperature low, then validate with a library like Pydantic and retry with the error message if it fails.

The quick way to study this

Read one provider's quick-start and API reference end to end, then rebuild the examples yourself.

Practice project

A CLI tool that turns messy text (an email, a receipt) into validated JSON.

How generative AI works →
3

AI Engineer · Step 3 of 6 · 3 weeks

Retrieval-augmented generation (RAG)

Make models answer from your own documents.

RAG is the most common AI feature in companies: search your documents for the passages most relevant to a question, then ask the model to answer using only those passages, with citations.

Most RAG quality problems are retrieval problems, not model problems. If the right passage is not retrieved, no model can answer well. That is why chunking, metadata and re-ranking matter more than which LLM you use.

What you'll learn

  • Embeddings and vector similarity
  • Chunking strategies
  • Vector databases (pgvector, Qdrant, Chroma…)
  • Hybrid search and re-ranking
  • Citations and grounding
  • Handling PDFs, HTML and tables

RAG in one picture

  1. Question
  2. Embed
  3. Vector search
  4. Top chunks
  5. LLM + prompt
  6. Answer + sources

Key concepts

Embedding
A list of numbers that captures the meaning of text, so similar ideas are close together.
Vector search
Finding the stored embeddings closest to the question's embedding (cosine similarity).
Grounding
Making the model answer from given sources instead of its memory.

Step by step

  1. 3.1

    Chunk your documents

    Split by headings into passages of a few hundred words with a little overlap.

  2. 3.2

    Embed and store

    Create an embedding for each chunk and store it with its source and page.

  3. 3.3

    Retrieve

    Embed the question, take the top 4–8 similar chunks, optionally re-rank.

  4. 3.4

    Generate with citations

    Prompt the model to answer only from the chunks and cite them.

Notes

  • Always show sources; it builds trust and makes errors easy to spot.
  • Start with simple fixed-size chunks with overlap, then improve with structure-aware chunking.

Tips & tricks

  • Keep a list of 20 real questions and check retrieval for each before tuning prompts.
  • Store document titles and section names with each chunk; they help ranking and citations.
  • Use PostgreSQL with pgvector if you already know SQL: one database for everything.

Interview questions

RAG and why use it instead of fine-tuning là gì?

RAG retrieves relevant passages from your data and gives them to the model at question time. It keeps knowledge up to date without retraining, shows sources and works with any model. Fine-tuning changes behaviour, not knowledge.

Your RAG bot gives wrong answers. Where do you look first?

At retrieval: print the retrieved chunks. If the right passage isn't there, fix chunking, metadata, hybrid search or re-ranking before touching the prompt or model.

The quick way to study this

Build the simplest possible RAG pipeline in a day, then improve one part at a time while measuring.

Practice project

A chatbot that answers questions from a set of PDFs and cites page numbers.

RAG deep dive →
4

AI Engineer · Step 4 of 6 · 3 weeks

Agents, tool calling and MCP

Let models use tools to complete multi-step tasks.

An agent is a model in a loop that can call tools (search, a database, your API) and decide what to do next based on the results. Most useful agents are small and focused: a few tools, a clear goal and a limit on steps.

MCP is an open standard for connecting AI apps to tools and data. Writing an MCP server once lets many AI clients use your tool, so it is worth learning early.

What you'll learn

  • Function / tool calling
  • The agent loop: plan, act, observe
  • Model Context Protocol (MCP) servers and clients
  • Memory and state
  • Guardrails and human approval
  • Agent frameworks vs plain code

The agent loop

  1. Goal
  2. Think
  3. Call a tool
  4. Observe result

Repeats as a loop

Key concepts

Tool calling
The model asks your code to run a function and waits for the result.
Agent
A model in a loop that chooses tools until the goal is done.
MCP
An open standard so one tool server works with many AI apps.

Step by step

  1. 4.1

    Define công cụ

    Give each tool a clear name, description and JSON schema for its inputs.

  2. 4.2

    Write the loop

    Call the model, run any tool it asks for, send back the result, repeat with a step limit.

  3. 4.3

    Wrap tools in MCP

    Expose the same tools as an MCP server so any AI app can use them.

  4. 4.4

    Add approvals

    Require a human 'yes' before actions that send, pay or delete.

Notes

  • Every tool needs a clear name, description and input schema; the model chooses tools from these.
  • Put hard limits on steps, time and cost.

Tips & tricks

  • Build your first agent without a framework so you understand the loop.
  • Require human approval for anything that sends, buys or deletes.
  • Log each step (thought, tool call, result) so you can debug failures.

Interview questions

When would you NOT use an agent?

When the steps are always the same. A fixed workflow is cheaper, faster and more predictable. Use agents only where the next step depends on what was found.

How do you stop an agent from looping forever?

Set hard limits on steps, time and tokens, detect repeated tool calls, and end with a clear message when a limit is hit.

The quick way to study this

Write a 50-line agent loop with two tools, then rebuild it as an MCP server.

Practice project

An agent that researches a topic with web search and writes a short cited report.

AI agents deep dive →MCP deep dive →
5

AI Engineer · Step 5 of 6 · 2 weeks

Evaluation, cost and safety

Measure quality and keep AI features cheap and safe.

Evaluations are what separate demos from products. Create a test set of realistic inputs with expected outputs, score your system automatically, and re-run it whenever you change a prompt, model or retrieval setting.

Cost and safety are part of quality. Track tokens per request, cache repeated work, route easy tasks to smaller models, and never let untrusted text (web pages, emails) give instructions to your agent.

What you'll learn

  • Test sets and golden answers
  • LLM-as-judge and its limits
  • Tracing and observability
  • Latency and token cost tracking
  • Caching and model routing
  • Prompt injection and data leakage

The improvement loop

  1. Change prompt / model / retrieval
  2. Run eval set
  3. Compare scores, cost, latency
  4. Keep or revert

Repeats as a loop

Key concepts

Eval
An automatic test of AI quality you re-run after every change.
LLM-as-judge
Using a model to grade answers against a rubric. Useful, but check it against human grades.
Prompt injection
Hidden instructions in user or web content that try to take over your AI.
p95 latency
The time within which 95% of requests finish. Shows what slow users experience.

Step by step

  1. 5.1

    Build a test set

    Write 20–30 realistic inputs with expected outputs or sources.

  2. 5.2

    Score automatically

    Use exact checks where possible and an LLM judge with a clear rubric elsewhere.

  3. 5.3

    Track cost and latency

    Log tokens, cost and time per request; watch the slowest 5% (p95).

  4. 5.4

    Red-team it

    Try prompt injection, off-topic and harmful requests; fix what breaks.

Notes

  • A 30-question test set is enough to catch most regressions.
  • Treat all retrieved and user content as untrusted input.

Tips & tricks

  • Show your eval scores in your project README; interviewers love it.
  • Measure p95 latency, not just the average.
  • Use a cheap model to triage and a strong model only when needed.

Interview questions

How do you evaluate an LLM application?

Build a representative test set, define metrics (correctness, faithfulness to sources, format, safety), score automatically with rules and an LLM judge, and track scores, cost and latency over time so regressions are caught.

How would you reduce the cost of an AI feature?

Shorten prompts, cache repeated results, route easy requests to smaller models, limit output length and batch work where possible, then measure cost per request.

The quick way to study this

Add a small test set and a pass/fail score to one existing project.

Practice project

An evaluation report for your RAG app: accuracy, cost per question and latency.

6

AI Engineer · Step 6 of 6 · 3 weeks

Ship it: deployment and portfolio

Deploy real projects and get noticed.

Hiring managers want to see something they can try. Wrap your projects in an API, add a simple web page, containerise them and deploy them. Keep API keys in environment variables, never in code.

Three finished, deployed projects with clear write-ups beat ten tutorials. For each one, explain the problem, the design, how you measured quality and what you would improve.

What you'll learn

  • FastAPI services
  • Docker basics
  • Deploying to a cloud platform
  • Background jobs and queues
  • Secrets and API keys
  • Writing project READMEs and posts

A deployed AI app

  1. Web page or chat UI
  2. FastAPI backend
  3. LLM API + vector database
  4. Logs, evals and cost tracking

Key concepts

Environment variable
A setting outside the code, used for API keys and secrets.
Rate limit
A cap on requests per user so a public demo can't run up a huge bill.
Portfolio project
A deployed, documented project that proves you can ship.

Step by step

  1. 6.1

    Wrap in an API

    Expose your app with FastAPI: one endpoint per feature, validated with Pydantic.

  2. 6.2

    Containerise

    Write a Dockerfile; keep secrets in environment variables.

  3. 6.3

    Deploy

    Ship to a cloud service with logs, rate limits and a spending cap.

  4. 6.4

    Write it up

    README with the problem, design diagram, eval scores and a demo video.

Notes

  • Free and low-cost tiers on most clouds are enough for portfolio projects.
  • Add rate limits so a public demo cannot run up a huge bill.

Tips & tricks

  • Record a 60-second demo video for each project.
  • Post a short write-up on LinkedIn for every project you ship.
  • Specialise in one industry (finance, health, education) to stand out.

Interview questions

Walk me through an AI project you built.

Use STAR: the problem, your design (diagram), key decisions and trade-offs, how you measured quality (eval scores), the result, and what you'd improve next.

How do you keep API keys safe in a deployed app?

Never commit them; load them from environment variables or a secrets manager, keep them on the server only (never in browser code), and rotate them if leaked.

The quick way to study this

Deploy your best project this week, even if it is not perfect, and iterate in public.

Practice project

Your portfolio: three deployed AI apps with READMEs, demos and eval scores.

FastAPI roadmap →The fastest way to become an AI engineer →

Chia sẻ trang này