AI engineering resources
Background
The most frustrating part about learning AI engineering was the amount of time I spent going down the wrong path or absorbing resources that ultimately weren’t useful.
I created this list to hopefully make your journey shorter and more straightforward. Luckily, once you have the resources, I think backend skills are pretty transferrable; for me, it was easier to learn AI engineering than infra.
Note: most of these resources explain how to do X. You still need to practice doing X, either for work or in your own projects.
What this list is and isn’t
This is an AI engineering list. Roughly, AI engineering is the set of relatively mature engineering practices using state-of-the-art models. It’s different from AI research, which explores more technical unknowns and is usually driven by the leading labs.
Fundamentals
AI Engineering
- AI Engineering book by Chip Huyen
- Good for skimming to get a general overview (eg what you should spend time on)
- As of 2025 Sept, is mostly up-to-date
- Good for quick checks
- Eg when to use PEFT instead of full fine-tuning
- Probably need to supplement with other resources if you want to dig deeper on any particular topic
- Good for skimming to get a general overview (eg what you should spend time on)
Prompt engineering
- Anthropic prompt eng tutorial
- The only resource I needed. Recommend this version (Jupyter notebooks) over the Google sheets version, which I couldn’t get to work.
Evals
Basics
- Hamel's evals blog
- Good for building intuition on how to start with evals
Potential tips
- https://github.com/huggingface/evaluation-guidebook
- https://huggingface.co/learn/cookbook/en/llm_judge
Tracing recs
Tracing helps you quickly check the quality of your system during development and in production. I find it useful to play around with implementing tracing from scratch so you can decide what you like and don't like when choosing between dedicated solutions.
For any non-trivial project, my rough requirements:
- Able to run locally for development
- Only tracing; not involved in the critical path of the application
Very nice-to-haves:
- Open-source
- Cost model seems “reasonable”
Langfuse seems to meet these requirements. I haven’t looked in detail at other tracing solutions, like Arize Phoenix.
Agents
Basics
MCP
Basics
Some issues I ran into when using MCP servers in practice
- https://www.reddit.com/r/mcp/comments/1k4tixp/mcp_protocol_needs_output_schemas/
- https://github.com/github/github-mcp-server/issues/142
Real codebases
In general I’m a fan of working in production-grade codebases as much as possible. This is especially true in AI, where I’m scared of picking up bad habits that wouldn’t fly for production applications.
I like the Cline codebase; I’ve only skimmed it when I needed to reference stuff, but it seems well-written.
Good tools to explore open-source codebases more quickly:
- Cline deepwiki
- Or download the repo locally and ask your favorite AI assistant
Optional
Neural nets → LLM fundamentals
Technically with the current state of AI engineering you don't need to know what's under the hood of an LLM. Also note that these resources reference models that are outdated, since current SOTA LLMs are proprietary; if you’re curious, I’d join one of the good labs :-).
Basics
- 3Blue1Brown Neural Nets playlist
- First resource that helped me understand transformers/attention
- Con: walks through intuition on what the network might be doing that isn’t actually what the network is doing
- Karpathy LLM Deep Dive video
- Probably one of my favorite resources
- Deep dives into a bunch of interesting discourse around LLMs if you treat it like a black box
- Karpathy resources are also useful for random recommendations, such as how he gets his AI news and his preferred cloud for renting GPUs
- Karpathy Intro to LLMs video
- GPT architecture
- Best and most concise resource on the GPT architecture
Other pointers to resources
Good
Haven’t tried