Andrew Ng maps the six skills AI engineers need to turn probabilistic models into reliable systems

Andrew Ng has laid out a six-part skill map for AI engineers, and he puts “building and deploying AI applications” at the center of the job. Ng, founder of DeepLearning.AI and a Stanford professor, said the role now breaks into six areas: LLM foundations, grounding models with data, agentic systems, evaluation-driven development, operating in production, and machine learning foundations. His core point is simple. AI software is not like traditional software because its output is uncertain. A developer can usually predict what conventional code will do under a given input. With large language models and machine learning systems, that kind of certainty does not exist. That is why Ng says AI engineering is about combining unreliable, probabilistic components into a reliable software system. He also argues that AI development is much more iterative than traditional software work. Engineers build a small part of the system, inspect the result, analyze the errors, and then decide whether to change the prompt, the data, the model, the tools, the agent architecture, or the evaluation method. In his view, the key skill is not finishing a full plan up front. It is deciding what experiment matters most after seeing the intermediate result. That is also why evaluation sits so high on his list. On LLM foundations, Ng says engineers need to understand how a model tokenizes input, generates output step by step, and where it tends to succeed or fail. He ties that to practical decisions such as how much information belongs in the context window, when multimodal models make more sense than text-only ones, how cache hits affect cost, what a model’s knowledge cutoff implies, how reasoning effort and sampling parameters change behavior, and when to use tool calling. Once engineers understand those basics, they can decide which model to use, whether to combine multiple models, and when fine-tuning or self-hosting is the better choice. For grounding models with data, Ng says RAG is no longer the whole story. Vector search was an early approach, but the list of options has widened. Engineers now need to decide what belongs in the prompt, what should be fetched in real time through tools, and how each data type should be represented. Document search may fit a vector index, complex entity relationships may call for a knowledge graph, and structured customer or order data may need a semantic layer. They also need to turn PDFs, HTML, images, and text files into input that a model can actually use, while keeping the pipeline clean, accurate, and up to date. His third area, agentic systems, covers everything from a fixed workflow that calls several LLMs in sequence to a more advanced agent harness that lets the model inspect state, decide on the next move, and act. The hard part, he says, is architecture. Engineers have to choose which steps should run in series or in parallel, what should stay in traditional code, what should be handed to the model, which tools the agent can call, how memory should work over long tasks, and whether multi-agent orchestration is really necessary. Ng also stresses that agents are much harder to ship than they are to demo. A prototype that works once is not the same as a system that can safely serve hundreds of thousands of users. Guardrails, adversarial input, data leakage, and governance all matter. A company agent that can read internal data and call external tools can turn a simple answer error into a real security incident if it falls victim to prompt injection. The fourth area, evaluation-driven development, is the one Ng says most clearly separates strong AI engineers from the rest. He describes a disciplined loop of “Eval → Error Analysis → Development.” Because models are stochastic, changing a prompt based on instinct alone can easily turn into random trial and error. Good evals tell teams how much the product improved and which class of problems should be tackled next. Ng adds that building the right evaluation system is itself a deep technical task. Engineers may need to inspect traces, review large volumes of model outputs, run exploratory data analysis, and combine that work with product and business knowledge to decide what should actually be measured. Some tasks can be checked deterministically. Others may need an LLM-as-a-judge. High-risk settings may still require human-in-the-loop review. Even the eval system itself has to be evaluated and updated as the product evolves. His fifth area is operating in production. Once AI software is live, engineers have to watch more than uptime. They need to track model drift, failure patterns for specific inputs, and security issues such as prompt injection. CI/CD and regression testing also need to change. Traditional software can often be checked against a single correct output, while AI systems usually need statistical evaluation, with the bar set by risk. A restaurant recommender that is wrong occasionally is not the same as a medical system that is wrong occasionally. Cost and latency are part of the product architecture too. As usage grows, the per-call inference cost that looked minor in a demo can quickly affect gross margin. Ng says engineers need to know how to mix models, optimize model choice, use distillation, fine-tuning, and simpler agent workflows. That is one of the main lines between AI engineering and prompt engineering. The former still carries responsibility for reliability, latency, and unit economics. His final area is machine learning foundations. After ChatGPT, some people argued that AI application builders no longer needed classic ML. Ng disagrees. He says the engineers who are truly good at building LLM systems almost always understand machine learning and deep learning at some level. That matters because LLMs are trained with supervised learning and reinforcement learning, and many real-world applications still rely on traditional ML models or even custom training. Engineers still need to understand trade-offs in accuracy, training speed, and inference speed. Bias/variance, error analysis, and data engineering remain essential frameworks for working with systems that produce uncertain outputs.