AI Engineering Demystified: From Vibe Coding to Systems Design

“AI Engineering Demystified for a Backend Dev”

“AI Engineering Demystified for a Backend Dev” is a video script by ICraftCode targeting backend engineers transitioning into AI. The script distinguishes using AI tools like ChatGPT from engineering with them, emphasizing that the former is an end-user skill, not a developer discipline.

Why This Matters

The video argues that while many organizations adopt AI through vibe-coding—describing requirements and letting tools generate code—this approach treats developers as advanced users rather than engineers. The core challenge is building reliable systems around an inherently unreliable component: an LLM can give different answers to the same question twice and state falsehoods with complete confidence, with no stack trace or error to debug. This shifts debugging from logic to behavior, requiring new engineering practices around prompting, memory injection (RAG), tool use, and measurement.

Key Insights

• LLMs are ‘very good next-piece-of-text guessers’ that read massive text corpora (most of the public internet, books, code) and guess plausible continuations—not understand or think (ICraftCode, 2026).
• Tokenization splits text into chunks (e.g., ‘unbelievable’ → ‘un’, ‘believ’, ‘able’), causing poor performance on tasks like counting letters in words or precise math on long numbers; APIs bill by token count (ICraftCode, 2026).
• Context windows impose hard caps on input size; models pay less attention to content buried in the middle of large inputs than at beginning or end—‘just paste the entire handbook in’ is a bug waiting to happen (ICraftCode, 2026).
• Temperature near zero yields deterministic output for precise tasks (e.g., extracting structured data); higher values (0.7–1) produce more creative but less reliable output, suitable for writing taglines but not customer order numbers (ICraftCode, 2026).
• Embeddings convert text into vectors whose proximity in number-space indicates semantic closeness; cosine similarity enables search by meaning over exact keywords—foundation for RAG (ICraftCode, 2026).

Practical Applications

• Use case: Building multi-step agents by chaining multiple LLM calls with guardrails so errors don’t compound across steps. Pitfall: Assuming each call produces correct output; since tokens cannot be undone once written, an early mistake propagates through all subsequent reasoning.
• Use case: Implementing RAG by feeding company-specific data into the context window at query time to ground answers in real information. Pitfall: Overloading the context window with irrelevant data causes the model to ignore critical details buried mid-input.
• Use case: Using low temperature settings for deterministic tasks like extracting order numbers or deciding which tool to call. Pitfall: Applying high temperature to extraction tasks introduces unpredictable variability that breaks downstream systems.

References:

• https://dev.to/icraftcode/ai-engg-episode-1-script-2fkp