
Amazon Web Services (AWS) is pushing back against a common production anti-pattern in generative AI engineering: routing deterministic workflow steps through large language models (LLMs). In a new AWS Machine Learning blog post published November 17, 2026, authors Meghan Kothari, Krishna Aswani, Pranav Patankar, and Vineet Kamdar argue that the largest cost lever in an AI architecture is deciding whether a step needs a model at all.
Why Prototype Defaults Become Production Bottlenecks
According to the AWS authors, teams frequently default to LLM calls during early prototyping for tasks such as input validation, request body reformatting, and intent routing because a single model invocation is inexpensive at low volume. Once deployed at scale, however, AWS notes that redundant model calls compound across three operational dimensions: token spend rises linearly with request volume even on low-value steps, stacked model calls add hundreds of milliseconds of latency per step compared with single-digit milliseconds for deterministic code, and probabilistic outputs complicate testing and compliance audits.
Rather than starting cost optimization with model selection—such as swapping a frontier model for a smaller variant—AWS frames the architectural decision one step earlier by separating rule-based work from open-ended judgment.
A Three-Question Filter for AI Workflows
To help builders audit existing pipelines and agentic systems, AWS outlines a sequential three-question filter before invoking a right-sized model on Amazon Bedrock:
- Can a deterministic rule check success? Tasks involving schemas, numeric ranges, formats, or exact keywords—such as validating an order number—belong in code using services like Amazon API Gateway request validation, AWS Lambda, or AWS Step Functions Choice states. For partial matches, AWS recommends a fast path that resolves obvious keywords like "refund" in code before escalating edge cases.
- Can a model write the rule once? Instead of calling an LLM at runtime to extract a carrier tracking number from every incoming email, developers can use a model during development to draft a regular expression or JSON schema, review it, and commit it as static code.
- Does the step choose from a fixed list? Classification or routing across defined categories should use a constrained decision call, escalating to a full generative model only when confidence is low.
Only when a task fails all three criteria—such as summarizing prose or drafting a tailored customer reply—does AWS recommend invoking a right-sized generative model.
Editorial Context and Practical Tradeoffs
AWS's guidance reflects a broader industry shift away from monolithic "LLM-for-everything" agent designs toward hybrid architectures that combine traditional software engineering with selective inference. For enterprise teams facing rising inference bills and latency targets, auditing pipelines for rule-shaped tasks offers an immediate path to lower operating costs without sacrificing capability where language judgment is genuinely required. Still, the approach shifts engineering effort back toward maintaining explicit validation rules, regex patterns, and escalation thresholds—tradeoffs teams must weigh against the speed of rapid prototyping.
Automated Vertex AI (gemini-4-hydrogen) primary-source report; reviewed 2026-11-17.



