Stream tokens: Deliver output incrementally, so users see progress immediately.

Cut time-to-first-token: Trim the prompt and use caching, so the first token comes fast.

Show status during work: Indicate tool calls and processing, so silence isn't mistaken for failure.

Return partial results: Deliver what's ready while more computes, so users get value sooner.

Acknowledge immediately: Confirm receipt fast, so users know the request is being handled.

Retrieve relevant tools: Surface only applicable tools per query, so the model chooses from a small set.

Write precise descriptions: Give clear, distinct tool specs, so selection is unambiguous.

Group tools by domain: Namespace or cluster them, so the model navigates fewer at once.

Provide selection examples: Show correct choices for representative tasks, so the model learns the mapping.

Validate the choice: Check the selected tool fits the task, so wrong picks are caught.

Scope all data by user: Namespace retrieval, memory, and cache per user, so cross-user access is structurally impossible.

Enforce at the data layer: Apply access control at the source, so filtering doesn't depend on the model.

Never share caches across users: Key caches by user, so one user's data can't surface for another.

Log data access per request: Record what data each request touched, so leaks are diagnosable.

Audit for isolation: Test that users can't access each other's data, so leaks are caught before production.

Parallelize independent calls: Run non-dependent calls concurrently, so latency is the slowest, not the sum.

Combine calls: Merge multiple reasoning steps into one prompt where possible, so you make fewer calls.

Use faster models for simple steps: Route lightweight steps to small models, so they don't add much latency.

Cache repeated calls: Reuse results for identical sub-calls, so you skip redundant work.

Stream the final step: Stream the last call's output, so users see progress sooner.

Track sources through the pipeline: Carry provenance from retrieval to answer, so attribution is available.

Require citations: Force the model to attribute claims to sources, so answers are cited.

Validate attribution presence: Check every answer has required citations, so uncited ones are caught.

Enforce post-generation: Add attribution structurally if missing, so it's never absent.

Verify citation accuracy: Confirm cited sources actually support the claims, so attribution is correct.

Route by difficulty: Send easy requests to cheaper models, so you spend big-model budget only where needed.

Trim prompt tokens: Cut redundant context and use caching, so per-call cost drops.

Cap output length: Limit generation to what's needed, so you don't pay for verbosity.

Cache aggressively: Reuse results for repeated queries, so you avoid redundant calls.

Measure quality per change: Validate each cost cut against evals, so quality doesn't silently drop.

Set tight timeouts: Bound the slow API, so it can't stall the agent indefinitely.

Cache its responses: Reuse recent results, so you avoid the slow call when possible.

Call it in parallel: Overlap the slow call with other work, so its latency is hidden.

Prefetch when predictable: Start the call early, so results are ready when needed.

Degrade gracefully: Proceed without it when it's too slow, so responsiveness holds.

Validate each step's output: Check intermediate results before passing them on, so errors are caught early.

Add sanity checks: Verify results fall in expected bounds, so implausible ones are flagged.

Verify against the goal: Confirm each step still serves the objective, so drift is caught.

Gate on confidence: Halt when a step is low-confidence, so uncertain results don't propagate.

Trace intermediate values: Log each step's output, so failures are diagnosable after the fact.

Ground extraction in source: Require values come from the provided text, so nothing is invented.

Verify against source: Check each extracted value appears in the input, so hallucinations are caught.

Allow null over invention: Let fields be empty when absent, so the model doesn't fabricate to fill them.

Cite source spans: Have the model point to where each value came from, so extraction is verifiable.

Score confidence per field: Flag low-confidence extractions for review, so uncertain values are checked.

Segment metrics by request type: Track quality per category, so category-specific decline is visible.

Baseline per category: Know normal performance for each type, so drift stands out.

Track trends over time: Watch metrics longitudinally, so slow degradation surfaces before it's severe.

Alert on category drift: Flag when a specific category declines, so you catch it early.

Diagnose per category: Investigate the failing type specifically, so fixes are targeted.

Use approximate search: Apply ANN indexes like HNSW, so search stays fast without exact scans.

Partition the index: Shard by metadata or namespace, so queries search a smaller space.

Filter before searching: Narrow with metadata first, so vector search runs on fewer candidates.

Tune the index: Balance recall against speed in ANN parameters, so latency meets your target.

Cache frequent queries: Serve hot queries from cache, so common requests skip the search.

Lower temperature: Reduce sampling randomness for stability-critical paths, so outputs are consistent.

Cache canonical answers: Serve a fixed answer for repeated inputs, so downstream sees stability.

Constrain output structure: Enforce a schema so the shape is always stable, so parsing doesn't break.

Pin the model version: Use a fixed model, so provider updates don't shift behavior.

Validate against a contract: Check output meets the downstream contract, so instability is caught before it propagates.

Use batch APIs: Exploit provider batch pricing for non-urgent volume, so per-request cost drops sharply.

Deduplicate inputs: Skip identical requests, so you don't pay for redundant work.

Route to cheaper models: Use the smallest model meeting quality for batch work, so cost falls.

Cache across the batch: Reuse results for similar inputs, so repeated work is avoided.

Self-host for steady volume: Run open models when batch volume justifies the infrastructure, so you avoid per-token API cost.

Summarize large outputs: Condense big tool results before feeding them back, so they fit the window.

Truncate intelligently: Keep the relevant parts of large outputs, so important content survives.

Store and reference: Save the full output and pass a reference, so the model retrieves detail on demand.

Paginate results: Return tool output in chunks, so the model processes it in pieces.

Cap output size: Limit what tools return, so responses stay manageable.

Isolate request handling: Process each request independently, so one failure doesn't affect others.

Catch exceptions per request: Wrap handling so errors are contained, so a crash doesn't propagate.

Validate inputs early: Reject malformed requests before they cause exceptions, so bad input fails cleanly.

Use bulkheads: Limit the blast radius of failures, so one bad request can't exhaust shared resources.

Fail the request, not the worker: Return an error for the bad one and keep serving others, so availability holds.

Test on your own data: Evaluate the model on your queries, not public benchmarks, so domain fit is measured.

Check version consistency: Ensure documents and queries use the same model, so mismatches aren't the cause.

Re-tune parameters: Recalibrate thresholds and top-k for the new model, so old settings don't hurt.

Compare retrieval directly: Diff what each model retrieves on the same queries, so the difference is visible.

Verify the migration: Confirm re-embedding completed correctly, so partial migration isn't the issue.

Use the saga pattern: Pair each action with a compensating undo, so failures can be rolled back.

Order for reversibility: Do reversible steps first and irreversible last, so failures are recoverable.

Make tools idempotent: Ensure retries don't duplicate effects, so recovery is safe.

Checkpoint between steps: Save state so you can resume or roll back, so partial failure is handled.

Confirm irreversible steps: Gate permanent actions until prerequisites succeed, so you don't commit prematurely.

Summarize old turns: Compress earlier history into a running summary, so key facts survive truncation.

Pin critical context: Keep essential early information in-window, so it isn't dropped.

Retrieve relevant history: Pull past turns on demand rather than keeping all, so the window stays lean.

Prioritize by relevance: Keep the most relevant history, not just the most recent, so importance drives retention.

Track a token budget: Bound history size explicitly, so the window never overflows.

Use a small model for routing: Route with a lightweight classifier, so you don't spend a big model on every request.

Try rules first: Handle clear-cut routing with deterministic rules, so most requests skip the LLM entirely.

Cache routing decisions: Reuse routes for similar requests, so repeated routing is free.

Route on embeddings: Classify by vector similarity, so routing avoids generation cost.

Skip routing when obvious: Bypass the router for unambiguous requests, so it only runs when needed.

Log retrieved context: Capture exactly what was retrieved per query, so you can check if the fact was there.

Check for the needed fact: Verify whether retrieval contained the answer, so you know which stage failed.

Test generation with gold context: Feed perfect context and check the answer, so you isolate generation quality.

Measure each stage separately: Score retrieval and generation independently, so failures are attributable.

Categorize failures: Label each as retrieval or generation, so patterns guide where to fix.