Centralize rate limiting: Track usage against the shared limit in one place, so concurrent requests coordinate.

Use a token bucket: Meter access with an atomic limiter, so the limit is respected under concurrency.

Queue when at the limit: Hold requests rather than failing, so bursts drain smoothly.

Allocate fairly: Prevent one request stream from monopolizing the key, so all get access.

Add capacity when needed: Use multiple keys or providers for sustained load, so the shared limit isn't a hard ceiling.

Tier your eval suite: Run a fast core set on every change and the full suite less often, so quick feedback is always available.

Parallelize eval runs: Execute cases concurrently, so wall-clock time drops.

Cache eval results: Skip re-running unchanged cases, so only affected ones rerun.

Sample for quick checks: Use a representative subset for fast iteration, so developers get signal in seconds.

Integrate into the workflow: Run evals automatically in CI, so they aren't a manual step to skip.

Set aggressive timeouts: Bound every call just above expected latency, so hangs are cut off fast.

Retry on a different path: Reissue timed-out calls to another provider or replica, so recovery is quick.

Use streaming to detect stalls: Watch for token flow stopping, so hangs are caught mid-stream.

Release resources on timeout: Free connections and memory when a call is abandoned, so hangs don't leak.

Alert on hang patterns: Track timeout frequency, so systemic issues surface.

Detect structural elements: Identify tables, code, and lists, so they can be handled specially.

Keep structures intact: Never split a table or code block across chunks, so they stay readable.

Chunk on semantic boundaries: Split at natural section breaks, so content stays coherent.

Attach context to structures: Include surrounding headers with a table or code, so it's interpretable alone.

Handle oversized structures: Split very large structures sensibly, so they fit while staying usable.

Migrate incrementally: Re-embed in batches over time, so the cost and load spread out.

Dual-index during transition: Serve from the old index while building the new, so there's no downtime.

Prioritize hot content: Re-embed frequently-retrieved documents first, so quality improves fastest where it matters.

Never mix embedding versions: Keep queries and documents on the same model, so retrieval stays valid.

Cut over atomically: Switch to the new index once complete, so there's a clean transition.

Move dynamic content late: Put variable data after static content, so the cacheable prefix stays stable.

Order by volatility: Arrange from most to least stable, so more of the prompt caches.

Separate the variable value: Isolate the dynamic bit so it doesn't break the prefix, so caching holds.

Cache the stable prefix: Explicitly mark the static portion for caching, so it's reused.

Measure cache hit rate: Track how often the cache hits, so prompt structure can be tuned.

Signal tool activity: Show a status indicator during tool calls, so the freeze is explained.

Stream around the call: Continue streaming text before and after, so the pause is minimal.

Parallelize where possible: Start tool calls early, so their latency overlaps with generation.

Set tool timeouts: Bound tool calls, so the stream doesn't hang indefinitely.

Fall back on failure: Continue with a degraded response if the tool fails, so the stream completes.

Use constrained generation: Enforce the schema at decode time where supported, so invalid args can't be produced.

Validate before executing: Check arguments against the schema, so bad ones are caught pre-execution.

Re-prompt on failure: Return the validation error to the model, so it corrects.

Coerce where safe: Fix minor type mismatches automatically, so trivial errors don't fail.

Provide clear schemas: Give precise types and examples, so the model produces valid args more often.

Shrink the candidate set: Rerank a small shortlist, not hundreds, so the reranker does less work.

Use a lighter reranker: Pick a smaller cross-encoder where quality allows, so latency drops.

Batch on GPU: Score candidates in one batched pass, so you exploit parallelism.

Cache rerank scores: Reuse rankings for repeated queries, so you skip recomputation.

Rerank conditionally: Skip it when first-stage confidence is high, so you pay only when needed.

Name what matters most: Explicitly define the highest-value goals, so they aren't crowded out by easy metrics.

Measure the important, even if hard: Invest in measuring what matters, so it isn't neglected for convenience.

Use qualitative signals: Supplement metrics with judgment where measurement is hard, so the important is tracked.

Weight priorities deliberately: Rank work by real value, not measurement ease, so focus stays right.

Guard against measurement bias: Watch for drift toward the easily-measured, so priorities stay honest.

Notice the conflict: Detect when stated and revealed preferences diverge, so it's not ignored.

Default to stated preferences: Honor what the user explicitly says, so you respect their agency.

Surface the discrepancy: Point out the apparent conflict, so the user can reconcile it.

Ask which to follow: Clarify when the conflict matters, so you don't guess wrong.

Adapt as they clarify: Adjust based on their response, so you serve their true preference.

Recognize questionable requests: Detect when a request warrants a second look, so it's not blindly executed.

Raise concerns constructively: Voice the issue helpfully, so the user isn't lectured.

Explain the reasoning: Show why you're questioning it, so the concern is credible.

Respect user autonomy: Proceed if they still want to after being informed, so you don't override them.

Decline only genuine harms: Refuse only when doing it would clearly cause serious harm, so you don't over-refuse.

Set honest expectations: Tell the user the task genuinely takes time, so they aren't misled.

Explain why: Show what makes it irreducibly slow, so the timeline is credible.

Offer a fast partial: Provide what can be done quickly, so users get some immediate value.

Don't fake speed with quality loss: Avoid rushing into a bad answer, so quality holds.

Move slow work to background: Run it async and notify on completion, so users aren't blocked waiting.

Prioritize correctness over approval: Aim for right answers, not just satisfying ones, so users aren't pleased into error.

Ground answers in evidence: Base them on real support, so satisfaction rests on accuracy.

Verify before delivering: Check answers for correctness, so satisfying-but-wrong ones are caught.

Flag uncertainty honestly: Don't project false confidence to please, so users aren't misled.

Measure outcomes, not just satisfaction: Track whether answers actually worked, so pleasing-but-wrong ones surface.

Prioritize honesty: Give the accurate assessment, so users aren't falsely reassured.

Deliver with empathy: Acknowledge the user's feelings, so honesty isn't cold.

Reassure where genuinely warranted: Offer real reassurance when it's true, so support is authentic.

Provide constructive next steps: Show what they can do, so honesty is actionable, not just deflating.

Don't substitute comfort for truth: Avoid empty reassurance, so the assessment stays valuable.

Anticipate use: Consider how the answer will be acted on, so foreseeable harm is caught.

Flag risky applications: Warn when acting on the answer could cause harm, so users are aware.

Add necessary safeguards: Include caveats or conditions for safe use, so harm is mitigated.

Withhold genuinely dangerous help: Decline when use would clearly cause serious harm, so you don't enable it.

Right-size the caution: Don't over-warn on benign answers, so caution stays proportionate.

Detect the mode: Recognize whether the user is exploring possibilities or executing a plan, so you respond right.

Support exploration broadly: Offer options and considerations when exploring, so they can range.

Support execution precisely: Give concrete, actionable steps when executing, so they can act.

Adapt as it shifts: Switch modes when the user moves from exploring to doing, so you stay aligned.

Confirm when ambiguous: Clarify the mode if unclear, so you serve the right one.

Read the user's goal: Judge whether they want the answer or the skill, so you serve the real need.

Answer directly when that's wanted: Give the result when speed and outcome matter, so you don't over-teach.

Teach when capability matters: Show the method when learning serves them, so they grow.

Do both when useful: Provide the answer and the how, so both needs are met.

Let users signal: Allow them to ask for one or the other, so you don't guess.

Take claims as a starting point: Use stated expertise to set initial depth, so you meet users where they say they are.

Verify through interaction: Adjust based on how they respond, so claims are validated in practice.

Adapt if the claim seems off: Recalibrate when responses don't match stated expertise, so you fit reality.

Avoid condescension or overshooting: Neither talk down to nor over-assume, so the level fits.

Let users recalibrate: Allow them to signal too basic or too advanced, so matching corrects.

Anticipate metric incentives: Consider what behavior a metric will encourage, so you avoid perverse incentives.

Watch for gaming: Detect when the system optimizes the metric at the expense of real value, so distortion is caught.

Use balanced metrics: Combine measures so no single one can be gamed harmfully, so behavior stays aligned.

Tie metrics to real goals: Ensure metrics proxy actual value, so optimizing them serves the intent.

Revise metrics that misdirect: Change measures that drive wrong behavior, so incentives stay right.