Practice GenAI Interview Questions

Add fallback providers: Route to a backup model when the primary fails, so outages don't halt you.

Abstract behind a gateway: Wrap providers uniformly, so failover is seamless.

Use circuit breakers: Stop calling a failing provider and switch fast, so you don't hang on a dead one.

Degrade gracefully: Serve cached or reduced responses during outages, so users retain some function.

Monitor provider health: Track latency and errors, so failover triggers before users notice.

Rerank and position well: Place the most relevant chunks at the start and end, so they aren't lost in the middle.

Limit context size: Provide fewer, more relevant chunks, so nothing important is buried.

Rerank aggressively: Ensure only the best chunks are included, so the model isn't diluted.

Force multi-source citation: Require citing several chunks, so the model must engage with all provided context.

Test middle-context recall: Evaluate whether mid-context facts are used, so the effect is measured.

Use continuous batching: Add and remove requests dynamically like vLLM, so requests don't wait for a full batch.

Set a max wait time: Flush partial batches after a deadline, so latency is bounded.

Separate latency tiers: Batch throughput-oriented work separately from interactive, so each gets suitable handling.

Size batches adaptively: Adjust batch size to load, so you balance throughput and latency dynamically.

Prioritize interactive requests: Let latency-sensitive ones jump the queue, so they aren't stuck.

Separate instructions from data: Structurally delimit untrusted content, so it's treated as data, not commands.

Apply least privilege: Limit tool scopes and credentials, so a successful injection has limited blast radius.

Gate high-impact actions: Require confirmation for consequential steps, so injection can't complete damage alone.

Block untrusted-driven egress: Prevent data flowing to endpoints suggested by content, so exfiltration is stopped.

Red-team continuously: Probe with novel injections, so defenses evolve against new attacks.

Evaluate on hard cases: Test quantized quality on the cases that matter, so degradation is measured where it counts.

Choose the right method: Compare AWQ, GPTQ, and FP8 for your model, so you pick the least lossy.

Try mixed precision: Keep sensitive layers at higher precision, so critical quality is preserved.

Tune the trade-off: Balance memory savings against quality loss, so you don't over-quantize.

Validate before deploying: Confirm quantized quality meets the bar, so regressions don't ship.

Retrieve iteratively: Use each hop's result to form the next query, so chained lookups work.

Decompose the question: Break multi-hop questions into sub-questions, so each hop is explicit.

Track intermediate findings: Carry results across hops, so later retrieval builds on earlier.

Bound the hops: Cap iterations, so multi-hop retrieval doesn't run away.

Verify the chain: Check each hop's result supports the next, so errors don't compound.

Use a supervisor: Have a coordinator break ties and assign turns, so agents don't circularly wait.

Set timeouts: Bound how long an agent waits, so deadlocks self-resolve.

Order resource acquisition: Impose a consistent acquisition order, so circular waits can't form.

Detect cycles: Monitor for mutual waiting, so deadlocks are caught and broken.

Define clear ownership: Assign task ownership unambiguously, so agents don't both wait to act.

Calibrate against humans: Validate judge scores on a human-labeled set, so agreement is measured.

Use explicit rubrics: Grade against clear, granular criteria, so scoring is consistent.

Control for bias: Counter position, length, and self-preference effects, so scores aren't skewed.

Ensemble judgments: Average multiple runs or models, so variance drops.

Audit disagreements: Review judge-human mismatches, so systematic errors are found and corrected.

Use paged attention: Allocate KV-cache in pages like vLLM, so memory is used efficiently and fragmentation drops.

Cap concurrent sequences: Limit simultaneous generations to what memory allows, so you don't over-commit.

Evict or preempt: Swap out lower-priority sequences when memory is tight, so high-priority ones proceed.

Bound context length: Limit sequence length, so per-request cache stays predictable.

Monitor cache pressure: Track memory usage, so you throttle before failures occur.

Propagate a trace ID: Attach one correlation ID from entry through every system, so the flow is linkable.

Centralize telemetry: Aggregate logs and metrics in one place, so you don't chase five dashboards.

Log at each hop: Capture input and output at every stage, so the path is reconstructable.

Visualize the trace: Render the full request flow, so failures are locatable at a glance.

Correlate across systems: Link related events by trace ID, so the complete picture assembles.

Test against an eval set: Measure each prompt variant on a fixed set, so improvements are objective.

Change one thing at a time: Vary single elements, so impact is attributable.

Ablate sections: Remove parts to find what matters, so dead weight is identified.

Version prompts: Track variants and their scores, so you can compare and revert.

Guard against overfitting: Validate on held-out cases, so gains generalize.

Validate and scan files: Check type and scan for threats before processing, so malicious files are caught.

Sandbox parsing: Extract content in isolation, so a malicious file can't compromise the system.

Treat content as untrusted: Never let file content issue instructions, so injection via files is blocked.

Limit resource use: Cap file size and processing, so files can't exhaust resources.

Fail safely: Handle unparseable or suspicious files cleanly, so failures don't cause harm.

Detect refusals: Recognize when the model declined rather than answered, so it's not treated as valid output.

Retry with rephrasing: Reformulate and retry legitimate requests, so occasional refusals are overcome.

Provide clear context: Frame internal requests to avoid triggering refusals, so they succeed.

Fall back gracefully: Handle persistent refusals without crashing, so the pipeline continues.

Log refusal patterns: Track what triggers them, so recurring false refusals are addressed.

Load-test at target scale: Simulate 10x traffic, so you find limits before production does.

Identify bottlenecks: Determine what breaks first, so you know where to reinforce.

Test dependencies too: Load the vector DB, provider limits, and tools, so the weakest link surfaces.

Verify horizontal scaling: Confirm adding capacity actually helps, so scaling works as assumed.

Provision headroom: Ensure capacity ahead of the ramp, so you're not scrambling.

Link answers to document versions: Track which version each answer used, so staleness is detectable.

Invalidate on update: Clear cached answers when their source document changes, so stale ones aren't served.

Re-index changed documents: Update retrieval when documents change, so new answers use current content.

Version cache keys: Include document version in keys, so updated docs miss stale cache.

Verify freshness on serve: Check the answer's source is current, so staleness is caught.

Version config: Track config in source control with history, so changes are auditable.

Require review: Gate config changes behind approval, so risky edits are caught.

Validate on load: Check config against a schema, so invalid values are rejected.

Tag requests with config version: Record which config each request used, so behavior maps to config.

Enable fast rollback: Revert config independently, so a bad change is quickly undone.

Enforce read-only access: Run generated SQL under a restricted account, so it can't write or lock harmfully.

Validate before executing: Parse and check the query against an allow-list, so dangerous queries are blocked.

Set query limits: Apply row limits and timeouts, so runaway queries are bounded.

Run against a replica: Execute on a read replica, so production isn't affected.

Preview before running: Show the query for confirmation on risky operations, so users catch problems.

Use hybrid search: Combine keyword and vector search, so exact and semantic queries both work.

Detect exact-match intent: Route code and quoted phrases to keyword search, so precision is preserved.

Weight by query type: Favor keyword matching for identifiers and semantic for concepts, so each fits.

Index for exact match: Maintain a keyword index alongside vectors, so exact lookups are fast.

Merge and rerank: Combine both result sets and rerank, so the best of each surfaces.

Speculate only on likely paths: Prefetch high-probability next steps, so waste stays low.

Gate on cost: Skip speculation for expensive operations, so wrong guesses don't cost much.

Cancel early: Abort speculative work once the real path is known, so waste is minimized.

Speculate for latency-critical paths only: Reserve it where speed matters most, so cost is justified.

Measure the trade-off: Track wasted cost against latency saved, so speculation stays worthwhile.

Build evals from production: Source test cases from real traffic, so evals reflect actual usage.

Include the hard long tail: Add real edge cases, so evals aren't dominated by easy ones.

Match production conditions: Test with realistic noise and formats, so results predict reality.

Correlate eval with live metrics: Check that eval wins translate to production gains, so you trust the evals.

Refresh continuously: Feed new production failures into evals, so the gap doesn't reopen.