Chunking, Reranking, Prompting: What I Learned Building RAG the Hard Way

I stopped reading about RAG and built one from scratch: FAISS + Sentence Transformers + a Cross-Encoder re-ranker + Groq + FastAPI. Every layer was mine to design, break and fix. Three things surprised me: 1️⃣ Chunking strategy matters as much as the embedding model. 2️⃣ Re-ranking did more for answer quality than anything else. 3️⃣ Your prompt is part of the system architecture, not an afterthought. Full write-up, including what broke first: https://lnkd.in/dnpVQr49 More coming. I've got follow-ups on what happened when I actually used it and measured it. #RAG #LLM #AIEngineering #Python

A RAG system was built from scratch using FAISS for vector search, Sentence Transformers for embeddings, a cross-encoder for re-ranking, Groq for LLM inference, and FastAPI to serve it, with each layer designed, tested, and debugged individually. Three findings stood out from the process: chunking strategy had as much impact on retrieval quality as the choice of embedding model, re-ranking improved answer quality more than any other single component, and prompt construction functioned as a core part of the system's architecture rather than a final step added at the end. The standard RAG pipeline follows two stages: an indexing stage, where documents are chunked, embedded, and stored in a vector database, and a query stage, where a user's query is embedded, used to retrieve relevant chunks from that database, and combined with the query into an augmented prompt sent to the LLM to generate a response.