Piotr Nowak

2025-02

Optimized Qdrant memory usage using scalar quantization (int8). Reduced vector storage footprint from 12GB to 3GB for our 1M+ vector collection.

Challenge: Scalar quantization slightly degrades recall. Initial quantization configuration dropped recall@10 from 0.97 to 0.88 — below our 0.95 threshold. Tuned the rescore parameter (oversampling factor) to recover recall at the cost of slightly higher query latency.

Learned: Quantization is a recall/latency/memory tradeoff — not a free lunch. Always measure recall degradation against a test set before deploying to production.

2024-10

Integrated citation and source attribution into RAG responses — every factual claim in the LLM answer is linked to the specific source chunk and document. Built a citation verification step.

Challenge: Getting GPT-4o to consistently follow the citation format was harder than expected. Prompt-only approaches had ~30% non-compliance. Switched to structured output (JSON with answer + citations array) and post-processing to render inline citations.

Learned: Citation fidelity (does the citation actually support the claim?) is a separate problem from citation format compliance. Even correctly formatted citations can misattribute claims. Added a spot-check validation step.

2024-05

Added metadata filtering to RAG — filter retrieved chunks by document type, date range, authoring team, and security classification. Implemented a natural language filter parser using GPT-4o to extract filter intent.

Challenge: Getting GPT-4o to reliably extract filter parameters from ambiguous queries like 'recent quality documents' required careful prompt design. Structured output (JSON mode) was essential — free text extraction was too noisy.

Learned: Qdrant's payload filtering is extremely fast — filtered queries over 1M vectors still return in <50ms. Investing in rich metadata at ingestion time pays off at query time.

2024-01

Implemented conversational RAG with LangChain ConversationalRetrievalChain — multi-turn chat that reformulates the user question based on conversation history before retrieval.

Challenge: Long conversations ballooned context window usage. At 10+ turns, each request was sending 8K+ tokens of history. Implemented a sliding window with summarization — last N turns in full, earlier turns compressed.

Learned: Conversation history management is a product decision, not just a technical one. Users expect the chatbot to 'remember' things from early in the conversation. Be explicit about what is kept and what is summarized.

2023-10

Migrated vector store from ChromaDB to Qdrant. ChromaDB was hitting memory limits at 200K+ chunks and query latency was degrading. Wrote migration script to re-embed and upsert all vectors.

Challenge: Configuring Qdrant on a dedicated server was painful — port 6333 was blocked by the corporate firewall. Took 3 days to get IT to approve the firewall exception. During that time, tested connectivity via SSH tunnel which also had issues due to bastion host configuration.

Learned: For enterprise deployments, network and firewall requirements should be scoped during architecture design, not discovered during setup. Now I include a 'network prerequisites' checklist in every deployment plan.

2023-06

Implemented hybrid search combining BM25 lexical search with vector similarity retrieval. Evaluated RRF (Reciprocal Rank Fusion) and linear weighted combination for merging ranked lists.

Challenge: Normalizing BM25 and cosine similarity scores for combination is non-trivial — their distributions are very different. RRF proved more robust than weighted combination because it is rank-based and requires no normalization.

Learned: Hybrid search improves recall for keyword-specific queries (part numbers, model IDs) significantly without hurting semantic query quality. Worth the added complexity.

2023-02

Built the first internal RAG prototype: LangChain + ChromaDB + GPT-3.5. Scraped and chunked 500+ internal engineering documents (PDFs, Word files, Confluence exports) into a searchable knowledge base.

Challenge: ChromaDB persistence was unreliable — data would disappear on container restart until pinning to version 0.4.x. Lost two days of indexing work before identifying the root cause.

Learned: Always pin dependency versions in AI projects. LLM library ecosystems move extremely fast and breaking changes are common between minor versions.