Marta Lewandowska

2025-03

Investigated upgrading from text-embedding-3-small to text-embedding-3-large. Ran systematic evaluation on our 200-query internal benchmark across all query categories.

Challenge: text-embedding-3-large costs 2x more per token and produces 3072-dimensional vectors (vs 1536 for small), doubling Qdrant storage requirements. The 4% recall@5 improvement is significant for precision-critical use cases but may not justify cost for bulk search.

Learned: Embedding model upgrades should be evaluated per use case, not applied uniformly. High-value expert-finding queries benefit from large model; bulk document search tolerates small model performance.

2024-11

Implemented multi-tenant vector store namespacing — each organization gets its own Qdrant collection with isolated data and independent indexing parameters.

Challenge: Collection naming conflicts when organization names contained special characters or were too long (Qdrant has a 255-char limit). Added a slugification step and a UUID suffix to guarantee uniqueness.

Learned: Never use user-provided names directly as identifiers in external systems. Always normalize, slug, and check for uniqueness before creating resources.

2024-07

Led migration of 800K vectors from ChromaDB to Qdrant. Wrote migration script: batch export from ChromaDB, transform metadata, batch upsert to Qdrant. Validated recall@10 before and after.

Challenge: Qdrant batch upsert has a 100MB payload limit per request. With 1536-dimensional float32 vectors, batches of 300+ vectors would hit the limit. Had to dynamically size batches based on vector dimensionality and metadata size.

Learned: Data migration for vector stores is trickier than for SQL databases because you also need to validate semantic quality (recall), not just data completeness (row count). Always run recall evaluation before decommissioning the source.

2024-03

Implemented cross-encoder re-ranking: after vector ANN retrieval of top-50 candidates, a cross-encoder re-ranks them to select the final top-5. Used ms-marco-MiniLM-L-6-v2.

Challenge: Cross-encoder added 350-500ms latency per query — almost doubling total response time. Deployed it as a separate GPU sidecar service and cached re-ranking results for repeated queries.

Learned: Re-ranking improves precision@5 by ~15-20% over vector-only retrieval on our test set. The latency cost is acceptable for precision-critical use cases (expert finding) but not for high-throughput search.

2023-10

Built async batch embedding pipeline with Redis queue: documents submitted to queue, worker processes up to 100 texts per API batch, results stored to Qdrant. Built-in token counting to stay within API limits.

Challenge: OpenAI embedding API rate limits are token-based. Batches with long texts would exceed the limit and fail. Added a token counter (tiktoken) before each batch that splits oversized batches automatically.

Learned: Token counting before API calls is essential for reliable batch embedding. The cost of counting (fast, local) is negligible compared to the cost of failed API calls and retries.

2023-06

Implemented and compared document chunking strategies: fixed-size (512 tokens, 64 overlap), sentence-based, and semantic chunking using embedding similarity between adjacent sentences.

Challenge: Semantic chunking produced much higher quality chunks but added 3-4 minutes of processing per document. For a corpus of 3000 documents, this was 150+ hours of compute — not acceptable for regular re-indexing.

Learned: Recursive character text splitter with 10% overlap is 80% as good as semantic chunking for RAG quality at 2% of the compute cost. Semantic chunking is worth it only for very long, heterogeneous documents.

2023-02

Benchmarked embedding models for Polish/English mixed technical text. Evaluated: OpenAI text-embedding-ada-002, sentence-transformers multilingual-e5-large, and multilingual-mpnet-base-v2 on an internal test set of 200 engineering queries.

Challenge: sentence-transformers required GPU for reasonable inference speed (CPU was 8x slower than GPU). Our development environment had no GPU access, making local iteration slow.

Learned: For MVP with mixed-language technical documents, OpenAI text-embedding-ada-002 offered the best quality/operability ratio. Local models are better for cost-sensitive production but require GPU infrastructure investment.