Agnieszka Dabrowska

2025-02

Designed server-sent events (SSE) streaming architecture for the chatbot UI — streaming LLM tokens from FastAPI directly to the browser for improved perceived latency.

Challenge: Handling partial JSON in streaming mode when the LLM outputs structured JSON — chunks can split mid-token. Built a stateful stream parser that buffers until a complete JSON object is received.

Learned: Streaming reduces perceived latency dramatically even when total response time stays the same. First-token latency matters more than total latency for user experience.

2024-10

Led design and execution of a retrieval evaluation framework. Built a golden dataset of 50 queries with annotated relevant documents, covering engineering Q&A, expert finding, and project search.

Challenge: High inter-annotator disagreement among domain experts — average Cohen's kappa of 0.61 for relevance judgments. Ran adjudication sessions to resolve disagreements and establish clear relevance criteria.

Learned: Retrieval evaluation in specialized domains needs clear relevance rubrics agreed upfront. Vague 'is this relevant?' questions produce unusable annotations.

2024-06

Investigated speech-to-text integration for a hands-free query interface targeting shop floor workers. Tested Whisper large-v3 on aviation domain vocabulary.

Challenge: Aviation jargon (FADEC, HPT, LPT, compressor bleed, Ni-based superalloy) was frequently misrecognized. Whisper transcribed 'FADEC' as 'faded' and 'HPT' as 'happy'. Had to build a post-processing correction layer with domain vocabulary.

Learned: Domain-specific ASR vocabulary correction via string matching on known acronyms and proper nouns dramatically improves transcription quality for specialized use cases, without needing model fine-tuning.

2024-02

Implemented agentic RAG where the LLM decides whether to retrieve or answer from parametric knowledge. Built a query router that classifies queries as 'retrieval needed' vs 'answer from model'.

Challenge: Router made wrong decisions on ambiguous queries 23% of the time. Feeding the router examples of borderline cases in few-shot prompts improved accuracy from 77% to 91%.

Learned: Routing logic needs its own evaluation set. Testing end-to-end RAG quality hides routing errors. Measuring routing accuracy separately allows targeted improvement.

2023-10

Built a multimodal retrieval prototype combining image and text queries. Used CLIP for image embeddings merged with text embeddings in a unified vector space for defect case retrieval.

Challenge: Image and text embedding spaces are not naturally aligned in ChromaDB — combined search returned semantically incoherent results. Explored joint embedding normalization and learned projection layers.

Learned: Keeping image and text in separate collections with a late-fusion retrieval step (rank merging after individual retrievals) outperformed forced joint embedding in our use case.

2023-05

Designed RAG architecture for technical documentation search across 3000+ engineering specs, maintenance manuals, and change notices. Defined chunking strategy for complex PDF documents with tables and figures.

Challenge: Nested tables in engineering PDFs destroy context when extracted as plain text. Standard PyPDF2 was producing garbled output for multi-column layouts. Switching to pdfplumber with custom table extraction improved chunk quality significantly.

Learned: No single PDF extraction library handles all cases. The combination of pdfplumber for tables and pypdf2 for text-heavy pages, with a routing heuristic, gave the best overall results.

2023-01

Explored vision-language models for inspection image analysis. Evaluated CLIP, BLIP-2, and LLaVA for classifying defect images from turbine component inspections.

Challenge: Severe domain shift — models trained on natural images performed poorly on industrial component photographs. CLIP zero-shot accuracy on defect classification was only 42%, far below usable threshold.

Learned: Industrial inspection images require domain-specific fine-tuning or at minimum domain-adapted prompting. Zero-shot vision models are not ready for aerospace inspection without adaptation.