Natalia Baran

Data Scientist · Data & AI Platform

Warsaw, Poland

Data Scientist working at the intersection of manufacturing data and modern ML. Focuses on training and evaluating models for time-series anomaly detection, NLP classification, and increasingly on evaluating and fine-tuning language models for domain-specific use cases. Collaborates closely with both the AI platform team and quality engineers to translate domain data into ML-ready datasets.

Expertise

ML model training and evaluation
anomaly detection
NLP fine-tuning
Bayesian statistics
embedding model evaluation

Technologies

PyTorch scikit-learn HuggingFace Transformers MLflow Python pandas SHAP Optuna Llama DPO sentence-transformers Jupyter

Work History

2025-02

Conducted systematic comparison of RAG vs fine-tuned small LLM for aerospace domain Q&A. Tested on 80-question benchmark spanning 3 difficulty levels.

Challenge: The comparison is sensitive to retrieval quality — RAG underperforms when the relevant document is not retrieved. Had to separate retrieval failures from generation failures in the analysis.

Learned: RAG wins for factual recall when relevant documents are available. Fine-tuned models win for reasoning tasks where the answer requires combining implicit domain knowledge not explicit in any single document. Production systems should consider combining both.

Python Qdrant Llama 3 GPT-4o MLflow pandas

2024-10

Built CT scan defect classification benchmark — 400 labeled images of turbine component cross-sections covering 4 defect types: porosity, inclusion, crack, and surface delamination. Evaluated ResNet-50, EfficientNet-B4, and ViT-B/16.

Challenge: Severe class imbalance — porosity defects accounted for 62% of samples, cracks only 8%. Standard cross-entropy produced a model that ignored the minority class. Combined focal loss with class-balanced sampling.

Learned: ViT outperforms CNNs on the rare defect classes (crack, surface delamination) likely because global attention captures long-range spatial patterns better than local convolutional filters. CNNs are faster but miss subtle distributed defect signatures.

PyTorch torchvision HuggingFace scikit-learn Python MLflow

2024-06

Fine-tuned Llama 3 8B on internal engineering Q&A pairs to create a domain-adapted LLM. Used DPO (Direct Preference Optimization) to improve response quality with limited labeled data.

Challenge: Only 800 Q&A pairs available — too few for stable SFT fine-tuning. DPO with preference pairs (preferred vs rejected responses) gave better sample efficiency. Generated preference pairs by asking engineers to rank two model outputs.

Learned: DPO requires preference pairs, not just correct answers. Collecting preference labels ('which answer is better') is faster for domain experts than writing correct answers from scratch.

Llama 3 PyTorch DPO HuggingFace MLflow Python NVIDIA A100

2024-01

Ran A/B evaluation comparing text-embedding-3-small vs multilingual-e5-large for the internal document search system. Measured precision@5, NDCG@10, and recall@20 on 200-query benchmark.

Challenge: The benchmark queries were originally written in English but engineers often query in Polish. Created a bilingual version of the benchmark (100 EN, 100 PL) to test multilingual performance separately.

Learned: text-embedding-3-small significantly outperforms multilingual-e5-large on English queries. For Polish queries, the gap narrows to within noise — both models handle Polish adequately for technical vocabulary.

OpenAI API sentence-transformers Python pandas Qdrant

2023-09

Performed feature importance analysis for dimensional drift prediction model using SHAP values. Identified spindle temperature and coolant flow rate as top 2 predictors. Presented findings to manufacturing engineering team.

Challenge: Translating SHAP force plots to actionable decisions for non-data-scientists. Manufacturing engineers understood the concept of 'this sensor matters' but not the direction and magnitude of SHAP values. Converted SHAP outputs to operational thresholds and decision rules.

Learned: ML explainability tools are for ML practitioners by default. For domain experts, you need to translate model insights back into the language of their domain — physical quantities, operational thresholds, not SHAP values.

SHAP scikit-learn Python pandas Jupyter

2023-05

Fine-tuned DistilBERT for classification of quality incident reports into 5 categories. Training set of 4000 labeled incidents curated from SAP QM records.

Challenge: Inter-annotator agreement was mediocre (Cohen's kappa = 0.67) — quality engineers often disagreed on category boundaries, especially between 'process deviation' and 'supplier quality issue'. Ran adjudication sessions for the 15% most ambiguous cases.

Learned: Label quality directly caps model quality. For classification tasks in regulated domains, a structured annotation guide with worked examples is as important as the model architecture.

HuggingFace Transformers DistilBERT PyTorch scikit-learn Python

2023-01

Built anomaly detection system for turbine sensor data — combined Isolation Forest for fast initial screening with an LSTM autoencoder for sequence-aware anomaly scoring.

Challenge: Isolation Forest false positive rate was 12% — too high for operator trust. Recalibrated the contamination parameter using 500 expert-labeled examples and tuned the LSTM reconstruction threshold on a separate validation set.

Learned: Anomaly detection thresholds are highly domain-specific and cannot be set from first principles. Operator-labeled examples are essential for calibration. False positive rate matters more than AUC for adoption.

scikit-learn PyTorch Python MLflow pandas