Parallel technical track · Applied MLE

Full-stack ML for
Ads Ranking

Build a coherent understanding of the advertising ML system, from event data and targets to candidate retrieval, ranking, bidding, experimentation and low-latency deployment. Keep this secondary to the active DS job search.

System map

Events & attribution → point-in-time features → candidate retrieval → pCTR / pCVR & ranker → expected value / bid controller → auction & pacing → serving & monitoring → online experiments.

This is a conceptual end-to-end learning project, not a claim about your employer's production architecture.

Suggested capstone

Build a synthetic-data sponsored-search ranking sandbox. Train a pCTR baseline and a tree-based ranking model, estimate expected value, introduce a budget constraint, expose a FastAPI scoring service, and evaluate offline versus simulated online outcomes.

Definition of mastery

Explain the causal or statistical reason behind a design choice.
Implement one small reproducible experiment.
State assumptions, limitations and trade-offs.
Communicate in an Applied MLE interview format.

This week's lightweight learning block

60–90 minutes: explain the difference between ranking by pCTR, pCVR and expected revenue, and why improving offline NDCG might not improve ROAS. Write one hypothesis and one test. Stop there if job-search tasks need attention.

Full-stack modules

01 · Problem formulation

Week 1–2

Define click, conversion, revenue and ROAS as distinct target/objective choicesWrite budget, CPC, latency, inventory and business constraintsUnderstand logged-policy bias and delayed conversionsSketch retrieval → ranking → bid optimization → serving

02 · Data & features

Week 3–4

Design impression/click/conversion event schema with timestampsPractice point-in-time joins and prevent feature leakageImplement negative sampling and temporal splitsDefine cold start, sparse advertiser and missing-attribution handling

03 · Baselines and GBDT

Week 5–6

Fit logistic regression baseline for pCTR/pCVRTrain LightGBM/XGBoost and calibrate predicted probabilitiesCompare PR-AUC/log loss/calibration, slices and uncertaintyExplain feature importance without treating it as causality

04 · Learning to rank

Week 7–8

Explain pairwise/listwise objectives and LambdaMARTCompute NDCG@K/MAP and recall@K on held-out queriesCompare retrieval recall vs reranker effectivenessStudy position bias and click-label selection bias

05 · Neural ranking

Week 9–10

Implement a PyTorch MLP from scratch for CTR/CVR predictionExperiment with categorical embeddings and shared-bottom multitask headsDescribe two-tower candidate retrieval vs cross-encoder rerankingProfile training/inference and compare to GBDT

06 · Auctions & bidding

Week 11–12

Formulate expected value, target ROAS and constrained bid decisionsStudy first/second-price and sponsored ad auction assumptionsCompare heuristic bid control to optimization-based policiesSimulate budget pacing and analyze failure modes

07 · Offline → online evaluation

Week 13–14

Define leakage-safe replay evaluation and model baselinesUnderstand off-policy evaluation assumptions and propensity weightingDesign A/B test guardrails, power and sequential monitoring caveatsCompare advertiser ROAS with platform/user metrics

08 · Serving & MLOps

Week 15–16

Implement batch feature pipeline with versioned artifactsBuild REST ranking endpoint (FastAPI) and basic Docker setupAdd latency monitoring, drift checks and rollback planDocument security, observability and retraining strategy

Weekly journal

Record one explanation, one experiment, and one unresolved question. Small entries count.