NannyML
Open-source library estimating post-deployment model performance.
About
NannyML is an open-source Python library for post-deployment data science: it estimates model performance without access to ground-truth targets, detects data drift, and intelligently links drift alerts back to changes in model performance. Built for data scientists, it is model-agnostic and supports tabular classification and regression use cases.
Its novel algorithms include confidence-based performance estimation (CBPE) and direct loss estimation (DLE), plus PCA-based multivariate drift detection that catches subtle structural changes univariate methods miss. Business value estimation ties model performance to monetary outcomes, and interactive visualizations make the whole loop explorable. The library is a staple of MLOps courses and a primary tool for teams fighting silent model failure.
What it can do
- Performance estimation without ground truth via CBPE and DLE
- Univariate and PCA-based multivariate drift detection
- Business value estimation tied to monetary outcomes
- Data quality checks for missing and unseen values
- Interactive visualizations for root cause analysis
Setup
pip install nannyml; provide reference and analysis datasets to start monitoring.
Prerequisites
Python environment; reference (test) and production (analysis) datasets.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 124; Trending score 76/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.