A hands-on exercise for packaging, serving, testing, monitoring, and rolling back a classification model with Python
Author
Complex Data Insights
Published
Aug 2026
COMPLEX DATA INSIGHTS
CDI Practicals
Hands-on exercises and step-by-step tools for doing, applying, and building evidence.
Model Deployment Practical
A hands-on exercise for packaging, serving, testing, monitoring, and rolling back a classification model with Python
AuthorComplex Data Insights
PublishedAug 2026
ID CDI-DATASCIENCE-PRACTICAL-010
Type Practical Product
Audience Students, researchers, analysts, professionals, mentors, and learners developing practical data science skills
Theme Responsible deployment turns a validated model into a controlled, observable, reversible service with explicit contracts and evidence
How to Use This Practical
Work through the practical in order. Each stage produces an artifact needed by the next stage. Run the code in a clean environment, read the interpretation after each output, and adapt the contracts and decision rules to your own project.
This practical uses a small public dataset and a fitted scikit-learn pipeline so that the complete workflow can run locally. The objective is not to create a production medical device. It is to learn the engineering and governance habits that make deployment defensible.
Browser-first workflow
If you are reading the rendered HTML, expand folded examples only when you need an implementation pattern. The main workflow contains everything required to complete the practical.
Recommended Approach
Pass 1: Define the release
Clarify intended use, ownership, the input/output contract, acceptance criteria, and rollback conditions before packaging anything.
Pass 2: Package and test
Create a versioned model bundle, validate requests, test prediction behavior, and verify that saved artifacts reproduce the expected result.
Pass 3: Serve and observe
Define the service boundary, prediction log, service indicators, and drift checks. Treat monitoring as part of deployment rather than a later add-on.
Pass 4: Rehearse recovery and communicate
Exercise a rollback decision, assemble release evidence, and state what the deployment may and may not be used for.
Practical Overview
A model is not deployed merely because it has been serialized. Deployment requires an explicit interface, reproducible packaging, controlled release, observable behavior, and a safe route back when assumptions fail.
In this practical you will turn a fitted binary-classification pipeline into a versioned release candidate. You will validate payloads, make local and batch predictions, generate a minimal API application, test success and failure paths, calculate operational and drift signals, rehearse rollback, and produce a deployment report.
The model predicts the demonstration target from numeric measurements in the scikit-learn breast-cancer dataset. The positive class used here is 1, corresponding to the dataset’s benign label. This choice is purely educational.
Learning exercise, not an operational clinical model
The dataset and outputs are used only to demonstrate deployment practice. Do not use this model for diagnosis, treatment, triage, or any decision affecting a person.
Learning Outcomes
By the end of this practical, you should be able to:
translate a model-use statement into deployment acceptance criteria;
define machine-readable input and output contracts;
package a complete preprocessing-and-model pipeline with metadata;
reject missing, extra, non-numeric, and implausible inputs;
implement deterministic local and batch prediction paths;
specify health and prediction endpoints for an API service;
test valid and invalid requests before release;
record privacy-conscious prediction events;
monitor service behavior, data quality, and population drift;
This practical requires Python 3.10 or later, Quarto, and these Python packages:
python-m pip install -r requirements.txt
The rendered exercise executes only the core data-science dependencies. FastAPI and Uvicorn are included in the generated service requirements for learners who choose to launch the optional local API.
After creating the environment, the complete workflow can also be run outside Quarto. This is the recommended verification path before rendering because it also executes API-level success and failure tests:
bash scripts/bash/run-workflow.sh
The command generates evidence locally; it does not deploy a service or approve the release. See README.md for the full setup, inspection, rendering, and manual packaging sequence.
Relationship to Practical 009
Practical 009 focused on building and selecting a complete model pipeline. Practical 010 begins at the handoff between model development and operations.
In a real project, copy the approved pipeline, feature dictionary, selection record, validation results, and model card from the previous practical. Here, the first executable stage recreates a compact validated pipeline so the deployment workflow is fully reproducible and does not depend on files outside this repository.
The deployment team should not silently change features, preprocessing, thresholds, or class definitions. Any such change creates a new model candidate that must return to model validation.
Project Setup
Create the folders, import the libraries, and fix the random seed. The paths are relative to the practical root so the same code works when the repository is moved.
The deployment brief connects technical implementation to a bounded use case. It also turns vague expectations into release gates.
Show code
deployment_brief = {"release_id": RELEASE_ID,"owner": "Complex Data Insights learning team","service_purpose": "Demonstrate controlled deployment of a binary classifier","intended_users": ["learners", "reviewers", "technical mentors"],"permitted_use": "Education, portfolio evidence, and local technical testing","prohibited_use": "Clinical or other real-world decisions affecting people","prediction_mode": ["single request", "batch"],"positive_class": {"value": 1, "meaning": "benign in the source dataset"},"decision_threshold": DECISION_THRESHOLD,"release_gates": {"minimum_holdout_roc_auc": 0.95,"contract_tests_must_pass": True,"artifact_reload_must_match": True,"maximum_missing_rate": 0.05,"maximum_population_stability_index": 0.25, },"rollback_triggers": ["schema or prediction failure rate exceeds 5% in a review window","p95 latency exceeds 250 ms in a review window","PSI exceeds 0.25 for any required feature","a critical safety, privacy, or integrity defect is confirmed", ],}brief_path = PATHS["config"] /"deployment-brief.json"brief_path.write_text(json.dumps(deployment_brief, indent=2))pd.DataFrame( [{"field": key, "value": json.dumps(value) ifisinstance(value, (dict, list)) else value}for key, value in deployment_brief.items()]).head(8)
field
value
0
release_id
model-010-v1
1
owner
Complex Data Insights learning team
2
service_purpose
Demonstrate controlled deployment of a binary classifier
3
intended_users
["learners", "reviewers", "technical mentors"]
4
permitted_use
Education, portfolio evidence, and local technical testing
5
prohibited_use
Clinical or other real-world decisions affecting people
6
prediction_mode
["single request", "batch"]
7
positive_class
{"value": 1, "meaning": "benign in the source dataset"}
Release gate versus monitoring alert
A release gate blocks promotion before deployment. A monitoring alert signals that a running release may need investigation or rollback. The same metric can have different thresholds for these two purposes, but both thresholds should be set before results are observed.
Prepare a Reproducible Model Handoff
Select a stable subset of numeric features, create a stratified holdout set, and fit a complete pipeline. The preprocessing objects remain inside the pipeline so inference applies exactly the same transformations as training.
The holdout score is a release-candidate check, not evidence of clinical validity. Deployment readiness also depends on contract, reliability, privacy, monitoring, and recovery evidence.
Define the Input and Output Contracts
What is a model contract?
A model contract is a set of rules for communicating with the model. It explains what information you must send and what the model promises to return.
Think of it like an order form:
The input contract specifies which boxes must be completed and what kind of information belongs in each box.
The output contract specifies what you receive after submitting a valid form.
An incomplete or invalid form is rejected before it reaches the model.
Valid request → accepted → prediction returned
Missing measurement → rejected
Text instead of a number → rejected
Unknown measurement → rejected
Implausible value → rejected
The contract helps the model and the system using it speak the same language. It catches input problems before they become unreliable predictions.
The input contract fixes feature names, types, nullability, and plausible demonstration ranges derived from the training data. Production ranges should come from domain knowledge and data-owner agreement, not automatically from observed extrema.
Save the fitted pipeline and metadata separately. A SHA-256 checksum detects accidental or unauthorized artifact changes; it does not replace access control or signed releases.
Reload the artifact in a fresh object, verify its checksum, and confirm that its probabilities match those generated before serialization.
Load only trusted model artifacts
joblib.load() can execute code embedded in a serialized file. A checksum can detect changed bytes, but it does not prove that the original artifact was safe. Load only artifacts produced through a trusted and controlled release process.
All artifact integrity and validation gates passed.
Implement Request Validation
Validation must fail clearly before the model receives malformed data. The function below rejects non-object payloads, missing and extra fields, booleans, non-numeric values, non-finite values, and values outside the agreed bounds.
Show code
def validate_payload(payload, schema=input_schema):ifnotisinstance(payload, dict):raiseValueError("Payload must be a JSON object.") required =set(schema["required"]) received =set(payload) missing =sorted(required - received) extra =sorted(received - required)if missing:raiseValueError(f"Missing required fields: {missing}")if extra:raiseValueError(f"Unexpected fields: {extra}") validated = {}for feature in schema["required"]: value = payload[feature]ifisinstance(value, bool) ornotisinstance(value, (int, float, np.integer, np.floating)):raiseValueError(f"{feature!r} must be numeric.") value =float(value)ifnot np.isfinite(value):raiseValueError(f"{feature!r} must be finite.") rules = schema["properties"][feature]if value < rules["minimum"] or value > rules["maximum"]:raiseValueError(f"{feature!r}={value} is outside the accepted range "f"[{rules['minimum']:.3f}, {rules['maximum']:.3f}]." ) validated[feature] = valuereturn validated
Create a Deterministic Prediction Function
The prediction function returns both the probability and the thresholded class. Returning the release identifier makes downstream logs and investigations traceable.
Show code
def validate_response(response, schema=output_schema): required =set(schema["required"]) received =set(response)if received != required:raiseValueError(f"Response fields do not match the contract: "f"missing={sorted(required - received)}; extra={sorted(received - required)}" )ifnotisinstance(response["release_id"], str):raiseValueError("release_id must be a string")ifisinstance(response["prediction"], bool) or response["prediction"] notin (0, 1):raiseValueError("prediction must be integer 0 or 1")for field in ["probability", "threshold"]: value = response[field]ifisinstance(value, bool) ornotisinstance(value, (int, float)) ornot0<= value <=1:raiseValueError(f"{field} must be numeric and between 0 and 1")return responsedef predict_one(payload, model=reloaded_pipeline, threshold=DECISION_THRESHOLD): validated = validate_payload(payload) frame = pd.DataFrame([validated], columns=feature_names) probability =float(model.predict_proba(frame)[0, 1]) response = {"release_id": RELEASE_ID,"prediction": int(probability >= threshold),"probability": round(probability, 6),"threshold": float(threshold), }return validate_response(response)example_request = {feature: float(X_holdout.iloc[0][feature]) for feature in feature_names}example_response = predict_one(example_request)(PATHS["outputs"] /"example-request.json").write_text(json.dumps(example_request, indent=2))(PATHS["outputs"] /"example-response.json").write_text(json.dumps(example_response, indent=2))example_response
'mean perimeter'=283.75 is outside the accepted range [21.895, 282.750].
Show code
assert contract_results["expectation_met"].all(), "One or more contract tests failed"print(f"Contract tests passed: {contract_results['expectation_met'].sum()}/{len(contract_results)}")
Contract tests passed: 5/5
Folded example: add a boundary-value test
Boundary tests are especially useful for numeric contracts. The following pattern checks that the declared maximum is accepted while a value immediately above it is rejected.
feature = feature_names[0]at_limit =dict(example_request)at_limit[feature] = input_schema["properties"][feature]["maximum"]predict_one(at_limit) # expected to succeedabove_limit =dict(at_limit)above_limit[feature] +=0.001predict_one(above_limit) # expected to raise ValueError
Run and Export Batch Predictions
Batch inference should reuse the same row-level contract. Include a request identifier supplied by the calling system, but avoid copying direct identifiers or raw feature values into general-purpose logs.
Show code
batch_input = X_holdout.head(12).reset_index(drop=True)batch_input.insert(0, "request_id", [f"demo-{i:03d}"for i inrange(len(batch_input))])batch_rows = []for record in batch_input.to_dict(orient="records"): request_id = record.pop("request_id") batch_rows.append({"request_id": request_id, **predict_one(record)})batch_predictions = pd.DataFrame(batch_rows)batch_predictions.to_csv(PATHS["outputs"] /"batch-predictions.csv", index=False)batch_predictions.head()
request_id
release_id
prediction
probability
threshold
0
demo-000
model-010-v1
1
0.985640
0.5
1
demo-001
model-010-v1
1
0.723189
0.5
2
demo-002
model-010-v1
0
0.089121
0.5
3
demo-003
model-010-v1
0
0.038742
0.5
4
demo-004
model-010-v1
1
0.997639
0.5
Generate the API Service Boundary
The service file below is a minimal FastAPI adapter around the same artifact and contract. It exposes /health for release identity and /predict for inference. The generated file is not launched during document rendering; contract behavior has already been executed directly in the notebook kernel.
Show code
app_source ='''from pathlib import Pathimport jsonimport joblibimport numpy as npimport pandas as pdfrom fastapi import FastAPI, HTTPExceptionROOT = Path(__file__).resolve().parents[1]RELEASE_ID = "model-010-v1"MODEL_PATH = ROOT / "artifacts" / f"{RELEASE_ID}.joblib"SCHEMA_PATH = ROOT / "config" / "input-schema.json"METADATA_PATH = ROOT / "artifacts" / f"{RELEASE_ID}.metadata.json"model = joblib.load(MODEL_PATH)schema = json.loads(SCHEMA_PATH.read_text())metadata = json.loads(METADATA_PATH.read_text())features = metadata["features"]threshold = float(metadata["threshold"])app = FastAPI(title="CDI Model 010 demonstration service", version=metadata["version"])def validate(payload): missing = sorted(set(features) - set(payload)) extra = sorted(set(payload) - set(features)) if missing or extra: raise ValueError(f"missing={missing}; extra={extra}") clean = {} for feature in features: value = payload[feature] if isinstance(value, bool) or not isinstance(value, (int, float)): raise ValueError(f"{feature} must be numeric") value = float(value) rules = schema["properties"][feature] if not np.isfinite(value) or not rules["minimum"] <= value <= rules["maximum"]: raise ValueError(f"{feature} is outside the accepted range") clean[feature] = value return clean@app.get("/health")def health(): return {"status": "ok", "release_id": RELEASE_ID}@app.post("/predict")def predict(payload: dict): try: clean = validate(payload) except ValueError as error: raise HTTPException(status_code=422, detail=str(error)) from error probability = float(model.predict_proba(pd.DataFrame([clean], columns=features))[0, 1]) return { "release_id": RELEASE_ID, "prediction": int(probability >= threshold), "probability": round(probability, 6), "threshold": threshold, }'''requirements ="""fastapi>=0.115,<1joblib>=1.4,<2numpy>=2.0,<3pandas>=2.2,<3scikit-learn>=1.5,<2uvicorn[standard]>=0.30,<1"""(PATHS["service"] /"app.py").write_text(app_source)(PATHS["service"] /"requirements.txt").write_text(requirements)compile(app_source, str(PATHS["service"] /"app.py"), "exec")print("Generated service/app.py and verified its Python syntax.")
Generated service/app.py and verified its Python syntax.
Folded example: launch and query the local API
From the repository root, install the service dependencies and start one local worker:
Send the contents of outputs/example-request.json to the prediction endpoint:
curl-X POST http://127.0.0.1:8000/predict \-H"Content-Type: application/json"\--data @outputs/example-request.json
Define Privacy-Conscious Prediction Logging
Operational logs should support debugging and monitoring without becoming an uncontrolled copy of sensitive inputs. This demonstration records a one-way request fingerprint, release identity, timing, status, predicted class, and probability. In a sensitive system, decide whether even prediction values are appropriate to retain.
Show code
def request_fingerprint(payload): canonical = json.dumps(payload, sort_keys=True, separators=(",", ":"))return hashlib.sha256(canonical.encode()).hexdigest()[:16]def predict_with_event(payload, request_id): started = time.perf_counter() event = {"timestamp_utc": datetime.now(timezone.utc).isoformat(),"request_id": request_id,"request_fingerprint": request_fingerprint(payload),"release_id": RELEASE_ID, }try: result = predict_one(payload) event.update({"status": "success","prediction": result["prediction"],"probability": result["probability"],"error_type": None, })exceptExceptionas error: result =None event.update({"status": "failure","prediction": None,"probability": None,"error_type": type(error).__name__, }) event["latency_ms"] =round((time.perf_counter() - started) *1000, 3)return result, eventevent_rows = []for index, record in X_holdout.head(40).reset_index(drop=True).iterrows(): _, event = predict_with_event(record.to_dict(), f"window-{index:03d}") event_rows.append(event)# Add one invalid request to exercise failure monitoring._, invalid_event = predict_with_event(missing_payload, "window-invalid-001")event_rows.append(invalid_event)event_log_path = PATHS["monitoring"] /"prediction-events.jsonl"event_log_path.write_text("\n".join(json.dumps(row) for row in event_rows) +"\n")events = pd.DataFrame(event_rows)events.head()
timestamp_utc
request_id
request_fingerprint
release_id
status
prediction
probability
error_type
latency_ms
0
2026-08-14T11:04:12.431586+00:00
window-000
09bc6dd6bec85bfa
model-010-v1
success
1.0
0.985640
None
1.653
1
2026-08-14T11:04:12.433374+00:00
window-001
0d56e5c859646e2c
model-010-v1
success
1.0
0.723189
None
1.138
2
2026-08-14T11:04:12.434546+00:00
window-002
a36cded479386475
model-010-v1
success
0.0
0.089121
None
0.632
3
2026-08-14T11:04:12.435206+00:00
window-003
2e54956733c0e4a0
model-010-v1
success
0.0
0.038742
None
0.567
4
2026-08-14T11:04:12.435798+00:00
window-004
6508993fed952a5f
model-010-v1
success
1.0
0.997639
None
0.724
Monitor Service-Level Indicators
Availability, failure rate, and latency describe whether the service is operating. They do not tell you whether the model remains useful or fair. Monitor service, data, prediction, outcome, and subgroup behavior as distinct layers.
The single deliberate invalid request remains below the > 5% failure trigger in this 41-request demonstration window. In practice, choose a minimum window size so one request cannot create or suppress an unstable alert.
Build a Reference Data Profile
Store aggregate reference statistics instead of the complete training dataset when that is sufficient. The profile below records quantiles used to create fixed monitoring bins.
Show code
profile_rows = []for feature in feature_names: values = X_train[feature].astype(float) quantiles = values.quantile([0, 0.1, 0.25, 0.5, 0.75, 0.9, 1.0]).to_dict() profile_rows.append({"feature": feature,"missing_rate": float(values.isna().mean()),"mean": float(values.mean()),"std": float(values.std()),**{f"q{int(q *100):02d}": float(value) for q, value in quantiles.items()}, })reference_profile = pd.DataFrame(profile_rows)reference_profile.to_csv(PATHS["monitoring"] /"reference-profile.csv", index=False)reference_profile.round(3)
feature
missing_rate
mean
std
q00
q10
q25
q50
q75
q90
q100
0
mean radius
0.0
14.121
3.613
6.981
10.220
11.650
13.270
15.780
19.574
28.110
1
mean texture
0.0
19.341
4.392
10.380
14.098
16.175
18.800
21.710
25.172
39.280
2
mean perimeter
0.0
91.933
24.961
43.790
65.196
74.720
85.790
103.900
129.880
188.500
3
mean area
0.0
656.740
362.958
143.500
320.960
416.150
545.200
782.650
1204.800
2501.000
4
mean smoothness
0.0
0.096
0.014
0.053
0.080
0.086
0.095
0.105
0.114
0.145
5
worst radius
0.0
16.290
4.985
7.930
11.160
12.980
14.900
18.775
23.726
36.040
6
worst texture
0.0
25.771
6.246
12.870
17.810
21.140
25.230
29.990
33.958
49.540
7
worst perimeter
0.0
107.416
34.652
50.410
71.850
84.135
97.170
125.250
159.400
251.200
8
worst area
0.0
888.260
592.667
185.200
377.980
513.500
677.900
1077.000
1718.000
4254.000
9
worst concavity
0.0
0.271
0.212
0.000
0.041
0.114
0.228
0.380
0.563
1.252
Calculate Population Stability Index
Population Stability Index (PSI) compares proportions in fixed reference bins. It is a screening signal, not proof that model performance changed. Small monitoring windows can be noisy, and thresholds must reflect the domain and consequences.
Population Stability Index by feature for the demonstration monitoring window.
Interpret Monitoring Signals Carefully
Use the signals as prompts for investigation:
a schema failure may indicate an upstream interface change;
rising latency may indicate resource pressure or dependency failure;
feature drift may reflect a real population change, a measurement change, or a pipeline defect;
prediction drift can occur even when individual feature PSI values remain modest;
stable inputs do not guarantee stable outcomes; and
performance monitoring requires delayed ground truth collected under an approved process.
Do not automatically retrain or roll back from a single noisy statistic. Confirm data lineage, sample size, timing, subgroup effects, operational incidents, and outcome evidence.
Folded example: delayed-outcome monitoring
When approved labels become available, join them to predictions using a controlled identifier and calculate metrics by time window and relevant subgroup.
Before implementing this pattern, document label delay, missing outcomes, intervention effects, consent, retention, access, and whether subgroup evaluation is statistically meaningful.
Define Alert Ownership and Response
An alert without an owner and action is noise. The response matrix below distinguishes investigation, escalation, and immediate rollback.
Show code
response_matrix = pd.DataFrame([ ["schema failure rate", "> 5%", "service owner", "inspect upstream payload changes", "pause traffic if sustained"], ["p95 latency", "> 250 ms", "platform owner", "inspect capacity and dependencies", "scale or revert release"], ["feature PSI", "> 0.25", "model owner", "validate lineage and population shift", "hold retraining decision for review"], ["critical safety/privacy defect", "confirmed", "incident lead", "contain and notify", "immediate rollback or shutdown"], ["outcome performance", "below approved floor", "model-risk owner", "review labels and subgroup results", "rollback or suspend"],], columns=["signal", "trigger", "owner", "first_action", "possible_decision"])response_matrix
signal
trigger
owner
first_action
possible_decision
0
schema failure rate
> 5%
service owner
inspect upstream payload changes
pause traffic if sustained
1
p95 latency
> 250 ms
platform owner
inspect capacity and dependencies
scale or revert release
2
feature PSI
> 0.25
model owner
validate lineage and population shift
hold retraining decision for review
3
critical safety/privacy defect
confirmed
incident lead
contain and notify
immediate rollback or shutdown
4
outcome performance
below approved floor
model-risk owner
review labels and subgroup results
rollback or suspend
Rehearse a Rollback Decision
Rollback must be possible before it is needed. A real rehearsal verifies artifact availability, configuration compatibility, data migration implications, routing, smoke tests, and authority to act.
This demonstration treats the intentionally shifted PSI signal as a reason to open an investigation. It does not automatically replace the release.
[previous artifact location defined, schema compatibility must be verified, traffic switch authority assigned, post-rollback smoke test specified]
True
A first release needs a safe fallback
This practical names a hypothetical previous version to make the rollback contract explicit. If no earlier safe model exists, the fallback may be a rules-based process, manual review, or service shutdown. “Keep the new model running” is not a fallback.
Create the Release Manifest
The release manifest gathers the identity, checksums, contracts, tests, monitoring configuration, and decision state needed for review.
The manual-review gate is intentionally False. Rendering the practical creates evidence for review; it does not authorize release.
Create the Model Deployment Report
The report is saved as a Markdown file. Expand the preview below only when you want to inspect its generated contents.
Preview the generated report
Show code
alerted_features = drift_summary.loc[drift_summary["psi_alert"], "feature"].tolist()report =f"""# Model Deployment Report## Release identity- Release: `{RELEASE_ID}`- Artifact: `{artifact_path.name}`- SHA-256: `{artifact_sha256}`- Framework: scikit-learn {sklearn.__version__}## Intended and prohibited useThis release candidate demonstrates controlled deployment of a binary classifier for education, portfolio evidence, and local technical testing. It must not be used for clinical or other real-world decisions affecting people.## Model handoff evidence- Required numeric features: {len(feature_names)}- Demonstration holdout rows: {len(X_holdout)}- Demonstration holdout ROC AUC: {holdout_auc:.4f}- Decision threshold: {DECISION_THRESHOLD:.2f}- Reloaded predictions matched pre-serialization predictions: {bool(integrity_checks.loc[integrity_checks['check'].eq('reloaded probabilities match'), 'passed'].iloc[0])}## Contract and service evidence- Contract tests passed: {int(contract_results['expectation_met'].sum())}/{len(contract_results)}- Single-request and batch inference paths executed successfully.- The generated FastAPI service passed Python syntax compilation.- Raw feature values are excluded from the demonstration prediction-event log.## Monitoring evidence- Requests in demonstration window: {int(service_window.loc[0, 'requests'])}- Failure rate: {service_window.loc[0, 'failure_rate']:.2%}- p95 local inference latency: {service_window.loc[0, 'p95_latency_ms']:.3f} ms- Features above PSI alert threshold: {alerted_features if alerted_features else'none'}The drift result is intentionally simulated to exercise investigation and rollback reasoning. It is not evidence of a real production shift.## Recovery readinessThe rollback decision logic was executed. The current recommendation is: **{recommended_action}**. A real release must verify the previous safe artifact or an alternative fallback, routing authority, schema compatibility, and a post-rollback smoke test.## Release decisionAutomated artifact and contract gates passed. The candidate remains **pending manual review**. Promotion requires an accountable reviewer to evaluate the evidence, resolve alerts, confirm operational ownership, and update the release manifest."""report_path = PATHS["reports"] /"model-deployment-report.md"report_path.write_text(report)display(Markdown(report))
This release candidate demonstrates controlled deployment of a binary classifier for education, portfolio evidence, and local technical testing. It must not be used for clinical or other real-world decisions affecting people.
Single-request and batch inference paths executed successfully.
The generated FastAPI service passed Python syntax compilation.
Raw feature values are excluded from the demonstration prediction-event log.
Monitoring evidence
Requests in demonstration window: 41
Failure rate: 2.44%
p95 local inference latency: 1.281 ms
Features above PSI alert threshold: [‘mean texture’]
The drift result is intentionally simulated to exercise investigation and rollback reasoning. It is not evidence of a real production shift.
Recovery readiness
The rollback decision logic was executed. The current recommendation is: investigate drift; keep rollback ready. A real release must verify the previous safe artifact or an alternative fallback, routing authority, schema compatibility, and a post-rollback smoke test.
Release decision
Automated artifact and contract gates passed. The candidate remains pending manual review. Promotion requires an accountable reviewer to evaluate the evidence, resolve alerts, confirm operational ownership, and update the release manifest.
Export and Verify Results
Verify that every expected artifact exists, is non-empty, and contains the expected release identity where applicable.
Use the completed repository to demonstrate that you can move beyond model training. Strong evidence includes:
a clear release brief and bounded model-use statement;
versioned, fingerprinted artifacts with reproducibility checks;
explicit input and output contracts;
executable success and failure tests;
a thin service boundary that reuses the complete pipeline;
privacy-conscious operational logging;
separate service, data, prediction, and outcome monitoring plans;
predeclared alert ownership and response actions; and
a rollback rehearsal with an honest fallback.
Do not present a rendered page alone as proof of production readiness. Explain which parts were executed locally, which controls are demonstrations, and what infrastructure or governance would still be required.
Quality Checks
Before requesting release approval, confirm that:
the serialized object contains preprocessing and model steps;
feature order comes from versioned metadata rather than caller order;
missing, extra, invalid, and implausible values are rejected;
class meaning and threshold are explicit;
model and schema versions are traceable in responses and logs;
tests cover both valid and invalid paths;
logs exclude unnecessary raw or identifying data;
monitoring windows and minimum sample sizes are defined;
drift alerts trigger investigation rather than blind retraining;
outcome and subgroup monitoring plans address label delay and missingness;
a safe fallback and accountable rollback authority exist; and
manual approval is recorded separately from automated test success.
Reflection and Next Steps
Write brief answers to these questions:
Which assumption in the input contract is most likely to fail first?
What information would you need to distinguish real population drift from a broken upstream pipeline?
Which logs are necessary for incident investigation, and which would create avoidable privacy risk?
What minimum evidence would justify continuing operation after a PSI alert?
If ground-truth outcomes arrive after 90 days, what early signals can you monitor without pretending they measure performance?
Who has authority to approve, pause, roll back, and retire the model?
What is the safest fallback if no earlier model version exists?
For a stronger extension, containerize the service, add automated API tests, generate a software bill of materials, sign the artifact, run load and security tests, and deploy to a non-production environment with controlled traffic.
Completion Checklist
Practical Complete
You have built a complete model-deployment evidence package: a reproducible artifact, explicit contracts, validated inference paths, an API boundary, operational logs, monitoring signals, response rules, rollback evidence, and a release manifest.
The final lesson is simple: deployment is not the moment a model goes live. It is the controlled process that makes the model identifiable, testable, observable, accountable, and reversible throughout its operational life.