Exploratory Data Analysis Practical

A hands-on exercise for investigating distributions, relationships, outliers, and patterns in a cleaned dataset with Python

Author

Complex Data Insights

Published

Aug 2026

COMPLEX DATA INSIGHTS
CDI Practicals

Hands-on exercises and step-by-step tools for doing, applying, and building evidence.

Exploratory Data Analysis Practical

A hands-on exercise for investigating distributions, relationships, outliers, and patterns in a cleaned dataset with Python

AuthorComplex Data Insights
PublishedAug 2026
ID CDI-DATASCIENCE-PRACTICAL-002
Type Practical Product
Audience Students, researchers, analysts, professionals, mentors, and learners developing practical data science skills
Theme Responsible exploratory analysis turns a cleaned dataset into questions, patterns, evidence, and clearly stated limitations

How to Use This Practical

This CDI Practical is designed to be completed actively rather than read passively. You will investigate a cleaned service-delivery dataset and build a reproducible exploratory data analysis (EDA) that moves from orientation to interpretation.

Work through the practical in order. Run each code block, inspect its output, and write short observations before moving on. Keep unusual values visible until you have evidence that they are errors. Exploratory findings are signals for further investigation, not proof of causes.

You are encouraged to:

  • begin with explicit exploratory questions;
  • connect each table or plot to a question;
  • examine missing values and outliers without automatically removing them;
  • distinguish description, interpretation, and explanation;
  • preserve the cleaned input dataset;
  • record limitations and possible next analyses;
  • adapt the workflow to a dataset relevant to your own work.

This practical is delivered as an online HTML page. Keep the practical open beside VS Code, work through it in order, and run or adapt each Python code block in your own environment. You may also use JupyterLab, Positron, Google Colab, or another environment that supports Python notebooks or scripts.

Practical Overview

Exploratory data analysis is the disciplined process of learning what a dataset can—and cannot—show before formal modelling or decision-making. It combines questions, summaries, visualizations, checks, and careful interpretation.

In this practical, one row represents one service record. You will examine who used services, where and when services were delivered, how satisfaction varied, which records remain incomplete, and which observations deserve review.

The workflow uses Python with pandas, numpy, matplotlib, and seaborn. It follows four linked stages:

  1. orient yourself to the dataset and define questions;
  2. explore individual variables and missingness;
  3. examine group differences, relationships, and outliers;
  4. communicate findings, uncertainty, and limitations.

Learning Outcomes

By completing this practical, you will be able to:

  • formulate useful exploratory questions;
  • confirm that a cleaned dataset is ready for analysis;
  • distinguish numerical, categorical, date, identifier, and text variables;
  • produce reproducible descriptive summaries and visualizations;
  • investigate missingness and potential outliers;
  • compare groups and examine relationships between variables;
  • avoid causal claims that the design cannot support;
  • export an exploratory evidence package;
  • present the work as a portfolio project.

What You Will Produce

Your completed practical will produce:

  • an exploratory analysis plan;
  • a dataset overview table;
  • numerical descriptive statistics;
  • categorical frequency summaries;
  • distribution and grouped-comparison plots;
  • numerical and categorical relationship plots;
  • missingness and outlier review tables;
  • an exploratory findings summary;
  • an interpretation and limitations statement;
  • portfolio-ready evidence.

Suggested output structure:

results/
├── figures/
│   ├── numerical-distributions.png
│   ├── categorical-patterns.png
│   ├── monthly-service-records.png
│   ├── age-outlier-review.png
│   ├── grouped-satisfaction.png
│   ├── age-satisfaction-relationship.png
│   ├── region-service-relationship.png
│   └── exploratory-visual-summary.png
└── tables/
    ├── dataset-overview.csv
    ├── numerical-summary.csv
    ├── categorical-summary.csv
    ├── temporal-summary.csv
    ├── missingness-summary.csv
    ├── outlier-review.csv
    ├── satisfaction-by-service.csv
    ├── region-service-percent.csv
    └── exploratory-findings.csv

Tools and Requirements

You need Python 3.10 or later and VS Code with the Python and Jupyter extensions. Run the commands below from the repository root to create a dedicated virtual environment for this practical.

Create the Virtual Environment

python3 -m venv .venv

Activate the Virtual Environment

On macOS or Linux:

source .venv/bin/activate

On Windows PowerShell:

.venv\Scripts\Activate.ps1

After activation, your terminal prompt should begin with (.venv).

Install the Required Packages

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Verify the Environment

python --version
python -c "import pandas, matplotlib, seaborn; print('Environment ready')"

When you return to the project later, activate .venv again before running any commands. To leave the environment, run deactivate.

The input for this practical is:

data/cleaned-service-data.csv

The project includes a deterministic generator for this synthetic input. You do not need Practical 001 to complete this product. If you use your own cleaned output from Practical 001, confirm that the column names and meanings match before running the code.

Step 1: Confirm the Starting Dataset

Begin inside the extracted project folder. The supplied project files already include:

data/cleaned-service-data.csv

Confirm that the file exists, then preview it without editing the source:

Show code
from pathlib import Path
import pandas as pd

preview_path = Path("data/cleaned-service-data.csv")
assert preview_path.exists(), "Confirm that data/cleaned-service-data.csv is present."
pd.read_csv(preview_path).head()
record_id participant_name region service_type age visit_date satisfaction_score follow_up_required contact_email requires_review
0 SR-0001 Synthetic Participant 001 North Outreach 40.0 2025-08-09 4.0 No synthetic001@example.invalid No
1 SR-0002 Synthetic Participant 002 East Consultation 48.0 2025-11-04 4.0 No synthetic002@example.invalid No
2 SR-0003 Synthetic Participant 003 Central Diagnostics 40.0 2025-09-14 4.0 No synthetic003@example.invalid No
3 SR-0004 Synthetic Participant 004 East Training NaN 2025-05-07 4.0 No synthetic004@example.invalid No
4 SR-0005 Synthetic Participant 005 West Consultation 28.0 2025-12-30 4.0 No synthetic005@example.invalid No

The project also includes a deterministic generator that can recreate the synthetic starting dataset when needed:

python scripts/python/generate-example-dataset.py

The generator uses a fixed random seed to create the same 240 fictional service records. It intentionally retains limited missingness, review flags, unequal groups, and a small number of unusual ages for investigation. Running it is optional and may replace the supplied file, so preserve any dataset you need before using it.

Reference results

The project files include author-generated tables, figures, and quality-control results. Use them to check your progress, but complete the guided analysis yourself so that you understand and can explain every analytical decision.

Step 2: Prepare the Guided Workflow

Open the project folder in VS Code and keep this practical open in your browser. Work through the remaining steps in order, run each code block in your Python environment, inspect its output, and record your own observations. The code will validate the dataset and create an evidence package under results/.

Project Structure

Open the extracted project folder in VS Code. Keep source data in data/ and write your outputs only to results/.

exploratory-data-analysis-practical/
├── assets/css/
├── data/
│   └── cleaned-service-data.csv
├── results/
│   ├── figures/
│   ├── quality-control/
│   └── tables/
├── scripts/
│   ├── bash/
│   └── python/
│       └── generate-example-dataset.py
├── README.md
└── requirements.txt

Step 3: Orient the Analysis

Dataset Orientation

The dataset contains service records with identifiers, participant details, service characteristics, dates, satisfaction scores, follow-up status, and a review flag. Names and email addresses are included only because the dataset is fictional. In real analysis, minimize access to direct identifiers and exclude them from exploratory outputs unless they are essential.

Variable Role in this analysis Intended meaning
record_id Identifier Unique service record
participant_name Direct identifier Fictional participant name
region Categorical Service-delivery region
service_type Categorical Type of service delivered
age Numerical Participant age in years
visit_date Date Date of service visit
satisfaction_score Numerical/ordinal Rating from 1 to 5
follow_up_required Categorical/binary Whether follow-up is required
contact_email Direct identifier Fictional contact detail
requires_review Categorical/binary Flag retained during cleaning

Define Exploratory Questions

Start with questions rather than charts. This prevents EDA from becoming an unstructured search for anything visually interesting.

Use these core questions:

  1. What does a typical service record look like?
  2. How are records distributed across regions and service types?
  3. What are the shapes and spreads of age and satisfaction score?
  4. Where does missingness remain, and is it concentrated in particular groups?
  5. Which observations are potential outliers and what context do they need?
  6. How does satisfaction vary across regions, service types, and follow-up status?
  7. Is age associated with satisfaction in this dataset?
  8. Which patterns are strong enough to motivate a later confirmatory analysis?

Exploratory Analysis Plan

For each question, record:

  • the question and why it matters;
  • the variables and unit of analysis;
  • the summary or visualization you will use;
  • the expected output;
  • one interpretation risk or limitation.

Add at least one question of your own before continuing.

Prepare the Analysis Environment

Run the following setup in VS Code as you work through the online practical.

Show code
from pathlib import Path
import sys

import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
Show code
pd.set_option("display.max_columns", 20)
pd.set_option("display.float_format", lambda value: f"{value:,.2f}")
sns.set_theme(style="whitegrid", context="notebook")

CDI_GREEN = "#2F6B4F"
CDI_GOLD = "#D7A84B"
CDI_BLUE = "#4878A8"
Show code
DATA_PATH = Path("data/cleaned-service-data.csv")
RESULTS_DIR = Path("results")
FIGURES_DIR = RESULTS_DIR / "figures"
TABLES_DIR = RESULTS_DIR / "tables"

for directory in [FIGURES_DIR, TABLES_DIR]:
    directory.mkdir(parents=True, exist_ok=True)
Show code
software_versions = pd.Series(
    {
        "python": sys.version.split()[0],
        "pandas": pd.__version__,
        "numpy": np.__version__,
        "matplotlib": matplotlib.__version__,
        "seaborn": sns.__version__,
    },
    name="version",
)
software_versions
python        3.12.6
pandas         2.3.3
numpy          2.5.2
matplotlib    3.11.1
seaborn       0.13.2
Name: version, dtype: object

Load the Cleaned Dataset

Load the source once and create an analysis copy. Do not overwrite the input file.

Show code
if not DATA_PATH.exists():
    raise FileNotFoundError(
        f"Expected {DATA_PATH}. Run the included generator or place a compatible cleaned dataset in data/."
    )

source_data = pd.read_csv(DATA_PATH)
df = source_data.copy()

df["visit_date"] = pd.to_datetime(df["visit_date"], errors="coerce")
for column in ["age", "satisfaction_score"]:
    df[column] = pd.to_numeric(df[column], errors="coerce")

df.head()
record_id participant_name region service_type age visit_date satisfaction_score follow_up_required contact_email requires_review
0 SR-0001 Synthetic Participant 001 North Outreach 40.00 2025-08-09 4.00 No synthetic001@example.invalid No
1 SR-0002 Synthetic Participant 002 East Consultation 48.00 2025-11-04 4.00 No synthetic002@example.invalid No
2 SR-0003 Synthetic Participant 003 Central Diagnostics 40.00 2025-09-14 4.00 No synthetic003@example.invalid No
3 SR-0004 Synthetic Participant 004 East Training NaN 2025-05-07 4.00 No synthetic004@example.invalid No
4 SR-0005 Synthetic Participant 005 West Consultation 28.00 2025-12-30 4.00 No synthetic005@example.invalid No

Confirm Analysis Readiness

Cleaning and EDA have different purposes, but EDA still begins with validation. Confirm shape, identifier uniqueness, expected columns, and ranges before interpreting results.

Show code
required_columns = {
    "record_id", "region", "service_type", "age", "visit_date",
    "satisfaction_score", "follow_up_required", "requires_review"
}
missing_columns = sorted(required_columns.difference(df.columns))
assert not missing_columns, f"Missing required columns: {missing_columns}"
Show code
readiness_checks = pd.Series(
    {
        "rows": len(df),
        "columns": df.shape[1],
        "duplicate_rows": int(df.duplicated().sum()),
        "missing_record_ids": int(df["record_id"].isna().sum()),
        "duplicate_record_ids": int(df["record_id"].duplicated().sum()),
        "invalid_visit_dates": int(df["visit_date"].isna().sum() - source_data["visit_date"].isna().sum()),
        "invalid_ages": int((~df["age"].between(0, 120) & df["age"].notna()).sum()),
        "invalid_satisfaction_scores": int(
            (~df["satisfaction_score"].between(1, 5) & df["satisfaction_score"].notna()).sum()
        ),
    },
    name="value",
)
readiness_checks
rows                           240
columns                         10
duplicate_rows                   0
missing_record_ids               0
duplicate_record_ids             0
invalid_visit_dates              0
invalid_ages                     0
invalid_satisfaction_scores      0
Name: value, dtype: int64
Stop if a structural check fails

Investigate missing columns, duplicate identifiers, invalid dates, or impossible values before continuing. Do not silently repair the dataset inside the EDA workflow; document the issue and return to the cleaning process when necessary.

Step 4: Profile the Dataset

Understand Variable Types

The same stored type can support different analytical roles. satisfaction_score, for example, is numeric but ordinal: the order is meaningful, while equal spacing between ratings should not be assumed automatically.

Show code
variable_roles = pd.DataFrame(
    {
        "role": ["identifier", "categorical", "categorical", "numerical", "date",
                 "ordinal", "categorical", "categorical"],
        "variable": ["record_id", "region", "service_type", "age", "visit_date",
                     "satisfaction_score", "follow_up_required", "requires_review"],
    }
)
variable_roles
role variable
0 identifier record_id
1 categorical region
2 categorical service_type
3 numerical age
4 date visit_date
5 ordinal satisfaction_score
6 categorical follow_up_required
7 categorical requires_review

Create a Dataset Overview

Show code
dataset_overview = pd.DataFrame(
    {
        "variable": df.columns,
        "stored_type": df.dtypes.astype(str).values,
        "non_missing": df.notna().sum().values,
        "missing": df.isna().sum().values,
        "missing_percent": (df.isna().mean().mul(100).round(1)).values,
        "unique_values": df.nunique(dropna=True).values,
    }
)
dataset_overview
variable stored_type non_missing missing missing_percent unique_values
0 record_id object 240 0 0.00 240
1 participant_name object 240 0 0.00 240
2 region object 240 0 0.00 4
3 service_type object 240 0 0.00 4
4 age float64 231 9 3.80 51
5 visit_date datetime64[ns] 240 0 0.00 178
6 satisfaction_score float64 233 7 2.90 5
7 follow_up_required object 240 0 0.00 2
8 contact_email object 240 0 0.00 240
9 requires_review object 240 0 0.00 2
Show code
dataset_overview.to_csv(TABLES_DIR / "dataset-overview.csv", index=False)

Use the overview to identify variables with little variation, unexpected missingness, or a suspicious number of categories. A table is an analytical checkpoint, not merely a required output.

Step 5: Explore Individual Variables

Summarize Numerical Variables

Show code
numerical_columns = ["age", "satisfaction_score"]
numerical_summary = (
    df[numerical_columns]
    .describe(percentiles=[0.25, 0.5, 0.75])
    .T
    .rename_axis("variable")
    .reset_index()
)
numerical_summary["missing"] = df[numerical_columns].isna().sum().values
numerical_summary["iqr"] = numerical_summary["75%"] - numerical_summary["25%"]
numerical_summary
variable count mean std min 25% 50% 75% max missing iqr
0 age 231.00 39.26 12.98 18.00 29.00 38.00 48.00 82.00 9 19.00
1 satisfaction_score 233.00 3.63 0.88 1.00 3.00 4.00 4.00 5.00 7 1.00
Show code
numerical_summary.to_csv(TABLES_DIR / "numerical-summary.csv", index=False)

Compare the mean and median, then examine the range and interquartile range. Differences may indicate skewness, outliers, or a bounded/ordinal scale.

Examine Numerical Distributions

Show code
fig, axes = plt.subplots(1, 2, figsize=(12, 4.5))

sns.histplot(data=df, x="age", bins=12, kde=True, color=CDI_GREEN, ax=axes[0])
axes[0].set(title="Distribution of participant age", xlabel="Age (years)", ylabel="Records")

score_order = range(1, 6)
sns.countplot(data=df, x="satisfaction_score", order=score_order, color=CDI_GOLD, ax=axes[1])
axes[1].set(title="Distribution of satisfaction scores", xlabel="Satisfaction score", ylabel="Records")

fig.tight_layout()
fig.savefig(FIGURES_DIR / "numerical-distributions.png", dpi=300, bbox_inches="tight")
plt.show()

Describe shape, centre, spread, gaps, and unusual values. Do not use “normal” to mean “acceptable”; in statistics, normality has a specific meaning.

Summarize Categorical Variables

Show code
categorical_columns = ["region", "service_type", "follow_up_required", "requires_review"]
categorical_summaries = []

for column in categorical_columns:
    summary = (
        df[column]
        .fillna("Missing")
        .value_counts(dropna=False)
        .rename_axis("category")
        .reset_index(name="count")
    )
    summary.insert(0, "variable", column)
    summary["percent"] = summary["count"].div(len(df)).mul(100).round(1)
    categorical_summaries.append(summary)

categorical_summary = pd.concat(categorical_summaries, ignore_index=True)
categorical_summary
variable category count percent
0 region Central 72 30.00
1 region East 68 28.30
2 region North 51 21.20
3 region West 49 20.40
4 service_type Consultation 84 35.00
5 service_type Outreach 60 25.00
6 service_type Diagnostics 60 25.00
7 service_type Training 36 15.00
8 follow_up_required No 186 77.50
9 follow_up_required Yes 54 22.50
10 requires_review No 226 94.20
11 requires_review Yes 14 5.80
Show code
categorical_summary.to_csv(TABLES_DIR / "categorical-summary.csv", index=False)

Visualize Categorical Patterns

Show code
fig, axes = plt.subplots(1, 2, figsize=(13, 4.8))

sns.countplot(data=df, y="region", order=df["region"].value_counts().index,
              color=CDI_GREEN, ax=axes[0])
axes[0].set(title="Service records by region", xlabel="Records", ylabel="Region")

sns.countplot(data=df, y="service_type", order=df["service_type"].value_counts().index,
              color=CDI_BLUE, ax=axes[1])
axes[1].set(title="Service records by service type", xlabel="Records", ylabel="Service type")

fig.tight_layout()
fig.savefig(FIGURES_DIR / "categorical-patterns.png", dpi=300, bbox_inches="tight")
plt.show()

Counts describe the dataset, not necessarily the population. A region with more records may have more clients, better reporting, a larger service network, or a different sampling process.

Examine Time Coverage and Activity

Dates reveal the observation window, gaps, and changes in record volume. They do not by themselves explain why activity changed.

Show code
monthly_records = (
    df.dropna(subset=["visit_date"])
    .assign(month=lambda x: x["visit_date"].dt.to_period("M").dt.to_timestamp())
    .groupby("month")
    .size()
    .rename("records")
    .reset_index()
)

temporal_summary = pd.DataFrame(
    {
        "metric": ["first_visit", "last_visit", "valid_visit_dates", "months_observed"],
        "value": [
            df["visit_date"].min(),
            df["visit_date"].max(),
            df["visit_date"].notna().sum(),
            monthly_records["month"].nunique(),
        ],
    }
)
temporal_summary
metric value
0 first_visit 2025-01-01 00:00:00
1 last_visit 2025-12-31 00:00:00
2 valid_visit_dates 240
3 months_observed 12
Show code
fig, ax = plt.subplots(figsize=(10, 4.5))
sns.lineplot(data=monthly_records, x="month", y="records", marker="o", color=CDI_GREEN, ax=ax)
ax.set(title="Service records by month", xlabel="Visit month", ylabel="Records")
fig.autofmt_xdate()
fig.tight_layout()
fig.savefig(FIGURES_DIR / "monthly-service-records.png", dpi=300, bbox_inches="tight")
plt.show()

Show code
temporal_summary.to_csv(TABLES_DIR / "temporal-summary.csv", index=False)

Look for gaps, isolated peaks, and incomplete first or last months. Confirm operational explanations with external context rather than inferring them from the chart alone.

Step 6: Investigate Missingness and Outliers

Investigate Missingness

Missing values can be informative. First locate them, then ask whether they are associated with other observed variables.

Show code
missingness_summary = (
    df.isna().sum()
    .rename("missing_count")
    .to_frame()
    .assign(missing_percent=lambda x: x["missing_count"].div(len(df)).mul(100).round(1))
    .query("missing_count > 0")
    .rename_axis("variable")
    .reset_index()
)
missingness_summary
variable missing_count missing_percent
0 age 9 3.80
1 satisfaction_score 7 2.90
Show code
missing_by_region = (
    df.assign(
        age_missing=df["age"].isna(),
        satisfaction_missing=df["satisfaction_score"].isna(),
    )
    .groupby("region", dropna=False)[["age_missing", "satisfaction_missing"]]
    .mean()
    .mul(100)
    .round(1)
    .reset_index()
)
missing_by_region
region age_missing satisfaction_missing
0 Central 5.60 2.80
1 East 2.90 7.40
2 North 2.00 0.00
3 West 4.10 0.00
Show code
missingness_summary.to_csv(TABLES_DIR / "missingness-summary.csv", index=False)

Avoid filling missing values simply to make a plot work. Each analysis should state how missing observations were handled and how many records contributed.

Identify and Review Outliers

Use the interquartile-range rule as a screening tool, not a deletion rule.

Show code
age_non_missing = df["age"].dropna()
q1, q3 = age_non_missing.quantile([0.25, 0.75])
iqr = q3 - q1
lower_bound = q1 - 1.5 * iqr
upper_bound = q3 + 1.5 * iqr

outlier_review = df.loc[
    df["age"].notna() & ~df["age"].between(lower_bound, upper_bound),
    ["record_id", "region", "service_type", "age", "satisfaction_score", "requires_review"],
].copy()
outlier_review["screening_rule"] = f"age outside {lower_bound:.1f} to {upper_bound:.1f}"
outlier_review["decision"] = "Review; do not automatically remove"
outlier_review
record_id region service_type age satisfaction_score requires_review screening_rule decision
31 SR-0032 Central Outreach 79.00 3.00 Yes age outside 0.5 to 76.5 Review; do not automatically remove
174 SR-0175 North Consultation 82.00 4.00 Yes age outside 0.5 to 76.5 Review; do not automatically remove
Show code
fig, ax = plt.subplots(figsize=(9, 3.5))
sns.boxplot(data=df, x="age", color=CDI_GOLD, ax=ax)
ax.set(title="Age distribution and potential outliers", xlabel="Age (years)")
fig.tight_layout()
fig.savefig(FIGURES_DIR / "age-outlier-review.png", dpi=300, bbox_inches="tight")
plt.show()

Show code
outlier_review.to_csv(TABLES_DIR / "outlier-review.csv", index=False)

For every flagged record, ask whether the value is possible, consistent with the source, influential, and relevant to the analysis. A valid extreme value may be one of the most important observations in the dataset.

Step 7: Explore Group Differences and Relationships

Compare Groups

Group summaries should show sample size as well as centre and spread.

Show code
satisfaction_by_service = (
    df.groupby("service_type", dropna=False)["satisfaction_score"]
    .agg(records="size", valid_scores="count", mean="mean", median="median", std="std")
    .round(2)
    .reset_index()
)
satisfaction_by_service
service_type records valid_scores mean median std
0 Consultation 84 82 3.71 4.00 0.92
1 Diagnostics 60 59 3.56 4.00 0.86
2 Outreach 60 58 3.41 3.50 0.86
3 Training 36 34 3.91 4.00 0.79
Show code
satisfaction_by_service.to_csv(TABLES_DIR / "satisfaction-by-service.csv", index=False)
Show code
fig, ax = plt.subplots(figsize=(10, 5))
sns.boxplot(data=df, x="service_type", y="satisfaction_score", color=CDI_GREEN, ax=ax)
sns.stripplot(data=df, x="service_type", y="satisfaction_score", color="black",
              alpha=0.35, jitter=0.18, size=3, ax=ax)
ax.set(title="Satisfaction by service type", xlabel="Service type", ylabel="Satisfaction score")
fig.tight_layout()
fig.savefig(FIGURES_DIR / "grouped-satisfaction.png", dpi=300, bbox_inches="tight")
plt.show()

Describe visible differences without claiming that service type caused them. Group composition, case complexity, timing, and selection may also matter.

Examine Numerical Relationships

Show code
age_satisfaction_complete = df[["age", "satisfaction_score"]].dropna()
age_satisfaction_correlation = age_satisfaction_complete.corr(method="spearman").iloc[0, 1]

pd.Series(
    {
        "complete_pairs": len(age_satisfaction_complete),
        "spearman_correlation": round(age_satisfaction_correlation, 3),
    },
    name="value",
)
complete_pairs         225.00
spearman_correlation    -0.02
Name: value, dtype: float64
Show code
fig, ax = plt.subplots(figsize=(9, 5))
sns.scatterplot(
    data=df, x="age", y="satisfaction_score",
    color=CDI_BLUE, alpha=0.55, s=45, ax=ax,
)
ax.set(title="Age and satisfaction score", xlabel="Age (years)", ylabel="Satisfaction score")
fig.tight_layout()
fig.savefig(FIGURES_DIR / "age-satisfaction-relationship.png", dpi=300, bbox_inches="tight")
plt.show()

Spearman correlation summarizes monotonic association and suits an ordinal score better than treating it as an unrestricted continuous measure. Still inspect the plot: a single coefficient can hide clusters, non-linearity, and uneven data density.

Examine Categorical Relationships

Show code
region_service_counts = pd.crosstab(df["region"], df["service_type"], margins=True)
region_service_percent = (
    pd.crosstab(df["region"], df["service_type"], normalize="index")
    .mul(100)
    .round(1)
)

region_service_counts
service_type Consultation Diagnostics Outreach Training All
region
Central 26 16 19 11 72
East 23 17 16 12 68
North 20 8 14 9 51
West 15 19 11 4 49
All 84 60 60 36 240
Show code
region_service_percent.rename_axis("region").reset_index().to_csv(
    TABLES_DIR / "region-service-percent.csv", index=False
)
Show code
plot_table = region_service_percent.copy()
fig, ax = plt.subplots(figsize=(9, 5))
sns.heatmap(plot_table, annot=True, fmt=".1f", cmap="YlGn", cbar_kws={"label": "Row percent"}, ax=ax)
ax.set(title="Service mix within each region", xlabel="Service type", ylabel="Region")
fig.tight_layout()
fig.savefig(FIGURES_DIR / "region-service-relationship.png", dpi=300, bbox_inches="tight")
plt.show()

Row percentages answer “within each region, what share of records belongs to each service type?” Counts and percentages answer different questions; keep the denominator explicit.

Build an Exploratory Visual Summary

Create one figure that presents the most useful patterns without trying to include every chart.

Show code
fig, axes = plt.subplots(2, 2, figsize=(13, 10))

sns.histplot(data=df, x="age", bins=12, color=CDI_GREEN, ax=axes[0, 0])
axes[0, 0].set_title("Participant age")

sns.countplot(data=df, y="region", order=df["region"].value_counts().index,
              color=CDI_BLUE, ax=axes[0, 1])
axes[0, 1].set_title("Records by region")

sns.boxplot(data=df, x="service_type", y="satisfaction_score",
            color=CDI_GOLD, ax=axes[1, 0])
axes[1, 0].set_title("Satisfaction by service type")
axes[1, 0].tick_params(axis="x", rotation=20)

sns.countplot(data=df, x="follow_up_required", hue="requires_review", ax=axes[1, 1])
axes[1, 1].set_title("Follow-up requirement and review flag")

fig.suptitle("Exploratory summary of service-delivery records", fontsize=16, y=1.01)
fig.tight_layout()
fig.savefig(FIGURES_DIR / "exploratory-visual-summary.png", dpi=300, bbox_inches="tight")
plt.show()

Check that titles state what is shown, axes and units are labelled, category order is understandable, colours are legible, and the figure can be interpreted without reading the code.

Step 8: Interpret and Communicate the Evidence

Record Key Findings

Create an evidence table that separates observation from interpretation and limitation. Replace the prompts with findings supported by your actual outputs.

Show code
exploratory_findings = pd.DataFrame(
    [
        {
            "question": "How are records distributed across regions?",
            "evidence": "Complete after inspecting the regional frequency table and chart.",
            "interpretation": "Describe the most and least represented regions.",
            "limitation": "Record counts may reflect service volume or reporting coverage.",
            "next_analysis": "Compare regional coverage with an external population denominator.",
        },
        {
            "question": "How does satisfaction vary by service type?",
            "evidence": "Complete after inspecting group sizes, medians, spreads, and the grouped plot.",
            "interpretation": "Describe differences without causal language.",
            "limitation": "Scores are ordinal and group composition may differ.",
            "next_analysis": "Use a pre-specified ordinal model with relevant confounders.",
        },
        {
            "question": "Which observations require further review?",
            "evidence": f"The IQR screen flagged {len(outlier_review)} age value(s).",
            "interpretation": "Flagged values require context; they are not automatically errors.",
            "limitation": "The IQR rule is sensitive to distribution shape and sample size.",
            "next_analysis": "Verify flagged records against source documentation.",
        },
    ]
)
exploratory_findings
question evidence interpretation limitation next_analysis
0 How are records distributed across regions? Complete after inspecting the regional frequen... Describe the most and least represented regions. Record counts may reflect service volume or re... Compare regional coverage with an external pop...
1 How does satisfaction vary by service type? Complete after inspecting group sizes, medians... Describe differences without causal language. Scores are ordinal and group composition may d... Use a pre-specified ordinal model with relevan...
2 Which observations require further review? The IQR screen flagged 2 age value(s). Flagged values require context; they are not a... The IQR rule is sensitive to distribution shap... Verify flagged records against source document...
Show code
exploratory_findings.to_csv(TABLES_DIR / "exploratory-findings.csv", index=False)

Interpretation and Limitations

Write a concise interpretation that answers the exploratory questions and refers to specific tables or figures. Use measured language such as “the dataset shows,” “records were concentrated,” or “scores tended to differ.” Avoid “proved,” “caused,” or “led to.”

Your limitations statement should address:

  • the fictional and service-record nature of the data;
  • whether records represent people, visits, or both;
  • missing age, satisfaction, or contact information;
  • any review flags retained after cleaning;
  • unequal group sizes;
  • the ordinal and bounded satisfaction scale;
  • the absence of a population denominator;
  • multiple patterns explored without confirmatory testing;
  • inability to infer causality from observational summaries.
Exploration is not confirmation

EDA helps generate and refine questions. If a finding will guide a high-stakes decision, test it with an analysis plan, suitable design, uncertainty estimates, and data collected for that purpose.

Step 9: Verify and Present the Evidence Package

Verify Exported Results

Confirm that the evidence package exists and can be regenerated from the source data.

Show code
expected_outputs = [
    TABLES_DIR / "dataset-overview.csv",
    TABLES_DIR / "numerical-summary.csv",
    TABLES_DIR / "categorical-summary.csv",
    TABLES_DIR / "temporal-summary.csv",
    TABLES_DIR / "missingness-summary.csv",
    TABLES_DIR / "outlier-review.csv",
    TABLES_DIR / "satisfaction-by-service.csv",
    TABLES_DIR / "region-service-percent.csv",
    TABLES_DIR / "exploratory-findings.csv",
    FIGURES_DIR / "numerical-distributions.png",
    FIGURES_DIR / "categorical-patterns.png",
    FIGURES_DIR / "monthly-service-records.png",
    FIGURES_DIR / "age-outlier-review.png",
    FIGURES_DIR / "grouped-satisfaction.png",
    FIGURES_DIR / "age-satisfaction-relationship.png",
    FIGURES_DIR / "region-service-relationship.png",
    FIGURES_DIR / "exploratory-visual-summary.png",
]

output_check = pd.DataFrame(
    {"path": [str(path) for path in expected_outputs], "exists": [path.exists() for path in expected_outputs]}
)
output_check
path exists
0 results/tables/dataset-overview.csv True
1 results/tables/numerical-summary.csv True
2 results/tables/categorical-summary.csv True
3 results/tables/temporal-summary.csv True
4 results/tables/missingness-summary.csv True
5 results/tables/outlier-review.csv True
6 results/tables/satisfaction-by-service.csv True
7 results/tables/region-service-percent.csv True
8 results/tables/exploratory-findings.csv True
9 results/figures/numerical-distributions.png True
10 results/figures/categorical-patterns.png True
11 results/figures/monthly-service-records.png True
12 results/figures/age-outlier-review.png True
13 results/figures/grouped-satisfaction.png True
14 results/figures/age-satisfaction-relationship.png True
15 results/figures/region-service-relationship.png True
16 results/figures/exploratory-visual-summary.png True
Show code
missing_outputs = output_check.loc[~output_check["exists"], "path"].tolist()
assert not missing_outputs, f"Expected outputs were not created: {missing_outputs}"

Compare your exported files with the supplied author-generated reference results. Small presentation differences may occur across software versions, but the data dimensions, summary values, file set, and analytical conclusions should agree.

Portfolio Evidence

Your portfolio evidence should show both the analytical result and the quality of your reasoning. Include:

  • a short project title and question;
  • the cleaned dataset and data dictionary;
  • the analysis plan;
  • one dataset overview table;
  • two or three strong figures rather than every exploratory plot;
  • a short findings table;
  • an outlier and missingness decision note;
  • a reproducible workflow description;
  • interpretation, limitations, and next steps;
  • a link to your repository or portfolio presentation when appropriate.

Project Question

Use this portfolio framing:

What patterns in service use, participant characteristics, satisfaction, follow-up requirements, and data completeness can be identified in the cleaned service-delivery dataset, and which patterns warrant further analysis?

Revise the question if you adapt the practical to your own dataset. A good question names the population or records, variables, purpose, and boundaries of the analysis.

Dataset

In your portfolio, describe:

  • what one row represents;
  • the number of rows and columns;
  • the relevant variables and their analytical roles;
  • where the cleaned dataset came from;
  • any access, privacy, or ethical constraints;
  • remaining missingness and review flags.

Do not publish direct identifiers from real data. Replace them with safe identifiers or use aggregated outputs.

Workflow

Summarize the work as a reproducible sequence:

  1. preserved and loaded the cleaned source;
  2. confirmed analysis readiness and variable roles;
  3. created dataset, numerical, and categorical summaries;
  4. examined distributions, missingness, and outliers;
  5. compared groups and variable relationships;
  6. recorded findings separately from interpretations;
  7. exported tables and figures;
  8. documented limitations and next analyses.

Main Result

Select one result that best answers the project question. Present the figure or table with a precise title, one paragraph describing the evidence, and one paragraph explaining why it matters. The main result should be understandable without requiring the reader to inspect every intermediate output.

Quality Checks

State the checks that support confidence in the workflow:

  • required columns were present;
  • identifiers and duplicate rows were checked;
  • dates and numeric fields were parsed explicitly;
  • valid and missing record counts accompanied summaries;
  • group sizes were reported;
  • outliers were reviewed rather than silently deleted;
  • exported files were verified;
  • completed outputs were compared with the supplied reference results.

Limitations

Keep limitations specific. Explain how each one affects interpretation and what additional data or analysis would reduce uncertainty. A limitation is not a ceremonial paragraph; it defines the boundary of the evidence.

Tools

List only tools you actually used. For the standard workflow:

  • Python for reproducible analysis;
  • pandas and numpy for data summaries;
  • matplotlib and seaborn for visualization;
  • VS Code for editing and running the workflow;
  • Git and GitHub for version control and portfolio sharing, if used.

Step 10: Reflect and Complete the Practical

Reflection and Next Steps

Write brief responses to the following prompts:

  1. Which exploratory question produced the most useful evidence, and why?
  2. Which chart changed or complicated your first interpretation?
  3. How did missingness affect the analyses you could perform?
  4. Which outlier decision required the most judgement?
  5. Where were you most at risk of making a causal claim?
  6. What would you pre-specify before a confirmatory analysis?
  7. How would you adapt this workflow to a larger or more sensitive dataset?

A natural next practical would move from exploration to a focused statistical analysis or predictive modelling workflow, using questions and limitations identified here.

Completion Checklist

Before marking the practical complete, confirm that you have:

Practical Complete

You have completed Data Science Practical 002: Exploratory Data Analysis Practical.

You now have a reproducible EDA workflow, a set of analytical outputs, and portfolio evidence that demonstrates more than chart-making. It shows that you can begin with questions, examine data responsibly, distinguish patterns from proof, and communicate what the evidence can and cannot support.

Keep the starting data, your analysis work, and exported results together. They form a complete record of the analysis and a reusable starting point for future exploratory work.

© Complex Data Insights | CDI Practicals

Hands-on exercises and step-by-step tools