CARVIEW |
Select Language
HTTP/2 200
date: Sat, 11 Oct 2025 09:59:52 GMT
server: Fly/6f91d33b9d (2025-10-08)
content-type: text/html; charset=utf-8
content-encoding: gzip
via: 2 fly.io, 2 fly.io
fly-request-id: 01K79BCSAJDEWY0ZFATR11DG17-bom
pytest coverage with context | Simon Willison’s TILs
pytest coverage with context
This tweet from @Mariatta tipped me off to the ability to measure "contexts" when running coverage - as a way to tell which tests exercise which specific lines of code.
My sqlite-utils project uses pytest
for the test suite. I decided to figure out how to get this working with pytest-cov.
After some experimentation, this is the recipe that worked for me:
# In the virtual environment, make sure pytest-cov is installed:
% pip install pytest-cov
# First, run pytest to calculate coverage of the `sqlite_utils` package, with context
% pytest --cov=sqlite_utils --cov-context=test
# The .coverage file is actually a SQLite database:
% ls -lah .coverage
-rw-r--r--@ 1 simon staff 716K Mar 4 16:39 .coverage
# This command generates the HTML coverage report in `htmlcov/`
% coverage html --show-contexts
# Open the report in a browser`
% open htmlcov/index.html
Here's what one of the pages looks like, displaying the context for some lines of code:
The .coverage schema
Since .coverage
is a SQLite database, here's the schema - generated using sqlite-utils schema .coverage
:
CREATE TABLE coverage_schema (
-- One row, to record the version of the schema in this db.
version integer
);
CREATE TABLE meta (
-- Key-value pairs, to record metadata about the data
key text,
value text,
unique (key)
-- Keys:
-- 'has_arcs' boolean -- Is this data recording branches?
-- 'sys_argv' text -- The coverage command line that recorded the data.
-- 'version' text -- The version of coverage.py that made the file.
-- 'when' text -- Datetime when the file was created.
);
CREATE TABLE file (
-- A row per file measured.
id integer primary key,
path text,
unique (path)
);
CREATE TABLE context (
-- A row per context measured.
id integer primary key,
context text,
unique (context)
);
CREATE TABLE line_bits (
-- If recording lines, a row per context per file executed.
-- All of the line numbers for that file/context are in one numbits.
file_id integer, -- foreign key to `file`.
context_id integer, -- foreign key to `context`.
numbits blob, -- see the numbits functions in coverage.numbits
foreign key (file_id) references file (id),
foreign key (context_id) references context (id),
unique (file_id, context_id)
);
CREATE TABLE arc (
-- If recording branches, a row per context per from/to line transition executed.
file_id integer, -- foreign key to `file`.
context_id integer, -- foreign key to `context`.
fromno integer, -- line number jumped from.
tono integer, -- line number jumped to.
foreign key (file_id) references file (id),
foreign key (context_id) references context (id),
unique (file_id, context_id, fromno, tono)
);
CREATE TABLE tracer (
-- A row per file indicating the tracer used for that file.
file_id integer primary key,
tracer text,
foreign key (file_id) references file (id)
);
Related
- pytest Code coverage using pytest and codecov.io - 2020-08-15
- pytest Snapshot testing with Syrupy - 2023-09-26
- github-actions Running tests against PostgreSQL in a service container - 2021-02-23
- pytest Using pytest and Playwright to test a JavaScript web application - 2022-07-24
- django Using pytest-django with a reusable Django application - 2024-08-07
- python Quickly testing code in a different Python version using pyenv - 2023-07-10
- python Running PyPy on macOS using Homebrew - 2022-09-14
- sphinx Adding Sphinx autodoc to a project, and configuring Read The Docs to build it - 2021-08-10
- cookiecutter Testing cookiecutter templates with pytest - 2021-01-27
- sqlite Building a specific version of SQLite with pysqlite on macOS/Linux - 2021-08-14
Created 2022-03-04T16:53:50-08:00 · Edit