Chapter 22 of 24 | Python

Build systems and professional workflow

A professional workflow makes development repeatable for you, collaborators, and automation. Keep source, tests, configuration, and documentation together; keep generated environments and caches out of version control.

Project configuration

Use pyproject.toml for tool and package configuration where supported. Declare the Python version range, dependencies, formatter, linter, and test settings. A virtual environment isolates installed packages but is generated locally, so provide instructions for recreating it.

python -m venv .venv
python -m pip install -e ".[dev]"
python -m unittest discover -s tests
python -m compileall src

Commands depend on the project's packaging tools, but should be documented and non-interactive where possible. Use python -m to run tools with the chosen interpreter. A lock file may improve repeatability for applications; libraries often declare compatible version ranges instead.

Formatting, linting, and types

Automated formatting reduces style-only debate. A linter catches suspicious patterns and unused imports. Static type checking can catch mismatches but works best with annotations and a deliberate strictness policy. These tools complement tests; they do not prove that program behavior matches requirements.

Version control and continuous integration

Use small, focused commits and review diffs for unintended files, secrets, generated output, and debugging code. Continuous integration should create a clean environment, install declared dependencies, run lint/type checks and tests, and report failures with the Python version used.

Test the supported version range, not only the newest local interpreter. Platform-dependent projects should exercise each claimed operating system. Reproducible builds require declared inputs and a documented toolchain, while dependency updates require an explicit review process.

Observability and release workflow

Applications need useful logs, clear error messages, and a way to identify their version. Avoid logging sensitive values. Releases should be built from a reviewed revision, tested as installed artifacts, and accompanied by concise change notes. For libraries, test the wheel in a separate environment so source-tree imports do not hide missing files.

Beginner-to-master progression

Exercises

  1. Create a project environment and document exact setup and test commands.
  2. Add a formatter, linter, and type checker, then fix one diagnostic from each.
  3. Build and install a package artifact into a clean environment.
  4. Design a CI matrix for two supported Python versions and two operating systems.

Reproducible environments and lock files

A virtual environment isolates installed packages but does not, by itself, guarantee the same dependency versions on another machine. Requirements files, lock files, constraints, and package metadata serve different purposes. Record direct dependencies and let the chosen resolver produce a consistent graph; include hashes or artifact verification when the deployment threat model requires them.

Separate development tools from runtime dependencies. A user installing a library should not need your test runner unless it is intentionally part of the product. For applications, include all runtime requirements in a reproducible deployment artifact. Rebuild environments from scratch regularly so undeclared packages do not hide missing metadata.

CI matrices and release confidence

A CI matrix can test supported Python versions and operating systems in parallel. Keep the matrix aligned with the project's stated support policy; testing every combination of every dependency may be too costly, so choose representative configurations and explain the coverage. Run lint, static typing, tests, and package-build checks as explicit jobs with actionable logs.

Release pipelines should build from a clean, reviewed revision, test the produced artifact, and preserve provenance. Secrets should be scoped to the job that needs them. A release should be reproducible or at least traceable to its source, toolchain, and dependencies.

Native extensions and ABI compatibility

Packages with C or C++ extensions have additional compatibility concerns: compiler toolchains, operating systems, CPU architectures, Python's extension ABI, and external system libraries. Wheels may use compatibility tags to describe supported combinations. Test source builds and binary wheels where relevant, and avoid assuming an extension that works on one platform will install everywhere.

Dependency updates and supply-chain controls

Automated update tools can propose version changes, but tests and human review remain important. Review changelogs for breaking changes, security fixes, and new transitive dependencies. Use trusted indexes and protect publishing credentials. A dependency policy should say how vulnerabilities are prioritized and how quickly updates are evaluated.

Mastery checkpoint

  1. Recreate a project environment from declared metadata in a clean machine or container.
  2. Build wheels for the supported Python versions and inspect their compatibility tags.
  3. Design CI jobs that fail clearly when tests, typing, formatting, or packaging fail.
  4. Write a dependency update and release provenance policy suitable for a small team.
0% read