This document defines the testing policy and workflows for the START project.
Flow
uv run pytest -qtest_common_*.pytest_domain.py, test_entity.pytest_config.py, test_languages_config.pytest_system_*.pytest_curriculum_entrypoints.pytest_core_hardening.pytest_remaining_source.pytest_generate_curriculum_gui.pytest_repos_*.pyUse the gate rather than retyping a command: uv run start-gate <arm> runs
exactly what CI runs. See Gate entry point for the arm list and
generated numbers.
# Default arm: tests not marked slow, network, or integration
uv run start-gate
# One arm at a time (each is also a CI step)
uv run start-gate fast
uv run start-gate full
uv run start-gate docs
uv run start-gate gnn
uv run start-gate artifacts
# Every arm with its exact argv
uv run start-gate --list
# Focused local runs (not gate arms)
uv run pytest -k "domain"
uv run pytest tests/test_domain.py
@pytest.mark.integration: end-to-end or cross-module behavior@pytest.mark.slow: long-running@pytest.mark.network: reserved for explicitly invoked external connectivity checkstmp_path for writable temp dirsdata/domain_research, data/audience_research, and the output directoriesdef test_file_processing(tmp_path):
source = tmp_path / "input.json"
source.write_text('{"a": 1}')
result = process_file(source)
assert result["a"] == 1
# Non-GUI matplotlib backend
export MPLBACKEND=Agg
# Optional: keys for an explicitly invoked live provider probe
export PERPLEXITY_API_KEY="..."
export OPENROUTER_API_KEY="..."
The full arm enforces the 80% branch-aware working baseline and is the only
coverage pass in CI; the measurement is recorded in
_generated/gate-timings.md. The manual release
workflow is fail-closed at the 90% branch-aware floor and adds the publication
output gate. Linux runs cover Python 3.11–3.12 (fast on 3.11, full on 3.12);
the macOS smoke job uses the locked Python 3.12 environment and runs the
smoke-macos arm.
The Linux matrix explicitly passes its Python version to both uv sync and
uv run; installing an interpreter alone does not override the repository’s
Python 3.12 pin. Before the selected gate runs, CI checks the actual version,
virtual-environment prefix and executable directory. The required test rollup
still requires every existing CI job to succeed; failed, cancelled or skipped
jobs cannot satisfy it.
uv run start-gate docs # strict MkDocs build + repository validation
uv run start-gate numbers # generated gate numbers match scripts/gate.py
The repository validator parses tracked JSON/YAML/TOML files, rejects duplicate YAML keys, checks authored Markdown links, rejects references to retired language-layout paths, and blocks project-authored terminology that would imply non-real provider or test behavior. Generated curriculum/research snapshots and vendored third-party artifacts are excluded from authored-text terminology checks while remaining subject to structural config parsing when applicable.
@pytest.mark.slow