tests.test_suite_hygiene module

Guard the test suite against parametrizing over an unordered collection.

pytest-xdist distributes work by test ID, so every worker must collect the same IDs in the same order. A @pytest.mark.parametrize fed a set or frozenset iterates in an order that depends on the process’s string-hash seed, so each worker derives different IDs and the whole run aborts with “Different tests were collected between gw0 and gw1” before a single test executes. The failure names the workers rather than the parametrization, and the suite still passes serially, which is what makes it worth a conformance test rather than a comment.

Wrapping the argument in sorted() fixes it.

Note

A dict is not a hazard: mappings iterate in insertion order, which is a language guarantee and identical in every process. Only the hash-ordered containers qualify, so SOME_DICT and SOME_DICT.items() are both accepted.

Two shapes evade a naive scan, and both have already shipped a latent break:

• An imported collection. from repomatic.registry import NON_REUSABLE_WORKFLOWS puts a frozenset in scope under a name this module never sees assigned, so the binding is resolved by importing it rather than by reading the source.

• A wrapper call. list(SOME_SET) and tuple(SOME_SET) are as unordered as the set they copy, and read as deliberate, which is worse.

tests.test_suite_hygiene.TESTS_DIR = PosixPath('/home/runner/work/repomatic/repomatic/tests')

Root of the test suite.

tests.test_suite_hygiene.TEST_FILES = [PosixPath('/home/runner/work/repomatic/repomatic/tests/__init__.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/conftest.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_actions.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_attestation.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_awesome_template.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_awesome_toc.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_binaries_page.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_binary.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_broken_links.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_cache.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_changelog.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_checksums.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_ci_status.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_claude_assets.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_cloudflare.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_cloudflare_r2.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_config.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_dep_graph.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_dep_policy.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_dep_report.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_dep_sources.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_dev_release.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_docs.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_docstrings.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_forge.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_frontmatter.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_gh.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_git_ops.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_gitignore.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_hashing.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_help.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_http.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_humanize.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_images.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_imports.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_init_project.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_issue.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_job_timings.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_labels.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_lint_repo.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_mailmap.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_matrix.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_metadata.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_metrics.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_npm.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_pages_redirects.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_platform_keys.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_plugin.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_plugin_update_detection.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_pr.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_pr_body.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_prepare_release.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_pypi.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_pyproject.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_readme.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_release_sync.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_releases.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_runner_catalog.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_runner_sync.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_setup_guide.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_skills.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_sphinx_crossrefs.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_status.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_suite_hygiene.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_sync_ops.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_tabular.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_todolist.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_tool_runner.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_unsubscribe.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_uv.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_version_sync.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_versions.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_virustotal.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_vulnerable_deps.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_workflow_sync.py'), PosixPath('/home/runner/work/repomatic/repomatic/tests/test_workflows.py')]

Every Python module of the test suite, sorted because pytest-xdist needs identical parametrize IDs in every worker.

tests.test_suite_hygiene.UNORDERED_CALLS = frozenset({'frozenset', 'set'})

Builtins whose result has no stable iteration order across processes.

tests.test_suite_hygiene.ORDER_PRESERVING_WRAPPERS = frozenset({'list', 'tuple'})

Builtins that copy an iterable without imposing an order of their own.

Applied to a set, they freeze whatever hash order that process happened to produce, which is exactly the failure they look like they are preventing.

tests.test_suite_hygiene.FILE_ENCODING = 'UTF-8'

The single spelling of the encoding= argument used across the suite.

Matching production’s dominant spelling. Both spellings are equally correct to Python, so this is purely about a reader never wondering whether the difference carries meaning.

tests.test_suite_hygiene.ENCODING_ARGUMENT_RE = re.compile('encoding=["\'](?!UTF-8["\'])([\\w-]+)["\']')

Match an encoding= keyword whose value is not the canonical spelling.

tests.test_suite_hygiene.test_test_suite_discovered()

The scan actually walks the suite, so a green result means something.

Return type:

None

tests.test_suite_hygiene.test_detector_flags_unordered_parametrize()

The detector catches each unordered shape, and clears the ordered ones.

Return type:

None

tests.test_suite_hygiene.test_no_unordered_parametrize(test_file)

No parametrization iterates a collection with an unstable order.

Return type:

None

tests.test_suite_hygiene.test_encoding_argument_spelling_is_uniform(test_file)

Every encoding= in the suite uses the one canonical spelling.

Python normalizes the codec name, so a mixed suite is correct but reads as though the difference means something. Pinning it is what keeps a reviewer from having to decide.

Return type:

None

tests.test_suite_hygiene.BARE_TEXT_IO_CALLS = frozenset({'open', 'read_text', 'write_text'})

Text-I/O calls that silently fall back to the platform encoding when bare.

tests.test_suite_hygiene.test_text_io_always_names_its_encoding(test_file)

Every text-mode I/O call passes encoding= explicitly.

Windows still defaults to cp1252, so a bare read_text() hides until the content grows a non-ASCII character and only fails in Windows CI. Ruff’s PLW1514 covers the receivers its inference can type; this sweep covers the unannotated Path locals it misses.

Return type:

None

tests.test_suite_hygiene.test_no_function_local_imports(test_file)

Imports live at module level, never inside test or helper bodies.

Function-local imports hide dependencies and bypass ruff’s import sorting; nothing in the suite needs one to break an import cycle.

Return type:

None

tests.test_suite_hygiene.INTEGRATION_TOOL_GUARD = 'skip_unless_tool_runs'

Marks a test that deliberately runs a tool for real, downloads included.

The opt-out for test_run_tool_calls_mock_the_binary_install(): a test calling this has declared that reaching the network is the point, and is skipped wholesale on a runner where the tool cannot run.

tests.test_suite_hygiene.BINARY_INSTALL_MOCK = '_install_binary'

The tool_runner internal a hermetic run_tool test has to patch.

tests.test_suite_hygiene.test_run_tool_calls_mock_the_binary_install(test_file)

A run_tool test on a downloading tool must patch its install.

Patching subprocess.run alone looks hermetic and is not: run_tool resolves the binary before it builds a command line, so the download runs for real and only the invocation is faked. Such a test passes in silence for as long as GitHub Releases is healthy, then fails the whole Tests job with an HTTPError naming a URL nothing in the test mentions. Three mdformat tests shipped this way and went red on a 503, mdformat being a uvx tool that still downloads shfmt to put on its PATH.

A test that means to hit the network says so with INTEGRATION_TOOL_GUARD and is exempt.

Return type:

None

tests.test_suite_hygiene.test_no_classes_for_grouping(test_file)

No Test* classes: tests are top-level functions, per claude.md.

Non-test helper classes (doubles, matcher sentinels) are fine; only a class pytest would collect as a grouping container is flagged.

Return type:

None