Testing and Quality Gates

This project uses a layered quality stack: pytest for behavior coverage, tox as the test runner wrapper, and pre-commit for linting, type checking, and secret scanning.

Warning: No repository CI pipeline files are currently present (.github/workflows, .gitlab-ci.yml, .circleci, Jenkinsfile, etc.). Quality gates are defined in local tooling (tox.toml, .pre-commit-config.yaml) and any external pre-commit service integration.

Pytest Coverage Areas

pytest is configured in pyproject.toml:

[project.optional-dependencies]
dev = ["pytest", "pytest-asyncio", "pytest-xdist", "httpx"]

[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
pythonpath = ["src"]

Current suite structure covers 149 tests across 13 test modules:

  • Auth, RBAC, session and access control: tests/test_auth.py (36), tests/test_main.py (15), tests/test_dashboard.py (4)
  • Storage and persistence behavior: tests/test_storage.py (33)
  • Generation/planning/parser/repository logic: tests/test_generator.py (8), tests/test_json_parser.py (15), tests/test_prompts.py (3), tests/test_repository.py (9)
  • Rendering and content safety: tests/test_renderer.py (11)
  • Contracts and configuration models: tests/test_config.py (3), tests/test_models.py (9), tests/test_ai_client.py (2)
  • End-to-end mocked flow: tests/test_integration.py (1)

Coverage Examples from Tests

SSRF hardening (private DNS/IP rejection) from tests/test_main.py:

async def test_reject_private_url_dns(monkeypatch: pytest.MonkeyPatch) -> None:
    """Test that SSRF protection rejects DNS names resolving to private IPs."""
    import socket

    from docsfy.main import _reject_private_url

    def mock_getaddrinfo(
        host: str, port: object, *args: object, **kwargs: object
    ) -> list[
        tuple[socket.AddressFamily, socket.SocketKind, int, str, tuple[str, int]]
    ]:
        return [(socket.AF_INET, socket.SOCK_STREAM, 0, "", ("192.168.1.1", 0))]

    monkeypatch.setattr(socket, "getaddrinfo", mock_getaddrinfo)

    with pytest.raises(HTTPException) as exc_info:
        await _reject_private_url("https://evil.com/org/repo")
    assert exc_info.value.status_code == 400

Role enforcement (viewer cannot generate docs) from tests/test_auth.py:

async def test_viewer_cannot_generate(_init_db: None) -> None:
    """A viewer should get 403 when trying to generate docs."""
    from docsfy.main import _generating, app
    from docsfy.storage import create_user

    _generating.clear()
    _, viewer_key = await create_user("viewer-gen", role="viewer")

    transport = ASGITransport(app=app)
    async with AsyncClient(
        transport=transport,
        base_url="http://test",
        headers={"Authorization": f"Bearer {viewer_key}"},
    ) as ac:
        response = await ac.post(
            "/api/generate",
            json={
                "repo_url": "https://github.com/org/repo",
                "project_name": "test-proj",
            },
        )
    assert response.status_code == 403
    assert "Write access required" in response.json()["detail"]
    _generating.clear()

Output sanitization (XSS vectors blocked) from tests/test_renderer.py:

def test_sanitize_html_unquoted_javascript() -> None:
    from docsfy.renderer import _sanitize_html

    result = _sanitize_html("<a href="#">)
    assert "javascript:" not in result

    result = _sanitize_html("<img src="#">)
    assert "javascript:" not in result

    result = _sanitize_html("<a href="#">)
    assert "data:" not in result

    result = _sanitize_html("<img src="#">)
    assert "data:" not in result

End-to-end API/docs artifact flow from tests/test_integration.py:

response = await client.get("/docs/test-repo/claude/opus/index.html")
assert response.status_code == 200
assert "test-repo" in response.text

response = await client.get("/api/projects/test-repo/claude/opus/download")
assert response.status_code == 200
assert response.headers["content-type"] == "application/gzip"

Note: There is no coverage threshold gate configured (no pytest-cov settings in pyproject.toml or tox.toml).

tox Usage

tox is configured in tox.toml with a single environment:

skipsdist = true

envlist = ["unittests"]

[env.unittests]
deps = ["uv"]
commands = [["uv", "run", "--extra", "dev", "pytest", "-n", "auto", "tests"]]

What this means:

  • tox runs the unittests env by default.
  • Tests run through uv with dev extras.
  • pytest-xdist is used with -n auto for parallel execution.
  • Packaging/build is skipped for test runs (skipsdist = true).

Tip: For parity with tox while debugging a single step, use the exact command from tox.toml: uv run --extra dev pytest -n auto tests.

Pre-commit Hooks

Hook orchestration is defined in .pre-commit-config.yaml:

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v6.0.0
    hooks:
      - id: check-added-large-files
      - id: check-docstring-first
      - id: check-executables-have-shebangs
      - id: check-merge-conflict
      - id: check-symlinks
      - id: detect-private-key
      - id: mixed-line-ending
      - id: debug-statements
      - id: trailing-whitespace
        args: [--markdown-linebreak-ext=md]
      - id: end-of-file-fixer
      - id: check-ast
      - id: check-builtin-literals
      - id: check-toml

It also wires lint/type/security hooks:

  # flake8 retained for RedHatQE M511 plugin; ruff handles standard linting
  - repo: https://github.com/PyCQA/flake8
    rev: 7.3.0
    hooks:
      - id: flake8
        args: [--config=.flake8]
        additional_dependencies:
          [git+https://github.com/RedHatQE/flake8-plugins.git, flake8-mutable]

  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.15.2
    hooks:
      - id: ruff
      - id: ruff-format

  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.19.1
    hooks:
      - id: mypy
        exclude: (tests/)

mypy Gate

Type checking is strict at project level in pyproject.toml:

[tool.mypy]
check_untyped_defs = true
disallow_any_generics = true
disallow_incomplete_defs = true
disallow_untyped_defs = true
no_implicit_optional = true
show_error_codes = true
warn_unused_ignores = true
strict_equality = true
extra_checks = true
warn_unused_configs = true
warn_redundant_casts = true

In pre-commit, mypy also installs extra stubs/deps:

  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.19.1
    hooks:
      - id: mypy
        exclude: (tests/)
        additional_dependencies:
          [types-requests, types-PyYAML, types-colorama, types-aiofiles, pydantic, types-Markdown]

Ruff Gate

Ruff is enforced via pre-commit:

  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.15.2
    hooks:
      - id: ruff
      - id: ruff-format

There is no dedicated [tool.ruff] section in pyproject.toml, so ruff runs with defaults unless overridden by hook-level args (none currently set).

Flake8 Compatibility Gate (M511)

flake8 is retained specifically for RedHatQE plugin checks:

[flake8]
select=M511

This keeps M511 enforcement while ruff handles general linting.

Secrets Scanning Gates

Secret scanning is layered in pre-commit:

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v6.0.0
    hooks:
      - id: detect-private-key

  - repo: https://github.com/Yelp/detect-secrets
    rev: v1.5.0
    hooks:
      - id: detect-secrets

  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.30.0
    hooks:
      - id: gitleaks

gitleaks has a repo-specific allowlist in .gitleaks.toml:

[extend]
useDefault = true

[allowlist]
paths = [
    '''tests/test_repository\.py''',
]

Test fixtures also use inline allowlist annotations where needed, for example in tests/test_repository.py:

assert sha == "abc123def"  # pragma: allowlist secret

Warning: Allowlisting should stay narrowly scoped to test fixtures only; broad allowlists can hide real leaks in production code.