Test Workflow

Loading…

Test Workflow

Relevant source files

• .github/workflows/tests.yml
• scripts/run-tests.sh

Purpose and Scope

This document describes the continuous integration testing workflow that validates code quality on every push to the main branch and on all pull requests. The workflow executes Python unit tests to ensure that core components function correctly before code is merged.

For information about the production build and deployment workflow, see Build and Deploy Workflow. For details on using the system as a reusable GitHub Action, see GitHub Action. For instructions on running tests locally during development, see Running Tests.

Workflow Configuration

The test workflow is defined in .github/workflows/tests.yml:1-26 It uses GitHub Actions to automatically execute the test suite in a clean Ubuntu environment.

Trigger Events

The workflow activates on two event types:

Sources: .github/workflows/tests.yml:3-7

Workflow Trigger and Outcome Logic

Sources: .github/workflows/tests.yml:3-7

Job Structure

The workflow contains a single job named pytest that runs on ubuntu-latest. The job executes five sequential steps to set up the environment and run tests.

pytest Job Execution Flow

graph TB
    subgraph "pytest Job Steps"
        S1["Step 1: Checkout\nactions/checkout@v4"]
S2["Step 2: Setup Python\nactions/setup-python@v5\npython-version: 3.12"]
S3["Step 3: Install Dependencies\npip install -r python/requirements.txt\npip install pytest"]
S4["Step 4: Run pytest\npytest python/tests/ -s"]
end
    
 
   S1 --> S2
 
   S2 --> S3
 
   S3 --> S4

Sources: .github/workflows/tests.yml:10-25

Step-by-Step Breakdown

Step 1: Repository Checkout

Uses actions/checkout@v4 to clone the repository at the commit being tested. For pull requests, this checks out the merge commit to test the exact code that would be merged.

Sources: .github/workflows/tests.yml:13-14

Step 2: Python Environment Setup

Configures Python 3.12 using actions/setup-python@v5. This matches the Python version used in the Docker container Dockerfile26 ensuring consistency between local development, CI testing, and production execution.

Sources: .github/workflows/tests.yml:15-18

Step 3: Dependency Installation

Installs required packages in three phases:

1. Upgrades pip to the latest version
2. Installs project dependencies from python/requirements.txt
3. Installs pytest testing framework

Sources: .github/workflows/tests.yml:19-23

Step 4: Test Execution

Runs pytest against the python/tests/ directory with the -s flag to show print statements. This executes all test modules discovered by pytest.

Sources: .github/workflows/tests.yml:24-25

Test Coverage

The workflow executes three test suites that validate core system components:

Test Module Coverage and Component Mapping

Sources: .github/workflows/tests.yml:24-25 scripts/run-tests.sh:11-30

Template Processor Tests

Validates the process_template function in python/process-template.py Tests verify:

• Variable substitution ({{REPO}}, {{BOOK_TITLE}}, etc.)
• Conditional rendering based on variable presence
• Special character handling in template values
• Fallback behavior when variables are undefined

Sources: scripts/run-tests.sh11

Mermaid Normalization Tests

Tests the seven-step normalization pipeline in python/deepwiki-scraper.py Validates:

• Edge label flattening for multiline labels
• State description syntax fixes
• Flowchart node cleanup
• Statement separator insertion
• Empty label fallback generation
• Gantt chart task ID synthesis

For detailed information on the normalization pipeline, see Mermaid Normalization.

Sources: scripts/run-tests.sh22

Numbering Tests

Verifies the page numbering and path resolution logic in python/deepwiki-scraper.py Tests confirm:

• Correct filename generation from hierarchical numbers (e.g., 9.2 → 9.2_test-workflow.md)
• Number parsing and validation
• Path calculations relative to markdown directory
• Numeric sorting behavior for SUMMARY.md generation

For detailed information on the numbering system, see Numbering and Path Resolution.

Sources: scripts/run-tests.sh30

graph LR
    subgraph "Development Flow"
        Dev["Developer\nCreates PR"]
Review["Code Review\nProcess"]
Merge["Merge to main"]
end
    
    subgraph "Test Workflow"
        TestTrigger["tests.yml\non: pull_request"]
PytestJob["pytest job"]
TestResult{{"Test Result"}}
end
    
    subgraph "Build Workflow"
        BuildTrigger["build-and-deploy.yml\non: push to main"]
BuildJob["Build & Deploy"]
end
    
 
   Dev --> TestTrigger
 
   TestTrigger --> PytestJob
 
   PytestJob --> TestResult
 
   TestResult -->|Pass| Review
 
   TestResult -->|Fail| Dev
 
   Review --> Merge
 
   Merge --> BuildTrigger
 
   BuildTrigger --> BuildJob

Integration with CI/CD Pipeline

The test workflow serves as a quality gate in the development process. Its relationship to other CI/CD components is shown below:

CI/CD Pipeline Integration and Quality Gate

Sources: .github/workflows/tests.yml:3-7

The workflow prevents code with failing tests from being merged, ensuring that the weekly build-and-deploy workflow (Build and Deploy Workflow) only processes validated code.

graph TB
    subgraph "GitHub Actions Execution"
        GHA["tests.yml workflow"]
GHAEnv["ubuntu-latest\nPython 3.12\nClean environment"]
GHATest["pytest python/tests/ -s"]
end
    
    subgraph "Local Execution"
        Local["scripts/run-tests.sh"]
LocalEnv["Developer machine\nAny Python 3.x\nCurrent environment"]
LocalTest["python3 -m pytest\nOR\npython3 test_*.py"]
end
    
    subgraph "Shared Test Suite"
        Tests["python/tests/\ntest_template_processor.py\ntest_mermaid_normalization.py\ntest_numbering.py"]
end
    
 
   GHA --> GHAEnv
 
   GHAEnv --> GHATest
 
   GHATest --> Tests
    
 
   Local --> LocalEnv
 
   LocalEnv --> LocalTest
 
   LocalTest --> Tests

Local Test Execution

Developers can run the same tests locally using the scripts/run-tests.sh script. This script provides an alternative execution path that doesn’t require GitHub Actions:

Dual Execution Paths for Test Suite

Sources: .github/workflows/tests.yml:1-26 scripts/run-tests.sh:1-43

The local script scripts/run-tests.sh:1-43 includes fallback logic: if pytest is not available, it runs test_template_processor.py directly using Python’s unittest framework, but skips the pytest-dependent tests. This ensures developers can run at least basic tests without installing pytest.

Workflow Best Practices

Pull Request Requirements

Before merging, pull requests must:

1. Pass all pytest tests
2. Not introduce new test failures
3. Update tests if behavior changes

Test Failure Investigation

When tests fail, the workflow output shows:

• Which test module failed
• The specific test function that failed
• Assertion details and error messages
• Full stdout/stderr due to the -s flag

The -s flag in .github/workflows/tests.yml25 ensures that print statements from the code under test are visible in the workflow logs, aiding debugging.

Adding New Tests

To add new tests to the workflow:

1. Create a new test module in python/tests/
2. Follow pytest conventions (test_*.py filename, test_* function names)
3. The workflow automatically discovers and runs the new tests
4. No workflow configuration changes required

Sources: .github/workflows/tests.yml:24-25

Comparison with Build Workflow

The following table highlights the differences between the test workflow and the build-and-deploy workflow:

Sources: .github/workflows/tests.yml:1-26

Dismiss

Refresh this wiki

Enter email to refresh