CI/CD Integration

Loading…

CI/CD Integration

Relevant source files

• .github/workflows/build-and-deploy.yml
• .github/workflows/tests.yml
• action.yml

Purpose and Scope

This page provides an overview of the continuous integration and continuous deployment (CI/CD) infrastructure for the DeepWiki-to-mdBook system. The system includes three distinct integration patterns:

1. Automated Build and Deploy Workflow - Weekly scheduled builds with manual trigger support (#9.1)
2. Continuous Testing Workflow - Automated Python tests on every push and pull request (#9.2)
3. Reusable GitHub Action - Packaged action for use in other repositories (#9.3)

All three patterns share the same underlying Docker infrastructure documented in Docker Multi-Stage Build, ensuring consistency between local development, testing, and production deployments.

For information about the Docker container architecture itself, see Docker Multi-Stage Build. For configuration options, see Configuration Reference.

CI/CD Architecture Overview

The CI/CD system consists of three independently triggered workflows that share a common Docker image but serve different purposes in the development lifecycle.

Diagram: CI/CD Trigger and Execution Patterns

graph TB
    subgraph "Trigger Sources"
        Schedule["Weekly Schedule\nSundays 00:00 UTC"]
ManualDispatch["Manual Dispatch\nworkflow_dispatch"]
Push["Push to main"]
PullRequest["Pull Request"]
ExternalRepo["External Repository\nuses action"]
end
    
    subgraph "Build and Deploy Workflow"
        BuildJob["build job\n.github/workflows/build-and-deploy.yml"]
ResolveStep["Resolve repository\nand book title"]
BuildxSetup["Setup Docker Buildx"]
DockerBuild1["Build deepwiki-to-mdbook\nDocker image"]
RunContainer1["Run docker container\nwith env vars"]
UploadArtifact["Upload Pages artifact\n./output/book"]
DeployJob["deploy job\nneeds: build"]
DeployPages["Deploy to GitHub Pages"]
end
    
    subgraph "Test Workflow"
        PytestJob["pytest job\n.github/workflows/tests.yml"]
SetupPython["Setup Python 3.12"]
InstallDeps["Install requirements.txt\nand pytest"]
RunTests["Run pytest python/tests/"]
end
    
    subgraph "Reusable Action"
        ActionDef["action.yml\ncomposite action"]
BuildImage["Build Docker image\nwith GITHUB_RUN_ID tag"]
RunContainer2["Run docker container\nwith input parameters"]
end
    
 
   Schedule --> BuildJob
 
   ManualDispatch --> BuildJob
 
   Push --> PytestJob
 
   PullRequest --> PytestJob
 
   ExternalRepo --> ActionDef
    
 
   BuildJob --> ResolveStep
 
   ResolveStep --> BuildxSetup
 
   BuildxSetup --> DockerBuild1
 
   DockerBuild1 --> RunContainer1
 
   RunContainer1 --> UploadArtifact
 
   UploadArtifact --> DeployJob
 
   DeployJob --> DeployPages
    
 
   PytestJob --> SetupPython
 
   SetupPython --> InstallDeps
 
   InstallDeps --> RunTests
    
 
   ActionDef --> BuildImage
 
   BuildImage --> RunContainer2

Sources: .github/workflows/build-and-deploy.yml:1-90 .github/workflows/tests.yml:1-26 action.yml:1-53

Workflow Triggers and Scheduling

The system employs multiple trigger mechanisms to balance automation, manual control, and external integration:

Sources: .github/workflows/build-and-deploy.yml:3-9 .github/workflows/tests.yml:3-8

Shared Docker Infrastructure

All three CI/CD patterns build and execute the same Docker image, ensuring consistent behavior across environments. The image build process follows this pattern:

Diagram: Docker Image Build and Execution Flow

graph LR
    subgraph "Source Files"
        Dockerfile["Dockerfile\nMulti-stage build"]
PythonScripts["Python scripts\ndeepwiki-scraper.py\nprocess-template.py"]
ShellScripts["Shell scripts\nbuild-docs.sh"]
Templates["templates/\nheader.html, footer.html"]
end
    
    subgraph "Build Stage"
        BuildCommand["docker build -t\ndeepwiki-to-mdbook"]
RustStage["Stage 1: rust:latest\nBuild mdBook binaries"]
PythonStage["Stage 2: python:3.12-slim\nInstall Python deps\nCopy executables"]
end
    
    subgraph "Execution"
        RunCommand["docker run --rm"]
EnvVars["Environment variables\nREPO, BOOK_TITLE, etc."]
VolumeMount["-v output:/output"]
Entrypoint["CMD: build-docs.sh"]
end
    
    subgraph "Output"
        OutputDir["./output/book/\nHTML documentation"]
MarkdownDir["./output/markdown/\nEnhanced markdown"]
end
    
 
   Dockerfile --> BuildCommand
 
   PythonScripts --> BuildCommand
 
   ShellScripts --> BuildCommand
 
   Templates --> BuildCommand
    
 
   BuildCommand --> RustStage
 
   RustStage --> PythonStage
 
   PythonStage --> RunCommand
    
 
   EnvVars --> RunCommand
 
   VolumeMount --> RunCommand
 
   RunCommand --> Entrypoint
    
 
   Entrypoint --> OutputDir
 
   Entrypoint --> MarkdownDir

Sources: .github/workflows/build-and-deploy.yml:60-72 action.yml:30-52

Permissions and Concurrency Control

The Build and Deploy workflow requires elevated permissions for GitHub Pages deployment, while the Test workflow operates with default read permissions.

Build and Deploy Permissions

Concurrency Management

The Build and Deploy workflow enforces single-instance execution to prevent deployment conflicts:

This configuration ensures that if a build is already running, subsequent triggers will wait rather than canceling the in-progress deployment.

Sources: .github/workflows/build-and-deploy.yml:11-20

Environment Variable Passing

All CI/CD patterns pass configuration to the Docker container via environment variables. The Build and Deploy workflow includes a resolution step that provides defaults when manual inputs are not specified:

Diagram: Environment Variable Resolution Flow

graph TD
    subgraph "Input Resolution"
        ManualInput["workflow_dispatch inputs\nrepo, book_title"]
CheckRepo{"repo input\nprovided?"}
CheckTitle{"book_title input\nprovided?"}
UseInputRepo["Use manual input"]
UseGitHubRepo["Use github.repository"]
UseInputTitle["Use manual input"]
GenTitle["Generate title from\nrepository name"]
end
    
    subgraph "Docker Execution"
        RepoVar["-e REPO"]
TitleVar["-e BOOK_TITLE"]
DockerRun["docker run deepwiki-to-mdbook"]
end
    
    subgraph "build-docs.sh"
        ReadEnv["Read environment variables"]
FetchWiki["Fetch from DeepWiki"]
BuildBook["Build mdBook"]
end
    
 
   ManualInput --> CheckRepo
 
   CheckRepo -->|Yes| UseInputRepo
 
   CheckRepo -->|No| UseGitHubRepo
    
 
   ManualInput --> CheckTitle
 
   CheckTitle -->|Yes| UseInputTitle
 
   CheckTitle -->|No| GenTitle
    
 
   UseInputRepo --> RepoVar
 
   UseGitHubRepo --> RepoVar
 
   UseInputTitle --> TitleVar
 
   GenTitle --> TitleVar
    
 
   RepoVar --> DockerRun
 
   TitleVar --> DockerRun
 
   DockerRun --> ReadEnv
 
   ReadEnv --> FetchWiki
 
   FetchWiki --> BuildBook

Sources: .github/workflows/build-and-deploy.yml:30-58 .github/workflows/build-and-deploy.yml:66-72

Job Dependencies and Artifact Flow

The Build and Deploy workflow uses a two-job structure with explicit dependency ordering:

1. Build Job - Executes Docker container, uploads Pages artifact
2. Deploy Job - Depends on build completion, deploys to GitHub Pages

The jobs communicate via the GitHub Actions artifact system:

Sources: .github/workflows/build-and-deploy.yml:74-89

Integration Points Summary

The following table summarizes how external systems interact with the CI/CD infrastructure:

Sources: .github/workflows/build-and-deploy.yml:1-90 action.yml:1-53

Related Documentation

• Build and Deploy Workflow - Detailed coverage of the weekly build workflow
• Test Workflow - Python test execution and validation
• GitHub Action - Using the action in external repositories
• Configuration Reference - Complete environment variable documentation
• Docker Multi-Stage Build - Container architecture details

Dismiss

Refresh this wiki

Enter email to refresh