Skip to content

Feat: devcontainer pytest integration #110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
junkataoka wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Feat: devcontainer pytest integration #110

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 113 additions & 59 deletions .azuredevops/ado-ci-pipeline-ms-hosted.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,59 +1,113 @@
# Azure DevOps pipeline for CI (Microsoft-hosted version)
# As the Microsoft-hosted agent option has a limit of 10GB of storage for disk outputs from a pipeline,
# this causes an issue when the Docker images for modules under src require more than 10GB of storage.
# If you will run into space issues (or other limitations with a Microsoft hosted agent option outlined in
# https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml#capabilities-and-limitations),
# consider using the .azuredevops/ado-ci-pipeline-self-hosted.yml version or using scale set agents, see
# this link for more info: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/scale-set-agents?view=azure-devops
# Note that docker images will only be build for src directories that contain at least one test file, so the
# total space consumed by Docker builds will be dependent on which modules under src contain tests.
# For setting up the pipeline in ADO see:
# https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/pools-queues?view=azure-devops&tabs=yaml%2Cbrowser


trigger:
- main

pool:
vmImage: 'ubuntu-latest'

steps:
- task: UsePythonVersion@0
displayName: "Use Python 3.11"
inputs:
versionSpec: 3.11

# Using pip here instead of uv/uvx because Azure DevOps doesn't have native uv support
# (unlike GitHub Actions which has astral-sh/setup-uv), and installing uv just to run
# these tools would add unnecessary overhead.
- script: |
pip install ruff pytest-azurepipelines
displayName: "Install ruff and pytest-azurepipelines"

- bash: |
ruff check --output-format azure
displayName: "Run ruff linter"

- task: Bash@3
inputs:
targetType: 'filePath'
filePath: ci-tests.sh
env:
BUILD_ARTIFACTSTAGINGDIRECTORY: $(Build.ArtifactStagingDirectory)
displayName: "Run pytest in docker containers"

- task: PublishTestResults@2
inputs:
testResultsFiles: '**/test-results-*.xml'
searchFolder: $(Build.ArtifactStagingDirectory)
condition: succeededOrFailed()

# Publish code coverage results
- task: PublishCodeCoverageResults@1
inputs:
codeCoverageTool: 'Cobertura' # Available options: 'JaCoCo', 'Cobertura'
summaryFileLocation: '$(Build.ArtifactStagingDirectory)/coverage.xml'
pathToSources: src/
#reportDirectory: # Optional
#additionalCodeCoverageFiles: # Optional
failIfCoverageEmpty: false # Optional
# Azure DevOps pipeline for CI (Microsoft-hosted version)
# As the Microsoft-hosted agent option has a limit of 10GB of storage for disk outputs from a pipeline,
# this causes an issue when the Docker images for modules under src require more than 10GB of storage.
# If you will run into space issues (or other limitations with a Microsoft hosted agent option outlined in
# https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml#capabilities-and-limitations),
# consider using the .azuredevops/ado-ci-pipeline-self-hosted.yml version or using scale set agents, see
# this link for more info: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/scale-set-agents?view=azure-devops
# Note that docker images will only be build for src directories that contain at least one test file, so the
# total space consumed by Docker builds will be dependent on which modules under src contain tests.
# For setting up the pipeline in ADO see:
# https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/pools-queues?view=azure-devops&tabs=yaml%2Cbrowser
Comment thread
fujikosu marked this conversation as resolved.


trigger:
- main

pool:
vmImage: 'ubuntu-latest'

steps:
- task: UsePythonVersion@0
displayName: "Use Python 3.11"
inputs:
versionSpec: 3.11

# Using pip here instead of uv/uvx because Azure DevOps doesn't have native uv support
# (unlike GitHub Actions which has astral-sh/setup-uv), and installing uv just to run
# ruff would add unnecessary overhead.
- script: pip install ruff
displayName: "Install ruff"

- bash: ruff check --output-format azure
displayName: "Run ruff linter"

- script: |
cp .env.example .env
# devcontainer.json runArgs use relative --env-file paths (../../.env for
# src/* projects, ../.env for notebooks) that resolve from the devcontainer
# CLI's CWD (= the workspace folder). Pre-create them at the expected host
# paths so docker can find them.
cp .env.example ../.env
cp .env.example ../../.env
displayName: "Setup .env file"

- script: npm install -g @devcontainers/cli
displayName: "Install devcontainer CLI"

- bash: |
set -e
WORKSPACE_ROOT="$(pwd)"
devcontainer up --workspace-folder "$WORKSPACE_ROOT" \
--config src/sample_cpu_project/.devcontainer/devcontainer.json
devcontainer exec --workspace-folder "$WORKSPACE_ROOT" \
--config src/sample_cpu_project/.devcontainer/devcontainer.json \
bash -c '
cd /workspaces/repo/src/sample_cpu_project &&
COVERAGE_FILE=/tmp/.coverage pytest tests/ \
--junitxml=/tmp/test-results-cpu.xml \
-o junit_suite_name=cpu-project \
--doctest-modules \
--cov=. \
--cov-config=/workspaces/repo/pyproject.toml \
--cov-report=xml:/tmp/coverage-cpu.xml &&
sed -i "s|<package name=\".\"|<package name=\"cpu-project\"|g" /tmp/coverage-cpu.xml &&
sudo cp /tmp/test-results-cpu.xml /workspaces/repo/test-results-cpu.xml &&
sudo cp /tmp/coverage-cpu.xml /workspaces/repo/coverage-cpu.xml
'
displayName: "Build and run pytest (cpu-project)"

- bash: |
set -e
WORKSPACE_ROOT="$(pwd)"
devcontainer up --workspace-folder "$WORKSPACE_ROOT" \
--config src/sample_pytorch_gpu_project/.devcontainer/devcontainer.json
devcontainer exec --workspace-folder "$WORKSPACE_ROOT" \
--config src/sample_pytorch_gpu_project/.devcontainer/devcontainer.json \
bash -c '
cd /workspaces/repo/src/sample_pytorch_gpu_project &&
COVERAGE_FILE=/tmp/.coverage pytest tests/ \
--junitxml=/tmp/test-results-gpu.xml \
-o junit_suite_name=gpu-project \
--doctest-modules \
--cov=. \
--cov-config=/workspaces/repo/pyproject.toml \
--cov-report=xml:/tmp/coverage-gpu.xml &&
sed -i "s|<package name=\".\"|<package name=\"gpu-project\"|g" /tmp/coverage-gpu.xml &&
sudo cp /tmp/test-results-gpu.xml /workspaces/repo/test-results-gpu.xml &&
sudo cp /tmp/coverage-gpu.xml /workspaces/repo/coverage-gpu.xml
'
displayName: "Build and run pytest (gpu-project)"

- bash: |
set -e
WORKSPACE_ROOT="$(pwd)"
devcontainer up --workspace-folder "$WORKSPACE_ROOT" \
--config notebooks/.devcontainer/devcontainer.json
devcontainer exec --workspace-folder "$WORKSPACE_ROOT" \
--config notebooks/.devcontainer/devcontainer.json \
bash -c 'python --version && pytest --version && ruff --version'
displayName: "Build and smoke test notebooks devcontainer"

- task: PublishTestResults@2
inputs:
testResultsFiles: 'test-results-*.xml'
condition: succeededOrFailed()

# Publish code coverage results (PublishCodeCoverageResults@2 supports glob patterns and merges multiple files)
- task: PublishCodeCoverageResults@2
inputs:
summaryFileLocation: 'coverage-*.xml'
pathToSources: src/
failIfCoverageEmpty: false
condition: succeededOrFailed()
207 changes: 128 additions & 79 deletions .azuredevops/ado-ci-pipeline-self-hosted.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,79 +1,128 @@
# Azure DevOps pipeline for CI (self-hoseted version)
# As the Microsoft-hosted agent option has a limit of 10GB of storage for disk outputs from a pipeline,
# this causes an issue when the Docker images for modules under src require more than 10GB of storage.
# The self-hosted agent option allows the storage to be increased based on the VM size. This version
# includes extra clean-up and space management steps relating to docker builds, but it otherwise equivalent
# to the .azuredevops/ado-ci-pipeline-ms-hosted.yml version.
# For setting up a CI pipeline with a self-hosted Linux agent see:
# https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/v2-linux?view=azure-devops
# Note that the CI scripts that this pipeline runs (ci-tests.sh) is designed to be run on a Linux agent,
# but could be adapated to other OSs.


trigger:
- main

pool:
name: Default
demands:
- agent.name -equals mc-ubuntu-agent
workspace:
clean: all

steps:
- script: |
docker image prune -f
docker container prune -f
displayName: "Docker Cleanup"

- script: |
df -h
displayName: "Check agent VM space"

- task: UsePythonVersion@0
displayName: "Use Python 3.11"
inputs:
versionSpec: 3.11

# Using pip here instead of uv/uvx because Azure DevOps doesn't have native uv support
# (unlike GitHub Actions which has astral-sh/setup-uv), and installing uv just to run
# these tools would add unnecessary overhead.
- script: |
pip install ruff pytest-azurepipelines
displayName: "Install ruff and pytest-azurepipelines"

- task: UseDotNet@2
inputs:
packageType: 'sdk'
workingDirectory: "src/"
version: '6.x'

- bash: |
ruff check --output-format azure
displayName: "Run ruff linter"

- task: Bash@3
inputs:
targetType: 'filePath'
filePath: ci-tests.sh
displayName: "Run pytest in docker containers"

- task: PublishTestResults@2
inputs:
testResultsFiles: "/tmp/artifact_output/**/test-results-*.xml"
condition: succeededOrFailed()

# Publish code coverage results
- task: PublishCodeCoverageResults@1
inputs:
codeCoverageTool: 'Cobertura' # Available options: 'JaCoCo', 'Cobertura'
summaryFileLocation: '/tmp/artifact_output/coverage.xml'
pathToSources: src/
#reportDirectory: # Optional
#additionalCodeCoverageFiles: # Optional
failIfCoverageEmpty: false # Optional

- bash: |
sudo rm -rfv /home/azureuser/myagent/_work/* /home/azureuser/myagent/_work/.* || true
displayName: "Clean-up _work dir"
condition: always()
# Azure DevOps pipeline for CI (self-hosted version)
# As the Microsoft-hosted agent option has a limit of 10GB of storage for disk outputs from a pipeline,
# this causes an issue when the Docker images for modules under src require more than 10GB of storage.
# The self-hosted agent option allows the storage to be increased based on the VM size. This version
# includes extra clean-up and space management steps relating to docker builds, but it is otherwise equivalent
# to the .azuredevops/ado-ci-pipeline-ms-hosted.yml version.
# For setting up a CI pipeline with a self-hosted Linux agent see:
# https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/v2-linux?view=azure-devops


trigger:
- main

pool:
name: Default
demands:
- agent.name -equals mc-ubuntu-agent
workspace:
clean: all

steps:
- script: |
docker image prune -f
docker container prune -f
displayName: "Docker Cleanup"

- script: |
df -h
displayName: "Check agent VM space"

- task: UsePythonVersion@0
displayName: "Use Python 3.11"
inputs:
versionSpec: 3.11

# Using pip here instead of uv/uvx because Azure DevOps doesn't have native uv support
# (unlike GitHub Actions which has astral-sh/setup-uv), and installing uv just to run
# ruff would add unnecessary overhead.
- script: pip install ruff
displayName: "Install ruff"

- bash: ruff check --output-format azure
displayName: "Run ruff linter"

- script: |
cp .env.example .env
# devcontainer.json runArgs use relative --env-file paths (../../.env for
# src/* projects, ../.env for notebooks) that resolve from the devcontainer
# CLI's CWD (= the workspace folder). Pre-create them at the expected host
# paths so docker can find them.
cp .env.example ../.env
cp .env.example ../../.env
displayName: "Setup .env file"

- script: npm install -g @devcontainers/cli
displayName: "Install devcontainer CLI"

- bash: |
set -e
WORKSPACE_ROOT="$(pwd)"
devcontainer up --workspace-folder "$WORKSPACE_ROOT" \
--config src/sample_cpu_project/.devcontainer/devcontainer.json
devcontainer exec --workspace-folder "$WORKSPACE_ROOT" \
--config src/sample_cpu_project/.devcontainer/devcontainer.json \
bash -c '
cd /workspaces/repo/src/sample_cpu_project &&
COVERAGE_FILE=/tmp/.coverage pytest tests/ \
--junitxml=/tmp/test-results-cpu.xml \
-o junit_suite_name=cpu-project \
--doctest-modules \
--cov=. \
--cov-config=/workspaces/repo/pyproject.toml \
--cov-report=xml:/tmp/coverage-cpu.xml &&
sed -i "s|<package name=\".\"|<package name=\"cpu-project\"|g" /tmp/coverage-cpu.xml &&
sudo cp /tmp/test-results-cpu.xml /workspaces/repo/test-results-cpu.xml &&
sudo cp /tmp/coverage-cpu.xml /workspaces/repo/coverage-cpu.xml
'
displayName: "Build and run pytest (cpu-project)"

- bash: |
set -e
WORKSPACE_ROOT="$(pwd)"
devcontainer up --workspace-folder "$WORKSPACE_ROOT" \
--config src/sample_pytorch_gpu_project/.devcontainer/devcontainer.json
devcontainer exec --workspace-folder "$WORKSPACE_ROOT" \
--config src/sample_pytorch_gpu_project/.devcontainer/devcontainer.json \
bash -c '
cd /workspaces/repo/src/sample_pytorch_gpu_project &&
COVERAGE_FILE=/tmp/.coverage pytest tests/ \
--junitxml=/tmp/test-results-gpu.xml \
-o junit_suite_name=gpu-project \
--doctest-modules \
--cov=. \
--cov-config=/workspaces/repo/pyproject.toml \
--cov-report=xml:/tmp/coverage-gpu.xml &&
sed -i "s|<package name=\".\"|<package name=\"gpu-project\"|g" /tmp/coverage-gpu.xml &&
sudo cp /tmp/test-results-gpu.xml /workspaces/repo/test-results-gpu.xml &&
sudo cp /tmp/coverage-gpu.xml /workspaces/repo/coverage-gpu.xml
'
displayName: "Build and run pytest (gpu-project)"

- bash: |
set -e
WORKSPACE_ROOT="$(pwd)"
devcontainer up --workspace-folder "$WORKSPACE_ROOT" \
--config notebooks/.devcontainer/devcontainer.json
devcontainer exec --workspace-folder "$WORKSPACE_ROOT" \
--config notebooks/.devcontainer/devcontainer.json \
bash -c 'python --version && pytest --version && ruff --version'
displayName: "Build and smoke test notebooks devcontainer"

- task: PublishTestResults@2
inputs:
testResultsFiles: 'test-results-*.xml'
condition: succeededOrFailed()

# Publish code coverage results (PublishCodeCoverageResults@2 supports glob patterns and merges multiple files)
- task: PublishCodeCoverageResults@2
inputs:
summaryFileLocation: 'coverage-*.xml'
pathToSources: src/
failIfCoverageEmpty: false
condition: succeededOrFailed()

- bash: |
sudo rm -rfv /home/azureuser/myagent/_work/* /home/azureuser/myagent/_work/.* || true
displayName: "Clean-up _work dir"
condition: always()
Loading
Loading