Skip to content

ci: move fio steps into separate job#185

Open
riley-dixon wants to merge 5 commits intodevelopfrom
rildixon/ci-fio-integration
Open

ci: move fio steps into separate job#185
riley-dixon wants to merge 5 commits intodevelopfrom
rildixon/ci-fio-integration

Conversation

@riley-dixon
Copy link
Collaborator

@riley-dixon riley-dixon commented Feb 12, 2026

Breaks apart our monolithic system test workflow to separate the hipFile system tests from the FIO tests.
The FIO job now consumes the packages built by the ROCm/fio build workflow. This is the same workflow that creates and publishes the fio packages with hipfile support.

AIHIPFILE-121

@riley-dixon riley-dixon self-assigned this Feb 12, 2026
Copilot AI review requested due to automatic review settings February 12, 2026 21:24
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the AIS system testing workflow by extracting FIO-related test steps from the main run_system_tests job into a separate run_FIO_tests job. The motivation (per the PR description) is to eventually use FIO packages produced by ROCm/fio instead of building FIO separately, though this PR currently still builds FIO from source as an intermediate step.

Changes:

  • Removed FIO repository checkout and build directory creation from the run_system_tests job
  • Added cleanup steps to the run_system_tests job to ensure proper resource cleanup
  • Created a new run_FIO_tests job that checks out the FIO repository and runs FIO-based tests

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@riley-dixon riley-dixon force-pushed the rildixon/ci-fio-integration branch from 03afdaa to a2b2af8 Compare February 12, 2026 22:21
Copilot AI review requested due to automatic review settings February 17, 2026 23:36
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

.github/workflows/test-ais-system.yml:306

  • The PR description states "TBD - Use the FIO packages produced by ROCm/fio instead of building it separately here", but the implementation still builds FIO from source in the run_FIO_tests job (lines 283-306). The current changes create a build_FIO job that isn't being used, making this change incomplete. The implementation should either: (1) download and use pre-built FIO packages from the build_FIO job, or (2) wait to introduce the build_FIO job until the integration is complete.
      - name: Configure fio
        run: |
          docker exec \
            -t \
            -w /ais/fio/build \
            "${AIS_CONTAINER_NAME}" \
            /bin/bash -c '
              ROCM=/opt/rocm-${ROCM_VERSION} \
              HIPFILE=/ais/hipFile \
              HIPFILELIB=${HIPFILE}/build/src/amd_detail/ \
              HIP_PLATFORM=amd \
              CFLAGS="-I${ROCM}/include" \
              LDFLAGS="-L${ROCM}/lib -Wl,-rpath,${ROCM}/lib" \
              ../configure --enable-libhipfile
            '
      - name: Build fio
        run: |
          docker exec \
            -t \
            -w /ais/fio/build \
            "${AIS_CONTAINER_NAME}" \
            /bin/bash -c '
              make -j
            '

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@riley-dixon riley-dixon force-pushed the rildixon/ci-fio-integration branch from 454445d to 89335b4 Compare February 17, 2026 23:45
Copilot AI review requested due to automatic review settings February 19, 2026 01:17
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

.github/workflows/test-ais-system.yml:303

  • The PR description states the motivation as "TBD - Use the FIO packages produced by ROCm/fio instead of building it separately here", suggesting the goal is to use pre-built FIO packages. However, the run_FIO_tests job still builds fio from source (steps "Configure fio" and "Build fio" at lines 280-303).

If the intention is to use packages from the build_FIO reusable workflow, there should be steps to download those artifacts instead of building from source. If building from source is still intended, the PR description should be updated to reflect the actual changes being made.

      - name: Configure fio
        run: |
          docker exec \
            -t \
            -w /ais/fio/build \
            "${AIS_CONTAINER_NAME}" \
            /bin/bash -c '
              ROCM=/opt/rocm-${ROCM_VERSION} \
              HIPFILE=/ais/hipFile \
              HIPFILELIB=${HIPFILE}/build/src/amd_detail/ \
              HIP_PLATFORM=amd \
              CFLAGS="-I${ROCM}/include" \
              LDFLAGS="-L${ROCM}/lib -Wl,-rpath,${ROCM}/lib" \
              ../configure --enable-libhipfile
            '
      - name: Build fio
        run: |
          docker exec \
            -t \
            -w /ais/fio/build \
            "${AIS_CONTAINER_NAME}" \
            /bin/bash -c '
              make -j
            '

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@riley-dixon riley-dixon force-pushed the rildixon/ci-fio-integration branch 4 times, most recently from f0dfb0f to 98d9f84 Compare February 24, 2026 21:17
This is in preparation for calling out to the ROCm/fio workflow
to build and package FIO with hipFile support. This new workflow
will then install the produced FIO packages instead of compiling
FIO separately. This lets us sanity check the FIO packages we
produce.
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@riley-dixon riley-dixon marked this pull request as ready for review February 26, 2026 17:45
@riley-dixon
Copy link
Collaborator Author

This PR needs ROCm/fio#5 to be accepted and merged first so that the targetted workflow in build_FIO exists on the hipFile branch within ROCm/fio.

Consume the packages produced by the ROCm/fio workflow instead
of building the library directly.

On Fedora-based distros, FIO's IO engines are dynamically linked
instead of built into the FIO executable. As a result, we need to
install multiple packages. Admittedly, it does make some steps
look more complex than they actually are...

(I would prefer if we could skip the cache when installing
FIO, but there are additional dependencies that FIO relies
upon.)
Most ROCm libraries don't seem to be concerned with RPM AutoReq's
AutoProv's. There are libs that do enable these features, but they
also manage CPACK packaging directly instead of through ROCm-CMake.
@riley-dixon riley-dixon force-pushed the rildixon/ci-fio-integration branch from 3902c07 to 19ff80a Compare February 26, 2026 18:45
Copilot AI review requested due to automatic review settings February 26, 2026 18:45
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +319 to +328
${{
case(
inputs.platform == 'rocky',
'docker cp \
"${GITHUB_WORKSPACE}/${FIO_HIPFILE_ENGINE_PKG_FILENAME}" \
"${AIS_CONTAINER_NAME}:/root"
',
''
)
}}
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using ${{ ... }} to inject a multi-line shell fragment (the conditional docker cp for Rocky) makes the run script hard to reason about and depends on the (likely unsupported) case(...) expression. Prefer a separate step with if: inputs.platform == 'rocky' for the second package copy, or do the conditional inside bash.

Suggested change
${{
case(
inputs.platform == 'rocky',
'docker cp \
"${GITHUB_WORKSPACE}/${FIO_HIPFILE_ENGINE_PKG_FILENAME}" \
"${AIS_CONTAINER_NAME}:/root"
',
''
)
}}
- name: Copy the FIO hipFile engine package into the container (rocky only)
if: inputs.platform == 'rocky'
run: |
docker cp \
"${GITHUB_WORKSPACE}/${{ needs.build_FIO.outputs.fio_pkg_fedora_hipfile_engine_filename }}" \
"${AIS_CONTAINER_NAME}:/root"

Copilot uses AI. Check for mistakes.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll change this if any humans agree with this take. I think running this copy in a single step looks better from a CI summary perspective.

Copilot AI review requested due to automatic review settings February 26, 2026 19:03
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@riley-dixon riley-dixon force-pushed the rildixon/ci-fio-integration branch from edd10ca to 5af116e Compare February 26, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants