Skip to content

Mount ReFS Dev Drive on Windows to speed up small-file I/O #5065

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pietern wants to merge 15 commits into from
Closed

Mount ReFS Dev Drive on Windows to speed up small-file I/O #5065

Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
0514262
ci: mount ReFS Dev Drive on Windows to speed up small-file I/O
pietern Apr 22, 2026
80ad9a4
Drop TEMP/TMP redirect from Dev Drive step
pietern Apr 22, 2026
3e90648
Junction the workspace onto the Dev Drive on Windows
pietern Apr 22, 2026
ecaeaa7
Revert "Junction the workspace onto the Dev Drive on Windows"
pietern Apr 22, 2026
2caaa88
Redirect TEMP + Go caches via C:\a\_fast junction
pietern Apr 22, 2026
e602ba3
Use directory symlink instead of junction for Dev Drive
pietern Apr 22, 2026
942125d
Revert "Use directory symlink instead of junction for Dev Drive"
pietern Apr 22, 2026
164569b
Revert "Redirect TEMP + Go caches via C:\a\_fast junction"
pietern Apr 22, 2026
0ebe781
ci: mount Dev Drive at C:\dd folder, junction workspace + redirect TEMP
pietern May 7, 2026
c6deaa7
ci: handle pre-existing workspace when junctioning to Dev Drive
pietern May 7, 2026
211541c
ci: move Dev Drive mount into push.yml jobs before actions/checkout
pietern May 7, 2026
6c119db
ci: cd out of GITHUB_WORKSPACE before deleting it on Windows
pietern May 7, 2026
73845ad
ci: mount Dev Drive volumes directly via Add-PartitionAccessPath
pietern May 7, 2026
07408b2
ci: skip checkout clean on Dev Drive to avoid System Volume Information
pietern May 7, 2026
d74e589
ci: strip System Volume Information from freshly-mounted ReFS workspace
pietern May 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
308 changes: 308 additions & 0 deletions .github/workflows/push.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,83 @@ jobs:
name: macos

steps:
# Mount a ReFS Dev Drive on Windows BEFORE the checkout so the checkout's
# files land on ReFS directly. The default C: drive on GH runners is
# ~4.3k IOPS; a Dev Drive is ~127k. Two access paths on the same volume:
# - $GITHUB_WORKSPACE for the checkout (mounted as folder, not a
# junction, so the runner's open handle on that dir is irrelevant)
# - C:\dd for TEMP and Go caches (separate namespace so it doesn't
# pollute the workspace tree)
# All paths start with C:\, so older databricks-bundles tests under
# bundle/python/* don't trip os.path.commonpath across drives.
- name: Mount ReFS Dev Drive (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'

# --- Volume A: workspace ---
$vhdA = "$env:RUNNER_TEMP\devdrive-ws.vhdx"
New-VHD -Path $vhdA -SizeBytes 15GB -Dynamic | Out-Null
$diskA = Mount-VHD -Path $vhdA -Passthru |
Initialize-Disk -PartitionStyle GPT -Passthru
$partA = New-Partition -DiskNumber $diskA.Number -UseMaximumSize -DriveLetter Y
try {
Format-Volume -DriveLetter Y -FileSystem ReFS -DevDrive `
-NewFileSystemLabel DevDriveWS -Confirm:$false -Force | Out-Null
} catch {
Format-Volume -DriveLetter Y -FileSystem ReFS `
-NewFileSystemLabel DevDriveWS -Confirm:$false -Force | Out-Null
}

$ws = $env:GITHUB_WORKSPACE
if ((Get-ChildItem -Force $ws | Measure-Object).Count -gt 0) {
throw "GITHUB_WORKSPACE ($ws) is not empty; this step must run before actions/checkout."
}
Add-PartitionAccessPath -DiskNumber $diskA.Number `
-PartitionNumber $partA.PartitionNumber -AccessPath $ws
Remove-PartitionAccessPath -DiskNumber $diskA.Number `
-PartitionNumber $partA.PartitionNumber -AccessPath 'Y:\'

# ReFS auto-creates "System Volume Information" at the volume root;
# it has restrictive ACLs that make scandir fail with EPERM, which
# trips actions/checkout. Take ownership and remove it.
$svi = Join-Path $ws 'System Volume Information'
if (Test-Path $svi) {
cmd /c "takeown /F `"$svi`" /A /R" | Out-Null
cmd /c "icacls `"$svi`" /grant Administrators:F /T /C" | Out-Null
Remove-Item -Path $svi -Recurse -Force -ErrorAction SilentlyContinue
}

# --- Volume B: TEMP + Go caches ---
$vhdB = "$env:RUNNER_TEMP\devdrive-cache.vhdx"
New-VHD -Path $vhdB -SizeBytes 10GB -Dynamic | Out-Null
$diskB = Mount-VHD -Path $vhdB -Passthru |
Initialize-Disk -PartitionStyle GPT -Passthru
$partB = New-Partition -DiskNumber $diskB.Number -UseMaximumSize -DriveLetter Z
try {
Format-Volume -DriveLetter Z -FileSystem ReFS -DevDrive `
-NewFileSystemLabel DevDriveCache -Confirm:$false -Force | Out-Null
} catch {
Format-Volume -DriveLetter Z -FileSystem ReFS `
-NewFileSystemLabel DevDriveCache -Confirm:$false -Force | Out-Null
}

$mount = 'C:\dd'
New-Item -ItemType Directory -Force -Path $mount | Out-Null
Add-PartitionAccessPath -DiskNumber $diskB.Number `
-PartitionNumber $partB.PartitionNumber -AccessPath $mount
Remove-PartitionAccessPath -DiskNumber $diskB.Number `
-PartitionNumber $partB.PartitionNumber -AccessPath 'Z:\'

New-Item -ItemType Directory -Path C:\dd\tmp,C:\dd\go-build,C:\dd\go-mod,C:\dd\go-tmp | Out-Null

"TMP=C:\dd\tmp" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"TEMP=C:\dd\tmp" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"GOCACHE=C:\dd\go-build" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"GOMODCACHE=C:\dd\go-mod" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"GOTMPDIR=C:\dd\go-tmp" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV

- name: Checkout repository and submodules
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

Expand Down Expand Up @@ -208,6 +285,83 @@ jobs:
# labels: windows-server-latest-large

steps:
# Mount a ReFS Dev Drive on Windows BEFORE the checkout so the checkout's
# files land on ReFS directly. The default C: drive on GH runners is
# ~4.3k IOPS; a Dev Drive is ~127k. Two access paths on the same volume:
# - $GITHUB_WORKSPACE for the checkout (mounted as folder, not a
# junction, so the runner's open handle on that dir is irrelevant)
# - C:\dd for TEMP and Go caches (separate namespace so it doesn't
# pollute the workspace tree)
# All paths start with C:\, so older databricks-bundles tests under
# bundle/python/* don't trip os.path.commonpath across drives.
- name: Mount ReFS Dev Drive (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'

# --- Volume A: workspace ---
$vhdA = "$env:RUNNER_TEMP\devdrive-ws.vhdx"
New-VHD -Path $vhdA -SizeBytes 15GB -Dynamic | Out-Null
$diskA = Mount-VHD -Path $vhdA -Passthru |
Initialize-Disk -PartitionStyle GPT -Passthru
$partA = New-Partition -DiskNumber $diskA.Number -UseMaximumSize -DriveLetter Y
try {
Format-Volume -DriveLetter Y -FileSystem ReFS -DevDrive `
-NewFileSystemLabel DevDriveWS -Confirm:$false -Force | Out-Null
} catch {
Format-Volume -DriveLetter Y -FileSystem ReFS `
-NewFileSystemLabel DevDriveWS -Confirm:$false -Force | Out-Null
}

$ws = $env:GITHUB_WORKSPACE
if ((Get-ChildItem -Force $ws | Measure-Object).Count -gt 0) {
throw "GITHUB_WORKSPACE ($ws) is not empty; this step must run before actions/checkout."
}
Add-PartitionAccessPath -DiskNumber $diskA.Number `
-PartitionNumber $partA.PartitionNumber -AccessPath $ws
Remove-PartitionAccessPath -DiskNumber $diskA.Number `
-PartitionNumber $partA.PartitionNumber -AccessPath 'Y:\'

# ReFS auto-creates "System Volume Information" at the volume root;
# it has restrictive ACLs that make scandir fail with EPERM, which
# trips actions/checkout. Take ownership and remove it.
$svi = Join-Path $ws 'System Volume Information'
if (Test-Path $svi) {
cmd /c "takeown /F `"$svi`" /A /R" | Out-Null
cmd /c "icacls `"$svi`" /grant Administrators:F /T /C" | Out-Null
Remove-Item -Path $svi -Recurse -Force -ErrorAction SilentlyContinue
}

# --- Volume B: TEMP + Go caches ---
$vhdB = "$env:RUNNER_TEMP\devdrive-cache.vhdx"
New-VHD -Path $vhdB -SizeBytes 10GB -Dynamic | Out-Null
$diskB = Mount-VHD -Path $vhdB -Passthru |
Initialize-Disk -PartitionStyle GPT -Passthru
$partB = New-Partition -DiskNumber $diskB.Number -UseMaximumSize -DriveLetter Z
try {
Format-Volume -DriveLetter Z -FileSystem ReFS -DevDrive `
-NewFileSystemLabel DevDriveCache -Confirm:$false -Force | Out-Null
} catch {
Format-Volume -DriveLetter Z -FileSystem ReFS `
-NewFileSystemLabel DevDriveCache -Confirm:$false -Force | Out-Null
}

$mount = 'C:\dd'
New-Item -ItemType Directory -Force -Path $mount | Out-Null
Add-PartitionAccessPath -DiskNumber $diskB.Number `
-PartitionNumber $partB.PartitionNumber -AccessPath $mount
Remove-PartitionAccessPath -DiskNumber $diskB.Number `
-PartitionNumber $partB.PartitionNumber -AccessPath 'Z:\'

New-Item -ItemType Directory -Path C:\dd\tmp,C:\dd\go-build,C:\dd\go-mod,C:\dd\go-tmp | Out-Null

"TMP=C:\dd\tmp" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"TEMP=C:\dd\tmp" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"GOCACHE=C:\dd\go-build" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"GOMODCACHE=C:\dd\go-mod" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"GOTMPDIR=C:\dd\go-tmp" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV

- name: Checkout repository and submodules
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

Expand Down Expand Up @@ -260,6 +414,83 @@ jobs:
labels: macos-latest

steps:
# Mount a ReFS Dev Drive on Windows BEFORE the checkout so the checkout's
# files land on ReFS directly. The default C: drive on GH runners is
# ~4.3k IOPS; a Dev Drive is ~127k. Two access paths on the same volume:
# - $GITHUB_WORKSPACE for the checkout (mounted as folder, not a
# junction, so the runner's open handle on that dir is irrelevant)
# - C:\dd for TEMP and Go caches (separate namespace so it doesn't
# pollute the workspace tree)
# All paths start with C:\, so older databricks-bundles tests under
# bundle/python/* don't trip os.path.commonpath across drives.
- name: Mount ReFS Dev Drive (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'

# --- Volume A: workspace ---
$vhdA = "$env:RUNNER_TEMP\devdrive-ws.vhdx"
New-VHD -Path $vhdA -SizeBytes 15GB -Dynamic | Out-Null
$diskA = Mount-VHD -Path $vhdA -Passthru |
Initialize-Disk -PartitionStyle GPT -Passthru
$partA = New-Partition -DiskNumber $diskA.Number -UseMaximumSize -DriveLetter Y
try {
Format-Volume -DriveLetter Y -FileSystem ReFS -DevDrive `
-NewFileSystemLabel DevDriveWS -Confirm:$false -Force | Out-Null
} catch {
Format-Volume -DriveLetter Y -FileSystem ReFS `
-NewFileSystemLabel DevDriveWS -Confirm:$false -Force | Out-Null
}

$ws = $env:GITHUB_WORKSPACE
if ((Get-ChildItem -Force $ws | Measure-Object).Count -gt 0) {
throw "GITHUB_WORKSPACE ($ws) is not empty; this step must run before actions/checkout."
}
Add-PartitionAccessPath -DiskNumber $diskA.Number `
-PartitionNumber $partA.PartitionNumber -AccessPath $ws
Remove-PartitionAccessPath -DiskNumber $diskA.Number `
-PartitionNumber $partA.PartitionNumber -AccessPath 'Y:\'

# ReFS auto-creates "System Volume Information" at the volume root;
# it has restrictive ACLs that make scandir fail with EPERM, which
# trips actions/checkout. Take ownership and remove it.
$svi = Join-Path $ws 'System Volume Information'
if (Test-Path $svi) {
cmd /c "takeown /F `"$svi`" /A /R" | Out-Null
cmd /c "icacls `"$svi`" /grant Administrators:F /T /C" | Out-Null
Remove-Item -Path $svi -Recurse -Force -ErrorAction SilentlyContinue
}

# --- Volume B: TEMP + Go caches ---
$vhdB = "$env:RUNNER_TEMP\devdrive-cache.vhdx"
New-VHD -Path $vhdB -SizeBytes 10GB -Dynamic | Out-Null
$diskB = Mount-VHD -Path $vhdB -Passthru |
Initialize-Disk -PartitionStyle GPT -Passthru
$partB = New-Partition -DiskNumber $diskB.Number -UseMaximumSize -DriveLetter Z
try {
Format-Volume -DriveLetter Z -FileSystem ReFS -DevDrive `
-NewFileSystemLabel DevDriveCache -Confirm:$false -Force | Out-Null
} catch {
Format-Volume -DriveLetter Z -FileSystem ReFS `
-NewFileSystemLabel DevDriveCache -Confirm:$false -Force | Out-Null
}

$mount = 'C:\dd'
New-Item -ItemType Directory -Force -Path $mount | Out-Null
Add-PartitionAccessPath -DiskNumber $diskB.Number `
-PartitionNumber $partB.PartitionNumber -AccessPath $mount
Remove-PartitionAccessPath -DiskNumber $diskB.Number `
-PartitionNumber $partB.PartitionNumber -AccessPath 'Z:\'

New-Item -ItemType Directory -Path C:\dd\tmp,C:\dd\go-build,C:\dd\go-mod,C:\dd\go-tmp | Out-Null

"TMP=C:\dd\tmp" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"TEMP=C:\dd\tmp" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"GOCACHE=C:\dd\go-build" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"GOMODCACHE=C:\dd\go-mod" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"GOTMPDIR=C:\dd\go-tmp" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV

- name: Checkout repository and submodules
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

Expand Down Expand Up @@ -312,6 +543,83 @@ jobs:
labels: macos-latest

steps:
# Mount a ReFS Dev Drive on Windows BEFORE the checkout so the checkout's
# files land on ReFS directly. The default C: drive on GH runners is
# ~4.3k IOPS; a Dev Drive is ~127k. Two access paths on the same volume:
# - $GITHUB_WORKSPACE for the checkout (mounted as folder, not a
# junction, so the runner's open handle on that dir is irrelevant)
# - C:\dd for TEMP and Go caches (separate namespace so it doesn't
# pollute the workspace tree)
# All paths start with C:\, so older databricks-bundles tests under
# bundle/python/* don't trip os.path.commonpath across drives.
- name: Mount ReFS Dev Drive (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'

# --- Volume A: workspace ---
$vhdA = "$env:RUNNER_TEMP\devdrive-ws.vhdx"
New-VHD -Path $vhdA -SizeBytes 15GB -Dynamic | Out-Null
$diskA = Mount-VHD -Path $vhdA -Passthru |
Initialize-Disk -PartitionStyle GPT -Passthru
$partA = New-Partition -DiskNumber $diskA.Number -UseMaximumSize -DriveLetter Y
try {
Format-Volume -DriveLetter Y -FileSystem ReFS -DevDrive `
-NewFileSystemLabel DevDriveWS -Confirm:$false -Force | Out-Null
} catch {
Format-Volume -DriveLetter Y -FileSystem ReFS `
-NewFileSystemLabel DevDriveWS -Confirm:$false -Force | Out-Null
}

$ws = $env:GITHUB_WORKSPACE
if ((Get-ChildItem -Force $ws | Measure-Object).Count -gt 0) {
throw "GITHUB_WORKSPACE ($ws) is not empty; this step must run before actions/checkout."
}
Add-PartitionAccessPath -DiskNumber $diskA.Number `
-PartitionNumber $partA.PartitionNumber -AccessPath $ws
Remove-PartitionAccessPath -DiskNumber $diskA.Number `
-PartitionNumber $partA.PartitionNumber -AccessPath 'Y:\'

# ReFS auto-creates "System Volume Information" at the volume root;
# it has restrictive ACLs that make scandir fail with EPERM, which
# trips actions/checkout. Take ownership and remove it.
$svi = Join-Path $ws 'System Volume Information'
if (Test-Path $svi) {
cmd /c "takeown /F `"$svi`" /A /R" | Out-Null
cmd /c "icacls `"$svi`" /grant Administrators:F /T /C" | Out-Null
Remove-Item -Path $svi -Recurse -Force -ErrorAction SilentlyContinue
}

# --- Volume B: TEMP + Go caches ---
$vhdB = "$env:RUNNER_TEMP\devdrive-cache.vhdx"
New-VHD -Path $vhdB -SizeBytes 10GB -Dynamic | Out-Null
$diskB = Mount-VHD -Path $vhdB -Passthru |
Initialize-Disk -PartitionStyle GPT -Passthru
$partB = New-Partition -DiskNumber $diskB.Number -UseMaximumSize -DriveLetter Z
try {
Format-Volume -DriveLetter Z -FileSystem ReFS -DevDrive `
-NewFileSystemLabel DevDriveCache -Confirm:$false -Force | Out-Null
} catch {
Format-Volume -DriveLetter Z -FileSystem ReFS `
-NewFileSystemLabel DevDriveCache -Confirm:$false -Force | Out-Null
}

$mount = 'C:\dd'
New-Item -ItemType Directory -Force -Path $mount | Out-Null
Add-PartitionAccessPath -DiskNumber $diskB.Number `
-PartitionNumber $partB.PartitionNumber -AccessPath $mount
Remove-PartitionAccessPath -DiskNumber $diskB.Number `
-PartitionNumber $partB.PartitionNumber -AccessPath 'Z:\'

New-Item -ItemType Directory -Path C:\dd\tmp,C:\dd\go-build,C:\dd\go-mod,C:\dd\go-tmp | Out-Null

"TMP=C:\dd\tmp" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"TEMP=C:\dd\tmp" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"GOCACHE=C:\dd\go-build" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"GOMODCACHE=C:\dd\go-mod" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV
"GOTMPDIR=C:\dd\go-tmp" | Out-File -Append -Encoding utf8 $env:GITHUB_ENV

- name: Checkout repository and submodules
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

Expand Down
Loading