Skip to content

petri: add NVMe emulator support#3368

Open
babayet2 wants to merge 2 commits intomicrosoft:mainfrom
babayet2:nvme-petri
Open

petri: add NVMe emulator support#3368
babayet2 wants to merge 2 commits intomicrosoft:mainfrom
babayet2:nvme-petri

Conversation

@babayet2
Copy link
Copy Markdown
Collaborator

Adds coverage for HyperV NVMe devices leveraging a closed source emulator

This test is currently marked as unstable for two reasons:

  1. The runners that pick up this job are not yet guaranteed to have NVMe emulators and setup scripts
  2. This test only passes locally when we force-enable bounce buffering. This is being investigated separately as a potential emulator bug, getting this running in CI will be a useful signal in the meantime

Copilot AI review requested due to automatic review settings April 24, 2026 05:12
@babayet2 babayet2 requested a review from a team as a code owner April 24, 2026 05:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Petri/Hyper-V support for attaching an NVMe emulator-backed disk and relaying it to the VTL0 guest via OpenHCL storvsp, along with an (unstable) x86_64 storage test that exercises the flow.

Changes:

  • Add an unstable x86_64 storage test that uses an NVMe emulator-backed VHD and validates guest I/O via storvsp.
  • Extend Petri VM config/builder to carry NVMe-emulator-to-SCSI LUN mapping intent.
  • Add Hyper-V backend + PowerShell plumbing to attach NVMe emulator devices post-VM-creation and patch VTL2 settings with resolved VSIDs + injected LUNs.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
vmm_tests/vmm_tests/tests/tests/x86_64/storage.rs Adds an unstable Hyper-V OpenHCL UEFI test covering NVMe-emulator-backed storage relayed via storvsp.
petri/src/vm/openvmm/construct.rs Ignores the new NVMe emulator mapping field for OpenVMM backend construction (Hyper-V only).
petri/src/vm/mod.rs Adds config field + builder API for NVMe emulator LUN mappings and a post-creation runtime config hook.
petri/src/vm/hyperv/powershell.rs Introduces env-var driven script discovery + PowerShell invocation to attach NVMe emulator devices.
petri/src/vm/hyperv/mod.rs Implements Hyper-V backend NVMe emulator attachment, VSID reconciliation, and VTL2 settings injection.

Comment thread vmm_tests/vmm_tests/tests/tests/x86_64/storage.rs
Comment thread vmm_tests/vmm_tests/tests/tests/x86_64/storage.rs Outdated
Comment on lines +249 to +268
crate::VmbusStorageType::Nvme => {
let mut vhd_paths = Vec::new();
for Drive { disk, is_dvd: _ } in controller.drives.values() {
let vhd = petri_disk_to_hyperv(disk.as_ref(), &temp_dir).await?;
if let Some(path) = vhd {
vhd_paths.push(path);
}
}
let vtl_num = match controller.target_vtl {
crate::Vtl::Vtl0 => 0u8,
crate::Vtl::Vtl2 => 2u8,
_ => {
anyhow::bail!(
"unsupported VTL {:?} for NVMe emulator device",
controller.target_vtl
)
}
};
nvme_emulator_attachments.push((*vsid, controller.clone(), vhd_paths, vtl_num));
}
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

controller.drives is a HashMap, so iterating values() yields a nondeterministic ordering. If the NVMe emulator script assigns namespace IDs based on the order of VhdPaths, this can attach the wrong VHD to a given NSID (especially when multiple namespaces are configured). Build vhd_paths in a deterministic order based on the drive key (NSID), and validate that each entry is a non-DVD drive with a backing disk (otherwise error out instead of silently skipping).

Copilot uses AI. Check for mistakes.
Comment on lines +418 to +425
.unwrap_or_else(|| {
tracing::warn!(
nvme_vsid = %mapping.nvme_vsid,
"NVMe emulator LUN mapping references unknown VSID; \
using placeholder (this will likely fail)"
);
mapping.nvme_vsid
});
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When an NvmeEmulatorLunMapping references a VSID that wasn't attached, the code logs a warning and continues with the placeholder VSID, which will likely fail later in a harder-to-debug way. Consider failing early (e.g., anyhow::bail!/ensure!) when the mapping references an unknown placeholder VSID.

Suggested change
.unwrap_or_else(|| {
tracing::warn!(
nvme_vsid = %mapping.nvme_vsid,
"NVMe emulator LUN mapping references unknown VSID; \
using placeholder (this will likely fail)"
);
mapping.nvme_vsid
});
.ok_or_else(|| {
anyhow::anyhow!(
"NVMe emulator LUN mapping references unknown placeholder VSID {} \
for controller {} LUN {} namespace {}",
mapping.nvme_vsid,
mapping.scsi_controller_instance_id,
mapping.scsi_lun,
mapping.nvme_namespace_id,
)
})?;

Copilot uses AI. Check for mistakes.
Comment on lines +769 to +778
controller.luns.push(
Vtl2LunBuilder::disk()
.with_location(scsi_lun)
.with_physical_device(Vtl2StorageBackingDeviceBuilder::new(
ControllerType::Nvme,
actual_vsid,
nvme_nsid,
))
.build(),
);
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inject_nvme_emulator_lun always appends a new LUN. If the target SCSI controller already has a LUN with the same location, this will create duplicates and can lead to confusing behavior. Consider checking for an existing LUN at scsi_lun and either replacing it or returning an error.

Suggested change
controller.luns.push(
Vtl2LunBuilder::disk()
.with_location(scsi_lun)
.with_physical_device(Vtl2StorageBackingDeviceBuilder::new(
ControllerType::Nvme,
actual_vsid,
nvme_nsid,
))
.build(),
);
let lun = Vtl2LunBuilder::disk()
.with_location(scsi_lun)
.with_physical_device(Vtl2StorageBackingDeviceBuilder::new(
ControllerType::Nvme,
actual_vsid,
nvme_nsid,
))
.build();
if let Some(existing_lun) = controller.luns.iter_mut().find(|lun| lun.location == scsi_lun) {
tracing::warn!(
%scsi_controller_id,
scsi_lun,
"replacing existing VTL2 SCSI LUN while injecting NVMe emulator LUN"
);
*existing_lun = lun;
} else {
controller.luns.push(lun);
}

Copilot uses AI. Check for mistakes.
Comment on lines +1246 to +1253
/// Environment variable name for the directory containing NVMe emulator
/// scripts (Register-NvmeEmulator.ps1, Add-NvmeEmulatorDevice.ps1,
/// NvmeEmulatorCommon.psm1).
pub const PETRI_NVME_EMULATOR_SCRIPTS_DIR: &str = "PETRI_NVME_EMULATOR_SCRIPTS_DIR";

/// Returns the path to the NVMe emulator scripts directory, read from the
/// `PETRI_NVME_EMULATOR_SCRIPTS_DIR` environment variable. Returns an error if the
/// env var is unset or the required scripts are missing.
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New environment variable PETRI_NVME_EMULATOR_SCRIPTS_DIR is required to run the NVMe emulator-backed tests. Please add/update developer documentation (likely Guide/src/dev_guide/tests/vmm.md or related) to describe how to set it up and what scripts are expected in that directory.

Copilot uses AI. Check for mistakes.
Comment on lines +1288 to +1295
let vhd_args: Vec<String> = vhd_paths.iter().map(|p| p.display().to_string()).collect();
let vhd_refs: Vec<&str> = vhd_args.iter().map(|s| s.as_str()).collect();
let script_str = format!("& \"{}\"", script.display());
let output = run_host_cmd(
PowerShellBuilder::new()
.cmdlet(script_str)
.arg("VmName", vm_name)
.arg("VhdPaths", ps::Array::new(&vhd_refs))
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

run_add_nvme_emulator_device converts PathBufs to String via display() and then builds &str references, which adds allocations and can be lossy. Since powershell_builder supports Path/PathBuf directly, consider passing vhd_paths directly to ps::Array::new(...) to avoid conversion.

Suggested change
let vhd_args: Vec<String> = vhd_paths.iter().map(|p| p.display().to_string()).collect();
let vhd_refs: Vec<&str> = vhd_args.iter().map(|s| s.as_str()).collect();
let script_str = format!("& \"{}\"", script.display());
let output = run_host_cmd(
PowerShellBuilder::new()
.cmdlet(script_str)
.arg("VmName", vm_name)
.arg("VhdPaths", ps::Array::new(&vhd_refs))
let script_str = format!("& \"{}\"", script.display());
let output = run_host_cmd(
PowerShellBuilder::new()
.cmdlet(script_str)
.arg("VmName", vm_name)
.arg("VhdPaths", ps::Array::new(vhd_paths))

Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 24, 2026 05:26
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Comment on lines +249 to +256
crate::VmbusStorageType::Nvme => {
let mut vhd_paths = Vec::new();
for Drive { disk, is_dvd: _ } in controller.drives.values() {
let vhd = petri_disk_to_hyperv(disk.as_ref(), &temp_dir).await?;
if let Some(path) = vhd {
vhd_paths.push(path);
}
}
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For NVMe controllers, controller.drives is a HashMap<u32, Drive>, but this code builds vhd_paths by iterating controller.drives.values(). That drops the namespace IDs and also makes the VHD ordering nondeterministic, which can cause the emulator to assign namespaces to the wrong VHD (breaking nvme_namespace_id/LUN mapping and making tests flaky). Build the VHD list in a deterministic order keyed by the namespace ID (and validate the IDs match what the PowerShell script expects), rather than iterating values().

Copilot uses AI. Check for mistakes.
Comment thread petri/src/vm/mod.rs
Comment on lines +1494 to +1509
pub fn add_nvme_emulator_lun_mapping(
mut self,
nvme_vsid: Guid,
nvme_namespace_id: u32,
scsi_controller_instance_id: Guid,
scsi_lun: u32,
) -> Self {
self.config
.nvme_emulator_lun_mappings
.push(NvmeEmulatorLunMapping {
nvme_vsid,
nvme_namespace_id,
scsi_controller_instance_id,
scsi_lun,
});
self
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add_nvme_emulator_lun_mapping only records the mapping, but it doesn’t ensure that OpenHCL VTL2 settings are initialized in the firmware config. The Hyper-V backend’s pre-start fixup path currently relies on firmware.openhcl_config().vtl2_settings being Some(...) to push corrected settings into the VM, so a caller that adds a mapping without otherwise touching VTL2 settings can end up with the mapping never being applied to the running VM. Consider initializing vtl2_settings here (e.g., via self.config.firmware.vtl2_settings() / default settings) and/or rejecting this API for non-OpenHCL firmware.

Copilot uses AI. Check for mistakes.
Comment on lines +1246 to +1259
/// Environment variable name for the directory containing NVMe emulator
/// scripts (Register-NvmeEmulator.ps1, Add-NvmeEmulatorDevice.ps1,
/// NvmeEmulatorCommon.psm1).
pub const PETRI_NVME_EMULATOR_SCRIPTS_DIR: &str = "PETRI_NVME_EMULATOR_SCRIPTS_DIR";

/// Returns the path to the NVMe emulator scripts directory, read from the
/// `PETRI_NVME_EMULATOR_SCRIPTS_DIR` environment variable. Returns an error if the
/// env var is unset or the required scripts are missing.
pub fn get_nvme_emulator_scripts_dir() -> anyhow::Result<PathBuf> {
let dir = std::env::var(PETRI_NVME_EMULATOR_SCRIPTS_DIR).map_err(|_| {
anyhow::anyhow!(
"NVMe emulator scripts not found. Set {PETRI_NVME_EMULATOR_SCRIPTS_DIR} to the \
directory containing NVMe emulation management scripts \
(Register-NvmeEmulator.ps1, Add-NvmeEmulatorDevice.ps1, NvmeEmulatorCommon.psm1)."
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This introduces a new Hyper-V/Petri configuration env var (PETRI_NVME_EMULATOR_SCRIPTS_DIR) that’s required to run the NVMe emulator path. The repo Guide already documents several PETRI_* variables for running VMM tests; please add this variable (and the required script names) to the relevant Guide page so others can run the new test locally/under CI images without reading code.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants