Add visualizers and scene data provider#4
Open
matthewtrepte wants to merge 46 commits intoooctipus:develop_simulation_contextfrom
Open
Conversation
matthewtrepte
requested review from
kellyguo11,
ooctipus and
pascal-roth
as code owners
matthewtrepte
changed the base branch from
main
to
develop_simulation_context
github-actions
Bot
added
enhancement
github-actions
Bot
added
documentation
ooctipus
force-pushed
the
develop_simulation_context
branch
from
ba468e6 to
07a8e1f
Compare
ooctipus
pushed a commit
that referenced
this pull request
May 7, 2026
…ic_write (isaac-sim#5380) ## Summary Replace the `sync_usd_on_fabric_write` workaround in `FabricFrameView` with proper `PrepareForReuse()` calls on the Fabric `PrimSelection`. This tells the renderer (FSD/Storm) that Fabric data has changed, so the next rendered frame reflects updated transforms — eliminating the need to copy Fabric writes back to USD. ## Motivation The existing `sync_usd_on_fabric_write` flag worked by mirroring every Fabric write back to USD, which defeated the performance benefits of Fabric. With `PrepareForReuse()`, the rendering pipeline is properly notified of Fabric data changes without any USD writeback. Additionally, the old code incorrectly fell back to USD for CPU devices — Warp handles CPU Fabric buffers correctly, so the fallback was unnecessary. This addresses two of the issues raised in @pbarejko Piotr's review of PR isaac-sim#4923: - **Issue #1** (USD write-back): Fabric writes no longer sync back to USD - **Issue #4** (PrepareForReuse): Renderer notification via `PrepareForReuse()` instead of USD writeback ## Changes ### Core (FabricFrameView) - Call `_prepare_for_reuse()` in write paths (`set_world_poses`, `set_scales`) to notify the renderer - Remove `sync_usd_on_fabric_write` parameter (accepted via `**kwargs` for backward compat) - Remove incorrect CPU/device fallback warnings — Warp handles CPU Fabric buffers correctly - Add `_rebuild_fabric_arrays()` for topology change recovery when `PrepareForReuse()` returns True, with assertion guarding the prim-count invariant ### Camera - Remove `sync_usd_on_fabric_write=True` from FrameView construction in `camera.py` ## Benchmark Results 1024 prims, 50 iterations, NVIDIA L40 GPU: | Operation | USD (ms) | Fabric (ms) | Speedup | |---|---|---|---| | Get World Poses | 14.71 | 0.07 | **203x** | | Set World Poses | 40.75 | 0.16 | **259x** | | Interleaved Set→Get | 55.90 | 0.24 | **232x** | | Get Local Poses | 11.08 | 11.12 | 1.0x | | Set Local Poses | 16.14 | 16.28 | 1.0x | Local poses fall back to USD (expected — Fabric only accelerates world poses via `omni:fabric:worldMatrix`). ## Tests Added | Test | What it validates | |------|------------------| | `test_camera_pose_update_reflected_in_render` | Camera pose changes propagate to rendered depth (close vs far) for CPU/GPU, tiled/non-tiled | | `test_fabric_set_world_does_not_write_back_to_usd` | Fabric writes stay in Fabric, USD prim unchanged | | `test_set_world_updates_local` (xfail) | Documents Issue #5: `set_world_poses` doesn't update local pose in Fabric mode | ## Test Results | Test Suite | Passed | Skipped | Xfailed | Total | |---|---|---|---|---| | Fabric contract tests (`test_views_xform_prim_fabric.py`) | 17 | 16 | 1 | 34 | | USD contract tests (`test_views_xform_prim.py`) | 45 | 0 | 0 | 45 | | Camera render test (`test_tiled_camera.py`) | 8 | 0 | 0 | 8 | ## Type of change - Performance improvement (removes redundant USD writeback on Fabric operations) ## Checklist - [x] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there *No doc changes needed (parameter wasn't referenced in any docs)*
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adding a scene data provider interface layer which shares simulation data from any physics backend to any renderer or visualizer.
Also, this change includes initial implementations for visualizers which use and test the interface layer.
Type of change
Screenshots
Please attach before and after screenshots of the change if applicable.
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there