Fix describe remote web content discovery and add regression tests#901
Open
lapfelix wants to merge 1 commit intofacebook:mainfrom
Open
Fix describe remote web content discovery and add regression tests#901lapfelix wants to merge 1 commit intofacebook:mainfrom
lapfelix wants to merge 1 commit intofacebook:mainfrom
Conversation
|
@lapfelix This is exactly what I've been looking for! Appreciate the work on this, it works well for me. Really hope it could be merged into the main branch. Have you looked into interacting with the "Sign in with Apple" modals? idb seems pretty unreliable in describing them and I'm wondering if it could be improved in a similar way? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
idb ui describewas not reliably returning accessibility elements inside embedded web content (WebView), even when those elements were present and VoiceOver-accessible.Root causes:
describepath did not enable remote-content discovery options.AXWeb*container roles, which could overstate “already covered” area and suppress remote probing.Implemented changes:
CompanionLib/FBIDBCommandExecutor.m: enabled remote discovery for describe calls by settingcollectFrameCoverageand defaultremoteContentOptions.FBSimulatorControl/Commands/FBSimulatorAccessibilityCommands.m: excluded web container roles (AXWeb*/Web*) from base frame coverage; added near-full-coverage fallback probing (coverage >= 0.98); allowed seen/frontmost/unknown PID hits in that fallback path to recover hidden content; skipped near-fullscreen proxy hits to avoid duplicate wrappers; kept explicit unsigned index conversions needed for strict warning-as-error builds.FBSimulatorControlTests/Utilities/AccessibilityDoubles.handFBSimulatorControlTests/Utilities/AccessibilityDoubles.m: added translation-to-element mapping support in the translator double so tests can model frontmost-tree vs hit-tested remote elements.FBSimulatorControlTests/Tests/Unit/FBSimulatorAccessibilityCommandsTests.m: added targeted regression tests:testCoverageCalculationSkipsWebContentRoleElementstestRemoteDiscoveryFindsHiddenContentWhenCoverageIsNearFulltestRemoteDiscoverySkipsFullscreenProxyInNearFullCoverageModeFBSimulatorControlTests/Tests/Unit/FBSimulatorAccessibilityCommandsTests.m: retained test-onlyint64_tcasts needed to satisfy strict sign-compare checks in this toolchain.Compatibility:
Test Plan
Build tests target:
Run accessibility unit suite:
Result:
21 tests, 0 failures(includes all new regression tests).Build companion libraries:
Manual runtime verification on booted simulator:
Verified output includes embedded webview labels/element.
Related PRs
None.
I accepted the CLA