Skip to content

Replace ScreenCaptureKit with SkyLight for tile previews (#21)#22

Merged
peterp merged 1 commit into
mainfrom
pp-fix-21-skylight-previews
May 14, 2026
Merged

Replace ScreenCaptureKit with SkyLight for tile previews (#21)#22
peterp merged 1 commit into
mainfrom
pp-fix-21-skylight-previews

Conversation

@peterp
Copy link
Copy Markdown
Owner

@peterp peterp commented May 14, 2026

Problem

The window picker grid used Apple's ScreenCaptureKit framework to render a live preview of every open window. That had three problems we wanted to address (issue #21):

  1. Starting a separate capture session per window is expensive. With many windows open, the capture setup was the slowest step of summoning the picker, and we had to add special locking last release just to keep it from crashing (issue Crash in Tile.start() — EXC_BAD_ACCESS in -[SCContentFilter initWithDesktopIndependentWindow:] (concurrent overlay install) #18).
  2. Each capture session ran continuously while the picker was open, which used noticeable CPU and battery once you had a dozen or more windows.
  3. The issue speculated that an internal Apple framework called SkyLight — the one Mission Control uses for its live previews — would let us skip the macOS Screen Recording permission and the recording indicator in the menu bar entirely. That part turned out to be wrong on current macOS, but we kept the migration for the other reasons.

Solution

  1. Stop using ScreenCaptureKit for live previews. Instead, ask the WindowServer for a fresh snapshot of each window through a private function in SkyLight, on a timer that fires fifteen times a second.
  2. Stop using ScreenCaptureKit to list the user's windows. Use the standard public window-listing function instead.
  3. Clean up the onboarding window: it used to be sized for two permissions with a big "Continue" button, but it now watches for the permission to be granted, automatically restarts the app the moment it flips on, and sizes itself to its content.

What this does not do: it does not remove the Screen Recording permission requirement, and it does not remove the menu-bar recording indicator while the picker is open. The SkyLight function still requires the permission on current macOS, and the system still attributes the capture to us in the indicator. Getting rid of those would mean a much larger change (mirroring each window's layer directly into our process), which we have not done here.

Technical Summary

  • New SkyLightCapture.swift: thin wrapper that looks up CGSMainConnectionID and CGSHWCaptureWindowList in /System/Library/PrivateFrameworks/SkyLight.framework using dlsym, so a future macOS that renames or removes those symbols degrades to a still-functional picker with no live thumbnails rather than a crash.
  • New WindowInfo.swift: plain struct populated from CGWindowListCopyWindowInfo + NSRunningApplication, replacing every place we used to depend on SCWindow.
  • Tile.swift was rewritten end-to-end. The SCStream / SCStreamOutput / SCStreamDelegate machinery, the stop-and-restart watchdog, the CVPixelBuffer to CGImage promotion, and the serialization queue from issue Crash in Tile.start() — EXC_BAD_ACCESS in -[SCContentFilter initWithDesktopIndependentWindow:] (concurrent overlay install) #18 are all gone. Capture is now a single DispatchSourceTimer that calls SkyLightCapture.captureImage(windowID:) and assigns the resulting CGImage to the tile's layer on the main queue. Net change: ~750 lines down to ~430.
  • Overlay.swift no longer imports ScreenCaptureKit. Window enumeration is WindowInfo.enumerate(). The prewarmShareable warm-up call and the SCShareableContent round trip are gone. The window-eligibility filter is rewritten against WindowInfo (and now requires kCGWindowLayer == 0, which SCShareableContent already implied).
  • main.swift drops its SCShareableContent.current warm-up.
  • Onboarding.swift: removes the explicit Continue button, replaces it with a 0.5 s poll on granted(), and on success relaunches the app via NSWorkspace.openApplication with createsNewApplicationInstance = true so any event tap re-registers under the fresh permission state. The window now sizes to content.fittingSize instead of a fixed 520×400. The permission row exposes a new "pending" state that swaps the rationale to "Find ⌘ ⌘ in the list and turn it on…" the moment the user clicks Grant.
  • StressTest.swift is left untouched. It still references SCK, but it is a debug-only harness behind --stress that exercised the very setup race we have now removed, and is no longer reachable from normal startup.

Move live tile previews off per-window SCStream onto a polled SkyLight
private API (CGSHWCaptureWindowList). Switch window enumeration from
SCShareableContent to CGWindowListCopyWindowInfo. Tighten the onboarding
window and auto-relaunch when Accessibility is granted.

ScreenCaptureKit is gone from the runtime path. Screen Recording is
still required on current macOS — CGSHWCaptureWindowList is gated on it
and the menu-bar recording indicator still attributes capture to us.
@peterp peterp merged commit c064ab1 into main May 14, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant