Fixing Hover State Loss During macOS Screenshot Capture

The Problem

When building a macOS screenshot app, there's a tricky issue you'll inevitably run into. You want to capture a button's hover state, but the moment the capture UI appears, the hover disappears.

The reason is straightforward. When the overlay window appears, NSApp.activate() gets called, deactivating the target app and dismissing its hover state. On macOS, hover effects only persist while the app is active.

User: hovers button → capture shortcut → overlay activates → target app deactivates → hover gone

Hover state captures are frequently needed for design system documentation, UI reviews, and bug reports. Not being able to capture them is a real pain point.

Root Cause

The existing overlay windows inherited from NSWindow and used the following flow to display:

// Old approach: activate app and wait
private func activateAppAndWait() async {
    guard !NSApp.isActive else { return }
    NSApp.activate(ignoringOtherApps: true)
    for _ in 0..<10 {
        if NSApp.isActive { return }
        try? await Task.sleep(nanoseconds: 10_000_000)
    }
}

And when showing the overlay:

// Old approach: activate then show window
await activateAppAndWait()
regionSelectorWindow = RegionSelectorWindow()

// Overlay window display
window.makeKeyAndOrderFront(nil)
NSApp.activate(ignoringOtherApps: true)

The moment NSApp.activate(ignoringOtherApps: true) fires, the macOS window server brings the capture app to the front and transitions all other apps to inactive state. This kills every visual effect that depends on active state: hovers, focus rings, tooltips, and more.

The Fix: NSPanel + nonactivatingPanel

The key is making the overlay window not steal activation from other apps. macOS has the exact tool for this: NSPanel with the .nonactivatingPanel style mask.

Converting NSWindow to NSPanel

// Before
class RegionOverlayWindow: NSWindow {
    init(screen: NSScreen) {
        super.init(
            contentRect: screen.frame,
            styleMask: .borderless,
            backing: .buffered,
            defer: false
        )
    }
}

// After
class RegionOverlayWindow: NSPanel {
    init(screen: NSScreen) {
        super.init(
            contentRect: screen.frame,
            styleMask: [.borderless, .nonactivatingPanel],
            backing: .buffered,
            defer: false
        )
        hidesOnDeactivate = false
    }
}

Three changes are needed:

1. NSWindow → NSPanel: NSPanel is a subclass of NSWindow designed for auxiliary windows (palettes, inspectors, etc.).
2. .nonactivatingPanel style mask: Prevents app activation when the window is displayed.
3. hidesOnDeactivate = false: NSPanel hides automatically when the app deactivates by default — this prevents that.

Changing Window Presentation

// Before
window.makeKeyAndOrderFront(nil)
NSApp.activate(ignoringOtherApps: true)

// After
window.orderFrontRegardless()
window.makeKey()

• orderFrontRegardless(): Brings the window to the front without activating the app.
• makeKey(): Accepts keyboard input without activating the entire app.

Removing activateAppAndWait()

Since the overlay no longer requires app activation, the activateAppAndWait() method and all its call sites were removed entirely.

// Deleted
await activateAppAndWait()

Result

User: hovers button → capture shortcut → overlay appears (non-activating panel) → target app stays active → hover preserved

Hover states, focus rings, and active state highlights are now all preserved during capture. The capture app's overlay floats above, but from the macOS window server's perspective, the target app remains the active app.

NSPanel and nonactivatingPanel Summary

NSPanel is a widely used pattern in Xcode's inspectors, Finder's Get Info window, the color picker, and more. It's the perfect fit for UI that needs to "float above other apps without disturbing them" — exactly what a screenshot tool needs.

Conclusion

The root cause of the hover capture problem was the default behavior of "overlay window = app activation." NSPanel's .nonactivatingPanel breaks that coupling.

If you're facing a similar issue, remember three things:

1. NSWindow → NSPanel
2. Add .nonactivatingPanel style mask
3. Remove NSApp.activate() calls

This combination lets you present overlays while preserving the target app's state.