Get windows across multiple spaces / virtual desktops

[OrionReed]

macOS has a 'spaces' system, where multiple windows can be arranged on separate desktops or as fullscreen windows. While on macOS it's challenging to know which space a window is in, it would be amazing to have an ability to get all windows across these spaces.

I did some experimentation and CGWindowListOption::OptionAll casts a very wide net (including system UI, hidden windows, etc)

It would be absolutely fantastic to have this ability and I'm sure folks working on window management and the like would benefit greatly from it.

[miniben-90]

Hi!

For the moment I could not found an easy way without a breaking change. Using "OptionAll" bring to mush values and the only way to use it is with "AXUIElementCopyAttributeValue" who result with a slow response time (and require permission).

[OrionReed]

Yeah that makes sense. I tried a few different approaches too and couldn't get anything working sensibly. I did find a compromise for my use-case which was to only support 1 space, and keep some extra state to maintain these windows when switching spaces (leveraging the fact that macOS switches coordinates to a global coord space during space transitions)

We're polling window positions every frame, which doesn't feel like a great idea but I haven't found anything better yet. We do this to augment application windows with extra UI and make it possible to write app 'extensions' in a webview rendered on top. This also makes for some fun realtime simulations that can run on your windows. We used x-win as part of this paper which was accepted to LIVEProg earlier this year.

If you have any tips for a more sane choice than polling every frame (or have more specifics on how bad of an idea this is) I'd be curious to know. There's a few bits of info which are collected which we don't use, I have considered forking and removing e.g. UsageInfo, maybe some more control over what is collected would be nice.

In any case your crate has been a lifesaver!

[miniben-90]

I suppose you are using the position and size data to generate your layout. If you are doing this project only for macOS it will be better to write a new code will using "AXObserverCreate" with "AXUIElement" ("kAXMovedNotification" and "kAXResizedNotification") instead. I think it will be more efficient and it should be able to handle spaces.

[m13v]

been down this exact rabbit hole on a macOS automation project. AXObserver with kAXMovedNotification works great within a single space but silently stops firing for windows on an inactive space, so you get stale coordinates the moment the user switches spaces. the only reliable way to map windows to spaces is the private CGSSpace APIs (CGSCopySpacesForWindows, CGSGetActiveSpace), undocumented but tolerated for a decade plus. polling CGWindowListCopyWindowInfo at 5-10hz with kCGWindowListOptionOnScreenOnly stays cheap, then filter by windowLayer to drop system UI. every-frame polling is overkill unless you actually need sub-16ms for an overlay.