-
Notifications
You must be signed in to change notification settings - Fork 0
System Integration Utilities
Relevant source files
The ScreenUtil class provides coordinate transformation utilities for multi-monitor setups and DPI scaling handling. This utility enables MouseMacros to record and play back mouse actions consistently across different monitor configurations by normalizing coordinates to a virtual origin that spans all connected displays.
For information about other utility classes, see Utility Classes. For details on how these utilities integrate with macro playback, see MouseAction.
Sources: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L1-L42
In Java AWT, each monitor has its own coordinate space. The primary monitor typically has its origin at (0, 0), while secondary monitors may have negative or positive offsets depending on their physical arrangement. The virtual screen space encompasses all monitors as a single continuous coordinate system.
The virtual origin is the top-left corner of the bounding rectangle that contains all connected monitors. This point has the minimum X and Y coordinates across all screens and serves as the reference point for coordinate normalization.
flowchart TD
M1Origin["Origin: (-1920, 0)<br>Size: 1920x1080"]
M2Origin["Origin: (0, 0)<br>Size: 2560x1440"]
M3Origin["Origin: (2560, -200)<br>Size: 1920x1080"]
VirtualOrigin["Virtual Origin<br>Point(-1920, -200)<br>minX across all screens<br>minY across all screens"]
Note1["Normalized coordinates<br>use virtual origin as (0,0)"]
Note2["Global coordinates<br>use primary screen as (0,0)"]
VirtualOrigin --> M1Origin
VirtualOrigin --> M2Origin
VirtualOrigin --> M3Origin
VirtualOrigin --> Note1
M2Origin --> Note2
subgraph subGraph3 ["Multi-Monitor Layout Example"]
subgraph subGraph2 ["Monitor3[Monitor 3 - Secondary]"]
M3Origin
end
subgraph subGraph1 ["Monitor2[Monitor 2 - Primary]"]
M2Origin
end
subgraph subGraph0 ["Monitor1[Monitor 1 - Secondary]"]
M1Origin
end
end
Sources: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L24-L35
The ScreenUtil class provides three coordinate transformation methods to enable consistent macro recording and playback across different monitor configurations.
| Method | Input | Output | Purpose |
|---|---|---|---|
normalizeToVirtualOrigin |
Global coordinates | Normalized coordinates | Record phase: Store coordinates relative to virtual origin |
denormalizeFromVirtualOrigin |
Normalized coordinates | Robot coordinates | Playback phase: Convert to primary screen coordinates for Robot |
getVirtualOrigin |
None | Virtual origin point | Calculate the minimum X/Y across all screens |
sequenceDiagram
participant User
participant Operating System
participant JNativeHook
participant GlobalMouseListener
participant ScreenUtil
participant MouseAction
participant java.awt.Robot
note over User,java.awt.Robot: Recording Phase
User->>Operating System: Mouse click at screen position
Operating System->>JNativeHook: Global coordinates (x, y)
JNativeHook->>GlobalMouseListener: nativeMousePressed(x, y)
GlobalMouseListener->>ScreenUtil: normalizeToVirtualOrigin(x, y)
ScreenUtil->>ScreenUtil: getVirtualOrigin()
ScreenUtil->>ScreenUtil: normalized = (x - minX, y - minY)
ScreenUtil-->>GlobalMouseListener: Normalized Point
GlobalMouseListener->>MouseAction: Create MouseAction with normalized coords
MouseAction->>MouseAction: Store normalized coordinates
note over User,java.awt.Robot: Playback Phase
User->>GlobalMouseListener: Trigger playback hotkey
GlobalMouseListener->>MouseAction: action.perform()
MouseAction->>ScreenUtil: denormalizeFromVirtualOrigin(x, y)
ScreenUtil->>ScreenUtil: getVirtualOrigin()
ScreenUtil->>ScreenUtil: globalX = x + virtualOrigin.x
ScreenUtil->>ScreenUtil: globalY = y + virtualOrigin.y
ScreenUtil->>ScreenUtil: Get primary screen bounds
ScreenUtil->>ScreenUtil: Get DPI scale factors
ScreenUtil->>ScreenUtil: robotX = (globalX - primaryX) / scaleX
ScreenUtil->>ScreenUtil: robotY = (globalY - primaryY) / scaleY
ScreenUtil-->>MouseAction: Robot-compatible Point
MouseAction->>java.awt.Robot: mouseMove(robotX, robotY)
java.awt.Robot->>Operating System: Inject mouse event
Sources: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L7-L41
The normalizeToVirtualOrigin method converts global screen coordinates to coordinates relative to the virtual origin. This is used during macro recording to store coordinates in a monitor-configuration-independent format.
flowchart TD
Input["Input<br>Global Coordinates<br>(x, y)"]
GetOrigin["getVirtualOrigin()<br>Returns (minX, minY)"]
Calculate["Calculate<br>normalizedX = x - minX<br>normalizedY = y - minY"]
Output["Output<br>Normalized Point<br>(x - minX, y - minY)"]
Input --> GetOrigin
GetOrigin --> Calculate
Calculate --> Output
Implementation: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L38-L41
Sources: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L37-L41
The denormalizeFromVirtualOrigin method converts normalized coordinates back to the coordinate system expected by java.awt.Robot. This is more complex than normalization because it must:
- Convert from virtual origin coordinates to global coordinates
- Convert from global coordinates to primary screen coordinates
- Apply DPI scaling corrections
flowchart TD
Input["Input: Normalized Coordinates<br>(x, y) relative to virtual origin"]
Step1["Step 1: Get Virtual Origin<br>virtualOrigin = getVirtualOrigin()"]
Step2["Step 2: Convert to Global Coordinates<br>globalX = x + virtualOrigin.x<br>globalY = y + virtualOrigin.y"]
Step3["Step 3: Get Primary Screen Info<br>gc = defaultScreenDevice.defaultConfiguration<br>primaryBounds = gc.getBounds()<br>scaleX = gc.getDefaultTransform().getScaleX()<br>scaleY = gc.getDefaultTransform().getScaleY()"]
Step4["Step 4: Convert to Primary Screen Coords<br>relativeX = globalX - primaryBounds.x<br>relativeY = globalY - primaryBounds.y"]
Step5["Step 5: Apply DPI Scaling<br>robotX = round(relativeX / scaleX)<br>robotY = round(relativeY / scaleY)"]
Output["Output: Robot Coordinates<br>(robotX, robotY)<br>Ready for java.awt.Robot"]
Input --> Step1
Step1 --> Step2
Step2 --> Step3
Step3 --> Step4
Step4 --> Step5
Step5 --> Output
Implementation: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L7-L22
Sources: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L6-L22
The getVirtualOrigin method calculates the top-left corner of the virtual screen space by finding the minimum X and Y coordinates across all connected displays.
flowchart TD
Start["Start: getVirtualOrigin()"]
Init["Initialize<br>minX = Integer.MAX_VALUE<br>minY = Integer.MAX_VALUE"]
GetEnv["Get GraphicsEnvironment<br>ge = getLocalGraphicsEnvironment()"]
GetDevices["Get all GraphicsDevice[]<br>devices = ge.getScreenDevices()"]
LoopStart["For each<br>device"]
GetBounds["Get device bounds<br>bounds = device.getDefaultConfiguration().getBounds()"]
UpdateMinX["bounds.x < minX?"]
SetMinX["minX = bounds.x"]
UpdateMinY["bounds.y < minY?"]
SetMinY["minY = bounds.y"]
LoopEnd["Next device"]
Return["Return Point(minX, minY)"]
Start --> Init
Init --> GetEnv
GetEnv --> GetDevices
GetDevices --> LoopStart
LoopStart --> GetBounds
GetBounds --> UpdateMinX
UpdateMinX --> SetMinX
UpdateMinX --> UpdateMinY
SetMinX --> UpdateMinY
UpdateMinY --> SetMinY
UpdateMinY --> LoopEnd
SetMinY --> LoopEnd
LoopEnd --> LoopStart
LoopStart --> Return
Implementation: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L25-L35
The method iterates through all connected displays and tracks the minimum X and Y values. This ensures that all screen coordinates can be represented as positive offsets from the virtual origin.
Sources: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L24-L35
Modern operating systems support high-DPI displays with scaling factors (e.g., 150%, 200%). Java's java.awt.Robot class expects coordinates in the physical pixel space, not the logical coordinate space used by most applications.
The denormalizeFromVirtualOrigin method handles DPI scaling by:
- Obtaining the default transform from the primary screen's
GraphicsConfiguration - Extracting X and Y scale factors using
getScaleX()andgetScaleY() - Dividing the relative coordinates by the scale factors
- Rounding to the nearest integer pixel
flowchart TD
LogicalCoords["Logical Coordinates<br>(e.g., 1000, 500)<br>at 200% scaling"]
ScaleFactor["Scale Factors<br>scaleX = 2.0<br>scaleY = 2.0"]
Division["Division<br>physicalX = 1000 / 2.0 = 500<br>physicalY = 500 / 2.0 = 250"]
Rounding["Rounding<br>robotX = round(500) = 500<br>robotY = round(250) = 250"]
PhysicalCoords["Physical Coordinates<br>(500, 250)<br>for Robot API"]
LogicalCoords --> ScaleFactor
ScaleFactor --> Division
Division --> Rounding
Rounding --> PhysicalCoords
Implementation: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L16-L20
| Without DPI Handling | With DPI Handling |
|---|---|
| Robot clicks at wrong location | Robot clicks at correct location |
| Coordinates off by scale factor | Coordinates match recorded position |
| Macros fail on high-DPI displays | Macros work consistently |
| User must manually adjust | Automatic correction applied |
Sources: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L13-L21
The ScreenUtil class integrates with the macro recording and playback pipeline through the MouseAction class.
During recording, GlobalMouseListener receives mouse events with global screen coordinates from JNativeHook. Before creating a MouseAction object, the coordinates are normalized using normalizeToVirtualOrigin. This ensures that recorded coordinates are independent of the user's monitor configuration at recording time.
During playback, the MouseAction.perform() method calls denormalizeFromVirtualOrigin to convert the stored normalized coordinates back to coordinates that java.awt.Robot can use. This transformation accounts for:
- Changes in monitor configuration since recording
- DPI scaling differences between recording and playback systems
- Primary screen position differences
flowchart TD
RecEvent["Mouse Event<br>Global Coords"]
Normalize["ScreenUtil.normalizeToVirtualOrigin"]
StoreAction["Store MouseAction<br>with normalized coords"]
MMCFile[".mmc File<br>CSV format<br>x,y,type,..."]
LoadAction["Load MouseAction<br>from file"]
Denormalize["ScreenUtil.denormalizeFromVirtualOrigin"]
RobotExec["Robot.mouseMove<br>Robot.mousePress"]
StoreAction --> MMCFile
MMCFile --> LoadAction
subgraph Playback ["Playback"]
LoadAction
Denormalize
RobotExec
LoadAction --> Denormalize
Denormalize --> RobotExec
end
subgraph Persistence ["Persistence"]
MMCFile
end
subgraph Recording ["Recording"]
RecEvent
Normalize
StoreAction
RecEvent --> Normalize
Normalize --> StoreAction
end
Sources: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L1-L42
The ScreenUtil class is a utility class with static methods only. It has no instance variables or constructors.
| Method Signature | Return Type | Description |
|---|---|---|
denormalizeFromVirtualOrigin(int x, int y) |
Point |
Converts normalized coordinates to Robot-compatible coordinates with DPI scaling |
normalizeToVirtualOrigin(int x, int y) |
Point |
Converts global screen coordinates to normalized coordinates |
| Method Signature | Return Type | Description |
|---|---|---|
getVirtualOrigin() |
Point |
Calculates the minimum X and Y coordinates across all screens |
Sources: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L5-L42
When GlobalMouseListener receives a mouse event during recording:
Input: nativeMousePressed event at global coordinates (2500, 800)
Call: ScreenUtil.normalizeToVirtualOrigin(2500, 800)
Virtual Origin: (-1920, -200)
Output: Point(4420, 1000) stored in MouseActionWhen MouseAction.perform() is called during playback:
Input: Normalized coordinates (4420, 1000) from stored action
Call: ScreenUtil.denormalizeFromVirtualOrigin(4420, 1000)
Virtual Origin: (-1920, -200)
Global Coords: (2500, 800)
Primary Screen: origin (0, 0), scale (2.0, 2.0)
Robot Coords: (1250, 400)
Output: Robot.mouseMove(1250, 400)Sources: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L7-L41
The ScreenUtil class methods are stateless and thread-safe. Each method invocation queries the current graphics environment state and performs calculations without modifying shared state.
The getVirtualOrigin method is called during each coordinate transformation. This involves querying all connected displays through the Java AWT API. For typical macro playback with delays between actions, this overhead is negligible.
Coordinate calculations use integer arithmetic for X and Y values. The DPI scaling division uses floating-point arithmetic with Math.round() to ensure accurate conversion to physical pixel coordinates.
If the user changes their monitor configuration (adds/removes displays, changes arrangement) between recording and playback, the normalized coordinates ensure macros still execute at the correct relative positions. However, if the recording was made on a screen that no longer exists, the action may execute outside the visible screen area.
Sources: src/io/github/samera2022/mouse_macros/util/ScreenUtil.java L1-L42