Skip to content

Feature: Upgrade camera motion detection to ML Kit Face Detection with instant state clearing#13

Open
adix992 wants to merge 10 commits intomsp1974:devfrom
adix992:dev
Open

Feature: Upgrade camera motion detection to ML Kit Face Detection with instant state clearing#13
adix992 wants to merge 10 commits intomsp1974:devfrom
adix992:dev

Conversation

@adix992
Copy link
Copy Markdown

@adix992 adix992 commented Feb 26, 2026

Title

Feature: Upgrade camera motion detection to ML Kit Face Detection with instant state clearing

Description

This PR upgrades the camera's motion detection system by replacing the pixel-based AggregateLumaMotionDetection with Google's ML Kit Face Detection.

The previous implementation relied on pixel changes, which could trigger false positives from shadows or pets, and relied on Home Assistant's 20-second timeout to clear the sensor. This update makes the tablet highly accurate at detecting actual human presence and makes the sensor instantly responsive.

Key Changes:

  • Added Google ML Kit: Integrated com.google.mlkit:face-detection via the version catalog.
  • Refactored CameraBackgroundTask: Swapped the legacy luma detector for ML Kit's FaceDetector, processing the InputImage directly from the camera frames.
  • Instant On / Instant Off: The app now fires a custom faceStateChanged event. It sends true the millisecond a face enters the frame, and explicitly sends false the exact moment the face leaves, bypassing the need to wait for Home Assistant's timeout.
  • Heartbeat Ping: To prevent the Wyoming integration's 20-second auto-clear timer from turning off the sensor while a user is actively looking at the tablet, the camera now sends a true heartbeat ping every 10 seconds (MOTION_INTERVAL) as long as a face remains in the frame.
  • Updated Routing: Added the faceStateChanged payload router in BackgroundTask.kt to seamlessly pass the new boolean states to Home Assistant.

Testing:
Tested locally on physical hardware connected to Home Assistant. The Wyoming sensor instantly triggers when a user approaches the tablet, stays active continuously while the user is present, and immediately clears when the user walks away.

Personal Note:
This is my first ever GitHub contribution! I am not professionally trained in any coding languages, so this feature was put together with the help of an Gemini.

Note to maintainer: This currently replaces the luma motion detection entirely. If you prefer to keep both and have this as a toggleable option in the app settings, let me know and I would be happy to adjust the code to support both pipelines!

@adix992
Copy link
Copy Markdown
Author

adix992 commented Feb 26, 2026

If anyone would like to test this face detection logic on their own tablet/phone without building from source, I have attached the compiled .apk to a release on my fork here: https://github.com/adix992/ViewAssistCompanionApp/releases/tag/0.9.2face

@msp1974
Copy link
Copy Markdown
Owner

msp1974 commented Mar 14, 2026

Hi @adix992 . Many thanks for this contribution. I am about to release v0.10.0 with microwakeword support and then will look to pull this into the next release if that is ok.

I would appreciate it if you could make this an optional detection mode as I don't like to remove features so bluntly that people may prefer for their use case. Happy to help with that if you tie it to a setting, I can do the HA side to set the mode.

@adix992
Copy link
Copy Markdown
Author

adix992 commented Mar 17, 2026

I'll have a look at synching my fork with 0.10.0 and re adding the original motion feature with a event hook for a toggle between motion and face detection in Home Assistant this week :).

Thanks for the great work btw, the integration is perfect for my desktop dashboard running on a tablet, The only feature I was missing was a way to prevent my cats from activating motion (Which neither pir or radar can do).

@adix992
Copy link
Copy Markdown
Author

adix992 commented Mar 26, 2026

Hey @msp1974, It took a little longer than a week for me to get around to it, but it's now synched with 0.10.0 and ready to use with both luma and face detection on a toggle.

To control the detection mode from the HA side, I have added a new event listener in BackgroundTask.kt that catches changes and routes them to the camera task.

You just need HA to send an event with the key motionDetectionMode.
The accepted string values are:

"luma" (This is the default, which runs the original pixel-based detection)
"face" (This routes the image reader to the new ML Kit face detection)

As requested, it defaults to luma upon initialization, so it is fully backwards compatible and won't break anything for users upgrading to this release before they touch the new HA settings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants