Adopt CameraX Feature Group API #456
Conversation
Summary of ChangesHello @Zeronfinity, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the camera's robustness by integrating the new CameraX Feature Combination Query API. The primary goal is to prevent application crashes and ensure a smoother user experience by dynamically checking and enforcing device compatibility for various camera settings. By mapping internal camera features to CameraX's Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request is an excellent and significant update that adopts the new CameraX Feature Group API. The migration from UseCaseGroup to SessionConfig for single camera sessions, along with the dynamic validation of feature combinations, is well-executed and crucial for preventing runtime failures on various devices. The introduction of FeatureGroupData provides a clean mapping layer, and the asynchronous updating of system constraints is a thoughtful performance consideration. My feedback is focused on improving test coverage for the complex new logic and ensuring adherence to the project's testing style guide.
core/camera/src/main/java/com/google/jetpackcamera/core/camera/CameraXCameraSystem.kt
Outdated
Show resolved
Hide resolved
core/camera/src/test/java/com/google/jetpackcamera/core/camera/FeatureGroupDataTest.kt
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Code Review
This pull request successfully adopts the new CameraX Feature Combination Query API, which is a significant and valuable improvement for ensuring device compatibility and preventing runtime errors with unsupported feature combinations. The refactoring from UseCaseGroup to SessionConfig for single camera sessions is well-executed and aligns with modern CameraX practices. The introduction of FeatureGroupData and the dynamic constraint filtering in CameraXCameraSystem are the core of this change and are logically sound.
My review includes a few critical and high-severity findings. One critical issue is a compilation error in the new filtering logic due to using a suspend function within a non-suspend lambda. Another high-severity issue relates to potential resource leaks from improper CoroutineScope creation within a frequently called helper function. I've also included suggestions for improving code structure and adhering to the project's testing style guide. Addressing these points will enhance the robustness and maintainability of this new implementation.
core/camera/src/main/java/com/google/jetpackcamera/core/camera/CameraXCameraSystem.kt
Outdated
Show resolved
Hide resolved
core/camera/src/main/java/com/google/jetpackcamera/core/camera/CameraXCameraSystem.kt
Outdated
Show resolved
Hide resolved
core/camera/src/main/java/com/google/jetpackcamera/core/camera/CameraSession.kt
Show resolved
Hide resolved
core/camera/src/test/java/com/google/jetpackcamera/core/camera/FeatureGroupDataTest.kt
Outdated
Show resolved
Hide resolved
Zeronfinity
force-pushed
the
tahsinm/cxfg-adoption
branch
2 times, most recently
from
43a556c to
70fa0fa
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request is a significant and well-executed refactoring to adopt the new CameraX Feature Combination Query API. The changes are extensive, touching on session creation, constraint management, and settings application, but they are well-structured and improve the overall architecture. The migration from UseCaseGroup to SessionConfig for single camera sessions is a great modernization. The introduction of FeatureGroupData and the associated logic to dynamically filter system constraints is a robust solution to prevent crashes from unsupported feature combinations.
I've identified one high-severity issue regarding side effects in a feature-checking function that should be addressed. I've also included a suggestion to enhance the new integration test to make it more robust. Overall, this is excellent work.
core/camera/src/main/java/com/google/jetpackcamera/core/camera/CameraXCameraSystem.kt
Outdated
Show resolved
Hide resolved
.../camera/src/androidTest/java/com/google/jetpackcamera/core/camera/CameraXCameraSystemTest.kt
Outdated
Show resolved
Hide resolved
Zeronfinity
force-pushed
the
tahsinm/cxfg-adoption
branch
from
f56b76a to
bde634e
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request successfully adopts the new CameraX Feature Group API, which is a significant and complex undertaking. The refactoring from UseCaseGroup to SessionConfig is well-executed, and the new logic for dynamically updating system constraints based on feature combinations is robust. I particularly appreciate the addition of comprehensive unit and integration tests, which ensure the new functionality is reliable. The fix to call setSurfaceProvider on the main thread is also a crucial correction. I have one suggestion to improve the efficiency of the feature support checks.
core/camera/src/main/java/com/google/jetpackcamera/core/camera/CameraXCameraSystem.kt
Outdated
Show resolved
Hide resolved
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request successfully adopts the new CameraX Feature Group API, which is a significant and valuable improvement. The changes allow for dynamic validation of feature combinations, preventing runtime crashes on devices with unsupported settings like HDR + 60 FPS. The refactoring from UseCaseGroup to SessionConfig for single camera sessions is well-executed and aligns with modern CameraX practices. The introduction of FeatureGroupData and the related mapping logic is clear and well-tested. The new integration test (setMultipleFeatures_systemConstraintsUpdatedAndFeaturesSetIfSupported) is particularly strong, as it verifies the dynamic constraint updates across various feature permutations. I have a couple of minor suggestions to improve code style and ensure documentation stays in sync with build changes. Overall, this is an excellent contribution that enhances the app's stability and correctness.
core/camera/src/main/java/com/google/jetpackcamera/core/camera/CameraSession.kt
Show resolved
Hide resolved
Zeronfinity
force-pushed
the
tahsinm/cxfg-adoption
branch
from
b8feb82 to
c577a74
Compare
core/camera/src/main/java/com/google/jetpackcamera/core/camera/FeatureGroupData.kt
Outdated
Show resolved
Hide resolved
core/camera/src/main/java/com/google/jetpackcamera/core/camera/FeatureGroupData.kt
Outdated
Show resolved
Hide resolved
core/camera/src/main/java/com/google/jetpackcamera/core/camera/FeatureGroupData.kt
Outdated
Show resolved
Hide resolved
core/camera/src/main/java/com/google/jetpackcamera/core/camera/FeatureGroupData.kt
Outdated
Show resolved
Hide resolved
core/camera/src/main/java/com/google/jetpackcamera/core/camera/CameraXCameraSystem.kt
Show resolved
Hide resolved
| captureMode: CaptureMode, | ||
| backgroundDispatcher: CoroutineDispatcher, | ||
| targetFrameRate: Int? = null, | ||
| stabilizationMode: StabilizationMode? = null, | ||
| dynamicRange: DynamicRange? = null, | ||
| videoQuality: VideoQuality? = null | ||
| ): VideoCapture<Recorder>? { | ||
| if (captureMode != CaptureMode.STANDARD && captureMode != CaptureMode.VIDEO_ONLY) { | ||
| return null | ||
| } |
There was a problem hiding this comment.
I suggest removing captureMode from this function and returning a non-nullable VideoCapture<Recorder>. The captureMode check should happen at the call site (similar to createImageUseCase) to avoid unnecessary argument passing and implicit null returns.
There was a problem hiding this comment.
I am fine with changing it back to that, but note that there are multiple call sites for this and the captureMode checking logic is duplicate in all of those places then (3 places in total with this PR). I had moved captureMode inside the function mainly to avoid this duplicate checking in all 3 places. WDYT?
| withContext(Dispatchers.Main) { | ||
| setSurfaceProvider { surfaceRequest -> | ||
| surfaceRequests.update { surfaceRequest } | ||
| } |
There was a problem hiding this comment.
I'm assuming this withContext was added so that this can be called from within the featuregroup check? Do we need to set a surface provider just to check for capabilities?
There was a problem hiding this comment.
Short answer: not really.
As explained below, I implemented it this way for a few reasons. Regardless, this should be a JCA decision and I am happy to change this to however you prefer (or at least leave some TODO comments for later in case we decide to do something a bit more complicated).
- I was trying to avoid different code flows for capability query and actual binding as much as possible.
Needing too many different code flow paths often leads to code smell and we've had developers complaining about similar situation recently as well. So, I tried to keep functions like createPreviewUseCase as generic as possible without having different code flows in these functions based on whether the intention is capability query or binding.
- In future, we may use the user-provided surface in query directly (only if users have already provided the surface).
In some cases, the framework-layer FCQ API wants to have concrete surfaces, not deferred ones. Right now, CameraX creates such surfaces for queries, even when surface provider is already providing one. However, this may lead to different results in query and binding (in rare cases though), and there's also an optimization opportunity here. So, we may end up using the user-provided surface in future (only if already provided, it won't be a hard requirement). But this is kinda moot point for this PR since we are using defaultCameraSessionContext during query here which probably doesn't set up the surfaceRequests flow correctly anyway.
| createSessionConfig( | ||
| cameraConstraints = cameraConstraints, | ||
| initialTransientSettings = transientSettings, | ||
| videoCaptureUseCase = videoCaptureUseCase, | ||
| sessionSettings = sessionSettings, | ||
| sessionScope = CoroutineScope(defaultDispatcher + job) | ||
| ) |
There was a problem hiding this comment.
This seems quite expensive. Any time we want to call supportsGrouping, which happens when we change a setting, we set up and creating an entire pipeline (including use cases, effects, etc.). Maybe it's unavoidable, but do we really need to do this every time we change settings? Are there enough combinations of settings that we couldn't just do this once up front during initialization? Maybe there is, but I would much prefer exploring that approach if it meant we could avoid this type of check whenever we change a setting.
There was a problem hiding this comment.
That is a valid concern regarding the overhead. The CameraX isSessionConfigSupported API for querying feature groups require a full SessionConfig because support can be influenced by session-wide variables like CameraEffect, use case resolutions, aspect ratios etc.
To ensure accuracy without assuming which scenarios might conflict in the JCA codebase (specially since I am not too well-versed with all scenarios in JCA and future plans), I have followed the framework's contract strictly for now. I agree that pre-calculation can be a great optimization since the number of factors that can change the result may not be too high for JCA, so I've added a TODO (in FeatureGroupHandler.isGroupingSupported) suggesting a persistence-based approach for the team to explore in future.
Does that work for this PR?
core/camera/src/main/java/com/google/jetpackcamera/core/camera/CameraSystem.kt
Outdated
Show resolved
Hide resolved
core/camera/src/main/java/com/google/jetpackcamera/core/camera/CameraXCameraSystem.kt
Outdated
Show resolved
Hide resolved
Zeronfinity
force-pushed
the
tahsinm/cxfg-adoption
branch
from
c577a74 to
99b56bd
Compare
Zeronfinity
force-pushed
the
tahsinm/cxfg-adoption
branch
from
99b56bd to
e8a4f78
Compare
Zeronfinity
force-pushed
the
tahsinm/cxfg-adoption
branch
from
e8a4f78 to
abaabeb
Compare
2 participants
Description
This PR adopts the new CameraX Feature Group API to validate feature combinations and dynamically update system constraints. This ensures that the app only allows supported combinations of settings (e.g., HDR, FPS, Stabilization), preventing runtime failures on devices that do not support specific combinations (like HDR + 60 FPS on Pixel 9).
Key Changes
1.6.0-SNAPSHOTto access the stable Feature Groups API.FeatureGroupDatato map JCA internal models (DynamicRange,VideoQuality,StabilizationMode, etc.) to CameraX'sGroupableFeature.updateSystemConstraintsByFeatureGroupsinCameraXCameraSystem. This method asynchronously validates the current session settings against the device capabilities and filters out unsupported options for other settings (e.g., if 60 FPS is selected, it might disable HDR if they can't be used together).UseCaseGrouptoSessionConfigfor single camera session creation to align with the new API usage and allow forisSessionConfigSupportedchecks.Bug Fixes & Enhancements
Known Limitations & Future Work
System constraints not updating for multiple updates from setting screen
The system constraints are currently updated only when the camera session is initialized or re-created (e.g., upon returning to the preview screen). This means that if a user sets feature options (like UHD and 60 FPS) directly from the settings menu without navigating back to the preview screen, the system constraints is not re-evaluated.
Consequently, it allows users to select a combination of features that are not mutually supported by the device, which can lead to issues when they eventually navigate back to the preview screen.
TODO: Addressing this limitation requires further architectural refactoring to trigger constraint updates dynamically upon any individual setting change. Or, the constraints repository should be updated with the info of all possible constraints at initialization, rather than only the settings state navigable from the current settings state.
Features incompatible with CameraX Feature Group API
CameraX depends on the Camera2 feature combination query (FCQ) API which guarantees support only a limited set of features. So, the scenarios using features incompatible with the FCQ API do not get much benefits from this PR.
Asynchronous system constraints update
System constraints are updated asynchronously, creating a window where a user might enable an unsupported combination of features before the update completes.
Future Work: This could be addressed by waiting for the constraints update to finish and restricting setting changes until it is complete.
Tests
FeatureGroupDataTest(unit tests) to verify mapping logic between JCA models and CameraXGroupableFeature.setMultipleFeatures_systemConstraintsUpdatedAndFeaturesSetIfSupportedtoCameraXCameraSystemTest(integration test) to verify that the camera system dynamically updates constraints and sets features correctly based on CameraX feature group compatibility.Additional Context
Related Issues