Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 0 additions & 58 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,64 +57,6 @@ jobs:
- name: Build package
run: yarn prepare

build-android:
runs-on: ubuntu-latest

env:
TURBO_CACHE_DIR: .turbo/android

steps:
- name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

- name: Setup
uses: ./.github/actions/setup

- name: Cache turborepo for Android
uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
with:
path: ${{ env.TURBO_CACHE_DIR }}
key: ${{ runner.os }}-turborepo-android-${{ hashFiles('yarn.lock') }}
restore-keys: |
${{ runner.os }}-turborepo-android-

- name: Check turborepo cache for Android
run: |
TURBO_CACHE_STATUS=$(node -p "($(yarn turbo run build:android --cache-dir="${{ env.TURBO_CACHE_DIR }}" --dry=json)).tasks.find(t => t.task === 'build:android').cache.status")

if [[ $TURBO_CACHE_STATUS == "HIT" ]]; then
echo "turbo_cache_hit=1" >> $GITHUB_ENV
fi

- name: Install JDK
if: env.turbo_cache_hit != 1
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4.7.1
with:
distribution: 'zulu'
java-version: '17'

- name: Finalize Android SDK
if: env.turbo_cache_hit != 1
run: |
/bin/bash -c "yes | $ANDROID_HOME/cmdline-tools/latest/bin/sdkmanager --licenses > /dev/null"

- name: Cache Gradle
if: env.turbo_cache_hit != 1
uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
with:
path: |
~/.gradle/wrapper
~/.gradle/caches
key: ${{ runner.os }}-gradle-${{ hashFiles('example/android/gradle/wrapper/gradle-wrapper.properties') }}
restore-keys: |
${{ runner.os }}-gradle-

- name: Build example for Android
env:
JAVA_OPTS: "-XX:MaxHeapSize=6g"
run: |
yarn turbo run build:android --cache-dir="${{ env.TURBO_CACHE_DIR }}"

build-ios:
runs-on: macos-latest

Expand Down
75 changes: 3 additions & 72 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# react-native-vision-rtc

React Native library for streaming Vision Camera, ARKit, OpenCV, or ML frames over WebRTC.
VisionRTC fills the gap that react-native-webrtc doesn’t: it enables zero-copy streaming of custom native GPU frames (from Vision Camera, ARKit, ML, etc.) into WebRTC tracks without requiring per-project native code.

It’s a plugin-based, type-safe, cross-platform pipeline for sources, processors (filters, ML, AR), and track management, while staying unopinionated about signaling and transport.

> 🚧 This library is a work in progress. APIs, behaviors, and native implementations may change without notice.

Expand All @@ -11,75 +13,4 @@ React Native library for streaming Vision Camera, ARKit, OpenCV, or ML frames ov
The package has not been published to npm yet. Installation instructions will be added once the first release is available.


## Usage


```ts
import {
createVisionCameraSource,
createWebRTCTrack,
disposeTrack,
getStats,
} from 'react-native-vision-rtc';

async function demo(reactTag: number) {
const source = await createVisionCameraSource(reactTag);
const { trackId } = await createWebRTCTrack(source, {
fps: 30,
resolution: { width: 1280, height: 720 },
});

const stats = (await getStats?.()) ?? null;
await disposeTrack(trackId);
return stats;
}

// Example invocation:
// demo(findNodeHandle(cameraRef));
```

### API

- **Functions**
- `createVisionCameraSource(viewTag: number)`: Links a native camera-like view to the library and returns a source you can stream from.
- `createWebRTCTrack(source, opts?)`: Creates a WebRTC video track from a source. You can pass simple options like `fps` and `resolution`.
- `replaceTrack(senderId: string, nextTrackId: string)`: Swaps the video track used by an existing WebRTC sender.
- `pauseTrack(trackId: string)`: Temporarily stops sending frames for that track (does not destroy it).
- `resumeTrack(trackId: string)`: Restarts frame sending for a paused track.
- `setTrackConstraints(trackId: string, opts)`: Changes track settings on the fly (for example, fps or resolution).
- `disposeTrack(trackId: string)`: Frees native resources for that track.
- `getStats()`: Returns basic runtime stats like `fps` and `droppedFrames` (if supported on the platform).

- **Component**
- `VisionRTCView`: A native view that can render a given `trackId`.
- Props:
- `trackId?: string | null`
- `style?: ViewStyle | ViewStyle[]`

- **Types**
- `TrackOptions`: Options for tracks. Common fields:
- `fps?: number` (frames per second)
- `resolution?: { width: number; height: number }`
- `bitrate?: number`
- `colorSpace?: 'auto' | 'sRGB' | 'BT.709' | 'BT.2020'`
- `orientationMode?: 'auto' | 'fixed-0' | 'fixed-90' | 'fixed-180' | 'fixed-270'`
- `mode?: 'null-gpu' | 'null-cpu' | 'external'`
- `VisionRTCTrack`: `{ trackId: string }` returned when you create a track.
- `VisionCameraSource`: Source handle returned by `createVisionCameraSource`.
- `NativePixelSource`: Low-level source if you already have native pixels (platform-specific shapes).
- `Resolution`: `{ width: number; height: number }`.


## Contributing

- [Development workflow](CONTRIBUTING.md#development-workflow)
- [Sending a pull request](CONTRIBUTING.md#sending-a-pull-request)
- [Code of conduct](CODE_OF_CONDUCT.md)

## License

MIT

---

Made with [create-react-native-library](https://github.com/callstack/react-native-builder-bob)
4 changes: 2 additions & 2 deletions VisionRtc.podspec
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ Pod::Spec.new do |s|
s.platforms = { :ios => min_ios_version_supported }
s.source = { :git => "https://github.com/gmemmy/react-native-vision-rtc.git", :tag => "#{s.version}" }

s.source_files = "ios/**/*.{h,m,mm,cpp,swift}"
s.private_header_files = "ios/**/*.h"
s.source_files = "packages/core/ios/**/*.{h,m,mm,cpp,swift}"
s.private_header_files = "packages/core/ios/**/*.h"
s.swift_version = "5.0"
s.requires_arc = true
s.pod_target_xcconfig = {
Expand Down
4 changes: 2 additions & 2 deletions babel.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ module.exports = {
overrides: [
{
exclude: /\/node_modules\//,
presets: ['module:react-native-builder-bob/babel-preset'],
presets: ["module:react-native-builder-bob/babel-preset"],
},
{
include: /\/node_modules\//,
presets: ['module:@react-native/babel-preset'],
presets: ["module:@react-native/babel-preset"],
},
],
};
2 changes: 1 addition & 1 deletion eslint.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,6 @@ export default defineConfig([
},
},
{
ignores: ['node_modules/', 'lib/'],
ignores: ['node_modules/', 'lib/', '**/lib/**', '**/dist/**'],
},
]);
10 changes: 5 additions & 5 deletions example/babel.config.js
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
const path = require('path');
const {getConfig} = require('react-native-builder-bob/babel-config');
const pkg = require('../package.json');
const path = require("path");
const {getConfig} = require("react-native-builder-bob/babel-config");
const pkg = require("../package.json");

const root = path.resolve(__dirname, '..');
const root = path.resolve(__dirname, "..");

module.exports = getConfig(
{
presets: ['module:@react-native/babel-preset'],
presets: ["module:@react-native/babel-preset"],
},
{root, pkg}
);
8 changes: 4 additions & 4 deletions example/index.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import {AppRegistry} from 'react-native';
import {SafeAreaProvider} from 'react-native-safe-area-context';
import App from './src/App';
import {name as appName} from './app.json';
import {AppRegistry} from "react-native";
import {SafeAreaProvider} from "react-native-safe-area-context";
import App from "./src/App";
import {name as appName} from "./app.json";

const Root = () => (
<SafeAreaProvider>
Expand Down
2 changes: 1 addition & 1 deletion example/ios/Podfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -2688,7 +2688,7 @@ SPEC CHECKSUMS:
ReactCommon: ce5d4226dfaf9d5dacbef57b4528819e39d3a120
SocketRocket: d4aabe649be1e368d1318fdf28a022d714d65748
VisionCamera: 30b358b807324c692064f78385e9a732ce1bebfe
VisionRtc: 56e770d48b49da73a130f18a3ca12148a12126e8
VisionRtc: bdeb26d44fd53fdf47446823629d2b15d4cdc7f2
WebRTC-SDK: 69d4e56b0b4b27d788e87bab9b9a1326ed05b1e3
Yoga: 11c9686a21e2cd82a094a723649d9f4507200fb0

Expand Down
58 changes: 29 additions & 29 deletions example/ios/VisionRtcExample.xcodeproj/project.pbxproj
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
objects = {

/* Begin PBXBuildFile section */
0C80B921A6F3F58F76C31292 /* libPods-VisionRtcExample.a in Frameworks */ = {isa = PBXBuildFile; fileRef = 5DCACB8F33CDC322A6C60F78 /* libPods-VisionRtcExample.a */; };
13B07FBF1A68108700A75B9A /* Images.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 13B07FB51A68108700A75B9A /* Images.xcassets */; };
1E4BE3763AA2517BC08B5391 /* libPods-VisionRtcExample.a in Frameworks */ = {isa = PBXBuildFile; fileRef = 174FB31A6D5F0125B480D4A1 /* libPods-VisionRtcExample.a */; };
761780ED2CA45674006654EE /* AppDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = 761780EC2CA45674006654EE /* AppDelegate.swift */; };
81AB9BB82411601600AC10FF /* LaunchScreen.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 81AB9BB72411601600AC10FF /* LaunchScreen.storyboard */; };
9E3279E85816207F5B476885 /* PrivacyInfo.xcprivacy in Resources */ = {isa = PBXBuildFile; fileRef = 13B07FB81A68108700A75B9A /* PrivacyInfo.xcprivacy */; };
Expand All @@ -19,11 +19,11 @@
13B07FB51A68108700A75B9A /* Images.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; name = Images.xcassets; path = VisionRtcExample/Images.xcassets; sourceTree = "<group>"; };
13B07FB61A68108700A75B9A /* Info.plist */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.plist.xml; name = Info.plist; path = VisionRtcExample/Info.plist; sourceTree = "<group>"; };
13B07FB81A68108700A75B9A /* PrivacyInfo.xcprivacy */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.plist.xml; name = PrivacyInfo.xcprivacy; path = VisionRtcExample/PrivacyInfo.xcprivacy; sourceTree = "<group>"; };
3B4392A12AC88292D35C810B /* Pods-VisionRtcExample.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-VisionRtcExample.debug.xcconfig"; path = "Target Support Files/Pods-VisionRtcExample/Pods-VisionRtcExample.debug.xcconfig"; sourceTree = "<group>"; };
5709B34CF0A7D63546082F79 /* Pods-VisionRtcExample.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-VisionRtcExample.release.xcconfig"; path = "Target Support Files/Pods-VisionRtcExample/Pods-VisionRtcExample.release.xcconfig"; sourceTree = "<group>"; };
5DCACB8F33CDC322A6C60F78 /* libPods-VisionRtcExample.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = "libPods-VisionRtcExample.a"; sourceTree = BUILT_PRODUCTS_DIR; };
174FB31A6D5F0125B480D4A1 /* libPods-VisionRtcExample.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = "libPods-VisionRtcExample.a"; sourceTree = BUILT_PRODUCTS_DIR; };
761780EC2CA45674006654EE /* AppDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; name = AppDelegate.swift; path = VisionRtcExample/AppDelegate.swift; sourceTree = "<group>"; };
7BFFF1555331CCECE3A07EFF /* Pods-VisionRtcExample.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-VisionRtcExample.release.xcconfig"; path = "Target Support Files/Pods-VisionRtcExample/Pods-VisionRtcExample.release.xcconfig"; sourceTree = "<group>"; };
81AB9BB72411601600AC10FF /* LaunchScreen.storyboard */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = file.storyboard; name = LaunchScreen.storyboard; path = VisionRtcExample/LaunchScreen.storyboard; sourceTree = "<group>"; };
C253A2DEAFE552D78C56BD9D /* Pods-VisionRtcExample.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-VisionRtcExample.debug.xcconfig"; path = "Target Support Files/Pods-VisionRtcExample/Pods-VisionRtcExample.debug.xcconfig"; sourceTree = "<group>"; };
ED297162215061F000B7C4FE /* JavaScriptCore.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = JavaScriptCore.framework; path = System/Library/Frameworks/JavaScriptCore.framework; sourceTree = SDKROOT; };
/* End PBXFileReference section */

Expand All @@ -32,7 +32,7 @@
isa = PBXFrameworksBuildPhase;
buildActionMask = 2147483647;
files = (
0C80B921A6F3F58F76C31292 /* libPods-VisionRtcExample.a in Frameworks */,
1E4BE3763AA2517BC08B5391 /* libPods-VisionRtcExample.a in Frameworks */,
);
runOnlyForDeploymentPostprocessing = 0;
};
Expand All @@ -55,7 +55,7 @@
isa = PBXGroup;
children = (
ED297162215061F000B7C4FE /* JavaScriptCore.framework */,
5DCACB8F33CDC322A6C60F78 /* libPods-VisionRtcExample.a */,
174FB31A6D5F0125B480D4A1 /* libPods-VisionRtcExample.a */,
);
name = Frameworks;
sourceTree = "<group>";
Expand Down Expand Up @@ -92,8 +92,8 @@
BBD78D7AC51CEA395F1C20DB /* Pods */ = {
isa = PBXGroup;
children = (
3B4392A12AC88292D35C810B /* Pods-VisionRtcExample.debug.xcconfig */,
5709B34CF0A7D63546082F79 /* Pods-VisionRtcExample.release.xcconfig */,
C253A2DEAFE552D78C56BD9D /* Pods-VisionRtcExample.debug.xcconfig */,
7BFFF1555331CCECE3A07EFF /* Pods-VisionRtcExample.release.xcconfig */,
);
path = Pods;
sourceTree = "<group>";
Expand All @@ -105,13 +105,13 @@
isa = PBXNativeTarget;
buildConfigurationList = 13B07F931A680F5B00A75B9A /* Build configuration list for PBXNativeTarget "VisionRtcExample" */;
buildPhases = (
C38B50BA6285516D6DCD4F65 /* [CP] Check Pods Manifest.lock */,
138B3C8B7A377E88F8A4D356 /* [CP] Check Pods Manifest.lock */,
13B07F871A680F5B00A75B9A /* Sources */,
13B07F8C1A680F5B00A75B9A /* Frameworks */,
13B07F8E1A680F5B00A75B9A /* Resources */,
00DD1BFF1BD5951E006B06BC /* Bundle React Native code and images */,
00EEFC60759A1932668264C0 /* [CP] Embed Pods Frameworks */,
E235C05ADACE081382539298 /* [CP] Copy Pods Resources */,
42C5CF001E9832EAC495F752 /* [CP] Embed Pods Frameworks */,
A7E999FA99B8EF4090BFEB40 /* [CP] Copy Pods Resources */,
);
buildRules = (
);
Expand Down Expand Up @@ -183,46 +183,46 @@
shellPath = /bin/sh;
shellScript = "set -e\n\nWITH_ENVIRONMENT=\"$REACT_NATIVE_PATH/scripts/xcode/with-environment.sh\"\nREACT_NATIVE_XCODE=\"$REACT_NATIVE_PATH/scripts/react-native-xcode.sh\"\n\n/bin/sh -c \"$WITH_ENVIRONMENT $REACT_NATIVE_XCODE\"\n";
};
00EEFC60759A1932668264C0 /* [CP] Embed Pods Frameworks */ = {
138B3C8B7A377E88F8A4D356 /* [CP] Check Pods Manifest.lock */ = {
isa = PBXShellScriptBuildPhase;
buildActionMask = 2147483647;
files = (
);
inputFileListPaths = (
"${PODS_ROOT}/Target Support Files/Pods-VisionRtcExample/Pods-VisionRtcExample-frameworks-${CONFIGURATION}-input-files.xcfilelist",
);
name = "[CP] Embed Pods Frameworks";
inputPaths = (
"${PODS_PODFILE_DIR_PATH}/Podfile.lock",
"${PODS_ROOT}/Manifest.lock",
);
name = "[CP] Check Pods Manifest.lock";
outputFileListPaths = (
"${PODS_ROOT}/Target Support Files/Pods-VisionRtcExample/Pods-VisionRtcExample-frameworks-${CONFIGURATION}-output-files.xcfilelist",
);
outputPaths = (
"$(DERIVED_FILE_DIR)/Pods-VisionRtcExample-checkManifestLockResult.txt",
);
runOnlyForDeploymentPostprocessing = 0;
shellPath = /bin/sh;
shellScript = "\"${PODS_ROOT}/Target Support Files/Pods-VisionRtcExample/Pods-VisionRtcExample-frameworks.sh\"\n";
shellScript = "diff \"${PODS_PODFILE_DIR_PATH}/Podfile.lock\" \"${PODS_ROOT}/Manifest.lock\" > /dev/null\nif [ $? != 0 ] ; then\n # print error to STDERR\n echo \"error: The sandbox is not in sync with the Podfile.lock. Run 'pod install' or update your CocoaPods installation.\" >&2\n exit 1\nfi\n# This output is used by Xcode 'outputs' to avoid re-running this script phase.\necho \"SUCCESS\" > \"${SCRIPT_OUTPUT_FILE_0}\"\n";
showEnvVarsInLog = 0;
};
C38B50BA6285516D6DCD4F65 /* [CP] Check Pods Manifest.lock */ = {
42C5CF001E9832EAC495F752 /* [CP] Embed Pods Frameworks */ = {
isa = PBXShellScriptBuildPhase;
buildActionMask = 2147483647;
files = (
);
inputFileListPaths = (
"${PODS_ROOT}/Target Support Files/Pods-VisionRtcExample/Pods-VisionRtcExample-frameworks-${CONFIGURATION}-input-files.xcfilelist",
);
inputPaths = (
"${PODS_PODFILE_DIR_PATH}/Podfile.lock",
"${PODS_ROOT}/Manifest.lock",
);
name = "[CP] Check Pods Manifest.lock";
name = "[CP] Embed Pods Frameworks";
outputFileListPaths = (
);
outputPaths = (
"$(DERIVED_FILE_DIR)/Pods-VisionRtcExample-checkManifestLockResult.txt",
"${PODS_ROOT}/Target Support Files/Pods-VisionRtcExample/Pods-VisionRtcExample-frameworks-${CONFIGURATION}-output-files.xcfilelist",
);
runOnlyForDeploymentPostprocessing = 0;
shellPath = /bin/sh;
shellScript = "diff \"${PODS_PODFILE_DIR_PATH}/Podfile.lock\" \"${PODS_ROOT}/Manifest.lock\" > /dev/null\nif [ $? != 0 ] ; then\n # print error to STDERR\n echo \"error: The sandbox is not in sync with the Podfile.lock. Run 'pod install' or update your CocoaPods installation.\" >&2\n exit 1\nfi\n# This output is used by Xcode 'outputs' to avoid re-running this script phase.\necho \"SUCCESS\" > \"${SCRIPT_OUTPUT_FILE_0}\"\n";
shellScript = "\"${PODS_ROOT}/Target Support Files/Pods-VisionRtcExample/Pods-VisionRtcExample-frameworks.sh\"\n";
showEnvVarsInLog = 0;
};
E235C05ADACE081382539298 /* [CP] Copy Pods Resources */ = {
A7E999FA99B8EF4090BFEB40 /* [CP] Copy Pods Resources */ = {
isa = PBXShellScriptBuildPhase;
buildActionMask = 2147483647;
files = (
Expand Down Expand Up @@ -255,7 +255,7 @@
/* Begin XCBuildConfiguration section */
13B07F941A680F5B00A75B9A /* Debug */ = {
isa = XCBuildConfiguration;
baseConfigurationReference = 3B4392A12AC88292D35C810B /* Pods-VisionRtcExample.debug.xcconfig */;
baseConfigurationReference = C253A2DEAFE552D78C56BD9D /* Pods-VisionRtcExample.debug.xcconfig */;
buildSettings = {
ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
CLANG_ENABLE_MODULES = YES;
Expand Down Expand Up @@ -286,7 +286,7 @@
};
13B07F951A680F5B00A75B9A /* Release */ = {
isa = XCBuildConfiguration;
baseConfigurationReference = 5709B34CF0A7D63546082F79 /* Pods-VisionRtcExample.release.xcconfig */;
baseConfigurationReference = 7BFFF1555331CCECE3A07EFF /* Pods-VisionRtcExample.release.xcconfig */;
buildSettings = {
ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
CLANG_ENABLE_MODULES = YES;
Expand Down
2 changes: 1 addition & 1 deletion example/jest.config.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module.exports = {
preset: 'react-native',
preset: "react-native",
};
8 changes: 4 additions & 4 deletions example/metro.config.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
const path = require('path');
const {getDefaultConfig} = require('@react-native/metro-config');
const {withMetroConfig} = require('react-native-monorepo-config');
const path = require("path");
const {getDefaultConfig} = require("@react-native/metro-config");
const {withMetroConfig} = require("react-native-monorepo-config");

const root = path.resolve(__dirname, '..');
const root = path.resolve(__dirname, "..");

/**
* Metro configuration
Expand Down
Loading
Loading