VR Network Avatar

A production-grade Unity package for real-time VR avatar synchronization in multiplayer environments. Built on Unity Netcode for GameObjects (NGO), this system provides seamless head, hand, and finger tracking across networked VR clients with support for Meta Quest, Unity XR Interaction Toolkit, and custom XR platforms.

This module was originally developed by me while building a production multiplayer VR experience at Evermorrow Labs. This repository is a cleaned-up, standalone version intended for reuse and documentation. No proprietary code or assets from the original project are included.

Features

Full 6DOF Tracking - Head, left hand, and right hand position & rotation synchronization
Optional Finger Tracking - Individual finger joint rotation sync for hand tracking devices
Multi-Platform XR Support - Meta Quest (OVR), Unity XR Interaction Toolkit, and extensible provider system
Client-Authoritative Movement - Low-latency avatar control with server validation
Network Optimized - Dead-zone filtering reduces bandwidth by 60-80% in typical scenarios
Physical Constraints - Arm length limits and head rotation clamping prevent unnatural poses
Player Identity System - Synchronized display names with TextMeshPro UI integration
Event System - Extensible avatar events for game-specific features (reactions, indicators, etc.)
Modular Architecture - Mix and match components based on your requirements

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                    VRNetworkAvatarManager                       │
│              (Singleton, Lifecycle Management)                  │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ┌──────────────────┐    ┌──────────────────────────────────┐   │
│  │ NetworkAvatar    │    │        XR Abstraction Layer      │   │
│  │    Spawner       │    │                                  │   │
│  │                  │    │  ┌────────────┐ ┌─────────────┐  │   │
│  │ • Spawn Points   │    │  │OVRRigProv. │ │GenericXRRig │  │   │
│  │ • Prefab Select  │    │  │(Meta/Quest)│ │ (Unity XR)  │  │   │
│  │ • Client Mapping │    │  └─────┬──────┘ └──────┬──────┘  │   │
│  └────────┬─────────┘    │        │    IXRRigProvider       │   │
│           │              │        └────────┴─────────────── │   │
│           ▼              └──────────────────────────────────┘   │
│  ┌─────────────────────────────────────────────────────────┐    │
│  │                  Network Avatar Instance                │    │
│  │  ┌───────────────┐ ┌─────────────┐ ┌─────────────────┐  │    │
│  │  │NetworkVRAvatar│ │AvatarFinger │ │NetworkPlayer    │  │    │
│  │  │               │ │   Sync      │ │   Identity      │  │    │
│  │  │• Head Tracking│ │             │ │                 │  │    │
│  │  │• Hand Tracking│ │• L/R Finger │ │• Display Name   │  │    │
│  │  │• Body Derive  │ │• Thresholds │ │• Name Plate UI  │  │    │
│  │  │• Constraints  │ │             │ │                 │  │    │
│  │  └───────────────┘ └─────────────┘ └─────────────────┘  │    │
│  └─────────────────────────────────────────────────────────┘    │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Design Patterns Implemented

Pattern	Application
Singleton	`VRNetworkAvatarManager` - Global access point with scene persistence
Strategy	`IXRRigProvider` - Swappable XR tracking implementations
Dependency Injection	Runtime provider assignment via `SetXRRigProvider()`
Registry	Client-to-avatar mapping for O(1) lookups
Observer	Event-driven architecture for avatar lifecycle
Component-Based	Modular, composable avatar functionality

Technical Highlights

Client-Authoritative Transform Synchronization

VR applications require client-authoritative movement to ensure responsive tracking. The system overrides NGO's default server authority:

// ClientAuthorityNetworkTransform.cs
public class ClientAuthorityNetworkTransform : NetworkTransform
{
    protected override bool OnIsServerAuthoritative()
    {
        return false; // Client owns their transform
    }
}

This eliminates server round-trip latency for local avatar movement while maintaining server authority for spawning and validation.

Dead-Zone Filtering Algorithm

Network traffic is optimized by only transmitting changes that exceed configurable thresholds:

// Position: Uses squared magnitude to avoid sqrt computation
if (Vector3.SqrMagnitude(newPosition - lastSyncedPosition) > positionThreshold)
{
    SyncPosition(newPosition);
    lastSyncedPosition = newPosition;
}

// Rotation: Quaternion angle comparison
if (Quaternion.Angle(newRotation, lastSyncedRotation) > rotationThreshold)
{
    SyncRotation(newRotation);
    lastSyncedRotation = newRotation;
}

Physical Constraint System

Prevents uncanny valley effects from tracking anomalies:

// Arm length constraint - prevents stretching artifacts
Vector3 ClampHandPosition(Vector3 handPos, Vector3 bodyPos, float maxArmLength)
{
    Vector3 offset = handPos - bodyPos;
    if (offset.magnitude > maxArmLength)
        return bodyPos + offset.normalized * maxArmLength;
    return handPos;
}

// Head rotation clamping - prevents unnatural head tilt
Vector3 ClampHeadRotation(Vector3 euler, float maxAngle)
{
    euler.x = Mathf.Clamp(euler.x, -maxAngle, maxAngle);
    euler.z = Mathf.Clamp(euler.z, -maxAngle, maxAngle);
    return euler;
}

Installation

Via Unity Package Manager (UPM)

Add to your Packages/manifest.json:

{
  "dependencies": {
    "com.vrnetworkavatar": "file:../path/to/VRNetworkAvatar"
  }
}

Or import via Window > Package Manager > + > Add package from disk and select package.json.

Dependencies

Package	Version	Required
Unity Netcode for GameObjects	1.5.0+	Yes
TextMeshPro	3.0.0+	Yes
Meta XR SDK	Any	Optional*
Unity XR Interaction Toolkit	Any	Optional*

*At least one XR SDK required for VR functionality

Quick Start

1. Scene Setup

Scene Hierarchy:
├── NetworkManager (Unity Netcode)
├── VRNetworkAvatarManager (from Prefabs/)
│   ├── VRNetworkAvatarManager.cs
│   └── NetworkAvatarSpawner.cs
└── XRRig
    ├── OVRCameraRig (or XROrigin)
    └── OVRRigProvider (or GenericXRRigProvider)

2. Avatar Prefab Structure

NetworkAvatar (Prefab)
├── NetworkObject
├── ClientAuthorityNetworkTransform
├── NetworkVRAvatar
├── NetworkPlayerIdentity
├── AvatarFingerSync (optional)
├── Head/
├── Body/
├── LeftHand/
└── RightHand/

3. Basic Integration

using VRNetworkAvatar;

public class GameManager : MonoBehaviour
{
    void Start()
    {
        // Subscribe to avatar events
        VRNetworkAvatarManager.Instance.OnAvatarRegistered += OnPlayerJoined;
        VRNetworkAvatarManager.Instance.OnAvatarUnregistered += OnPlayerLeft;
    }

    void OnPlayerJoined(ulong clientId, GameObject avatar)
    {
        var identity = avatar.GetComponent<NetworkPlayerIdentity>();
        Debug.Log($"Player joined: {identity.DisplayName.Value}");
    }

    void OnPlayerLeft(ulong clientId)
    {
        Debug.Log($"Player {clientId} disconnected");
    }
}

Core Systems

Avatar Synchronization

The NetworkVRAvatar component handles real-time tracking synchronization:

// Synchronization occurs in LateUpdate for post-physics accuracy
private void LateUpdate()
{
    if (!IsOwner || !xrRigProvider.IsInitialized) return;

    // Sync tracking data with thresholds
    UpdateHeadTransform();
    UpdateHandTransform(HandSide.Left);
    UpdateHandTransform(HandSide.Right);
    UpdateBodyTransform(); // Derived from head position
}

Key Properties:

headTransform, leftHandTransform, rightHandTransform - Avatar bone references
bodyOffset - Vertical offset from head to body pivot
syncSettings - Per-avatar threshold configuration

XR Platform Abstraction

The IXRRigProvider interface enables multi-platform support:

public interface IXRRigProvider
{
    Transform HeadTransform { get; }
    Transform LeftHandTransform { get; }
    Transform RightHandTransform { get; }
    Transform RigRootTransform { get; }

    bool SupportsFingerTracking { get; }
    IReadOnlyList<Transform> GetLeftFingerTransforms();
    IReadOnlyList<Transform> GetRightFingerTransforms();

    bool IsInitialized { get; }
    void Initialize();
}

Included Implementations:

Provider	Platform	Finger Tracking
`OVRRigProvider`	Meta Quest, Rift	Yes
`GenericXRRigProvider`	Any Unity XR device	No

Custom Provider Example:

public class PicoRigProvider : MonoBehaviour, IXRRigProvider
{
    // Implement interface for Pico SDK
    public Transform HeadTransform => picoManager.HeadPose;
    public Transform LeftHandTransform => picoManager.LeftController;
    // ... etc
}

Network Optimization

Bandwidth Reduction Techniques:

Threshold Filtering - Skip updates below movement threshold
Squared Magnitude - Avoid sqrt in distance calculations
Selective Sync - Only changed values transmitted
Optional Components - Finger tracking disabled by default

Typical Bandwidth Savings:

Scenario	Without Optimization	With Optimization	Reduction
Standing still	60 updates/sec	2-5 updates/sec	~95%
Slow movement	60 updates/sec	15-20 updates/sec	~70%
Active gesturing	60 updates/sec	30-40 updates/sec	~40%

Constraint System

Prevents tracking artifacts and maintains avatar believability:

[System.Serializable]
public class AvatarConstraints
{
    [Tooltip("Maximum arm reach from body pivot")]
    public float maxArmLength = 0.7f;

    [Tooltip("Maximum head tilt angle (X and Z axes)")]
    public float headRotationClamp = 45f;

    [Tooltip("Interpolation speed for remote avatars")]
    public float interpolationSpeed = 15f;

    [Tooltip("Maximum extrapolation distance")]
    public float extrapolationLimit = 0.1f;
}

Configuration

Global Configuration (ScriptableObject)

Create via Assets > Create > VRNetworkAvatar > Configuration

[CreateAssetMenu(menuName = "VRNetworkAvatar/Configuration")]
public class VRNetworkAvatarConfig : ScriptableObject
{
    public GameObject defaultAvatarPrefab;
    public AvatarSyncSettings syncSettings;
    public string localAvatarLayer = "LocalAvatar";
    public bool disableLocalColliders = true;
    public bool spawnOnConnect = true;
    public bool despawnOnDisconnect = true;
    public XRPlatformPreference xrPlatform = XRPlatformPreference.Auto;
    public bool enableDebugLogging = false;
}

Sync Settings (Per-Avatar Tuning)

[CreateAssetMenu(menuName = "VRNetworkAvatar/Sync Settings")]
public class AvatarSyncSettings : ScriptableObject
{
    [Header("Position Sync")]
    [Range(0.0001f, 0.01f)]
    public float positionThreshold = 0.001f;

    [Header("Rotation Sync")]
    [Range(0.01f, 1f)]
    public float rotationThreshold = 0.1f;

    [Header("Finger Tracking")]
    [Range(0.01f, 1f)]
    public float fingerRotationThreshold = 0.1f;

    [Header("Constraints")]
    public float maxArmLength = 0.7f;
    public float headRotationClampAngle = 45f;

    [Header("Interpolation")]
    public float interpolationSpeed = 15f;
    public float extrapolationLimit = 0.1f;
}

Extending the System

Custom Avatar Events

Broadcast game-specific events to all clients:

// Send an event (e.g., wave gesture)
NetworkVRAvatar avatar = GetLocalAvatar();
avatar.SendAvatarEvent(AvatarEventType.Wave);

// Receive events
avatar.OnAvatarEvent += (clientId, eventType) =>
{
    if (eventType == AvatarEventType.Wave)
        PlayWaveAnimation(clientId);
};

Avatar Prefab Selection

Dynamic avatar assignment based on player data:

NetworkAvatarSpawner spawner = VRNetworkAvatarManager.Instance
    .GetComponent<NetworkAvatarSpawner>();

spawner.OnSelectAvatarPrefab = (clientId) =>
{
    // Return different prefabs based on player data
    PlayerData data = GetPlayerData(clientId);
    return avatarPrefabs[data.selectedAvatarIndex];
};

Custom Spawn Points

spawner.GetSpawnPosition = (clientId) =>
{
    // Custom spawn logic
    return spawnPoints[clientId % spawnPoints.Length].position;
};

spawner.GetSpawnRotation = (clientId) =>
{
    return Quaternion.Euler(0, clientId * 45f, 0);
};

API Reference