Add Sentry PowerShell SDK integration for telemetry

[vaind]

Overview

Integrate sentry-powershell SDK to collect telemetry from app-runner and sentry-api-client modules. This will provide operational visibility into test infrastructure failures, device connection issues, and automation errors.

Implementation Plan

Phase 1: Core Infrastructure

1. Create utils/TrySentry.psm1

Internal function:

• Ensure-SentryReady - Centralized function that:
  • Checks if $env:SENTRY_DSN is empty/null → return false (disabled)
  • Checks if [Sentry.SentrySdk] type exists (module already loaded)
  • If not loaded, attempts Import-Module Sentry
  • Checks [Sentry.SentrySdk]::IsEnabled (already initialized)
  • If not initialized, calls Start-Sentry -Dsn ($env:SENTRY_DSN ?? $script:DefaultDsn)
  • Returns true on success, false on failure
  • All failures are silent (Write-Debug only)

Public functions:

# Optional early initialization with context
TryStart-Sentry -ModuleName <string> -ModuleVersion <string> -Tags <hashtable>

# Wrapper functions (all call Ensure-SentryReady internally)
TryOut-Sentry -InputObject <object> -Tag <hashtable> -Level <string>
TryAdd-SentryBreadcrumb -Message <string> -Category <string> -Data <hashtable>
TryEdit-SentryScope -ScopeSetup <scriptblock>

# Performance monitoring
TryStart-SentryTransaction -Name <string> -Operation <string>
# Returns transaction object that can be used to create spans and finish

Configuration:

• Default DSN: Hardcoded $script:DefaultDsn (single project for all modules)
• Override: $env:SENTRY_DSN takes precedence
• Disable: Set $env:SENTRY_DSN to empty string or $null

Behavior:

• All wrappers fail silently if SDK unavailable or disabled
• Auto-load module on first use
• Auto-initialize on first use
• Accept external initialization (if caller already started Sentry, use their DSN)

---

Phase 2: Module Integration

2. Initialize in app-runner/SentryAppRunner.psm1

At the top of module file:

Import-Module (Join-Path $PSScriptRoot '../utils/TrySentry.psm1')
TryStart-Sentry -ModuleName 'SentryAppRunner' -ModuleVersion $MyInvocation.MyCommand.ScriptBlock.Module.Version

Sets default tags:

• module_name
• module_version
• powershell_version
• os

3. Initialize in sentry-api-client/SentryApiClient.psm1

Same pattern as app-runner module.

---

Phase 3: Error & Diagnostic Integration

4. Enhance app-runner/Private/ErrorHandling.ps1

Modify ErrorHandler::LogError() method:

static [void]LogError([ConsoleException]$exception) {
    # Existing error log code...
    
    # Send to Sentry
    TryOut-Sentry -InputObject $exception.GetDetailedMessage() -Tag @{
        category = $exception.Category.ToString()
        platform = $exception.Platform
        session_id = $exception.SessionId
        error_id = $errorId
    }
}

Includes all exception context (category, platform, session ID, inner exception).

5. Add breadcrumbs to critical operations

Key locations:

• app-runner/Public/Connect-Device.ps1

  • Before/after device lock acquisition
  • Before/after provider creation
  • Before/after connection establishment

• app-runner/Public/Invoke-DeviceApp.ps1

  • Before app installation
  • Before app launch
  • Before log collection

• utils/Integration.TestUtils.psm1

  • Get-SentryTestEvent: Before polling, on timeout
  • Get-EventIds: When EVENT_CAPTURED lines missing

Example:

TryAdd-SentryBreadcrumb -Message "Acquiring device lock: $resourceName" -Category "device"
# ... operation ...
TryAdd-SentryBreadcrumb -Message "Device lock acquired" -Category "device"

6. Add performance monitoring to long operations

Use transactions and spans to track performance of critical operations:

Example - Device Connection:

$transaction = TryStart-SentryTransaction -Name "Connect-Device" -Operation "device.connect"
try {
    $span = $transaction?.StartChild("device.lock.acquire")
    # ... acquire lock ...
    $span?.Finish()
    
    $span = $transaction?.StartChild("device.provider.create")
    # ... create provider ...
    $span?.Finish()
    
    $span = $transaction?.StartChild("device.connection.establish")
    # ... establish connection ...
    $span?.Finish()
}
finally {
    $transaction?.Finish()
}

Target operations:

• Connect-Device - Track lock acquisition, provider creation, connection time
• Invoke-DeviceApp - Track app installation, launch, and execution time
• Get-SentryTestEvent - Track polling duration and API call performance
• Invoke-CMakeBuild - Track build duration per target

---

Phase 4: Documentation

7. Update root README.md

Add new section:

## Telemetry

This toolkit automatically collects operational telemetry using Sentry to improve reliability and diagnose issues. Events collected include:

- Module errors and exceptions
- Device connection failures
- Test infrastructure issues
- Diagnostic operation breadcrumbs
- Performance metrics for critical operations

**To disable telemetry:**
```powershell
$env:SENTRY_DSN = $null  # or empty string

To use a custom Sentry project:

$env:SENTRY_DSN = 'https://your-key@o123.ingest.sentry.io/your-project'

DSNs are public client keys and safe to expose. See Sentry DSN documentation.

---

## Technical Decisions

| Decision | Rationale |
|----------|-----------|
| Single DSN for all modules | Simplifies management, all events in one project |
| Empty `$env:SENTRY_DSN` to disable | Standard Sentry SDK pattern across languages |
| Accept external initialization | If caller already initialized, use their configuration |
| Silent failures in wrappers | Don't break functionality if Sentry unavailable |
| Auto-load on first use | Lazy loading, no hard dependency |
| Performance monitoring | Track operation duration to identify bottlenecks |

---

## Testing Checklist

- [ ] Module loads successfully without Sentry installed
- [ ] Module loads successfully with Sentry installed
- [ ] Telemetry disabled when `$env:SENTRY_DSN = $null`
- [ ] Events appear in Sentry when errors occur
- [ ] Breadcrumbs attached to error events
- [ ] Module tags present on all events
- [ ] External initialization respected (if caller starts Sentry first)
- [ ] Transactions and spans appear in Sentry performance monitoring
- [ ] All existing tests still pass

---

## Dependencies

- **Optional:** `Sentry` PowerShell module from PSGallery
  - Install: `Install-Module -Name Sentry`
  - Repo: https://github.com/getsentry/sentry-powershell
  - Module automatically loaded on first use if available

---

## Related Documentation

- [Sentry PowerShell SDK Usage](https://docs.sentry.io/platforms/powershell/usage/)
- [Sentry DSN Explainer](https://docs.sentry.io/product/sentry-basics/dsn-explainer/)
- [Sentry Performance Monitoring](https://docs.sentry.io/platforms/powershell/tracing/)