Skip to content

etw-internals.md

Latest commit

 

History

History
170 lines (125 loc) · 5.73 KB

etw-internals.md

File metadata and controls

170 lines (125 loc) · 5.73 KB

ETW Internals Deep-Dive

Overview

Event Tracing for Windows (ETW) is the core tracing facility built into the Windows kernel. It provides a high-performance, low-overhead mechanism for logging events from both kernel-mode and user-mode components.

This document explores the internal architecture of ETW relevant to security research and telemetry analysis.

Architecture

Three-Component Model

ETW operates on a provider-session-consumer model:

  1. Providers — Generate events (kernel drivers, user-mode DLLs, applications)
  2. Sessions (Controllers) — Manage trace sessions, configure buffering, route events
  3. Consumers — Receive and process events in real-time or from log files

Kernel-Mode Components

The ETW subsystem lives primarily in the Windows kernel (ntoskrnl.exe):

  • EtwpLogger — Core logging engine that manages trace sessions
  • EtwpRegistrationTable — Hash table of all registered providers
  • WMI_LOGGER_CONTEXT — Per-session structure containing buffers, configuration
  • ETW_REG_ENTRY — Per-provider registration entry with callback information

User-Mode Components

The primary user-mode interface is through ntdll.dll:

  • EtwEventRegister — Register a provider with a GUID and callback
  • EtwEventWrite — Write an event to all enabled sessions
  • EtwEventWriteFull — Extended event write with activity correlation
  • EtwEventUnregister — Unregister a provider

Event Flow

Provider Registration

Application -> ntdll!EtwEventRegister
           -> ntdll!NtTraceControl (TraceControlRegisterProvider)
           -> nt!EtwpRegisterProvider
           -> Creates ETW_REG_ENTRY in EtwpRegistrationTable

Event Writing

Application -> ntdll!EtwEventWrite
           -> ntdll!EtwEventWriteFull (internal)
           -> ntdll!NtTraceEvent (syscall)
           -> nt!EtwpEventWriteFull
           -> Copies event to session buffer(s)
           -> Consumer receives via ProcessTrace callback

Buffer Management

ETW uses a per-session buffer pool:

  • Buffer Size: Configurable (4 KB to 1 MB, default 64 KB)
  • Buffer Count: Min/Max configurable
  • Free List: Unused buffers available for allocation
  • Flush List: Filled buffers waiting to be written/consumed
  • Allocation: Lock-free per-processor buffer allocation for performance

Buffer Lifecycle

  1. Thread acquires a buffer from the free list (per-processor for scalability)
  2. Event data is copied into the buffer
  3. When buffer is full, it moves to the flush list
  4. Consumer or log file writer drains the flush list
  5. Drained buffers return to the free list

Session Configuration

EVENT_TRACE_PROPERTIES

Key fields:

  • BufferSize — Size of each buffer in KB
  • MinimumBuffers / MaximumBuffers — Buffer pool bounds
  • FlushTimer — Automatic flush interval in seconds
  • LogFileMode — Flags: real-time, file, circular, buffering-only
  • EnableFlags — For kernel logger: which system events to trace

Trace Session Limits

  • Maximum Sessions: 64 on most Windows versions (increased in newer builds)
  • Session 0: Reserved for the NT Kernel Logger
  • AutoLogger Sessions: Start automatically at boot via registry configuration

Provider Enable Parameters

When enabling a provider on a session via EnableTraceEx2:

  • Level — Minimum severity (Verbose=5, Info=4, Warning=3, Error=2, Critical=1)
  • MatchAnyKeyword — Event must match at least one of these keyword bits
  • MatchAllKeyword — Event must match ALL of these keyword bits
  • EnableProperty — Additional flags (stack trace, security ID, etc.)
  • Filters — Schematized filters for event-ID-level filtering

Security-Relevant Providers

Microsoft-Windows-Threat-Intelligence

This kernel-mode provider is critical for security products:

  • Requires Protected Process Light (PPL) to consume
  • Provides memory operation events (allocation, protection changes)
  • Used by Windows Defender ATP / Microsoft Defender for Endpoint

ETW Provider Hierarchy

Kernel Providers (nt!EtwpSystemTraceProvider)
  |-- Process events (create, terminate, image load)
  |-- Thread events (create, terminate, context switch)
  |-- File I/O events
  |-- Registry events
  |-- Network events (TCP/IP, UDP)
  |-- Memory events (page fault, virtual alloc)

User-Mode Providers
  |-- Microsoft-Windows-PowerShell
  |-- Microsoft-Windows-DotNETRuntime
  |-- Microsoft-Windows-DNS-Client
  |-- Microsoft-Windows-Security-Auditing
  |-- (thousands more...)

Relevant Structures

ETW_REG_ENTRY (Undocumented)

typedef struct _ETW_REG_ENTRY {
    LIST_ENTRY RegList;          // Global registration list
    LIST_ENTRY GroupRegList;     // Group registration list
    GUID ProviderId;             // Provider GUID
    PETWENABLECALLBACK Callback; // Enable/disable notification callback
    PVOID CallbackContext;       // Callback context pointer
    ULONG Index;                 // Registration index
    ULONG Flags;                 // Registration flags
    // ... additional fields vary by Windows version
} ETW_REG_ENTRY;

WMI_LOGGER_CONTEXT (Undocumented)

This structure represents an active trace session in the kernel:

typedef struct _WMI_LOGGER_CONTEXT {
    ULONG LoggerId;
    ULONG BufferSize;
    ULONG MaximumBuffers;
    ULONG MinimumBuffers;
    volatile ULONG NumberOfBuffers;
    volatile ULONG FreeBuffers;
    // ... many more fields
} WMI_LOGGER_CONTEXT;

References