feat: Support AdCP v3 structured geo targeting (#1006)

[KonstantinMirin]

Summary

Implements AdCP v3 structured geo targeting syntax. What started as a schema migration revealed high coupling between targeting types, validation layers, adapters, and serialization paths — each layer had assumptions about field names and data shapes that required coordinated changes.

Why this is larger than "just add structured fields"

The original Targeting class was a standalone Pydantic model with flat string lists (geo_country_any_of: list[str]). The AdCP v3 spec requires inheriting from the library's TargetingOverlay with typed structured objects (geo_metros: list[GeoMetro] where GeoMetro has a system enum and values list). Making this switch exposed problems at every layer:

1. Schema layer — Targeting couldn't just add new fields; it had to inherit from TargetingOverlay to maintain isinstance checks and protocol compliance. This changed the class hierarchy and required overriding frequency_cap to use our extended type with scope.

2. Serialization layer — GeoMetro.system is a MetroAreaSystem enum. model_dump() (default mode='python') keeps it as an enum object. When psycopg2 calls json.dumps() for JSONB storage, it raises TypeError: Object of type MetroAreaSystem is not JSON serializable. This means any buyer sending metro or postal targeting would crash the media buy create/update flow at session.commit(). Fixed by defaulting Targeting.model_dump() to mode='json'.

3. Validation layer — The overlay validator used a suffix-stripping heuristic (geo_country_any_of → strip _any_of → geo_country) to map field names to targeting dimensions. V3 field names (geo_countries, geo_metros) don't follow this convention. The heuristic silently passed unknown fields through without error. Replaced with an explicit FIELD_TO_DIMENSION mapping table, and added model_extra inspection to catch truly unknown fields (typos, bogus names).

4. Adapter layer — All 5 adapters (GAM, Kevel, Triton, Xandr, Mock) read targeting fields by the old flat names. Each needed updating to read from the new structured types. GAM additionally needed to extract system and values from GeoMetro/GeoPostalArea objects and validate the system matches its capabilities.

5. Normalizer — Existing database records store flat field names. A legacy normalizer converts v2 → v3 at parse time so old data reconstructs correctly. This required hardening for edge cases: bare region codes without country prefix, both v2+v3 fields present simultaneously, and the removed geo_city dimension.

6. Codebase-wide rename — ~60 test files and fixtures referenced the old field names (geo_country_any_of, geo_metro_any_of, etc.). These needed updating to v3 names to match the new schema.

Bugs found and fixed during implementation

Bug	Severity	Impact
model_dump() produces non-JSON-serializable enum objects	P1	Metro/postal targeting crashes DB writes at session.commit()
Suffix-stripping heuristic fails for v3 field names	P2	Unknown/unmapped targeting fields silently accepted
model_extra fields never inspected	P2	Buyer typos (geo_contries) silently ignored
City targeting silently dropped	P2	geo_city_any_of accepted but never reached adapter
Bare region codes ("CA") not normalized	P2	Region targeting fails unless buyer knows to send "US-CA"
Storage key mismatch after schema change	P2	Targeting data lost on DB round-trip

Changes

Core schema (src/core/schemas.py)

• Targeting now extends TargetingOverlay from adcp library
• FrequencyCap extends LibraryFrequencyCap with scope field
• Added geo exclusion fields (geo_countries_exclude, geo_metros_exclude, etc.)
• Legacy normalizer converts flat v2 field names to v3 structured format
• model_dump() / model_dump_internal() default to mode='json' for DB safety

Adapters (src/adapters/)

• GAM: Full v3 structured geo handling — extracts system/values from GeoMetro/GeoPostalArea, validates system against adapter capabilities, handles exclusion variants
• Kevel/Triton/Xandr/Mock: Updated to read from v3 field names
• Base adapter: Added validate_geo_systems() — adapters declare which metro/postal systems they support, requests using unsupported systems get clear errors

Validation (src/services/targeting_capabilities.py)

• Replaced suffix-stripping heuristic with explicit FIELD_TO_DIMENSION mapping
• Added validate_unknown_targeting_fields() using Pydantic model_extra
• Both inclusion and exclusion field variants mapped to the same capability dimension
• Added validate_geo_overlap() — rejects same-value overlap between inclusion and exclusion fields at the same geo level (AdCP SHOULD requirement from adcp PR fix: preserve format dimensions during media buy approval #1010)

Test coverage (12 new test files, ~220 new tests)

• test_v3_geo_targeting.py — Type construction, model_dump JSON safety, FrequencyCap inheritance
• test_v3_targeting_roundtrip.py — construct→dump→reconstruct identity, DB storage simulation
• test_gam_targeting_v3.py — GAM adapter structured geo handling
• test_validate_geo_systems.py — Adapter system validation
• test_targeting_normalizer.py — Legacy v2→v3 conversion
• test_overlay_validation_v3.py — Explicit field mapping validation
• test_unknown_targeting_fields.py — model_extra unknown field detection
• test_city_targeting_rejected.py — Removed dimension rejection
• test_targeting_storage_key.py — DB round-trip key consistency
• test_adapter_v3_geo_fields.py — Non-GAM adapter v3 consumption
• test_geo_overlap_validation.py — Geo inclusion/exclusion same-value overlap rejection
• test_adcp_contract.py — Updated for new type hierarchy

Test plan

• All 1918 unit tests pass (uv run pytest tests/unit/ -x)
• AdCP contract compliance: isinstance(Targeting(...), TargetingOverlay) verified
• DB serialization safety: json.dumps(Targeting(geo_metros=[...]).model_dump()) succeeds
• Legacy backward compatibility: v2 field names normalize to v3 at parse time
• Adapter system validation: GAM rejects eurostat_nuts2, accepts nielsen_dma
• Geo overlap validation: same value in inclusion + exclusion at same level → rejected

Closes #1006

---

Update: Post-review changes (3 new commits)

1. had_city_targeting Pydantic Field cleanup

Replaced the _had_city_targeting transient attribute (manually managed via __dict__ and hardcoded exclusions in model_dump()) with a proper Pydantic Field(default=False, exclude=True). Pydantic forbids underscore-prefixed field names, so renamed to had_city_targeting. The exclude=True annotation handles serialization exclusion automatically — no more hardcoded string lists in model_dump() and model_dump_internal().

2. CreateMediaBuyRequest.packages type override

When a buyer sends targeting_overlay inside a package, the request JSON is first parsed by CreateMediaBuyRequest. The parent class (LibraryCreateMediaBuyRequest) defines packages: list[LibraryPackageRequest], and the library's PackageRequest types targeting_overlay as TargetingOverlay. This means Pydantic constructs a TargetingOverlay instance — which bypasses our legacy normalizer and doesn't carry our extension fields.

Later, when the code constructs MediaPackage(targeting_overlay=...), it fails because MediaPackage expects Targeting (our extended type), not the library's TargetingOverlay.

Fix: override packages on CreateMediaBuyRequest to reference our PackageRequest (which already overrides targeting_overlay: Targeting). This follows the same inheritance pattern used throughout the codebase — CreateMediaBuyRequest had a gap where it extended the library but forgot to override packages to use our PackageRequest.

3. Geo overlap integration test

Added tests/integration/test_targeting_validation_chain.py — tests the full validation chain with a real database, confirming that geo overlap rejection works end-to-end (not just in unit isolation).

Review response

Addressed @bokelley's comment on targeting_capabilities.py — annotated both TARGETING_CAPABILITIES and FIELD_TO_DIMENSION to distinguish AdCP-defined fields from seller extensions, with a note that the seller extensions are candidates for upstream AdCP inclusion.

[github-actions]

All contributors have agreed to the IPR Policy. Thank you!
_{Posted by the CLA Assistant Lite bot.}

[KonstantinMirin]

I have read the IPR policy

[bokelley]

not all of these are in the adcp list - we could conceptually add them

[KonstantinMirin]

Good catch. Checked against adcp.types.TargetingOverlay — it defines 8 fields:

• geo_countries, geo_regions, geo_metros, geo_postal_areas
• frequency_cap, property_list
• axe_include_segment, axe_exclude_segment

Of the 26 entries in FIELD_TO_DIMENSION, only 5 map to actual AdCP fields (the 4 geo fields + frequency_cap). The geo exclusion fields (geo_countries_exclude, etc.) are our PR #1006 extensions. The rest — device_type_any_of, os_any_of, browser_any_of, media_type_any_of, audiences_any_of, content_cat_any_of, custom — are seller extensions carried forward from the original implementation, which had Targeting as a standalone BaseModel that never inherited from the AdCP library.

These seller extensions are actively consumed by adapters (GAM, Kevel, Triton, Xandr all read device type, media type, etc.) — they're standard ad-server dimensions that AdCP hasn't adopted yet. We've annotated both TARGETING_CAPABILITIES and FIELD_TO_DIMENSION to clearly distinguish AdCP-defined fields from seller extensions, with a note that the extensions are candidates for upstream AdCP inclusion.