Add global Lakehouse Schemas for Managing Batch Diagnostic Information #1918
Description
What are the schemas that are affected by the issue
No existing schemas are impacted.
New schemas created are:
- aep-lakehouse-batch-input-files-timeseries.schema.json
- aep-lakehouse-batch-errors-timeseries.schema.json
What are examples of products that are impacted by the issue
AEP Foundation/Data Landing
Relevant Documentation - https://experienceleague.adobe.com/en/docs/experience-platform/ingestion/quality/error-diagnostics
Internal JIRA - https://jira.corp.adobe.com/browse/PLAT-214882
Context
We are deprecating the Data Access API service. which is provides access to Batch Diagnostic data. The need for this deprecation has been accelerated due to support in AWS where the Data Access API is unavailable. The Lakehouse team is providing alternative access to this data through system datasets that reference these global schemas.