[Draft]Datafusion and Parquet plugin config interaction

[alchemist51]

The PR brings in the pub-sub model for Dataformat and Engine plugins for settings interactions.

We have followed the following hierarchy model:
Override Settings in Dataformat >> Defaults in Dataformat >> Settings defined in Engine plugin.

In case of no overrides in Dataformat, Engine will decide the settings to be used. However in case of conflicts, dataformat will get the priority.

The below diagram depicts the engine initialisation:

┌─────────────────┐
│ DataFusion      │
│ Engine          │
└────────┬────────┘
         │ 1. Get base config
         │
         ▼
┌─────────────────────────┐
│ DataFusionEngineConfig  │
│ batchSize = 4096        │
└────────┬────────────────┘
         │ 2. Request override
         │
         ▼
┌─────────────────────────┐
│ ParquetDataSourceCodec  │
│ updateEngineConfig()    │
└────────┬────────────────┘
         │ 3. Override values
         │
         ▼
┌─────────────────────────┐
│ EngineConfig            │
│ batchSize = 8192        │
│ (Parquet overrides)     │
└─────────────────────────┘

The below diagram depicts on how the setting flow work in this model

┌─────────────────┐
│ Parquet Plugin  │ 
│ (Publisher)     │
└────────┬────────┘
         │ 1. Settings change
         │
         ▼
┌─────────────────────────┐
│ ParquetSessionConfig    │
│ setBatchSize(8192)      │
└────────┬────────────────┘
         │ 2. Publish update
         │
         ▼
┌─────────────────────────┐
│ SessionConfigRegistry   │
│ publishSessionConfig()  │
└────────┬────────────────┘
         │ 3. Notify listeners
         │
         ▼
┌─────────────────────────┐
│ DataFusionPlugin        │
│ (Subscriber)            │
│ onSessionConfigUpdate() │
└────────┬────────────────┘
         │ 4. Merge config
         │
         ▼
┌─────────────────────────┐
│ DataFusionEngineConfig  │
│ updateSessionConfig()   │
└─────────────────────────┘

There are few problems which I could see with this approach:

1. In case of 1000+ listeners from a dataformat, the subscribers might see delay in the updates
2. In case of index settings being introduced, we will need a subscriber method to listen to index setting updates. Probably clusterStateUpdate event? Though I'm not sure on how it will create an impact since it will get called in everyUpdate and will need to be propagated to all of the subscriber.