Skip to content

Conversation

@minhmo1620
Copy link
Contributor

@minhmo1620 minhmo1620 commented Jan 2, 2026

Problem Statement

Introduce a new feature flag for Flink-based venice views. This feature flag will be used as SOT for the views behavior.
Right now, default views behavior is on server ingestion path. However, we want to introduce a new view path where the ingestion goes from Flink producer directly.

Solution

This feature flag for now will disable the non-necessary write from controllers and servers such as EOP or VS since Flink connectors will handle that in the flink job.

AdminOperation v92->v93

diff services/venice-controller/src/main/resources/avro/AdminOperation/v92/AdminOperation.avsc services/venice-controller/src/main/resources/avro/AdminOperation/v93/AdminOperation.avsc
752a753,758
>             },
>             {
>               "name": "flinkVeniceViewsEnabled",
>               "doc": "Whether this store is enabled to use Flink-based Venice views",
>               "type": "boolean",
>               "default": false

StoreMetaValue.avsc v37 -> v38

diff internal/venice-common/src/main/resources/avro/StoreMetaValue/v37/StoreMetaValue.avsc internal/venice-common/src/main/resources/avro/StoreMetaValue/v38/StoreMetaValue.avsc 
312c312,313
<             {"name": "keyUrnFields", "type": {"type": "array", "items": "string"}, "default": [], "doc": "List of fields in the key schema that will be eligible for key urn compression"}
---
>             {"name": "keyUrnFields", "type": {"type": "array", "items": "string"}, "default": [], "doc": "List of fields in the key schema that will be eligible for key urn compression"},
>             {"name": "flinkVeniceViewsEnabled", "type": "boolean", "default": false, "doc": "Flag to indicate whether Flink Venice Views are enabled for this store."}

Code changes

  • Added new code behind a config. If so list the config names and their default values in the PR description.
  • Introduced new log lines.
    • Confirmed if logs need to be rate limited to avoid excessive logging.

Concurrency-Specific Checks

Both reviewer and PR author to verify

  • Code has no race conditions or thread safety issues.
  • Proper synchronization mechanisms (e.g., synchronized, RWLock) are used where needed.
  • No blocking calls inside critical sections that could lead to deadlocks or performance degradation.
  • Verified thread-safe collections are used (e.g., ConcurrentHashMap, CopyOnWriteArrayList).
  • Validated proper exception handling in multi-threaded code to avoid silent thread termination.

How was this PR tested?

  • New unit tests added.
  • New integration tests added.
  • Modified or extended existing tests.
  • Verified backward compatibility (if applicable).

Does this PR introduce any user-facing or breaking changes?

  • No. You can skip the rest of this section.
  • Yes. Clearly explain the behavior change and its impact.

@minhmo1620 minhmo1620 marked this pull request as ready for review January 2, 2026 19:59
ZacAttack
ZacAttack previously approved these changes Jan 2, 2026
Copy link
Contributor

@ZacAttack ZacAttack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No logic changes in this one. Just the protocol upgrade. Probably nothing can go wrong. Ship it!

@minhmo1620 minhmo1620 merged commit 394ce61 into linkedin:main Jan 5, 2026
50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants