[2/3] sdks/python: enrich data with Milvus Search [Vector, Keyword, Hybrid]

[mohamedawnallah]

Description

• Make Milvus as extra dependency
• Mark Milvus itests as require_docker_in_docker to run ubuntu-latest

Towards #35046.
Prev #35216.
Next #35467.

Motivation and Context

Follow-up on the discussion here #35216 (comment).

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

[github-actions]

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

[github-actions]

Assigning reviewers:

R: @claudevdm for label python.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[mohamedawnallah]

stop reviewer notifications

[github-actions]

Stopping reviewer notifications for this pull request: requested by reviewer. If you'd like to restart, comment assign set of reviewers

[mohamedawnallah]

Hey @damccorm, I haven't experimented with this yet and not sure about the feasibility of this suggestion I think if we update the ubuntu version to latest for self-hosted runners it would probably solve this docker_in_docker environment issue entirely and may be future related issues

beam/.github/gh-actions-self-hosted-runners/self-hosted-linux/docker/Dockerfile

Line 19 in e8d4395

FROM ubuntu:20.04

The minimum recommend ubuntu version for docker to run stably is Ubuntu Jammy 22.04 (LTS) as seen in https://docs.docker.com/engine/install/ubuntu/#os-requirements

I am still uncertain if there are beam related dependencies in the CI testing environment that depends on the existing ubuntu version 20.04

What do you think?

cc @Abacn (may help here as well – it is related to infra)

EDIT:
Just saw Danny's comment I missed those github notifications:
#35598 (comment)

[mohamedawnallah]

Hey @damccorm, With this PR we would have milvus tests marked with docker_in_docker as well as making it extra dependency. After this PR, we can have ubuntu-latest on postcommit python and precommit ml as in this PR #35816

EDIT:
I have looked the test failures. They appear to me flaky?

[damccorm]

This LGTM - I'm going to let tests run to try to get a green signal and then I'll merge if they don't raise any real problems

[damccorm]

Flakes look unrelated