Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
cebe49c
sdks/python: add pymilvus dependency
mohamedawnallah Jun 9, 2025
9f3507d
sdks/python: add `MilvusSearchEnrichmentHandler`
mohamedawnallah Jun 9, 2025
153bff9
sdks/python: test `MilvusSearchEnrichmentHandler`
mohamedawnallah Jun 9, 2025
602f5f7
sdks/python: itest `MilvusSearchEnrichmentHandler`
mohamedawnallah Jun 9, 2025
7851651
examples: add `MilvusSearchEnrichmentHandler`
mohamedawnallah Jun 9, 2025
b00b640
sdks/python: combine milvus search strategies in one
mohamedawnallah Jun 14, 2025
8716ac4
Merge remote-tracking branch 'upstream/master' into enrichWithMilvus
mohamedawnallah Jun 14, 2025
716a89b
Merge remote-tracking branch 'upstream/master' into enrichWithMilvus
mohamedawnallah Jun 14, 2025
4e34b2d
sdks/python/container: update image requirements
mohamedawnallah Jun 14, 2025
f8b8413
sdks/python: add license for `milvus_search.py`
mohamedawnallah Jun 14, 2025
53d55e1
sdks/python: add docstrings for `milvus_search.py`
mohamedawnallah Jun 15, 2025
f029d69
sdks/python: unit test milvus search handler
mohamedawnallah Jun 15, 2025
3830c04
sdks/python: update docstrings for milvus_search
mohamedawnallah Jun 15, 2025
f262a86
sdks/python: fix linting for `milvus_search.py`
mohamedawnallah Jun 15, 2025
71a6dfe
sdks/python: add more unit tests for milvus search
mohamedawnallah Jun 15, 2025
23e60ad
sdks/python: combine test classes in one
mohamedawnallah Jun 15, 2025
3206df7
sdks/python: add `setuptools` as dependency
mohamedawnallah Jun 15, 2025
1abd50e
sdks/python: update container image requirements
mohamedawnallah Jun 15, 2025
37c360b
sdks/python: update definition for `ANNS` field
mohamedawnallah Jun 15, 2025
ea6bf84
.github/workflows: upgrade pip & setuptools
mohamedawnallah Jun 15, 2025
f1521e4
sdks/python: fix linting issue for `milvus_search`
mohamedawnallah Jun 15, 2025
ac440fa
sdks/python: group I/O types together milvus_search
mohamedawnallah Jun 15, 2025
d326f37
.github/workflows: upgrade pip & setuptools
mohamedawnallah Jun 15, 2025
079e620
.github: unify upgrading setuptools & pip
mohamedawnallah Jun 15, 2025
7f433d9
sdks/python: fix linting for `milvus_search.py`
mohamedawnallah Jun 15, 2025
56873df
sdks/python: update grpcio for py<=3.12
mohamedawnallah Jun 16, 2025
3494543
sdks/python: update image requirements
mohamedawnallah Jun 16, 2025
9bc2297
sdks/python: add `milvus-lite` manual license
mohamedawnallah Jun 16, 2025
ecc7725
Merge remote-tracking branch 'upstream/master' into enrichWithMilvus
mohamedawnallah Jun 17, 2025
96acb9a
sdks/python: fix `milvus_search_it_test` failed cases
mohamedawnallah Jun 17, 2025
fa9bc82
sdks/python: unify access to sparse/dense embeddings
mohamedawnallah Jun 18, 2025
3aa177c
sdks/python: invoke `unittest.main` on milvus search
mohamedawnallah Jun 18, 2025
bcdb65f
Merge remote-tracking branch 'upstream/master' into enrichWithMilvus
mohamedawnallah Jun 18, 2025
6191c13
sdks/pyhon: make `MilvusSearchDBContainerInfo` optional for linting
mohamedawnallah Jun 18, 2025
40e390d
sdks/python+website: update docs
mohamedawnallah Jun 19, 2025
297f658
sdks/python: fix linting issues for `milvus_search` component
mohamedawnallah Jun 19, 2025
4813efd
sdks/python: fix linting issues for milvus search component
mohamedawnallah Jun 19, 2025
8c09fd0
website: add missing doc for milvus search
mohamedawnallah Jun 19, 2025
1020c34
sdks/python: add itests for milvus search
mohamedawnallah Jun 19, 2025
2f5e45a
sdks/python: complete itests for milvus search
mohamedawnallah Jun 21, 2025
904376d
sdks/python: fix linting
mohamedawnallah Jun 21, 2025
01f1ada
sdks/python: address (can be) flaky test cases
mohamedawnallah Jun 21, 2025
d355653
website: update relase version for `enrichment-milvus.md`
mohamedawnallah Jun 21, 2025
0f4ac91
sdks/python: fix failed unit tests for milvus search
mohamedawnallah Jun 21, 2025
7aa2555
sdks/python: fix linting for milvus search itests
mohamedawnallah Jun 21, 2025
5a8dc48
website: update docs html to ref milvus enrichment handler
mohamedawnallah Jun 21, 2025
d057a6e
sdks/python: avoid port collision for milvus container
mohamedawnallah Jun 21, 2025
e43e7de
Merge remote-tracking branch 'upstream/master' into enrichWithMilvus
mohamedawnallah Jun 21, 2025
a50b001
sdks/python: remove free port allocation for milvus search
mohamedawnallah Jun 21, 2025
ede2dcd
sdks/python: fix formatting issues for milvus search
mohamedawnallah Jun 21, 2025
82249ab
sdks/python: fix linting for milvus_search_it_test
mohamedawnallah Jun 21, 2025
89cdc77
sdks/python: handle port collisions for milvus search itest
mohamedawnallah Jun 21, 2025
4bf0967
sdks/python: increase timeout for milvus container
mohamedawnallah Jun 21, 2025
b9139b2
sdks/python: experiment being explicit about the port solve the CI issue
mohamedawnallah Jun 22, 2025
cbcd386
sdks+.github: experiment running ml deps CI test onubuntu solve issue
mohamedawnallah Jun 22, 2025
0c05e3d
.github/workflwos: revert python precommit ml changes
mohamedawnallah Jun 22, 2025
6e87939
sdks/python: fix CI issues for itests
mohamedawnallah Jun 22, 2025
edf53b2
sdks/python: fix linting for milvus search itests
mohamedawnallah Jun 22, 2025
01988df
examples/notebook: update milvus enrichment transform
mohamedawnallah Jun 22, 2025
c3eeb4b
website: update milvus enrichment transform
mohamedawnallah Jun 22, 2025
fc60683
CHANGES.md: add note for milvus enrichment handler
mohamedawnallah Jun 22, 2025
468e630
sdks/python: update itests for milvus search
mohamedawnallah Jun 22, 2025
98f9381
sdks/python: fix linting issues
mohamedawnallah Jun 22, 2025
daaf172
multi: update
mohamedawnallah Jun 23, 2025
a0deccc
multi: update
mohamedawnallah Jun 23, 2025
faa8b3f
updatet
mohamedawnallah Jun 23, 2025
b5d26c0
update
mohamedawnallah Jun 23, 2025
ad81ff3
update
mohamedawnallah Jun 23, 2025
d4552cf
sdks/python: fix linting issues
mohamedawnallah Jun 23, 2025
22456e0
sdks/python: see what CI workflows would fail
mohamedawnallah Jun 23, 2025
af4143f
.github: run beam_PreCommit_Python_ML only on ubuntu-20.04 runner
mohamedawnallah Jun 23, 2025
0eba232
.github: test workflow
mohamedawnallah Jun 23, 2025
78bc5bc
.github: revert changes
mohamedawnallah Jun 23, 2025
6569a1f
.github: add milvus-integration-tests.yml
mohamedawnallah Jun 23, 2025
84ae1f3
.github: update milvus it workflow
mohamedawnallah Jun 23, 2025
17afaa5
update
mohamedawnallah Jun 23, 2025
9c82d87
.github: update milvus-tests workflow
mohamedawnallah Jun 23, 2025
13baf7c
.github: try to use ubuntu version `ubuntu-20.04`
mohamedawnallah Jun 23, 2025
8c99d06
.github+sdks/python: update itests
mohamedawnallah Jun 23, 2025
d73c020
.github: update gh runner for milvus itests
mohamedawnallah Jun 23, 2025
789ff01
.github: update milvus itests workflow
mohamedawnallah Jun 23, 2025
a3695df
.github+sdks/python: update itests
mohamedawnallah Jun 23, 2025
131fff8
.github: remove `milvus-integration-tests.yml` for the PR review
mohamedawnallah Jun 23, 2025
685c26f
sdks/python: skip itests properly if milvus db container failed to start
mohamedawnallah Jun 23, 2025
4281e41
skds/python: restructure the code order in the example
mohamedawnallah Jun 23, 2025
f4a2552
sdks/python: reduce number of retries to avoid test timeout
mohamedawnallah Jun 23, 2025
d871144
sdks/python: set internal testcontainer env variable for max retries
mohamedawnallah Jun 23, 2025
257b0cd
sdks/python: update tc max retries
mohamedawnallah Jun 23, 2025
c01945d
sdks/python: update
mohamedawnallah Jun 23, 2025
85a7c11
sdks/python: use dynamic milvus service and healthcheck ports
mohamedawnallah Jun 23, 2025
229abb6
sdks/python: fix linting issues for milvus search itest
mohamedawnallah Jun 24, 2025
0f07595
sdks/python: fixing linting issues for milvus search itests
mohamedawnallah Jun 24, 2025
a3dedc9
.github+sdks/python: reconfigure dependencies
mohamedawnallah Jun 27, 2025
622a1c5
sdks/python: address Danny's feedback (2)
mohamedawnallah Jun 27, 2025
6b849b5
examples/notebooks: update `milvus_enrichment_transform`
mohamedawnallah Jun 27, 2025
fa4c285
website+examples: remove non-functional docs/examples
mohamedawnallah Jun 27, 2025
b536b8c
Merge remote-tracking branch 'upstream/master' into enrichWithMilvus
mohamedawnallah Jun 27, 2025
ac54a8b
website: revert updated `enrichment.md`
mohamedawnallah Jun 27, 2025
847dc6f
sdks/python: remove duplicated `HybridSearchParameters`
mohamedawnallah Jun 27, 2025
dbfd5f5
sdks/python: fix linting for milvus search
mohamedawnallah Jun 27, 2025
ff5d35a
sdks/python: remove examples from this PR
mohamedawnallah Jun 27, 2025
998dbfb
.github/workflows: remove unnecesssary changes
mohamedawnallah Jun 27, 2025
aa2231f
CHANGES.md: undo the feature template
mohamedawnallah Jun 29, 2025
27107b6
sdks/python: remove `pip` and `setuptools` as explicit dependency
mohamedawnallah Jun 30, 2025
65fc24b
Merge remote-tracking branch 'upstream/master' into enrichWithMilvus
mohamedawnallah Jun 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,9 @@

* X feature added (Java/Python) ([#X](https://github.com/apache/beam/issues/X)).
* Add pip-based install support for JupyterLab Sidepanel extension ([#35397](https://github.com/apache/beam/issues/#35397)).
* Milvus enrichment handler added (Python) ([#35216](https://github.com/apache/beam/pull/35216)).
Beam now supports Milvus enrichment handler capabilities for vector, keyword,
and hybrid search operations.

## Breaking Changes

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -220,11 +220,11 @@ def format_query(self, chunks: List[Chunk]) -> str:
# Create embeddings subquery for this group
embedding_unions = []
for chunk in group_chunks:
if chunk.embedding is None or chunk.embedding.dense_embedding is None:
if not chunk.dense_embedding:
raise ValueError(f"Chunk {chunk.id} missing embedding")
embedding_str = (
f"SELECT '{chunk.id}' as id, "
f"{[float(x) for x in chunk.embedding.dense_embedding]} "
f"{[float(x) for x in chunk.dense_embedding]} "
f"as embedding")
embedding_unions.append(embedding_str)
group_embeddings = " UNION ALL ".join(embedding_unions)
Expand Down
Loading
Loading