Skip to content

Conversation

@lampajr
Copy link

@lampajr lampajr commented Oct 19, 2023

Hello team,

In recent times, we had the opportunity to experiment with the usage of ml-metadata following the official documentation, specifically focusing on using MLMD with a gRPC server [1].
Initially, based on the examples in the doc currently, we were led to believe users are forced to handle RPC requests/responses, thus requiring a different approach compared to what is shown in the non-gRPC server scenario [2].

After some code investigation, we discovered that it is actually possible to use the MetadataStore by providing it with a MetadataStoreClientConfig as a parameter (with proper MLMD gRPC host and port).
This way, users do not need to manage gRPC requests and responses themselves, unless they have a valid reason to do so.

client_connection_config = metadata_store_pb2.MetadataStoreClientConfig()
client_connection_config.host = 'localhost'
client_connection_config.port = 8080

store = metadata_store.MetadataStore(client_connection_config)

In order to demonstrate its usage, I have created a simple notebook [3] that precisely executes all the steps described in the documentation [2] by making use of a MetadataStore configured with a MetadataStoreClientConfig.

We believe that adding this information to the documentation could be very useful and helpful for all ml-metadata users.

Hope this helps!

[1] https://www.tensorflow.org/tfx/guide/mlmd#use_mlmd_with_a_remote_grpc_server
[2] https://www.tensorflow.org/tfx/guide/mlmd#integrate_ml_metadata_into_your_ml_workflows
[3] https://github.com/lampajr/demo20231018-mlmd-showcase/blob/d9c6d31cb4c9221e62f6915c41d73cad132505e4/mlmd-grpc-store.ipynb

Co-authored-by: Matteo Mortari <matteo.mortari@gmail.com>
@lampajr
Copy link
Author

lampajr commented Nov 10, 2023

Hi @XinranTang, have you had a chance to take a look at this?

@lampajr
Copy link
Author

lampajr commented Dec 13, 2023

Any update here?

@XinranTang
Copy link
Contributor

Hi @lampajr, looks good to me here. We will be able to merge this pr after internal tests passed.

@lampajr
Copy link
Author

lampajr commented Dec 19, 2023

Thanks a lot @XinranTang , looking forward to your updates 🙏

@lampajr
Copy link
Author

lampajr commented Jan 30, 2024

Hi @lampajr, looks good to me here. We will be able to merge this pr after internal tests passed.

Any update @XinranTang ?

@lampajr
Copy link
Author

lampajr commented Apr 25, 2024

Hi @lampajr, looks good to me here. We will be able to merge this pr after internal tests passed.

Hi @XinranTang I see a new release has been done but this hasn't been included, any update so far?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants