- Introduce
google.protobuf.Structas an additionalValuetype for storing complex, JSON-objects in MLMD.
- MLMD wheel for MacOS is now built with minimum OS version 10.9. This will improve compatibility for the older MacOS versions.
- N/A
- N/A
- Exposes limit and order_by parameters to get_artifacts, get_executions and get_contexts API in python MetadataStore client to allow users to specify the maximum results to retrieve and the field the results to be ordered by.
- Adds
attrsas py client dependency.
- N/A
- N/A
- Adds
reuse_context_if_already_existoption toput_executionpython API to better support concurrent execution publishing with the same new context. - Supports pagination and ordering options in GetExecutionsByContext and GetArtifactsByContext APIs.
- N/A
- N/A
- N/A
-
Supports MetadataStoreClientConfig options
client_timeout_secfrom the python client. The grpc APIs would return DeadlineExceededError when server does not respond withinclient_timeout_ms. -
From this release MLMD will also be hosting nightly packages on https://pypi-nightly.tensorflow.org. To install the nightly package use the following command:
pip install -i https://pypi-nightly.tensorflow.org/simple ml-metadataNote: These nightly packages are unstable and breakages are likely to happen. The fix could often take a week or more depending on the complexity involved for the wheels to be available on the PyPI cloud service. You can always use the stable version of MLMD available on PyPI by running the command
pip install ml-metadata. -
Upgrades MLMD schema version to 6.
- Add
ParentTypetable for supporting type inheritance. - Add
Type.versioncolumn for Type evolution development. - Add
Type.idx_type_nameindex for type lookup APIs. - Add
Type.descriptioncolumn for capturing static information about Type. - Add
ParentContexttable for supporting context parental relationship. - Add
Artifact.idx_artifact_urifor filtering artifacts by uri. - Add
Event.idx_event_artifact_idandidx_event_execution_idfor lineage traversal APIs. - Add indices on
create_time_since_epoch,last_update_time_since_epochforArtifact,ExecutionandContextfor sorted listing queries.
- Add
-
Allows omitting stored properties when using
put_artifact_type,put_execution_type,put_context_type, to help writing forward compatibility MLMD type registration calls.
- Optimizes GetContext*/GetArtifact*/GetExecution* and corresponding List* calls to reduce number of backend queries.
- Documentation fixes for QueryExecutor methods.
- N/A
- Deprecates
all_fields_matchandcan_delete_fieldsfrom python APIsput_artifact_type,put_execution_type,put_context_type. In previous releases these parameters can only be set with default values, otherwise Unimplemented error returns. This change should be no-op for all existing users.
- Improves building wheels from source with
setup.py. - Supports configure GRPC options
max_receive_message_lengthfrom the python client. - Adds python 3.8 support.
- Adds
grpcioas py client dependency. - Replaces the C++ MOCK_METHOD
<n>family of macros with the new MOCK_METHOD - Updates node's
last_update_time_since_epochwhen changing (custom)properties. - Disables incompatible Golang BUILD targets temporarily.
- Support GetArtifactByTypeAndName, GetExecutionByTypeAndName, GetContextByTypeAndName Go API
- N/A
- Deprecated py3.5 support
- GetArtifacts, GetExecutions and GetContexts now supports pagination and ordering results by ID, Create time and last update time fields.
- Python MetadataStore now exposes get_artifact_by_type_and_name and get_execution_by_type_and_name methods.
- Improves query performance of get_events_by_execution_ids and get_events_by_artifact_ids by combing multiple queries.
- Drops python dependency on tensorflow to make ml-metadata be friendly with non-TFX use cases.
- Python MetadataStore APIs return mlmd errors instead of tensorflow errors.
- Note: We plan to remove Python 3.5 support after this release.
- Uses metadata_store per request for grpc server to improve scalability.
- Uses Iterable[int] instead Sequence[int] for listing APIs accepting ids.
- Depends on
tensorflow>=1.15,!=2.0.*,<3
- Drops Python 2 support and stops releasing py2 wheels.
- Upgrades MLMD schema version to 5.
- Added state columns to persistent Artifact.state, Execution.last_known_state
- Added user-given unique name per type column to Artifact and Execution.
- Added create_time_since_epoch, last_update_time_since_epoch to all Nodes.
- Provides GetArtifactByTypeAndName and GetExecutionByTypeAndName API to get artifact/execution by type and name.
- Refactors transaction executions using TransactionExecutor class.
- Supports storing/retrieving Artifact.state and Execution.last_known_state.
- Returns explicit InvalidArgument for get_artifacts_by_uri from 0.15.x clients when using 0.21.0+ server.
- Updates logging level for python mlmd client.
- Refactoring MetadataAccessObject to allow for more flexibility.
- Release a script to generate Python API documentation.
- GetArtifacts/Executions/Contexts returns OK instead of NotFound to align with other listing APIs.
- Handles mysql stale connection error 2006 by client-side reconnection.
- Handles mysql innodb deadlock error (1213) and lock timeout (1205) via client-side retry.
- Avoids update node or properties without changes.
- Adding artifact states.
- Supporting connection retries to gRPC server.
- Allowing the Python API put_execution to update or insert related contexts.
- Adding a new execution state: CANCELED. This indicates an execution being canceled.
- Adding two event types: INTERNAL_INPUT and INTERNAL_OUTPUT indended to be used by mlmd powered systems (e.g., orchestrator).
- Add support to pass migration options as command line parameters to the MLMD gRPC server.
- Adding a new Python API get_context_by_type_and_name to allow querying a context by its type and context name at the same time.
- Refactoring MetadataAccessObject to allow for more flexibility.
- The Python API put_execution will need an extra input argument to pass in contexts and return updated context_ids. Users using the old API could pass in None or an empty list as context and add another variable to hold the returned context_ids to migrate.
- Passes bytes instead of string to grpc.ssl_channel_credentials.
- Align GRPC python client stub error code with swig client error code.
- Add verify_server_cert support to MySQL source SSL options.
- Add migration options to gRPC MetadataStoreServerConfig.
- Disable auto schema migration by default during connection. The user needs to explicitly enable it when connecting an older database.
- Support SSL options when using MySQL metadata source.
- Fixes MySQL errors with concurrent connection to an empty database. Now, MLMD returns Aborted when concurrent connection error happens and the caller can retry appropriately.
- Deprecates proto field MigrationOptions.disable_upgrade_migration.
- Deprecates
disable_upgrade_migrationin python MetadataStore constructor.
- Add Dockerfile for building a containerized version of the MLMD gRPC server.
- Add support for connecting to a MYSQL Metadata Source via Unix sockets.
- Add support to pass mysql connection configuration as command line parameters to the MLMD gRPC server.
- Provides the ability for metadata_store.py to communicate with the MLMD gRPC server, as an alternative to connecting directly with a database.
- Supports Sqlite for Windows and adds scripts to build wheels for python3 in Windows.
- Provides GetContextTypes to list all Context Types.
- MLMD ConnectionConfig provides an option to disable an automatic upgrade.
- Supports downgrade of the database schema version to older versions.
- Depended on
tensorflow>=1.15,<3- Starting from 1.15, package
tensorflowcomes with GPU support. Users won't need to choose betweentensorflowandtensorflow-gpu. - Caveat:
tensorflow2.0.0 is an exception and does not have GPU support. Iftensorflow-gpu2.0.0 is installed before installingml_metadata, it will be replaced withtensorflow2.0.0. Re-installtensorflow-gpu2.0.0 if needed.
- Starting from 1.15, package
- Add Context and ContextType to MLMD data model, which are used for capturing grouping concepts (e.g., Project, Pipeline, Owner, etc) of Artifacts and Executions.
- Add CACHED state to Execution state enum to model an execution that is skipped due to cached results.
- Add the ability to list all instances of ArtifactType and ExecutionType.
- Support Type update and enforce backward compatibility.
- Support atomic creation and publishing of an execution.
- Support building a manylinux2010 compliant wheel in docker so that it works in other linux OS outside of Ubuntu.
- Provide MLMD migration scheme to migrate out-of-date MLMD instance.
- Support creating and querying ContextType.
- Support creating, updating and querying Context instances.
- Support grouping artifact and executions into contexts.
- Support querying related artifacts and executions through contexts.
- Established ML Metadata as a standalone package.
- Provides a way to store information about how each artifact (e.g. file) was generated.
- Provides tools for determining provenance.