This folder contains the source code and configuration necessary to serve the model on Vertex AI. The implementation follows this container architecture.
The serving container can be used in both online and batch prediction workflows:
-
Online predictions: Deploy the container as a REST endpoint, like a Vertex AI endpoint. This allows you to access the model for real-time predictions via the REST Application Programming Interface (API).
-
Batch predictions: Use the container to run large-scale Vertex AI batch prediction jobs to process many inputs at once.
Note: PETE is an acronym used throughout the code that stands for Pathology Encoder Tech Engine.
-
serving_framework/: A library for implementing Vertex AI-compatible HTTP servers. -
vertex_schemata/: Folder containing YAML files that define the PredictSchemata for Vertex AI endpoints. -
abstract_pete_predictor.py: DefinesAbstractPetePredictor, an abstract base class that provides a blueprint for PETE predictor classes. Subclasses must implement the predict method to provide concrete prediction logic. -
Dockerfile: Defines the Docker image for serving the model. -
entrypoint.sh: A bash script that is used as the Docker entrypoint. It sets up the necessary environment variables, copies the TensorFlow SavedModel(s) locally and launches the TensorFlow server and the frontend HTTP server. -
pete_error_mapping.py: Defines mappings between errors in Python and error codes returned in API responses. -
pete_errors.py: Defines error classes. It includes the base classPeteErrorand various specific error classes that inherit from it. -
pete_flags.py: Defines flags configured by environmental variables that configure container. -
pete_icc_profile_cache.py: Enables ICC profile caching using Redis or Cloud Storage. -
pete_prediction_executor.py: Defines the prediction executor for PETE. It includes the main function that runs the prediction executor loop. -
pete_predictor_v2.py: Defines the PETE predictor v2 class. It includes the predict function that runs inference on provided patches. -
requirements.txt: Lists the required Python packages. -
server_gunicorn.py: Creates the HTTP server that launches the prediction executor.
data_processing/: A library for data retrieval and processing.