ADR-0003: TrustyAI Service Deployment using Operator by ruivieira · Pull Request #5 · trustyai-explainability/community

ruivieira · 2023-05-22T08:54:05Z

This is a proposal for the deployment of the TrustyAI service using an Operator (ADR-0003).

Some questions (open for discussion) are:

Should the custom resource allow for custom TrustyAI images (e.g. for testing purposes), using tagged releases by default?
Should the operator check for ServiceMesh availability and use it if available?

ruivieira · 2023-05-23T09:58:35Z

cc: @anishasthana @Jooho @etirelli Your feedback would be most valuable, thank you!
@cam-garrison @bartoszmajsak @InnocentK Your feedback, from a service mesh perspective, would also be most valuable, thank you!

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md

anishasthana · 2023-05-23T10:22:45Z

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md

+
+If such configuration is not provided, the operator will use the default configuration.
+
+### Route


Copying a comment from the google doc -- we should think through the servicemesh integration (if any) here too.

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md

anishasthana · 2023-05-23T10:29:04Z

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md

+
+### Route
+
+If deployed on OpenShift, the Operator will also create a `Route` object to expose the TrustyAI Service to external clients. The `Route` object will have the following configuration:


Does the service have auth enabled by default? If not, we will need to think about that too since otherwise TrustyAI data would be available to the wider internet by default

@anishasthana Good point, thanks! (no it doesn't)

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md

elmiko

nice design doc. one thing i like to recommend when designing a new operator is considering what events you will emit as well. conditions, metrics, and logs are good to surface issues but also consider adding events to your operator to help understand what is doing.

i recommend this article as a good primer about some of the differences.

elmiko · 2023-05-23T12:37:10Z

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md

+
+## Proposal
+
+We propose to use a stand-alone TrustyAI Kubernetes Operator which would create and manage the required Deployment, Service, ConfigMap, Route, and ServiceMonitor resources based on a simple Custom Resource while keeping the state consistent with the desired one [^1].


this sounds like a big win to me

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md

Jooho · 2023-05-25T13:54:51Z

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md

+
+* `replicas` is an optional field that specifies the number of replicas of the TrustyAI service that you want to run. If not provided, the default is one replica.
+* `storage` is a mandatory field that specifies the storage details. It has two nested fields:
+  * `format` - the storage format, (example: a Persistent Volume Claim (PVC)).


Is there a specific AccessMode PVC needed? (RWX,RWO)

Is it using default StorageClass?

So default StorageClass will be a prerequisite?

If there is no default storageClass, what will happen?

@Jooho Good point, I'll add this info.
RWO is what the manifests have been specifying so far, so I think we could keep with that.
Regarding the StorageClass, the initial implementation disables dynamic provisioning and binds to already existing PVs.
Along the lines of

spec: storage: pv: "mypv" size: 1Gi

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md

Jooho · 2023-05-25T14:01:40Z

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md

+```
+
+
+Note that TrustyAI isn't currently implementing HTTPS endpoints, so the `tls` field will be set to `null` for now. Once HTTPS is implemented, the `tls` field will be updated to include the TLS configuration.


We can use Edge TLS. Is there a reason to use reencrypt or passthough TLS?

israel-hdez · 2023-05-26T23:45:03Z

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md

+
+### ModelMesh Serving Integration
+
+The operator also ensures the correct configuration of the ModelMesh Serving component. Once the TrustyAI Service is deployed and reachable, the operator will patch the ModelMesh Serving configuration to include a custom payload processor and it will be configured to point to the consumer endpoint of the deployed TrustyAI Service.


I wonder if this is really OK to do.
When installing ModelMesh via operator, this patch may be rolled back, no?

ruivieira · 2023-06-06T12:56:05Z

@anishasthana @danielezonca Related to the question of custom images, a new section was added on how to provide custom service images.

Add operator ADR-0003

ec37d5f

ruivieira added ADR Architecture Decision Record ADR/under-discussion Architecture Decision Record under discussion labels May 22, 2023

ruivieira requested review from a team and danielezonca May 22, 2023 08:54

ruivieira self-assigned this May 22, 2023

ruivieira requested review from RobGeada and tteofili and removed request for a team May 22, 2023 08:54

Update proposal

ef42a27

ruivieira requested a review from taneem-ibrahim May 23, 2023 09:42

anishasthana reviewed May 23, 2023

View reviewed changes

elmiko reviewed May 23, 2023

View reviewed changes

ruivieira mentioned this pull request May 25, 2023

Implement TrustyAI operator PVC management trustyai-explainability/trustyai-service-operator#10

Merged

Jooho reviewed May 25, 2023

View reviewed changes

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md Outdated Show resolved Hide resolved

Jooho reviewed May 25, 2023

View reviewed changes

adr/ADR-0003-trustyai-service-deployment-using-operator-pattern.md Show resolved Hide resolved

Jooho reviewed May 25, 2023

View reviewed changes

ruivieira added 6 commits May 25, 2023 18:21

Remove OLM dependency

80e86ad

Add OperatorHub as a non-goal

e733daf

Remove OperatorHub paragraph

65ea473

Add storage and envtest

07f30cd

Remove hardcoded namespaces

a75bb57

Add full service hostname

557376e

israel-hdez reviewed May 26, 2023

View reviewed changes

Add custom image via ConfigMap

7e9f4c5

ruivieira mentioned this pull request Jun 8, 2023

Operator v1 trustyai-explainability/trustyai-service-operator#44

Closed

Add storage scenario for multiple CRs in the same namespace

6768c6a


		If such configuration is not provided, the operator will use the default configuration.

		### Route


		### Route

		If deployed on OpenShift, the Operator will also create a `Route` object to expose the TrustyAI Service to external clients. The `Route` object will have the following configuration:


		## Proposal

		We propose to use a stand-alone TrustyAI Kubernetes Operator which would create and manage the required Deployment, Service, ConfigMap, Route, and ServiceMonitor resources based on a simple Custom Resource while keeping the state consistent with the desired one [^1].

		```


		Note that TrustyAI isn't currently implementing HTTPS endpoints, so the `tls` field will be set to `null` for now. Once HTTPS is implemented, the `tls` field will be updated to include the TLS configuration.


		### ModelMesh Serving Integration

		The operator also ensures the correct configuration of the ModelMesh Serving component. Once the TrustyAI Service is deployed and reachable, the operator will patch the ModelMesh Serving configuration to include a custom payload processor and it will be configured to point to the consumer endpoint of the deployed TrustyAI Service.

Conversation

ruivieira commented May 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ruivieira commented May 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elmiko left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ruivieira commented Jun 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ruivieira commented May 22, 2023 •

edited

Loading

ruivieira commented May 23, 2023 •

edited

Loading