anyscale-operator-1.3.1

Release 1.3.1

This release adds a bugfix to correctly handle workloads using the Kueue integration. Prior to this fix, when the operator created a pod it would wait 5 minutes for that pod to become scheduled and delete it if it was unscheduled. This led to pods using Kueue to repeatedly be deleted and placed back in the beginning of the LocalQueue. Now the operator will allow a pod to remain unscheduled as long as the workload itself has not passed its initialization timeout.

Full Changelog: anyscale-operator-1.3.0...anyscale-operator-1.3.1

This release is backward compatible and recommended for all users.

anyscale-operator-1.3.0

Release 1.3.0

This release introduces several utilities into the chart, including patch generation for GCP credential mounting and ingress class name overrides, improved customization to the operator's deployment metadata, and various bugfixes.

New Values

GCP Credential Mounting Utility

This implements a chart utility for automatically mounting a GCP Service Account JSON key into the Anyscale Operator as well as all workloads it manages. This is useful for providing GCP credentials when workload identity is not possible, such as for non-GKE environments.

Name	Description	Default
global.gcp.projectId	The project ID to set in the environment variables of workloads	""
credentialMount.gcp.enabled	Whether to enable the credential mounting utility	false
credentialMount.gcp.fromSecret.name	The name of the Kubernetes Secret to load the credential json key from	"anyscale-gcp-credentials"
credentialMount.gcp.fromSecret.operatorMountPath	The file path in the operator to mount the credential key to	"/var/secrets"
credentialMount.gcp.fromSecret.podMountPath	The file path in the workload pods to mount the credential key to	"/var/secrets"
credentialMount.gcp.createSecret.create	Whether to manage secret creation through the chart. If set to false you must manually manage a secret with name credentialMount.gcp.FromSecret.name	false
credentialMount.gcp.createSecret.keyJsonB64	The credential key json in base64 encoded format	""

Operator Service Account Name Customization

The operator's service account name can now be customized.

Name	Description	Default
operator.serviceAccount.name	The name of the operator service account to create	"anyscale-operator"

Ingress Class Name Override

If you wish to use a custom ingress class, you can now specify an override and the chart will automatically generate the required patches to attach this to all workloads.

Name	Description	Default
networking.ingress.classNameOverride	The name of the ingress class to use, if unset the operator will default to nginx	""

Operator Deployment Metadata Customization

The operator's deployment metadata can now be customized. This can be useful for various systems that require annotations and labels to categorize workloads.

Name	Description	Default
operator.deployment.annotations	Annotations to apply to the Operator's Deployment	{}
operator.deployment.labels	Labels to apply to the Operator's Deployment	{}

Other

• Add logic to pass in AWS_CONFIG_FILE environment variable to be used with AWS credential mounting
• Added an annotation on the operator pod template including global.auth.iamIdentity to ensure the operator pod is recreated if this value is altered
• Added support for RTX-PRO-6000 within workload.accelerator.nodeSelectors.gcp

Bugfixes

• Fixed a bug where operator.affinity would be ignored if operator.nodeSelector was set

Full Changelog: anyscale-operator-1.2.1...anyscale-operator-1.3.0

This release is backward-compatible and recommended for all users.

anyscale-operator-1.2.1

Release 1.2.1

New Values

Improved Cross Kubernetes Namespace Support for Workloads

This release extends cross-namespace functionality for the operator

Name	Description	Default
workloads.managedNamespaces	The namespaces the operator should manage in addition to the namespace it is deployed in	""

Example:

workloads:
  # Enable cross-namespace resource management
  # When enabled, the operator can manage resources (services, ingresses, pods, etc.) across multiple namespaces
  enableCrossNamespaceResourceManagement: true
  
  # List of namespaces where the operator will sync secrets
  # The operator will periodically copy all secrets from its own namespace to these namespaces
  managedNamespaces:
    - namespace1
    - namespace2
    - namespace3

Once this field is set, you can deploy workloads in any one of the managed namespaces by setting the target_kubernetes_namespace flag in your compute config.

flags:
  target_kubernetes_namespace: "my-custom-namespace"

Important Notes:
When enableCrossNamespaceResourceManagement is enabled, the Anyscale Operator will use ClusterRole permissions to manage resources across all namespaces, rather than Role permissions.

Full Changelog: anyscale-operator-1.2.0...anyscale-operator-1.2.1

This release is backward-compatible and recommended for all users.

anyscale-operator-1.2.0

Release 1.2.0

This PR adds support for ARM architecture for the Anyscale Operator and support for Azure Managed Identity auth. All Operator container images are now distributed as multi-arch manifests, guaranteeing that deployments are compatible with both AMD64 (Intel/AMD) and ARM64 (Graviton/Apple Silicon) node pools.

New Values

Azure Auth

Name	Description	Default
global.auth.audience	The audience to set when requesting a token for operator auth. Required for Azure, should be set to Anyscale's Azure Multi-Tenant App	""

You can now utilize Azure Managed Identity for the operator auth flow. This requires:

• global.auth.audience to be set to Anyscale's Azure Multi-Tenant App
• global.auth.iamIdentity to be set to your Azure Managed Identity Client ID
• Your Azure Tenant ID and Managed Identity Principal ID to be registered in your Cloud Resource

Other

Multi-Arch Images

The default image tag in values.yaml has been updated, this tag is multi-arch

Full Changelog: anyscale-operator-1.1.0...anyscale-operator-1.2.0

This release is backward-compatible and recommended for all users.

anyscale-operator-1.1.0

Release 1.1.0

This release adds a new INSTANCE_TYPES operator status check which validates the schema for the instance types ConfigMap and fixes a minor bug in image pulling events.

This release is backward-compatible and is recommended for all users. See New Values for information on how to disable the new INSTANCE_TYPES.

New Values

Instance Type Status Check

The new Instance Type status check is enabled by default and will report whether the instance type schema is valid. This is useful for cases that are not caught by the validating webhook, such as the first operator install, or if the webhook is misconfigured. This check can be disabled by adding INSTANCE_TYPES to .Values.operator.config.status.excludeComponentVerification

Other

The operator will now filter out Azure Workload Identity Proxy images from image pulling events, preventing confusing output in the console.

Using enableCrossNamespaceResourceManagement for cross-namespace support is a beta release. Contact Anyscale support to enable support for this feature in your Anyscale organization.

Full Changelog: anyscale-operator-1.0.1...anyscale-operator-1.1.0

This release is backward compatible and recommended for all users.

anyscale-operator-1.0.1

Release 1.0.1

This release adds a bugfix to properly scope the ingress-nginx controller to the custom anyscale-nginx ingress class. Before this fix, it was possible for the ingress controller installed through the subchart to monitor the default nginx ingress class, and for an nginx controller installed through the chart externally to monitor the anyscale-nginx ingress class.

Full Changelog: anyscale-operator-1.0.0...anyscale-operator-1.0.1

This release is backward compatible and recommended for all users.

anyscale-operator-1.0.0

Release 1.0.0

This is a major release that introduces a new hierarchical Helm values structure for improved maintainability and organization. While the operator functionality remains largely the same, the values.yaml format has been completely restructured to group related configurations together.

This release requires migration of your values.yaml file. Please see the Migration Guide for detailed instructions on upgrading from previous versions.

Breaking Changes

New Helm Values Structure

The most significant change in v1.0.0 is the reorganization of the Helm chart values into a hierarchical structure. This affects all users upgrading from previous versions.

What this means for you:

• Your existing values.yaml file will need to be updated to the new format
• Configuration keys have been moved to logical groupings (e.g., operator.*, workloads.*, networking.*, global.*)
• The new structure is more intuitive and easier to maintain

Migration is straightforward: Most keys are simple renames or moves to new sections. The Migration Guide provides:

• Complete key mapping tables
• Side-by-side examples of old vs. new format
• Step-by-step migration instructions
• Validation commands to verify your migration

AWS Region Configuration

For AWS deployments using workload identity (IRSA), region configuration has changed:

Old format:

region: "us-west-2"

New format:

global:
  aws:
    region: "us-west-2"  # Required for AWS STS authentication

Important: The global.aws.region field is now required when using AWS with workload identity authentication (when global.auth.anyscaleCliToken is not provided). This region must match the AWS region of your Kubernetes cluster and is used by the operator for AWS STS authentication during registration.

The generic global.region field is deprecated and will be removed in a future release.

New Features

Enhanced Control Over Defaults

The new values structure introduces fine-grained control over default configurations:

• workloads.instanceTypes.enableDefaults: Control whether default instance types are included
• workloads.accelerator.enableDefaults: Control whether default accelerator configurations are included
• workloads.marketType.enableDefaults: Control whether default market type tolerations are included

This allows you to:

• Start with a minimal configuration and add only what you need
• Override defaults completely with your own configurations
• Reduce chart complexity for simpler deployments

Improved Configuration Organization

All configuration keys are now organized into logical sections:

• global.*: Cloud provider, deployment ID, authentication, and provider-specific settings
• operator.*: Operator deployment settings (replicas, images, resources, node selection)
• workloads.*: Pod/workload configuration (instance types, accelerators, service accounts)
• networking.*: Ingress and gateway configuration
• patches.*: Custom Kubernetes resource patches
• credentialMount.*: Credential mounting configuration

AWS-Specific Configuration Section

A new global.aws.* section groups all AWS-specific settings:

• global.aws.region: Region for AWS STS authentication (required for IRSA)
• global.aws.s3.usePathStyle: S3 path-style access configuration

This pattern will be extended to other cloud providers in future releases.

Deprecations

Deprecated GPU Types

The following GPU types are deprecated and are no longer supported:

• V100
• P100

Deprecated Region Field

The generic global.region field is deprecated in favor of provider-specific region configuration:

• For AWS, use global.aws.region
• For other cloud providers, region will be automatically sourced from the cloud registration response

The global.region field will continue to work for backward compatibility but will be removed in v2.0.0.

Migration Guide

All users must migrate their values.yaml file when upgrading to v1.0.0.

The complete migration guide is available at: MIGRATION_README.md

Quick Migration Steps

1. Backup your current configuration:

   helm get values <release-name> > values-backup.yaml

2. Review the migration guide to understand which keys need to change

3. Update your values file using the mapping tables and examples in the guide

4. Validate your migration:

   helm repo update anyscale
   helm template -f values-new.yaml <release-name> anyscale/anyscale-operator > new-templates.yaml
   kubectl diff -f new-templates.yaml

   NOTE: some difference is expected in the configmap_patches.yaml file since it has been reorganized, but no functionality changes are expected. Aside from this your templates should remain the same.

5. Upgrade your release:

   helm upgrade <release-name> anyscale/anyscale-operator -f values-new.yaml

Common Migration Examples

Basic Deployment

# OLD FORMAT
cloudDeploymentId: "prod-123"
cloudProvider: "aws"
region: "us-west-2"

# NEW FORMAT
global:
  cloudDeploymentId: "prod-123"
  cloudProvider: "aws"
  aws:
    region: "us-west-2"

Operator Resources

# OLD FORMAT
operatorResources:
  operator:
    requests:
      memory: 1Gi
      cpu: 2

# NEW FORMAT
operator:
  container:
    resources:
      requests:
        memory: 1Gi
        cpu: 2

See the Migration Guide for complete examples covering all configuration scenarios.

Bug Fixes

Region Handling Improvements

• Deprecated the use of the region flag for non-AWS deployments. All non-AWS deployments will now use the region field configured in the cloud resource for routing logic. This helps prevent bugs from inputting in an incorrect region value. AWS deployments still need region to be specified since it is used as part of the registration flow.

Technical Details

Operator Version

This release updates the Kubernetes operator binary with support for the new region logic.

Backward Compatibility

While the values structure has changed significantly, the operator itself is entirely backwards compatible still. Performing this upgrade will not cause any issues with existing workloads your operator is managing, and you will not need to make any changes to your cloud. This change is purely for simplifying maintaining the Helm chart and making it easier to configure.

However, your values.yaml file must be updated to the new format to deploy v1.0.0.

Full Changelog

Full Changelog: anyscale-operator-0.9.2...anyscale-operator-1.0.0

Support

If you encounter issues during migration:

1. Review the Migration Guide
2. Use helm template and kubectl diff to preview changes before applying
3. Contact Anyscale support for assistance

Note: We strongly recommend testing the migration in a non-production environment first.

anyscale-operator-0.9.2

Release 0.9.2

This is a patch adds support for configuring the operator pod's annotations and labels (#39). This release is backwards-compatible.

What's Changed

• Add support for customizing the operator pod's annotations and labels @BrendanGalloway in #39

Full Changelog: anyscale-operator-0.9.1...anyscale-operator-0.9.2

anyscale-operator-0.9.1

Release 0.9.1

This release adds a bugfix to include region in the operator if it was specified, regardless if cli token is used. This is backwards compatible and recommended for all users.

Bugfixes

• add logic to include region in the operator start command if it is present when anyscale cli token is set

Full Changelog: anyscale-operator-0.9.0...anyscale-operator-0.9.1

This release is backward compatible and recommended for all users.

anyscale-operator-0.9.0

Release 0.9.0

This release adds an optional feature to create and mount hard-coded AWS credentials onto the operator pod and pods it manages. This is not a recommended practice and should only be used when workload based access is not possible. This release also contains patches to enable market type support on AKS.

This release is backward-compatible and is recommended for all users. See New Values for information on how to configure AWS credential secret support.

New Values

AWS Credential Secret

You can now mount a Kubernetes Secret containing AWS credential and config file contents into the operator pod and the pods it manages.

Name	Description	Default
aws.credentialSecret.enabled	Whether to enable AWS credential secret support. If true, the secret referenced by aws.credentialSecret.name will be mounted into the operator pod and all pods it manages.	false
aws.credentialSecret.name	The name of the secret storing the credential and config file content. See templates/credentials.yaml for the expected format	anyscale-aws-credentials
aws.credentialSecret.create	Whether to create the credential secret as part of the chart using credentials passed into the values file. Set to false if you would like to have the operator use an externally created secret and set aws.credentialSecret.name to that secret's name.	false
aws.credentialSecret.accessKeyId	The AWS Access Key ID stored in the credential secret created if aws.credentialSecret.create is set to true	""
aws.credentialSecret.secretAccessKey	The AWS Secret Access Key stored in the credential secret created if aws.credentialSecret.create is set to true	""
aws.credentialSecret.endpointUrl	The endpoint url to store in the config section in the credential secret created if aws.credentialSecret.create is set to true. If unset, defaults to region	""
operatorMountPath	The path to mount the credential secret as a volume in the operator.	/root/.aws
podMountPath	The path to mount the credential secret as a volume in the pods the operator manages.	/tmp/.aws

Other

AKS Market Place Patches

If cloudProvider is set to azure we now correctly generate patches to add nodeSelector and nodeAffinity to place spot and on-demand workloads.

Full Changelog: anyscale-operator-0.8.1...anyscale-operator-0.9.0

This release is backward compatible and recommended for all users.