diff --git a/docs/products/kafka/get-started/create-inkless-service.md b/docs/products/kafka/get-started/create-inkless-service.md index d7675f449..b763bbdfc 100644 --- a/docs/products/kafka/get-started/create-inkless-service.md +++ b/docs/products/kafka/get-started/create-inkless-service.md @@ -10,7 +10,6 @@ import ConsoleLabel from "@site/src/components/ConsoleIcons"; import RelatedPages from "@site/src/components/RelatedPages"; Create an Inkless Apache Kafka® service on Aiven Cloud or using Bring Your Own Cloud (BYOC). -Inkless runs Kafka 4.x and supports both classic and diskless topics. ## Prerequisites @@ -34,8 +33,8 @@ configure the service by selecting ingress capacity and retention. 1. In **Deployment mode**, select **Aiven cloud**. 1. Select a cloud provider and region. 1. Select the **Ingress capacity** for the service. -1. Optional: Enable **Diskless topics**, if available. They are not suitable for - smaller workloads. +1. Optional: Enable **Diskless topics**. You can enable diskless topics later in + > **Advanced configuration**. 1. Select a **Retention** period. 1. In **Service basics**, enter: - **Name:** Enter a name for the service. You cannot change the service name after @@ -47,11 +46,70 @@ configure the service by selecting ingress capacity and retention. -:::note -Inkless on Aiven Cloud uses throughput-based offerings rather than fixed plans. -The Aiven CLI does not list these offerings. -Create Inkless services on Aiven Cloud using the Aiven Console. -::: +Create an Inkless Kafka service on Aiven Cloud using the Aiven CLI. + +1. List available Inkless offerings for the project: + + ```bash + avn inkless offering list \ + --organization-id ORGANIZATION_ID \ + --project PROJECT_NAME + ``` + + The command returns available offerings with their maximum ingress and + egress throughput. + +1. Optional: Filter offerings by required ingress throughput: + + ```bash + avn inkless offering list \ + --organization-id ORGANIZATION_ID \ + --project PROJECT_NAME \ + --ingress REQUIRED_MBPS + ``` + +1. View pricing rates for the offerings: + + ```bash + avn inkless offering rates \ + --organization-id ORGANIZATION_ID \ + --project PROJECT_NAME \ + --cloud-provider CLOUD_PROVIDER + ``` + + Optional: Filter rates by offering with `--offering-name OFFERING_NAME` or by + region with `--cloud-name CLOUD_NAME`. + +1. Create the Inkless Kafka service using an offering as the plan: + + ```bash + avn service create SERVICE_NAME \ + --project PROJECT_NAME \ + --service-type kafka \ + --cloud CLOUD_REGION \ + --plan OFFERING_NAME \ + -c kafka_version=4.0 \ + -c tiered_storage.enabled=true \ + -c inkless.enabled=true + ``` + +Parameters: + +- `ORGANIZATION_ID`: Organization ID that owns the project. +- `PROJECT_NAME`: Aiven project name. +- `CLOUD_PROVIDER`: Cloud provider, such as `aws`, `google`, or `azure`. +- `CLOUD_REGION`: Cloud region for the service, such as `aws-us-east-1`. +- `OFFERING_NAME`: Inkless offering returned by `avn inkless offering list`. +- `SERVICE_NAME`: Name of the Kafka service. + +Optional: To enable diskless topics when creating the service, add this option to +the create command: + +```bash +-c kafka_diskless.enabled=true +``` + +You can also enable diskless topics later in the service configuration. @@ -59,7 +117,7 @@ Create Inkless services on Aiven Cloud using the Aiven Console. ## Create an Inkless service on Bring Your Own Cloud (BYOC) You can run Inkless Kafka clusters in your own cloud account using -Bring Your Own Cloud (BYOC). Inkless clusters support classic and diskless topics. +Bring Your Own Cloud (BYOC). Inkless on BYOC supports classic and diskless topics. Before creating services on BYOC, configure a BYOC environment. For instructions, see [Create a custom cloud (BYOC)](/docs/platform/howto/byoc/create-cloud/create-custom-cloud). @@ -67,7 +125,7 @@ For instructions, see [Create a custom cloud (BYOC)](/docs/platform/howto/byoc/c -1. In the Aiven Console, open the project and select . +1. In the [Aiven Console](https://console.aiven.io), open the project and select . 1. Click **Create service**. 1. Select **Apache Kafka®**. 1. In **Service tier**, select **Professional**. @@ -86,21 +144,29 @@ For instructions, see [Create a custom cloud (BYOC)](/docs/platform/howto/byoc/c Create an Inkless Kafka service in a BYOC environment: -```bash -avn service create SERVICE_NAME \ - --project PROJECT_NAME \ - --service-type kafka \ - --cloud CUSTOM_CLOUD_REGION \ - --plan INKLESS_PLAN \ - -c kafka_version=4.0 \ - -c inkless.enabled=true -``` +1. List available Kafka plans for your BYOC cloud and region. Plans that support Inkless + end with `-inkless`. -To enable diskless topics, add the following option: + ```bash + avn service plans --service-type kafka --cloud CUSTOM_CLOUD_REGION + ``` -```bash --c kafka_diskless.enabled=true -``` +1. Create the service using an Inkless-capable plan: + + ```bash + avn service create SERVICE_NAME \ + --project PROJECT_NAME \ + --service-type kafka \ + --cloud CUSTOM_CLOUD_REGION \ + --plan INKLESS_PLAN \ + -c kafka_version=4.0 \ + -c tiered_storage.enabled=true \ + -c kafka_diskless.enabled=true + ``` + + :::note + Set both `tiered_storage.enabled=true` and `kafka_diskless.enabled=true`. + ::: Parameters: @@ -109,26 +175,13 @@ Parameters: - `CUSTOM_CLOUD_REGION`: BYOC region, such as `custom-aws-eu-central-1`. - `INKLESS_PLAN`: Inkless-capable plan for the selected BYOC environment. -To list available Inkless-capable plans for a BYOC environment and region: - -```bash -avn service plans --service-type kafka --cloud CUSTOM_CLOUD_REGION -``` - -:::note -Diskless topics require `kafka_diskless.enabled=true`. The selected plan must support -diskless topics. -::: -## Topic defaults +After creating the service, create topics to store data streams. To create topics, +see [Create Kafka topics](/docs/products/kafka/howto/create-topic). -- **Classic topics:** - - Remote storage is enabled automatically when you create a classic topic. - - Local retention settings are enforced by the service and cannot be changed. -- **Diskless topics:** Available only if enabled when creating the service. diff --git a/docs/products/kafka/howto/create-topic.md b/docs/products/kafka/howto/create-topic.md index b7c49ef34..45ea6839b 100644 --- a/docs/products/kafka/howto/create-topic.md +++ b/docs/products/kafka/howto/create-topic.md @@ -15,7 +15,7 @@ Create topics in your Aiven for Apache Kafka® service to organize message strea A topic in Aiven for Apache Kafka is a named stream of messages. Producers write data to topics, and consumers read from them. -Topic types depend on your Kafka service type. +Available topic types depend on the Kafka service type. ### Topic types @@ -26,18 +26,20 @@ Topic types depend on your Kafka service type. storage is enabled. - In **Inkless Kafka** services, classic topics use remote storage by default. You cannot change the storage mode or local retention settings. + - Support log compaction. - **Diskless topics** - - They are available only in **Inkless Kafka** services. - - They store topic data directly in cloud object storage. - - They require diskless support to be enabled on the service before you create - diskless topics. - - You choose classic or diskless per topic when you create the topic. + - Available only in **Inkless Kafka** services. + - Store topic data directly in cloud object storage. + - Require a service plan that supports diskless. On Aiven Cloud, enable diskless + topics during service creation or later in + > **Advanced configuration**. + - Select the topic type when creating the topic. - You cannot change the topic type after creation. In Inkless Kafka services, classic and diskless topics can coexist. To choose between them, see -[Compare diskless and classic topics](/docs/products/kafka/diskless/concepts/topics-vs-classic). +[Compare diskless and classic topics](/docs/products/kafka/diskless/concepts/topics-vs-classic#compare-classic-and-diskless-topics). For diskless topic limitations, see [Limitations of diskless topics](/docs/products/kafka/diskless/concepts/limitations). @@ -114,7 +116,8 @@ between classic and diskless topics. as retention, cleanup policy, and message size limits. :::note - You cannot create compacted topics on Inkless Kafka services. + Diskless topics do not support compaction. + Create a **classic topic** if your workload requires compaction. ::: 1. Click **Create topic**. @@ -157,10 +160,12 @@ avn service topic-create \ TOPIC_NAME \ --project PROJECT_NAME \ --partitions PARTITION_COUNT \ - --replication REPLICATION_FACTOR \ + --replication 1 \ --diskless-enable ``` +For diskless topics, set `--replication 1`. + Use the `--diskless-enable` flag to create a diskless topic. Enable diskless topics on the service before creating them. On Inkless services, omitting the flag creates a classic topic with remote storage. diff --git a/docs/products/kafka/inkless-overview.md b/docs/products/kafka/inkless-overview.md index a4a4ea8ea..ecc51a0cc 100644 --- a/docs/products/kafka/inkless-overview.md +++ b/docs/products/kafka/inkless-overview.md @@ -46,10 +46,8 @@ Use Inkless when you need to: ## Existing Classic Kafka services -Existing Classic Kafka services continue to run unchanged. - -Classic Kafka remains available for existing deployments and is visible in the Aiven -Console only for projects that already include a Classic Kafka service. +Existing Classic Kafka services continue to run unchanged. Classic Kafka remains +available when you create a new Kafka service. Upgrading or migrating an existing Classic Kafka service to Inkless is not supported at this time. The cluster type is fixed at service creation. To use Inkless, create a