Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 91 additions & 38 deletions docs/products/kafka/get-started/create-inkless-service.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ import ConsoleLabel from "@site/src/components/ConsoleIcons";
import RelatedPages from "@site/src/components/RelatedPages";

Create an Inkless Apache Kafka® service on Aiven Cloud or using Bring Your Own Cloud (BYOC).
Inkless runs Kafka 4.x and supports both classic and diskless topics.

## Prerequisites

Expand All @@ -34,8 +33,8 @@ configure the service by selecting ingress capacity and retention.
1. In **Deployment mode**, select **Aiven cloud**.
1. Select a cloud provider and region.
1. Select the **Ingress capacity** for the service.
1. Optional: Enable **Diskless topics**, if available. They are not suitable for
smaller workloads.
1. Optional: Enable **Diskless topics**. You can enable diskless topics later in
<ConsoleLabel name="service settings" /> > **Advanced configuration**.
1. Select a **Retention** period.
1. In **Service basics**, enter:
- **Name:** Enter a name for the service. You cannot change the service name after
Expand All @@ -47,27 +46,86 @@ configure the service by selecting ingress capacity and retention.
</TabItem>
<TabItem value="cli" label="CLI">

:::note
Inkless on Aiven Cloud uses throughput-based offerings rather than fixed plans.
The Aiven CLI does not list these offerings.
Create Inkless services on Aiven Cloud using the Aiven Console.
:::
Create an Inkless Kafka service on Aiven Cloud using the Aiven CLI.

1. List available Inkless offerings for the project:

```bash
avn inkless offering list \
--organization-id ORGANIZATION_ID \
--project PROJECT_NAME
```

The command returns available offerings with their maximum ingress and
egress throughput.

1. Optional: Filter offerings by required ingress throughput:

```bash
avn inkless offering list \
--organization-id ORGANIZATION_ID \
--project PROJECT_NAME \
--ingress REQUIRED_MBPS
```

1. View pricing rates for the offerings:

```bash
avn inkless offering rates \
--organization-id ORGANIZATION_ID \
--project PROJECT_NAME \
--cloud-provider CLOUD_PROVIDER
```

Optional: Filter rates by offering with `--offering-name OFFERING_NAME` or by
region with `--cloud-name CLOUD_NAME`.

1. Create the Inkless Kafka service using an offering as the plan:

```bash
avn service create SERVICE_NAME \
--project PROJECT_NAME \
--service-type kafka \
--cloud CLOUD_REGION \
--plan OFFERING_NAME \
-c kafka_version=4.0 \
-c tiered_storage.enabled=true \
-c inkless.enabled=true
```

Parameters:

- `ORGANIZATION_ID`: Organization ID that owns the project.
- `PROJECT_NAME`: Aiven project name.
- `CLOUD_PROVIDER`: Cloud provider, such as `aws`, `google`, or `azure`.
- `CLOUD_REGION`: Cloud region for the service, such as `aws-us-east-1`.
- `OFFERING_NAME`: Inkless offering returned by `avn inkless offering list`.
- `SERVICE_NAME`: Name of the Kafka service.

Optional: To enable diskless topics when creating the service, add this option to
the create command:

```bash
-c kafka_diskless.enabled=true
```

You can also enable diskless topics later in the service configuration.

</TabItem>
</Tabs>

## Create an Inkless service on Bring Your Own Cloud (BYOC)

You can run Inkless Kafka clusters in your own cloud account using
Bring Your Own Cloud (BYOC). Inkless clusters support classic and diskless topics.
Bring Your Own Cloud (BYOC). Inkless on BYOC supports classic and diskless topics.

Before creating services on BYOC, configure a BYOC environment.
For instructions, see [Create a custom cloud (BYOC)](/docs/platform/howto/byoc/create-cloud/create-custom-cloud).

<Tabs groupId="inkless-byoc">
<TabItem value="console" label="Console" default>

1. In the Aiven Console, open the project and select <ConsoleLabel name="services" />.
1. In the [Aiven Console](https://console.aiven.io), open the project and select <ConsoleLabel name="services" />.
1. Click **Create service**.
1. Select **Apache Kafka®**.
1. In **Service tier**, select **Professional**.
Expand All @@ -86,21 +144,29 @@ For instructions, see [Create a custom cloud (BYOC)](/docs/platform/howto/byoc/c

Create an Inkless Kafka service in a BYOC environment:

```bash
avn service create SERVICE_NAME \
--project PROJECT_NAME \
--service-type kafka \
--cloud CUSTOM_CLOUD_REGION \
--plan INKLESS_PLAN \
-c kafka_version=4.0 \
-c inkless.enabled=true
```
1. List available Kafka plans for your BYOC cloud and region. Plans that support Inkless
end with `-inkless`.

To enable diskless topics, add the following option:
```bash
avn service plans --service-type kafka --cloud CUSTOM_CLOUD_REGION
```

```bash
-c kafka_diskless.enabled=true
```
1. Create the service using an Inkless-capable plan:

```bash
avn service create SERVICE_NAME \
--project PROJECT_NAME \
--service-type kafka \
--cloud CUSTOM_CLOUD_REGION \
--plan INKLESS_PLAN \
-c kafka_version=4.0 \
-c tiered_storage.enabled=true \
-c kafka_diskless.enabled=true
```

:::note
Set both `tiered_storage.enabled=true` and `kafka_diskless.enabled=true`.
:::

Parameters:

Expand All @@ -109,26 +175,13 @@ Parameters:
- `CUSTOM_CLOUD_REGION`: BYOC region, such as `custom-aws-eu-central-1`.
- `INKLESS_PLAN`: Inkless-capable plan for the selected BYOC environment.

To list available Inkless-capable plans for a BYOC environment and region:

```bash
avn service plans --service-type kafka --cloud CUSTOM_CLOUD_REGION
```

:::note
Diskless topics require `kafka_diskless.enabled=true`. The selected plan must support
diskless topics.
:::

</TabItem>
</Tabs>

## Topic defaults
After creating the service, create topics to store data streams. To create topics,
see [Create Kafka topics](/docs/products/kafka/howto/create-topic).

- **Classic topics:**
- Remote storage is enabled automatically when you create a classic topic.
- Local retention settings are enforced by the service and cannot be changed.
- **Diskless topics:** Available only if enabled when creating the service.

<RelatedPages />

Expand Down
23 changes: 14 additions & 9 deletions docs/products/kafka/howto/create-topic.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Create topics in your Aiven for Apache Kafka® service to organize message strea
A topic in Aiven for Apache Kafka is a named stream of messages.
Producers write data to topics, and consumers read from them.

Topic types depend on your Kafka service type.
Available topic types depend on the Kafka service type.

### Topic types

Expand All @@ -26,18 +26,20 @@ Topic types depend on your Kafka service type.
storage is enabled.
- In **Inkless Kafka** services, classic topics use remote storage by default. You
cannot change the storage mode or local retention settings.
- Support log compaction.

- **Diskless topics**
- They are available only in **Inkless Kafka** services.
- They store topic data directly in cloud object storage.
- They require diskless support to be enabled on the service before you create
diskless topics.
- You choose classic or diskless per topic when you create the topic.
- Available only in **Inkless Kafka** services.
- Store topic data directly in cloud object storage.
- Require a service plan that supports diskless. On Aiven Cloud, enable diskless
topics during service creation or later in
<ConsoleLabel name="service settings" /> > **Advanced configuration**.
- Select the topic type when creating the topic.
- You cannot change the topic type after creation.

In Inkless Kafka services, classic and diskless topics can coexist.
To choose between them, see
[Compare diskless and classic topics](/docs/products/kafka/diskless/concepts/topics-vs-classic).
[Compare diskless and classic topics](/docs/products/kafka/diskless/concepts/topics-vs-classic#compare-classic-and-diskless-topics).

For diskless topic limitations, see
[Limitations of diskless topics](/docs/products/kafka/diskless/concepts/limitations).
Expand Down Expand Up @@ -114,7 +116,8 @@ between classic and diskless topics.
as retention, cleanup policy, and message size limits.

:::note
You cannot create compacted topics on Inkless Kafka services.
Diskless topics do not support compaction.
Create a **classic topic** if your workload requires compaction.
:::

1. Click **Create topic**.
Expand Down Expand Up @@ -157,10 +160,12 @@ avn service topic-create \
TOPIC_NAME \
--project PROJECT_NAME \
--partitions PARTITION_COUNT \
--replication REPLICATION_FACTOR \
--replication 1 \
--diskless-enable
```

For diskless topics, set `--replication 1`.

Use the `--diskless-enable` flag to create a diskless topic. Enable diskless topics on
the service before creating them. On Inkless services, omitting the flag creates a
classic topic with remote storage.
Expand Down
6 changes: 2 additions & 4 deletions docs/products/kafka/inkless-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,8 @@ Use Inkless when you need to:

## Existing Classic Kafka services

Existing Classic Kafka services continue to run unchanged.

Classic Kafka remains available for existing deployments and is visible in the Aiven
Console only for projects that already include a Classic Kafka service.
Existing Classic Kafka services continue to run unchanged. Classic Kafka remains
available when you create a new Kafka service.

Upgrading or migrating an existing Classic Kafka service to Inkless is not supported at
this time. The cluster type is fixed at service creation. To use Inkless, create a
Expand Down