Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/admin/troubleshooting/jcmd/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Let's start by starting a `cratedb` container.
```console
docker run --rm -it --name=cratedb \
--publish=4200:4200 --publish=5432:5432 \
--env=CRATE_HEAP_SIZE=2g crate/crate:nightly \
--env=CRATE_HEAP_SIZE=1g crate/crate:nightly \
-Cdiscovery.type=single-node
```

Expand Down
4 changes: 2 additions & 2 deletions docs/feature/search/fts/analyzer.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,10 +165,10 @@ s(900516492,3,'Bill',n,1,0).
```

```shell
docker run --rm -it --name=cratedb --publish=4200:4200 --env=CRATE_HEAP_SIZE=2g \
docker run --rm -it --name=cratedb --publish=4200:4200 --env=CRATE_HEAP_SIZE=1g \
--volume="$PWD/synonyms-solr.txt:/crate/config/synonyms-solr.txt" \
--volume="$PWD/synonyms-wordnet.txt:/crate/config/synonyms-wordnet.txt" \
crate -Cdiscovery.type=single-node
crate
```

This example uses the `synonyms-solr.txt` in Solr format.
Expand Down
10 changes: 5 additions & 5 deletions docs/install/container/docker.rst
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Breaking the command down:
- Puts the container into the ``crate`` network and maps port ``4201`` on your
host machine to port ``4200`` on the container (admin UI).
- Defines the environment variable:ref:`CRATE_HEAP_SIZE <manual:conf-env-heap-size>`,
which is used by CrateDB to allocate 2G for its heap.
which is used by CrateDB to allocate 1 GB for its heap memory.
- Runs the command ``crate`` inside the container with parameters:
* ``network.host``: The ``_site_`` value results in the binding of the
CrateDB process to a site-local IP address.
Expand Down Expand Up @@ -298,7 +298,7 @@ define your services like this:
restart_policy:
condition: on-failure
environment:
- CRATE_HEAP_SIZE=2g
- CRATE_HEAP_SIZE=1g

cratedb02:
image: crate:latest
Expand All @@ -320,7 +320,7 @@ define your services like this:
restart_policy:
condition: on-failure
environment:
- CRATE_HEAP_SIZE=2g
- CRATE_HEAP_SIZE=1g

cratedb03:
image: crate:latest
Expand All @@ -342,7 +342,7 @@ define your services like this:
restart_policy:
condition: on-failure
environment:
- CRATE_HEAP_SIZE=2g
- CRATE_HEAP_SIZE=1g

In the file above:

Expand Down Expand Up @@ -485,7 +485,7 @@ example::

$ docker run -d \
--cpus 1.5 \
--memory 2g \
--memory 1g \
--env CRATE_HEAP_SIZE=1g \
crate \
crate -Cnetwork.host=_site_
Expand Down
6 changes: 3 additions & 3 deletions docs/integrate/airflow/data-retention-hot-cold.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ services:
"-Cgateway.expected_data_nodes=3",
"-Cgateway.recover_after_data_nodes=2"]
environment:
- CRATE_HEAP_SIZE=2g
- CRATE_HEAP_SIZE=1g

cratedb02:
image: crate:latest
Expand All @@ -53,7 +53,7 @@ services:
"-Cgateway.expected_data_nodes=3",
"-Cgateway.recover_after_data_nodes=2"]
environment:
- CRATE_HEAP_SIZE=2g
- CRATE_HEAP_SIZE=1g

cratedb03:
image: crate:latest
Expand All @@ -72,7 +72,7 @@ services:
"-Cgateway.expected_data_nodes=3",
"-Cgateway.recover_after_data_nodes=2"]
environment:
- CRATE_HEAP_SIZE=2g
- CRATE_HEAP_SIZE=1g
```

Start the cluster with `docker compose up`. For details, see the linked documentation.
Expand Down
2 changes: 1 addition & 1 deletion docs/integrate/airflow/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ You will:
:::{rubric} Apache Airflow
:::
Apache Airflow programmatically creates, schedules, and monitors workflows
\[[Official documentation](https://airflow.apache.org/docs/)\]. A workflow
([Official documentation](https://airflow.apache.org/docs/)). A workflow
is a directed acyclic graph (DAG) where each node represents a task. Each
task runs independently; the DAG tracks dependencies. Run DAGs on demand
or on schedules (for example, twice a week).
Expand Down
28 changes: 28 additions & 0 deletions docs/integrate/amqp/compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Service composition file for Docker Compose or Podman Compose
# https://cratedb.com/docs/guide/integrate/amqp/

services:

cratedb:
image: docker.io/crate/crate:latest
ports:
- "4200:4200/tcp"
- "5432:5432/tcp"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:4200/"]
interval: 5s
timeout: 30s
retries: 5

rabbitmq:
image: docker.io/rabbitmq:3
ports:
- "5672:5672/tcp"

lorrystream:
image: ghcr.io/daq-tools/lorrystream:latest
profiles: ["cli"]

amqpcat:
image: docker.io/cloudamqp/amqpcat:latest
profiles: ["cli"]
88 changes: 32 additions & 56 deletions docs/integrate/amqp/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,84 +11,60 @@ pipeline element.

## Prerequisites

Docker is used for running all components. This approach works consistently
Use Docker or Podman to run all components. This approach works consistently
across Linux, macOS, and Windows.

Alternatively, you can use Podman. You can also use a different AMQP broker such as
Apache Qpid, Apache ActiveMQ, IBM MQ, or Solace. Azure Event Hubs and Azure Service
Bus speak AMQP as well, but with protocol and authentication specifics; adjust
settings accordingly.
### Files

Create a shared network.
```shell
docker network create cratedb-demo
```
First, download and save all required files to your machine.
- {download}`compose.yaml`

Start CrateDB.
```shell
docker run --name=cratedb --rm --network=cratedb-demo \
--publish=4200:4200 --publish=5432:5432 \
--env=CRATE_HEAP_SIZE=2g docker.io/crate -Cdiscovery.type=single-node
```
### Services

Start RabbitMQ.
```shell
docker run --name=rabbitmq --rm --network=cratedb-demo \
--publish=5672:5672 docker.io/rabbitmq:3
```
> Note: This broker configuration allows anonymous access for demonstration purposes only.
> Do not expose it to untrusted networks. For production, configure authentication/TLS.
Start services using Docker Compose or Podman Compose.
If you use Podman, replace `docker` with `podman` (or enable the podman‑docker
compatibility shim) and run `podman compose up`.

Prepare shortcuts for the CrateDB shell, LorryStream, and the AMQP client
programs.

::::{tab-set}
You can also use a different AMQP broker such as
Apache Qpid, Apache ActiveMQ, IBM MQ, or Solace. Azure Event Hubs and Azure Service
Bus speak AMQP as well, but with protocol and authentication specifics; adjust
settings accordingly.

:::{tab-item} Linux and macOS
To make the settings persistent, add them to your shell profile (e.g., `~/.profile` or `~/.zshrc`).
```shell
alias crash="docker run --rm -it --network=cratedb-demo ghcr.io/crate/cratedb-toolkit crash"
alias lorry="docker run --rm -i --network=cratedb-demo ghcr.io/daq-tools/lorrystream lorry"
alias amqpcat="docker run --rm -i --network=cratedb-demo docker.io/cloudamqp/amqpcat amqpcat"
```
:::
:::{tab-item} Windows PowerShell
To make the settings persistent, add them to your PowerShell profile (`$PROFILE`).
```powershell
function crash { docker run --rm -it --network=cratedb-demo ghcr.io/crate/cratedb-toolkit crash @args }
function lorry { docker run --rm -i --network=cratedb-demo ghcr.io/daq-tools/lorrystream lorry @args }
function amqpcat { docker run --rm -i --network=cratedb-demo docker.io/cloudamqp/amqpcat amqpcat @args }
docker compose up
```
:::
:::{tab-item} Windows Command
```shell
doskey crash=docker run --rm -it --network=cratedb-demo ghcr.io/crate/cratedb-toolkit crash $*
doskey lorry=docker run --rm -i --network=cratedb-demo ghcr.io/daq-tools/lorrystream lorry $*
doskey amqpcat=docker run --rm -i --network=cratedb-demo docker.io/cloudamqp/amqpcat amqpcat $*
```
:::

::::
:::{note}
The AMQP broker configuration used here allows anonymous access for
demonstration purposes only. Do not expose it to untrusted networks. For
production, configure authentication/TLS.
:::

## Usage
## Submit data

Invoke the data transfer pipeline.
```shell
lorry relay \
"amqp://guest:guest@rabbitmq:5672/%2F?exchange=default&queue=default&routing-key=testdrive&setup=exchange,queue,bind&content-type=json" \
"crate://cratedb/?table=testdrive"
docker compose run --rm --no-TTY lorrystream lorry relay "amqp://guest:guest@rabbitmq:5672/%2F?exchange=default&queue=default&routing-key=testdrive&setup=exchange,queue,bind&content-type=json" "crate://cratedb/?table=amqp_demo"
```

Publish a JSON message to AMQP.
```shell
echo '{"temperature": 42.84, "humidity": 83.1}' | \
amqpcat --producer --uri='amqp://guest:guest@rabbitmq:5672/%2F' \
--exchange=default --queue=default --routing-key=testdrive
echo '{"temperature": 42.84, "humidity": 83.1}' | docker compose run --rm --no-TTY amqpcat amqpcat --producer --uri='amqp://guest:guest@rabbitmq:5672/%2F' --exchange=default --queue=default --routing-key=testdrive
```

## Explore data

Inspect data stored in CrateDB.
```shell
crash --hosts cratedb -c "SELECT * FROM testdrive"
docker compose exec cratedb crash -c "SELECT * FROM doc.amqp_demo"
```
```psql
+-------------+----------+
| temperature | humidity |
+-------------+----------+
| 42.84 | 83.1 |
+-------------+----------+
SELECT 1 row in set (0.023 sec)
```


Expand Down
2 changes: 2 additions & 0 deletions docs/integrate/collectd/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# https://cratedb.com/docs/guide/integrate/collectd/

FROM docker.io/debian:13-slim

# Configure system environment.
Expand Down
2 changes: 2 additions & 0 deletions docs/integrate/collectd/collectd-cratedb.conf
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# collectd configuration for storing metrics into CrateDB.
# https://cratedb.com/docs/guide/integrate/collectd/

# https://collectd.org/documentation/manpages/collectd.conf.html#plugin-postgresql

LoadPlugin postgresql
Expand Down
2 changes: 2 additions & 0 deletions docs/integrate/collectd/collectd-telegraf.conf
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# collectd configuration for sending metrics to Telegraf.
# https://cratedb.com/docs/guide/integrate/collectd/

# https://collectd.org/documentation/manpages/collectd.conf.html#plugin-network

LoadPlugin network
Expand Down
42 changes: 42 additions & 0 deletions docs/integrate/collectd/compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Service composition file for Docker Compose or Podman Compose
# https://cratedb.com/docs/guide/integrate/collectd/

services:

cratedb:
image: docker.io/crate/crate:latest
ports:
- "4200:4200/tcp"
- "5432:5432/tcp"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:4200/"]
interval: 5s
timeout: 30s
retries: 5

collectd-cratedb:
build: .
volumes:
- ./collectd-cratedb.conf:/etc/collectd/collectd.conf.d/collectd-cratedb.conf:ro
depends_on:
- cratedb

collectd-telegraf:
build: .
volumes:
- ./collectd-telegraf.conf:/etc/collectd/collectd.conf.d/collectd-telegraf.conf:ro
depends_on:
- cratedb

telegraf:
image: docker.io/telegraf:latest
ports:
- "8125:8125/udp" # StatsD
volumes:
- ./telegraf.conf:/etc/telegraf/telegraf.conf:ro
depends_on:
- cratedb

postgresql:
image: docker.io/postgres:18
profiles: ["cli"]
3 changes: 3 additions & 0 deletions docs/integrate/collectd/telegraf.conf
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# Telegraf configuration with CrateDB output.
# https://cratedb.com/docs/guide/integrate/collectd/

[[inputs.socket_listener]]
service_address = "udp://:25826"

Expand Down
Loading
Loading