diff --git a/docs/toolkit/creating-derivatives/_index.md b/docs/toolkit/creating-derivatives/_index.md new file mode 100644 index 000000000..69ed27aa0 --- /dev/null +++ b/docs/toolkit/creating-derivatives/_index.md @@ -0,0 +1,27 @@ + +--- +title: "Creating derivatives" +sidebar_label: "Creating derivatives" +weight: 4 +date: 2023-05-11 +description: > + Documents various methods for creating Elemental derivatives +--- + +A derivative is a standard container image which can be booted by Elemental. + +We can identify a build phase where we build the derivative, and a "runtime phase" where we consume it. + +The image is described by a Dockerfile, composed of a base OS of choice (e.g. openSUSE, Ubuntu, etc. ) and the Elemental toolkit itself in order to be consumed by Elemental and allow to be upgraded from by other derivatives. + +elemental-toolkit then converts the OCI artifact into a bootable medium (ISO, packer, ova, etc) and the image itself then can be used to bootstrap other derivatives, which can in turn upgrade to any derivative built with Elemental. + +A derivative can also be later re-used again as input as base-image for downstream derivatives. + +All the documentation below imply that the container image generated will be the booting one, there are however several configuration entrypoint that you should keep in mind while building the image which are general across all the implementation: + +- Custom persistent runtime configuration has to be provided in `/system/oem` for derivatives, [see also the documentation section](../customizing/configuration_persistency). Everything under `/system/oem` will be loaded during the various stages (boot, network, initramfs). +- `/etc/cos/bootargs.cfg` contains the booting options required to boot the image with GRUB, [see grub customization](../customizing/configure_grub) + +Derivatives inherits `Elemental` defaults, which you can override during the build process, however there are some defaults which are relevant and listed below: + diff --git a/docs/toolkit/creating-derivatives/build_disk.md b/docs/toolkit/creating-derivatives/build_disk.md new file mode 100644 index 000000000..6171709a3 --- /dev/null +++ b/docs/toolkit/creating-derivatives/build_disk.md @@ -0,0 +1,61 @@ +--- +title: "Build disk images with Elemental" +sidebar_label: "Build disk images with Elemental" +--- + +Requirements: + +* `qemu-img` utility +* `elemental` binary +* elemental runtime dependencies + +The suggested approach is based on using the Elemental installer (`elemental install` command) to run the installation +from a Linux to a loop device. The loop device can be a raw image created with `qemu-img create` that can easily be +converted to other formats after the installation by using `qemu-img convert`. + +## Prepare the loop device + +Preparing the a loop device for the installation is simple and straight forward. + +```bash +# Create a raw image of 32G +> qemu-img create -f raw disk.img 32G + +# Set the disk image as a loop device +> sudo losetup -f --show disk.img + +``` + +## Run elemental installation + +Execute the elemental installation as described in [installing](../getting-started/install): + +```bash +> sudo elemental install --firmware efi --system.uri oci: +``` + +Where `` is the Elemental derivative container image we want to use for the disk creation and `` is the +loop device previously created with `losetup` (e.g. `/dev/loop0`). + + +## Convert the RAW image to desired format + +Once the installation is done just unsetting the loop device and converting the image to the desired format is missing: + +```bash +# Unset the loop device +> sudo losetup -d + +# Convert the RAW image to qcow2 +> qemu-img convert -f raw -O qcow2 disk.img disk.qcow2 +``` + +QEMU supports a wide range of formats including common ones such as `vdi`, `vmdk` or `vhdx`. + +The result can be easily tested on QEMU with: + +```bash +> qemu -m 4096 -hda disk.qcow2 -bios /usr/share/qemu/ovmf-x86_64.bin +``` + +Note the firmware image path varies depending on the host distro, the path provided in this example is based on openSUSE Leap. diff --git a/docs/toolkit/creating-derivatives/build_iso.md b/docs/toolkit/creating-derivatives/build_iso.md new file mode 100644 index 000000000..394c08943 --- /dev/null +++ b/docs/toolkit/creating-derivatives/build_iso.md @@ -0,0 +1,184 @@ +--- +title: "Build ISOs" +sidebar_label: "Build ISOs" +--- + +In order to build an ISO we rely on `elemental build-iso` command. It accepts a YAML file denoting the sources to bundle in an ISO. In addition it can also overlay custom files or use container images from a registry as packages. + +To build an ISO, just run: + +```bash +docker run --rm -ti -v $(pwd):/build ghcr.io/rancher/elemental-toolkit/elemental-cli:latest --debug build-iso -o /build $SOURCE +``` + +Where `$SOURCE` might be the container image you want to build the ISO for, you might want to check on [how to build bootable images](creating_bootable_images). Argument `$SOURCE` might be the reference to the directory, file, container image or channel we are building the ISO for, it should be provided as uri in following format `:`, where: + * `` - might be ["dir", "file", "oci", "docker"], as default is taken "docker" + * `` - is path to file or directory, channel or image name with tag version (if tag was not provided then "latest" is used) + +`elemental build-iso` command also supports reading a configuration `manifest.yaml` file. It is loaded form the directory specified by `--config-dir` elemental's flag. + +An example of a yaml file using the bootloader from the contained image: + +```yaml +iso: + bootloader-in-rootfs: true + grub-entry-name: "Installer" + +name: "Elemental-0" +date: true +``` + +## What's next? + +- Check out on how to [build an image](build_disk) from the ISO we have just created + +## Syntax + +Below you can find a full reference about the yaml file format. + +```yaml +iso: + # Sources to be installed in the rootfs + rootfs: + - .. + # Sources to be installed in the uefi image + uefi: + - .. + # Sources to be installed in the iso image + image: + - .. + label: "COS_LIVE" +``` + +Sources can be an image reference (then an explicit tag is required) or a local path. Sources are stacked in the given order, so one can easily overwrite or append data by simply adding a local path as the last source. + +### Command flags + +- **name**: Name of the ISO image. It will be used to generate the `*.iso` file name +- **output**: Path of the destination folder of created images +- **date**: If present it includes the date in the generated file name +- **overlay-rootfs**: Sets the path of a tree to overlay on top of the system root-tree +- **overlay-uefi**: Sets the path of a tree to overaly on top of the EFI image root-tree +- **overlay-iso**: Sets the path of a tree to overlay on top of the ISO filesystem root-tree +- **label**: Sets the volume label of the ISO filesystem + +## Configuration reference + +### `iso.rootfs` + +A list of sources in uri format (container image or local path) [ "docker", "oci", "dir", "file" ] to install in the rootfs. The rootfs will be squashed to a `rootfs.squashfs` file + +### `iso.uefi` + +A list of sources in uri format (container image or local path) [ "docker", "oci", "dir", "file" ] to install in the efi FAT image or partition. + +### `iso.image` + +A list of sources in uri format (container image or local path) [ "docker", "oci", "dir", "file" ] to install in ISO filesystem. + +### `iso.label` + +The label of the ISO filesystem. Defaults to `COS_LIVE`. Note this value is tied with the bootloader and kernel parameters to identify the root device. + +### `name` + +A string representing the ISO final image name without including the `.iso` + +### `date` + +Boolean indicating if the output image name has to contain the date + +### `output` + +Folder destination of the built artifacts. It attempts to create if it doesn't exist. + +## Customize bootloader with GRUB + +Boot menu and other bootloader parameters can then be easily customized by using the overlay parameters within the ISO config yaml manifest. + +Assuming the ISO being built includes: + +```yaml +iso: + rootfs: + - ... + uefi: + - oci:example-grub2-efi-image:latest + image: + - oci:example-grub2:latest + - oci:example-grub2-efi-image:latest +``` + +We can customize either the `image` packages (in the referrence image `live/grub2` package +includes bootloader configuration) or make use of the overlay concept to include or +overwrite addition files for `image` section. + +Consider the following example: + +```yaml +iso: + rootfs: + - ... + uefi: + - oci:example-grub2-efi-image:latest + image: + - oci:example-grub2:latest + - oci:example-grub2-efi-image:latest + - dir:/my/path/to/overlay/iso +``` + +With the above the ISO will also include the files under `/my/path/to/overlay/iso` path. To customize the boot +menu parameters consider copy and modify relevant files from `example-grub2:latest` image. In this example the +`overlay` folder files list could be: + +```bash +# image files for grub2 boot +boot/grub2/grub.cfg +``` + +Being `boot/grub2/grub.cfg` a custom grub2 configuration including custom boot menu entries. Consider the following `grub.cfg` example: + +``` +search --file --set=root /boot/kernel.xz +set default=0 +set timeout=10 +set timeout_style=menu +set linux=linux +set initrd=initrd +if [ "${grub_cpu}" = "x86_64" -o "${grub_cpu}" = "i386" ];then + if [ "${grub_platform}" = "efi" ]; then + set linux=linuxefi + set initrd=initrdefi + fi +fi + +set font=($root)/boot/x86_64/loader/grub2/fonts/unicode.pf2 +if [ -f ${font} ];then + loadfont ${font} +fi + +menuentry "Custom grub2 menu entry" --class os --unrestricted { + echo Loading kernel... + $linux ($root)/boot/kernel.xz cdroot root=live:CDLABEL=COS_LIVE rd.live.dir=/ rd.live.squashimg=rootfs.squashfs console=tty1 console=ttyS0 rd.cos.disable + echo Loading initrd... + $initrd ($root)/boot/rootfs.xz +} +``` + +## Separate recovery + +To make an ISO with a separate recovery image as squashfs, you can either use the default from `Elemental`, by adding it in the iso yaml file: + +```yaml +iso: + rootfs: + .. + uefi: + .. + image: + ... + - oci:example-recovery:latest +``` + +The installer will detect the squashfs file in the iso, and will use it when installing the system. You can customize the recovery image as well by providing your own. + diff --git a/docs/toolkit/creating-derivatives/creating_bootable_images.md b/docs/toolkit/creating-derivatives/creating_bootable_images.md new file mode 100644 index 000000000..198cf6ddf --- /dev/null +++ b/docs/toolkit/creating-derivatives/creating_bootable_images.md @@ -0,0 +1,56 @@ +--- +title: "Creating bootable images" +sidebar_label: "Creating bootable images" +--- + +![](https://docs.google.com/drawings/d/e/2PACX-1vSmIZ5FTInGjtkGonUOgwhti6DZnSoeexGmWL9CAmbdiIGtBGnzDuGNj80Lj_206hP0MOxQGpEdYFvK/pub?w=1223&h=691) + +A derivative is a simple container image which can be processed by the Elemental toolkit in order to be bootable and installable. This section describes the requirements to create a container image that can be run by `Elemental`. + +## Requirements +{{}} + +Bootable images are standard container images, that means the usual `build` and `push` workflow applies, and building images is also a way to persist [oem customizations](../customizing/oem_configuration). + +The base image can be any Linux distribution that is compatible with our flavors. + +The image needs to ship: +- parts of the elemental-toolkit (required, see below) +- kernel (required) +- initrd (required) +- grub2 (required) +- dracut (required) +- microcode (optional, not required in order to boot, but recomended) + +## Example + +An illustrative example can be: + + + + +In the example above, the elemental-toolkit parts that are **required** are pulled in by `COPY --from=TOOLKIT /install-root /`. + +## Initrd +The image should provide at least `grub`, `systemd`, `dracut`, a kernel and an initrd. Those are the common set of packages between derivatives. See also [package stack](package_stack). +By default the initrd is expected to be symlinked to `/boot/initrd` and the kernel to `/boot/vmlinuz`, otherwise you can specify a custom path while [building an iso](build_iso) and [by customizing grub](../customizing/configure_grub). + +## Building + +![](https://docs.google.com/drawings/d/e/2PACX-1vS6eRyjnjdQI7OBO0laYD6vJ2rftosmh5eAog6vk_BVj8QYGGvnZoB0K8C6Qdu7SDz7p2VTxejcZsF6/pub?w=956&h=339) + +The workflow would be then: + +1) `docker build` the image +2) `docker push` the image to some registry +3) `elemental upgrade --docker-image $IMAGE` from a Elemental machine or (`elemental reset` if bootstrapping a cloud image) + +The following can be incorporated in any standard gitops workflow. + +You can explore more examples in the [example section](../examples/creating_bootable_images) on how to create bootable images. + +## What's next? + +Now that we have created our derivative container, we can either: + +- [Build an iso](build_iso) diff --git a/docs/toolkit/creating-derivatives/package_stack.md b/docs/toolkit/creating-derivatives/package_stack.md new file mode 100644 index 000000000..8a1159387 --- /dev/null +++ b/docs/toolkit/creating-derivatives/package_stack.md @@ -0,0 +1,11 @@ +--- +title: "Package stack" +sidebar_label: "Package stack" +--- + + +When building a `elemental-toolkit` derivative, a common set of packages are required with a common default configuration. Some of the most notably are: + +- systemd as init system +- grub for boot loader +- dracut for initramfs diff --git a/docs/toolkit/customizing/_index.md b/docs/toolkit/customizing/_index.md new file mode 100644 index 000000000..9c496bcae --- /dev/null +++ b/docs/toolkit/customizing/_index.md @@ -0,0 +1,9 @@ + +--- +title: "Customizing" +sidebar_label: "Customizing" +weight: 3 +date: 2017-01-05 +description: > + This section contains various articles relative on how to customize Elemental, branding and behavior. +--- diff --git a/docs/toolkit/customizing/configuration_persistency.md b/docs/toolkit/customizing/configuration_persistency.md new file mode 100644 index 000000000..14775821c --- /dev/null +++ b/docs/toolkit/customizing/configuration_persistency.md @@ -0,0 +1,45 @@ +--- +title: "Configuration persistency" +sidebar_label: "Configuration persistency" +--- + + +By default Elemental and derivatives are reading and executing cloud-init files in (lexicopgrahic) sequence inside: + +- `/system/oem` +- `/usr/local/cloud-config` +- `/oem` + +It is also possible to run cloud-init file in a different location (URLs included, too) from boot cmdline by using the `cos.setup=..` option. + +{{% alert title="Note" %}} +It is possible to install a custom [cloud-init style file](../reference/cloud_init/) during install with `--cloud-init` flag on `elemental install` command or, it's possible to add one or more files manually inside the `/oem` directory after installation. +{{% /alert %}} + +While `/system/oem` is reserved for system configurations to be included directly in the derivative container image, the `/oem` folder instead is reserved for persistent cloud-init files that can be extended in runtime. + +For example, if you want to change `/etc/issue` of the system persistently, you can create `/usr/local/cloud-config/90_after_install.yaml` or alternatively in `/oem/90_after_install.yaml` with the following content: + +```yaml +# The following is executed before fs is setted up: +stages: + fs: + - name: "After install" + files: + - path: /etc/issue + content: | + Welcome, have fun! + permissions: 0644 + owner: 0 + group: 0 + - name: "After install (second step)" + files: + - path: /etc/motd + content: | + Welcome, have more fun! + permissions: 0644 + owner: 0 + group: 0 +``` + +For more examples you can find `/system/oem` inside Elemental vanilla images containing files used to configure on boot a pristine `Elemental`. diff --git a/docs/toolkit/customizing/configure_grub.md b/docs/toolkit/customizing/configure_grub.md new file mode 100644 index 000000000..db2b33153 --- /dev/null +++ b/docs/toolkit/customizing/configure_grub.md @@ -0,0 +1,191 @@ +--- +title: "GRUB" +sidebar_label: "GRUB" +--- + +Elemental is set to deploy a persistent `grub.cfg` into the `COS_STATE` partition during +the system installation or image creation. Elemental grub configuration +includes three menu entries: first for the main OS system, second for the +fallback OS system and a third for the recovery OS. + +For example the main OS system menu entry could be something like: + +``` +menuentry "Elemental" --id elemental { + search.fs_label COS_STATE root + set img=/cOS/active.img + set label=COS_ACTIVE + loopback loop0 /$img + set root=($root) + source (loop0)/etc/cos/bootargs.cfg + linux (loop0)$kernel $kernelcmd + initrd (loop0)$initramfs +} +``` + +{{% alert title="Kernel parameters" %}} +The kernel parameters are not part of the persistent `grub.cfg` file stored in +`COS_STATE` partition. Kernel parameters are sourced from the loop device of +the OS image to boot. This is mainly to keep kernel parameters consistent +across different potential OS images or system upgrades. +{{% /alert %}} + +## Specifying default custom boot options + +Elemental images and its derivatives, are expected to include a +`/etc/cos/bootargs.cfg` file which provides the definition of the following +variables: + +* `$kernel`: Path of the kernel binary +* `$kernelcmd`: Kernel parameters +* `$initramfs`: Path of the initrd binary + +This is the mechanism any Elemental image or Elemental derivative has to communicate +its boot parameters (kernel, kernel params and initrd file) to GRUB2. + +For example, the default Elemental bootarg.cfg file is: + +``` +set kernel=/boot/vmlinuz +if [ -n "$recoverylabel" ]; then + # Boot arguments when the image is used as recovery + set kernelcmd="console=tty1 root=live:CDLABEL=$recoverylabel rd.live.dir=/ rd.live.squashimg=$img panic=5" +else + # Boot arguments when the image is used as active/passive + set kernelcmd="console=tty1 root=LABEL=$label iso-scan/filename=$img panic=5 security=selinux rd.cos.oemlabel=COS_OEM selinux=1" +fi + +set initramfs=/boot/initrd +``` + +You can tweak that file to suit your needs if you need to specify persistent boot arguments. + +{{% alert title="Note" %}} +`rd.cos.oemlabel=COS_OEM` is required inside the bootargs in order to access to +the `/oem` mount within the rootfs stage. `COS_OEM` is the default label for +the oem partition. +{{% /alert %}} + +## Grub environment variables + +Elemental (since v0.5.8) makes use of the GRUB2 environment block which can used to define +persistent GRUB2 variables across reboots. + +Use `grub2-editenv` command line utility to define the desired values. + +| Variable | Description | +|------------------------|---------------------------------------------------------| +| next_entry | Set the next reboot entry | +| saved_entry | Set the default boot entry | +| default_menu_entry | Set the name entries on the GRUB menu | +| extra_active_cmdline | Set additional boot commands when booting into active | +| extra_passive_cmdline | Set additional boot commands when booting into passive | +| extra_recovery_cmdline | Set additional boot commands when booting into recovery | +| extra_cmdline | Set additional boot commands for all entries | +| default_fallback | Sets default fallback logic | + + +For instance use the following command to reboot to recovery system only once: + +```bash +> grub2-editenv /oem/grubenv set next_entry=recovery +``` + +{{% alert title="Note" %}} +The examples below make use of the `COS_STATE` device, only files in the state +and oem partitions will be used when booting. +{{% /alert %}} + +### Default boot entry + +The default grub configuration loads the `/grubenv` of the COS_OEM partition +and evaluates on `next_entry` variable and `saved_entry` variable. By default +none is set. + +The default boot entry is set to the value of `saved_entry`, in case the variable +is not set grub just defaults to the first menu entry. + +`next_entry` variable can be used to overwrite the default boot entry for a single +boot. If `next_entry` variable is set this is only being used once, GRUB2 will +unset it after reading it for the first time. This is helpful to define the menu entry +to reboot to without having to make any permanent config change. + +Use `grub2-editenv` command line utility to define desired values. + +For instance use the following command to reboot to recovery system only once: + +```bash +> grub2-editenv /oem/grubenv set next_entry=recovery +``` + +Or to set the default entry to `fallback` system: + +```bash +> grub2-editenv /oem/grubenv set saved_entry=fallback +``` + +## Boot menu + +By default `Elemental` and derivatives shows the default boot menu entry while booting (`Elemental`). + +The grub menu entry is generated during installation and can be configured by setting `GRUB_ENTRY_NAME` in the `/etc/os-release` file inside the derivative, or either via the [general configuration](../customizing/general_configuration) to specify installation details. + +For example, specifying in `/etc/elemental/config.yaml`: + +```bash +install: + + ... + + grub-entry-name: myOS + + ... +``` + +will automatically set the GRUB menu entries for active, passive and recovery to the specified value. + +The grub menu boot entry can also be set with `grub2-editenv`: + +```bash +> grub2-editenv /oem/grubenv set default_menu_entry=fooOS +``` + + + +Since `{{}}` >= 0.0.3-16 it is possible to add multiple custom menu entries to GRUB by creating a `/grubcustom` config file in the state partition during boot. +The `grubcustom` file will be sourced at the end of the boot process, and can contain several `menuentry` blocks. + +{{% /alert %}} + +## Persistent boot option flags + +It is possible to define persistent boot flag for each menu entry also via `grub2-editenv`: + +- `extra_active_cmdline`: extra bootflags to be applied only on active boot +- `extra_passive_cmdline`: extra bootflags to be applied only on passive boot +- `extra_recovery_cmdline`: extra bootflags to be applied only on recovery +- `extra_cmdline`: will be applied to each boot entry + +## Renaming partition labels + +During boot the GRUB2 configuration is set to load the `grub_oem_env` file from the state partition. In this file the following variables are set in order to find system partitions: + +- `state_label`: Label of state partition. +- `active_label`: Filesystem label of active image. +- `passive_label`: Filesystem label of passive image. +- `recovery_label`: Label of recovery partition. +- `system_label`: Filesystem label of recovery image. +- `oem_label`: Label of OEM partition. +- `persistent_label`: Label of persistent partition. + +## Customizing fallback logic + +By default Elemental boots into active, and if there are failures will boot into the passive, and finally if keeps failing, will boot into recovery. + +It is possible to override the default fallback logic by setting `default_fallback` as grub environment, consider for example: + +```bash +> grub2-editenv /oem/grubenv set default_fallback="2 0 1" +``` + +Will set the default fallback to "2 0 1" instead of the default "0 1 2". diff --git a/docs/toolkit/customizing/embedded_features.md b/docs/toolkit/customizing/embedded_features.md new file mode 100644 index 000000000..13cfb0bd2 --- /dev/null +++ b/docs/toolkit/customizing/embedded_features.md @@ -0,0 +1,28 @@ +--- +title: "Embedded configuration" +sidebar_label: "Embedded configuration" +--- + +Elemental-toolkit provides some default configuration files for the following components: + +- GRUB2 +- Dracut +- Cloud init files +- Boot assessment + +These configuration files can be installed into a Derivative using the `elemental init`-command + +The `init`-command should be used inside the Dockerfile as in the following example: + + + +The current features available for the `init`-command is: + +- immutable-rootfs: dracut configuration for mounting the immutable root filesystem. +- grub-config: grub configuration for booting the derivative. +- elemental-setup: services used for booting the system and running cloud-init files at boot/install/upgrade. +- dracut-config: default dracut configuration for generating an initrd. +- cloud-config-defaults: optional default settings for a derivative. +- cloud-config-essentials: essential cloud-init files. + + diff --git a/docs/toolkit/customizing/general_configuration.md b/docs/toolkit/customizing/general_configuration.md new file mode 100644 index 000000000..95bc5232e --- /dev/null +++ b/docs/toolkit/customizing/general_configuration.md @@ -0,0 +1,33 @@ +--- +title: "General Configuration" +sidebar_label: "General Configuration" +--- + + +Elemental during installation, reset and upgrade (`elemental install`, `elemental reset` and `elemental upgrade` respectively) will read a configuration file in order to apply derivative customizations. The configuration files are sourced in precedence order and can be located in the following places: + +- `/etc/os-release` +- `/config.yaml` +- `/config.d/*.yaml` + +By default `` is set to `/etc/elemental` however this can be changed to any custom path by using the `--config-dir` runtime flag. + +Below you can find an example of the config file including most of the available options: + + + + +The `system` and `recovery-system` objects are an image specification. An image specification is defined by: + +- **fs**: defines the filesystem of the image. Currently only `ext2` and `squashfs` should be used for images and `squashfs` is only supported for the `recovery-system` image. +- **label**: defines the filesystem label. It is strongly recommended to use default labels as it is easy to fall into inconsistent states when changing labels as all changes should also be reflected in several other parts such as the bootloader configuration. This attribute has no effect for `squashfs` filesystems. +- **uri**: defines the source of the image. The uri must include a valid scheme to identify the type of source. It supports `oci`, `dir` and `file` schemes. +- **size**: defines the filesystem image size in MiB, it must be big enough to store the defined image source. This attribute has no effect for `squashfs` filesystems. + + +The `partitions` object lists partition specifications. A partition specifications is defined by: + +- **fs**: defines the filesystem of the partition. Currently only `ext2`, `ext4` and `xfs` are supported being `ext4` the default. +- **label**: defines the label of the filesystem of the partition. It is strongly recommended to use default labels as it is easy to fall into inconsistent states when changing labels as all changes should also be reflected in several other parts such as the bootloader configuration. +- **size**: defines the partition size in MiB. A zero size means use all available disk, obviously this only makes sense for the last partition, the `persistent` partition. +- **flags**: is a list of strings, this is used as additional partition flags that are passed to `parted` (e.g. `boot` flag). Defaults should be just fine for most of the cases. diff --git a/docs/toolkit/customizing/login.md b/docs/toolkit/customizing/login.md new file mode 100644 index 000000000..7b0767be1 --- /dev/null +++ b/docs/toolkit/customizing/login.md @@ -0,0 +1,11 @@ +--- +title: "Login" +sidebar_label: "Login" +--- + +By default you can login with the user `root` and `cos` in a vanilla Elemental image, this is also set automatically by the `cloud-config-defaults` feature if used by a derivative. + +You can change this by overriding `/system/oem/04_accounting.yaml` in the container image if present, or via [cloud-init](../reference/cloud_init/#stagesstage_idstep_nameusers). + +## Examples +- [Example accounting file](https://github.com/rancher/elemental/blob/main/framework/files/system/oem/04_accounting.yaml) diff --git a/docs/toolkit/customizing/oem_configuration.md b/docs/toolkit/customizing/oem_configuration.md new file mode 100644 index 000000000..90501c43c --- /dev/null +++ b/docs/toolkit/customizing/oem_configuration.md @@ -0,0 +1,31 @@ +--- +title: "OEM configuration" +sidebar_label: "OEM configuration" +--- + +There are several way to customize Elemental and a elemental-toolkit derivative: + +- declaratively in runtime with cloud-config file (by overriding, or extending) +- stateful, embedding any configuration in the container image to be booted. + +For runtime persistence configuration, the only supported way is with cloud-config files, [see the relevant docs](configuration_persistency). + +A derivative automatically loads and executes cloud-config files during the various system [stages](stages) also inside `/system/oem` which is read-only and reserved to the system. + +Derivatives that wish to override default configurations can do that by placing extra cloud-init file, or overriding completely `/system/oem` in the target image. + +This is to setup for example, the default root password or the preferred upgrade channel. + +The following are the `Elemental` default oem files, which are shipped within the `cloud-config-defaults` and `cloud-config-essentials` features: + +``` +/system/oem/00_rootfs.yaml - defines the rootfs mountpoint layout setting +/system/oem/01_defaults.yaml - systemd defaults (keyboard layout, timezone) +/system/oem/02_upgrades.yaml - Settings for Elemental vanilla channel upgrades +/system/oem/03_branding.yaml - Branding setting, Derivative name, /etc/issue content +/system/oem/04_accounting.yaml - Default user/pass +/system/oem/05_network.yaml - Default network setup +/system/oem/06_recovery.yaml - Executes additional commands when booting in recovery mode +``` + +You can either override the above files, or alternatively not consume the above features while building a derivative. diff --git a/docs/toolkit/customizing/runtime_persistent_changes.md b/docs/toolkit/customizing/runtime_persistent_changes.md new file mode 100644 index 000000000..ffeba2c28 --- /dev/null +++ b/docs/toolkit/customizing/runtime_persistent_changes.md @@ -0,0 +1,65 @@ +--- +title: "Runtime persistent changes" +sidebar_label: "Runtime persistent changes" +--- + +Elemental and derivatives are [immutable](../reference/immutable_rootfs) systems. That means that any change in the running OS will not persist after a reboot. + +While [configurations can be persisted](configuration_persistency), there are occasions where installing a custom package or provide additional persistent files in the end system is needed. + +We will see here a way to install packages, drivers, or apply any modification we might want to do in the OS image during runtime, without any need to rebuild the derivative container image. This will let any user (and not derivative developer) to apply any needed customization and to be able to persist across upgrades. + +## Transient changes + +To apply transient changes, it's possible to boot a Elemental derivative in read/write mode by specifying `rd.cos.debugrw` [see here for more details](../reference/immutable_rootfs). This allows to do any change and will persist into the active/passive booting system (does NOT apply for recovery). Altough this methodology should be only considered for debugging purposes. + +## Persist changes with Cloud init files + +Elemental allows to apply a set of commands, or cloud-init steps, during upgrade, deploy, install and reset in the context of the target image, in RW capabilities. This allows to carry on changes during upgrades on the target image without the need to re-build or have a custom derivative image. + +All the configuration that we want to apply to the system will run each time we do an upgrade, a reset or an installation on top of the new downloaded image (in case of upgrade) or the image which is the target system. + +Between the available [stages](stages) in the [cloud-init](../reference/cloud_init/) there are `after-upgrade-chroot`, `after-install-chroot`, `after-reset-chroot` and `after-deploy-chroot`, for example, consider the following cloud-init file: + +```yaml +stages: +name: "Install something" +stages: + after-upgrade-chroot: + - commands: + - zypper in -y ... + after-reset-chroot: + - commands: + - zypper in -y ... + after-deploy-chroot: + - commands: + - zypper in -y ... + after-install-chroot: + - commands: + - zypper in -y ... +``` + +It will run the `zypper in -y ...` calls during each stage, in the context of the target system, allowing to customize the target image with additional packages. + +{{% alert title="Note" %}} +`zypper` calls here are just an example. We could have used `dnf` for fedora based, or `apt-get` for ubuntu based images. +{{% /alert %}} + +When running the cloud-init steps the `/oem` partition and `/usr/local` will be mounted to `COS_OEM` and `COS_PERSISTENT` respectively, allowing to load extra data (e.g. rpm files, or configuration). + +## Example + +If an user wants to install an additional package in the running system, and keep having that persistent across upgrades, he can copy the following file (`install.yaml`) inside the `/oem` folder, or `/usr/local/cloud-config`: + +```yaml +stages: +name: "Install something" +stages: + after-upgrade-chroot: + - commands: + - zypper in -y vim +``` + +and run `elemental upgrade`. + +It will automatically upgrade the system with the changes above included. diff --git a/docs/toolkit/customizing/selinux_support.md b/docs/toolkit/customizing/selinux_support.md new file mode 100644 index 000000000..87b2c48bb --- /dev/null +++ b/docs/toolkit/customizing/selinux_support.md @@ -0,0 +1,65 @@ +--- +title: "SELinux Support" +sidebar_label: "SELinux Support" +--- + +Elemental includes basic support for SELinux. From an elemental perspective SELinux is some custom configurationt that requires special treatment. Being specific it mostly nails down to apply SELinux labels at install and upgrade time. Since the rootfs is readonly they can't be easily applied at runtime or at boot time. As consequence of that SELinux autorelabel service should not be used within elemental as it expects a RW root and persistency across reboots (it essentially reboots after appliying labels). For the time being Elemental only considers the `targeted` SELinux policy. + +`elemental-cli` utility applies SELinux contexts to the installed/upgraded system if three conditions are met: + +* the installed system includes the `setfiles` command +* the installed system includes the targeted files context (`/etc/selinux/targeted/contexts/files/file_contexts` file) +* the binary for `targeted` policy is also present (`/etc/selinux/targeted/policy/policy.*` file) + +In an Elemental workflow SElinux context labels should be applied at install/upgrade time for the readonly areas, but this is not enough as it doesn't cover the ephemeral filesystems (overlayfs on top of tmpfs), which are usually sensitive paths like `/etc/`, `/var`, `/srv`, etc. In order to properly apply file contexts over the ephemeral paths the relabelling has to happen at boot time once those overlayfs are created. The appropriate stage for that is in initrd before switching root. In fact, it can be done as a cloud-init step as part of the `initramfs` stage, using the packaged `10_selinux.yaml` with: + + + +Note it is required to load the policy in advance to be capable to apply the `restorecon` command. The `restorecon` command should be applied to all ephemeral paths and, depending on the specific use case, to the persistent paths too. Note that without restoring context on the ephemeral `/etc` it is unlikely the system is capable of properly booting, hence this is a very important step if SELinux is intended to used. + +## Using custom SELinux modules + +Making use of `selinux` and including SELinux utilities and targeted policy within the base OS it is enough to get started with SELinux, however there is a great chance that this is too generic and requires some additional policy modules to be fully functional according to each specific use case. + +The Type Enforcement file was created by booting an Elemental OS on permissive mode using `audit2allow` and other SELinux related utilities to generate the custom module out of the reported denials. Something like: + +```bash +# Create the type enforcement file +cat /var/log/audit/audit.log | audit2allow -m elemental > elemental.te + +# Create the policy module +checkmodule -M -m -o elemental.mod elemental.te + +# Create the policy package out of the module +semodule_package -o elemental.pp -m elemental.mod +``` + +To make effective the policy package it has to be loaded or installed within the selinux policy, this can be easily done with the `semodule -i /usr/share/elemental/selinux/elemental.pp` command. So from a derivative perspective and following the example from [Creating bootable image](../creating-derivatives/creating_bootable_images/#example) section adding the following lines to the Dockerfile should be enough to enable SELinux in enforcing mode: + +```Dockerfile +# Install the custom policy package if any and the restore context stage in cloud-init config +RUN elemental init --force --features=cloud-config + +# Load the policy package +RUN semodule -i /usr/share/elemental/selinux/elemental.pp + +# Enable selinux in enforcing mode +RUN sed -i "s|^SELINUX=.*|SELINUX=enforcing|g" /etc/selinux/config +``` + +The above assumes the base image already includes the SELinux packages and utilities provided by the underlaying distro. It is suggested to set the enforcing mode via the config file rather than setting grub with the selinux kernel parameter (`enforcing=1`), this way it is easier, at any time, to temporarily add `enforcing=0` at runtime within the grub2 shell and temporarily set SELinux in permissive mode. + +Notes when using a SELinux version prior to v3.4. If `libsemanage` version is lower than v3.4, it is likely that the `semodule -i *.pp` command fails with a cross-device linking issue, this is a known [issue](https://github.com/SELinuxProject/selinux/issues/343) upstream and already fixed since v3.4. Command `selinux -i ` mutates files under `/var/lib/selinux/targeted` and used to rename some files, this can be tricky when executed inside a container as hardlinks across filesystems are not permitted and this is actually what happens if the overlayfs driver is used. This can be worked around if all the originally mutated files are already modified within the execution layer (so they are part of the upper layer of the overlayfs). So the above specific example could be rewritten as: + +```Dockerfile +# Install the custom policy package if any and the restore context stage in cloud-init config +RUN elemental init --force --features=cloud-config + +# Artificially modify selinux files to copy them in within the overlyfs and then load the policy package +RUN mv /var/lib/selinux/targeted/active /var/lib/selinux/targeted/previous &&\ + cp --link --recursive /var/lib/selinux/targeted/previous /var/lib/selinux/targeted/active &&\ + semodule -i /usr/share/elemental/selinux/elemental.pp + +# Enable selinux in enforcing mode +RUN sed -i "s|^SELINUX=.*|SELINUX=enforcing|g" /etc/selinux/config +``` diff --git a/docs/toolkit/customizing/stages.md b/docs/toolkit/customizing/stages.md new file mode 100644 index 000000000..f9286c133 --- /dev/null +++ b/docs/toolkit/customizing/stages.md @@ -0,0 +1,383 @@ +--- +title: "Stages" +sidebar_label: "Stages" +--- + +We have a custom augmented cloud-init syntax that allows to hook into various stages of the system, for example: +- Initramfs load +- Boot +- Network availability +- During upgrades, installation, deployments , and resets + +Cloud-init files in `/system/oem`, `/oem` and `/usr/local/oem` are applied in 5 different phases: `boot`, `network`, `fs`, `initramfs` and `reconcile`. All the available cloud-init keywords can be used in each stage. Additionally, it's possible also to hook before or after a stage has run, each one has a specific stage which is possible to run steps: `boot.after`, `network.before`, `fs.after` etc. + +Multiple stages can be specified in a single cloud-init file. + +{{% alert title="Note" %}} +When a Elemental derivative boots it creates sentinel files in order to allow to execute cloud-init steps programmaticaly. + +- `/run/cos/recovery_mode` is being created when booting from the recovery partition +- `/run/cos/live_mode` is created when booting from the LiveCD + +To execute a block using the sentinel files you can specify: `if: '[ -f "/run/cos/..." ]'`, see the examples below. +{{% /alert %}} + +## Stages + +Below there is a detailed list of the stages available that can be used in the cloud-init configuration files + +### `rootfs` + +This is the earliest stage, running before switching root, just right after the +root is mounted in `/sysroot` and before applying the immutable rootfs configuration. +This stage is executed over initrd root, no chroot is applied. + +Example: +```yaml +name: "Set persistent devices" +stage: + rootfs: + - name: "Layout configuration" + environment_file: /run/cos/cos-layout.env + environment: + VOLUMES: "LABEL=COS_OEM:/oem LABEL=COS_PERSISTENT:/usr/local" + OVERLAY: "tmpfs:25%" +``` + +### `initramfs` + +This is still an early stage, running before switching root. Here you can apply radical changes to the booting setup of `Elemental`. +Despite this is executed before switching root this exection runs chrooted into the target root after the immutable rootfs is set up and ready. + +Example: +```yaml +name: "Run something on initramfs" +stages: + initramfs: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + touch /etc/something_important + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + +### `boot` + +This stage is executed after initramfs has switched root, during the `systemd` bootup process. + +Example: +```yaml +name: "Run something on boot" +stages: + boot: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + +### `fs` + +This stage is executed when fs is mounted and is guaranteed to have access to `COS_STATE` and `COS_PERSISTENT`. + +Example: +```yaml +name: "Run something on boot" +stages: + fs: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + touch /usr/local/something + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + + +### `network` + +This stage is executed when network is available + +Example: +```yaml +name: "Run something on boot" +stages: + network: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Network is available, do something.. +``` + +### `reconcile` + +This stage is executed `5m` after boot and periodically each `60m`. + +Example: +```yaml +name: "Run something on boot" +stages: + reconcile: + - name: "Setting" + if: '[ ! -f "/run/sentinel" ]' + commands: + - | + touch /run/sentinel +``` + +### `post-install` + +This stage is executed after installation of the OS has ended (last step of `elemental install`). + +Example: +```yaml +name: "Run something after installation" +stages: + post-install: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + +### `after-install-chroot` + +This stage is executed after installation of the OS filesystem image has completed. +{{% alert title="Note" %}} +Steps executed at this stage are running *inside* the new OS as chroot, allowing to write persisting changes to the image, +for example by installing additional software. +{{% /alert %}} + +Example: +```yaml +name: "Run something after installation" +stages: + after-install-chroot: + - name: "Setting" + commands: + - | + ... +``` + +### `after-install` + +This stage is executed after installation of the OS filesystem image has completed and just after the chroot hook. +{{% alert title="Note" %}} +Steps executed at this stage are running when the new image and all the relevant partitions are still mounted in rw mode, allowing to write persisting changes to the image, +for example installing additional software. +{{% /alert %}} + +Example: +```yaml +name: "Run something after installation" +stages: + after-install: + - name: "Setting" + commands: + - | + ... +``` + + +### `post-upgrade` + +This stage is executed after upgrade of the OS has ended (last step of `elemental upgrade`). + +Example: +```yaml +name: "Run something after upgrade" +stages: + post-upgrade: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + +### `after-upgrade-chroot` + +This stage is executed after installation of the OS filesystem image has completed. +{{% alert title="Note" %}} +Steps executed at this stage are running *inside* the new OS as chroot, allowing to write persisting changes to the image, +for example by downloading and installing additional software. +{{% /alert %}} + +{{% alert title="Note" %}} +Steps executed at this stage are based on stages found within the chroot, hence any new (not present in the current host) upgrade specific +hook that requires to be executed during upgrade should be included here. Otherwise it will not be seen by elemental-cli during the upgrade. +{{% /alert %}} + +Example: +```yaml +name: "Run something after upgrade" +stages: + after-upgrade-chroot: + - name: "Setting" + commands: + - | + ... +``` + +### `after-upgrade` + +This stage is executed after installation of the OS filesystem image has completed and just after the chroot hook. + + +Example: +```yaml +name: "Run something after upgrade" +stages: + after-upgrade: + - name: "Setting" + commands: + - | + ... +``` + +### `post-reset` + +This stage is executed after reset of the OS has ended (last step of `elemental reset`). + +Example: +```yaml +name: "Run something after reset" +stages: + post-reset: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + +### `after-reset-chroot` + +This stage is executed after installation of the OS filesystem image has completed. +{{% alert title="Note" %}} +Steps executed at this stage are running *inside* the new OS as chroot, allowing to write persisting changes to the image, +for example by installing additional software. +{{% /alert %}} + +Example: +```yaml +name: "Run something after installation" +stages: + after-reset-chroot: + - name: "Setting" + commands: + - | + ... +``` + +### `after-reset` + +This stage is executed after installation of the OS filesystem image has completed and just after the chroot hook. + +Example: +```yaml +name: "Run something after installation" +stages: + after-reset: + - name: "Setting" + commands: + - | + ... +``` + +### `before-install` + +This stage is executed before installation (executed during `elemental install`). + +Example: +```yaml +name: "Run something before installation" +stages: + before-install: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + + +### `before-upgrade` + +This stage is executed before upgrade of the OS (executed during `elemental upgrade`). + +Example: +```yaml +name: "Run something before upgrade" +stages: + before-upgrade: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + +### `before-reset` + +This stage is executed before reset of the OS (executed during `elemental reset`). + +Example: +```yaml +name: "Run something before reset" +stages: + before-reset: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` diff --git a/docs/toolkit/customizing/upgrades.md b/docs/toolkit/customizing/upgrades.md new file mode 100644 index 000000000..1ef5a6c9d --- /dev/null +++ b/docs/toolkit/customizing/upgrades.md @@ -0,0 +1,33 @@ +--- +title: "Upgrades" +sidebar_label: "Upgrades" +--- + +`Elemental` vanilla images by default are picking upgrades by the standard upgrade channel. It means it will always get the latest published `Elemental` version by our CI. + +However, it's possible to tweak the default behavior of `elemental upgrade` to point to a specific OCI image/tag, or a different release channel. + +## Configuration + +`elemental upgrade` during start reads the [Elemental configuration file](general_configuration) and allows to tweak the following: + +```yaml +# configuration used for the 'ugrade' command +upgrade: + # if set to true upgrade command will upgrade recovery system instead + # of main active system + recovery: false + + # image used to upgrade main OS + # size in MiB + system: + uri: + + # image used to upgrade recovery OS + # recovery images can be set to use squashfs + recovery-system: + fs: squashfs + uri: oci:recovery/cos +``` + +The `system` and `recovery-system` objects define the OS image used for the main active system and the recovery system respectively. They both are fined by a ``. diff --git a/docs/toolkit/development/_index.md b/docs/toolkit/development/_index.md new file mode 100644 index 000000000..db08a1c55 --- /dev/null +++ b/docs/toolkit/development/_index.md @@ -0,0 +1,103 @@ + +--- +title: "Development" +sidebar_label: "Development" +weight: 8 +date: 2023-05-31 +description: > + How to build Elemental? +--- + +Welcome! + +The Elemental (containerized OS) distribution is entirely built over GitHub. You can check the pipelines in the `.github` folder to see how the process looks like. + +## Forking and test on your own + +By forking the `Elemental-toolkit` repository, you already have the Github Action workflow configured to start building and pushing your own `Elemental` fork. + +## Building locally + +The elemental-cli can be built locally using go: + +From your git folder: + +```bash +$> make build-cli +$> build/elemental version +v0.2.5+g4d5d1be +``` + +### Build an example locally + +Building locally has a [set of dependencies](dependencies.md) that +should be satisfied. + +Then you can run +``` +# make build-os +``` + +### Build ISO + +If using SLES or openSUSE, first install the required deps: + +``` +# zypper in -y squashfs xorriso dosfstools +``` + +and then, simply run + +``` +# make build-iso +``` + +### Run with qemu + +After you have the iso locally, run + +``` + +$> make prepare-installer-test + +``` + +This will create a disk image and boot from the ISO. + +> +> If the image already exists, it will NOT be overwritten. +> +> You need to run an explicit `make test-clean` to wipe the image and +> start over. +> + +#### Installing + +After booting from the ISO you can log in as `root` with password `cos` using ssh `ssh root@localhost:2222` and install Elemental on +the disk image with: + +``` +# elemental install /dev/sda +``` + +### Run tests + +Requires: ginkgo, qemu + +We have a test suite which runs over SSH. + +To create the disk image: + +``` + +$> make build-disk + +``` + +To run the tests: + +``` + +$> make test-smoke + +``` diff --git a/docs/toolkit/development/creating_derivatives.md b/docs/toolkit/development/creating_derivatives.md new file mode 100644 index 000000000..d8f801e09 --- /dev/null +++ b/docs/toolkit/development/creating_derivatives.md @@ -0,0 +1,41 @@ +--- +title: "Creating derivatives" +sidebar_label: "Creating derivatives" +--- + +`elemental-toolkit` is a manifest to share a common abstract layer between derivatives inheriting the same featureset. + +## High level workflow + +The building workflow can be resumed in the following steps: + +- Build a container image (`docker build` / `nerdctl build` / `buildah` ..) +- Publish the image (`docker push` / `nerdctl push` ) +- Build an ISO (`elemental build-iso`) +- Boot a machine using the ISO and run installation (`elemental install`) +- Reboot into the installed system + +While on the client side, the upgrade workflow is: +- `elemental upgrade --system.uri=oci:` + +## Single image OS + +Derivatives are composed by a combination of specs to form a single image OS. + +The container image during installation and upgrade, is converted to an image file with a backing ext2 fs. + +## Build ISO + +To build an iso for a derivative image `elemental build-iso` command can be used: + +```bash +elemental build-iso -n $NAME $SOURCE +``` + +Where `$NAME` is the name of the ISO and `$SOURCE` might be the reference to the directory, file, container image or chaneel we are building the ISO for. `$SOURCE` should be provided as uri in following format `:`, where: + * `` - might be ["dir", "file", "oci", "docker"], as default is taken "oci" + * `` - is path to file or directory, channel or image name with tag version (if tag was not provided then "latest" is used) + +Some examples for $SOURCE argument "dir:/cOS/system", "oci:quay.io/repository/costoolkit/releases-green:cos-system-0.8.14-10" + +See also [building ISOs](../creating-derivatives/build_iso) diff --git a/docs/toolkit/development/dependencies.md b/docs/toolkit/development/dependencies.md new file mode 100644 index 000000000..34ecc6649 --- /dev/null +++ b/docs/toolkit/development/dependencies.md @@ -0,0 +1,38 @@ +--- +title: "Build requirements" +sidebar_label: "Build requirements" +--- + +### Installing required dependencies for local build + +To get requirements installed locally, run: + +```bash +$> make deps +``` + +or you need: + +- [`elemental-cli`](https://github.com/rancher/elemental-toolkit) +- [`squashfs-tools`](https://github.com/plougher/squashfs-tools) + - `zypper in squashfs` on SLES or openSUSE +- [`xorriso`](https://dev.lovelyhq.com/libburnia/web/wiki/Xorriso) + - `zypper in xorriso` on SLES or openSUSE +- [`mtools`](https://www.gnu.org/software/mtools/) + - `zypper in mtools` on SLES or openSUSE +- `yq` ([version `4.x`](https://github.com/mikefarah/yq/releases)), installed via [packages/toolchain/yq](https://github.com/rancher/elemental-toolkit/tree/main/packages/toolchain/yq) (optional) +- [`jq`](https://stedolan.github.io/jq), installed via [packages/utils/jq](https://github.com/rancher/elemental-toolkit/tree/main/packages/utils/jq) (optional) + +#### elemental + +`elemental` comes [with Elemental-toolkit](https://github.com/rancher/elemental-toolkit) + +You can grab the binary from [elemental](https://github.com/rancher/elemental-toolkit) releases. + + +#### yq and jq +`yq` (version `4.x`) and `jq` are used to retrieve the list of +packages to build in order to produce the final ISOs. Those are not +strictly required, see the Note below. + +_Note_: `yq` and `jq` are just used to generate the list of packages to build, and you don't need to have them installed if you manually specify the packages to be compiled. diff --git a/docs/toolkit/examples/_index.md b/docs/toolkit/examples/_index.md new file mode 100644 index 000000000..af7c7faf7 --- /dev/null +++ b/docs/toolkit/examples/_index.md @@ -0,0 +1,9 @@ + +--- +title: "Examples" +sidebar_label: "Examples" +weight: 5 +date: 2017-01-05 +description: > + Examples and recipes for building Elemental derivatives +--- diff --git a/docs/toolkit/examples/cloud_config.md b/docs/toolkit/examples/cloud_config.md new file mode 100644 index 000000000..12120c8fd --- /dev/null +++ b/docs/toolkit/examples/cloud_config.md @@ -0,0 +1,112 @@ +--- +title: "Cloud config examples" +sidebar_label: "Cloud config examples" +--- + +You can find here examples on how to tweak a system via cloud-config various aspects of an Elemental-toolkit derivative. + +The examples are meant to be placed as yaml files under `/oem` ore either `/usr/local/cloud-config`. They can be also given as input cloud-config while calling `elemental install`. + +## Networking with NetworkManager + +By default all interfaces will get automatically an IP address, however, there are situations where a static IP is desired, or custom configuration to be specified, here you can find some network settings with NetworkManager. + +### Additional NIC + +Set static IP to an additional NIC: + +```yaml +name: "Default network configuration" +stages: + boot: + - commands: + - nmcli dev up eth1 + - name: "Setup network" + files: + - path: /etc/sysconfig/network/ifcfg-eth1 + content: | + BOOTPROTO='static' + IPADDR='192.168.1.2/24' + permissions: 0600 + owner: 0 + group: 0 +``` + +### Static IP + +Set static IP to default interface: + +```yaml +name: "Default network configuration" +stages: + boot: + - commands: + - nmcli dev up eth0 + initramfs: + - name: "Setup network" + files: + - path: /etc/sysconfig/network/ifcfg-eth0 + content: | + BOOTPROTO='static' + IPADDR='192.168.1.2/24' + permissions: 0600 + owner: 0 + group: 0 +``` + +### DHCP + +```yaml +name: "Default network configuration" +stages: + boot: + - commands: + - nmcli dev up eth1 + initramfs: + - name: "Setup network" + files: + - path: /etc/sysconfig/network/ifcfg-eth1 + content: | + BOOTPROTO='dhcp' + STARTMODE='onboot' + permissions: 0600 + owner: 0 + group: 0 +``` + +## Additional files + +### K3s manifests + +Add k3s manifests: + +```yaml +name: "k3s" +stages: + network: + - if: '[ ! -f "/run/cos/recovery_mode" ]' + name: "Fleet deployment" + files: + - path: /var/lib/rancher/k3s/server/manifests/fleet-config.yaml + content: | + apiVersion: v1 + kind: Namespace + metadata: + name: cattle-system + --- + apiVersion: helm.cattle.io/v1 + kind: HelmChart + metadata: + name: fleet-crd + namespace: cattle-system + spec: + chart: https://github.com/rancher/fleet/releases/download/v0.3.8/fleet-crd-0.3.8.tgz + --- + apiVersion: helm.cattle.io/v1 + kind: HelmChart + metadata: + name: fleet + namespace: cattle-system + spec: + chart: https://github.com/rancher/fleet/releases/download/v0.3.8/fleet-0.3.8.tgz +``` diff --git a/docs/toolkit/examples/creating_bootable_images.md b/docs/toolkit/examples/creating_bootable_images.md new file mode 100644 index 000000000..2571e1e22 --- /dev/null +++ b/docs/toolkit/examples/creating_bootable_images.md @@ -0,0 +1,38 @@ +--- +title: "Creating bootable images" +sidebar_label: "Creating bootable images" +--- + +You can find the examples below in the [examples](https://github.com/rancher/elemental-toolkit/tree/main/examples) folder. + +## From standard images + +Besides using the `elemental-toolkit` toolchain, it's possible to create standard container images which are consumable by the vanilla `Elemental` images (ISO, Cloud Images, etc.) during the upgrade and deploy phase. + +An example of a Dockerfile image can be: + + + +We can just run docker to build the image with + +```bash +docker build -t $IMAGE . +``` + +The important piece is that an image needs to ship at least: + +``` +grub2 +systemd +kernel +dracut +``` + +And then extract the configuration for the system using the `elemental init`-command. + +## Customizations + +All the method above imply that the image generated will be the booting one, there are however several configuration entrypoint that you should keep in mind while building the image: + +- Everything under `/system/oem` will be loaded during the various stage (boot, network, initramfs). You can check [here](https://github.com/rancher/elemental-toolkit/tree/e411d8b3f0044edffc6fafa39f3097b471ef46bc/packages/cloud-config/oem) for the `Elemental` defaults. See `00_rootfs.yaml` to customize the booting layout. +- `/etc/cos/bootargs.cfg` contains the booting options required to boot the image with GRUB diff --git a/docs/toolkit/examples/embedded_images.md b/docs/toolkit/examples/embedded_images.md new file mode 100644 index 000000000..3976495b2 --- /dev/null +++ b/docs/toolkit/examples/embedded_images.md @@ -0,0 +1,332 @@ +--- +title: "Creating embedded images" +sidebar_label: "Creating embedded images" +--- + +This guide will guide in a step-by-step process to build a derivative which is fully compatible with Elemental, and will illustrate how to make customization on such image, by adding for example a default set of services and a custom user. + +The derivative will be based on openSUSE and embed k3s, and a custom user `joe` which will be already set to allow us to login. + +## Prerequisites + +- Docker installed + +## 1) Create a Dockerfile + +Let's create a workspace directory and move into it: + +```bash +$> mkdir derivative +$> cd derivative +``` + +Let's create now a `Dockerfile` for our image inside that directory, which will be represent running system: + +```Dockerfile +FROM ghcr.io/rancher/elemental-toolkit/elemental-cli:v0.11.0 AS elemental + +FROM registry.suse.com/suse/sle-micro-rancher/5.2 +ARG K3S_VERSION=v1.20.4+k3s1 +ARG ARCH=amd64 +ENV ARCH=${ARCH} + +COPY --from=elemental /install-root / +COPY --from=elemental /usr/bin/elemental /usr/bin/elemental + +# Install k3s server/agent +ENV INSTALL_K3S_VERSION=${K3S_VERSION} +RUN curl -sfL https://get.k3s.io > installer.sh && \ + INSTALL_K3S_SKIP_START="true" INSTALL_K3S_SKIP_ENABLE="true" sh installer.sh && \ + INSTALL_K3S_SKIP_START="true" INSTALL_K3S_SKIP_ENABLE="true" sh installer.sh agent && \ + rm -rf installer.sh + +## System layout + +# Required by k3s etc. +RUN mkdir /usr/libexec && touch /usr/libexec/.keep + +# Copy custom files +# COPY files/ / + +# Copy cloud-init default configuration +COPY cloud-init.yaml /system/oem/ + +# Generate initrd +RUN elemental init --force + +# OS level configuration +RUN echo "VERSION=999" > /etc/os-release +RUN echo "GRUB_ENTRY_NAME=derivative" >> /etc/os-release +RUN echo "welcome to our derivative" >> /etc/issue.d/01-derivative + +# Copy cloud-init default configuration +COPY cloud-init.yaml /system/oem/ +``` + +## 2) Configuration + +At the end of the Dockerfile, you can see that we copy over a custom [cloud-init](../reference/cloud_init) file: + +```Dockerfile +# Copy cloud-init default configuration +COPY cloud-init.yaml /system/oem/ +``` + +Create a `cloud-init.yaml` file as the `derivative/cloud-init.yaml` with the following content: + +```yaml +# See https://rancher.github.io/elemental-toolkit/docs/reference/cloud_init/ for a full syntax reference +name: "Default settings" +stages: + initramfs: + # Setup default hostname + - name: "Branding" + hostname: "derivative" + # Setup an admin group with sudo access + - name: "Setup groups" + ensure_entities: + - entity: | + kind: "group" + group_name: "admin" + password: "x" + gid: 900 + # Setup network - openSUSE specific + - name: "Network setup" + files: + - path: /etc/sysconfig/network/ifcfg-eth0 + content: | + BOOTPROTO='dhcp' + STARTMODE='onboot' + permissions: 0600 + owner: 0 + group: 0 + # Setup a custom user + - name: "Setup users" + users: + # Replace the default user name here and settings + joe: + # Comment passwd for no password + passwd: "joe" + shell: /bin/bash + homedir: "/home/joe" + groups: + - "admin" + #authorized_keys: + # Replace here with your ssh keys + # joe: + # - ssh-rsa .... + # Setup sudo + - name: "Setup sudo" + files: + - path: "/etc/sudoers" + owner: 0 + group: 0 + permsisions: 0600 + content: | + Defaults always_set_home + Defaults secure_path="/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/usr/local/sbin" + Defaults env_reset + Defaults env_keep = "LANG LC_ADDRESS LC_CTYPE LC_COLLATE LC_IDENTIFICATION LC_MEASUREMENT LC_MESSAGES LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE LC_ATIME LC_ALL LANGUAGE LINGUAS XDG_SESSION_COOKIE" + Defaults !insults + root ALL=(ALL) ALL + %admin ALL=(ALL) NOPASSWD: ALL + @includedir /etc/sudoers.d + commands: + - passwd -l root + # Setup persistency so k3s works properly + # See also: https://rancher.github.io/elemental-toolkit/docs/reference/immutable_rootfs/#configuration-with-an-environment-file + rootfs.after: + - name: "Immutable Layout configuration" + environment_file: /run/cos/cos-layout.env + environment: + VOLUMES: "LABEL=COS_OEM:/oem LABEL=COS_PERSISTENT:/usr/local" + OVERLAY: "tmpfs:25%" + RW_PATHS: "/var /etc /srv" + PERSISTENT_STATE_PATHS: >- + /etc/systemd + /etc/rancher + /etc/ssh + /etc/iscsi + /etc/cni + /home + /opt + /root + /usr/libexec + /var/log + /var/lib/rancher + /var/lib/kubelet + /var/lib/NetworkManager + /var/lib/longhorn + /var/lib/cni + PERSISTENT_STATE_BIND: "true" + # Finally, let's start k3s when network is available, and download the SSH key from github for the joe user + network: + - name: "Start k3s" + systemctl: + start: + - k3s + - authorized_keys: + # Replace here with your ssh keys or github handle + joe: + - github:joe +``` + +Done! We are now ready to build the container image. + +The file structure should be like the following: + +``` +$> tree ./derivative +derivative +├── cloud-init.yaml +├── Dockerfile +├── iso.yaml +└── repositories.yaml +``` + +## 3) Build it! + +Now we are ready to build our docker image: + +```bash +$~/derivative> docker build -t derivative:latest . +... + ---> Running in a9c33b42f567 +Removing intermediate container a9c33b42f567 + ---> 8e83191d29df +Step 19/19 : COPY cloud-init.yaml /system/oem/ + ---> 38cc4c8b173a +Successfully built 38cc4c8b173a +Successfully tagged derivative:latest +``` + +After the process completed, we are ready to consume our docker image. If you push the image over a container registry, you can then or use a running `Elemental` system to upgrade to it, or deploy it directly [see getting started](../getting-started/install). + +### Build an ISO image + +We can at this point also create a ISO from it. + +Create a `manifest.yaml` file with the following content inside the `derivative` folder: + +```yaml +iso: + bootloader-in-rootfs: true + grub-entry-name: "Derivative Installer" + +squash-no-compression: true +``` + +Now, we can build the ISO with: + +```bash +$~/derivative> elemental build-iso --config-dir=./ derivative:latest +.... +INFO[0114] Copying BIOS kernels +INFO[0114] Create squashfs +Parallel mksquashfs: Using 8 processors +Creating 4.0 filesystem on /tmp/elemental-geniso4082786464/tempISO/rootfs.squashfs, block size 1048576. +.... +INFO[0247] 🍹 Generate ISO derivative-0.20210909.iso +xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project. + +Drive current: -outdev 'stdio:derivative-0.20210909.iso' +Media current: stdio file, overwriteable +Media status : is blank +Media summary: 0 sessions, 0 data blocks, 0 data, 448g free +Added to ISO image: directory '/'='/tmp/elemental-geniso4082786464/tempISO' +xorriso : UPDATE : 599 files added in 1 seconds +xorriso : UPDATE : 599 files added in 1 seconds +xorriso : NOTE : Copying to System Area: 512 bytes from file '/tmp/elemental-geniso4082786464/tempISO/boot/x86_64/loader/boot_hybrid.img' +xorriso : WARNING : Boot image load size exceeds 65535 blocks of 512 bytes. Will record 0 in El Torito to extend ESP to end-of-medium. +libisofs: NOTE : Aligned image size to cylinder size by 137 blocks +xorriso : UPDATE : 12.35% done +xorriso : UPDATE : 42.73% done +ISO image produced: 282624 sectors +Written to medium : 282624 sectors at LBA 0 +Writing to 'stdio:derivative-0.20210909.iso' completed successfully. +``` + +After the process completes, we should have a ISO in our folder ready to be used. See the [build ISOs section](../creating-derivatives/build_iso) for all the available options. + + +## Customization + +Here follows a break down of the steps above + +### Adding packages +Feel free to edit the Dockerfile with the packages you want to embed in our image. You can install any packages available in the openSUSE repositories by +tweaking + +```Dockerfile +# Install packages from the base image +RUN zypper in -y \ + .... # Add more packages here! +``` + +### System layout and k3s + +We set some default layouts and install k3s: + +``` +# Install k3s server/agent +ENV INSTALL_K3S_VERSION=${K3S_VERSION} +RUN curl -sfL https://get.k3s.io > installer.sh && \ + INSTALL_K3S_SKIP_START="true" INSTALL_K3S_SKIP_ENABLE="true" sh installer.sh && \ + INSTALL_K3S_SKIP_START="true" INSTALL_K3S_SKIP_ENABLE="true" sh installer.sh agent && \ + rm -rf installer.sh + +## System layout + +# Required by k3s etc. +RUN mkdir /usr/libexec && touch /usr/libexec/.keep + +# Copy custom files +# COPY files/ / + +# Generate initrd +RUN elemental init --force + +# OS level configuration +RUN echo "VERSION=999" > /etc/os-release +RUN echo "GRUB_ENTRY_NAME=derivative" >> /etc/os-release +RUN echo "welcome to our derivative" >> /etc/issue.d/01-derivative +``` + +As our target here is to install `k3s` we do install both k3s `agent` and `server`, so the image can work in both modes. Here we could have installed any service or binary that we want to embed in our container image. We setup the system layout by creating needed paths for `k3s` and set up a os-release which identifies the OS version. Afterward we regenerate the initrd which is required in order to boot, [see also the Initrd section](../creating-derivatives/creating_bootable_images/#initrd). + +### Cloud-init, custom SSH access + +The `cloud-init.yaml` file above configures a user, `joe` and attaches a ssh key to it in order to login. It also sets up a default password, which is optional. + +To specify any additional ssh key installed within the user, we do: + +```yaml +network: +- authorized_keys: + joe: + - github:joe +``` + +which you can replace with your github handle, or by specifying directly an ssh key. In case you specify the SSH key directly, you don't need to run the step in the `network` stage. + +The user will be part of the `admin` group which is allowed to use `sudo`. + +As our target is to run `k3s`, but could have been any other service, we tweak the immutable setup by specifying sensible path required for `k3s` in order to function properly, see [immutable rootfs](../reference/immutable_rootfs) for all the available options. + +Finally, we start k3s. Note, we could have tweaked that part slightly to provide `k3s` configurations via systemd env files, or boot up for example the agent instead of the server: + +```yaml + network: + - if: '[ ! -f "/run/cos/recovery_mode" ]' + name: "Setup k3s" + # Setup environment file for custom k3s arguments + environment_file: "/etc/systemd/system/k3s.service.env" + environment: + FOO: "bar" + systemctl: + start: + - k3s + - commands: + - | + chmod 600 /etc/systemd/system/k3s.service.env +``` diff --git a/docs/toolkit/getting-started/_index.md b/docs/toolkit/getting-started/_index.md new file mode 100644 index 000000000..4b0cf63dc --- /dev/null +++ b/docs/toolkit/getting-started/_index.md @@ -0,0 +1,59 @@ +--- +title: "Getting Started" +sidebar_label: "Getting Started" +weight: 1 +description: > + Getting started with Elemental +--- + +![](https://docs.google.com/drawings/d/e/2PACX-1vRSuocC4_2rHeJAWW2vqinw_EZeZxTzJFo5ZwnJaL_sdKab_R_OsCTLT_LFh1_L5fUcA_2i9FIe-k69/pub?w=1223&h=691) + + +Elemental toolkit provides a runtime and buildtime framework in order to boot containers in VMs, Baremetals and Cloud. + +You can either choose to **build** a Elemental derivative or **run** Elemental to boostrap a new system. + +Elemental vanilla images are published to allow to deploy user-built derivatives. + +Elemental is designed to run, deploy and upgrade derivatives that can be built just as standard OCI container images. Elemental assets can be used to either drive unattended deployments of a derivative or used to create custom images (with packer). + +## Philosophy + +{{}} + +Philosophy behind elemental-toolkit is simple: it allows you to create Linux derivatives from container images. + +- **Container registry as a single source of truth** +- Hybrid way to access your image for different scopes (development, debugging, ..) +- No more inconsistent states between nodes. A “Store” to keep your (tagged) shared states where you can rollback and upgrade into. +- “Stateless”: Images with upgrades are rebuilt from scratch instead of applying upgrades. +- A/B upgrades, immutable systems + +The container image is booted as-is, encapsulating all the needed components (kernel, initrd included) and can be pulled locally for inspection, development and debugging. At the same time it can be used also to create installation medium as ISO, Raw images, OVA or Cloud specific images. + +A derivative automatically inherits the following featureset: +- [Can upgrade to another container image](./upgrading) +- [Can deploy a system from scratch from an image](./deploy) +- [Reset or recovery to a specific image](./recovery) +- [Customize the image during runtime to persist changes across reboots](../customizing/runtime_persistent_changes) +- [Perform an installation from the LiveCD medium](./booting) + +## Building Elemental derivatives + +The starting point to use elemental-toolkit is to check out our [examples](https://github.com/rancher/elemental-toolkit/tree/main/examples) and our [creating bootable images](../creating-derivatives/creating_bootable_images) section. + +The only requirement to build derivatives with `elemental-toolkit` is Docker installed. If you are interested in building elemental-toolkit itself, see [Development notes](../development). + +The toolkit itself is delivered as a set of standalone, re-usable OCI artifacts which are tagged and tracked as standard OCI images and it is installed inside the container image to provide the same featureset among derivatives, see [how to create bootable images](../creating-derivatives/creating_bootable_images). + +## Vanilla images + +`Elemental` releases are composed of vanilla images that are used internally for testing and can be used as a starting point to deploy derivatives in specific environments (e.g. AWS) or just to try out the Elemental featureset. + +The vanilla images ships no specific business-logic aside serving as a base for testing and deploying other derivatives. + +### What to do next? + +Check out [how to create bootable images](../creating-derivatives/creating_bootable_images) or [download the Elemental vanilla images](../getting-started/download) to give Elemental a try! + +Here below you will find the common documentation that applies to any derivative built with Elemental and the Elemental vanilla images. diff --git a/docs/toolkit/getting-started/deploy.md b/docs/toolkit/getting-started/deploy.md new file mode 100644 index 000000000..38aed3138 --- /dev/null +++ b/docs/toolkit/getting-started/deploy.md @@ -0,0 +1,43 @@ +--- +title: "Deploying" +sidebar_label: "Deploying" +--- + + +Elemental vanilla images, like ISOs, cloud images or raw disks can be used to deploy another derivative image. + +## `elemental reset` + +`elemental reset` can be used to reset the system from the recovery image or from a custom image. Vanilla images only include a minimal recovery partition and system. + +It can be either invoked manually with `elemental reset --system.uri ` or used in conjuction with a cloud-init configuration, for example consider the following [cloud-init configuration file](../reference/cloud_init) that creates the `state` and `persistent` partitions during first boot (this is required on Elemental vanilla images): + + +```yaml +name: "Default deployment" +stages: + rootfs.after: + - name: "Repart image" + layout: + # It will partition a device including the given filesystem label or part label (filesystem label matches first) + device: + label: COS_RECOVERY + add_partitions: + - fsLabel: COS_STATE + # 10Gb for COS_STATE, so the disk should have at least 16Gb + size: 10240 + pLabel: state + - fsLabel: COS_PERSISTENT + # unset size or 0 size means all available space + pLabel: persistent + network: + - if: '[ -f "/run/cos/recovery_mode" ]' + name: "Deploy Elemental system" + commands: + - | + # Use `elemental reset --system.uri docker:` to deploy a custom image + # By default the recovery Elemental gets deployed + elemental reset --reboot --system.uri docker:$IMAGE +``` + +The following will first repartition the image after the `rootfs` [stage](../customizing/stages) and will run `elemental reset` when booting into [recovery mode](recovery). RAW vanilla disk images automatically boot by default into recovery, so the first thing upon booting is deploying the system diff --git a/docs/toolkit/getting-started/download.md b/docs/toolkit/getting-started/download.md new file mode 100644 index 000000000..39da076c3 --- /dev/null +++ b/docs/toolkit/getting-started/download.md @@ -0,0 +1,29 @@ +--- +title: "Download" +sidebar_label: "Download" +--- + +Elemental-toolkit consists of a CLI program that is used to install a system and build bootable sources. The CLI also embeds configuration needed for a bootable derivative. + +## Download Elemental + +Elemental toolkit can be run directly using a container runtime such as docker: + +```bash +docker run -it --rm ghcr.io/rancher/elemental-toolkit/elemental-cli:latest version +``` + +## Building from source + +The CLI can also be built from source by checking out the repo and running make: + +```bash +git clone https://github.com/rancher/elemental-toolkit +cd elemental-toolkit +make build-cli +./build/elemental version +``` + +## What to do next? + +Check out [the customization section](../customizing/stages) to build a custom `Elemental` derivative or [the example section](../examples/creating_bootable_images) for some already prepared recipe examples. diff --git a/docs/toolkit/getting-started/install.md b/docs/toolkit/getting-started/install.md new file mode 100644 index 000000000..ad4150d8e --- /dev/null +++ b/docs/toolkit/getting-started/install.md @@ -0,0 +1,93 @@ +--- +title: "Installing" +sidebar_label: "Installing" +--- + + +Elemental (or any Elemental derivative built with elemental-toolkit) can be installed with `elemental install`: + +```bash +elemental install [options] +``` + +| Option | Description | +|------------------------------|--------------------------------------------------------------------------------------------------------------| +| --cloud-init string | Cloud-init config file | +| --cosign | Enable cosign verification (requires images with signatures) | +| --cosign-key string | Sets the URL of the public key to be used by cosign validation | +| --eject-cd | Try to eject the cd on reboot, only valid if booting from iso | +| --firmware string | Firmware to install for ('esp' or 'bios') (default "efi") | +| --force | Force install | +| --help | help for install | +| --iso string | Performs an installation from the ISO url | +| --no-format | Don’t format disks. It is implied that COS_STATE, COS_RECOVERY, COS_PERSISTENT, COS_OEM are already existing | +| --verify | Enable mtree checksum verification (requires images manifests generated with mtree separately) | +| --part-table string | Partition table type to use (default "gpt") | +| --poweroff | Shutdown the system after install | +| --reboot | Reboot the system after install | +| --recovery-system.uri string | Sets the recovery image source and its type (e.g. 'docker:registry.org/image:tag') | +| --system.uri string | Sets the system image source and its type (e.g. 'docker:registry.org/image:tag') | +| --strict | Enable strict check of hooks (They need to exit with 0) | +| --tty string | Add named tty to grub | + + +### Custom OEM configuration + +During installation it can be specified a [cloud-init config file](../reference/cloud_init), that will be installed and persist in the system after installation: + +```bash +elemental install --cloud-init [url|path] +``` + +### Custom partitioning + +When installing it's possible to specify a custom partition sizes via the configuration file (`/etc/elemental/config.yaml` by default). + +```yaml +install + partitions: + state: + # All sizes are in MiB + size: 8192 + recovery: + size: 4096 + oem: + size: 64 + persistent: + # zero size tells parted to use all the available + # disk, note this is only functional for the last partition + size: 0 +``` + +Refer to the [config file docs](../customizing/general_configuration) for further details about all partitioning options. + +In order to create additional partitions please consider the layout section on [cloud-init config file reference](../reference/cloud_init) + +### Installation from 3rd party LiveCD or rescue mediums + +The installer can be used to perform installations also from outside the Elemental or standard derivative ISOs. + +For instance, it is possible to install Elemental (or any derivative) with the installer from another bootable medium, or a rescue mode which is booting from RAM, given there is enough free RAM available. + +#### With Docker + +If in the rescue system, or LiveCD you have docker available, it can be used to perform an installation + +```bash +docker run --privileged -v /dev/:/dev/ -ti ghcr.io/rancher/elemental-toolkit/elemental-cli:latest install --system.uri $IMAGE $DEVICE +``` + +Where `$IMAGE` is the container image that we want to install (e.g. `oci:ghcr.io/rancher/elemental-toolkit/elemental-green:v0.10.7` ), elemental identifies the type of source by the URI scheme (`docker`, `channel`, `dir` or `file`). `$DEVICE` is the device where to perform the installation to (e.g. `/dev/sda`). + + +Note, we used the `ghcr.io/rancher/elemental-toolkit/elemental-cli:latest` image which contains the latest stable installer and the dependencies. +You can see all the versions at [GitHub Container Registry](https://ghcr.io/rancher/elemental-toolkit/elemental-cli). + + +#### By using manually the Elemental installer + +Similarly, the same mechanism can be used without docker. Install elemental using the [Download guide](download.md) and run the follow as root: + +```bash +elemental install --system.uri $IMAGE $DEVICE +``` diff --git a/docs/toolkit/getting-started/recovery.md b/docs/toolkit/getting-started/recovery.md new file mode 100644 index 000000000..6f7d52f52 --- /dev/null +++ b/docs/toolkit/getting-started/recovery.md @@ -0,0 +1,37 @@ +--- +title: "Recovery" +sidebar_label: "Recovery" +--- + +Elemental derivatives shares a common recovery mechanism built-in which can be leveraged to restore the system to a known point. At installation time, the recovery partition is created from the installation medium. + +The recovery system can be accessed during boot by selecting the last entry in the menu (labeled by "recovery"). + +A derivative can be recovered anytime by booting into the ` recovery` partition and by running `elemental reset` from it. + +This command will regenerate the bootloader and the images in the `COS_STATE` partition by using the recovery image. + +### Upgrading the recovery partition + +From either the active or passive system, the recovery partition can also be upgraded by running + +```bash +elemental upgrade --recovery +``` + +It also supports to specify docker images directly: + +```bash +elemental upgrade --recovery --recovery-system.uri +``` + +Where `` can be an opaque URI of `docker` scheme (e.g. `docker:registry.org/some/image:tag`). + +### Upgrading the active system from the recovery + +The recovery system can upgrade also the active system by running `elemental upgrade`, and it also supports to specify docker images directly: + +```bash +elemental upgrade --system.uri +``` + diff --git a/docs/toolkit/getting-started/upgrading.md b/docs/toolkit/getting-started/upgrading.md new file mode 100644 index 000000000..c3fa7f1aa --- /dev/null +++ b/docs/toolkit/getting-started/upgrading.md @@ -0,0 +1,60 @@ +--- +title: "Upgrading" +sidebar_label: "Upgrading" +--- + +Elemental and every derivative can upgrade, rollback or just switch to different versions in runtime by using the toolkit installed inside the image. + +To upgrade an installed system, just run `elemental upgrade` and reboot. + +This will perform an upgrade based on the default derivative configuration for the image. See [general configuration](../customizing/general_configuration) on how to configure defaults when building a derivative. + +"Upgrades" are not carried over the usual way of treating each single package individually: Elemental considers the container image as a new system where to boot into. It will pull a new container image during this phase, which will be booted on the next reboot. + +## Upgrade to a specific container image + +To specify a specific container image to upgrade to instead of the regular upgrade channels, run `elemental upgrade --system.uri imagei-uri`. + +_Note_ by default `elemental upgrade --system.uri` runs an mtree checksum verificatiom (requires images manifests generated with mtree separately). To disable image checksum verification, run `elemental upgrade --verify --system.uri`. + +## Integration with System Upgrade Controller + +If running a kubernetes cluster on the `Elemental` system, you can leverage the [system-upgrade-controller](https://github.com/rancher/system-upgrade-controller) to trigger upgrades to specific image versions, for example: + +```yaml +--- +apiVersion: upgrade.cattle.io/v1 +kind: Plan +metadata: + name: elemental-upgrade + namespace: system-upgrade + labels: + k3s-upgrade: server +spec: + concurrency: 1 + version: fleet-sample # Image tag + nodeSelector: + matchExpressions: + - {key: k3s.io/hostname, operator: Exists} + serviceAccountName: system-upgrade + cordon: true +# drain: +# force: true + upgrade: + image: quay.io/costoolkit/test-images # Image upgrade reference + command: + - "/usr/sbin/suc-upgrade" +``` + +See also [trigger upgrades with fleet](../tutorials/trigger_upgrades_with_fleet) + +## From ISO + +The ISO can be also used as a recovery medium: type `elemental upgrade` from a LiveCD. It will then try to upgrade the image of the active partition installed in the system. + +## How it works +Elemental during installation sets two `.img` images files in the `COS_STATE` partition: +- `/cOS/active.img` labeled `COS_ACTIVE`: Where `Elemental` typically boots from +- `/cOS/passive.img` labeled `COS_PASSIVE`: Where `Elemental` boots for fallback + +Those are used by the upgrade mechanism to prepare and install a pristine `Elemental` each time an upgrade is attempted. diff --git a/docs/toolkit/index.md b/docs/toolkit/index.md new file mode 100644 index 000000000..112c9df50 --- /dev/null +++ b/docs/toolkit/index.md @@ -0,0 +1,56 @@ +--- +title: "Overview" +sidebar_label: "Overview" +--- + +## What is Elemental toolkit? + +Elemental is a toolkit which allows container images to be bootable in VMs, baremetals, embedded devices, and much more. + +Elemental allows to create meta-Linux derivatives which are configured throughout cloud-init configuration files and are immutable by default. + +Elemental and derivatives shares a common feature set, can be upgraded with a A/B mechanism, and upgrades are delivered with standard container registries. + +Elemental comes also with vanilla images that can be used to boot directly container images built with the toolkit. + +## Why Elemental? + +Elemental allows to create custom OS versions in your cluster with standard container images with a high degree of customization. It can also be used in its vanilla form - Elemental enables then everyone to build their own derivative and access it in various formats. + +Building a bootable image is as simple as running `docker build`. + +* **What is it good for?**: Embedded, Cloud, Containers, VM, Baremetals, Servers, IoT, Edge + +## Design goals + +- A Manifest for container-based OS. It contains just the common bits to make a container image bootable and to be upgraded from, with few customization on top +- Everything is an OCI artifact from the ground-up +- Immutable-first, but with a flexible layout +- Cloud-init driven +- Based on systemd +- Built and upgraded from containers - It is a [single image OS](https://quay.io/repository/costoolkit/releases-teal)! +- A/B updates +- Easy to customize +- Cryptographically verified +- Instant switch from different versions +- Recovery mechanism with `Elemental` vanilla images (or bring your own) + +## Mission + +The elemental-toolkit project is under the Elemental umbrella. + +Elemental-toolkit provides a unique container based approach to define the system lifecycle of an immutable Linux derivative, without any string attached to a specific Linux distribution. + +At its heart, Elemental-toolkit is the abstraction layer between Linux distro management and the specific purpose of the OS. + +Elemental-toolkit empowers anyone to create derivatives from standard OCI images. Frees whoever wants to create a Linux derivative from handling the heavy bits of packaging and managing entire repositories to propagate upgrades, simplifying the entire process by using container images as base for OS. +At the same time, Elemental-toolkit provides an highly integrated ecosystem which is designed to be container-first, cloud native, and immutable. +Anyone can tweak Elemental-toolkit derivatives from the bottom-up to enable and disable its featureset. + +As the Elemental team, the [elemental](https://github.com/rancher/elemental) project is our point of reference. + +`Elemental` is a complete derivative built with elemental-toolkit tied with the rancher ecosystem and full cycle node management solution with Kubernetes. + +We are supporting directly and indirectly `elemental` within changes also in the Elemental ecosystem. + +`Elemental` is our main show-case, and as the Elemental team we are committed to it. It encompasses several technologies to create a Kubernetes-focused Linux derivative which lifecycle is managed entirely from Kubernetes itself, [Secure Device Onboarding](https://www.intel.it/content/www/it/it/internet-of-things/secure-device-onboard.html) included, and automatic provisioning via cloud-init. diff --git a/docs/toolkit/reference/_index.md b/docs/toolkit/reference/_index.md new file mode 100644 index 000000000..8f0f3313c --- /dev/null +++ b/docs/toolkit/reference/_index.md @@ -0,0 +1,14 @@ +--- +title: "Reference" +sidebar_label: "Reference" +weight: 7 +description: > + References for Elemental derivatives, like common featuresets, high level architecture +--- + +- [Github project](https://github.com/orgs/rancher/projects/9/views/1) for our sprint board + + +### Samples repositories + +- [Build a derivative and upgrade it with fleet](https://github.com/rancher-sandbox/cos-fleet-upgrades-sample) diff --git a/docs/toolkit/reference/built_with_elemental.md b/docs/toolkit/reference/built_with_elemental.md new file mode 100644 index 000000000..0373d77bc --- /dev/null +++ b/docs/toolkit/reference/built_with_elemental.md @@ -0,0 +1,24 @@ +--- +title: "Built with Elemental" +sidebar_label: "Built with Elemental" +--- + +Here is a not-exhaustive lists of derivatives built with Elemental. + + +## Run by Rancher + +| Name | Description | Link | +|-----------|-----------------------------------------------------------------------------------------------------------------|----------------------------------------| +| Harvester | Open source hyperconverged infrastructure (HCI) software | https://github.com/harvester/harvester | +| Elemental | Immutable Linux distribution built to run Rancher and its corresponding Kubernetes distributions RKE2 and k3s. | https://github.com/rancher/elemental | + + +## Run by Community + +The following derivatives are not run by Rancher, but are community efforts. +Open up an issue or create a PR to get yours added to the list! + +| Name | Description | Link | +|-----------|-----------------------------------------------------------------------------------------------------------------|----------------------------------------| +| Kairos | Immutable OS for Automated (Decentralized) Kubernetes clusters with k3s, for homelab and beyond | https://github.com/kairos-io/kairos | diff --git a/docs/toolkit/reference/cloud_init.md b/docs/toolkit/reference/cloud_init.md new file mode 100644 index 000000000..0a1280547 --- /dev/null +++ b/docs/toolkit/reference/cloud_init.md @@ -0,0 +1,440 @@ +--- +title: "Cloud-init support" +sidebar_label: "Cloud-init support" +--- + +Below is a reference of all keys available in the cloud-init style files. + +```yaml +stages: + # "network" is the stage where network is expected to be up + # It is called internally when network is available from + # the cos-setup-network unit. + network: + # Here there are a list of + # steps to be run in the network stage + - name: "Some setup happening" + files: + - path: /tmp/foo + content: | + test + permissions: 0777 + owner: 1000 + group: 100 + commands: + - echo "test" + modules: + - nvidia + environment: + FOO: "bar" + systctl: + debug.exception-trace: "0" + hostname: "foo" + systemctl: + enable: + - foo + disable: + - bar + start: + - baz + mask: + - foobar + authorized_keys: + user: + - "github:suse" + - "ssh-rsa ...." + dns: + path: /etc/resolv.conf + nameservers: + - 8.8.8.8 + ensure_entities: + - path: /etc/passwd + entity: | + kind: "user" + username: "foo" + password: "pass" + uid: 0 + gid: 0 + info: "Foo!" + homedir: "/home/foo" + shell: "/bin/bash" + delete_entities: + - path: /etc/passwd + entity: | + kind: "user" + username: "foo" + password: "pass" + uid: 0 + gid: 0 + info: "Foo!" + homedir: "/home/foo" + shell: "/bin/bash" + datasource: + providers: + - "aws" + - "digitalocean" + path: "/etc/cloud-data" +``` + +The default cloud-config format is split into *stages* (*initramfs*, *boot*, *network*, *initramfs*, *reconcile*, called generically **STAGE_ID** below) [see also stages](../customizing/stages) that are emitted internally during the various phases by calling `cos-setup STAGE_ID`. +*steps* (**STEP_NAME** below) defined for each stage are executed in order. + +Each cloud-config file is loaded and executed only at the apprioriate stage, this allows further components to emit their own stages at the desired time. + +{{% pageinfo %}} +The [cloud-init tool](https://github.com/mudler/yip#readme) can be also run standalone, this helps debugging locally and also during development, you can find separate [releases here](https://github.com/mudler/yip/releases). +{{% /pageinfo %}} + +_Note_: Each cloud-init option can be either run in *dot notation* ( e.g. `stages.network[0].authorized_keys.user=github:user` ) in the boot args or either can supply a cloud-init URL at boot with the `cos.setup=$URL` parameter. + +### Using templates + +With Cloud Init support, templates can be used to allow dynamic configuration. More information about templates can be found [here](https://github.com/mudler/yip#node-data-interpolation) and also [here for sprig](http://masterminds.github.io/sprig/) functions. + +### Compatibility with Cloud Init format + +A subset of the official [cloud-config spec](http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data) is implemented. + +If a yaml file starts with `#cloud-config` it is parsed as a standard cloud-init and automatically associated it to the `boot` stage. For example: + +```yaml +#cloud-config +growpart: + mode: auto + devices: ['/'] + +users: +- name: "bar" + passwd: "foo" + lock_passwd: true + uid: "1002" + groups: "users" + ssh_authorized_keys: + - faaapploo +ssh_authorized_keys: + - asdd +runcmd: +- foo +hostname: "bar" +write_files: +- encoding: b64 + content: CiMgVGhpcyBmaWxlIGNvbnRyb2xzIHRoZSBzdGF0ZSBvZiBTRUxpbnV4 + path: /foo/bar + permissions: "0644" + owner: "bar" +``` + +Is executed at boot, by using the standard `cloud-config` format. + +{{% alert title="Note" %}} +You can't mix extended syntax with legacy cloud-init syntax. By pre-pending the cloud-config with the `#cloud-config` header you enable the legacy notation, and the extended one ( `stages`.. ) will be ignored. +{{% /alert %}} + + +### `stages.STAGE_ID.STEP_NAME.name` + +A description of the stage step. Used only when printing output to console. + +### `stages.STAGE_ID.STEP_NAME.files` + +A list of files to write to disk. + +```yaml +stages: + default: + - files: + - path: /tmp/bar + content: | + #!/bin/sh + echo "test" + permissions: 0777 + owner: 1000 + group: 100 +``` + +### `stages.STAGE_ID.STEP_NAME.directories` + +A list of directories to be created on disk. Runs before `files`. + +```yaml +stages: + default: + - name: "Setup folders" + directories: + - path: "/etc/foo" + permissions: 0600 + owner: 0 + group: 0 +``` + +### `stages.STAGE_ID.STEP_NAME.dns` + +A way to configure the `/etc/resolv.conf` file. + +```yaml +stages: + default: + - name: "Setup dns" + dns: + nameservers: + - 8.8.8.8 + - 1.1.1.1 + search: + - foo.bar + options: + - .. + path: "/etc/resolv.conf.bak" +``` +### `stages.STAGE_ID.STEP_NAME.hostname` + +A string representing the machine hostname. It sets it in the running system, updates `/etc/hostname` and adds the new hostname to `/etc/hosts`. +Templates can be used to allow dynamic configuration. For example in mass-install scenario it could be needed (and easier) to specify hostnames for multiple machines from a single cloud-init config file. + +```yaml +stages: + default: + - name: "Setup hostname" + hostname: "node-{{ trunc 4 .MachineID }}" +``` +### `stages.STAGE_ID.STEP_NAME.sysctl` + +Kernel configuration. It sets `/proc/sys/` accordingly, similarly to `sysctl`. + +```yaml +stages: + default: + - name: "Setup exception trace" + systctl: + debug.exception-trace: "0" +``` + +### `stages.STAGE_ID.STEP_NAME.authorized_keys` + +A list of SSH authorized keys that should be added for each user. +SSH keys can be obtained from GitHub user accounts by using the format github:${USERNAME}, similarly for Gitlab with gitlab:${USERNAME}. + +```yaml +stages: + default: + - name: "Setup exception trace" + authorized_keys: + joe: + - github:joe + - ssh-rsa: ... +``` + +### `stages.STAGE_ID.STEP_NAME.node` + +If defined, the node hostname where this stage has to run, otherwise it skips the execution. The node can be also a regexp in the Golang format. + +```yaml +stages: + default: + - name: "Setup logging" + node: "bastion" +``` + +### `stages.STAGE_ID.STEP_NAME.users` + +A map of users and user info to set. Passwords can be also encrypted. + +The `users` parameter adds or modifies the specified list of users. Each user is an object which consists of the following fields. Each field is optional and of type string unless otherwise noted. +In case the user is already existing, the entry is ignored. + +- **name**: Required. Login name of user +- **gecos**: GECOS comment of user +- **passwd**: Hash of the password to use for this user. Unencrypted strings are supported too. +- **homedir**: User's home directory. Defaults to /home/*name* +- **no-create-home**: Boolean. Skip home directory creation. +- **primary-group**: Default group for the user. Defaults to a new group created named after the user. +- **groups**: Add user to these additional groups +- **no-user-group**: Boolean. Skip default group creation. +- **ssh-authorized-keys**: List of public SSH keys to authorize for this user +- **system**: Create the user as a system user. No home directory will be created. +- **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases. +- **shell**: User's login shell. + +```yaml +stages: + default: + - name: "Setup users" + users: + bastion: + passwd: "strongpassword" + homedir: "/home/foo +``` + +### `stages.STAGE_ID.STEP_NAME.ensure_entities` + +A `user` or a `group` in the [entity](https://github.com/mudler/entities) format to be configured in the system + +```yaml +stages: + default: + - name: "Setup users" + ensure_entities: + - path: /etc/passwd + entity: | + kind: "user" + username: "foo" + password: "x" + uid: 0 + gid: 0 + info: "Foo!" + homedir: "/home/foo" + shell: "/bin/bash" +``` +### `stages.STAGE_ID.STEP_NAME.delete_entities` + +A `user` or a `group` in the [entity](https://github.com/mudler/entities) format to be pruned from the system + +```yaml +stages: + default: + - name: "Setup users" + delete_entities: + - path: /etc/passwd + entity: | + kind: "user" + username: "foo" + password: "x" + uid: 0 + gid: 0 + info: "Foo!" + homedir: "/home/foo" + shell: "/bin/bash" +``` +### `stages.STAGE_ID.STEP_NAME.modules` + +A list of kernel modules to load. + +```yaml +stages: + default: + - name: "Setup users" + modules: + - nvidia +``` +### `stages.STAGE_ID.STEP_NAME.systemctl` + +A list of systemd services to `enable`, `disable`, `mask` or `start`. + +```yaml +stages: + default: + - name: "Setup users" + systemctl: + enable: + - systemd-timesyncd + - cronie + mask: + - purge-kernels + disable: + - crond + start: + - cronie +``` +### `stages.STAGE_ID.STEP_NAME.environment` + +A map of variables to write in `/etc/environment`, or otherwise specified in `environment_file` + +```yaml +stages: + default: + - name: "Setup users" + environment: + FOO: "bar" +``` +### `stages.STAGE_ID.STEP_NAME.environment_file` + +A string to specify where to set the environment file + +```yaml +stages: + default: + - name: "Setup users" + environment_file: "/home/user/.envrc" + environment: + FOO: "bar" +``` +### `stages.STAGE_ID.STEP_NAME.timesyncd` + +Sets the `systemd-timesyncd` daemon file (`/etc/system/timesyncd.conf`) file accordingly. The documentation for `timesyncd` and all the options can be found [here](https://www.freedesktop.org/software/systemd/man/timesyncd.conf.html). + +```yaml +stages: + default: + - name: "Setup NTP" + systemctl: + enable: + - systemd-timesyncd + timesyncd: + NTP: "0.pool.org foo.pool.org" + FallbackNTP: "" + ... +``` + +### `stages.STAGE_ID.STEP_NAME.commands` + +A list of arbitrary commands to run after file writes and directory creation. + +```yaml +stages: + default: + - name: "Setup something" + commands: + - echo 1 > /bar +``` + +### `stages.STAGE_ID.STEP_NAME.datasource` + +Sets to fetch user data from the specified cloud providers. It populates +provider specific data into `/run/config` folder and the custom user data +is stored into the provided path. + + +```yaml +stages: + default: + - name: "Fetch cloud provider's user data" + datasource: + providers: + - "aws" + - "digitalocean" + path: "/etc/cloud-data" +``` + +### `stages.STAGE_ID.STEP_NAME.layout` + + +Sets additional partitions on disk free space, if any, and/or expands the last +partition. All sizes are expressed in MiB only and default value of `size: 0` +means all available free space in disk. This plugin is useful to be used in +oem images where the default partitions might not suit the actual disk geometry. + + +```yaml +stages: + default: + - name: "Repart disk" + layout: + device: + # It will partition a device including the given filesystem label + # or partition label (filesystem label matches first) or the device + # provided in 'path'. The label check has precedence over path when + # both are provided. + label: "COS_RECOVERY" + path: "/dev/sda" + # Only last partition can be expanded and it happens before any other + # partition is added. size: 0 means all available free space + expand_partition: + size: 4096 + add_partitions: + - fsLabel: "COS_STATE" + size: 8192 + # No partition label is applied if omitted + pLabel: "state" + - fsLabel: "COS_PERSISTENT" + # default filesystem is ext2 if omitted + filesystem: "ext4" +``` diff --git a/docs/toolkit/reference/elemental.md b/docs/toolkit/reference/elemental.md new file mode 100644 index 000000000..745f5d462 --- /dev/null +++ b/docs/toolkit/reference/elemental.md @@ -0,0 +1,29 @@ +--- +title: "elemental" +sidebar_label: "elemental" +--- +## elemental + +Elemental + +### Options + +``` + --config-dir string Set config dir + --debug Enable debug output + -h, --help help for elemental + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental build-iso](elemental_build-iso.md) - Build bootable installation media ISOs +* [elemental cloud-init](elemental_cloud-init.md) - Run cloud-init +* [elemental install](elemental_install.md) - Elemental installer +* [elemental pull-image](elemental_pull-image.md) - Pull remote image to local file +* [elemental reset](elemental_reset.md) - Reset OS +* [elemental run-stage](elemental_run-stage.md) - Run stage from cloud-init +* [elemental upgrade](elemental_upgrade.md) - Upgrade the system +* [elemental version](elemental_version.md) - Print the version + diff --git a/docs/toolkit/reference/elemental_build-iso.md b/docs/toolkit/reference/elemental_build-iso.md new file mode 100644 index 000000000..3030a932f --- /dev/null +++ b/docs/toolkit/reference/elemental_build-iso.md @@ -0,0 +1,52 @@ +--- +title: "elemental build-iso" +sidebar_label: "elemental build-iso" +--- + +Build bootable installation media ISOs + +### Synopsis + +Build bootable installation media ISOs + +SOURCE - should be provided as uri in following format `:` + * `` - might be ["dir", "file", "oci", "docker", "channel"], as default is "docker" + * `` - is path to file or directory, image name with tag version or channel name + +``` +elemental build-iso SOURCE [flags] +``` + +### Options + +``` + --bootloader-in-rootfs Fetch ISO bootloader binaries from the rootfs + --cosign Enable cosign verification (requires images with signatures) + --cosign-key string Sets the URL of the public key to be used by cosign validation + --date Adds a date suffix into the generated ISO file + -h, --help help for build-iso + --label string Label of the ISO volume + --local Use an image from local cache + -n, --name string Basename of the generated ISO file + -o, --output string Output directory (defaults to current directory) + --overlay-iso string Path of the overlayed iso data + --overlay-rootfs string Path of the overlayed rootfs data + --overlay-uefi string Path of the overlayed uefi data + --platform string Platform to build the image for (default "linux/amd64") + -x, --squash-compression stringArray cmd options for compression to pass to mksquashfs. Full cmd including --comp as the whole values will be passed to mksquashfs. For a full list of options please check mksquashfs manual. (default value: '-comp xz -Xbcj ARCH') + --squash-no-compression Disable squashfs compression. Overrides any values on squash-compression +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/docs/toolkit/reference/elemental_cloud-init.md b/docs/toolkit/reference/elemental_cloud-init.md new file mode 100644 index 000000000..c02032e89 --- /dev/null +++ b/docs/toolkit/reference/elemental_cloud-init.md @@ -0,0 +1,32 @@ +--- +title: "elemental cloud-init" +sidebar_label: "elemental cloud-init" +--- + +Run cloud-init + +``` +elemental cloud-init [flags] +``` + +### Options + +``` + -d, --dotnotation stages.foo.name=.. Parse input in dotnotation ( e.g. stages.foo.name=.. ) + -h, --help help for cloud-init + -s, --stage string Stage to apply (default "default") +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/docs/toolkit/reference/elemental_convert-disk.md b/docs/toolkit/reference/elemental_convert-disk.md new file mode 100644 index 000000000..f4fd34da5 --- /dev/null +++ b/docs/toolkit/reference/elemental_convert-disk.md @@ -0,0 +1,32 @@ +--- +title: "elemental convert-disk" +sidebar_label: "elemental convert-disk" +--- + +converts between a raw disk and a cloud operator disk image (azure,gce) + +``` +elemental convert-disk RAW_DISK [flags] +``` + +### Options + +``` + -h, --help help for convert-disk + --keep-source Keep the source image, otherwise it will delete it once transformed. + -t, --type string Type of image to create (default "azure") +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/docs/toolkit/reference/elemental_exit-codes.md b/docs/toolkit/reference/elemental_exit-codes.md new file mode 100644 index 000000000..00b8dd297 --- /dev/null +++ b/docs/toolkit/reference/elemental_exit-codes.md @@ -0,0 +1,80 @@ +--- +title: "Exit codes for elemental CLI" +sidebar_label: "Exit codes for elemental CLI" +--- + +| Exit code | Meaning | +| :----: | :---- | +| 10 | Error closing a file| +| 11 | Error running a command| +| 12 | Error copying data| +| 13 | Error copying a file| +| 14 | Wrong cosign flags used in cmd| +| 15 | Error creating a dir| +| 16 | Error creating a file| +| 17 | Error creating a temporal dir| +| 18 | Error dumping the source| +| 19 | Error creating a gzip writer| +| 20 | Error trying to identify the source| +| 21 | Error calling mkfs| +| 22 | There is not packages for the given architecture| +| 24 | Error opening a file| +| 25 | Output file already exists| +| 26 | Error reading the build config| +| 27 | Error reading the build-disk config| +| 28 | Error running stat on a file| +| 29 | Error creating a tar archive| +| 30 | Error truncating a file| +| 31 | Error reading the run config| +| 32 | Error reading the install/upgrade flags| +| 33 | Error reading the config for the command| +| 34 | Error mounting state partition| +| 35 | Error mounting recovery partition| +| 36 | Error during before-upgrade hook| +| 37 | Error during before-upgrade-chroot hook| +| 38 | Error during after-upgrade hook| +| 39 | Error during after-upgrade-chroot hook| +| 40 | Error moving file| +| 41 | Error occurred during cleanup| +| 42 | Error occurred trying to reboot| +| 43 | Error occurred trying to shutdown| +| 44 | Error occurred when labeling partition| +| 45 | Error setting default grub entry| +| 46 | Error occurred during selinux relabeling| +| 47 | Error invalid device specified| +| 48 | Error deploying image to file| +| 49 | Error installing GRUB| +| 50 | Error during before-install hook| +| 51 | Error during after-install hook| +| 52 | Error during after-install-chroot hook| +| 53 | Error during file download| +| 54 | Error mounting partitions| +| 55 | Error deactivating active devices| +| 56 | Error during device partitioning| +| 57 | Device already contains an install| +| 58 | Command requires root privileges| +| 59 | Error occurred when unmounting partitions| +| 60 | Error occurred when formatting partitions| +| 61 | Error during before-reset hook| +| 62 | Error during after-reset-chroot hook| +| 63 | Error during after-reset hook| +| 64 | Unsupported flavor| +| 65 | Error encountered during cloud-init run-stage| +| 66 | Error unpacking image| +| 67 | Error reading file| +| 68 | No source was provided for the command| +| 69 | Error removing a file| +| 70 | Error calculating checksum| +| 71 | Error occurred when unmounting image| +| 72 | Error occurred during post-upgrade hook| +| 73 | Error occurred during post-reset hook| +| 74 | Error occurred during post-install hook| +| 75 | Error occurred while preparing the image root tree| +| 76 | Error occurred while creating the OS filesystem image| +| 77 | Error occurred while copying the filesystem image and setting new labels| +| 78 | Error setting persistent GRUB variables| +| 255 | Unknown error| + +### SEE ALSO + +* [elemental](elemental.md) - Elemental diff --git a/docs/toolkit/reference/elemental_install.md b/docs/toolkit/reference/elemental_install.md new file mode 100644 index 000000000..bec9d63c2 --- /dev/null +++ b/docs/toolkit/reference/elemental_install.md @@ -0,0 +1,50 @@ +--- +title: "elemental install" +sidebar_label: "elemental install" +--- + +Elemental installer + +``` +elemental install DEVICE [flags] +``` + +### Options + +``` + -c, --cloud-init strings Cloud-init config files + --cosign Enable cosign verification (requires images with signatures) + --cosign-key string Sets the URL of the public key to be used by cosign validation + --disable-boot-entry Dont create an EFI entry for the system install. + --eject-cd Try to eject the cd on reboot, only valid if booting from iso + --firmware string Firmware to install for: 'efi' or 'bios'. (defaults to 'efi') (default "efi") + --force Force install + -h, --help help for install + -i, --iso string Performs an installation from the ISO url + --local Use an image from local cache + --no-format Don’t format disks. It is implied that COS_STATE, COS_RECOVERY, COS_PERSISTENT, COS_OEM are already existing + --part-table string Partition table type to use (default "gpt") + --platform string Platform to build the image for (default "linux/amd64") + --poweroff Shutdown the system after install + --reboot Reboot the system after install + --recovery-system.uri string Sets the recovery image source and its type (e.g. 'docker:registry.org/image:tag') + -x, --squash-compression stringArray cmd options for compression to pass to mksquashfs. Full cmd including --comp as the whole values will be passed to mksquashfs. For a full list of options please check mksquashfs manual. (default value: '-comp xz -Xbcj ARCH') + --squash-no-compression Disable squashfs compression. Overrides any values on squash-compression + --strict Enable strict check of hooks (They need to exit with 0) + --system.uri string Sets the system image source and its type (e.g. 'docker:registry.org/image:tag') + --verify Enable mtree checksum verification (requires images manifests generated with mtree separately) +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/docs/toolkit/reference/elemental_new.md b/docs/toolkit/reference/elemental_new.md new file mode 100644 index 000000000..46454eb4f --- /dev/null +++ b/docs/toolkit/reference/elemental_new.md @@ -0,0 +1,31 @@ +--- +title: "elemental new" +sidebar_label: "elemental new" +--- + +Create skeleton Dockerfile for a derivative + +``` +elemental new FLAVOR [flags] +``` + +### Options + +``` + --arch string X86_64 or aarch64 architectures + -h, --help help for new +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/docs/toolkit/reference/elemental_pull-image.md b/docs/toolkit/reference/elemental_pull-image.md new file mode 100644 index 000000000..025c3343a --- /dev/null +++ b/docs/toolkit/reference/elemental_pull-image.md @@ -0,0 +1,32 @@ +--- +title: "elemental pull-image" +sidebar_label: "elemental pull-image" +--- + +Pull remote image to local file + +``` +elemental pull-image IMAGE DESTINATION [flags] +``` + +### Options + +``` + -h, --help help for pull-image + --local Use an image from local cache + --platform string Platform to build the image for (default "linux/amd64") +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/docs/toolkit/reference/elemental_reset.md b/docs/toolkit/reference/elemental_reset.md new file mode 100644 index 000000000..f287b107d --- /dev/null +++ b/docs/toolkit/reference/elemental_reset.md @@ -0,0 +1,41 @@ +--- +title: "elemental reset" +sidebar_label: "elemental reset" +--- + +Reset OS + +``` +elemental reset [flags] +``` + +### Options + +``` + -c, --cloud-init strings Cloud-init config files + --cosign Enable cosign verification (requires images with signatures) + --cosign-key string Sets the URL of the public key to be used by cosign validation + --disable-boot-entry Dont create an EFI entry for the system install. + -h, --help help for reset + --poweroff Shutdown the system after install + --reboot Reboot the system after install + --reset-oem Clear OEM partitions + --reset-persistent Clear persistent partitions + --strict Enable strict check of hooks (They need to exit with 0) + --system.uri string Sets the system image source and its type (e.g. 'docker:registry.org/image:tag') + --verify Enable mtree checksum verification (requires images manifests generated with mtree separately) +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/docs/toolkit/reference/elemental_run-stage.md b/docs/toolkit/reference/elemental_run-stage.md new file mode 100644 index 000000000..4c08d924b --- /dev/null +++ b/docs/toolkit/reference/elemental_run-stage.md @@ -0,0 +1,31 @@ +--- +title: "elemental run-stage" +sidebar_label: "elemental run-stage" +--- + +Run stage from cloud-init + +``` +elemental run-stage STAGE [flags] +``` + +### Options + +``` + -h, --help help for run-stage + --strict Set strict checking for errors, i.e. fail if errors were found +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/docs/toolkit/reference/elemental_upgrade.md b/docs/toolkit/reference/elemental_upgrade.md new file mode 100644 index 000000000..c431ab3c0 --- /dev/null +++ b/docs/toolkit/reference/elemental_upgrade.md @@ -0,0 +1,42 @@ +--- +title: "elemental upgrade" +sidebar_label: "elemental upgrade" +--- + +Upgrade the system + +``` +elemental upgrade [flags] +``` + +### Options + +``` + --cosign Enable cosign verification (requires images with signatures) + --cosign-key string Sets the URL of the public key to be used by cosign validation + -h, --help help for upgrade + --local Use an image from local cache + --poweroff Shutdown the system after install + --reboot Reboot the system after install + --recovery Upgrade the recovery + --recovery-system.uri string Sets the recovery image source and its type (e.g. 'docker:registry.org/image:tag') + -x, --squash-compression stringArray cmd options for compression to pass to mksquashfs. Full cmd including --comp as the whole values will be passed to mksquashfs. For a full list of options please check mksquashfs manual. (default value: '-comp xz -Xbcj ARCH') + --squash-no-compression Disable squashfs compression. Overrides any values on squash-compression + --strict Enable strict check of hooks (They need to exit with 0) + --system.uri string Sets the system image source and its type (e.g. 'docker:registry.org/image:tag') + --verify Enable mtree checksum verification (requires images manifests generated with mtree separately) +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/docs/toolkit/reference/elemental_version.md b/docs/toolkit/reference/elemental_version.md new file mode 100644 index 000000000..bcc104e86 --- /dev/null +++ b/docs/toolkit/reference/elemental_version.md @@ -0,0 +1,31 @@ +--- +title: "elemental version" +sidebar_label: "elemental version" +--- + +Print the version + +``` +elemental version [flags] +``` + +### Options + +``` + -h, --help help for version + --long Show long version info +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/docs/toolkit/reference/high_level_architecture.md b/docs/toolkit/reference/high_level_architecture.md new file mode 100644 index 000000000..c8ffab901 --- /dev/null +++ b/docs/toolkit/reference/high_level_architecture.md @@ -0,0 +1,47 @@ +--- +title: "High level architecture" +sidebar_label: "High level architecture" +--- + +# Elemental toolkit High level Architecture + +This page tries to encompass the [`elemental-toolkit`](https://github.com/rancher/elemental-toolkit) structure and the high level architecture, along with all the involved components. + + +## Design goals + +- Blueprints to build immutable Linux derivatives from container images +- A workflow to maintain, support and deliver custom-OS and upgrades to end systems +- Derivatives have the same “foundation” manifest - easy to customize on top, add packages: `systemd`, `dracut` and `grub` as a foundation stack. +- Upgrades delivered with container registry images ( also workflow with `docker run` && `docker commit` supported! ) +
The content of the container image is the system which is booted. + + +## High level overview + +elemental-toolkit encompasses several components required for building and distributing OS images. [This issue](https://github.com/rancher/elemental-toolkit/issues/108) summarize the current state, and how we plan to integrate them in a single CLI to improve the user experience. + +elemental-toolkit is also a manifest, which includes package definitions of how the underlying OS is composed. It forms an abstraction layer, which is then translated to Dockerfiles and built by our CI (optionally) for re-usal. A derivative can be built by parts of the manifest, or reusing it entirely, container images included. + +![High level overview](https://docs.google.com/drawings/d/e/2PACX-1vQQJOaISPbMxMYU44UT-M3ou9uGYOrzbXCRXMLPU8m7_ie3ke_08xCsyRLkFZJRB4VnzIeobPciEoQv/pub?w=942&h=532) + +The fundamental phases can be summarized in the following steps: + +- Build packages from container images (and optionally keep build caches) +- Extract artefacts from containers +- Add metadata(s) and create a repository +- (optionally) publish the repository and the artefacts + +The developer of the derivative applies a customization layer during build, which is an augmentation layer in the same form of `elemental-toolkit` itself. + +## Distribution + +The OS delivery mechanism is done via container registries. The developer that wants to provide upgrades for the custom OS will push the resulting container images to the container registry. It will then be used by the installed system to pull upgrades from. + +![](https://docs.google.com/drawings/d/e/2PACX-1vQrTArCYgu-iscf29v1sl1sEn2J81AqBpi9D5xpwGKr9uxR2QywoSqCmsSaJLxRRacoRr0Kq40a7jPF/pub?w=969&h=464) + +## Upgrade mechanism + +There are two different upgrade mechanisms available that can be used from a maintainer perspective: (a) release channels or (b) providing a container image reference ( `e.g. my.registry.com/image:tag` ) [that can be tweaked in the customization phases](https://github.com/rancher/elemental-toolkit#default-oem) to achieve the desired effect. + + \ No newline at end of file diff --git a/docs/toolkit/reference/immutable_rootfs.md b/docs/toolkit/reference/immutable_rootfs.md new file mode 100644 index 000000000..6ca97cfdd --- /dev/null +++ b/docs/toolkit/reference/immutable_rootfs.md @@ -0,0 +1,23 @@ +--- +title: "Immutable Root Filesystem" +sidebar_label: "Immutable Rootfs" +--- + +The immutable rootfs concept in Elemental is provided by a dracut module. +By default, `elemental` and derivatives will inherit an immutable setup. + +![Partitioning layout](https://docs.google.com/drawings/d/e/2PACX-1vR-I5ZwwB5EjpsymUfcNADRTTKXrNMnlZHgD8RjDpzYhyYiz_JrWJwvpcfMcwfYet1oWCZVWH22aj1k/pub?w=533&h=443) + +A running system will look like as follows: + +``` +/usr/local - persistent (COS_PERSISTENT) +/oem - persistent (COS_OEM) +/etc - ephemeral +/usr - read only +/ immutable +``` + +This means that any changes that are not specified as cloud-init configuration are not persisting across reboots. + +You can place persisting cloud-init files either in `/oem` or `/usr/local/oem`, `Elemental` already supports cloud-init [datasources](https://cloudinit.readthedocs.io/en/latest/topics/datasources.html), so you can use also load cloud-init configuration as standard userdata, depending on the platform. For more details on the cloud-init syntax, see the [cloud-init configuration reference](cloud_init). diff --git a/docs/toolkit/reference/layout.md b/docs/toolkit/reference/layout.md new file mode 100644 index 000000000..13cb7e5eb --- /dev/null +++ b/docs/toolkit/reference/layout.md @@ -0,0 +1,42 @@ +--- +title: "Runtime layout" +sidebar_label: "Runtime layout" +--- + +This section describes the runtime layout of a derivative (or a Elemental Vanilla image) once booted in a system. + +The Elemental toolkit performs during installation a common setup which is equivalent across all derivatives. + +This mechanism ensures that a layout: + +- it's simple and human friendly +- allows to switch easily derivatives +- allows to perform recovery tasks +- is resilient to upgrade failures + +## Layout + +The basic setup consists of: + +- an `A/B` partitioning style. We have an 'active' and a 'passive' system too boot from in case of failures +- a Recovery system which allows to perform emergency tasks in case of failure of the 'A/B' partitions +- a Fallback mechanism that boots the partitions in this sequence: "A -> B -> Recovery" in case of booting failures + +The upgrade happens in a transition image and take places only after all the necessary steps are completed. An upgrade of the 'A/B' partitions can be done by booting into them and running `elemental upgrade`. This will create a new pristine image that will be selected as active for the next reboot, the old one will be flagged as passive. If we are performing the same from the passive system, only the active is subject to changes. + +Similarly, a recovery system can be upgraded as well by running `elemental upgrade --recovery`. This will upgrade the recovery system instead of the active/passive. Note both commands needs to be run inside the active or passive system. + +## Partitions + +![](https://docs.google.com/drawings/d/e/2PACX-1vSP-Pz9l9hwYDeIlej7qXzzcMzGYBiKjyFpiYYKlbNR3H37n_R_c0eBNeYa3msouOupmDim3ZYYBSxS/pub?w=812&h=646) + +The default partitioning is created during installation and is expected to be present in a booted Elemental system: + +- a `COS_STATE` partition that will contain our active, passive and recovery images. The images are located under the `/cOS` directory +- a `COS_PERSISTENT` partition which contains the persistent user data. This directory is mounted over `/usr/local` during runtime +- a `COS_OEM` partition which contains the cloud-init oem files, which is mounted over `/oem` during runtime +- a `COS_RECOVERY` partition which contains the recovery system image + +The `COS_STATE` partitions contains the `active`, `passive` . While the `active` and `passive` are `.img` files which are loopback mounted, the `recovery` system is in `COS_RECOVERY` and can also be a `squashfs` file (provided in `/cOS/recovery.squashfs`). This ensures the immutability aspect and ease out building derivative in constrained environments (e.g. when we have restricted permissions and we can't mount). + +For more information about the immutability aspect of Elemental, see [Immutable rootfs](immutable_rootfs) diff --git a/docs/toolkit/reference/troubleshooting.md b/docs/toolkit/reference/troubleshooting.md new file mode 100644 index 000000000..47a3ad62c --- /dev/null +++ b/docs/toolkit/reference/troubleshooting.md @@ -0,0 +1,73 @@ +--- +title: "Troubleshooting" +sidebar_label: "Troubleshooting" +--- + +{{% pageinfo color="warning"%}} +Section under construction. +{{% /pageinfo %}} + +While building a derivative, or on a running system things can go really wrong, the guide is aimed to give tips while building derivatives and also debugging running systems. + +Don't forget tocheck the known issues for the [release you're using](https://github.com/rancher/elemental-toolkit/issues). + +Before booting, [several kernel parameters](immutable_rootfs) can be used to help during debugging (also when booting an ISO). Those are meant to be used only while debugging, and they might defeat the concept of immutability. + +## Disable Immutability + +By adding `rd.cos.debugrw` to the boot parameters read only mode will be disabled. See [Immutable setup](immutable_rootfs) for more options. + +The derivative will boot into RW mode, that means any change made during runtime will persist across reboots. Use this feature with caution as defeats the concept of immutability. + +`rd.cos.debugrw` applies only to active and passive partitions. The recovery image can't be mutated. + +{{% alert title="Note" %}} +The changes made will persist during reboots but won't persist across upgrades. If you need to persist changes across upgrades in runtime (for example by adding additional packages on top of the derivative image), see [how to apply persistent changes](../customizing/runtime_persistent_changes). +{{% /alert %}} + +## Debug initramfs issues + +As derivative can ship and build their own initrd, the [official debug docs](https://fedoraproject.org/wiki/How_to_debug_Dracut_problems) contains valid information that can be used for troubleshooting. + +For example: + +- `rd.break=pre-mount rd.shell`: Drop a shell before setting up mount points +- `rd.break=pre-pivot rd.shell`: Drop a shell before switch-root + +## Recovery partition + +If you can boot into the system, the recovery partition can be used to reset the state of the active/passive, but can also be used to upgrade to specific images. Be sure to read the [Recovery section in the docs](../getting-started/recovery). + +## Mutating derivative images + +It can be useful to mutate derivative images and commit a container’s file changes or settings into a new image. +This allows you to debug a container by running an interactive shell, and re-use the mutated image in Elemental systems. Generally, it is better to use Dockerfiles to manage your images in a documented and maintainable way. [Read more about creating bootable images](../creating-derivatives/creating_bootable_images). + +Let's suppose we have the derivative original image at `$IMAGE` and we want to mutate it. We will push it later with another name `$NEW_IMAGE` and use it to our node downstream. + +Run the derivative image locally, and perform any necessary change (e.g. add additional software): +```bash +$> docker run --entrypoint /bin/bash -ti --name updated-image $IMAGE +``` + +Commit any changes to a new image `$NEW_IMAGE`: +```bash +$> docker commit updated-image $NEW_IMAGE +``` + +And push the image to the container registry: +```bash +$> docker push $NEW_IMAGE +``` + +In the derivative then it's sufficient to upgrade to that image with `elemental upgrade`: + +```bash +$> elemental upgrade --docker-image $NEW_IMAGE +``` + +## Adding login keys at boot + +To add users key from the GRUB menu prompt, edit the boot cmdline and add the following kernel parameters: + +`stages.boot[0].authorized_keys.root[0]=github:suse` diff --git a/docs/toolkit/tutorials/_index.md b/docs/toolkit/tutorials/_index.md new file mode 100644 index 000000000..d9a0a2990 --- /dev/null +++ b/docs/toolkit/tutorials/_index.md @@ -0,0 +1,9 @@ + +--- +title: "Tutorials" +sidebar_label: "Tutorials" +weight: 6 +date: 2017-01-04 +description: > + Elemental tutorials and real life use-case samples +--- diff --git a/docs/toolkit/tutorials/k3s_and_fleet_on_vanilla_image_example.md b/docs/toolkit/tutorials/k3s_and_fleet_on_vanilla_image_example.md new file mode 100644 index 000000000..e6508fddf --- /dev/null +++ b/docs/toolkit/tutorials/k3s_and_fleet_on_vanilla_image_example.md @@ -0,0 +1,109 @@ +--- +title: "K3s + Fleet" +sidebar_label: "K3s and Fleet" +--- + +This is a work in progress example of how to deploy K3S + Fleet + System Upgrade Controller over a Elemental vanilla image only +by using cloud-init yaml configuration files. The config file reproduced here is meant to be included +as a user-data in a cloud provider (aws, gcp, azure, etc) or as part of a cdrom (Elemental-Recovery will try to fetch `/userdata` file +from a cdrom device). + +A vanilla image is an image that only provides the Elemental-Recovery system on a `COS_RECOVERY` partition. It does not include any other +system and it is meant to be dumped to a bigger disk and deploy a Elemental system or a derivative system over the free space in disk. +COS vanilla images are build as part of the CI workflow, see CI artifacts to download one of those. + +The configuration file of this example has two purposes: first it deploys Elemental, second in reboots on the deployed OS and deploys +K3S + Fleet + System Upgrades Controller. + +On first boot it will fail to boot Elemental grub menu entry and fallback +to Elemental-Recovery system. From there it will partition the vanilla image to create the main system partition (`COS_STATE`) +and add an extra partition for persistent data (`COS_PERSISTENT`). It will use the full disk, a disk of at least 20GiB +is recommended. After partitioning it will deploy the main system on `COS_STATE` and reboot to it. + +On consequent boots it will simply boot from `COS_STATE`, there it prepares the persistent areas of the system (arranges few bind +mounts inside `COS_PERSISTENT`) and then it runs an standard installation of K3s, Fleet and System Upgrade Controller. After few +minutes after the system is up the K3s cluster is up and running. + +Note this setup similar to the [derivative example](https://github.com/rancher-sandbox/cos-fleet-upgrades-sample) using Fleet. +The main difference is that this example does not require to build any image, it is pure cloud-init configuration based. + +### User data configuration file +```yaml +name: "Default deployment" +stages: + rootfs.after: + - if: '[ -f "/run/cos/recovery_mode" ]' + name: "Repart image" + layout: + # It will partition a device including the given filesystem label or part label (filesystem label matches first) + device: + label: COS_RECOVERY + add_partitions: + - fsLabel: COS_STATE + # 15Gb for COS_STATE, so the disk should have, at least, 20Gb + size: 15360 + pLabel: state + - fsLabel: COS_PERSISTENT + # unset size or 0 size means all available space + pLabel: persistent + - if: '[ ! -f "/run/cos/recovery_mode" ]' + name: "Persistent state" + environment_file: /run/cos/cos-layout.env + environment: + VOLUMES: "LABEL=COS_OEM:/oem LABEL=COS_PERSISTENT:/usr/local" + OVERLAY: "tmpfs:25%" + RW_PATHS: "/var /etc /srv" + PERSISTENT_STATE_PATHS: "/root /opt /home /var/lib/rancher /var/lib/kubelet /etc/systemd /etc/rancher /etc/ssh" + network.before: + - name: "Setup SSH keys" + authorized_keys: + root: + # It can download ssh key from remote places, such as github user keys (e.g. `github:my_user`) + - my_custom_ssh_key + - if: '[ ! -f "/run/cos/recovery_mode" ]' + name: "Fleet deployment" + files: + - path: /etc/k3s/manifests/fleet-config.yaml + content: | + apiVersion: helm.cattle.io/v1 + kind: HelmChart + metadata: + name: fleet-crd + namespace: kube-system + spec: + chart: https://github.com/rancher/fleet/releases/download/v0.3.3/fleet-crd-0.3.3.tgz + --- + apiVersion: helm.cattle.io/v1 + kind: HelmChart + metadata: + name: fleet + namespace: kube-system + spec: + chart: https://github.com/rancher/fleet/releases/download/v0.3.3/fleet-0.3.3.tgz + network: + - if: '[ -f "/run/cos/recovery_mode" ]' + name: "Deploy cos-system" + commands: + # Deploys the recovery image. + # use --docker-image to deploy a custom image + # e.g. `elemental reset --docker-image quay.io/my_custom_repo:my_image` + - elemental reset --reboot + - if: '[ ! -f "/run/cos/recovery_mode" ]' + name: "Setup k3s" + directories: + - path: "/usr/local/bin" + permissions: 0755 + owner: 0 + group: 0 + commands: + - | + curl -sfL https://get.k3s.io | \ + INSTALL_K3S_VERSION="v1.20.4+k3s1" \ + INSTALL_K3S_EXEC="--tls-san {{.Values.node.hostname}}" \ + INSTALL_K3S_SELINUX_WARN="true" \ + sh - + # Install fleet + kubectl apply -f /etc/k3s/manifests/fleet-config.yaml + # Install system-upgrade-controller + kubectl apply -f https://raw.githubusercontent.com/rancher/system-upgrade-controller/v0.6.2/manifests/system-upgrade-controller.yaml +``` diff --git a/docs/toolkit/tutorials/trigger_upgrades_with_fleet.md b/docs/toolkit/tutorials/trigger_upgrades_with_fleet.md new file mode 100644 index 000000000..7e139f219 --- /dev/null +++ b/docs/toolkit/tutorials/trigger_upgrades_with_fleet.md @@ -0,0 +1,63 @@ +--- +title: "Trigger upgrades with K3s and Fleet" +sidebar_label: "Trigger upgrades with K3s and Fleet" +--- + +![](https://docs.google.com/drawings/d/e/2PACX-1vQPv9TI3D95vocG7oCHmVmNuuvBYuc2_0kaxAc6xnCBM9mFTnUTFIIDkzZKUFFP-xyw2Hg4q9XhxLD8/pub?w=1185&h=712) + +In this tutorial we will: + +1) Build a custom OS image to deploy in our cluster +2) Setup a cluster with Elemental, k3s and fleet +3) Upgrade the cluster to our custom OS image with fleet + +[This repository](https://github.com/rancher-sandbox/cos-fleet-upgrades-sample/) contains the full example code. + +## 1) Build the OS image + +```bash +# IMAGE=quay.io/costoolkit/test-images:fleet-sample +# cd os +# docker build -t $IMAGE . +``` + +## 2) Push the docker image + + +```bash +# docker push $IMAGE +``` + +## 3) Prepare a Elemental VM + +Download an ISO, or a qcow image from the Github artifacts of Elemental. + +If deploying on AWS/openstack/Cloud, use the `fleet-cloud-init.yaml` file as userdata. If deploying on baremetal/VMs, place `fleet-cloud-init.yaml` in `/oem` after install (or run the installer with `elemental install --cloud-init https://raw.githubusercontent.com/rancher-sandbox/cos-fleet-upgrades-sample/main/fleet-cloud-init.yaml $DEVICE`). + +Reboot, after some bootstraping time (check until all pods are running with `watch kubectl get pods -A`), you should have a k3s cluster with fleet and [system-upgrade-controller](https://github.com/rancher/system-upgrade-controller) deployed. + +## 4) Upgrade with fleet + +Add your fleet repository to the fleet cluster: + +```bash +cat > example.yaml << "EOF" +apiVersion: fleet.cattle.io/v1alpha1 +kind: GitRepo +metadata: + name: upgrade + # This namespace is special and auto-wired to deploy to the local cluster + namespace: fleet-local +spec: + # Everything from this repo will be ran in this cluster. You trust me right? + repo: "https://github.com/rancher-sandbox/cos-fleet-upgrades-sample" + branch: "main" + paths: + - manifests +EOF + +kubectl apply -f example.yaml +``` + +An example of how to trigger an upgrade with fleet is in `manifests/upgrade.yaml`. Edit the image with the one generated in the previous steps, and commit it to your **fleet repository**, At this point you should see the upgrade job to kick-in, the system will reboot afterwards. + diff --git a/docusaurus.config.js b/docusaurus.config.js index 5b8074985..c307a02ca 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -7,7 +7,7 @@ const darkCodeTheme = require('prism-react-renderer').themes.dracula; /** @type {import('@docusaurus/types').Config} */ const config = { title: 'Elemental - Immutable Linux for Rancher', - url: 'https://rancher.github.io', + url: 'https://elemental.docs.rancher.com', baseUrl: '/', onBrokenLinks: 'throw', onBrokenMarkdownLinks: 'warn', @@ -74,6 +74,12 @@ const config = { position: 'left', label: 'Documentation', }, + { + type: 'docSidebar', + sidebarId: 'toolkit', + position: 'left', + label: 'Toolkit', + }, { type: 'docsVersionDropdown', position: 'right', diff --git a/sidebars.js b/sidebars.js index 12f037fbe..064388c81 100644 --- a/sidebars.js +++ b/sidebars.js @@ -77,11 +77,11 @@ const sidebars = { ] }, { - type: 'category', - collapsible: true, - collapsed: true, - label: 'How to', - items: [ + "type": 'category', + "collapsible": true, + "collapsed": true, + "label": 'How to', + "items": [ 'wifi', 'elemental_behind_proxy', 'hostname', @@ -93,11 +93,11 @@ const sidebars = { ] }, { - type: 'category', - collapsible: true, - collapsed: true, - label: 'Troubleshooting', - items: [ + "type": 'category', + "collapsible": true, + "collapsed": true, + "label": 'Troubleshooting', + "items": [ { "type": "category", "collapsible": true, @@ -138,12 +138,112 @@ const sidebars = { }, "release-notes", ], - + "toolkit": [ + "toolkit/index", + { + "type": "category", + "collapsible": true, + "collapsed": false, + "label": "Getting Started", + "items": [ + "toolkit/getting-started/download", + "toolkit/getting-started/install", + "toolkit/getting-started/upgrading", + "toolkit/getting-started/recovery", + "toolkit/getting-started/deploy" + ], + }, + { + "type": "category", + "label": "Customizing", + "collapsible": true, + "collapsed": true, + "items": [ + "toolkit/customizing/stages", + "toolkit/customizing/configuration_persistency", + "toolkit/customizing/embedded_features", + "toolkit/customizing/login", + "toolkit/customizing/oem_configuration", + "toolkit/customizing/runtime_persistent_changes", + "toolkit/customizing/general_configuration", + "toolkit/customizing/upgrades", + "toolkit/customizing/configure_grub", + "toolkit/customizing/selinux_support" + ], + }, + { + "type": "category", + "label": "Creating Derivatives", + "collapsible": true, + "collapsed": true, + "items": [ + "toolkit/creating-derivatives/package_stack", + "toolkit/creating-derivatives/creating_bootable_images", + "toolkit/creating-derivatives/build_disk", + "toolkit/creating-derivatives/build_iso" + ], + }, + { + "type": "category", + "label": "Examples", + "items": [ + "toolkit/examples/creating_bootable_images", + "toolkit/examples/cloud_config", + "toolkit/examples/embedded_images" + ], + }, + { + "type": "category", + "label": "Tutorials", + "items": [ + "toolkit/tutorials/k3s_and_fleet_on_vanilla_image_example", + "toolkit/tutorials/trigger_upgrades_with_fleet" + ], + }, + { + "type": "category", + "label": "Reference", + "items": [ + "toolkit/reference/cloud_init", + "toolkit/reference/immutable_rootfs", + "toolkit/reference/layout", + "toolkit/reference/troubleshooting", + "toolkit/reference/built_with_elemental", + "toolkit/reference/high_level_architecture", + { + "type": "category", + "label": "Command Line Interface", + "items": [ + "toolkit/reference/elemental", + "toolkit/reference/elemental_build-iso", + "toolkit/reference/elemental_cloud-init", + "toolkit/reference/elemental_convert-disk", + "toolkit/reference/elemental_exit-codes", + "toolkit/reference/elemental_install", + "toolkit/reference/elemental_new", + "toolkit/reference/elemental_pull-image", + "toolkit/reference/elemental_reset", + "toolkit/reference/elemental_run-stage", + "toolkit/reference/elemental_upgrade", + "toolkit/reference/elemental_version" + ], + }, + ], + }, + { + "type": "category", + "label": "Development", + "items": [ + "toolkit/development/creating_derivatives", + "toolkit/development/dependencies" + ], + }, + ], // But you can create a sidebar manually /* tutorialSidebar: [ { - type: "category", + "type": "category", label: "Tutorial", items: ["hello"], }, diff --git a/versioned_docs/version-1.2/toolkit/creating-derivatives/_index.md b/versioned_docs/version-1.2/toolkit/creating-derivatives/_index.md new file mode 100644 index 000000000..69ed27aa0 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/creating-derivatives/_index.md @@ -0,0 +1,27 @@ + +--- +title: "Creating derivatives" +sidebar_label: "Creating derivatives" +weight: 4 +date: 2023-05-11 +description: > + Documents various methods for creating Elemental derivatives +--- + +A derivative is a standard container image which can be booted by Elemental. + +We can identify a build phase where we build the derivative, and a "runtime phase" where we consume it. + +The image is described by a Dockerfile, composed of a base OS of choice (e.g. openSUSE, Ubuntu, etc. ) and the Elemental toolkit itself in order to be consumed by Elemental and allow to be upgraded from by other derivatives. + +elemental-toolkit then converts the OCI artifact into a bootable medium (ISO, packer, ova, etc) and the image itself then can be used to bootstrap other derivatives, which can in turn upgrade to any derivative built with Elemental. + +A derivative can also be later re-used again as input as base-image for downstream derivatives. + +All the documentation below imply that the container image generated will be the booting one, there are however several configuration entrypoint that you should keep in mind while building the image which are general across all the implementation: + +- Custom persistent runtime configuration has to be provided in `/system/oem` for derivatives, [see also the documentation section](../customizing/configuration_persistency). Everything under `/system/oem` will be loaded during the various stages (boot, network, initramfs). +- `/etc/cos/bootargs.cfg` contains the booting options required to boot the image with GRUB, [see grub customization](../customizing/configure_grub) + +Derivatives inherits `Elemental` defaults, which you can override during the build process, however there are some defaults which are relevant and listed below: + diff --git a/versioned_docs/version-1.2/toolkit/creating-derivatives/build_disk.md b/versioned_docs/version-1.2/toolkit/creating-derivatives/build_disk.md new file mode 100644 index 000000000..6171709a3 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/creating-derivatives/build_disk.md @@ -0,0 +1,61 @@ +--- +title: "Build disk images with Elemental" +sidebar_label: "Build disk images with Elemental" +--- + +Requirements: + +* `qemu-img` utility +* `elemental` binary +* elemental runtime dependencies + +The suggested approach is based on using the Elemental installer (`elemental install` command) to run the installation +from a Linux to a loop device. The loop device can be a raw image created with `qemu-img create` that can easily be +converted to other formats after the installation by using `qemu-img convert`. + +## Prepare the loop device + +Preparing the a loop device for the installation is simple and straight forward. + +```bash +# Create a raw image of 32G +> qemu-img create -f raw disk.img 32G + +# Set the disk image as a loop device +> sudo losetup -f --show disk.img + +``` + +## Run elemental installation + +Execute the elemental installation as described in [installing](../getting-started/install): + +```bash +> sudo elemental install --firmware efi --system.uri oci: +``` + +Where `` is the Elemental derivative container image we want to use for the disk creation and `` is the +loop device previously created with `losetup` (e.g. `/dev/loop0`). + + +## Convert the RAW image to desired format + +Once the installation is done just unsetting the loop device and converting the image to the desired format is missing: + +```bash +# Unset the loop device +> sudo losetup -d + +# Convert the RAW image to qcow2 +> qemu-img convert -f raw -O qcow2 disk.img disk.qcow2 +``` + +QEMU supports a wide range of formats including common ones such as `vdi`, `vmdk` or `vhdx`. + +The result can be easily tested on QEMU with: + +```bash +> qemu -m 4096 -hda disk.qcow2 -bios /usr/share/qemu/ovmf-x86_64.bin +``` + +Note the firmware image path varies depending on the host distro, the path provided in this example is based on openSUSE Leap. diff --git a/versioned_docs/version-1.2/toolkit/creating-derivatives/build_iso.md b/versioned_docs/version-1.2/toolkit/creating-derivatives/build_iso.md new file mode 100644 index 000000000..394c08943 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/creating-derivatives/build_iso.md @@ -0,0 +1,184 @@ +--- +title: "Build ISOs" +sidebar_label: "Build ISOs" +--- + +In order to build an ISO we rely on `elemental build-iso` command. It accepts a YAML file denoting the sources to bundle in an ISO. In addition it can also overlay custom files or use container images from a registry as packages. + +To build an ISO, just run: + +```bash +docker run --rm -ti -v $(pwd):/build ghcr.io/rancher/elemental-toolkit/elemental-cli:latest --debug build-iso -o /build $SOURCE +``` + +Where `$SOURCE` might be the container image you want to build the ISO for, you might want to check on [how to build bootable images](creating_bootable_images). Argument `$SOURCE` might be the reference to the directory, file, container image or channel we are building the ISO for, it should be provided as uri in following format `:`, where: + * `` - might be ["dir", "file", "oci", "docker"], as default is taken "docker" + * `` - is path to file or directory, channel or image name with tag version (if tag was not provided then "latest" is used) + +`elemental build-iso` command also supports reading a configuration `manifest.yaml` file. It is loaded form the directory specified by `--config-dir` elemental's flag. + +An example of a yaml file using the bootloader from the contained image: + +```yaml +iso: + bootloader-in-rootfs: true + grub-entry-name: "Installer" + +name: "Elemental-0" +date: true +``` + +## What's next? + +- Check out on how to [build an image](build_disk) from the ISO we have just created + +## Syntax + +Below you can find a full reference about the yaml file format. + +```yaml +iso: + # Sources to be installed in the rootfs + rootfs: + - .. + # Sources to be installed in the uefi image + uefi: + - .. + # Sources to be installed in the iso image + image: + - .. + label: "COS_LIVE" +``` + +Sources can be an image reference (then an explicit tag is required) or a local path. Sources are stacked in the given order, so one can easily overwrite or append data by simply adding a local path as the last source. + +### Command flags + +- **name**: Name of the ISO image. It will be used to generate the `*.iso` file name +- **output**: Path of the destination folder of created images +- **date**: If present it includes the date in the generated file name +- **overlay-rootfs**: Sets the path of a tree to overlay on top of the system root-tree +- **overlay-uefi**: Sets the path of a tree to overaly on top of the EFI image root-tree +- **overlay-iso**: Sets the path of a tree to overlay on top of the ISO filesystem root-tree +- **label**: Sets the volume label of the ISO filesystem + +## Configuration reference + +### `iso.rootfs` + +A list of sources in uri format (container image or local path) [ "docker", "oci", "dir", "file" ] to install in the rootfs. The rootfs will be squashed to a `rootfs.squashfs` file + +### `iso.uefi` + +A list of sources in uri format (container image or local path) [ "docker", "oci", "dir", "file" ] to install in the efi FAT image or partition. + +### `iso.image` + +A list of sources in uri format (container image or local path) [ "docker", "oci", "dir", "file" ] to install in ISO filesystem. + +### `iso.label` + +The label of the ISO filesystem. Defaults to `COS_LIVE`. Note this value is tied with the bootloader and kernel parameters to identify the root device. + +### `name` + +A string representing the ISO final image name without including the `.iso` + +### `date` + +Boolean indicating if the output image name has to contain the date + +### `output` + +Folder destination of the built artifacts. It attempts to create if it doesn't exist. + +## Customize bootloader with GRUB + +Boot menu and other bootloader parameters can then be easily customized by using the overlay parameters within the ISO config yaml manifest. + +Assuming the ISO being built includes: + +```yaml +iso: + rootfs: + - ... + uefi: + - oci:example-grub2-efi-image:latest + image: + - oci:example-grub2:latest + - oci:example-grub2-efi-image:latest +``` + +We can customize either the `image` packages (in the referrence image `live/grub2` package +includes bootloader configuration) or make use of the overlay concept to include or +overwrite addition files for `image` section. + +Consider the following example: + +```yaml +iso: + rootfs: + - ... + uefi: + - oci:example-grub2-efi-image:latest + image: + - oci:example-grub2:latest + - oci:example-grub2-efi-image:latest + - dir:/my/path/to/overlay/iso +``` + +With the above the ISO will also include the files under `/my/path/to/overlay/iso` path. To customize the boot +menu parameters consider copy and modify relevant files from `example-grub2:latest` image. In this example the +`overlay` folder files list could be: + +```bash +# image files for grub2 boot +boot/grub2/grub.cfg +``` + +Being `boot/grub2/grub.cfg` a custom grub2 configuration including custom boot menu entries. Consider the following `grub.cfg` example: + +``` +search --file --set=root /boot/kernel.xz +set default=0 +set timeout=10 +set timeout_style=menu +set linux=linux +set initrd=initrd +if [ "${grub_cpu}" = "x86_64" -o "${grub_cpu}" = "i386" ];then + if [ "${grub_platform}" = "efi" ]; then + set linux=linuxefi + set initrd=initrdefi + fi +fi + +set font=($root)/boot/x86_64/loader/grub2/fonts/unicode.pf2 +if [ -f ${font} ];then + loadfont ${font} +fi + +menuentry "Custom grub2 menu entry" --class os --unrestricted { + echo Loading kernel... + $linux ($root)/boot/kernel.xz cdroot root=live:CDLABEL=COS_LIVE rd.live.dir=/ rd.live.squashimg=rootfs.squashfs console=tty1 console=ttyS0 rd.cos.disable + echo Loading initrd... + $initrd ($root)/boot/rootfs.xz +} +``` + +## Separate recovery + +To make an ISO with a separate recovery image as squashfs, you can either use the default from `Elemental`, by adding it in the iso yaml file: + +```yaml +iso: + rootfs: + .. + uefi: + .. + image: + ... + - oci:example-recovery:latest +``` + +The installer will detect the squashfs file in the iso, and will use it when installing the system. You can customize the recovery image as well by providing your own. + diff --git a/versioned_docs/version-1.2/toolkit/creating-derivatives/creating_bootable_images.md b/versioned_docs/version-1.2/toolkit/creating-derivatives/creating_bootable_images.md new file mode 100644 index 000000000..198cf6ddf --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/creating-derivatives/creating_bootable_images.md @@ -0,0 +1,56 @@ +--- +title: "Creating bootable images" +sidebar_label: "Creating bootable images" +--- + +![](https://docs.google.com/drawings/d/e/2PACX-1vSmIZ5FTInGjtkGonUOgwhti6DZnSoeexGmWL9CAmbdiIGtBGnzDuGNj80Lj_206hP0MOxQGpEdYFvK/pub?w=1223&h=691) + +A derivative is a simple container image which can be processed by the Elemental toolkit in order to be bootable and installable. This section describes the requirements to create a container image that can be run by `Elemental`. + +## Requirements +{{}} + +Bootable images are standard container images, that means the usual `build` and `push` workflow applies, and building images is also a way to persist [oem customizations](../customizing/oem_configuration). + +The base image can be any Linux distribution that is compatible with our flavors. + +The image needs to ship: +- parts of the elemental-toolkit (required, see below) +- kernel (required) +- initrd (required) +- grub2 (required) +- dracut (required) +- microcode (optional, not required in order to boot, but recomended) + +## Example + +An illustrative example can be: + + + + +In the example above, the elemental-toolkit parts that are **required** are pulled in by `COPY --from=TOOLKIT /install-root /`. + +## Initrd +The image should provide at least `grub`, `systemd`, `dracut`, a kernel and an initrd. Those are the common set of packages between derivatives. See also [package stack](package_stack). +By default the initrd is expected to be symlinked to `/boot/initrd` and the kernel to `/boot/vmlinuz`, otherwise you can specify a custom path while [building an iso](build_iso) and [by customizing grub](../customizing/configure_grub). + +## Building + +![](https://docs.google.com/drawings/d/e/2PACX-1vS6eRyjnjdQI7OBO0laYD6vJ2rftosmh5eAog6vk_BVj8QYGGvnZoB0K8C6Qdu7SDz7p2VTxejcZsF6/pub?w=956&h=339) + +The workflow would be then: + +1) `docker build` the image +2) `docker push` the image to some registry +3) `elemental upgrade --docker-image $IMAGE` from a Elemental machine or (`elemental reset` if bootstrapping a cloud image) + +The following can be incorporated in any standard gitops workflow. + +You can explore more examples in the [example section](../examples/creating_bootable_images) on how to create bootable images. + +## What's next? + +Now that we have created our derivative container, we can either: + +- [Build an iso](build_iso) diff --git a/versioned_docs/version-1.2/toolkit/creating-derivatives/package_stack.md b/versioned_docs/version-1.2/toolkit/creating-derivatives/package_stack.md new file mode 100644 index 000000000..8a1159387 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/creating-derivatives/package_stack.md @@ -0,0 +1,11 @@ +--- +title: "Package stack" +sidebar_label: "Package stack" +--- + + +When building a `elemental-toolkit` derivative, a common set of packages are required with a common default configuration. Some of the most notably are: + +- systemd as init system +- grub for boot loader +- dracut for initramfs diff --git a/versioned_docs/version-1.2/toolkit/customizing/_index.md b/versioned_docs/version-1.2/toolkit/customizing/_index.md new file mode 100644 index 000000000..9c496bcae --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/customizing/_index.md @@ -0,0 +1,9 @@ + +--- +title: "Customizing" +sidebar_label: "Customizing" +weight: 3 +date: 2017-01-05 +description: > + This section contains various articles relative on how to customize Elemental, branding and behavior. +--- diff --git a/versioned_docs/version-1.2/toolkit/customizing/configuration_persistency.md b/versioned_docs/version-1.2/toolkit/customizing/configuration_persistency.md new file mode 100644 index 000000000..14775821c --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/customizing/configuration_persistency.md @@ -0,0 +1,45 @@ +--- +title: "Configuration persistency" +sidebar_label: "Configuration persistency" +--- + + +By default Elemental and derivatives are reading and executing cloud-init files in (lexicopgrahic) sequence inside: + +- `/system/oem` +- `/usr/local/cloud-config` +- `/oem` + +It is also possible to run cloud-init file in a different location (URLs included, too) from boot cmdline by using the `cos.setup=..` option. + +{{% alert title="Note" %}} +It is possible to install a custom [cloud-init style file](../reference/cloud_init/) during install with `--cloud-init` flag on `elemental install` command or, it's possible to add one or more files manually inside the `/oem` directory after installation. +{{% /alert %}} + +While `/system/oem` is reserved for system configurations to be included directly in the derivative container image, the `/oem` folder instead is reserved for persistent cloud-init files that can be extended in runtime. + +For example, if you want to change `/etc/issue` of the system persistently, you can create `/usr/local/cloud-config/90_after_install.yaml` or alternatively in `/oem/90_after_install.yaml` with the following content: + +```yaml +# The following is executed before fs is setted up: +stages: + fs: + - name: "After install" + files: + - path: /etc/issue + content: | + Welcome, have fun! + permissions: 0644 + owner: 0 + group: 0 + - name: "After install (second step)" + files: + - path: /etc/motd + content: | + Welcome, have more fun! + permissions: 0644 + owner: 0 + group: 0 +``` + +For more examples you can find `/system/oem` inside Elemental vanilla images containing files used to configure on boot a pristine `Elemental`. diff --git a/versioned_docs/version-1.2/toolkit/customizing/configure_grub.md b/versioned_docs/version-1.2/toolkit/customizing/configure_grub.md new file mode 100644 index 000000000..db2b33153 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/customizing/configure_grub.md @@ -0,0 +1,191 @@ +--- +title: "GRUB" +sidebar_label: "GRUB" +--- + +Elemental is set to deploy a persistent `grub.cfg` into the `COS_STATE` partition during +the system installation or image creation. Elemental grub configuration +includes three menu entries: first for the main OS system, second for the +fallback OS system and a third for the recovery OS. + +For example the main OS system menu entry could be something like: + +``` +menuentry "Elemental" --id elemental { + search.fs_label COS_STATE root + set img=/cOS/active.img + set label=COS_ACTIVE + loopback loop0 /$img + set root=($root) + source (loop0)/etc/cos/bootargs.cfg + linux (loop0)$kernel $kernelcmd + initrd (loop0)$initramfs +} +``` + +{{% alert title="Kernel parameters" %}} +The kernel parameters are not part of the persistent `grub.cfg` file stored in +`COS_STATE` partition. Kernel parameters are sourced from the loop device of +the OS image to boot. This is mainly to keep kernel parameters consistent +across different potential OS images or system upgrades. +{{% /alert %}} + +## Specifying default custom boot options + +Elemental images and its derivatives, are expected to include a +`/etc/cos/bootargs.cfg` file which provides the definition of the following +variables: + +* `$kernel`: Path of the kernel binary +* `$kernelcmd`: Kernel parameters +* `$initramfs`: Path of the initrd binary + +This is the mechanism any Elemental image or Elemental derivative has to communicate +its boot parameters (kernel, kernel params and initrd file) to GRUB2. + +For example, the default Elemental bootarg.cfg file is: + +``` +set kernel=/boot/vmlinuz +if [ -n "$recoverylabel" ]; then + # Boot arguments when the image is used as recovery + set kernelcmd="console=tty1 root=live:CDLABEL=$recoverylabel rd.live.dir=/ rd.live.squashimg=$img panic=5" +else + # Boot arguments when the image is used as active/passive + set kernelcmd="console=tty1 root=LABEL=$label iso-scan/filename=$img panic=5 security=selinux rd.cos.oemlabel=COS_OEM selinux=1" +fi + +set initramfs=/boot/initrd +``` + +You can tweak that file to suit your needs if you need to specify persistent boot arguments. + +{{% alert title="Note" %}} +`rd.cos.oemlabel=COS_OEM` is required inside the bootargs in order to access to +the `/oem` mount within the rootfs stage. `COS_OEM` is the default label for +the oem partition. +{{% /alert %}} + +## Grub environment variables + +Elemental (since v0.5.8) makes use of the GRUB2 environment block which can used to define +persistent GRUB2 variables across reboots. + +Use `grub2-editenv` command line utility to define the desired values. + +| Variable | Description | +|------------------------|---------------------------------------------------------| +| next_entry | Set the next reboot entry | +| saved_entry | Set the default boot entry | +| default_menu_entry | Set the name entries on the GRUB menu | +| extra_active_cmdline | Set additional boot commands when booting into active | +| extra_passive_cmdline | Set additional boot commands when booting into passive | +| extra_recovery_cmdline | Set additional boot commands when booting into recovery | +| extra_cmdline | Set additional boot commands for all entries | +| default_fallback | Sets default fallback logic | + + +For instance use the following command to reboot to recovery system only once: + +```bash +> grub2-editenv /oem/grubenv set next_entry=recovery +``` + +{{% alert title="Note" %}} +The examples below make use of the `COS_STATE` device, only files in the state +and oem partitions will be used when booting. +{{% /alert %}} + +### Default boot entry + +The default grub configuration loads the `/grubenv` of the COS_OEM partition +and evaluates on `next_entry` variable and `saved_entry` variable. By default +none is set. + +The default boot entry is set to the value of `saved_entry`, in case the variable +is not set grub just defaults to the first menu entry. + +`next_entry` variable can be used to overwrite the default boot entry for a single +boot. If `next_entry` variable is set this is only being used once, GRUB2 will +unset it after reading it for the first time. This is helpful to define the menu entry +to reboot to without having to make any permanent config change. + +Use `grub2-editenv` command line utility to define desired values. + +For instance use the following command to reboot to recovery system only once: + +```bash +> grub2-editenv /oem/grubenv set next_entry=recovery +``` + +Or to set the default entry to `fallback` system: + +```bash +> grub2-editenv /oem/grubenv set saved_entry=fallback +``` + +## Boot menu + +By default `Elemental` and derivatives shows the default boot menu entry while booting (`Elemental`). + +The grub menu entry is generated during installation and can be configured by setting `GRUB_ENTRY_NAME` in the `/etc/os-release` file inside the derivative, or either via the [general configuration](../customizing/general_configuration) to specify installation details. + +For example, specifying in `/etc/elemental/config.yaml`: + +```bash +install: + + ... + + grub-entry-name: myOS + + ... +``` + +will automatically set the GRUB menu entries for active, passive and recovery to the specified value. + +The grub menu boot entry can also be set with `grub2-editenv`: + +```bash +> grub2-editenv /oem/grubenv set default_menu_entry=fooOS +``` + + + +Since `{{}}` >= 0.0.3-16 it is possible to add multiple custom menu entries to GRUB by creating a `/grubcustom` config file in the state partition during boot. +The `grubcustom` file will be sourced at the end of the boot process, and can contain several `menuentry` blocks. + +{{% /alert %}} + +## Persistent boot option flags + +It is possible to define persistent boot flag for each menu entry also via `grub2-editenv`: + +- `extra_active_cmdline`: extra bootflags to be applied only on active boot +- `extra_passive_cmdline`: extra bootflags to be applied only on passive boot +- `extra_recovery_cmdline`: extra bootflags to be applied only on recovery +- `extra_cmdline`: will be applied to each boot entry + +## Renaming partition labels + +During boot the GRUB2 configuration is set to load the `grub_oem_env` file from the state partition. In this file the following variables are set in order to find system partitions: + +- `state_label`: Label of state partition. +- `active_label`: Filesystem label of active image. +- `passive_label`: Filesystem label of passive image. +- `recovery_label`: Label of recovery partition. +- `system_label`: Filesystem label of recovery image. +- `oem_label`: Label of OEM partition. +- `persistent_label`: Label of persistent partition. + +## Customizing fallback logic + +By default Elemental boots into active, and if there are failures will boot into the passive, and finally if keeps failing, will boot into recovery. + +It is possible to override the default fallback logic by setting `default_fallback` as grub environment, consider for example: + +```bash +> grub2-editenv /oem/grubenv set default_fallback="2 0 1" +``` + +Will set the default fallback to "2 0 1" instead of the default "0 1 2". diff --git a/versioned_docs/version-1.2/toolkit/customizing/embedded_features.md b/versioned_docs/version-1.2/toolkit/customizing/embedded_features.md new file mode 100644 index 000000000..13cfb0bd2 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/customizing/embedded_features.md @@ -0,0 +1,28 @@ +--- +title: "Embedded configuration" +sidebar_label: "Embedded configuration" +--- + +Elemental-toolkit provides some default configuration files for the following components: + +- GRUB2 +- Dracut +- Cloud init files +- Boot assessment + +These configuration files can be installed into a Derivative using the `elemental init`-command + +The `init`-command should be used inside the Dockerfile as in the following example: + + + +The current features available for the `init`-command is: + +- immutable-rootfs: dracut configuration for mounting the immutable root filesystem. +- grub-config: grub configuration for booting the derivative. +- elemental-setup: services used for booting the system and running cloud-init files at boot/install/upgrade. +- dracut-config: default dracut configuration for generating an initrd. +- cloud-config-defaults: optional default settings for a derivative. +- cloud-config-essentials: essential cloud-init files. + + diff --git a/versioned_docs/version-1.2/toolkit/customizing/general_configuration.md b/versioned_docs/version-1.2/toolkit/customizing/general_configuration.md new file mode 100644 index 000000000..95bc5232e --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/customizing/general_configuration.md @@ -0,0 +1,33 @@ +--- +title: "General Configuration" +sidebar_label: "General Configuration" +--- + + +Elemental during installation, reset and upgrade (`elemental install`, `elemental reset` and `elemental upgrade` respectively) will read a configuration file in order to apply derivative customizations. The configuration files are sourced in precedence order and can be located in the following places: + +- `/etc/os-release` +- `/config.yaml` +- `/config.d/*.yaml` + +By default `` is set to `/etc/elemental` however this can be changed to any custom path by using the `--config-dir` runtime flag. + +Below you can find an example of the config file including most of the available options: + + + + +The `system` and `recovery-system` objects are an image specification. An image specification is defined by: + +- **fs**: defines the filesystem of the image. Currently only `ext2` and `squashfs` should be used for images and `squashfs` is only supported for the `recovery-system` image. +- **label**: defines the filesystem label. It is strongly recommended to use default labels as it is easy to fall into inconsistent states when changing labels as all changes should also be reflected in several other parts such as the bootloader configuration. This attribute has no effect for `squashfs` filesystems. +- **uri**: defines the source of the image. The uri must include a valid scheme to identify the type of source. It supports `oci`, `dir` and `file` schemes. +- **size**: defines the filesystem image size in MiB, it must be big enough to store the defined image source. This attribute has no effect for `squashfs` filesystems. + + +The `partitions` object lists partition specifications. A partition specifications is defined by: + +- **fs**: defines the filesystem of the partition. Currently only `ext2`, `ext4` and `xfs` are supported being `ext4` the default. +- **label**: defines the label of the filesystem of the partition. It is strongly recommended to use default labels as it is easy to fall into inconsistent states when changing labels as all changes should also be reflected in several other parts such as the bootloader configuration. +- **size**: defines the partition size in MiB. A zero size means use all available disk, obviously this only makes sense for the last partition, the `persistent` partition. +- **flags**: is a list of strings, this is used as additional partition flags that are passed to `parted` (e.g. `boot` flag). Defaults should be just fine for most of the cases. diff --git a/versioned_docs/version-1.2/toolkit/customizing/login.md b/versioned_docs/version-1.2/toolkit/customizing/login.md new file mode 100644 index 000000000..7b0767be1 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/customizing/login.md @@ -0,0 +1,11 @@ +--- +title: "Login" +sidebar_label: "Login" +--- + +By default you can login with the user `root` and `cos` in a vanilla Elemental image, this is also set automatically by the `cloud-config-defaults` feature if used by a derivative. + +You can change this by overriding `/system/oem/04_accounting.yaml` in the container image if present, or via [cloud-init](../reference/cloud_init/#stagesstage_idstep_nameusers). + +## Examples +- [Example accounting file](https://github.com/rancher/elemental/blob/main/framework/files/system/oem/04_accounting.yaml) diff --git a/versioned_docs/version-1.2/toolkit/customizing/oem_configuration.md b/versioned_docs/version-1.2/toolkit/customizing/oem_configuration.md new file mode 100644 index 000000000..90501c43c --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/customizing/oem_configuration.md @@ -0,0 +1,31 @@ +--- +title: "OEM configuration" +sidebar_label: "OEM configuration" +--- + +There are several way to customize Elemental and a elemental-toolkit derivative: + +- declaratively in runtime with cloud-config file (by overriding, or extending) +- stateful, embedding any configuration in the container image to be booted. + +For runtime persistence configuration, the only supported way is with cloud-config files, [see the relevant docs](configuration_persistency). + +A derivative automatically loads and executes cloud-config files during the various system [stages](stages) also inside `/system/oem` which is read-only and reserved to the system. + +Derivatives that wish to override default configurations can do that by placing extra cloud-init file, or overriding completely `/system/oem` in the target image. + +This is to setup for example, the default root password or the preferred upgrade channel. + +The following are the `Elemental` default oem files, which are shipped within the `cloud-config-defaults` and `cloud-config-essentials` features: + +``` +/system/oem/00_rootfs.yaml - defines the rootfs mountpoint layout setting +/system/oem/01_defaults.yaml - systemd defaults (keyboard layout, timezone) +/system/oem/02_upgrades.yaml - Settings for Elemental vanilla channel upgrades +/system/oem/03_branding.yaml - Branding setting, Derivative name, /etc/issue content +/system/oem/04_accounting.yaml - Default user/pass +/system/oem/05_network.yaml - Default network setup +/system/oem/06_recovery.yaml - Executes additional commands when booting in recovery mode +``` + +You can either override the above files, or alternatively not consume the above features while building a derivative. diff --git a/versioned_docs/version-1.2/toolkit/customizing/runtime_persistent_changes.md b/versioned_docs/version-1.2/toolkit/customizing/runtime_persistent_changes.md new file mode 100644 index 000000000..ffeba2c28 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/customizing/runtime_persistent_changes.md @@ -0,0 +1,65 @@ +--- +title: "Runtime persistent changes" +sidebar_label: "Runtime persistent changes" +--- + +Elemental and derivatives are [immutable](../reference/immutable_rootfs) systems. That means that any change in the running OS will not persist after a reboot. + +While [configurations can be persisted](configuration_persistency), there are occasions where installing a custom package or provide additional persistent files in the end system is needed. + +We will see here a way to install packages, drivers, or apply any modification we might want to do in the OS image during runtime, without any need to rebuild the derivative container image. This will let any user (and not derivative developer) to apply any needed customization and to be able to persist across upgrades. + +## Transient changes + +To apply transient changes, it's possible to boot a Elemental derivative in read/write mode by specifying `rd.cos.debugrw` [see here for more details](../reference/immutable_rootfs). This allows to do any change and will persist into the active/passive booting system (does NOT apply for recovery). Altough this methodology should be only considered for debugging purposes. + +## Persist changes with Cloud init files + +Elemental allows to apply a set of commands, or cloud-init steps, during upgrade, deploy, install and reset in the context of the target image, in RW capabilities. This allows to carry on changes during upgrades on the target image without the need to re-build or have a custom derivative image. + +All the configuration that we want to apply to the system will run each time we do an upgrade, a reset or an installation on top of the new downloaded image (in case of upgrade) or the image which is the target system. + +Between the available [stages](stages) in the [cloud-init](../reference/cloud_init/) there are `after-upgrade-chroot`, `after-install-chroot`, `after-reset-chroot` and `after-deploy-chroot`, for example, consider the following cloud-init file: + +```yaml +stages: +name: "Install something" +stages: + after-upgrade-chroot: + - commands: + - zypper in -y ... + after-reset-chroot: + - commands: + - zypper in -y ... + after-deploy-chroot: + - commands: + - zypper in -y ... + after-install-chroot: + - commands: + - zypper in -y ... +``` + +It will run the `zypper in -y ...` calls during each stage, in the context of the target system, allowing to customize the target image with additional packages. + +{{% alert title="Note" %}} +`zypper` calls here are just an example. We could have used `dnf` for fedora based, or `apt-get` for ubuntu based images. +{{% /alert %}} + +When running the cloud-init steps the `/oem` partition and `/usr/local` will be mounted to `COS_OEM` and `COS_PERSISTENT` respectively, allowing to load extra data (e.g. rpm files, or configuration). + +## Example + +If an user wants to install an additional package in the running system, and keep having that persistent across upgrades, he can copy the following file (`install.yaml`) inside the `/oem` folder, or `/usr/local/cloud-config`: + +```yaml +stages: +name: "Install something" +stages: + after-upgrade-chroot: + - commands: + - zypper in -y vim +``` + +and run `elemental upgrade`. + +It will automatically upgrade the system with the changes above included. diff --git a/versioned_docs/version-1.2/toolkit/customizing/selinux_support.md b/versioned_docs/version-1.2/toolkit/customizing/selinux_support.md new file mode 100644 index 000000000..87b2c48bb --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/customizing/selinux_support.md @@ -0,0 +1,65 @@ +--- +title: "SELinux Support" +sidebar_label: "SELinux Support" +--- + +Elemental includes basic support for SELinux. From an elemental perspective SELinux is some custom configurationt that requires special treatment. Being specific it mostly nails down to apply SELinux labels at install and upgrade time. Since the rootfs is readonly they can't be easily applied at runtime or at boot time. As consequence of that SELinux autorelabel service should not be used within elemental as it expects a RW root and persistency across reboots (it essentially reboots after appliying labels). For the time being Elemental only considers the `targeted` SELinux policy. + +`elemental-cli` utility applies SELinux contexts to the installed/upgraded system if three conditions are met: + +* the installed system includes the `setfiles` command +* the installed system includes the targeted files context (`/etc/selinux/targeted/contexts/files/file_contexts` file) +* the binary for `targeted` policy is also present (`/etc/selinux/targeted/policy/policy.*` file) + +In an Elemental workflow SElinux context labels should be applied at install/upgrade time for the readonly areas, but this is not enough as it doesn't cover the ephemeral filesystems (overlayfs on top of tmpfs), which are usually sensitive paths like `/etc/`, `/var`, `/srv`, etc. In order to properly apply file contexts over the ephemeral paths the relabelling has to happen at boot time once those overlayfs are created. The appropriate stage for that is in initrd before switching root. In fact, it can be done as a cloud-init step as part of the `initramfs` stage, using the packaged `10_selinux.yaml` with: + + + +Note it is required to load the policy in advance to be capable to apply the `restorecon` command. The `restorecon` command should be applied to all ephemeral paths and, depending on the specific use case, to the persistent paths too. Note that without restoring context on the ephemeral `/etc` it is unlikely the system is capable of properly booting, hence this is a very important step if SELinux is intended to used. + +## Using custom SELinux modules + +Making use of `selinux` and including SELinux utilities and targeted policy within the base OS it is enough to get started with SELinux, however there is a great chance that this is too generic and requires some additional policy modules to be fully functional according to each specific use case. + +The Type Enforcement file was created by booting an Elemental OS on permissive mode using `audit2allow` and other SELinux related utilities to generate the custom module out of the reported denials. Something like: + +```bash +# Create the type enforcement file +cat /var/log/audit/audit.log | audit2allow -m elemental > elemental.te + +# Create the policy module +checkmodule -M -m -o elemental.mod elemental.te + +# Create the policy package out of the module +semodule_package -o elemental.pp -m elemental.mod +``` + +To make effective the policy package it has to be loaded or installed within the selinux policy, this can be easily done with the `semodule -i /usr/share/elemental/selinux/elemental.pp` command. So from a derivative perspective and following the example from [Creating bootable image](../creating-derivatives/creating_bootable_images/#example) section adding the following lines to the Dockerfile should be enough to enable SELinux in enforcing mode: + +```Dockerfile +# Install the custom policy package if any and the restore context stage in cloud-init config +RUN elemental init --force --features=cloud-config + +# Load the policy package +RUN semodule -i /usr/share/elemental/selinux/elemental.pp + +# Enable selinux in enforcing mode +RUN sed -i "s|^SELINUX=.*|SELINUX=enforcing|g" /etc/selinux/config +``` + +The above assumes the base image already includes the SELinux packages and utilities provided by the underlaying distro. It is suggested to set the enforcing mode via the config file rather than setting grub with the selinux kernel parameter (`enforcing=1`), this way it is easier, at any time, to temporarily add `enforcing=0` at runtime within the grub2 shell and temporarily set SELinux in permissive mode. + +Notes when using a SELinux version prior to v3.4. If `libsemanage` version is lower than v3.4, it is likely that the `semodule -i *.pp` command fails with a cross-device linking issue, this is a known [issue](https://github.com/SELinuxProject/selinux/issues/343) upstream and already fixed since v3.4. Command `selinux -i ` mutates files under `/var/lib/selinux/targeted` and used to rename some files, this can be tricky when executed inside a container as hardlinks across filesystems are not permitted and this is actually what happens if the overlayfs driver is used. This can be worked around if all the originally mutated files are already modified within the execution layer (so they are part of the upper layer of the overlayfs). So the above specific example could be rewritten as: + +```Dockerfile +# Install the custom policy package if any and the restore context stage in cloud-init config +RUN elemental init --force --features=cloud-config + +# Artificially modify selinux files to copy them in within the overlyfs and then load the policy package +RUN mv /var/lib/selinux/targeted/active /var/lib/selinux/targeted/previous &&\ + cp --link --recursive /var/lib/selinux/targeted/previous /var/lib/selinux/targeted/active &&\ + semodule -i /usr/share/elemental/selinux/elemental.pp + +# Enable selinux in enforcing mode +RUN sed -i "s|^SELINUX=.*|SELINUX=enforcing|g" /etc/selinux/config +``` diff --git a/versioned_docs/version-1.2/toolkit/customizing/stages.md b/versioned_docs/version-1.2/toolkit/customizing/stages.md new file mode 100644 index 000000000..f9286c133 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/customizing/stages.md @@ -0,0 +1,383 @@ +--- +title: "Stages" +sidebar_label: "Stages" +--- + +We have a custom augmented cloud-init syntax that allows to hook into various stages of the system, for example: +- Initramfs load +- Boot +- Network availability +- During upgrades, installation, deployments , and resets + +Cloud-init files in `/system/oem`, `/oem` and `/usr/local/oem` are applied in 5 different phases: `boot`, `network`, `fs`, `initramfs` and `reconcile`. All the available cloud-init keywords can be used in each stage. Additionally, it's possible also to hook before or after a stage has run, each one has a specific stage which is possible to run steps: `boot.after`, `network.before`, `fs.after` etc. + +Multiple stages can be specified in a single cloud-init file. + +{{% alert title="Note" %}} +When a Elemental derivative boots it creates sentinel files in order to allow to execute cloud-init steps programmaticaly. + +- `/run/cos/recovery_mode` is being created when booting from the recovery partition +- `/run/cos/live_mode` is created when booting from the LiveCD + +To execute a block using the sentinel files you can specify: `if: '[ -f "/run/cos/..." ]'`, see the examples below. +{{% /alert %}} + +## Stages + +Below there is a detailed list of the stages available that can be used in the cloud-init configuration files + +### `rootfs` + +This is the earliest stage, running before switching root, just right after the +root is mounted in `/sysroot` and before applying the immutable rootfs configuration. +This stage is executed over initrd root, no chroot is applied. + +Example: +```yaml +name: "Set persistent devices" +stage: + rootfs: + - name: "Layout configuration" + environment_file: /run/cos/cos-layout.env + environment: + VOLUMES: "LABEL=COS_OEM:/oem LABEL=COS_PERSISTENT:/usr/local" + OVERLAY: "tmpfs:25%" +``` + +### `initramfs` + +This is still an early stage, running before switching root. Here you can apply radical changes to the booting setup of `Elemental`. +Despite this is executed before switching root this exection runs chrooted into the target root after the immutable rootfs is set up and ready. + +Example: +```yaml +name: "Run something on initramfs" +stages: + initramfs: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + touch /etc/something_important + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + +### `boot` + +This stage is executed after initramfs has switched root, during the `systemd` bootup process. + +Example: +```yaml +name: "Run something on boot" +stages: + boot: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + +### `fs` + +This stage is executed when fs is mounted and is guaranteed to have access to `COS_STATE` and `COS_PERSISTENT`. + +Example: +```yaml +name: "Run something on boot" +stages: + fs: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + touch /usr/local/something + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + + +### `network` + +This stage is executed when network is available + +Example: +```yaml +name: "Run something on boot" +stages: + network: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Network is available, do something.. +``` + +### `reconcile` + +This stage is executed `5m` after boot and periodically each `60m`. + +Example: +```yaml +name: "Run something on boot" +stages: + reconcile: + - name: "Setting" + if: '[ ! -f "/run/sentinel" ]' + commands: + - | + touch /run/sentinel +``` + +### `post-install` + +This stage is executed after installation of the OS has ended (last step of `elemental install`). + +Example: +```yaml +name: "Run something after installation" +stages: + post-install: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + +### `after-install-chroot` + +This stage is executed after installation of the OS filesystem image has completed. +{{% alert title="Note" %}} +Steps executed at this stage are running *inside* the new OS as chroot, allowing to write persisting changes to the image, +for example by installing additional software. +{{% /alert %}} + +Example: +```yaml +name: "Run something after installation" +stages: + after-install-chroot: + - name: "Setting" + commands: + - | + ... +``` + +### `after-install` + +This stage is executed after installation of the OS filesystem image has completed and just after the chroot hook. +{{% alert title="Note" %}} +Steps executed at this stage are running when the new image and all the relevant partitions are still mounted in rw mode, allowing to write persisting changes to the image, +for example installing additional software. +{{% /alert %}} + +Example: +```yaml +name: "Run something after installation" +stages: + after-install: + - name: "Setting" + commands: + - | + ... +``` + + +### `post-upgrade` + +This stage is executed after upgrade of the OS has ended (last step of `elemental upgrade`). + +Example: +```yaml +name: "Run something after upgrade" +stages: + post-upgrade: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + +### `after-upgrade-chroot` + +This stage is executed after installation of the OS filesystem image has completed. +{{% alert title="Note" %}} +Steps executed at this stage are running *inside* the new OS as chroot, allowing to write persisting changes to the image, +for example by downloading and installing additional software. +{{% /alert %}} + +{{% alert title="Note" %}} +Steps executed at this stage are based on stages found within the chroot, hence any new (not present in the current host) upgrade specific +hook that requires to be executed during upgrade should be included here. Otherwise it will not be seen by elemental-cli during the upgrade. +{{% /alert %}} + +Example: +```yaml +name: "Run something after upgrade" +stages: + after-upgrade-chroot: + - name: "Setting" + commands: + - | + ... +``` + +### `after-upgrade` + +This stage is executed after installation of the OS filesystem image has completed and just after the chroot hook. + + +Example: +```yaml +name: "Run something after upgrade" +stages: + after-upgrade: + - name: "Setting" + commands: + - | + ... +``` + +### `post-reset` + +This stage is executed after reset of the OS has ended (last step of `elemental reset`). + +Example: +```yaml +name: "Run something after reset" +stages: + post-reset: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + +### `after-reset-chroot` + +This stage is executed after installation of the OS filesystem image has completed. +{{% alert title="Note" %}} +Steps executed at this stage are running *inside* the new OS as chroot, allowing to write persisting changes to the image, +for example by installing additional software. +{{% /alert %}} + +Example: +```yaml +name: "Run something after installation" +stages: + after-reset-chroot: + - name: "Setting" + commands: + - | + ... +``` + +### `after-reset` + +This stage is executed after installation of the OS filesystem image has completed and just after the chroot hook. + +Example: +```yaml +name: "Run something after installation" +stages: + after-reset: + - name: "Setting" + commands: + - | + ... +``` + +### `before-install` + +This stage is executed before installation (executed during `elemental install`). + +Example: +```yaml +name: "Run something before installation" +stages: + before-install: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + + +### `before-upgrade` + +This stage is executed before upgrade of the OS (executed during `elemental upgrade`). + +Example: +```yaml +name: "Run something before upgrade" +stages: + before-upgrade: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` + +### `before-reset` + +This stage is executed before reset of the OS (executed during `elemental reset`). + +Example: +```yaml +name: "Run something before reset" +stages: + before-reset: + - name: "Setting" + if: '[ ! -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in active or passive + - name: "Setting" + if: '[ -f "/run/cos/recovery_mode" ]' + commands: + - | + # Run something when we are booting in recovery mode +``` diff --git a/versioned_docs/version-1.2/toolkit/customizing/upgrades.md b/versioned_docs/version-1.2/toolkit/customizing/upgrades.md new file mode 100644 index 000000000..1ef5a6c9d --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/customizing/upgrades.md @@ -0,0 +1,33 @@ +--- +title: "Upgrades" +sidebar_label: "Upgrades" +--- + +`Elemental` vanilla images by default are picking upgrades by the standard upgrade channel. It means it will always get the latest published `Elemental` version by our CI. + +However, it's possible to tweak the default behavior of `elemental upgrade` to point to a specific OCI image/tag, or a different release channel. + +## Configuration + +`elemental upgrade` during start reads the [Elemental configuration file](general_configuration) and allows to tweak the following: + +```yaml +# configuration used for the 'ugrade' command +upgrade: + # if set to true upgrade command will upgrade recovery system instead + # of main active system + recovery: false + + # image used to upgrade main OS + # size in MiB + system: + uri: + + # image used to upgrade recovery OS + # recovery images can be set to use squashfs + recovery-system: + fs: squashfs + uri: oci:recovery/cos +``` + +The `system` and `recovery-system` objects define the OS image used for the main active system and the recovery system respectively. They both are fined by a ``. diff --git a/versioned_docs/version-1.2/toolkit/development/_index.md b/versioned_docs/version-1.2/toolkit/development/_index.md new file mode 100644 index 000000000..db08a1c55 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/development/_index.md @@ -0,0 +1,103 @@ + +--- +title: "Development" +sidebar_label: "Development" +weight: 8 +date: 2023-05-31 +description: > + How to build Elemental? +--- + +Welcome! + +The Elemental (containerized OS) distribution is entirely built over GitHub. You can check the pipelines in the `.github` folder to see how the process looks like. + +## Forking and test on your own + +By forking the `Elemental-toolkit` repository, you already have the Github Action workflow configured to start building and pushing your own `Elemental` fork. + +## Building locally + +The elemental-cli can be built locally using go: + +From your git folder: + +```bash +$> make build-cli +$> build/elemental version +v0.2.5+g4d5d1be +``` + +### Build an example locally + +Building locally has a [set of dependencies](dependencies.md) that +should be satisfied. + +Then you can run +``` +# make build-os +``` + +### Build ISO + +If using SLES or openSUSE, first install the required deps: + +``` +# zypper in -y squashfs xorriso dosfstools +``` + +and then, simply run + +``` +# make build-iso +``` + +### Run with qemu + +After you have the iso locally, run + +``` + +$> make prepare-installer-test + +``` + +This will create a disk image and boot from the ISO. + +> +> If the image already exists, it will NOT be overwritten. +> +> You need to run an explicit `make test-clean` to wipe the image and +> start over. +> + +#### Installing + +After booting from the ISO you can log in as `root` with password `cos` using ssh `ssh root@localhost:2222` and install Elemental on +the disk image with: + +``` +# elemental install /dev/sda +``` + +### Run tests + +Requires: ginkgo, qemu + +We have a test suite which runs over SSH. + +To create the disk image: + +``` + +$> make build-disk + +``` + +To run the tests: + +``` + +$> make test-smoke + +``` diff --git a/versioned_docs/version-1.2/toolkit/development/creating_derivatives.md b/versioned_docs/version-1.2/toolkit/development/creating_derivatives.md new file mode 100644 index 000000000..d8f801e09 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/development/creating_derivatives.md @@ -0,0 +1,41 @@ +--- +title: "Creating derivatives" +sidebar_label: "Creating derivatives" +--- + +`elemental-toolkit` is a manifest to share a common abstract layer between derivatives inheriting the same featureset. + +## High level workflow + +The building workflow can be resumed in the following steps: + +- Build a container image (`docker build` / `nerdctl build` / `buildah` ..) +- Publish the image (`docker push` / `nerdctl push` ) +- Build an ISO (`elemental build-iso`) +- Boot a machine using the ISO and run installation (`elemental install`) +- Reboot into the installed system + +While on the client side, the upgrade workflow is: +- `elemental upgrade --system.uri=oci:` + +## Single image OS + +Derivatives are composed by a combination of specs to form a single image OS. + +The container image during installation and upgrade, is converted to an image file with a backing ext2 fs. + +## Build ISO + +To build an iso for a derivative image `elemental build-iso` command can be used: + +```bash +elemental build-iso -n $NAME $SOURCE +``` + +Where `$NAME` is the name of the ISO and `$SOURCE` might be the reference to the directory, file, container image or chaneel we are building the ISO for. `$SOURCE` should be provided as uri in following format `:`, where: + * `` - might be ["dir", "file", "oci", "docker"], as default is taken "oci" + * `` - is path to file or directory, channel or image name with tag version (if tag was not provided then "latest" is used) + +Some examples for $SOURCE argument "dir:/cOS/system", "oci:quay.io/repository/costoolkit/releases-green:cos-system-0.8.14-10" + +See also [building ISOs](../creating-derivatives/build_iso) diff --git a/versioned_docs/version-1.2/toolkit/development/dependencies.md b/versioned_docs/version-1.2/toolkit/development/dependencies.md new file mode 100644 index 000000000..34ecc6649 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/development/dependencies.md @@ -0,0 +1,38 @@ +--- +title: "Build requirements" +sidebar_label: "Build requirements" +--- + +### Installing required dependencies for local build + +To get requirements installed locally, run: + +```bash +$> make deps +``` + +or you need: + +- [`elemental-cli`](https://github.com/rancher/elemental-toolkit) +- [`squashfs-tools`](https://github.com/plougher/squashfs-tools) + - `zypper in squashfs` on SLES or openSUSE +- [`xorriso`](https://dev.lovelyhq.com/libburnia/web/wiki/Xorriso) + - `zypper in xorriso` on SLES or openSUSE +- [`mtools`](https://www.gnu.org/software/mtools/) + - `zypper in mtools` on SLES or openSUSE +- `yq` ([version `4.x`](https://github.com/mikefarah/yq/releases)), installed via [packages/toolchain/yq](https://github.com/rancher/elemental-toolkit/tree/main/packages/toolchain/yq) (optional) +- [`jq`](https://stedolan.github.io/jq), installed via [packages/utils/jq](https://github.com/rancher/elemental-toolkit/tree/main/packages/utils/jq) (optional) + +#### elemental + +`elemental` comes [with Elemental-toolkit](https://github.com/rancher/elemental-toolkit) + +You can grab the binary from [elemental](https://github.com/rancher/elemental-toolkit) releases. + + +#### yq and jq +`yq` (version `4.x`) and `jq` are used to retrieve the list of +packages to build in order to produce the final ISOs. Those are not +strictly required, see the Note below. + +_Note_: `yq` and `jq` are just used to generate the list of packages to build, and you don't need to have them installed if you manually specify the packages to be compiled. diff --git a/versioned_docs/version-1.2/toolkit/examples/_index.md b/versioned_docs/version-1.2/toolkit/examples/_index.md new file mode 100644 index 000000000..af7c7faf7 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/examples/_index.md @@ -0,0 +1,9 @@ + +--- +title: "Examples" +sidebar_label: "Examples" +weight: 5 +date: 2017-01-05 +description: > + Examples and recipes for building Elemental derivatives +--- diff --git a/versioned_docs/version-1.2/toolkit/examples/cloud_config.md b/versioned_docs/version-1.2/toolkit/examples/cloud_config.md new file mode 100644 index 000000000..12120c8fd --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/examples/cloud_config.md @@ -0,0 +1,112 @@ +--- +title: "Cloud config examples" +sidebar_label: "Cloud config examples" +--- + +You can find here examples on how to tweak a system via cloud-config various aspects of an Elemental-toolkit derivative. + +The examples are meant to be placed as yaml files under `/oem` ore either `/usr/local/cloud-config`. They can be also given as input cloud-config while calling `elemental install`. + +## Networking with NetworkManager + +By default all interfaces will get automatically an IP address, however, there are situations where a static IP is desired, or custom configuration to be specified, here you can find some network settings with NetworkManager. + +### Additional NIC + +Set static IP to an additional NIC: + +```yaml +name: "Default network configuration" +stages: + boot: + - commands: + - nmcli dev up eth1 + - name: "Setup network" + files: + - path: /etc/sysconfig/network/ifcfg-eth1 + content: | + BOOTPROTO='static' + IPADDR='192.168.1.2/24' + permissions: 0600 + owner: 0 + group: 0 +``` + +### Static IP + +Set static IP to default interface: + +```yaml +name: "Default network configuration" +stages: + boot: + - commands: + - nmcli dev up eth0 + initramfs: + - name: "Setup network" + files: + - path: /etc/sysconfig/network/ifcfg-eth0 + content: | + BOOTPROTO='static' + IPADDR='192.168.1.2/24' + permissions: 0600 + owner: 0 + group: 0 +``` + +### DHCP + +```yaml +name: "Default network configuration" +stages: + boot: + - commands: + - nmcli dev up eth1 + initramfs: + - name: "Setup network" + files: + - path: /etc/sysconfig/network/ifcfg-eth1 + content: | + BOOTPROTO='dhcp' + STARTMODE='onboot' + permissions: 0600 + owner: 0 + group: 0 +``` + +## Additional files + +### K3s manifests + +Add k3s manifests: + +```yaml +name: "k3s" +stages: + network: + - if: '[ ! -f "/run/cos/recovery_mode" ]' + name: "Fleet deployment" + files: + - path: /var/lib/rancher/k3s/server/manifests/fleet-config.yaml + content: | + apiVersion: v1 + kind: Namespace + metadata: + name: cattle-system + --- + apiVersion: helm.cattle.io/v1 + kind: HelmChart + metadata: + name: fleet-crd + namespace: cattle-system + spec: + chart: https://github.com/rancher/fleet/releases/download/v0.3.8/fleet-crd-0.3.8.tgz + --- + apiVersion: helm.cattle.io/v1 + kind: HelmChart + metadata: + name: fleet + namespace: cattle-system + spec: + chart: https://github.com/rancher/fleet/releases/download/v0.3.8/fleet-0.3.8.tgz +``` diff --git a/versioned_docs/version-1.2/toolkit/examples/creating_bootable_images.md b/versioned_docs/version-1.2/toolkit/examples/creating_bootable_images.md new file mode 100644 index 000000000..2571e1e22 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/examples/creating_bootable_images.md @@ -0,0 +1,38 @@ +--- +title: "Creating bootable images" +sidebar_label: "Creating bootable images" +--- + +You can find the examples below in the [examples](https://github.com/rancher/elemental-toolkit/tree/main/examples) folder. + +## From standard images + +Besides using the `elemental-toolkit` toolchain, it's possible to create standard container images which are consumable by the vanilla `Elemental` images (ISO, Cloud Images, etc.) during the upgrade and deploy phase. + +An example of a Dockerfile image can be: + + + +We can just run docker to build the image with + +```bash +docker build -t $IMAGE . +``` + +The important piece is that an image needs to ship at least: + +``` +grub2 +systemd +kernel +dracut +``` + +And then extract the configuration for the system using the `elemental init`-command. + +## Customizations + +All the method above imply that the image generated will be the booting one, there are however several configuration entrypoint that you should keep in mind while building the image: + +- Everything under `/system/oem` will be loaded during the various stage (boot, network, initramfs). You can check [here](https://github.com/rancher/elemental-toolkit/tree/e411d8b3f0044edffc6fafa39f3097b471ef46bc/packages/cloud-config/oem) for the `Elemental` defaults. See `00_rootfs.yaml` to customize the booting layout. +- `/etc/cos/bootargs.cfg` contains the booting options required to boot the image with GRUB diff --git a/versioned_docs/version-1.2/toolkit/examples/embedded_images.md b/versioned_docs/version-1.2/toolkit/examples/embedded_images.md new file mode 100644 index 000000000..3976495b2 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/examples/embedded_images.md @@ -0,0 +1,332 @@ +--- +title: "Creating embedded images" +sidebar_label: "Creating embedded images" +--- + +This guide will guide in a step-by-step process to build a derivative which is fully compatible with Elemental, and will illustrate how to make customization on such image, by adding for example a default set of services and a custom user. + +The derivative will be based on openSUSE and embed k3s, and a custom user `joe` which will be already set to allow us to login. + +## Prerequisites + +- Docker installed + +## 1) Create a Dockerfile + +Let's create a workspace directory and move into it: + +```bash +$> mkdir derivative +$> cd derivative +``` + +Let's create now a `Dockerfile` for our image inside that directory, which will be represent running system: + +```Dockerfile +FROM ghcr.io/rancher/elemental-toolkit/elemental-cli:v0.11.0 AS elemental + +FROM registry.suse.com/suse/sle-micro-rancher/5.2 +ARG K3S_VERSION=v1.20.4+k3s1 +ARG ARCH=amd64 +ENV ARCH=${ARCH} + +COPY --from=elemental /install-root / +COPY --from=elemental /usr/bin/elemental /usr/bin/elemental + +# Install k3s server/agent +ENV INSTALL_K3S_VERSION=${K3S_VERSION} +RUN curl -sfL https://get.k3s.io > installer.sh && \ + INSTALL_K3S_SKIP_START="true" INSTALL_K3S_SKIP_ENABLE="true" sh installer.sh && \ + INSTALL_K3S_SKIP_START="true" INSTALL_K3S_SKIP_ENABLE="true" sh installer.sh agent && \ + rm -rf installer.sh + +## System layout + +# Required by k3s etc. +RUN mkdir /usr/libexec && touch /usr/libexec/.keep + +# Copy custom files +# COPY files/ / + +# Copy cloud-init default configuration +COPY cloud-init.yaml /system/oem/ + +# Generate initrd +RUN elemental init --force + +# OS level configuration +RUN echo "VERSION=999" > /etc/os-release +RUN echo "GRUB_ENTRY_NAME=derivative" >> /etc/os-release +RUN echo "welcome to our derivative" >> /etc/issue.d/01-derivative + +# Copy cloud-init default configuration +COPY cloud-init.yaml /system/oem/ +``` + +## 2) Configuration + +At the end of the Dockerfile, you can see that we copy over a custom [cloud-init](../reference/cloud_init) file: + +```Dockerfile +# Copy cloud-init default configuration +COPY cloud-init.yaml /system/oem/ +``` + +Create a `cloud-init.yaml` file as the `derivative/cloud-init.yaml` with the following content: + +```yaml +# See https://rancher.github.io/elemental-toolkit/docs/reference/cloud_init/ for a full syntax reference +name: "Default settings" +stages: + initramfs: + # Setup default hostname + - name: "Branding" + hostname: "derivative" + # Setup an admin group with sudo access + - name: "Setup groups" + ensure_entities: + - entity: | + kind: "group" + group_name: "admin" + password: "x" + gid: 900 + # Setup network - openSUSE specific + - name: "Network setup" + files: + - path: /etc/sysconfig/network/ifcfg-eth0 + content: | + BOOTPROTO='dhcp' + STARTMODE='onboot' + permissions: 0600 + owner: 0 + group: 0 + # Setup a custom user + - name: "Setup users" + users: + # Replace the default user name here and settings + joe: + # Comment passwd for no password + passwd: "joe" + shell: /bin/bash + homedir: "/home/joe" + groups: + - "admin" + #authorized_keys: + # Replace here with your ssh keys + # joe: + # - ssh-rsa .... + # Setup sudo + - name: "Setup sudo" + files: + - path: "/etc/sudoers" + owner: 0 + group: 0 + permsisions: 0600 + content: | + Defaults always_set_home + Defaults secure_path="/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/usr/local/sbin" + Defaults env_reset + Defaults env_keep = "LANG LC_ADDRESS LC_CTYPE LC_COLLATE LC_IDENTIFICATION LC_MEASUREMENT LC_MESSAGES LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE LC_ATIME LC_ALL LANGUAGE LINGUAS XDG_SESSION_COOKIE" + Defaults !insults + root ALL=(ALL) ALL + %admin ALL=(ALL) NOPASSWD: ALL + @includedir /etc/sudoers.d + commands: + - passwd -l root + # Setup persistency so k3s works properly + # See also: https://rancher.github.io/elemental-toolkit/docs/reference/immutable_rootfs/#configuration-with-an-environment-file + rootfs.after: + - name: "Immutable Layout configuration" + environment_file: /run/cos/cos-layout.env + environment: + VOLUMES: "LABEL=COS_OEM:/oem LABEL=COS_PERSISTENT:/usr/local" + OVERLAY: "tmpfs:25%" + RW_PATHS: "/var /etc /srv" + PERSISTENT_STATE_PATHS: >- + /etc/systemd + /etc/rancher + /etc/ssh + /etc/iscsi + /etc/cni + /home + /opt + /root + /usr/libexec + /var/log + /var/lib/rancher + /var/lib/kubelet + /var/lib/NetworkManager + /var/lib/longhorn + /var/lib/cni + PERSISTENT_STATE_BIND: "true" + # Finally, let's start k3s when network is available, and download the SSH key from github for the joe user + network: + - name: "Start k3s" + systemctl: + start: + - k3s + - authorized_keys: + # Replace here with your ssh keys or github handle + joe: + - github:joe +``` + +Done! We are now ready to build the container image. + +The file structure should be like the following: + +``` +$> tree ./derivative +derivative +├── cloud-init.yaml +├── Dockerfile +├── iso.yaml +└── repositories.yaml +``` + +## 3) Build it! + +Now we are ready to build our docker image: + +```bash +$~/derivative> docker build -t derivative:latest . +... + ---> Running in a9c33b42f567 +Removing intermediate container a9c33b42f567 + ---> 8e83191d29df +Step 19/19 : COPY cloud-init.yaml /system/oem/ + ---> 38cc4c8b173a +Successfully built 38cc4c8b173a +Successfully tagged derivative:latest +``` + +After the process completed, we are ready to consume our docker image. If you push the image over a container registry, you can then or use a running `Elemental` system to upgrade to it, or deploy it directly [see getting started](../getting-started/install). + +### Build an ISO image + +We can at this point also create a ISO from it. + +Create a `manifest.yaml` file with the following content inside the `derivative` folder: + +```yaml +iso: + bootloader-in-rootfs: true + grub-entry-name: "Derivative Installer" + +squash-no-compression: true +``` + +Now, we can build the ISO with: + +```bash +$~/derivative> elemental build-iso --config-dir=./ derivative:latest +.... +INFO[0114] Copying BIOS kernels +INFO[0114] Create squashfs +Parallel mksquashfs: Using 8 processors +Creating 4.0 filesystem on /tmp/elemental-geniso4082786464/tempISO/rootfs.squashfs, block size 1048576. +.... +INFO[0247] 🍹 Generate ISO derivative-0.20210909.iso +xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project. + +Drive current: -outdev 'stdio:derivative-0.20210909.iso' +Media current: stdio file, overwriteable +Media status : is blank +Media summary: 0 sessions, 0 data blocks, 0 data, 448g free +Added to ISO image: directory '/'='/tmp/elemental-geniso4082786464/tempISO' +xorriso : UPDATE : 599 files added in 1 seconds +xorriso : UPDATE : 599 files added in 1 seconds +xorriso : NOTE : Copying to System Area: 512 bytes from file '/tmp/elemental-geniso4082786464/tempISO/boot/x86_64/loader/boot_hybrid.img' +xorriso : WARNING : Boot image load size exceeds 65535 blocks of 512 bytes. Will record 0 in El Torito to extend ESP to end-of-medium. +libisofs: NOTE : Aligned image size to cylinder size by 137 blocks +xorriso : UPDATE : 12.35% done +xorriso : UPDATE : 42.73% done +ISO image produced: 282624 sectors +Written to medium : 282624 sectors at LBA 0 +Writing to 'stdio:derivative-0.20210909.iso' completed successfully. +``` + +After the process completes, we should have a ISO in our folder ready to be used. See the [build ISOs section](../creating-derivatives/build_iso) for all the available options. + + +## Customization + +Here follows a break down of the steps above + +### Adding packages +Feel free to edit the Dockerfile with the packages you want to embed in our image. You can install any packages available in the openSUSE repositories by +tweaking + +```Dockerfile +# Install packages from the base image +RUN zypper in -y \ + .... # Add more packages here! +``` + +### System layout and k3s + +We set some default layouts and install k3s: + +``` +# Install k3s server/agent +ENV INSTALL_K3S_VERSION=${K3S_VERSION} +RUN curl -sfL https://get.k3s.io > installer.sh && \ + INSTALL_K3S_SKIP_START="true" INSTALL_K3S_SKIP_ENABLE="true" sh installer.sh && \ + INSTALL_K3S_SKIP_START="true" INSTALL_K3S_SKIP_ENABLE="true" sh installer.sh agent && \ + rm -rf installer.sh + +## System layout + +# Required by k3s etc. +RUN mkdir /usr/libexec && touch /usr/libexec/.keep + +# Copy custom files +# COPY files/ / + +# Generate initrd +RUN elemental init --force + +# OS level configuration +RUN echo "VERSION=999" > /etc/os-release +RUN echo "GRUB_ENTRY_NAME=derivative" >> /etc/os-release +RUN echo "welcome to our derivative" >> /etc/issue.d/01-derivative +``` + +As our target here is to install `k3s` we do install both k3s `agent` and `server`, so the image can work in both modes. Here we could have installed any service or binary that we want to embed in our container image. We setup the system layout by creating needed paths for `k3s` and set up a os-release which identifies the OS version. Afterward we regenerate the initrd which is required in order to boot, [see also the Initrd section](../creating-derivatives/creating_bootable_images/#initrd). + +### Cloud-init, custom SSH access + +The `cloud-init.yaml` file above configures a user, `joe` and attaches a ssh key to it in order to login. It also sets up a default password, which is optional. + +To specify any additional ssh key installed within the user, we do: + +```yaml +network: +- authorized_keys: + joe: + - github:joe +``` + +which you can replace with your github handle, or by specifying directly an ssh key. In case you specify the SSH key directly, you don't need to run the step in the `network` stage. + +The user will be part of the `admin` group which is allowed to use `sudo`. + +As our target is to run `k3s`, but could have been any other service, we tweak the immutable setup by specifying sensible path required for `k3s` in order to function properly, see [immutable rootfs](../reference/immutable_rootfs) for all the available options. + +Finally, we start k3s. Note, we could have tweaked that part slightly to provide `k3s` configurations via systemd env files, or boot up for example the agent instead of the server: + +```yaml + network: + - if: '[ ! -f "/run/cos/recovery_mode" ]' + name: "Setup k3s" + # Setup environment file for custom k3s arguments + environment_file: "/etc/systemd/system/k3s.service.env" + environment: + FOO: "bar" + systemctl: + start: + - k3s + - commands: + - | + chmod 600 /etc/systemd/system/k3s.service.env +``` diff --git a/versioned_docs/version-1.2/toolkit/getting-started/_index.md b/versioned_docs/version-1.2/toolkit/getting-started/_index.md new file mode 100644 index 000000000..4b0cf63dc --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/getting-started/_index.md @@ -0,0 +1,59 @@ +--- +title: "Getting Started" +sidebar_label: "Getting Started" +weight: 1 +description: > + Getting started with Elemental +--- + +![](https://docs.google.com/drawings/d/e/2PACX-1vRSuocC4_2rHeJAWW2vqinw_EZeZxTzJFo5ZwnJaL_sdKab_R_OsCTLT_LFh1_L5fUcA_2i9FIe-k69/pub?w=1223&h=691) + + +Elemental toolkit provides a runtime and buildtime framework in order to boot containers in VMs, Baremetals and Cloud. + +You can either choose to **build** a Elemental derivative or **run** Elemental to boostrap a new system. + +Elemental vanilla images are published to allow to deploy user-built derivatives. + +Elemental is designed to run, deploy and upgrade derivatives that can be built just as standard OCI container images. Elemental assets can be used to either drive unattended deployments of a derivative or used to create custom images (with packer). + +## Philosophy + +{{}} + +Philosophy behind elemental-toolkit is simple: it allows you to create Linux derivatives from container images. + +- **Container registry as a single source of truth** +- Hybrid way to access your image for different scopes (development, debugging, ..) +- No more inconsistent states between nodes. A “Store” to keep your (tagged) shared states where you can rollback and upgrade into. +- “Stateless”: Images with upgrades are rebuilt from scratch instead of applying upgrades. +- A/B upgrades, immutable systems + +The container image is booted as-is, encapsulating all the needed components (kernel, initrd included) and can be pulled locally for inspection, development and debugging. At the same time it can be used also to create installation medium as ISO, Raw images, OVA or Cloud specific images. + +A derivative automatically inherits the following featureset: +- [Can upgrade to another container image](./upgrading) +- [Can deploy a system from scratch from an image](./deploy) +- [Reset or recovery to a specific image](./recovery) +- [Customize the image during runtime to persist changes across reboots](../customizing/runtime_persistent_changes) +- [Perform an installation from the LiveCD medium](./booting) + +## Building Elemental derivatives + +The starting point to use elemental-toolkit is to check out our [examples](https://github.com/rancher/elemental-toolkit/tree/main/examples) and our [creating bootable images](../creating-derivatives/creating_bootable_images) section. + +The only requirement to build derivatives with `elemental-toolkit` is Docker installed. If you are interested in building elemental-toolkit itself, see [Development notes](../development). + +The toolkit itself is delivered as a set of standalone, re-usable OCI artifacts which are tagged and tracked as standard OCI images and it is installed inside the container image to provide the same featureset among derivatives, see [how to create bootable images](../creating-derivatives/creating_bootable_images). + +## Vanilla images + +`Elemental` releases are composed of vanilla images that are used internally for testing and can be used as a starting point to deploy derivatives in specific environments (e.g. AWS) or just to try out the Elemental featureset. + +The vanilla images ships no specific business-logic aside serving as a base for testing and deploying other derivatives. + +### What to do next? + +Check out [how to create bootable images](../creating-derivatives/creating_bootable_images) or [download the Elemental vanilla images](../getting-started/download) to give Elemental a try! + +Here below you will find the common documentation that applies to any derivative built with Elemental and the Elemental vanilla images. diff --git a/versioned_docs/version-1.2/toolkit/getting-started/deploy.md b/versioned_docs/version-1.2/toolkit/getting-started/deploy.md new file mode 100644 index 000000000..38aed3138 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/getting-started/deploy.md @@ -0,0 +1,43 @@ +--- +title: "Deploying" +sidebar_label: "Deploying" +--- + + +Elemental vanilla images, like ISOs, cloud images or raw disks can be used to deploy another derivative image. + +## `elemental reset` + +`elemental reset` can be used to reset the system from the recovery image or from a custom image. Vanilla images only include a minimal recovery partition and system. + +It can be either invoked manually with `elemental reset --system.uri ` or used in conjuction with a cloud-init configuration, for example consider the following [cloud-init configuration file](../reference/cloud_init) that creates the `state` and `persistent` partitions during first boot (this is required on Elemental vanilla images): + + +```yaml +name: "Default deployment" +stages: + rootfs.after: + - name: "Repart image" + layout: + # It will partition a device including the given filesystem label or part label (filesystem label matches first) + device: + label: COS_RECOVERY + add_partitions: + - fsLabel: COS_STATE + # 10Gb for COS_STATE, so the disk should have at least 16Gb + size: 10240 + pLabel: state + - fsLabel: COS_PERSISTENT + # unset size or 0 size means all available space + pLabel: persistent + network: + - if: '[ -f "/run/cos/recovery_mode" ]' + name: "Deploy Elemental system" + commands: + - | + # Use `elemental reset --system.uri docker:` to deploy a custom image + # By default the recovery Elemental gets deployed + elemental reset --reboot --system.uri docker:$IMAGE +``` + +The following will first repartition the image after the `rootfs` [stage](../customizing/stages) and will run `elemental reset` when booting into [recovery mode](recovery). RAW vanilla disk images automatically boot by default into recovery, so the first thing upon booting is deploying the system diff --git a/versioned_docs/version-1.2/toolkit/getting-started/download.md b/versioned_docs/version-1.2/toolkit/getting-started/download.md new file mode 100644 index 000000000..39da076c3 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/getting-started/download.md @@ -0,0 +1,29 @@ +--- +title: "Download" +sidebar_label: "Download" +--- + +Elemental-toolkit consists of a CLI program that is used to install a system and build bootable sources. The CLI also embeds configuration needed for a bootable derivative. + +## Download Elemental + +Elemental toolkit can be run directly using a container runtime such as docker: + +```bash +docker run -it --rm ghcr.io/rancher/elemental-toolkit/elemental-cli:latest version +``` + +## Building from source + +The CLI can also be built from source by checking out the repo and running make: + +```bash +git clone https://github.com/rancher/elemental-toolkit +cd elemental-toolkit +make build-cli +./build/elemental version +``` + +## What to do next? + +Check out [the customization section](../customizing/stages) to build a custom `Elemental` derivative or [the example section](../examples/creating_bootable_images) for some already prepared recipe examples. diff --git a/versioned_docs/version-1.2/toolkit/getting-started/install.md b/versioned_docs/version-1.2/toolkit/getting-started/install.md new file mode 100644 index 000000000..ad4150d8e --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/getting-started/install.md @@ -0,0 +1,93 @@ +--- +title: "Installing" +sidebar_label: "Installing" +--- + + +Elemental (or any Elemental derivative built with elemental-toolkit) can be installed with `elemental install`: + +```bash +elemental install [options] +``` + +| Option | Description | +|------------------------------|--------------------------------------------------------------------------------------------------------------| +| --cloud-init string | Cloud-init config file | +| --cosign | Enable cosign verification (requires images with signatures) | +| --cosign-key string | Sets the URL of the public key to be used by cosign validation | +| --eject-cd | Try to eject the cd on reboot, only valid if booting from iso | +| --firmware string | Firmware to install for ('esp' or 'bios') (default "efi") | +| --force | Force install | +| --help | help for install | +| --iso string | Performs an installation from the ISO url | +| --no-format | Don’t format disks. It is implied that COS_STATE, COS_RECOVERY, COS_PERSISTENT, COS_OEM are already existing | +| --verify | Enable mtree checksum verification (requires images manifests generated with mtree separately) | +| --part-table string | Partition table type to use (default "gpt") | +| --poweroff | Shutdown the system after install | +| --reboot | Reboot the system after install | +| --recovery-system.uri string | Sets the recovery image source and its type (e.g. 'docker:registry.org/image:tag') | +| --system.uri string | Sets the system image source and its type (e.g. 'docker:registry.org/image:tag') | +| --strict | Enable strict check of hooks (They need to exit with 0) | +| --tty string | Add named tty to grub | + + +### Custom OEM configuration + +During installation it can be specified a [cloud-init config file](../reference/cloud_init), that will be installed and persist in the system after installation: + +```bash +elemental install --cloud-init [url|path] +``` + +### Custom partitioning + +When installing it's possible to specify a custom partition sizes via the configuration file (`/etc/elemental/config.yaml` by default). + +```yaml +install + partitions: + state: + # All sizes are in MiB + size: 8192 + recovery: + size: 4096 + oem: + size: 64 + persistent: + # zero size tells parted to use all the available + # disk, note this is only functional for the last partition + size: 0 +``` + +Refer to the [config file docs](../customizing/general_configuration) for further details about all partitioning options. + +In order to create additional partitions please consider the layout section on [cloud-init config file reference](../reference/cloud_init) + +### Installation from 3rd party LiveCD or rescue mediums + +The installer can be used to perform installations also from outside the Elemental or standard derivative ISOs. + +For instance, it is possible to install Elemental (or any derivative) with the installer from another bootable medium, or a rescue mode which is booting from RAM, given there is enough free RAM available. + +#### With Docker + +If in the rescue system, or LiveCD you have docker available, it can be used to perform an installation + +```bash +docker run --privileged -v /dev/:/dev/ -ti ghcr.io/rancher/elemental-toolkit/elemental-cli:latest install --system.uri $IMAGE $DEVICE +``` + +Where `$IMAGE` is the container image that we want to install (e.g. `oci:ghcr.io/rancher/elemental-toolkit/elemental-green:v0.10.7` ), elemental identifies the type of source by the URI scheme (`docker`, `channel`, `dir` or `file`). `$DEVICE` is the device where to perform the installation to (e.g. `/dev/sda`). + + +Note, we used the `ghcr.io/rancher/elemental-toolkit/elemental-cli:latest` image which contains the latest stable installer and the dependencies. +You can see all the versions at [GitHub Container Registry](https://ghcr.io/rancher/elemental-toolkit/elemental-cli). + + +#### By using manually the Elemental installer + +Similarly, the same mechanism can be used without docker. Install elemental using the [Download guide](download.md) and run the follow as root: + +```bash +elemental install --system.uri $IMAGE $DEVICE +``` diff --git a/versioned_docs/version-1.2/toolkit/getting-started/recovery.md b/versioned_docs/version-1.2/toolkit/getting-started/recovery.md new file mode 100644 index 000000000..6f7d52f52 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/getting-started/recovery.md @@ -0,0 +1,37 @@ +--- +title: "Recovery" +sidebar_label: "Recovery" +--- + +Elemental derivatives shares a common recovery mechanism built-in which can be leveraged to restore the system to a known point. At installation time, the recovery partition is created from the installation medium. + +The recovery system can be accessed during boot by selecting the last entry in the menu (labeled by "recovery"). + +A derivative can be recovered anytime by booting into the ` recovery` partition and by running `elemental reset` from it. + +This command will regenerate the bootloader and the images in the `COS_STATE` partition by using the recovery image. + +### Upgrading the recovery partition + +From either the active or passive system, the recovery partition can also be upgraded by running + +```bash +elemental upgrade --recovery +``` + +It also supports to specify docker images directly: + +```bash +elemental upgrade --recovery --recovery-system.uri +``` + +Where `` can be an opaque URI of `docker` scheme (e.g. `docker:registry.org/some/image:tag`). + +### Upgrading the active system from the recovery + +The recovery system can upgrade also the active system by running `elemental upgrade`, and it also supports to specify docker images directly: + +```bash +elemental upgrade --system.uri +``` + diff --git a/versioned_docs/version-1.2/toolkit/getting-started/upgrading.md b/versioned_docs/version-1.2/toolkit/getting-started/upgrading.md new file mode 100644 index 000000000..c3fa7f1aa --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/getting-started/upgrading.md @@ -0,0 +1,60 @@ +--- +title: "Upgrading" +sidebar_label: "Upgrading" +--- + +Elemental and every derivative can upgrade, rollback or just switch to different versions in runtime by using the toolkit installed inside the image. + +To upgrade an installed system, just run `elemental upgrade` and reboot. + +This will perform an upgrade based on the default derivative configuration for the image. See [general configuration](../customizing/general_configuration) on how to configure defaults when building a derivative. + +"Upgrades" are not carried over the usual way of treating each single package individually: Elemental considers the container image as a new system where to boot into. It will pull a new container image during this phase, which will be booted on the next reboot. + +## Upgrade to a specific container image + +To specify a specific container image to upgrade to instead of the regular upgrade channels, run `elemental upgrade --system.uri imagei-uri`. + +_Note_ by default `elemental upgrade --system.uri` runs an mtree checksum verificatiom (requires images manifests generated with mtree separately). To disable image checksum verification, run `elemental upgrade --verify --system.uri`. + +## Integration with System Upgrade Controller + +If running a kubernetes cluster on the `Elemental` system, you can leverage the [system-upgrade-controller](https://github.com/rancher/system-upgrade-controller) to trigger upgrades to specific image versions, for example: + +```yaml +--- +apiVersion: upgrade.cattle.io/v1 +kind: Plan +metadata: + name: elemental-upgrade + namespace: system-upgrade + labels: + k3s-upgrade: server +spec: + concurrency: 1 + version: fleet-sample # Image tag + nodeSelector: + matchExpressions: + - {key: k3s.io/hostname, operator: Exists} + serviceAccountName: system-upgrade + cordon: true +# drain: +# force: true + upgrade: + image: quay.io/costoolkit/test-images # Image upgrade reference + command: + - "/usr/sbin/suc-upgrade" +``` + +See also [trigger upgrades with fleet](../tutorials/trigger_upgrades_with_fleet) + +## From ISO + +The ISO can be also used as a recovery medium: type `elemental upgrade` from a LiveCD. It will then try to upgrade the image of the active partition installed in the system. + +## How it works +Elemental during installation sets two `.img` images files in the `COS_STATE` partition: +- `/cOS/active.img` labeled `COS_ACTIVE`: Where `Elemental` typically boots from +- `/cOS/passive.img` labeled `COS_PASSIVE`: Where `Elemental` boots for fallback + +Those are used by the upgrade mechanism to prepare and install a pristine `Elemental` each time an upgrade is attempted. diff --git a/versioned_docs/version-1.2/toolkit/index.md b/versioned_docs/version-1.2/toolkit/index.md new file mode 100644 index 000000000..112c9df50 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/index.md @@ -0,0 +1,56 @@ +--- +title: "Overview" +sidebar_label: "Overview" +--- + +## What is Elemental toolkit? + +Elemental is a toolkit which allows container images to be bootable in VMs, baremetals, embedded devices, and much more. + +Elemental allows to create meta-Linux derivatives which are configured throughout cloud-init configuration files and are immutable by default. + +Elemental and derivatives shares a common feature set, can be upgraded with a A/B mechanism, and upgrades are delivered with standard container registries. + +Elemental comes also with vanilla images that can be used to boot directly container images built with the toolkit. + +## Why Elemental? + +Elemental allows to create custom OS versions in your cluster with standard container images with a high degree of customization. It can also be used in its vanilla form - Elemental enables then everyone to build their own derivative and access it in various formats. + +Building a bootable image is as simple as running `docker build`. + +* **What is it good for?**: Embedded, Cloud, Containers, VM, Baremetals, Servers, IoT, Edge + +## Design goals + +- A Manifest for container-based OS. It contains just the common bits to make a container image bootable and to be upgraded from, with few customization on top +- Everything is an OCI artifact from the ground-up +- Immutable-first, but with a flexible layout +- Cloud-init driven +- Based on systemd +- Built and upgraded from containers - It is a [single image OS](https://quay.io/repository/costoolkit/releases-teal)! +- A/B updates +- Easy to customize +- Cryptographically verified +- Instant switch from different versions +- Recovery mechanism with `Elemental` vanilla images (or bring your own) + +## Mission + +The elemental-toolkit project is under the Elemental umbrella. + +Elemental-toolkit provides a unique container based approach to define the system lifecycle of an immutable Linux derivative, without any string attached to a specific Linux distribution. + +At its heart, Elemental-toolkit is the abstraction layer between Linux distro management and the specific purpose of the OS. + +Elemental-toolkit empowers anyone to create derivatives from standard OCI images. Frees whoever wants to create a Linux derivative from handling the heavy bits of packaging and managing entire repositories to propagate upgrades, simplifying the entire process by using container images as base for OS. +At the same time, Elemental-toolkit provides an highly integrated ecosystem which is designed to be container-first, cloud native, and immutable. +Anyone can tweak Elemental-toolkit derivatives from the bottom-up to enable and disable its featureset. + +As the Elemental team, the [elemental](https://github.com/rancher/elemental) project is our point of reference. + +`Elemental` is a complete derivative built with elemental-toolkit tied with the rancher ecosystem and full cycle node management solution with Kubernetes. + +We are supporting directly and indirectly `elemental` within changes also in the Elemental ecosystem. + +`Elemental` is our main show-case, and as the Elemental team we are committed to it. It encompasses several technologies to create a Kubernetes-focused Linux derivative which lifecycle is managed entirely from Kubernetes itself, [Secure Device Onboarding](https://www.intel.it/content/www/it/it/internet-of-things/secure-device-onboard.html) included, and automatic provisioning via cloud-init. diff --git a/versioned_docs/version-1.2/toolkit/reference/_index.md b/versioned_docs/version-1.2/toolkit/reference/_index.md new file mode 100644 index 000000000..8f0f3313c --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/_index.md @@ -0,0 +1,14 @@ +--- +title: "Reference" +sidebar_label: "Reference" +weight: 7 +description: > + References for Elemental derivatives, like common featuresets, high level architecture +--- + +- [Github project](https://github.com/orgs/rancher/projects/9/views/1) for our sprint board + + +### Samples repositories + +- [Build a derivative and upgrade it with fleet](https://github.com/rancher-sandbox/cos-fleet-upgrades-sample) diff --git a/versioned_docs/version-1.2/toolkit/reference/built_with_elemental.md b/versioned_docs/version-1.2/toolkit/reference/built_with_elemental.md new file mode 100644 index 000000000..0373d77bc --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/built_with_elemental.md @@ -0,0 +1,24 @@ +--- +title: "Built with Elemental" +sidebar_label: "Built with Elemental" +--- + +Here is a not-exhaustive lists of derivatives built with Elemental. + + +## Run by Rancher + +| Name | Description | Link | +|-----------|-----------------------------------------------------------------------------------------------------------------|----------------------------------------| +| Harvester | Open source hyperconverged infrastructure (HCI) software | https://github.com/harvester/harvester | +| Elemental | Immutable Linux distribution built to run Rancher and its corresponding Kubernetes distributions RKE2 and k3s. | https://github.com/rancher/elemental | + + +## Run by Community + +The following derivatives are not run by Rancher, but are community efforts. +Open up an issue or create a PR to get yours added to the list! + +| Name | Description | Link | +|-----------|-----------------------------------------------------------------------------------------------------------------|----------------------------------------| +| Kairos | Immutable OS for Automated (Decentralized) Kubernetes clusters with k3s, for homelab and beyond | https://github.com/kairos-io/kairos | diff --git a/versioned_docs/version-1.2/toolkit/reference/cloud_init.md b/versioned_docs/version-1.2/toolkit/reference/cloud_init.md new file mode 100644 index 000000000..0a1280547 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/cloud_init.md @@ -0,0 +1,440 @@ +--- +title: "Cloud-init support" +sidebar_label: "Cloud-init support" +--- + +Below is a reference of all keys available in the cloud-init style files. + +```yaml +stages: + # "network" is the stage where network is expected to be up + # It is called internally when network is available from + # the cos-setup-network unit. + network: + # Here there are a list of + # steps to be run in the network stage + - name: "Some setup happening" + files: + - path: /tmp/foo + content: | + test + permissions: 0777 + owner: 1000 + group: 100 + commands: + - echo "test" + modules: + - nvidia + environment: + FOO: "bar" + systctl: + debug.exception-trace: "0" + hostname: "foo" + systemctl: + enable: + - foo + disable: + - bar + start: + - baz + mask: + - foobar + authorized_keys: + user: + - "github:suse" + - "ssh-rsa ...." + dns: + path: /etc/resolv.conf + nameservers: + - 8.8.8.8 + ensure_entities: + - path: /etc/passwd + entity: | + kind: "user" + username: "foo" + password: "pass" + uid: 0 + gid: 0 + info: "Foo!" + homedir: "/home/foo" + shell: "/bin/bash" + delete_entities: + - path: /etc/passwd + entity: | + kind: "user" + username: "foo" + password: "pass" + uid: 0 + gid: 0 + info: "Foo!" + homedir: "/home/foo" + shell: "/bin/bash" + datasource: + providers: + - "aws" + - "digitalocean" + path: "/etc/cloud-data" +``` + +The default cloud-config format is split into *stages* (*initramfs*, *boot*, *network*, *initramfs*, *reconcile*, called generically **STAGE_ID** below) [see also stages](../customizing/stages) that are emitted internally during the various phases by calling `cos-setup STAGE_ID`. +*steps* (**STEP_NAME** below) defined for each stage are executed in order. + +Each cloud-config file is loaded and executed only at the apprioriate stage, this allows further components to emit their own stages at the desired time. + +{{% pageinfo %}} +The [cloud-init tool](https://github.com/mudler/yip#readme) can be also run standalone, this helps debugging locally and also during development, you can find separate [releases here](https://github.com/mudler/yip/releases). +{{% /pageinfo %}} + +_Note_: Each cloud-init option can be either run in *dot notation* ( e.g. `stages.network[0].authorized_keys.user=github:user` ) in the boot args or either can supply a cloud-init URL at boot with the `cos.setup=$URL` parameter. + +### Using templates + +With Cloud Init support, templates can be used to allow dynamic configuration. More information about templates can be found [here](https://github.com/mudler/yip#node-data-interpolation) and also [here for sprig](http://masterminds.github.io/sprig/) functions. + +### Compatibility with Cloud Init format + +A subset of the official [cloud-config spec](http://cloudinit.readthedocs.org/en/latest/topics/format.html#cloud-config-data) is implemented. + +If a yaml file starts with `#cloud-config` it is parsed as a standard cloud-init and automatically associated it to the `boot` stage. For example: + +```yaml +#cloud-config +growpart: + mode: auto + devices: ['/'] + +users: +- name: "bar" + passwd: "foo" + lock_passwd: true + uid: "1002" + groups: "users" + ssh_authorized_keys: + - faaapploo +ssh_authorized_keys: + - asdd +runcmd: +- foo +hostname: "bar" +write_files: +- encoding: b64 + content: CiMgVGhpcyBmaWxlIGNvbnRyb2xzIHRoZSBzdGF0ZSBvZiBTRUxpbnV4 + path: /foo/bar + permissions: "0644" + owner: "bar" +``` + +Is executed at boot, by using the standard `cloud-config` format. + +{{% alert title="Note" %}} +You can't mix extended syntax with legacy cloud-init syntax. By pre-pending the cloud-config with the `#cloud-config` header you enable the legacy notation, and the extended one ( `stages`.. ) will be ignored. +{{% /alert %}} + + +### `stages.STAGE_ID.STEP_NAME.name` + +A description of the stage step. Used only when printing output to console. + +### `stages.STAGE_ID.STEP_NAME.files` + +A list of files to write to disk. + +```yaml +stages: + default: + - files: + - path: /tmp/bar + content: | + #!/bin/sh + echo "test" + permissions: 0777 + owner: 1000 + group: 100 +``` + +### `stages.STAGE_ID.STEP_NAME.directories` + +A list of directories to be created on disk. Runs before `files`. + +```yaml +stages: + default: + - name: "Setup folders" + directories: + - path: "/etc/foo" + permissions: 0600 + owner: 0 + group: 0 +``` + +### `stages.STAGE_ID.STEP_NAME.dns` + +A way to configure the `/etc/resolv.conf` file. + +```yaml +stages: + default: + - name: "Setup dns" + dns: + nameservers: + - 8.8.8.8 + - 1.1.1.1 + search: + - foo.bar + options: + - .. + path: "/etc/resolv.conf.bak" +``` +### `stages.STAGE_ID.STEP_NAME.hostname` + +A string representing the machine hostname. It sets it in the running system, updates `/etc/hostname` and adds the new hostname to `/etc/hosts`. +Templates can be used to allow dynamic configuration. For example in mass-install scenario it could be needed (and easier) to specify hostnames for multiple machines from a single cloud-init config file. + +```yaml +stages: + default: + - name: "Setup hostname" + hostname: "node-{{ trunc 4 .MachineID }}" +``` +### `stages.STAGE_ID.STEP_NAME.sysctl` + +Kernel configuration. It sets `/proc/sys/` accordingly, similarly to `sysctl`. + +```yaml +stages: + default: + - name: "Setup exception trace" + systctl: + debug.exception-trace: "0" +``` + +### `stages.STAGE_ID.STEP_NAME.authorized_keys` + +A list of SSH authorized keys that should be added for each user. +SSH keys can be obtained from GitHub user accounts by using the format github:${USERNAME}, similarly for Gitlab with gitlab:${USERNAME}. + +```yaml +stages: + default: + - name: "Setup exception trace" + authorized_keys: + joe: + - github:joe + - ssh-rsa: ... +``` + +### `stages.STAGE_ID.STEP_NAME.node` + +If defined, the node hostname where this stage has to run, otherwise it skips the execution. The node can be also a regexp in the Golang format. + +```yaml +stages: + default: + - name: "Setup logging" + node: "bastion" +``` + +### `stages.STAGE_ID.STEP_NAME.users` + +A map of users and user info to set. Passwords can be also encrypted. + +The `users` parameter adds or modifies the specified list of users. Each user is an object which consists of the following fields. Each field is optional and of type string unless otherwise noted. +In case the user is already existing, the entry is ignored. + +- **name**: Required. Login name of user +- **gecos**: GECOS comment of user +- **passwd**: Hash of the password to use for this user. Unencrypted strings are supported too. +- **homedir**: User's home directory. Defaults to /home/*name* +- **no-create-home**: Boolean. Skip home directory creation. +- **primary-group**: Default group for the user. Defaults to a new group created named after the user. +- **groups**: Add user to these additional groups +- **no-user-group**: Boolean. Skip default group creation. +- **ssh-authorized-keys**: List of public SSH keys to authorize for this user +- **system**: Create the user as a system user. No home directory will be created. +- **no-log-init**: Boolean. Skip initialization of lastlog and faillog databases. +- **shell**: User's login shell. + +```yaml +stages: + default: + - name: "Setup users" + users: + bastion: + passwd: "strongpassword" + homedir: "/home/foo +``` + +### `stages.STAGE_ID.STEP_NAME.ensure_entities` + +A `user` or a `group` in the [entity](https://github.com/mudler/entities) format to be configured in the system + +```yaml +stages: + default: + - name: "Setup users" + ensure_entities: + - path: /etc/passwd + entity: | + kind: "user" + username: "foo" + password: "x" + uid: 0 + gid: 0 + info: "Foo!" + homedir: "/home/foo" + shell: "/bin/bash" +``` +### `stages.STAGE_ID.STEP_NAME.delete_entities` + +A `user` or a `group` in the [entity](https://github.com/mudler/entities) format to be pruned from the system + +```yaml +stages: + default: + - name: "Setup users" + delete_entities: + - path: /etc/passwd + entity: | + kind: "user" + username: "foo" + password: "x" + uid: 0 + gid: 0 + info: "Foo!" + homedir: "/home/foo" + shell: "/bin/bash" +``` +### `stages.STAGE_ID.STEP_NAME.modules` + +A list of kernel modules to load. + +```yaml +stages: + default: + - name: "Setup users" + modules: + - nvidia +``` +### `stages.STAGE_ID.STEP_NAME.systemctl` + +A list of systemd services to `enable`, `disable`, `mask` or `start`. + +```yaml +stages: + default: + - name: "Setup users" + systemctl: + enable: + - systemd-timesyncd + - cronie + mask: + - purge-kernels + disable: + - crond + start: + - cronie +``` +### `stages.STAGE_ID.STEP_NAME.environment` + +A map of variables to write in `/etc/environment`, or otherwise specified in `environment_file` + +```yaml +stages: + default: + - name: "Setup users" + environment: + FOO: "bar" +``` +### `stages.STAGE_ID.STEP_NAME.environment_file` + +A string to specify where to set the environment file + +```yaml +stages: + default: + - name: "Setup users" + environment_file: "/home/user/.envrc" + environment: + FOO: "bar" +``` +### `stages.STAGE_ID.STEP_NAME.timesyncd` + +Sets the `systemd-timesyncd` daemon file (`/etc/system/timesyncd.conf`) file accordingly. The documentation for `timesyncd` and all the options can be found [here](https://www.freedesktop.org/software/systemd/man/timesyncd.conf.html). + +```yaml +stages: + default: + - name: "Setup NTP" + systemctl: + enable: + - systemd-timesyncd + timesyncd: + NTP: "0.pool.org foo.pool.org" + FallbackNTP: "" + ... +``` + +### `stages.STAGE_ID.STEP_NAME.commands` + +A list of arbitrary commands to run after file writes and directory creation. + +```yaml +stages: + default: + - name: "Setup something" + commands: + - echo 1 > /bar +``` + +### `stages.STAGE_ID.STEP_NAME.datasource` + +Sets to fetch user data from the specified cloud providers. It populates +provider specific data into `/run/config` folder and the custom user data +is stored into the provided path. + + +```yaml +stages: + default: + - name: "Fetch cloud provider's user data" + datasource: + providers: + - "aws" + - "digitalocean" + path: "/etc/cloud-data" +``` + +### `stages.STAGE_ID.STEP_NAME.layout` + + +Sets additional partitions on disk free space, if any, and/or expands the last +partition. All sizes are expressed in MiB only and default value of `size: 0` +means all available free space in disk. This plugin is useful to be used in +oem images where the default partitions might not suit the actual disk geometry. + + +```yaml +stages: + default: + - name: "Repart disk" + layout: + device: + # It will partition a device including the given filesystem label + # or partition label (filesystem label matches first) or the device + # provided in 'path'. The label check has precedence over path when + # both are provided. + label: "COS_RECOVERY" + path: "/dev/sda" + # Only last partition can be expanded and it happens before any other + # partition is added. size: 0 means all available free space + expand_partition: + size: 4096 + add_partitions: + - fsLabel: "COS_STATE" + size: 8192 + # No partition label is applied if omitted + pLabel: "state" + - fsLabel: "COS_PERSISTENT" + # default filesystem is ext2 if omitted + filesystem: "ext4" +``` diff --git a/versioned_docs/version-1.2/toolkit/reference/elemental.md b/versioned_docs/version-1.2/toolkit/reference/elemental.md new file mode 100644 index 000000000..745f5d462 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/elemental.md @@ -0,0 +1,29 @@ +--- +title: "elemental" +sidebar_label: "elemental" +--- +## elemental + +Elemental + +### Options + +``` + --config-dir string Set config dir + --debug Enable debug output + -h, --help help for elemental + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental build-iso](elemental_build-iso.md) - Build bootable installation media ISOs +* [elemental cloud-init](elemental_cloud-init.md) - Run cloud-init +* [elemental install](elemental_install.md) - Elemental installer +* [elemental pull-image](elemental_pull-image.md) - Pull remote image to local file +* [elemental reset](elemental_reset.md) - Reset OS +* [elemental run-stage](elemental_run-stage.md) - Run stage from cloud-init +* [elemental upgrade](elemental_upgrade.md) - Upgrade the system +* [elemental version](elemental_version.md) - Print the version + diff --git a/versioned_docs/version-1.2/toolkit/reference/elemental_build-iso.md b/versioned_docs/version-1.2/toolkit/reference/elemental_build-iso.md new file mode 100644 index 000000000..3030a932f --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/elemental_build-iso.md @@ -0,0 +1,52 @@ +--- +title: "elemental build-iso" +sidebar_label: "elemental build-iso" +--- + +Build bootable installation media ISOs + +### Synopsis + +Build bootable installation media ISOs + +SOURCE - should be provided as uri in following format `:` + * `` - might be ["dir", "file", "oci", "docker", "channel"], as default is "docker" + * `` - is path to file or directory, image name with tag version or channel name + +``` +elemental build-iso SOURCE [flags] +``` + +### Options + +``` + --bootloader-in-rootfs Fetch ISO bootloader binaries from the rootfs + --cosign Enable cosign verification (requires images with signatures) + --cosign-key string Sets the URL of the public key to be used by cosign validation + --date Adds a date suffix into the generated ISO file + -h, --help help for build-iso + --label string Label of the ISO volume + --local Use an image from local cache + -n, --name string Basename of the generated ISO file + -o, --output string Output directory (defaults to current directory) + --overlay-iso string Path of the overlayed iso data + --overlay-rootfs string Path of the overlayed rootfs data + --overlay-uefi string Path of the overlayed uefi data + --platform string Platform to build the image for (default "linux/amd64") + -x, --squash-compression stringArray cmd options for compression to pass to mksquashfs. Full cmd including --comp as the whole values will be passed to mksquashfs. For a full list of options please check mksquashfs manual. (default value: '-comp xz -Xbcj ARCH') + --squash-no-compression Disable squashfs compression. Overrides any values on squash-compression +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/versioned_docs/version-1.2/toolkit/reference/elemental_cloud-init.md b/versioned_docs/version-1.2/toolkit/reference/elemental_cloud-init.md new file mode 100644 index 000000000..c02032e89 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/elemental_cloud-init.md @@ -0,0 +1,32 @@ +--- +title: "elemental cloud-init" +sidebar_label: "elemental cloud-init" +--- + +Run cloud-init + +``` +elemental cloud-init [flags] +``` + +### Options + +``` + -d, --dotnotation stages.foo.name=.. Parse input in dotnotation ( e.g. stages.foo.name=.. ) + -h, --help help for cloud-init + -s, --stage string Stage to apply (default "default") +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/versioned_docs/version-1.2/toolkit/reference/elemental_convert-disk.md b/versioned_docs/version-1.2/toolkit/reference/elemental_convert-disk.md new file mode 100644 index 000000000..f4fd34da5 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/elemental_convert-disk.md @@ -0,0 +1,32 @@ +--- +title: "elemental convert-disk" +sidebar_label: "elemental convert-disk" +--- + +converts between a raw disk and a cloud operator disk image (azure,gce) + +``` +elemental convert-disk RAW_DISK [flags] +``` + +### Options + +``` + -h, --help help for convert-disk + --keep-source Keep the source image, otherwise it will delete it once transformed. + -t, --type string Type of image to create (default "azure") +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/versioned_docs/version-1.2/toolkit/reference/elemental_exit-codes.md b/versioned_docs/version-1.2/toolkit/reference/elemental_exit-codes.md new file mode 100644 index 000000000..00b8dd297 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/elemental_exit-codes.md @@ -0,0 +1,80 @@ +--- +title: "Exit codes for elemental CLI" +sidebar_label: "Exit codes for elemental CLI" +--- + +| Exit code | Meaning | +| :----: | :---- | +| 10 | Error closing a file| +| 11 | Error running a command| +| 12 | Error copying data| +| 13 | Error copying a file| +| 14 | Wrong cosign flags used in cmd| +| 15 | Error creating a dir| +| 16 | Error creating a file| +| 17 | Error creating a temporal dir| +| 18 | Error dumping the source| +| 19 | Error creating a gzip writer| +| 20 | Error trying to identify the source| +| 21 | Error calling mkfs| +| 22 | There is not packages for the given architecture| +| 24 | Error opening a file| +| 25 | Output file already exists| +| 26 | Error reading the build config| +| 27 | Error reading the build-disk config| +| 28 | Error running stat on a file| +| 29 | Error creating a tar archive| +| 30 | Error truncating a file| +| 31 | Error reading the run config| +| 32 | Error reading the install/upgrade flags| +| 33 | Error reading the config for the command| +| 34 | Error mounting state partition| +| 35 | Error mounting recovery partition| +| 36 | Error during before-upgrade hook| +| 37 | Error during before-upgrade-chroot hook| +| 38 | Error during after-upgrade hook| +| 39 | Error during after-upgrade-chroot hook| +| 40 | Error moving file| +| 41 | Error occurred during cleanup| +| 42 | Error occurred trying to reboot| +| 43 | Error occurred trying to shutdown| +| 44 | Error occurred when labeling partition| +| 45 | Error setting default grub entry| +| 46 | Error occurred during selinux relabeling| +| 47 | Error invalid device specified| +| 48 | Error deploying image to file| +| 49 | Error installing GRUB| +| 50 | Error during before-install hook| +| 51 | Error during after-install hook| +| 52 | Error during after-install-chroot hook| +| 53 | Error during file download| +| 54 | Error mounting partitions| +| 55 | Error deactivating active devices| +| 56 | Error during device partitioning| +| 57 | Device already contains an install| +| 58 | Command requires root privileges| +| 59 | Error occurred when unmounting partitions| +| 60 | Error occurred when formatting partitions| +| 61 | Error during before-reset hook| +| 62 | Error during after-reset-chroot hook| +| 63 | Error during after-reset hook| +| 64 | Unsupported flavor| +| 65 | Error encountered during cloud-init run-stage| +| 66 | Error unpacking image| +| 67 | Error reading file| +| 68 | No source was provided for the command| +| 69 | Error removing a file| +| 70 | Error calculating checksum| +| 71 | Error occurred when unmounting image| +| 72 | Error occurred during post-upgrade hook| +| 73 | Error occurred during post-reset hook| +| 74 | Error occurred during post-install hook| +| 75 | Error occurred while preparing the image root tree| +| 76 | Error occurred while creating the OS filesystem image| +| 77 | Error occurred while copying the filesystem image and setting new labels| +| 78 | Error setting persistent GRUB variables| +| 255 | Unknown error| + +### SEE ALSO + +* [elemental](elemental.md) - Elemental diff --git a/versioned_docs/version-1.2/toolkit/reference/elemental_install.md b/versioned_docs/version-1.2/toolkit/reference/elemental_install.md new file mode 100644 index 000000000..bec9d63c2 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/elemental_install.md @@ -0,0 +1,50 @@ +--- +title: "elemental install" +sidebar_label: "elemental install" +--- + +Elemental installer + +``` +elemental install DEVICE [flags] +``` + +### Options + +``` + -c, --cloud-init strings Cloud-init config files + --cosign Enable cosign verification (requires images with signatures) + --cosign-key string Sets the URL of the public key to be used by cosign validation + --disable-boot-entry Dont create an EFI entry for the system install. + --eject-cd Try to eject the cd on reboot, only valid if booting from iso + --firmware string Firmware to install for: 'efi' or 'bios'. (defaults to 'efi') (default "efi") + --force Force install + -h, --help help for install + -i, --iso string Performs an installation from the ISO url + --local Use an image from local cache + --no-format Don’t format disks. It is implied that COS_STATE, COS_RECOVERY, COS_PERSISTENT, COS_OEM are already existing + --part-table string Partition table type to use (default "gpt") + --platform string Platform to build the image for (default "linux/amd64") + --poweroff Shutdown the system after install + --reboot Reboot the system after install + --recovery-system.uri string Sets the recovery image source and its type (e.g. 'docker:registry.org/image:tag') + -x, --squash-compression stringArray cmd options for compression to pass to mksquashfs. Full cmd including --comp as the whole values will be passed to mksquashfs. For a full list of options please check mksquashfs manual. (default value: '-comp xz -Xbcj ARCH') + --squash-no-compression Disable squashfs compression. Overrides any values on squash-compression + --strict Enable strict check of hooks (They need to exit with 0) + --system.uri string Sets the system image source and its type (e.g. 'docker:registry.org/image:tag') + --verify Enable mtree checksum verification (requires images manifests generated with mtree separately) +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/versioned_docs/version-1.2/toolkit/reference/elemental_new.md b/versioned_docs/version-1.2/toolkit/reference/elemental_new.md new file mode 100644 index 000000000..46454eb4f --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/elemental_new.md @@ -0,0 +1,31 @@ +--- +title: "elemental new" +sidebar_label: "elemental new" +--- + +Create skeleton Dockerfile for a derivative + +``` +elemental new FLAVOR [flags] +``` + +### Options + +``` + --arch string X86_64 or aarch64 architectures + -h, --help help for new +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/versioned_docs/version-1.2/toolkit/reference/elemental_pull-image.md b/versioned_docs/version-1.2/toolkit/reference/elemental_pull-image.md new file mode 100644 index 000000000..025c3343a --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/elemental_pull-image.md @@ -0,0 +1,32 @@ +--- +title: "elemental pull-image" +sidebar_label: "elemental pull-image" +--- + +Pull remote image to local file + +``` +elemental pull-image IMAGE DESTINATION [flags] +``` + +### Options + +``` + -h, --help help for pull-image + --local Use an image from local cache + --platform string Platform to build the image for (default "linux/amd64") +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/versioned_docs/version-1.2/toolkit/reference/elemental_reset.md b/versioned_docs/version-1.2/toolkit/reference/elemental_reset.md new file mode 100644 index 000000000..f287b107d --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/elemental_reset.md @@ -0,0 +1,41 @@ +--- +title: "elemental reset" +sidebar_label: "elemental reset" +--- + +Reset OS + +``` +elemental reset [flags] +``` + +### Options + +``` + -c, --cloud-init strings Cloud-init config files + --cosign Enable cosign verification (requires images with signatures) + --cosign-key string Sets the URL of the public key to be used by cosign validation + --disable-boot-entry Dont create an EFI entry for the system install. + -h, --help help for reset + --poweroff Shutdown the system after install + --reboot Reboot the system after install + --reset-oem Clear OEM partitions + --reset-persistent Clear persistent partitions + --strict Enable strict check of hooks (They need to exit with 0) + --system.uri string Sets the system image source and its type (e.g. 'docker:registry.org/image:tag') + --verify Enable mtree checksum verification (requires images manifests generated with mtree separately) +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/versioned_docs/version-1.2/toolkit/reference/elemental_run-stage.md b/versioned_docs/version-1.2/toolkit/reference/elemental_run-stage.md new file mode 100644 index 000000000..4c08d924b --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/elemental_run-stage.md @@ -0,0 +1,31 @@ +--- +title: "elemental run-stage" +sidebar_label: "elemental run-stage" +--- + +Run stage from cloud-init + +``` +elemental run-stage STAGE [flags] +``` + +### Options + +``` + -h, --help help for run-stage + --strict Set strict checking for errors, i.e. fail if errors were found +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/versioned_docs/version-1.2/toolkit/reference/elemental_upgrade.md b/versioned_docs/version-1.2/toolkit/reference/elemental_upgrade.md new file mode 100644 index 000000000..c431ab3c0 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/elemental_upgrade.md @@ -0,0 +1,42 @@ +--- +title: "elemental upgrade" +sidebar_label: "elemental upgrade" +--- + +Upgrade the system + +``` +elemental upgrade [flags] +``` + +### Options + +``` + --cosign Enable cosign verification (requires images with signatures) + --cosign-key string Sets the URL of the public key to be used by cosign validation + -h, --help help for upgrade + --local Use an image from local cache + --poweroff Shutdown the system after install + --reboot Reboot the system after install + --recovery Upgrade the recovery + --recovery-system.uri string Sets the recovery image source and its type (e.g. 'docker:registry.org/image:tag') + -x, --squash-compression stringArray cmd options for compression to pass to mksquashfs. Full cmd including --comp as the whole values will be passed to mksquashfs. For a full list of options please check mksquashfs manual. (default value: '-comp xz -Xbcj ARCH') + --squash-no-compression Disable squashfs compression. Overrides any values on squash-compression + --strict Enable strict check of hooks (They need to exit with 0) + --system.uri string Sets the system image source and its type (e.g. 'docker:registry.org/image:tag') + --verify Enable mtree checksum verification (requires images manifests generated with mtree separately) +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/versioned_docs/version-1.2/toolkit/reference/elemental_version.md b/versioned_docs/version-1.2/toolkit/reference/elemental_version.md new file mode 100644 index 000000000..bcc104e86 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/elemental_version.md @@ -0,0 +1,31 @@ +--- +title: "elemental version" +sidebar_label: "elemental version" +--- + +Print the version + +``` +elemental version [flags] +``` + +### Options + +``` + -h, --help help for version + --long Show long version info +``` + +### Options inherited from parent commands + +``` + --config-dir string Set config dir + --debug Enable debug output + --logfile string Set logfile + --quiet Do not output to stdout +``` + +### SEE ALSO + +* [elemental](elemental.md) - Elemental + diff --git a/versioned_docs/version-1.2/toolkit/reference/high_level_architecture.md b/versioned_docs/version-1.2/toolkit/reference/high_level_architecture.md new file mode 100644 index 000000000..c8ffab901 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/high_level_architecture.md @@ -0,0 +1,47 @@ +--- +title: "High level architecture" +sidebar_label: "High level architecture" +--- + +# Elemental toolkit High level Architecture + +This page tries to encompass the [`elemental-toolkit`](https://github.com/rancher/elemental-toolkit) structure and the high level architecture, along with all the involved components. + + +## Design goals + +- Blueprints to build immutable Linux derivatives from container images +- A workflow to maintain, support and deliver custom-OS and upgrades to end systems +- Derivatives have the same “foundation” manifest - easy to customize on top, add packages: `systemd`, `dracut` and `grub` as a foundation stack. +- Upgrades delivered with container registry images ( also workflow with `docker run` && `docker commit` supported! ) +
The content of the container image is the system which is booted. + + +## High level overview + +elemental-toolkit encompasses several components required for building and distributing OS images. [This issue](https://github.com/rancher/elemental-toolkit/issues/108) summarize the current state, and how we plan to integrate them in a single CLI to improve the user experience. + +elemental-toolkit is also a manifest, which includes package definitions of how the underlying OS is composed. It forms an abstraction layer, which is then translated to Dockerfiles and built by our CI (optionally) for re-usal. A derivative can be built by parts of the manifest, or reusing it entirely, container images included. + +![High level overview](https://docs.google.com/drawings/d/e/2PACX-1vQQJOaISPbMxMYU44UT-M3ou9uGYOrzbXCRXMLPU8m7_ie3ke_08xCsyRLkFZJRB4VnzIeobPciEoQv/pub?w=942&h=532) + +The fundamental phases can be summarized in the following steps: + +- Build packages from container images (and optionally keep build caches) +- Extract artefacts from containers +- Add metadata(s) and create a repository +- (optionally) publish the repository and the artefacts + +The developer of the derivative applies a customization layer during build, which is an augmentation layer in the same form of `elemental-toolkit` itself. + +## Distribution + +The OS delivery mechanism is done via container registries. The developer that wants to provide upgrades for the custom OS will push the resulting container images to the container registry. It will then be used by the installed system to pull upgrades from. + +![](https://docs.google.com/drawings/d/e/2PACX-1vQrTArCYgu-iscf29v1sl1sEn2J81AqBpi9D5xpwGKr9uxR2QywoSqCmsSaJLxRRacoRr0Kq40a7jPF/pub?w=969&h=464) + +## Upgrade mechanism + +There are two different upgrade mechanisms available that can be used from a maintainer perspective: (a) release channels or (b) providing a container image reference ( `e.g. my.registry.com/image:tag` ) [that can be tweaked in the customization phases](https://github.com/rancher/elemental-toolkit#default-oem) to achieve the desired effect. + + \ No newline at end of file diff --git a/versioned_docs/version-1.2/toolkit/reference/immutable_rootfs.md b/versioned_docs/version-1.2/toolkit/reference/immutable_rootfs.md new file mode 100644 index 000000000..6ca97cfdd --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/immutable_rootfs.md @@ -0,0 +1,23 @@ +--- +title: "Immutable Root Filesystem" +sidebar_label: "Immutable Rootfs" +--- + +The immutable rootfs concept in Elemental is provided by a dracut module. +By default, `elemental` and derivatives will inherit an immutable setup. + +![Partitioning layout](https://docs.google.com/drawings/d/e/2PACX-1vR-I5ZwwB5EjpsymUfcNADRTTKXrNMnlZHgD8RjDpzYhyYiz_JrWJwvpcfMcwfYet1oWCZVWH22aj1k/pub?w=533&h=443) + +A running system will look like as follows: + +``` +/usr/local - persistent (COS_PERSISTENT) +/oem - persistent (COS_OEM) +/etc - ephemeral +/usr - read only +/ immutable +``` + +This means that any changes that are not specified as cloud-init configuration are not persisting across reboots. + +You can place persisting cloud-init files either in `/oem` or `/usr/local/oem`, `Elemental` already supports cloud-init [datasources](https://cloudinit.readthedocs.io/en/latest/topics/datasources.html), so you can use also load cloud-init configuration as standard userdata, depending on the platform. For more details on the cloud-init syntax, see the [cloud-init configuration reference](cloud_init). diff --git a/versioned_docs/version-1.2/toolkit/reference/layout.md b/versioned_docs/version-1.2/toolkit/reference/layout.md new file mode 100644 index 000000000..13cb7e5eb --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/layout.md @@ -0,0 +1,42 @@ +--- +title: "Runtime layout" +sidebar_label: "Runtime layout" +--- + +This section describes the runtime layout of a derivative (or a Elemental Vanilla image) once booted in a system. + +The Elemental toolkit performs during installation a common setup which is equivalent across all derivatives. + +This mechanism ensures that a layout: + +- it's simple and human friendly +- allows to switch easily derivatives +- allows to perform recovery tasks +- is resilient to upgrade failures + +## Layout + +The basic setup consists of: + +- an `A/B` partitioning style. We have an 'active' and a 'passive' system too boot from in case of failures +- a Recovery system which allows to perform emergency tasks in case of failure of the 'A/B' partitions +- a Fallback mechanism that boots the partitions in this sequence: "A -> B -> Recovery" in case of booting failures + +The upgrade happens in a transition image and take places only after all the necessary steps are completed. An upgrade of the 'A/B' partitions can be done by booting into them and running `elemental upgrade`. This will create a new pristine image that will be selected as active for the next reboot, the old one will be flagged as passive. If we are performing the same from the passive system, only the active is subject to changes. + +Similarly, a recovery system can be upgraded as well by running `elemental upgrade --recovery`. This will upgrade the recovery system instead of the active/passive. Note both commands needs to be run inside the active or passive system. + +## Partitions + +![](https://docs.google.com/drawings/d/e/2PACX-1vSP-Pz9l9hwYDeIlej7qXzzcMzGYBiKjyFpiYYKlbNR3H37n_R_c0eBNeYa3msouOupmDim3ZYYBSxS/pub?w=812&h=646) + +The default partitioning is created during installation and is expected to be present in a booted Elemental system: + +- a `COS_STATE` partition that will contain our active, passive and recovery images. The images are located under the `/cOS` directory +- a `COS_PERSISTENT` partition which contains the persistent user data. This directory is mounted over `/usr/local` during runtime +- a `COS_OEM` partition which contains the cloud-init oem files, which is mounted over `/oem` during runtime +- a `COS_RECOVERY` partition which contains the recovery system image + +The `COS_STATE` partitions contains the `active`, `passive` . While the `active` and `passive` are `.img` files which are loopback mounted, the `recovery` system is in `COS_RECOVERY` and can also be a `squashfs` file (provided in `/cOS/recovery.squashfs`). This ensures the immutability aspect and ease out building derivative in constrained environments (e.g. when we have restricted permissions and we can't mount). + +For more information about the immutability aspect of Elemental, see [Immutable rootfs](immutable_rootfs) diff --git a/versioned_docs/version-1.2/toolkit/reference/troubleshooting.md b/versioned_docs/version-1.2/toolkit/reference/troubleshooting.md new file mode 100644 index 000000000..47a3ad62c --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/reference/troubleshooting.md @@ -0,0 +1,73 @@ +--- +title: "Troubleshooting" +sidebar_label: "Troubleshooting" +--- + +{{% pageinfo color="warning"%}} +Section under construction. +{{% /pageinfo %}} + +While building a derivative, or on a running system things can go really wrong, the guide is aimed to give tips while building derivatives and also debugging running systems. + +Don't forget tocheck the known issues for the [release you're using](https://github.com/rancher/elemental-toolkit/issues). + +Before booting, [several kernel parameters](immutable_rootfs) can be used to help during debugging (also when booting an ISO). Those are meant to be used only while debugging, and they might defeat the concept of immutability. + +## Disable Immutability + +By adding `rd.cos.debugrw` to the boot parameters read only mode will be disabled. See [Immutable setup](immutable_rootfs) for more options. + +The derivative will boot into RW mode, that means any change made during runtime will persist across reboots. Use this feature with caution as defeats the concept of immutability. + +`rd.cos.debugrw` applies only to active and passive partitions. The recovery image can't be mutated. + +{{% alert title="Note" %}} +The changes made will persist during reboots but won't persist across upgrades. If you need to persist changes across upgrades in runtime (for example by adding additional packages on top of the derivative image), see [how to apply persistent changes](../customizing/runtime_persistent_changes). +{{% /alert %}} + +## Debug initramfs issues + +As derivative can ship and build their own initrd, the [official debug docs](https://fedoraproject.org/wiki/How_to_debug_Dracut_problems) contains valid information that can be used for troubleshooting. + +For example: + +- `rd.break=pre-mount rd.shell`: Drop a shell before setting up mount points +- `rd.break=pre-pivot rd.shell`: Drop a shell before switch-root + +## Recovery partition + +If you can boot into the system, the recovery partition can be used to reset the state of the active/passive, but can also be used to upgrade to specific images. Be sure to read the [Recovery section in the docs](../getting-started/recovery). + +## Mutating derivative images + +It can be useful to mutate derivative images and commit a container’s file changes or settings into a new image. +This allows you to debug a container by running an interactive shell, and re-use the mutated image in Elemental systems. Generally, it is better to use Dockerfiles to manage your images in a documented and maintainable way. [Read more about creating bootable images](../creating-derivatives/creating_bootable_images). + +Let's suppose we have the derivative original image at `$IMAGE` and we want to mutate it. We will push it later with another name `$NEW_IMAGE` and use it to our node downstream. + +Run the derivative image locally, and perform any necessary change (e.g. add additional software): +```bash +$> docker run --entrypoint /bin/bash -ti --name updated-image $IMAGE +``` + +Commit any changes to a new image `$NEW_IMAGE`: +```bash +$> docker commit updated-image $NEW_IMAGE +``` + +And push the image to the container registry: +```bash +$> docker push $NEW_IMAGE +``` + +In the derivative then it's sufficient to upgrade to that image with `elemental upgrade`: + +```bash +$> elemental upgrade --docker-image $NEW_IMAGE +``` + +## Adding login keys at boot + +To add users key from the GRUB menu prompt, edit the boot cmdline and add the following kernel parameters: + +`stages.boot[0].authorized_keys.root[0]=github:suse` diff --git a/versioned_docs/version-1.2/toolkit/tutorials/_index.md b/versioned_docs/version-1.2/toolkit/tutorials/_index.md new file mode 100644 index 000000000..d9a0a2990 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/tutorials/_index.md @@ -0,0 +1,9 @@ + +--- +title: "Tutorials" +sidebar_label: "Tutorials" +weight: 6 +date: 2017-01-04 +description: > + Elemental tutorials and real life use-case samples +--- diff --git a/versioned_docs/version-1.2/toolkit/tutorials/k3s_and_fleet_on_vanilla_image_example.md b/versioned_docs/version-1.2/toolkit/tutorials/k3s_and_fleet_on_vanilla_image_example.md new file mode 100644 index 000000000..e6508fddf --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/tutorials/k3s_and_fleet_on_vanilla_image_example.md @@ -0,0 +1,109 @@ +--- +title: "K3s + Fleet" +sidebar_label: "K3s and Fleet" +--- + +This is a work in progress example of how to deploy K3S + Fleet + System Upgrade Controller over a Elemental vanilla image only +by using cloud-init yaml configuration files. The config file reproduced here is meant to be included +as a user-data in a cloud provider (aws, gcp, azure, etc) or as part of a cdrom (Elemental-Recovery will try to fetch `/userdata` file +from a cdrom device). + +A vanilla image is an image that only provides the Elemental-Recovery system on a `COS_RECOVERY` partition. It does not include any other +system and it is meant to be dumped to a bigger disk and deploy a Elemental system or a derivative system over the free space in disk. +COS vanilla images are build as part of the CI workflow, see CI artifacts to download one of those. + +The configuration file of this example has two purposes: first it deploys Elemental, second in reboots on the deployed OS and deploys +K3S + Fleet + System Upgrades Controller. + +On first boot it will fail to boot Elemental grub menu entry and fallback +to Elemental-Recovery system. From there it will partition the vanilla image to create the main system partition (`COS_STATE`) +and add an extra partition for persistent data (`COS_PERSISTENT`). It will use the full disk, a disk of at least 20GiB +is recommended. After partitioning it will deploy the main system on `COS_STATE` and reboot to it. + +On consequent boots it will simply boot from `COS_STATE`, there it prepares the persistent areas of the system (arranges few bind +mounts inside `COS_PERSISTENT`) and then it runs an standard installation of K3s, Fleet and System Upgrade Controller. After few +minutes after the system is up the K3s cluster is up and running. + +Note this setup similar to the [derivative example](https://github.com/rancher-sandbox/cos-fleet-upgrades-sample) using Fleet. +The main difference is that this example does not require to build any image, it is pure cloud-init configuration based. + +### User data configuration file +```yaml +name: "Default deployment" +stages: + rootfs.after: + - if: '[ -f "/run/cos/recovery_mode" ]' + name: "Repart image" + layout: + # It will partition a device including the given filesystem label or part label (filesystem label matches first) + device: + label: COS_RECOVERY + add_partitions: + - fsLabel: COS_STATE + # 15Gb for COS_STATE, so the disk should have, at least, 20Gb + size: 15360 + pLabel: state + - fsLabel: COS_PERSISTENT + # unset size or 0 size means all available space + pLabel: persistent + - if: '[ ! -f "/run/cos/recovery_mode" ]' + name: "Persistent state" + environment_file: /run/cos/cos-layout.env + environment: + VOLUMES: "LABEL=COS_OEM:/oem LABEL=COS_PERSISTENT:/usr/local" + OVERLAY: "tmpfs:25%" + RW_PATHS: "/var /etc /srv" + PERSISTENT_STATE_PATHS: "/root /opt /home /var/lib/rancher /var/lib/kubelet /etc/systemd /etc/rancher /etc/ssh" + network.before: + - name: "Setup SSH keys" + authorized_keys: + root: + # It can download ssh key from remote places, such as github user keys (e.g. `github:my_user`) + - my_custom_ssh_key + - if: '[ ! -f "/run/cos/recovery_mode" ]' + name: "Fleet deployment" + files: + - path: /etc/k3s/manifests/fleet-config.yaml + content: | + apiVersion: helm.cattle.io/v1 + kind: HelmChart + metadata: + name: fleet-crd + namespace: kube-system + spec: + chart: https://github.com/rancher/fleet/releases/download/v0.3.3/fleet-crd-0.3.3.tgz + --- + apiVersion: helm.cattle.io/v1 + kind: HelmChart + metadata: + name: fleet + namespace: kube-system + spec: + chart: https://github.com/rancher/fleet/releases/download/v0.3.3/fleet-0.3.3.tgz + network: + - if: '[ -f "/run/cos/recovery_mode" ]' + name: "Deploy cos-system" + commands: + # Deploys the recovery image. + # use --docker-image to deploy a custom image + # e.g. `elemental reset --docker-image quay.io/my_custom_repo:my_image` + - elemental reset --reboot + - if: '[ ! -f "/run/cos/recovery_mode" ]' + name: "Setup k3s" + directories: + - path: "/usr/local/bin" + permissions: 0755 + owner: 0 + group: 0 + commands: + - | + curl -sfL https://get.k3s.io | \ + INSTALL_K3S_VERSION="v1.20.4+k3s1" \ + INSTALL_K3S_EXEC="--tls-san {{.Values.node.hostname}}" \ + INSTALL_K3S_SELINUX_WARN="true" \ + sh - + # Install fleet + kubectl apply -f /etc/k3s/manifests/fleet-config.yaml + # Install system-upgrade-controller + kubectl apply -f https://raw.githubusercontent.com/rancher/system-upgrade-controller/v0.6.2/manifests/system-upgrade-controller.yaml +``` diff --git a/versioned_docs/version-1.2/toolkit/tutorials/trigger_upgrades_with_fleet.md b/versioned_docs/version-1.2/toolkit/tutorials/trigger_upgrades_with_fleet.md new file mode 100644 index 000000000..7e139f219 --- /dev/null +++ b/versioned_docs/version-1.2/toolkit/tutorials/trigger_upgrades_with_fleet.md @@ -0,0 +1,63 @@ +--- +title: "Trigger upgrades with K3s and Fleet" +sidebar_label: "Trigger upgrades with K3s and Fleet" +--- + +![](https://docs.google.com/drawings/d/e/2PACX-1vQPv9TI3D95vocG7oCHmVmNuuvBYuc2_0kaxAc6xnCBM9mFTnUTFIIDkzZKUFFP-xyw2Hg4q9XhxLD8/pub?w=1185&h=712) + +In this tutorial we will: + +1) Build a custom OS image to deploy in our cluster +2) Setup a cluster with Elemental, k3s and fleet +3) Upgrade the cluster to our custom OS image with fleet + +[This repository](https://github.com/rancher-sandbox/cos-fleet-upgrades-sample/) contains the full example code. + +## 1) Build the OS image + +```bash +# IMAGE=quay.io/costoolkit/test-images:fleet-sample +# cd os +# docker build -t $IMAGE . +``` + +## 2) Push the docker image + + +```bash +# docker push $IMAGE +``` + +## 3) Prepare a Elemental VM + +Download an ISO, or a qcow image from the Github artifacts of Elemental. + +If deploying on AWS/openstack/Cloud, use the `fleet-cloud-init.yaml` file as userdata. If deploying on baremetal/VMs, place `fleet-cloud-init.yaml` in `/oem` after install (or run the installer with `elemental install --cloud-init https://raw.githubusercontent.com/rancher-sandbox/cos-fleet-upgrades-sample/main/fleet-cloud-init.yaml $DEVICE`). + +Reboot, after some bootstraping time (check until all pods are running with `watch kubectl get pods -A`), you should have a k3s cluster with fleet and [system-upgrade-controller](https://github.com/rancher/system-upgrade-controller) deployed. + +## 4) Upgrade with fleet + +Add your fleet repository to the fleet cluster: + +```bash +cat > example.yaml << "EOF" +apiVersion: fleet.cattle.io/v1alpha1 +kind: GitRepo +metadata: + name: upgrade + # This namespace is special and auto-wired to deploy to the local cluster + namespace: fleet-local +spec: + # Everything from this repo will be ran in this cluster. You trust me right? + repo: "https://github.com/rancher-sandbox/cos-fleet-upgrades-sample" + branch: "main" + paths: + - manifests +EOF + +kubectl apply -f example.yaml +``` + +An example of how to trigger an upgrade with fleet is in `manifests/upgrade.yaml`. Edit the image with the one generated in the previous steps, and commit it to your **fleet repository**, At this point you should see the upgrade job to kick-in, the system will reboot afterwards. + diff --git a/versioned_sidebars/version-1.2-sidebars.json b/versioned_sidebars/version-1.2-sidebars.json index 1815d316c..f4bca89f3 100644 --- a/versioned_sidebars/version-1.2-sidebars.json +++ b/versioned_sidebars/version-1.2-sidebars.json @@ -109,5 +109,106 @@ ] }, "release-notes" + ], + "toolkit": [ + "toolkit/index", + { + "type": "category", + "collapsible": true, + "collapsed": false, + "label": "Getting Started", + "items": [ + "toolkit/getting-started/download", + "toolkit/getting-started/install", + "toolkit/getting-started/upgrading", + "toolkit/getting-started/recovery", + "toolkit/getting-started/deploy" + ] + }, + { + "type": "category", + "label": "Customizing", + "collapsible": true, + "collapsed": true, + "items": [ + "toolkit/customizing/stages", + "toolkit/customizing/configuration_persistency", + "toolkit/customizing/embedded_features", + "toolkit/customizing/login", + "toolkit/customizing/oem_configuration", + "toolkit/customizing/runtime_persistent_changes", + "toolkit/customizing/general_configuration", + "toolkit/customizing/upgrades", + "toolkit/customizing/configure_grub", + "toolkit/customizing/selinux_support" + ] + }, + { + "type": "category", + "label": "Creating Derivatives", + "collapsible": true, + "collapsed": true, + "items": [ + "toolkit/creating-derivatives/package_stack", + "toolkit/creating-derivatives/creating_bootable_images", + "toolkit/creating-derivatives/build_disk", + "toolkit/creating-derivatives/build_iso" + ] + }, + { + "type": "category", + "label": "Examples", + "items": [ + "toolkit/examples/creating_bootable_images", + "toolkit/examples/cloud_config", + "toolkit/examples/embedded_images" + ] + }, + { + "type": "category", + "label": "Tutorials", + "items": [ + "toolkit/tutorials/k3s_and_fleet_on_vanilla_image_example", + "toolkit/tutorials/trigger_upgrades_with_fleet" + ] + }, + { + "type": "category", + "label": "Reference", + "items": [ + "toolkit/reference/cloud_init", + "toolkit/reference/immutable_rootfs", + "toolkit/reference/layout", + "toolkit/reference/troubleshooting", + "toolkit/reference/built_with_elemental", + "toolkit/reference/high_level_architecture", + { + "type": "category", + "label": "Command Line Interface", + "items": [ + "toolkit/reference/elemental", + "toolkit/reference/elemental_build-iso", + "toolkit/reference/elemental_cloud-init", + "toolkit/reference/elemental_convert-disk", + "toolkit/reference/elemental_exit-codes", + "toolkit/reference/elemental_install", + "toolkit/reference/elemental_new", + "toolkit/reference/elemental_pull-image", + "toolkit/reference/elemental_reset", + "toolkit/reference/elemental_run-stage", + "toolkit/reference/elemental_upgrade", + "toolkit/reference/elemental_version" + ] + } + ] + }, + { + "type": "category", + "label": "Development", + "items": [ + "toolkit/development/creating_derivatives", + "toolkit/development/dependencies" + ] + } ] }