Elektrobit · thir820 · Oct 24, 2024 · Oct 24, 2024 · Oct 24, 2024 · Oct 24, 2024
diff --git a/.gtihub/pull_request_template.md → .github/pull_request_template.md b/.gtihub/pull_request_template.md → .github/pull_request_template.md
diff --git a/.vscode/settings.json b/.vscode/settings.json
@@ -0,0 +1,6 @@
+{
+    "cSpell.language": "en,en-US,en-GB",
+    "cSpell.words": [
+        "Embdgen"
+    ]
+}
diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
@@ -3,3 +3,13 @@
 [EB corbos Linux dev container](README.md)
 
 - [Overview](overview.md)
+- [Testing the container](tests.md)
+- [Layers](layers/index.md)
+    - [Base Layer](layers/base.md)
+    - [PBuilder Layer](layers/pbuilder.md)
+    - [Rust Layer](layers/rust.md)
+    - [Kiwi-ng Layer](layers/kiwi.md)
+    - [AppDev Layer](layers/appdev.md)
+    - [Embdgen Layer](layers/embdgen.md)
+    - [VS Code Layer](layers/vscode.md)
+    - [Elbe Layer](layers/elbe.md)
diff --git a/docs/layers/appdev.md b/docs/layers/appdev.md
@@ -0,0 +1 @@
+# AppDev Layer
diff --git a/docs/layers/base.md b/docs/layers/base.md
@@ -0,0 +1,16 @@
+# Base Layer
+
+The base layer does the basic configuration needed for all additional layers, and installs the common tools. These configurations are:
+
+- [Setup the **ebcl** user](https://github.com/Elektrobit/ebcl_dev_container/blob/9727219bfd009b717d7b29d1eb54d42586e9eebd/layers/base/Dockerfile#L3).
+- [Setup the apt sources](https://github.com/Elektrobit/ebcl_dev_container/blob/9727219bfd009b717d7b29d1eb54d42586e9eebd/layers/base/Dockerfile#L8).
+- [Install the common packages](https://github.com/Elektrobit/ebcl_dev_container/blob/9727219bfd009b717d7b29d1eb54d42586e9eebd/layers/base/Dockerfile#L26).
+- [Make the _addon_ packages available](https://github.com/Elektrobit/ebcl_dev_container/blob/9727219bfd009b717d7b29d1eb54d42586e9eebd/layers/base/Dockerfile#L38).
+- [Configure the expected folder structure](https://github.com/Elektrobit/ebcl_dev_container/blob/9727219bfd009b717d7b29d1eb54d42586e9eebd/layers/base/Dockerfile#L56).
+- [Configure the shell environment and common scripts](https://github.com/Elektrobit/ebcl_dev_container/blob/9727219bfd009b717d7b29d1eb54d42586e9eebd/layers/base/Dockerfile#L70).
+- [Prepare a common Python virtual environment](https://github.com/Elektrobit/ebcl_dev_container/blob/9727219bfd009b717d7b29d1eb54d42586e9eebd/layers/base/Dockerfile#L100).
+
+## Remarks
+
+- The _bashrc_ is used to source a EB corbos Linux environment setup file _/build/bin/ebcl_env_. This file is _layers/base/scripts/bash/ebcl_env_.
+- The base layer provides some helper scripts to generate a user-specific GPG key. See _layers/base/scripts/gpg_.
diff --git a/docs/layers/elbe.md b/docs/layers/elbe.md
@@ -0,0 +1 @@
+# Elbe Layer
diff --git a/docs/layers/embdgen.md b/docs/layers/embdgen.md
@@ -0,0 +1 @@
+# Embdgen Layer
diff --git a/docs/layers/index.md b/docs/layers/index.md
@@ -0,0 +1,17 @@
+# Layers
+
+The different EB corbos Linux dev container features are grouped into separate Docker container layers.
+Each layer should be independent of the other layers.
+The default layers are contained in _docs/layers_.
+
+## Customer specific layers
+
+You can add your own layers by using an own container configuration file.
+The default configuration is contained in _configuration/build_config.yaml_.
+
+To create an own layer follow these steps:
+
+- Prepare your layer, i.e. a Docker image in a separate folder.
+- Copy the configuration from _configuration/build_config.yaml_ and add the path to your layer.
+- Make sure the Python dependencies are available, e.g. by creating a venv and installing the dependencies form _requirements.txt_.
+- Run the build using your configuration: `python builder/build_container.py path/to/your/configuration.yaml`
diff --git a/docs/layers/kiwi.md b/docs/layers/kiwi.md
@@ -0,0 +1 @@
+# Kiwi-ng Layer
diff --git a/docs/layers/pbuilder.md b/docs/layers/pbuilder.md
@@ -0,0 +1,11 @@
+# PBuilder Layer
+
+The pbuilder layer adds support for the Debian pbuilder tool.
+This tool is used for application packaging.
+This layer also provides the helper scripts for application packaging and generating apt repository metadata, see _layers/pbuilder/scripts/pbuilder_ and _layers/pbuilder/scripts/deb_.
+
+## Remarks
+
+- By default, pbuilder fails for Ubuntu because the repository archive.ubuntu.com lists the arm64 architecture, but doesn't provide package indices. This is OK from a specification point of view, but causes an apt exit code != 0, which causes a pbuilder failure. The workaround is _layers/pbuilder/conf/pbuilder/G99apt_arch_, which modifies the apt sources and adds and architecture filter.
+- Pbuilder is configured ot use the Canonical and Elektrobit apt repositories, see _layers/pbuilder/conf/pbuilder/pbuilderrc_.
+- Pbuilder always makes use of the **latest EB corbos Linux release** provided by the public repository. To change this, the file _layers/pbuilder/conf/pbuilder/pbuilderrc_ (_/home/ebcl/.pbuilderrc_) needs to be adapted.
diff --git a/docs/layers/rust.md b/docs/layers/rust.md
@@ -0,0 +1 @@
+# Rust Layer
diff --git a/docs/layers/vscode.md b/docs/layers/vscode.md
@@ -0,0 +1 @@
+# VS Code Layer
diff --git a/docs/overview.md b/docs/overview.md
@@ -1,2 +1,29 @@
 # Overview
 
+The EB corbos Linux dev container provides a stable and versioned built environment 
+for building images and applications. This container is used as base for the 
+EB corbos Linux SDK workspace. It can be used for local development and for automated
+application and image building in a CI environment.
+
+To improve maintainability and variant support, the EB corbos Linux dev container
+is implemented as a set of layers. Each layer is a default Docker container, and
+the layer are stacked on top of each other. The overall container is defined using
+the configuration file _configuration/build_config.yaml_. It defines the base container,
+some metadata like the container version, and the used layers, as list of relative paths
+to the folders containing the containers.
+
+For building the container a Python script, _builder/build_container.py_, is used with takes 
+care of running the Docker builds for the layers, and stacking the resulting containers.
+The dependencies of this script are also contained in _requirements.txt_.
+The shell script _builder/build_container_ takes care that a virtual environment with the 
+dependencies is available and runs the Python script to build the dev container.
+
+At the moment, the container is distributed as exported image, and this image is generated
+using _builder/export_container_.
+
+There are also some helper scripts which supports and demos how to use the container stand-alone.
+These scripts are contained in _usage_.
+The script _usage/run_container_ runs the dev container for interactive usage.
+The script _usage/run_background_ runs the container in the background.
+This script is also used for the robot tests.
+The script _usage/stop_container_ stops a dev container running in the background.
diff --git a/docs/tests.md b/docs/tests.md
@@ -0,0 +1,16 @@
+# Testing the dev container
+
+## Tooling tests
+
+The image build tests are contained in _tests_.
+You can run these tests by running the _tests/run_tests_ script.
+This script will ensure that a proper Python virtual environment is available
+and then run all robot tests contained in _tests_.
+
+## Image build tests
+
+To ensure a good test coverage, also the build tests for the EB corbos Linux template workspace should be used.
+Running these tests is supported by the shell script _tests/image_build_tests/run_tests_.
+This script clones the current EB corbos Linux template workspace,
+mounts is into the latest local built dev container,
+and runs the image build robot tests from the EB corbos Linux template workspace in this container.
diff --git a/layers/base/Dockerfile b/layers/base/Dockerfile
@@ -82,12 +82,6 @@ COPY scripts/bash/* /build/bin/
 # Copy GnuPG helper scripts
 COPY scripts/gpg/* /build/bin/
 
-# Copy QEMU helper scripts
-COPY scripts/qemu/* /build/bin/
-
-# Copy mount helper scripts
-COPY scripts/mount/* /build/bin/
-
 # Ensure correct attributes for scripts
 RUN chown -R $CONTAINER_USER:$CONTAINER_USER /build/bin && \
     chmod +x /build/bin/*

diff --git a/layers/base/scripts/bash/ebcl_env b/layers/base/scripts/bash/ebcl_env
@@ -45,13 +45,8 @@ if [ -f "$ENV" ]; then
     source $ENV
 fi
 
-if [ ! -d "/tmp/build/images" ]; then
-    bash -c "overlay_mount /build/images"
-fi
-
 source /build/venv/bin/activate
 
-
 # Run user setup
 if [ -d "/workspace/tools/user_config" ]; then
     echo "Running user extensions..."

diff --git a/layers/base/scripts/mount/overlay_mount b/layers/base/scripts/mount/overlay_mount
diff --git a/layers/base/scripts/qemu/qemu_aarch64 b/layers/base/scripts/qemu/qemu_aarch64
diff --git a/layers/base/scripts/qemu/qemu_efi_aarch64 b/layers/base/scripts/qemu/qemu_efi_aarch64
diff --git a/layers/base/scripts/qemu/qemu_efi_x86_64 b/layers/base/scripts/qemu/qemu_efi_x86_64
diff --git a/layers/base/scripts/qemu/qemu_x86_64 b/layers/base/scripts/qemu/qemu_x86_64
diff --git a/layers/pbuilder/conf/pbuilder/pbuilderrc b/layers/pbuilder/conf/pbuilder/pbuilderrc
@@ -1,7 +1,7 @@
 HOOKDIR="/home/ebcl/.pbuilder/hooks/"
 DISTRIBUTION=jammy
 MIRRORSITE=http://archive.ubuntu.com/ubuntu
-OTHERMIRROR="deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports jammy main universe | deb [arch=amd64] http://archive.ubuntu.com/ubuntu jammy-security main universe | deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports jammy-security main universe | deb http://linux.elektrobit.com/eb-corbos-linux/1.1.1 ebcl prod dev"
+OTHERMIRROR="deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports jammy main universe | deb [arch=amd64] http://archive.ubuntu.com/ubuntu jammy-security main universe | deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports jammy-security main universe | deb http://linux.elektrobit.com/eb-corbos-linux/latest ebcl prod dev"
 PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends-apt"
 BUILDRESULT=/build/result_app
 COMPONENTS="main universe"