` (see src/main.ts in the [TypeScript module template](https://github.com/bitfocus/companion-module-template-ts)).
+
+Secrets is a new feature (since [API 1.13 - Companion 4.1](../api-changes/v1.13#secrets)). By defining a field as a secret, Companion will store its values in a separate secrets object. This allows Companion to be more careful in avoiding logging of this object, and allows the user to easily omit these values when exporting their configuration, which is beneficial for sharing with others.
+
+The fields available for secrets is quite limited, as we expect it to only be useful for api keys, usernames, passwords and similar things. If other field types is useful, let us know and we can look at adding more.
+
+## API calls: `getConfigFields()`, `saveConfig()`, `configUpdated()`
+
+As part of creating a module, you should implement the [`getConfigFields()` method](https://bitfocus.github.io/companion-module-base/classes/InstanceBase.html#getconfigfields).
+
+Companion will call this when the configuration panel is opened for your module, so that it can present the correct fields to the user. See the next section, below, for details.
+
+When your module is initialised, you will be provided a copy of the config in the [`init()`](https://bitfocus.github.io/companion-module-base/classes/InstanceBase.html#init) method, and any time the user changes the configuration, [`configUpdated()`](https://bitfocus.github.io/companion-module-base/classes/InstanceBase.html#configupdated) will be called.
+
+If you need to programmatically change the config object, for example to save some persistent state or to allow actions to change the config, call `saveConfig(newconfig)`. (This will not trigger `configUpdated()`.)
+
+:::tip
+
+The `saveConfig()`, `configUpdated()` and `init()` methods also provide or accept a secrets object when you are defining any secret fields.
+
+:::
+
+## User-Config definitions
+
+The fields you can use here are similar to the ones for actions and feedbacks, but with more limitations. See the [list of field types](./input-field-types.md) for more details. The linked documentation states any limitations that apply when used for the configuration, or if they are not allowed.
+
+The contents of the config and secrets objects must be JSON safe values, due to how they are stored. Make sure to not try and use a non-json safe object or it either won't be saved, or will throw an error that can crash your module.
+
+### Layout (deprecated)
+
+:::tip
+
+Since API v1.14, ee are working to unify the layout of this configuration to match elsewhere in Companion, meaning this value is not respected by default.
+
+At the moment (in Companion v4.2/v4.3) it is possible to opt into the old layout, if you are not ready to ensure this works well for your module, you can opt out by adding in the constructor `this.options.disableNewConfigLayout = true`. This should only be done as a temporary measure, at some point in the future this will not be supported.
+
+If you do this, let us know what is missing for you to switch. We recognise that there may be functionality you need and want to expand upon what the new layout offers. Reach out on [Github](https://github.com/bitfocus/companion/issues) to let us know what you need to be able to migrate.
+
+:::
+
+Each field is required to have a `width` property. This should be a value 1-12, specifying how many columns the field needs in the UI.
+
+### Device discovery
+
+If you are connecting over the network, your device may be discoverable using the Bonjour protocol. See the advanced topic [Bonjour Device Discovery](../connection-advanced/bonjour-device-discovery.md) for further details.
+
+If your device uses a different discovery mechanism, we would like to hear about it so that we can expand this support for more devices. Reach out on [Github](https://github.com/bitfocus/companion/issues)
+
+## Further reading
+
+- [Autogenerated docs for the module `InstanceBase` class](https://bitfocus.github.io/companion-module-base/classes/InstanceBase.html)
+- [Input-field Types (Options)](./input-field-types.md)
+- [Bonjour Device Discovery](../connection-advanced/bonjour-device-discovery.md)
+- [Upgrade scripts](./upgrade-scripts.md)
diff --git a/for-developers/module-development/connection-basics/variables.md b/for-developers/module-development/connection-basics/variables.md
new file mode 100644
index 0000000..adecd4a
--- /dev/null
+++ b/for-developers/module-development/connection-basics/variables.md
@@ -0,0 +1,69 @@
+---
+title: Module Variable Definitions
+sidebar_label: Variables
+sidebar_position: 19
+description: Module variable definition details.
+---
+
+Variables are a way for modules to expose values to the user, which can be used as part of the button text, as input to some actions or feedbacks and more. This section explains how to define variables and update their values.
+
+The basic workflow is to define your variables using `setVariableDefinitions()`, then set or update the values using `setVariableValues()`. Both of these methods are defined by the module InstanceBase class.
+
+## API call: `setVariableDefinitions()`
+
+Your module should define the list of variables it exposes by making a call to `this.setVariableDefinitions({ ...some variables here... })`. You will need to do this as part of your `init()` method, but can also call it at any other time if you wish to change the list of variables exposed.
+
+:::warning
+Please try not to call this method too often, as updating the list has a cost. If you are calling it multiple times in a short span of time, consider if it would be possible to batch the calls so it is only done once.
+:::
+
+## Variable definitions
+
+The [TypeScript module template](https://github.com/bitfocus/companion-module-template-ts) includes a file `src/variables.ts`, which is where your variables should be defined. It is not required to use this structure, but it keeps it more readable than having everything in one file. More complex modules will likely want to split the variable definitions into even more files/folders.
+
+All the variable definitions are passed in as a single javascript array, in the form of:
+
+```js
+;[
+ { variableId: 'variable1', name: 'My first variable' },
+ { variableId: 'variable2', name: 'My second variable' },
+ { variableId: 'variable3', name: 'Another variable' },
+]
+```
+
+:::important
+
+VariableId must only use letters [a-zA-Z], numbers, underscore, hyphen.
+
+:::
+
+## API call: `setVariableValues()`
+
+At any point in your module you can call `this.setVariableValues({ ... new values ... })`. You can specify as many or few variables as you wish in this call. Only the variables you specify will be updated.
+
+For example:
+
+```js
+this.setVariableValues({
+ 'variable1': 'new value'
+ 'variable2': 99,
+ 'old_variable': undefined, // This unsets a value
+ 'array_variable': [1, 2, 3, 4],
+})
+```
+
+Variables can have values of any type, the user can use expressions to manipulate the values you provide.
+
+:::warning
+
+Please try to batch variable updates whenever possible, as updating the values has a cost. If you are calling it multiple times in a short span of time, consider if it would be possible to batch the calls so it is only done once.
+
+:::
+
+## Further Reading
+
+- [Autogenerated docs for the module `InstanceBase` class](https://bitfocus.github.io/companion-module-base/classes/InstanceBase.html)
+- [User-configuration management](./user-configuration.md)
+- [Actions](./actions.md)
+- [Feedbacks](./feedbacks.md)
+- [Presets](./presets.md)
diff --git a/for-developers/module-development/home.md b/for-developers/module-development/home.md
index 1d9aea2..fa17b61 100644
--- a/for-developers/module-development/home.md
+++ b/for-developers/module-development/home.md
@@ -1,5 +1,5 @@
---
-title: Module Development Setup
+title: Getting Started with Modules
sidebar_label: Getting Started with Modules
sidebar_position: 1
description: Developer environment setup specific to module development.
@@ -53,9 +53,9 @@ With over 700 published modules, you may find a module already written for you d
1. Rename your module's directory _companion-module-mymanufacturer-myproduct_, replacing _mymanufacturer-myproduct_ with appropriate names. Try to think of what is most appropriate for your device: Are there other similar devices by the manufacturer that use the same protocol that the module could support later on? If so try and name it to more easily allow for that.
-2. In at least _package.json_, _companion/manifest.json_ and _companion/HELP.md_, edit the name and description of the module to match what yours is called. The search feature in your IDE is really helpful to find all of the places the name shows up! See the [Module Configuration section](./module-config/file-structure.md) and especially the [documentation for the manifest.json file](./module-config/manifest.json.md) for further details.
+2. In at least _package.json_, _companion/manifest.json_ and _companion/HELP.md_, edit the name and description of the module to match what yours is called. The search feature in your IDE is really helpful to find all of the places the name shows up! See the [Module Configuration section](./module-setup/file-structure.md) and especially the [documentation for the manifest.json file](./module-setup/manifest.json.md) for further details.
-3. Please see [Module Configuration](./module-config/file-structure.md) and the other pages in that section for more details and more options on starting your own module.
+3. Please see [Module Configuration](./module-setup/file-structure.md) and the other pages in that section for more details and more options on starting your own module.
## Install the dependencies
@@ -73,7 +73,7 @@ If you are using an IDE such as [VS Code](https://code.visualstudio.com/), make
You are now ready to start developing your module. Here are our suggested next steps:
-- Familiarize yourself with the [Module Configuration](module-config/file-structure.md) to understand the general file structure and configuration options, especially if working on a new module.
+- Familiarize yourself with the [Module Configuration](module-setup/file-structure.md) to understand the general file structure and configuration options, especially if working on a new module.
- Read [Module development 101](./module-development-101.md) for an overview of the development lifecycle.
- Review the recommended [GitHub Workflow](../git-workflows/github-workflow.md) to learn best practices for new features to your codebase.
diff --git a/for-developers/module-development/images/bonjour.png b/for-developers/module-development/images/bonjour.png
new file mode 100644
index 0000000..7ab6839
Binary files /dev/null and b/for-developers/module-development/images/bonjour.png differ
diff --git a/for-developers/module-development/index.md b/for-developers/module-development/index.md
new file mode 100644
index 0000000..c1293af
--- /dev/null
+++ b/for-developers/module-development/index.md
@@ -0,0 +1,8 @@
+---
+title: Module Development Files
+description: Outline of the Module Development section.
+auto_toc: 2
+---
+
+This section describes everything you need to know to develop your own modules for Companion. Below is
+an outline of the top-level pages and folders in this section. For pages, the main headings inside each file are listed as bulleted lines. Folders are show preceded by '> ', and the immediate contents of that folder are shown below it using "└─ " to indicate pages (or subfolders) inside that folder.
diff --git a/for-developers/module-development/module-config/_category_.json b/for-developers/module-development/module-config/_category_.json
deleted file mode 100644
index 9639e8b..0000000
--- a/for-developers/module-development/module-config/_category_.json
+++ /dev/null
@@ -1,11 +0,0 @@
-{
- "label": "Module Configuration",
- "position": 10,
- "link": {
- "type": "generated-index",
- "title": "Configuring your companion module"
- },
- "customProps": {
- "description": "The files and file structure necessary to configure a module repository."
- }
-}
diff --git a/for-developers/module-development/module-debugging.md b/for-developers/module-development/module-debugging.md
new file mode 100644
index 0000000..a62155b
--- /dev/null
+++ b/for-developers/module-development/module-debugging.md
@@ -0,0 +1,93 @@
+---
+title: Debugging Modules
+sidebar_label: Debugging Modules
+sidebar_position: 7
+description: How to debug your Companion module during development.
+---
+
+Once you've started coding, debugging the code will be come essential to your success. We consider it
+important enough that we're covering it here before you dive into the details in the following sections.
+
+There are three main routes to debugging: log through the API, console.log, interactive debugging
+
+## Log through the API
+
+Companion provides individual log pages for each module that are different from the main Companion log page.
+
+You can open the module specific log view from the connections page by clicking the ⋮ button and selecting "View logs":
+
+
+
+The module InstanceBase class provides a logging function that allows you to specify the log-level as one of: `"info"`, `"warn"`, `"error"`, or `"debug"`. For example:
+
+```ts
+this.log('debug', 'My debug message.')
+```
+
+These messages will show up in the module-specific log page and can be filtered by selecting/deselecting the "Debug" button (or whichever log-level you chose) in the top-left of the window:
+
+
+> 26.01.01 00:00:00 **Module**: My debug message.
+
+
+## Log to the console
+
+The simplest debugging method is to log to the console.
+
+`console.log('your data/message');`
+
+These messages will still show up in the module-specific log page and can be filtered by selecting/deselecting the "Console" button in the top-left of the window:
+
+
+> 26.01.01 00:00:00 **Console**: your data/message
+
+
+:::note
+Depending on buffering, several `console.log()` messages may be grouped together, so only the first will appear to have a timestamp. Use the API `this.log()` function described above, if seeing things reported in exactly the call order is important.
+:::
+
+## Attach a debugger
+
+Often it can be useful and more convenient to attach a debugger to your module so you can create breakpoints from which you can inspect its state while the module is running. This method also has the advantage of not requiring the addition of numerous logging statements.
+
+To attach to your module's process, first add a file named _DEBUG-INSPECT_ (no suffix) in the root of your module folder. The file can be empty or have a single number in it, which specifies the debugger port number. Next time you start Companion, it will launch your module with the remote debugging protocol enabled.
+
+By default Companion will pick and use a random port that will change each launch, alternatively, you can specify a port number inside the `DEBUG-INSPECT` file to use a fixed port for debugging.
+
+You can use any compatible debugger such as the builtin VS Code debugger, or Chrome inspector to connect to your process.
+
+:::warning
+
+It may not be possible to debug the `init` method from your module with this, Companion still imposes the same launch timeout as usual here. But you can attach after this and see what is going on.
+
+:::
+
+:::tip[VS Code]
+
+VS Code users can store a setup which allows you to use F5 to initiate debugging as follows:
+
+1. Put a port number in DEBUG-INSPECT -- for this example it is 12345.
+
+2. Put the following into the file _.vscode/launch.json_ (where the value of "port" matches the value in DEBUG-INSPECT).
+
+```json
+{
+ "configurations": [
+ {
+ "name": "Attach to Companion module",
+ "port": 12345,
+ "request": "attach",
+ "skipFiles": ["/**"],
+ "type": "node"
+ }
+ ]
+}
+```
+
+:::
+
+## Further reading
+
+- [Autogenerated docs for the InstanceBase log method](https://bitfocus.github.io/companion-module-base/classes/InstanceBase.html#log)
+- [Module development 101](./module-development-101.md)
+- [Module Basics Overview](./connection-basics/overview.md)
diff --git a/for-developers/module-development/module-development-101.md b/for-developers/module-development/module-development-101.md
index 9e2035d..4ad02a5 100644
--- a/for-developers/module-development/module-development-101.md
+++ b/for-developers/module-development/module-development-101.md
@@ -15,18 +15,18 @@ If you are creating a new module, then the very first thing you'll want to do is
## Configure the module
-There are a few files that make up every module. Please familiarize yourself with the basic structure described in our pages on [Module Configuration](module-config/file-structure). In particular, _package.json_,
-[_companion/manifest.json_](./module-config/manifest.json.md) and _companion/HELP.md_ define the identity of the module. Once these are defined, you will spend most of your time crafting the module source code.
+There are a few files that make up every module. Please familiarize yourself with the basic structure described in our pages on [Module Configuration](module-setup/file-structure). In particular, _package.json_,
+[_companion/manifest.json_](./module-setup/manifest.json.md) and _companion/HELP.md_ define the identity of the module. Once these are defined, you will spend most of your time crafting the module source code.
## Program the module
-While you can handle all your module's code in one big file, we strongly recommend splitting it across several files as illustrated [here](./module-config/file-structure#file-structure).
+While you can handle all your module's code in one big file, we strongly recommend splitting it across several files as illustrated in our [file structure overview](./module-setup/file-structure#file-structure).
To understand what is needed in a module it helps to understand how the code is used. Your module is presented to Companion as a class that extends the module base class. A user can add one or more _instances_ of your module to their Companion site. When Companion starts up, it initializes each instance of the module by starting a new process and passing configuration information, as described next.
### The module class and entrypoint (Module base class, configs)
-In the [typical module structure](./module-config/file-structure#file-structure), the entrypoint and module class are defined in _src/main.ts_. When your module is started, first the `constructor` of your module's class will be called, followed by your [upgrade scripts](https://github.com/bitfocus/companion-module-base/wiki/Upgrade-scripts) and then the `init` method.
+In the [typical module structure](./module-setup/file-structure#file-structure), the entrypoint and module class are defined in _src/main.ts_. When your module is started, first the `constructor` of your module's class will be called, followed by your [upgrade scripts](./connection-basics/upgrade-scripts.md) and then the `init` method.
Your constructor should only do some minimal class setup. It does not have access to the configuration information, so it should not be used to start doing things. Instead...
@@ -40,13 +40,13 @@ When the module gets deleted or disabled the `destroy` function is called. here
Your module provides interaction with the user by defining user-configurations, actions, feedbacks, and variables. In addition you can define "preset" buttons that predefine combinations of common actions and feedbacks for the user's convenience. These presets can be dragged onto the button grid for "instant button configuration".
-TODO: Update links
-
-- [Module Configuration](module-config/file-structure.md)
-- [Actions](https://github.com/bitfocus/companion-module-base/wiki/Actions)
-- [Feedbacks](https://github.com/bitfocus/companion-module-base/wiki/Feedbacks)
-- [Presets](https://github.com/bitfocus/companion-module-base/wiki/Presets)
-- [Variables](https://github.com/bitfocus/companion-module-base/wiki/Variables)
+- [Module Setup](module-setup/file-structure.md)
+- [Module Basics Overview](./connection-basics/overview.md)
+- [Actions](./connection-basics/actions.md)
+- [Feedbacks](./connection-basics/feedbacks.md)
+- [Presets API v2.x](./connection-basics/presets.md)
+- [Presets API v1.x](./connection-basics/presets-1.x.md)
+- [Variables](./connection-basics/variables.md)
### Log module activity (optional)
@@ -58,6 +58,8 @@ For printing to the module debug log use:
And if you want it in the log in the web interface, see [the log method](https://bitfocus.github.io/companion-module-base/classes/InstanceBase.html#log).
+See also our instructions for [debugging your module](./module-debugging.md)
+
## Test the module
In any case, your module should be tested throughout at different stages of its life.
@@ -66,8 +68,8 @@ And last but not least you should check **all** your actions with **all** the op
## Share your code
-Once your module is tested and you are ready to release it publicly, you will use the [BitFocus Developer Portal](https://developer.bitfocus.io/modules/companion-connection/discover) to list it with Companion. Please follow the guide for [releasing your module](https://github.com/bitfocus/companion-module-base/wiki/Releasing-your-module).
+The first step in sharing your code, whether to share privately or distribute through Companion is to [package your module](./module-lifecycle/module-packaging.md).
-If your module it not intended for public release, or you want to share it locally for testing, you can also read the guide on [packaging your module](https://github.com/bitfocus/companion-module-base/wiki/Module-packaging).
+Once your packaged module has passed your quality-control and you are ready to release it publicly, you will use the [BitFocus Developer Portal](https://developer.bitfocus.io/modules/companion-connection/discover) to list it with Companion. Please follow the guide for [releasing your module](./module-lifecycle/releasing-your-module.md).
Questions? Reach out on [SLACK](https://bfoc.us/uu1kmq6qs4)! :)
diff --git a/for-developers/module-development/module-lifecycle/_category_.json b/for-developers/module-development/module-lifecycle/_category_.json
index 8a18914..8f53358 100644
--- a/for-developers/module-development/module-lifecycle/_category_.json
+++ b/for-developers/module-development/module-lifecycle/_category_.json
@@ -1,11 +1,11 @@
{
- "label": "Module LifeCycle",
+ "label": "Module Release and Maintenance",
"position": 30,
"link": {
- "type": "generated-index",
- "title": "Maintaining you Companion module over time"
+ "type": "doc",
+ "id": "index"
},
"customProps": {
- "description": "The task necessary to maintain and upgrade a module repository over time."
+ "description": "The task necessary to release, maintain and upgrade a module repository over time."
}
}
diff --git a/for-developers/module-development/module-lifecycle/companion-module-library.md b/for-developers/module-development/module-lifecycle/companion-module-library.md
index 0770e34..678a015 100644
--- a/for-developers/module-development/module-lifecycle/companion-module-library.md
+++ b/for-developers/module-development/module-lifecycle/companion-module-library.md
@@ -1,7 +1,7 @@
---
title: 'The Companion Module Libraries'
sidebar_label: '@companion-module Libraries'
-sidebar_position: 7
+sidebar_position: 1
description: Explanation of the companion module library.
---
@@ -9,7 +9,7 @@ Since Companion version 3.0, we have used `@companion-module/base` and `@compani
The libraries are available on [npm](https://www.npmjs.com/) and are installed automatically when you run `yarn install`.
-## What is the purpose of each?
+## What is the purpose of each library?
### @companion-module/base
@@ -48,3 +48,9 @@ This was the main issue that was blocking our ability to support adding newer ve
Everything that modules need to access Companion is now located inside of `@companion-module/base`. Some things have not been made available, as they were determined to not be appropriate and alternatives will be recommended to the few module authors who utilised them.
Another benefit of this separation is that it allows us to better isolate modules from being tied to specific versions of Companion. The `@companion-module/base` acts as a stable barrier between the two. It has intentionally been kept separate from the rest of the Companion code, so that changes made here get an extra level of scrutiny, as we want to guarantee backwards compatibility as much as possible. For example, in Companion version 2.x changes made to the module api occasionally required fixes to be applied to multiple modules. By having this barrier, we can avoid such problems for as long as possible, and can more easily create compatibility layers as needed.
+
+## Further reading
+
+- [Introduction to modules](../home.md)
+- [Module development 101](../module-development-101.md)
+- [Module Basics](../connection-basics/)
diff --git a/for-developers/module-development/module-lifecycle/index.md b/for-developers/module-development/module-lifecycle/index.md
new file mode 100644
index 0000000..51294ab
--- /dev/null
+++ b/for-developers/module-development/module-lifecycle/index.md
@@ -0,0 +1,7 @@
+---
+title: 'Module Development Lifecycle: Release and Maintenance'
+description: The task necessary to release, maintain and upgrade a module repository over time.
+auto_toc: 2
+---
+
+This section describes the task necessary to package, deliver, maintain and upgrade a module repository over time.
diff --git a/for-developers/module-development/module-lifecycle/module-packaging.md b/for-developers/module-development/module-lifecycle/module-packaging.md
new file mode 100644
index 0000000..52c209d
--- /dev/null
+++ b/for-developers/module-development/module-lifecycle/module-packaging.md
@@ -0,0 +1,154 @@
+---
+title: 'Packaging a Companion Module'
+sidebar_label: 'Package a module'
+sidebar_position: 2
+description: How to package your module for delivery to others.
+---
+
+## Background
+
+Starting with Companion 3.0, modules must be packaged with some special tooling. This is done to reduce the number and total size of files in your module. Combining your module into just a few files can often reduce the size from multiple mb, to a few hundred kb, lead to much shorter load times.
+
+:::warning
+
+Sometimes the build process introduces or reveals issues that prevent the module from running, so be sure to test it before distributing. In our experience, issues often occur when working with files from disk, or introducing a new dependency that doesn't play nice.
+
+:::
+
+## Packaging for testing
+
+If you are using one of our [recommended module templates](../module-setup/file-structure.md) you can package your module for distribution by running
+
+```bash
+ yarn companion-module-build --dev
+```
+
+If successful, there will now be a `pkg/` folder at your module root folder a `.tgz` file in the root folder. The module name of version are taken from your _package.json_ file, so for example, if the module is named 'generic-animation' and the version number in _package.json_ is 0.70, then the file will be named _generic-animation-0.7.0.tgz_.
+
+If you need to debug the package code rather than the dev code, create an empty file `DEBUG-PACKAGED` in your module folder. Companion will read the code from `pkg` instead of your source folders.
+
+You probably don't need to do a very thorough test, as long as it starts and connects to your device and a couple of actions work it should be fine.
+
+:::tip
+
+Due to how the packaging is done, it can result in some errors producing unreadable stack traces, or for a wall of code to be shown in the log making it unreadable. While using `DEBUG-PACKAGED`, if you run `yarn companion-module-build --dev` (the `--dev` parameter is key here) it will produce a larger build of your module that will retain original line numbers and formatting, making it much easier to read any output.
+
+:::
+
+If you need help, don't hesitate to reach out on the Module Developer's [Slack channel](https://bfoc.us/ke7e9dqgaz) and we will be happy to assist you.
+
+## Packaging for distribution
+
+When you run `yarn companion-module-build` (without the --dev), it produces the _.tgz_ file described above. This file contains everything a user needs to be able to run your module. A `tgz` file is like a `zip` file, but different encoding. You can extract it into an [appropriate folder](../local-modules.md) or load it from the Companion Modules page (starting with Companion 4.0) and Companion will be able to run it.
+
+## Customising the packaging
+
+For more complex modules, it is possible to adjust some settings in the packaging process.
+
+Be careful when using these, as changing these settings are advanced features that can cause the build process to fail.
+
+To start off, create a file `build-config.cjs` in your module folder, with the contents `module.exports = {}`. Each of these customisation sections will add some data to this file.
+
+### Changing `__dirname`
+
+Webpack has a few options for how `__dirname` behaves in packaged code. By default it gets replaced with `/` which makes everything relative to the bundled main file. Alternatively, you can set `useOriginalStructureDirname: true` and the original paths will be preserved.
+
+### Including extra data files
+
+Sometimes it can be useful to store some extra data as text files, or other formats on disk. Once your module is packaged, it wont have access to any files, from the repository unless they are javascript which gets included in the bundle or the files are explicitly copied. You will need to do this to allow any `fs.readFile()` or similar calls to work.
+
+You can include these files by adding something like the following to your `build-config.cjs`:
+
+```js
+module.exports = {
+ extraFiles: ['*.txt'],
+}
+```
+
+You can use any glob pattern to define files to be copied.
+All files will be copied to the root folder of the package, which is the same folder where the packaged main script is in. Make sure that there are no name conflicts when copying files from different folders.
+Make sure you don't copy files you don't need, as these files will be included in the installation for all users of Companion.
+
+### Using native dependencies
+
+Native dependencies are not possible to bundle in the same way as js ones. So to support these requires a bit of extra work on your part.
+
+It is not yet possible to use all native dependencies. We only support ones who publish prebuilt binaries as part of their npm package.
+This means that `sharp` and `node-hid` are not yet possible to use. Reach out if you are affected by this, we would appreciate some input. (`node-hid` has a PR to resolve this, and elsewhere in Companion we are using a fork base on this PR)
+
+To support these modules, you should make one of two changes to your build-config.cjs, depending on how the library works.
+
+If the library is using [`prebuild-install`](https://www.npmjs.com/package/prebuild-install), then it will not work. With `prebuild-install` it is only possible to have the binaries for one platform installed, which isn't compatible with our bundling process. If you need one of these libraries, let us know and we can try and get this working.
+
+If the library is using [`pkg-prebuilds`](https://www.npmjs.com/package/pkg-prebuilds) for loading the prebuilt binaries, then you can use the following syntax.
+
+```js
+module.exports = {
+ prebuilds: ['@julusian/freetype2'],
+}
+```
+
+If the library is using `node-gyp-build`, then there are a couple of options.
+The preferred method is to set `useOriginalStructureDirname: true` in `build-config.cjs`. This changes the value of `__dirname` in your built module, and allows `node-gyp-build` to find its prebuilds.
+
+If you are not able to use `useOriginalStructureDirname: true`, then you can instead mark the dependency as an external:
+
+```js
+module.exports = {
+ externals: [
+ {
+ 'node-hid': 'commonjs node-hid',
+ },
+ ],
+}
+```
+
+This isn't the most efficient solution, as it still results in a lot of files on disk. We are looking into whether we can package them more efficiently, but are currently blocked on how most of these dependencies locate the native portion of themselves to load.
+
+If the library is using something else, let us know which of these approaches works, and we can update this page to include it.
+
+### Using extra plugins
+
+Sometimes the standard webpack functionality is not enough to produce working modules for the node runtime, but there is a [webpack plugin](https://webpack.js.org/plugins/) which tackles your problem.
+
+You can include additional plugins by adding something like the following to your `build-config.cjs`:
+
+```js
+module.exports = {
+ plugins: [new webpack.ProgressPlugin()],
+}
+```
+
+### Disabling minification
+
+Some libraries can break when minified, if they are relying on names of objects in a way that webpack doesn't expect. This can lead to cryptic runtime errors.
+
+You can disable minification (module-tools >=v1.4) with:
+
+```js
+module.exports = {
+ disableMinifier: true,
+}
+```
+
+Alternatively, if you are having issues with error reports from users that have unreadable stack traces due to this minification, it can be disabled. We would prefer it to remain on for all modules to avoid bloating the install size (it can triple the size of a module), we do not mind modules enabling if it they have a reason to.
+
+### Using worker threads
+
+Worker threads need their own entrypoints, and so need their own built file to execute.
+
+We need to investigate how to handle this correctly. Reach out if you have ideas.
+
+## Common issues
+
+This process can often introduce some unexpected issues, here are some of the more common ones and solutions:
+
+_TODO_
+
+## Further reading
+
+- [Introduction to modules](../home.md)
+- [Module development 101](../module-development-101.md)
+- [Debugging Modules](../module-debugging.md)
+- [Module Setup](../module-setup/)
+- [Module Basics](../connection-basics/)
diff --git a/for-developers/module-development/module-lifecycle/releasing-your-module.md b/for-developers/module-development/module-lifecycle/releasing-your-module.md
new file mode 100644
index 0000000..2fce233
--- /dev/null
+++ b/for-developers/module-development/module-lifecycle/releasing-your-module.md
@@ -0,0 +1,49 @@
+---
+title: 'Releasing a Companion Module'
+sidebar_label: 'Release a module'
+sidebar_position: 3
+description: How to release your module for delivery to others using Companion's "web store".
+---
+
+_TODO: Update this whole page?_
+
+## First Release
+
+If this is the first release of your module, you will need to request the repository on [Slack](https://bfoc.us/ke7e9dqgaz).
+
+Please post a message in the `#module-development` channel the includes your GitHub username and the desired name of your module in the `manufacturer-product` format.
+
+_TODO: This or the previous or both?_
+
+You will need to use the [BitFocus Developer Portal](https://developer.bitfocus.io/modules/my-list) to list it with Companion.
+
+## Releasing a New Version
+
+When a new version of your module has been tested and is ready for distribution, use the following guide to submit it for review:
+
+1. **Update `package.json` version**
+ - Use the `major.minor.patch` format (e.g., `1.2.3`).
+ - Refer to the [Versioning of Modules guide](../../git-workflows/versioning.md#version-of-modules) for details.
+
+2. **(Optional) Update `companion/manifest.json` version**
+ - This is not required; the build process will override this value with the version from the `package.json`.
+
+3. **Create a Git tag**
+ - Prefix the version number with `v` (e.g., `v1.2.3`).
+ - You can create the tag by:
+ - [Creating a release on GitHub](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository#creating-a-release), or
+ - [Creating a local tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging) and pushing it to the repository.
+
+4. **Submit the new version in the [Bitfocus Developer Portal](https://developer.bitfocus.io/)** (login with GitHub).
+ - In the sidebar, go to **My Connections** and select the module.
+ - Click **Submit Version** (bottom of page).
+ - Choose the Git Tag to submit.
+ - (Optional) Check **Is Prerelease** if this is a beta release.
+ - Click **Submit** to send for review. The version status should show "Pending" after this.
+
+**Notes about the review process:**
+
+- Only versions submitted to the developer portal will be reviewed.
+- Reviews are done by volunteers, so review time depends on member availability and the complexity of code changes to be reviewed.
+- If adjustments are needed, you'll receive feedback through the developer portal.
+- Once approved, the new version of the module becomes immediately available to download for users on Companion v4.0.0+.
diff --git a/for-developers/module-development/module-lifecycle/renaming-your-module.md b/for-developers/module-development/module-lifecycle/renaming-your-module.md
new file mode 100644
index 0000000..c965e57
--- /dev/null
+++ b/for-developers/module-development/module-lifecycle/renaming-your-module.md
@@ -0,0 +1,32 @@
+---
+title: 'Renaming a Companion Module'
+sidebar_label: 'Rename a module'
+sidebar_position: 4
+description: How to rename your module after having released it.
+---
+
+Occasionally you will need to rename a module, perhaps for example, to make the name more inclusive as you add more devices, or the manufacturer releases a new device.
+
+_TODO: update this. I'm pretty sure it's still possible_
+
+## This process needs a rework
+
+Due to some technical changes on the module inclusion process, this flow may not be possible anymore. We need to review it and reconsider what is possible and allowed
+
+---
+
+1. Make sure you are happy for any changes in the main branch of the module to be included into beta builds of Companion
+
+2. Ask in the module-development slack for approval on the new name
+ This is so that we can be sure the new name conforms to our standard structure of `companion-module-manufacturer-product` (or `companion-module-manufacturer-protocol`).
+
+3. Once approved, the team will be able to rename the GitHub repository for you.
+
+4. Some additional changes need to be made. This might be done for you, or might be left for you to do:
+ - Update the `name` and any urls in the `package.json`
+ - Update the `name` and related fields in `companion/manifest.json`
+ - Add the old name to the `legacyIds` array in `companion/manifest.json`. This lets Companion know of the rename, so that existing users will be migrated across.
+
+5. Once everything is updated, this can be included in the builds
+
+A note for the maintainers running the build script, the old module name will need to be manually removed from the bundled-modules github repository.
diff --git a/for-developers/module-development/module-lifecycle/testing-a-custom-version-of-@companion-module-base.md b/for-developers/module-development/module-lifecycle/testing-a-custom-version-of-@companion-module-base.md
new file mode 100644
index 0000000..fcee57b
--- /dev/null
+++ b/for-developers/module-development/module-lifecycle/testing-a-custom-version-of-@companion-module-base.md
@@ -0,0 +1,28 @@
+---
+title: 'Using a custom @companion-module/base library'
+sidebar_label: 'Custom @companion-module/base'
+sidebar_position: 8
+description: How to test and use your module with a non-release version of the companion modules.
+---
+
+Sometimes it may be useful to test or use an unreleased version of `@companion-module/base`. This commonly happens when new features are added to this library, before they are deemed ready for general use.
+
+:::warning
+
+Modules using unreleased versions of `@companion-module/base` should not be distributed. There will often be subtle differences between the unreleased version and the final version that will cause bugs. Make sure to only use released versions in any distributed/published module versions.
+
+:::
+
+## Using an unreleased version
+
+This can sometimes be done against a normal build of companion, unless the changes to this library require changes in companion too.
+
+1. clone this repository somewhere on your computer if you havent already `git clone https://github.com/bitfocus/companion-module-base.git`
+2. cd into the cloned folder `cd companion-module-base`
+3. If the version you want is a branch, checkout the branch, or main if you want the primary branch. eg `git checkout main`
+4. Install dependencies `yarn install`
+5. Build the library `yarn build`. If this step fails with an error, you will need to resolve this
+6. Make the library available through `yarn link` (if you have done this before, it can be skipped)
+7. Inside your module folder, link in the local version `yarn link @companion-module/base`. This may need to be repeated anytime you run add/install/remove any dependencies from your module.
+
+Note: be aware that the custom version will stay with your module as you switch branches. Once you are done, the best way to ensure you are using the version defined in your `package.json` is to delete your `node_modules` folder and run `yarn install` to regenerate it
diff --git a/for-developers/module-development/module-lifecycle/updating-nodejs.md b/for-developers/module-development/module-lifecycle/updating-nodejs.md
index 6b68fcf..4782c62 100644
--- a/for-developers/module-development/module-lifecycle/updating-nodejs.md
+++ b/for-developers/module-development/module-lifecycle/updating-nodejs.md
@@ -1,8 +1,8 @@
---
title: 'Updating the Node.JS Version in your Module'
sidebar_label: 'Updating Node.JS'
-sidebar_position: 8
-description: Explanation of the companion module library.
+sidebar_position: 5
+description: How to update your module to use a newer version of Node.JS.
---
Nodejs often makes new releases. In the major version jumps, these can include some breaking changes which could impact your module, so we don't want to do that without you being aware of it.
diff --git a/for-developers/module-development/module-lifecycle/upgrading-a-module-built-for-companion-2.x.md b/for-developers/module-development/module-lifecycle/upgrading-a-module-built-for-companion-2.x.md
new file mode 100644
index 0000000..ab1f636
--- /dev/null
+++ b/for-developers/module-development/module-lifecycle/upgrading-a-module-built-for-companion-2.x.md
@@ -0,0 +1,522 @@
+---
+title: 'Upgrading a module built for Companion 2.x'
+sidebar_label: 'Upgrading from Companion 2.x'
+sidebar_position: 10
+description: How to update a module that was built for Companion 2 or earlier.
+---
+
+## Background
+
+In Companion 3.0, we rewrote the module-api from scratch. We chose to do this because the old api had grown very organically over 5 years, and was starting to show various issues. The main issues was modules were running in the main companion thread, and method signatures were assuming various calls to be synchronous, and modules being able to access various internals of companion (and some making concerningly large use of that ability).
+Rather than trying to quickly fix up the api, it was decided to rethink it entirely.
+
+The general shape of the api should be familiar, but many methods names have been changed, and the whole api is written to rely on promises a lot more when before either callbacks were abused or methods would be synchronous.
+
+Technology has also evolved since many of the modules were written, with many of them using js syntax that was superseded in 2015! As part of this overhaul, we have imposed some minimum requirements for tooling and code style. This shouldnt be an issue, but if you run into any issues let us know on slack and we will help you out.
+
+Note: This guide is written assuming your module is an ES6 class (`class YourModule extends InstanceSkel {`). If it is not, then it would be best to convert it to that format first, to simplify this process. That can be done before starting this conversion for Companion 3.0, as it is supported in all older versions of companion
+
+## First steps
+
+You no longer need a developer version of companion to develop your module! It can now be done with the version you usually run, or you can keep to the old way if you prefer, but you shall have to refer to the main application development guides for getting that setup and running again.
+
+Once you have the companion launcher open, click the cog in the top right. This will reveal a 'Developer modules path' field. Use this to select a `companion-module-dev` folder you created containing your modules code. See the page on [Setting up the Development Folder](../local-modules.md)
+
+Companion will load in any modules it finds from that folder when starting, and will restart itself whenever a file inside that folder is changed.
+
+You can now clone or move or existing clone into the folder you specified for development modules. Once you have done this, launch companion and it will report your module as crashing in its status, as your code needs updating.
+
+While developing, you can view a log for an instance in the ui. This will show all messages written with `this.log()`, `this.updateStatus()` and anything written to the console. Some of this is omitted from other logs, this is the best place to see output from your module while it is running.
+
+
+
+### 1) Add new dependencies
+
+To help with this new flow, any files you previously imported from companion itself (eg '../../../instance_skel') have been rewritten and are now located in a dedicated npm package. And there is a second package which provides some tooling to help with packaging your module for distribution.
+
+You can add these to your module by running `yarn add @companion-module/base -T` and `yarn add --dev @companion-module/tools`
+
+Note: We recommend `@companion-module/base` to be installed as `~1.4` rather than `^1.4.1` as the version of this will dictate the versions of companion your module is compatible with. Read the documentation on module versioning if you want more details.
+
+Add a new file (if not exist already) : `.yarnrc.yml` and put the following code in:
+
+```
+nodeLinker: node-modules
+```
+
+You should also add the following to your `.gitignore` file:
+
+```
+/pkg
+/pkg.tgz
+```
+
+### 2) Create the companion manifest
+
+Previously, Companion would read some special properties from your package.json file to know about your module. We have decided that we should instead be using our own file, so that we can avoid properties we do not need, and reduce ambiguation in the contents. This will also allow us to handle modules which are not node/npm based.
+
+To start, run the command `yarn companion-generate-manifest`. This will generate the `companion` folder based on your package.json.
+
+As part of the process, you should notice that the `HELP.md` file has moved into the folder too. The `HELP.md` is also expected to be inside of this folder, as well as any images it uses.
+
+Please give the manifest a read, it should be fairly self explanatory. You should verify that the `runtime.entrypoint` property is a relative path to your main javascript file. It should have been generated correctly, but be aware that you will need to change this if you move/rename the file it references.
+
+### 3) package.json cleanup
+
+Now that you have the new manifest, you can cleanup the old properties from your package.json.
+
+The following properties can be removed:
+
+- api_version
+- keywords
+- legacy
+- manufacturer
+- product
+- shortname
+- description
+- bugs
+- homepage
+- author
+- contributors
+
+Note: some of these are standard npm properties, but they are no longer necessary and are defined in the manifest.json
+
+If you have a `postinstall` script defined to build your module, that should now be removed.
+
+If you are using typescript, you can move any `@types/*` packages from dependencies to devDependencies, but this is not required. The build script should be changed from `npx rimraf dist && npx typescript@~4.5 -p tsconfig.build.json` to `rimraf dist && yarn build:main`.
+
+As this is quite a large change, we should update the version number too. To make the size of the change here clear, we should increment the first number in the version. For example, `1.5.11` should become `2.0.0` or `3.0.1` should become `4.0.0`. This classes it as a breaking change. Make sure to change the number in package.json, as that is the master location for that. Refer to the [Versioning of Modules guide](../../git-workflows/versioning.md#version-of-modules) for details.
+
+### 4) Prepare to update your code
+
+You are now ready to begin updating your code. Many properties and methods have been renamed, so this can take some time in larger modules. But the hope is that it provides a much more consistent and easier to understand api than before.
+
+Because of the amount of changes, we recommend following our steps for what to do, then to test and debug it at the very end. If you attempt to run it part way through it will most likely produce weird errors or constantly crash.
+
+Before we begin, it is recommended that you ensure you have some understanding of async & Promises in nodejs. Previously the api was very synchronous, due to everything sharing a single thread, but some methods are now asynchronous as each instance of your module is run in its own thread. It is important to understand how to use promises, as you will need that to get any values from companion, and any promises left 'floating' will crash your module.
+
+TODO - write more about async or find some good tutorials/docs/examples
+
+If you are ever unsure on how something should look, we recommend looking at the following modules. This is a curated list of up-to-date and well maintained modules. If you are still unsure, then reach out in #module-development in slack.
+
+- [generic-osc](https://github.com/bitfocus/companion-module-generic-osc)
+- [homeassistant-server](https://github.com/bitfocus/companion-module-homeassistant-server)
+
+Advanced users: You are now able to make your module be ESM if you wish. It is completely optional and should require no special configuration for companion.
+
+Finally, look through your code and make sure that any dependencies you use are defined in your `package.json`. Many modules have been accidentally using the dependencies that companion defined, but in this new structure that is not possible. You may need to spend some time updating your code to work with the latest version of a library, but this is a good idea to do every now and then anyway.
+
+### 5) Update your actions
+
+If you are using typescript, `CompanionAction` and `CompanionActions` have been renamed to `CompanionActionDefinition` and `CompanionActionDefinitions` and should be imported from `'@companion-module/base'`.
+
+For the action definitions, the following changes have been made:
+
+- `label` has been renamed to `name`
+- `options` is required, but can be an empty array (`[]`)
+- `callback` is now required. This is the only way an action can be executed (more help is below)
+
+Tip: While you are making these changes, does the value of `name` still make sense? Perhaps you could set a `description` for the action too?
+
+Some changes to `options` may be required, but we shall cover those in a later step, as the same changes will need to be done for feedbacks and config_fields.
+
+If you aren't already aware, there are some other properties you can implement if you need then.
+You can find the typescript definitions as well as descriptions of each property you can set [in the module-base repo](https://github.com/bitfocus/companion-module-base/blob/main/src/module-api/action.ts). Do let us know if anything needs more explanation/clarification.
+
+The `callback` property is the only way for an action to be executed. In previous versions it was possible to do this by implementing the `action()` function in the main class too. We found that using this callback made modules more maintainable as everything for an action was better grouped together, especially when the other methods are implemented.
+If you need help figuring out how to convert your code from the old `action` method to these callbacks then reach out on slack.
+
+The parameters supplied to the `callback` or `action` function have been restructured too:
+
+- The second parameter has been merged into the first, with some utility methods provided in the second parameter.
+- `action` has been renamed to `actionId` to be more consistent with elsewhere. This is the `id` you gave your actionDefinition.
+- `deviceId` is no longer provided as we don't see a use case for why this is useful. Let us know if you have one and we shall consider re-adding it
+- `bank` and `page` are no longer provided. These have been replaced by `controlId`. Actions can reside in a trigger rather than on a bank, and controlId lets us express that better. `controlId` is a unique identifier that will be the same for all actions & feedbacks on the same button or inside the same trigger, but its value should not be treated as meaningful. In a later version of companion the value of the `controlId` will change.
+- The new second parameter contains an implementation of `parseVariablesInString`, this has more purpose for feedbacks, and is provided to actions for consistency
+
+Note: there are some properties on the object with names starting with an underscore that are not mentioned here. You must not use these, as these are temporary properties. They will be removed without notice in a future version.
+
+If you are using `subscribe`, `unsubscribe` or `learn`, the object parameter has changed a little, and is a subset of the properties supplied to `callback`.
+
+Finally, to pass the action definitions to companion has changed. It is now `this.setActionDefinitions(actions)` instead of `this.setActions(actions)` or `self.system.emit('instance_actions', self.id, actions)`
+
+### 6) Updating your feedbacks
+
+If you are using typescript, `CompanionFeedback` and `CompanionFeedbacks` have been renamed to `CompanionFeedbackDefinition` and `CompanionFeedbackDefinitions` and should be imported from `'@companion-module/base'`.
+
+For the feedback definitions, the following changes have been made:
+
+- `type` is now required. If you don't have it set already, then it should be set to `'advanced'`
+- `label` has been renamed to `name`
+- `options` is required, but can be an empty array (`[]`)
+- `callback` is now required. This is the only way an action can be executed (more help is below)
+- If your feedback is of `type: 'boolean'` then `style` has been renamed to `defaultStyle`
+
+Additionally, the `rgb` and `rgbRev` methods no longer exist on your instance class. They have been renamed to `combineRgb` and `splitRgb` and should be imported from `'@companion-module/base'`.
+
+Tip: While you are making these changes, does the value of `name` still make sense? Perhaps you could set a `description` for the action too?
+
+Some changes to `options` may be required, but we shall cover those in a later step, as the same changes will need to be done for feedbacks and config_fields.
+
+If you aren't already aware, there are some other properties you can implement if you need then.
+You can find the typescript definitions as well as descriptions of each property you can set [in the module-base repo](https://github.com/bitfocus/companion-module-base/blob/main/src/module-api/feedback.ts). Do let us know if anything needs more explanation/clarification.
+
+The `callback` property is the only way for an feedback to be checked. In previous versions it was possible to do this by implementing the `feedback()` function in the main class too. We found that using this callback made modules more maintainable as everything for a feedback was better grouped together, especially when the other methods are implemented.
+If you need help figuring out how to convert your code from the old `feedback` method to these callbacks then reach out on slack.
+
+The parameters supplied to the `callback` or `feedback` function have been restructured too:
+
+- The parameters have been merged into one, with some utility methods provided in the second parameter.
+- `type` has been renamed to `feedbackId` to be more consistent with elsewhere. This is the `id` you gave your feedbackDefinition.
+- `bank` and `page` are no longer provided. These have been replaced by `controlId`. Feedbacks can reside in a trigger rather than on a bank, and controlId lets us express that better. `controlId` is a unique identifier that will be the same for all actions & feedbacks on the same button or inside the same trigger, but its value should not be treated as meaningful. In a later version of companion the value of the `controlId` will change.
+- It is no longer possible to get the complete 'bank' object that was provided before. Do let us know if you have a use case for it, we couldn't think of one.
+- For 'advanced' feedbacks, the `width` and `height` properties are now represented by a single `image` property. This helps clarify when they will be defined. Also note that these may not be present for all feedbacks in a future version.
+- The new second parameter contains an implementation of `parseVariablesInString`. You **must** use this instead of the similar method on the class, otherwise your feedback will not be rechecked when the variables change.
+
+Note: there are some properties on the object with names starting with an underscore that are not mentioned here. You must not use these, as these are temporary properties. They will be removed without notice in a future version.
+
+If you are using `subscribe`, `unsubscribe` or `learn`, the object parameter has changed a little, and is a subset of the properties supplied to `callback`.
+
+### 7) Updating your presets
+
+If you are using typescript, `CompanionPreset` has been renamed to `CompanionButtonPresetDefinition`. There is also a `CompanionPresetDefinitions` defining a collection of these objects.
+
+The first change to be aware of, is that the `setPresetDefinitions()` function is now expecting an object of presets, rather than array. This is for better consistency with actions and feedbacks, and also provides a unique id for each preset.
+
+For the preset definitions, the following changes have been made for both types:
+
+- `type` is a new property that must be `button`
+- `label` has been renamed to `name`
+- `options` is a new property of some behavioural properties for the button. Its contents is explained below.
+- `bank` has been renamed to `style`. Additionally some properties have been moved out of this object, listed below.
+- `bank.style` has been removed
+- `bank.relative_delay` has been moved to `options.relativeDelay`
+- `bank.latch` has been removed. This will be handled below when changing the actions
+- `feedbacks` The objects inside this have changed slightly. The `type` property has been renamed to `feedbackId`, for better consistency.
+- `actions` and `release_actions` have been combined into `steps`, the exact structure is documented below.
+
+Note: Even when there are no actions or feedbacks, you would need to put those objects in : `feedbacks: []` & `steps: []`.
+
+Note: Remember that the `rgb` and `rgbRev` methods no longer exist on your instance class. They have been renamed to `combineRgb` and `splitRgb` and should be imported from `'@companion-module/base'`.
+
+The latch button mode has been replaced with a more flexible 'stepped' system. On each button, you can have as many steps as you wish, each with their own down/up/rotate actions. When releasing the button, it will by default automatically progress to the next step, or you can disable that and use an internal action to change the current step of a button.
+While is is wordier and more complex to make a latched button, this allows for much more complex flows, such as using another button as a 'shift' key, or having one 'go' button which runs the whole pre-programmed show.
+
+The `steps` property is an array of objects. Each object should be at a minimum `{ down: [], up: [] }` (`rotate_left` and `rotate_right` are also valid here).
+
+For example, a normal button will only have one step:
+
+```javascript
+steps: [
+ {
+ down: [],
+ up: [],
+ },
+]
+```
+
+Or you can match the old latching behaviour with:
+
+```javascript
+steps: [
+ {
+ // Put the 'latch' actions here
+ down: [],
+ up: [],
+ },
+ {
+ // Put the 'unlatch' actions here
+ down: [],
+ up: [],
+ },
+]
+```
+
+For the action objects inside of each of these steps, the `action` property renamed to `actionId`, for better consistency with elsewhere.
+
+### 8) Updating options and config fields
+
+The options/input fields available to use as options for actions and feedbacks, as well as for your config fields has changed a little.
+
+There may be additional new properties not listed here. You can check [the docs](../connection-basics/input-field-types.md) for full details on the available input fields.
+
+#### common changes
+
+The optional `isVisible` function has changed, its parameter is now the options object for the action/feedback, not the whole action/feedback object
+
+#### text
+
+The `text` type has been renamed to `static-text`
+
+#### textinput & textwithvariables
+
+These have been combined into one `textinput`. With a new `useVariables` property to select the mode.
+
+#### dropdown
+
+This has been split into `dropdown` and `multidropdown`.
+
+`minSelection` and `maximumSelectionLength` are only valid for `multidropdown`
+
+`maximumSelectionLength` has been renamed to `maxSelection`
+
+`regex` and `allowCustom` are only valid for `dropdown`
+
+#### multiselect
+
+`multidropdown` should be used instead
+
+#### checkbox & colorpicker & number
+
+These are unchanged
+
+#### Anything else
+
+That should be all of them, if you have something else then it has likely been forgotten/removed.
+
+Either use a supported input type or let us know what we missed.
+
+### 9) Updating variables
+
+If you are using typescript, `CompanionVariable` has been renamed to `CompanionVariableDefinition` and should be imported from `'@companion-module/base'`.
+
+For the variable definitions, the following changes have been made:
+
+- `name` has been renamed to `variableId`
+- `label` has been renamed to `name`
+
+We acknowledge that this change is rather confusing, but it felt necessary for consistency with elsewhere
+
+To set the value of variables, the `setVariables` method has been renamed to `setVariableValues`, and the `setVariable` method has been removed.
+This is to encourage multiple values to be set in the one call which will make companion more responsive by being able to process multiple changes at a time. Please try to combine calls to `setVariableValues` where possible.
+
+For example, before:
+
+```js
+this.setVariable('one', 'word')
+this.setVariable('two', 34)
+this.setVariable('three, undefined)
+```
+
+After:
+
+```js
+this.setVariableValues({
+ one: 'word',
+ two: 34,
+ three: undefined,
+})
+```
+
+### 10) Updating upgrade-scripts
+
+If you are using typescript, `CompanionStaticUpgradeScript` should be imported from `'@companion-module/base'`.
+
+If you are using the `CreateConvertToBooleanFeedbackUpgradeScript` helper method on the InstanceSkel, this has moved and should be imported from `'@companion-module/base'`.
+
+The upgrade script parameters and return value have been completely reworked, as the old functions did not fit into the new flow well.
+
+The first parameter (`context`) to the functions remains, but currently has no methods. We expect to add some methods in the future.
+The remaining parameters have been combined into a single `props` object.
+
+This object is composed as
+
+```js
+{
+ config: Object | null,
+ actions: [],
+ feedbacks: [],
+}
+```
+
+Tip: These upgrade scripts can be called at any time. They can be called to upgrade existing configuration, or when data is imported into companion.
+
+At times the config property will be null, if the config does not need updating. There may be no actions or no feedbacks, depending on what needs upgrading.
+
+Previously, the return type was simply `boolean`, to state whether anything had changed. The return type is now expected to be an object of the format:
+
+```js
+{
+ updatedConfig: Object | null,
+ updatedActions: [],
+ updatedFeedbacks: [],
+}
+```
+
+If an action or feedback is not included in the output type, then it is assumed to not have been changed and any changes you have made to the object passed in will be lost.
+If `updatedConfig` is not set, then it is assumed that the config has not changed, and any changes will be lost.
+
+This allows companion to better understand what has changed in an optimal way, and also allows you more flexibility in your code as you are no longer forced to update in place.
+
+Tip: Make sure you don't change the id fields, or it won't track the changes!
+
+The config object is the same as you receive in the module, and is the same as before.
+
+The action and feedback objects have a similar shape to how they are provided to their callbacks. You can see more about the exact structure of the objects [in the module-base repo](https://github.com/bitfocus/companion-module-base/blob/main/src/module-api/upgrade.ts). This is harder to document the changes, and upgrade scripts are not that widely used.
+
+Finally, these are no longer provided via the static method. They will be passed into a function call later described later on. We suggest making them an array outside of the class that can be used later on.
+
+### 11) Updating any uses of the TCP/UDP/Telnet helpers
+
+If you are importing the old socket helpers, you will need to make some changes to handle some changes that have been made to them.
+
+#### UDP
+
+This should be imported as `UDPHelper` from `@companion-module/base` instead of the old path.
+
+Most usage of the class is the same, only the differences are documented:
+
+- The `addMembership` method has been removed. It was previously broken and appeared unused by any module. If you have need for it, it can be reimplemented
+- The `send` method no longer accepts a callback. Instead it returns a promise that must be handled.
+- The `status_change` event emits different statuses, to match how they have changed elsewhere. It now emits `'ok' | 'unknown_error'`
+- There are some new readonly properties added: `isDestroyed`
+- Any previously visible class members have been hidden. If you rely on any, let us know and we shall look at extending the wrapper or making things public again
+
+#### TCP
+
+This should be imported as `TCPHelper` from `@companion-module/base` instead of the old path.
+
+Most usage of the class is the same, only the differences are documented:
+
+- The `send` method no longer accepts a callback. Instead it returns a promise that must be handled.
+- The `status_change` event emits different statuses, to match how they have changed elsewhere. It now emits `'ok' | 'connecting' | 'disconnected' | 'unknown_error'`
+- There are some new readonly properties added: `isConnected`, `isConnecting` and `isDestroyed`
+- The `write` and `send` methods have been combined into a single `send` method. Behaviour is the same, just the name has changed
+- Any previously visible class members have been hidden. If you rely on any, let us know and we shall look at extending the wrapper or making things public again
+
+#### Telnet
+
+This should be imported as `TelnetHelper` from `@companion-module/base` instead of the old path.
+
+Most usage of the class is the same, only the differences are documented:
+
+- The `send` method no longer accepts a callback. Instead it returns a promise that must be handled.
+- The `status_change` event emits different statuses, to match how they have changed elsewhere. It now emits `'ok' | 'connecting' | 'disconnected' | 'unknown_error'`
+- There are some new readonly properties added: `isConnected`, `isConnecting` and `isDestroyed`
+- The `write` and `send` methods have been combined into a single `send` method. Behaviour is the same, just the name has changed
+- Any previously visible class members have been hidden. If you rely on any, let us know and we shall look at extending the wrapper or making things public again
+
+### 12) Updating your main class
+
+Finally, we are ready to look at your main class.
+
+To start, you should change your class to extend `InstanceBase`, which can be imported from `@companion-module/base`.
+
+There are a few fundamental changes to `InstanceBase` compared to the old `InstanceSkel`.
+
+- You should be doing very little in the constructor. You do not have the config for the instance at this point, or the ability to call any companion methods here. The intention is to make sure various class properties are defined here, with the module truly starting to work in the `init()` method.
+- `this.config` is no longer provided for you by the base class. You are provided a new config object in `init()` and `updateConfig()`, it is up to you to to an `this.config = config` when receiving that, or to do something else with the config object.
+- `this.system` is no longer available. That was previously an internal api that too many modules were hooking into. This caused many of them to break at unexpected times, and is no longer possible in this new api. If we haven't exposed something you are using on system as a proper method on `InstanceBase`, then do let us know on slack or github, and we will either implement it or help you with an alternative.
+
+The constructor has changed to have a single parameter called `internal`. This is of typescript type `unknown`. You must not use this object for anything yourself, the value of this is an internal detail, and will change in unexpected ways without notice.
+You should start your constructor with the following
+
+```js
+constructor(internal) {
+ super(internal)
+```
+
+Remember that you do not have access to the config here, you may need to move some stuff into `init()`
+
+Hopefully you have an understanding of async code, because we now need to start using it.
+It is expected that `init()` will return a Promise. The simplest way to do this is to mark it as async.
+
+```js
+ async init(config) {
+```
+
+`init()` also has a parameter, of your config object.
+
+Inside of init, you are free to do things as you wish. But make sure that any method returning a promise isn't left floating, so that errors propagate correctly.
+
+Tip: if you do a `this.checkFeedbacks()` inside of `init()`, you can remove that line
+
+The method `config_fields` should be renamed to `getConfigFields`.
+The method `updateConfig` should be renamed to `configUpdated`, it too is expected to return a Promise and should also be marked as async.
+The method `destroy` is expected to return a Promise and should also be marked as async.
+
+If you have a method called `action` or `feedback` still, make sure it has been migrated fully in the earlier steps, then remove it as it will no longer be called.
+
+Some other methods provided by companion have been changed.
+Any where the name starts with an underscore must not be used as they are internal methods that will change without notice.
+
+- The static method `CreateConvertToBooleanFeedbackUpgradeScript` has been removed and can be imported from `@companion-module/base` instead
+- The method `saveConfig` now expects a parameter of the config object to be saved
+- The method `setActions` has been renamed to `setActionDefinitions`, as explained earlier
+- The method `setPresetDefinitions` now expects an object of presets, as explained earlier
+- The method `setVariables` has been renamed to `setVariableValues`, as explained earlier
+- The method `setVariable` has been removed, as explained earlier
+- The method `getVariable` has been renamed to `getVariableValue`.
+- The method `checkFeedbacks` now accepts multiple feedbacks to check. eg `this.checkFeedbacks('one', 'two, 'three')`
+- The method `parseVariables` has been renamed to `parseVariablesInString`. Note that this method returns a promise instead of accepting a callback.
+- The method `getAllFeedbacks` has been removed, with no replacement
+- The method `getAllActions` has been removed, with no replacement
+- The method `oscSend` is unchanged
+- The method `status` has been renamed to `updateStatus`. Additionally, the first parameter has been changed and should be provided a string with one of the values from below. The second parameter remains as an optional string for more details. Note: calls to this also now add a log line when the status changes.
+- The method `log` is unchanged.
+- The method `debug` has been removed, use `log` instead
+- The methods `rgb` and `rgbRev` have been removed. They can be imported as `combineRgb` and `splitRgb` from `@companion-module/base` instead
+- The `STATUS_*` properties have been removed, they are no longer useful
+- The `REGEX_*` properties have been removed. Instead you can import `Regex` from `@companion-module/base` and use them from there.
+- The method `defineConst` has been removed. Instead you can either define it as a constant variable outside your class `const YOUR_THING = value` (perhaps in a separate constants.js file?) or as a normal member variable `this.YOUR_THING = value`
+- Anything else? It has likely been removed or forgotten. Ask us on slack
+
+Possible values for status: 'ok', 'connecting', 'disconnected', 'connection_failure', 'bad_config', 'unknown_error', 'unknown_warning'
+Note: More will be added in the future, let us know if you have any ideas for common statuses.
+
+Once you have made sure that any method calls to methods exposed on InstanceBase have been updated, there is only a little bit more.
+
+At the bottom of your file you should fine a line looking something like `export = MyInstance`.
+You should replace this line with `runEntrypoint(MyInstance, UpgradeScripts)`, making sure to pass your class as the first parameter, and your array of upgrade scripts to the second. If you have no upgrade scripts then make the second parameter be `[]`.
+
+Congratulations. You have now converted all of your code!
+
+It is time to take a break, as next we shall be running and testing it.
+
+### 13) Run it and debug!
+
+Startup companion, and make sure you have it setup so that it will find your custom module.
+
+If all went well, then your module will run and connect to your device.
+
+If it does not, hopefully you can figure out what has broken. There are too many possibilities for us to document here, but we can try to document some common issues.
+
+Once it is running, make sure to test every action, feedback, variable and preset. It is very easy to make a mistake during the upgrading process. Whatever bugs you find now means less to be found by other users.
+
+When you are happy with it being stable enough for others to test, let us know and we shall include it in the beta builds.
+
+### 14) Package it and test
+
+For modules in this new format, we require them to be packaged with some special tooling. This is done to both reduce the number of files that your module spans on disk, and also the size. Loading code spread across hundreds of files is surprisingly slow on some oses, any by combining it all into a few files, we can often reduce the size from multiple mb, to a few hundred kb.
+
+During the build process of the releases your module package will be generated automatically and bundled with the application, so you have to make sure that the final package is working. If you are developing with a complete dev environment, it is not sufficient if your module works in the dev environment.
+
+Sometimes once built there are issues that prevent a package from running, so it is mandatory to test it before distributing it. In our experience, issues often occur when working with files from disk, or introducing a new dependency that doesn't play nice. Once you have done this once, if you are just changing some internal logic you probably don't need to repeat this unless you want to be sure.
+
+You can build your module into this format with `yarn companion-module-build`. If this was successful, there should now be a `pkg` folder and a `pkg.tgz` file. These both contain the build version of your module!
+If you create an empty file `DEBUG-PACKAGED` in your module folder, then companion will read the code from `pkg` instead. This will let you test it.
+You probably don't need to do a very thorough test, as long as it starts and connects to your device and a couple of actions work it should be fine.
+
+Make sure to remove this `DEBUG-PACKAGED` file once you are done testing it, or next time you try to update your module it will keep loading the copy from `pkg`!
+
+If you encountered any issues in this process that you need help with, then have a look at the [Module packaging](./module-packaging.md) page for additional tips.
+
+### 15) Feedback
+
+Have any thoughts/feedback on this process? Anything in the docs that should be improved? Do [let us know](https://github.com/bitfocus/companion-module-base/issues), we are interested in your feedback!
+We won't be able to cater to everything you dislike about the changes, as other modules are already using these new apis, but perhaps we can make the transition smoother?
+
+Have an idea of a new connection helper that would be beneficical to you? Or have some utility code that you are copying into multiple modules? We are interested to hear this. We are happy to add more to `@companion-module/base` if it will be useful to many modules and is unlikely to result in breaking changes.
+
+We appreciate that this update is throwing a lot of changes at you, but changes of this scale are a rare occurrence and shouldn't be necessary to repeat for at least another 5 years.
+
+### 16) Follow up recommendations
+
+There are some extra steps that we recommend modules take, but are completely optional.
+
+- [Setting up code formatting](../module-setup/code-quality.md)
+- [Reusable Typescript config](../module-setup/typescript-config.md)
+- [Could any feedbacks be converted to booleans?](../connection-advanced/migrating-legacy-to-boolean-feedbacks.md)
+- [Using the subscribe/unsubscribe callbacks in actions](../connection-basics/actions.md#subscribe--unsubscribe-flow)
+- [Using the subscribe/unsubscribe callbacks in feedbacks](../connection-basics/feedbacks.md#subscribe--unsubscribe-flow)
+- [Using the learn callbacks](../connection-advanced/learn-action-feedback-values.md)
diff --git a/for-developers/module-development/module-setup/_category_.json b/for-developers/module-development/module-setup/_category_.json
new file mode 100644
index 0000000..13f7692
--- /dev/null
+++ b/for-developers/module-development/module-setup/_category_.json
@@ -0,0 +1,11 @@
+{
+ "label": "Module Setup",
+ "position": 10,
+ "link": {
+ "type": "doc",
+ "id": "index"
+ },
+ "customProps": {
+ "description": "The files and file structure necessary to create and configure a module repository."
+ }
+}
diff --git a/for-developers/module-development/module-config/code-quality.md b/for-developers/module-development/module-setup/code-quality.md
similarity index 95%
rename from for-developers/module-development/module-config/code-quality.md
rename to for-developers/module-development/module-setup/code-quality.md
index fea8756..423d4cd 100644
--- a/for-developers/module-development/module-config/code-quality.md
+++ b/for-developers/module-development/module-setup/code-quality.md
@@ -55,7 +55,7 @@ If using typescript, you should specify a `typescriptRoot`
import { generateEslintConfig } from '@companion-module/tools/eslint/config.mjs'
export default generateEslintConfig({
- enableTypescript: true,
+ enableTypescript: true,
})
```
@@ -63,7 +63,7 @@ You can now run `yarn lint` and see any linter errors. If you need any help unde
:::note
-If you have any suggestions on changes to make to this eslint config, do [open an issue](https://github.com/bitfocus/companion-module-tools/issues) to let us know. We hope that this set of rules will evolve over time based on what is and isnt useful to module developers.
+If you have any suggestions on changes to make to this eslint config, do [open an issue](https://github.com/bitfocus/companion-module-tools/issues) to let us know. We hope that this set of rules will evolve over time based on what is and isn't useful to module developers.
:::
@@ -153,18 +153,18 @@ You can easily override rules in this setup with:
import { generateEslintConfig } from '@companion-module/tools/eslint/config.mjs'
const baseConfig = await generateEslintConfig({
- enableTypescript: true,
+ enableTypescript: true,
})
const customConfig = [
- ...baseConfig,
-
- {
- rules: {
- 'n/no-missing-import': 'off',
- 'node/no-unpublished-import': 'off',
- },
- },
+ ...baseConfig,
+
+ {
+ rules: {
+ 'n/no-missing-import': 'off',
+ 'node/no-unpublished-import': 'off',
+ },
+ },
]
export default customConfig
diff --git a/for-developers/module-development/module-config/file-structure.md b/for-developers/module-development/module-setup/file-structure.md
similarity index 100%
rename from for-developers/module-development/module-config/file-structure.md
rename to for-developers/module-development/module-setup/file-structure.md
diff --git a/for-developers/module-development/module-setup/index.md b/for-developers/module-development/module-setup/index.md
new file mode 100644
index 0000000..2e0d24b
--- /dev/null
+++ b/for-developers/module-development/module-setup/index.md
@@ -0,0 +1,7 @@
+---
+title: Module Setup and Structure
+description: The files and file structure necessary to create a module repository.
+auto_toc: 3
+---
+
+This section describes the files and file structure necessary to create a module repository.
diff --git a/for-developers/module-development/module-config/manifest.json.md b/for-developers/module-development/module-setup/manifest.json.md
similarity index 72%
rename from for-developers/module-development/module-config/manifest.json.md
rename to for-developers/module-development/module-setup/manifest.json.md
index 31d9b2e..80a363a 100644
--- a/for-developers/module-development/module-config/manifest.json.md
+++ b/for-developers/module-development/module-setup/manifest.json.md
@@ -7,46 +7,46 @@ description: Specification of the Companion manifest file
Starting with Companion 3.0, Companion looks at the `companion/manifest.json` file for module information (before 3.0 it looked at `package.json`) This provides a companion specific and programming language agnostic manifest about your module. In the future this will allow us to do more powerful things!
-You can see the auto-generated documentation for this file as a typescript interface [here](https://bitfocus.github.io/companion-module-base/interfaces/ModuleManifest.html).
+Read the [auto-generated documentation for manifest.json](https://bitfocus.github.io/companion-module-base/interfaces/ModuleManifest.html) for more details.
Tip: At any point you can validate your `manifest.json` by running `yarn companion-module-check`.
-### Format
+## Format
-If you are comfortable reading or working with JSON Schema, you can find the formal definition [here](https://github.com/bitfocus/companion-module-base/blob/main/assets/manifest.schema.json)
+If you are comfortable reading or working with JSON Schema, you can find the [formal definition in the module-base repo](https://github.com/bitfocus/companion-module-base/blob/main/assets/manifest.schema.json)
A full manifest definition is:
```json
{
- "id": "fake-module",
- "name": "fake module",
- "shortname": "fake",
- "description": "Fake Module",
- "manufacturer": "Fake Module",
- "products": ["Fake"],
- "keywords": ["Fake"],
- "version": "0.0.0",
- "license": "MIT",
- "repository": "git+https://github.com/bitfocus/companion-module-fake-module.git",
- "bugs": "https://github.com/bitfocus/companion-module-fake-module/issues",
- "maintainers": [
- {
- "name": "Your name",
- "email": "your.email@example.com"
- }
- ],
- "legacyIds": [],
- "runtime": {
- "type": "node22",
- "api": "nodejs-ipc",
- "apiVersion": "0.0.0",
- "entrypoint": "../dist/index.js"
- }
+ "id": "fake-module",
+ "name": "fake module",
+ "shortname": "fake",
+ "description": "Fake Module",
+ "manufacturer": "Fake Module",
+ "products": ["Fake"],
+ "keywords": ["Fake"],
+ "version": "0.0.0",
+ "license": "MIT",
+ "repository": "git+https://github.com/bitfocus/companion-module-fake-module.git",
+ "bugs": "https://github.com/bitfocus/companion-module-fake-module/issues",
+ "maintainers": [
+ {
+ "name": "Your name",
+ "email": "your.email@example.com"
+ }
+ ],
+ "legacyIds": [],
+ "runtime": {
+ "type": "node22",
+ "api": "nodejs-ipc",
+ "apiVersion": "0.0.0",
+ "entrypoint": "../dist/index.js"
+ }
}
```
-### Properties
+## Properties
- `id` unique id of your module. This has to match the repository name excluding the `companion-module-`
- `name` ???
diff --git a/for-developers/module-development/module-config/typescript-config.md b/for-developers/module-development/module-setup/typescript-config.md
similarity index 54%
rename from for-developers/module-development/module-config/typescript-config.md
rename to for-developers/module-development/module-setup/typescript-config.md
index b343bab..b2766f0 100644
--- a/for-developers/module-development/module-config/typescript-config.md
+++ b/for-developers/module-development/module-setup/typescript-config.md
@@ -12,22 +12,22 @@ config file is included and you will generally not want to change it.
:::
-The [recommended templates](./file-structure.md) provide typescript config presets in _tsconfig.json_ that we believe to be best practise, but they can be configured to be too strict for some, or may need to be modified if you change the name of the source or destination directories.
+The [recommended templates](./file-structure.md) provide typescript config presets in _tsconfig.json_ that we believe to be best practice, but they can be configured to be too strict for some, or may need to be modified if you change the name of the source or destination directories.
A typical _tsconfig.json_ file looks like:
```json
{
- "extends": "@companion-module/tools/tsconfig/node22/recommended",
- "include": ["src/**/*.ts"],
- "exclude": ["node_modules/**", "src/**/*spec.ts", "src/**/__tests__/*", "src/**/__mocks__/*"],
- "compilerOptions": {
- "outDir": "./dist",
- "baseUrl": "./",
- "paths": {
- "*": ["./node_modules/*"]
- }
- }
+ "extends": "@companion-module/tools/tsconfig/node22/recommended",
+ "include": ["src/**/*.ts"],
+ "exclude": ["node_modules/**", "src/**/*spec.ts", "src/**/__tests__/*", "src/**/__mocks__/*"],
+ "compilerOptions": {
+ "outDir": "./dist",
+ "baseUrl": "./",
+ "paths": {
+ "*": ["./node_modules/*"]
+ }
+ }
}
```
@@ -36,30 +36,30 @@ Our TypeScript template splits it into two files:
```json
//tsconfig.json
{
- "extends": "./tsconfig.build.json",
- "include": ["src/**/*.ts"],
- "exclude": ["node_modules/**"],
- "compilerOptions": {
- "types": ["node"]
- }
+ "extends": "./tsconfig.build.json",
+ "include": ["src/**/*.ts"],
+ "exclude": ["node_modules/**"],
+ "compilerOptions": {
+ "types": ["node"]
+ }
}
```
```json
// tsconfig.build.json
{
- "extends": "@companion-module/tools/tsconfig/node22/recommended",
- "include": ["src/**/*.ts"],
- "exclude": ["node_modules/**", "src/**/*spec.ts", "src/**/__tests__/*", "src/**/__mocks__/*"],
- "compilerOptions": {
- "outDir": "./dist",
- "baseUrl": "./",
- "paths": {
- "*": ["./node_modules/*"]
- },
- "module": "Node16",
- "moduleResolution": "Node16"
- }
+ "extends": "@companion-module/tools/tsconfig/node22/recommended",
+ "include": ["src/**/*.ts"],
+ "exclude": ["node_modules/**", "src/**/*spec.ts", "src/**/__tests__/*", "src/**/__mocks__/*"],
+ "compilerOptions": {
+ "outDir": "./dist",
+ "baseUrl": "./",
+ "paths": {
+ "*": ["./node_modules/*"]
+ },
+ "module": "Node16",
+ "moduleResolution": "Node16"
+ }
}
```
diff --git a/for-developers/module-development/module-config/unit-testing.md b/for-developers/module-development/module-setup/unit-testing.md
similarity index 100%
rename from for-developers/module-development/module-config/unit-testing.md
rename to for-developers/module-development/module-setup/unit-testing.md
diff --git a/src/css/custom.css b/src/css/custom.css
index 88368f6..232540f 100644
--- a/src/css/custom.css
+++ b/src/css/custom.css
@@ -244,11 +244,6 @@
color: #6b7280;
}
-/* In markdown pages indent all text, including subheaders, under H2 headers */
-.markdown h2 ~ :not(h1, h2) {
- margin-left: 2rem;
-}
-
/* Auto-generated TOC heading links should be white, not red */
.auto-toc-heading a {
color: var(--ifm-heading-color);
@@ -259,3 +254,32 @@
color: var(--ifm-heading-color);
text-decoration: underline;
}
+
+/* In markdown pages indent all text, including subheaders, under H2 headers */
+.markdown h2 ~ :not(h1, h2) {
+ margin-left: 2rem;
+}
+
+/* Some default numbers: in markdown, the ifm defaults are overridden
+ --ifm-heading-line-height: 1.25;
+ --ifm-h1-font-size: 2rem; but 3rem for top of file (.markdown h1:first-child),
+ --ifm-h2-font-size: 2rem; in markdown (default is 1.5 rem)
+ --ifm-h3-font-size: 1.5 in markdown; normal is 1.25rem;
+ --ifm-h4-font-size: 1rem; (unchanged)
+ --ifm-h5-font-size: 0.875rem;
+ --ifm-h6-font-size: 0.85rem;
+ */
+.markdown h1 {
+ --ifm-h1-vertical-rhythm-bottom: 1;
+ font-size: 2em; /* setting --ifm-h1-font-size doesn't work (because of specificity?) */
+}
+
+.markdown h2 {
+ --ifm-h2-vertical-rhythm-top: 1; /* default 2 is too much. Also, we already indent everything under H2 to make H2 breaks clear */
+ --ifm-heading-vertical-rhythm-bottom: calc(0.5 / 1.25); /* this is multiplied by --ifm-leading = 1.25 */
+ font-size: 1.5em;
+}
+
+.markdown h3 {
+ font-size: 1.25em;
+}
diff --git a/src/remark/autoTocPlugin.mjs b/src/remark/autoTocPlugin.mjs
index 67f20dd..98a86a1 100644
--- a/src/remark/autoTocPlugin.mjs
+++ b/src/remark/autoTocPlugin.mjs
@@ -3,12 +3,18 @@
*
* When a document has `auto_toc: true` in its frontmatter, this plugin reads
* all sibling `.md` files in the same directory, extracts their titles and
- * headings, and appends an organized TOC to the document.
+ * headings, and appends an bulleted TOC to the document.
+ *
+ * If subdirectories exist, they will be listed with their files as the "content"
+ *
+ * auto_toc: number | boolean - if number, only include headers to that H# level. If `true`, go up to H3.
*/
-import { readFileSync, readdirSync } from 'node:fs'
-import { dirname, join } from 'node:path'
+import { readFileSync, readdirSync, statSync, existsSync } from 'node:fs'
+import { dirname, basename, join } from 'node:path'
-/** Parse YAML frontmatter from a markdown string (simple key: value only). */
+/** Parse YAML frontmatter from a markdown string (simple key: value only).
+ * Note: a possibly better way to do this would be with npm gray-matter. Leaving this for now
+ */
function parseFrontmatter(content) {
const match = content.match(/^---\r?\n([\s\S]*?)\r?\n---/)
if (!match) return { frontmatter: {}, body: content }
@@ -84,22 +90,54 @@ function parseInlineContent(text) {
export default function autoTocPlugin() {
return (tree, vfile) => {
- if (!vfile.data?.frontMatter?.auto_toc) return
+ const auto_toc = vfile.data?.frontMatter?.auto_toc
+ if (!auto_toc) return
+ const depth = typeof auto_toc === 'number' ? auto_toc : 3 // max header-level depth
const dir = dirname(vfile.path)
- const files = readdirSync(dir).filter((f) => f.endsWith('.md') && !f.startsWith('index'))
+ const vfilename = basename(vfile.path)
+ const files = readdirSync(dir).filter((f) => f.endsWith('.md') && !f.startsWith(vfilename))
+ const subdirs = readdirSync(dir).filter((f) => existsSync(join(dir, f, '_category_.json')))
+
+ const getPages = (fileList, dir) =>
+ fileList.map((f) => {
+ const content = readFileSync(join(dir, f), 'utf-8')
+ const { frontmatter, body } = parseFrontmatter(content)
+ let title = frontmatter.sidebar_label?.replace(/└─ */, '') || frontmatter.title || f.replace(/\.md$/, '')
+ // replace enclosing quotes, if present. putting this in parseFrontmatter didn't work for some reason.
+ title = title.replaceAll(/^["']|["']$/g, '')
+ return {
+ slug: f.replace(/\.md$/, ''),
+ title: title,
+ sidebarPosition: parseInt(frontmatter.sidebar_position || '0', 10),
+ headings: extractHeadings(body),
+ files: [],
+ }
+ })
- const pages = files.map((f) => {
- const content = readFileSync(join(dir, f), 'utf-8')
- const { frontmatter, body } = parseFrontmatter(content)
+ let pages = getPages(files, dir)
+
+ // TODO: handle directories inside the subdir
+ // compile a list of files inside a subdir:
+ const dirs = subdirs.map((d) => {
+ const subdir = join(dir, d)
+ const raw = readFileSync(join(subdir, '_category_.json'), 'utf-8')
+ const frontmatter = JSON.parse(raw)
+ const indexfile = frontmatter.link?.id ?? 'index.md' // if link is missing, it appears to default to a doc named index.md, if present.
+ const subfiles = readdirSync(subdir).filter((f) => f.endsWith('.md') && !f.startsWith(indexfile))
+ const subpages = getPages(subfiles, subdir)
+ subpages.sort((a, b) => a.sidebarPosition - b.sidebarPosition)
return {
- slug: f.replace(/\.md$/, ''),
- title: frontmatter.title || f.replace(/\.md$/, ''),
- sidebarPosition: parseInt(frontmatter.sidebar_position || '0', 10),
- headings: extractHeadings(body),
+ slug: d,
+ title: '> ' + (frontmatter.label || d),
+ sidebarPosition: frontmatter.position || 0,
+ headings: [],
+ files: subpages,
}
})
+ pages = pages.concat(dirs)
+
// Ascending sidebar_position → most-negative first → newest version on top
pages.sort((a, b) => a.sidebarPosition - b.sidebarPosition)
@@ -108,7 +146,7 @@ export default function autoTocPlugin() {
// ### [Page Title](./slug)
nodes.push({
type: 'heading',
- depth: 3,
+ depth: 3, // make page name h3
data: {
hProperties: {
className: 'auto-toc-heading',
@@ -117,7 +155,9 @@ export default function autoTocPlugin() {
children: [
{
type: 'link',
- url: `./${page.slug}`,
+ // folder URLs need a trailing slash in order for links on the target page (not the link itself) to work!
+ // i.e. w/o this the url is: ""./category" instead of ""./category/"" and the links in index.md will go to ""./subfile" instead of ""./category/subfile"
+ url: `./${page.slug}` + (page.files.length > 0 ? '/' : ''),
children: [{ type: 'text', value: page.title }],
},
],
@@ -129,24 +169,43 @@ export default function autoTocPlugin() {
type: 'list',
ordered: false,
spread: false,
- children: page.headings.map((h) => ({
- type: 'listItem',
- spread: false,
- children: [
- {
- type: 'paragraph',
- children: [
- {
- type: 'link',
- url: `./${page.slug}#${h.id}`,
- children: parseInlineContent(h.text),
- },
- ],
- },
- ],
- })),
+ children: page.headings
+ .filter((h) => h.level <= depth)
+ .map((h) => ({
+ type: 'listItem',
+ spread: false,
+ children: [
+ {
+ type: 'paragraph',
+ children: [
+ {
+ type: 'link',
+ url: `./${page.slug}#${h.id}`,
+ children: parseInlineContent(h.text),
+ },
+ ],
+ },
+ ],
+ })),
})
}
+
+ // Bullet list of each subdirectory file as a link
+ if (page.files.length > 0) {
+ for (const subfile of page.files) {
+ nodes.push({
+ type: 'text',
+ // note: the "spaces" before are U+2800: unicode Braille Pattern Blank
+ value: subfile === page.files.at(-1) ? '⠀⠀└── ' : '⠀⠀├── ',
+ })
+ nodes.push({
+ type: 'link',
+ url: `./${page.slug}/${subfile.slug}`,
+ children: [{ type: 'text', value: subfile.title }],
+ })
+ nodes.push({ type: 'break' })
+ }
+ }
}
tree.children.push(...nodes)
```
-At the points where it says "ref=wiki" replace the word wiki with a reference to you or your product (only letters and numbers are allowed).
-You can scale the image to fit in your website, as long as it is still readable, you may not not change its color, rotation, animate it or change it in any other way than scaling.
+At the points where it says `"ref=bitfocus"` replace the word _bitfocus_ with a reference to you or your product (only letters and numbers are allowed).
+
+You can scale the image to fit in your website, as long as it is still readable, you may not change its color, rotation, animate it or change it in any other way than scaling.
You are not allowed to:
diff --git a/for-developers/module-development/_category_.json b/for-developers/module-development/_category_.json
index 9d59ff2..99d1ae7 100644
--- a/for-developers/module-development/_category_.json
+++ b/for-developers/module-development/_category_.json
@@ -1,7 +1,8 @@
{
- "label": "Companion Modules",
+ "label": "Module Development",
"position": 3,
"link": {
- "type": "generated-index"
+ "type": "doc",
+ "id": "index"
}
}
diff --git a/for-developers/module-development/api-changes/v1.10.md b/for-developers/module-development/api-changes/v1.10.md
index 3285e7d..1263239 100644
--- a/for-developers/module-development/api-changes/v1.10.md
+++ b/for-developers/module-development/api-changes/v1.10.md
@@ -3,15 +3,15 @@ title: API 1.10 (Companion 3.4+)
sidebar_position: -110
---
-### Custom headlines for preset actions, feedbacks and steps {#preset-headlines}
+## Custom headlines for preset actions, feedbacks and steps {#preset-headlines}
Presets can now define a `headline` value for each action, feedback and step, to be used as the user editable label for the action/feedback/step.
-### Array support for merging multiple Bonjour queries {#bonjour-arrays}
+## Array support for merging multiple Bonjour queries {#bonjour-arrays}
Inside your `companion/manifest.json`, each bonjour query defined can now be an array of queries, allowing two queries to be merged and used for one config field.
-### Extended format support for feedback image buffers {#image-buffers}
+## Extended format support for feedback image buffers {#image-buffers}
The `imageBuffer` property from advanced feedbacks can now be in more formats.
diff --git a/for-developers/module-development/api-changes/v1.11.md b/for-developers/module-development/api-changes/v1.11.md
index da48ead..1b15b4a 100644
--- a/for-developers/module-development/api-changes/v1.11.md
+++ b/for-developers/module-development/api-changes/v1.11.md
@@ -4,6 +4,6 @@ sidebar_position: -111
description: Node.js 22
---
-### Updated to Node.js 22 runtime {#nodejs-22}
+## Updated to Node.js 22 runtime {#nodejs-22}
Modules can now use node.js 22. We recommend all modules update when possible.
diff --git a/for-developers/module-development/api-changes/v1.12.md b/for-developers/module-development/api-changes/v1.12.md
index f692864..7ea9191 100644
--- a/for-developers/module-development/api-changes/v1.12.md
+++ b/for-developers/module-development/api-changes/v1.12.md
@@ -4,7 +4,7 @@ sidebar_position: -112
description: Module permissions & isVisibleExpression
---
-### Module permissions for enhanced security {#permissions}
+## Module permissions for enhanced security {#permissions}
As of @companion-module/base v1.12, modules will be run with the nodejs permissions model enabled. This will allow us to inform users about the requirements of modules.
@@ -30,7 +30,7 @@ We would recommend planning for this in your module implementation.
:::
-### Expression syntax support for `isVisible` on option fields {#isvisible-expressions}
+## Expression syntax support for `isVisible` on option fields {#isvisible-expressions}
The `isVisible` functions that can be defined on option fields can now be written in the companion expression syntax through a new `isVisibleExpression` property.
@@ -40,11 +40,11 @@ We advise all uses to be updated to the new syntax, the old syntax is now deprec
:::
-### Port-based filtering for Bonjour queries {#bonjour-port}
+## Port-based filtering for Bonjour queries {#bonjour-port}
Bonjour queries in the module manifest can now specify a filter based on port number.
-### New utility methods for escape character handling {#utility-methods}
+## New utility methods for escape character handling {#utility-methods}
- parseEscapeCharacters
- substituteEscapeCharacters
diff --git a/for-developers/module-development/api-changes/v1.13.md b/for-developers/module-development/api-changes/v1.13.md
index 1a4a90a..01fd9ed 100644
--- a/for-developers/module-development/api-changes/v1.13.md
+++ b/for-developers/module-development/api-changes/v1.13.md
@@ -3,7 +3,7 @@ title: API 1.13 (Companion 4.1+)
sidebar_position: -113
---
-### Variables in `textinput` fields are now automatically parsed {#variable-parsing}
+## Variables in `textinput` fields are now automatically parsed {#variable-parsing}
You no longer need to call `parseVariablesInString` in most circumstances. Any `textinput` field with `useVariables` defined will automatically have variables parsed before the action/feedback is called.
@@ -21,13 +21,13 @@ This is groundwork to better allow us to handle expressions for you, expect more
:::
-### Value-type feedbacks (support for local variables) {#value-feedbacks}
+## Value-type feedbacks (support for local variables) {#value-feedbacks}
As part of the new local variables support in Companion, you can now define `value` feedbacks. These are similar to `boolean` feedbacks, but you can return any type of value.
Within Companion, the user is able to store the value you provide into a local variable. They can also do this with `boolean` feedbacks, but `boolean` feedbacks can also be used directly in styling a button, a `value` feedback cannot
-### `options:` improvements: new `description` field; multiline `textinput` field; "infinity" bounds for `number` fields {#new-options}
+## `options:` improvements: new `description` field; multiline `textinput` field; "infinity" bounds for `number` fields {#new-options}
Every field can now specify a `description`. This is intended to be a some hint to the user that should always be visible, and gets shown below the input field.
@@ -35,7 +35,7 @@ The `textinput` field type can now request to be multiline. This will provide a
The `number` field type can opt to show the defined min or max values as infinity. This is common behaviour for audio mixers, where they treat some low value such as -90 as -infinity
-### `secret-text` field-type for Configs {#secrets}
+## `secret-text` field-type for Configs {#secrets}
There is a new `secret-text` field type available to the connection config panel.
@@ -49,7 +49,7 @@ Make sure to not mix them, or it defeats the purpose of them being separate
:::
-### subscribe/unsubscribe: More flexibility for Action/ less for feedbacks {#subscribe}
+## subscribe/unsubscribe: More flexibility for Action/ less for feedbacks {#subscribe}
Actions can opt out of their unsubscribe method being called when the options change.
diff --git a/for-developers/module-development/api-changes/v1.14.md b/for-developers/module-development/api-changes/v1.14.md
index 4247050..ed0ebc8 100644
--- a/for-developers/module-development/api-changes/v1.14.md
+++ b/for-developers/module-development/api-changes/v1.14.md
@@ -4,7 +4,7 @@ sidebar_position: -114
description: Config layout changes, Connection processes now include the label
---
-### Automated layout for Config parameters {#config-layout}
+## Automated layout for Config parameters {#config-layout}
The connection config panel allowed modules to customise the layout of input fields. This was not possible elsewhere in Companion, and is not consistent with elsewhere within the panel where fields are in the simplified layout.
@@ -26,6 +26,6 @@ Let us know if there is extra configurability that you need, we are open to rest
See the [PR](https://github.com/bitfocus/companion/pull/3569) for more examples on the impact.
-### Connection processes now include the label (name) of the connection {#connection-process-name}
+## Connection processes now include the label (name) of the connection {#connection-process-name}
To improve the developer experience, when looking at each process in task manager or activity monitor, each process is now labelled with the label of the connection.
diff --git a/for-developers/module-development/api-changes/v1.5.md b/for-developers/module-development/api-changes/v1.5.md
index c0e2f43..51ff325 100644
--- a/for-developers/module-development/api-changes/v1.5.md
+++ b/for-developers/module-development/api-changes/v1.5.md
@@ -3,7 +3,7 @@ title: API 1.5 (Companion 3.1+)
sidebar_position: -105
---
-### Automatic invert property for boolean feedbacks {#feedback-invert}
+## Automatic invert property for boolean feedbacks {#feedback-invert}
Boolean feedbacks now automatically get an 'inverted' property added by Companion.
diff --git a/for-developers/module-development/api-changes/v1.7.md b/for-developers/module-development/api-changes/v1.7.md
index 4d77a96..902346f 100644
--- a/for-developers/module-development/api-changes/v1.7.md
+++ b/for-developers/module-development/api-changes/v1.7.md
@@ -3,12 +3,12 @@ title: API 1.7 (Companion 3.2+)
sidebar_position: -107
---
-### Bonjour/MDNS device discovery config field {#bonjour-discovery}
+## Bonjour/MDNS device discovery config field {#bonjour-discovery}
Many devices use Bonjour or MDNS for discovery.
There is a new `bonjour-device` config field type that you can use to easily perform this discovery for your users as they open the configuration panel.
-### Hex color format and alpha channel support {#color-picker}
+## Hex color format and alpha channel support {#color-picker}
The `color` input field type now supports providing colors as `#000000` format hex strings, and can have alpha support enabled
diff --git a/for-developers/module-development/api-changes/v1.8.md b/for-developers/module-development/api-changes/v1.8.md
index 9b9c1e7..490337e 100644
--- a/for-developers/module-development/api-changes/v1.8.md
+++ b/for-developers/module-development/api-changes/v1.8.md
@@ -3,13 +3,13 @@ title: API 1.8 (Companion 3.3+)
sidebar_position: -108
---
-### Text preset type for organizing presets with headings {#text-presets}
+## Text preset type for organizing presets with headings {#text-presets}
A new 'text' preset type has been added, to allow you to put some headings and blocks of text into the presets panel.
These will also split the presets into chunks around them, allowing you to organise presets better.
-### Support for button-scoped local variables {#local-variables}
+## Support for button-scoped local variables {#local-variables}
:::tip
@@ -24,7 +24,7 @@ Due to the way the `parseVariablesInString` method works, Companion often doesn'
In order to support these, in your action/feedback callback, there is a second `context` parameter which holds an alternate `parseVariablesInString` implementation. This implementation is specific to that callback, so Companion knows what control it belongs to, and can handle the variables.
Additionally, you can indicate that you are doing this and support these variables by setting the `useVariables` property to an object like `{ local: true }` to indicate this support. This allows us to show a hint to the user about this support, and suggest them whilst they type.
-### Shared UDP port listener for devices with hardcoded ports {#shared-udp}
+## Shared UDP port listener for devices with hardcoded ports {#shared-udp}
A few devices have been found which are not cooperative when it comes to control, and expect to send all messages to a hardcoded UDP port.
This makes it hard to support these, as by default only one connection can listen on a port at a time.
diff --git a/for-developers/module-development/api-changes/v2.0.md b/for-developers/module-development/api-changes/v2.0.md
index 44fd8a2..423f8a9 100644
--- a/for-developers/module-development/api-changes/v2.0.md
+++ b/for-developers/module-development/api-changes/v2.0.md
@@ -70,9 +70,9 @@ If you don't already, we also recommend defining the `$schema` property to give
```json
{
- "$schema": "../node_modules/@companion-module/base/assets/manifest.schema.json",
- "type": "connection",
- "id": "your-module-name"
+ "$schema": "../node_modules/@companion-module/base/assets/manifest.schema.json",
+ "type": "connection",
+ "id": "your-module-name"
}
```
@@ -301,20 +301,20 @@ The `structure` array describes how to present these presets in the UI. For exam
```js
const structure = [
- {
- id: 'a',
- name: 'Main A',
- // optional description
- definitions: [
- {
- id: 'b',
- type: 'simple',
- name: 'Output 1',
- // optional description
- presets: ['something', 'another'],
- },
- ],
- },
+ {
+ id: 'a',
+ name: 'Main A',
+ // optional description
+ definitions: [
+ {
+ id: 'b',
+ type: 'simple',
+ name: 'Output 1',
+ // optional description
+ presets: ['something', 'another'],
+ },
+ ],
+ },
]
```
@@ -421,11 +421,11 @@ If no type is specified, the default is:
```ts
export interface InstanceTypes {
- config: JsonObject
- secrets: JsonObject | undefined
- actions: Record