MicroSFSC

The Shop-Floor Service Connector (SFSC) is an easy-to-use, service-orientated communication framework for Industry 4.0, developed at the University of Stuttgart. MircoSFSC is a microcontroller ready implementation of the SFSC adapter role. This documentation provides a quick overview about the usage and the features of the MicroSFSC framework, whereas an in deep analysis can be found in this white-paper(German). This README presumes that you are familiar with SFSC and its concepts like adapters, cores, services (i.e. server services and publisher services), subscriptions, requests, the service registry, etc.

This implementation is written in C and conforms the C99 freestanding standard. If your compiler only supports ANSI-C, some platform depended adjustments are required.

The RAM and ROM footprint is configuration depended; in default configuration 25kB ROM and about 7-11kB RAM are needed. A configuration for your specific use-case will most likely result in lower resource requirements.

This framework does not use dynamic memory allocation.

Repository structure

This repository is structured the following way:

• The src/sfsc folder contains the actual source code you should copy into your project.
• The src/platforms folder contains implementations for the platform dependencies for some platforms. Platform dependencies are explained in the next section.
• The docs folder contains a doxygen html documentation of the public header. It can be accessed via github pages. Also, the docs folder contains latex code and the complete documentation as PDF.
• The src/examples/scenarios folder contains the actual examples. Every example comes with its own preprocessor directive that must be defined to enable that example. Only one of the examples should be active at the same time. Where to define the directives is up to the used build system. The src/examples/shared folder contains shared example code for the platforms, most notably a common API to log things. The other subfolders of the src/examples folder contain the initialization logic and example build instructions to the get the examples running on the corresponding platform. A good starting point for the examples is the pubsub example, then you can study the a bit more complex reqrepack example and finally the query example.

Porting to your platform

The sfsc/platform folder contains headers you have edit or implement for your platform if you want to use this framework. There are some existing platform ports in the platforms folder in the root of this repository. For example, there are implementations for POSIX (and Windows with MinGW) systems and the ESP32 microcontroller family. This framework does not contain an IP stack, so you need to provide one (most network ready platforms will already include one). Below are some more information on the headers you need to look at.

sfsc_types.h

sfsc_types.h contains declarations of different datatypes. If your platform provides stdint.h and stdbool.h (like all conforming freestanding C99 environments should), there is no need to edit this file. The other needed header files are from the ANSI-C (C90) standard.

sfsc_strings.h

This framework needs some functions from the strings.h header (namely memcpy, memset and strlen), which is not part of any freestanding C standard. Many platforms will provide it anyway, and if you platform does, make sure to define HAS_STRING_H in sfsc_adapter_cofig.h (more about configuration can be read below). If this define is missing, the framework will fall back to self-provided, but not as efficient implementations of these functions.

sfsc_platform.h

This header provides four functions you need to implement. They should behave as their function documentation demands, some important additional details are given here:

• time_ms is used for timing and must return some means of the current time, in millisecond resolution. It is not necessary (but allowed) to provide the absolute unix time, a relative value to indicate time since the system start (or even first call of the time_ms function) is sufficient.
• random_bytes needs to generate the requested amount of random bytes and write them to the specified buffer. These bytes are not used for cryptographically functions, so it's ok for them to be pseudo-random. On the other hand, it is very important that the generated byte sequences are different on each system start! If your platform does not provide such a mechanism you have to do this yourself: You can use a pseudo-random generator algorithms seeded with some random sensor noise (read more on the von Neumann whitening algorithm) or the absolute unix time (if you can access it). An other approach is to store the last generated bytes in persistent memory (EEPROM), and seed your pseudo-random algorithm with them on system start. If you use this framework on multiple microcontrollers simultaneously, the generated random sequences must not be the same, too!
• The lock and unlock functions are only needed for multi-threading, if you don't use multi-threading you can provide empty implementations of these functions. If you use multi-threading you have to ensure that only one thread is accessing a socket for writing at the same time. This should be no problem, since the most kernels and operating systems (which most likely provide the multi-threading in the first place) provide synchronization mechanisms. The framework will call you with an address to an uint_8. The address identifies the socket that should be locked/unlocked for access through other threads (you can treat the address like a numerical handle). The uint_8t is a single byte you can freely use as memory if needed (e.g. to store locking information).

sfsc_sockets.h

SFSC adapters communicate with SFSC cores using TCP/IP, and you need a to provide functions to establish this connection. The framework does not contain a own network-stack, since if your platform is IP-ready, it will most likely feature a platform optimized implementation. The API demanded by this framework is based on the POSIX Socket API (also known as BSD Socket), with the extension that most functions need to operate in a non-blocking way. Examples for providing this functions can be found in the platforms folder.

Protobuf

This framework uses NanoPB v0.4.2 for protobuf serialization/deserialization. It is configured to not use dynamic memory allocation, and is pointed towards sfsc_strings.h instead of the normal strings.h. NanoPB also depends on stdbool.h, stdint.h, stddef.h, and limits.h in its pb.h. If your compiler does not conform the freestanding C99 standard, you might need to edit the pb.h header. For more information on NanoPB see it's official page.

The Execution model

The framework needs some background tasks to be handled. But it does not include any concept of threading (since your platform might already have a concept, or is not powerful enough to have such a concept). Instead, it is your responsibility to call the background tasks in a cyclic manor. There are two main tasks you need to call: The system task and the user task. Both tasks are designed to be non-blocking, but for the user task, there are some restrictions.

System Task

The system task handles connection setup and heartbeating. It will also read the network using your sfsc_socket.h implementation. It is important that the system task is called with a high enough frequency, or it will fail to send heartbeats to the core in time. If this happens, the core will treat the corresponding adapter as disconnected. As a guideline, try to call the system task at least once every 5ms. The system tasks runtime is designed to be constant, which is achieved by only using non-blocking operations. As a consequence, the system task does not invoke your callbacks, as the framework can not know, what you are doing in your callbacks. So, instead of executing callbacks based on the data the system task receives from the network, it writes the data to an intermediate ring buffer, called the user ring.

User Task

Among other things, the user task will then go ahead and read data from the user ring, and based on them, invoke callbacks you defined during an API call. Since the user task directly executes your callbacks, what you do in them will influence the runtime of the user task. Reading one entry of data from the user ring and invoking a callback is called a micro step. How many micro steps should be taken in a single call to the user task function can be configured (using REPLAYS_PER_TASK). The data supplied as parameters to your callbacks are only valid during the current micro step, meaning that after your callback returns, the data will be removed from the user ring and you should no longer try to access them. If you need the data from the callback outside the callback, you have two options:

1. Simply copy the data somewhere else. This is the easiest and safest way, but requires additional memory.
2. If you can not afford to copy the data, you can enter the user task pause state. In the user task pause state, the user task won't advance to the next micro step after your callback returns. You will need to leave the pause state explicitly by calling a function. Keep in mind, that as long as you are in the pause state, no further callbacks will be invoked (exceptions are listed below). Note that even while you are currently in the pause state, you still must ensure that the system and user task functions are called (since as noted above, the user tasks needs to do other things then taking micro steps, too)!

Exceptions to the user task pause state

The pause state actually only affects messages that are stored in the user ring. Some callbacks that are triggered by the user task are not subject to data from the user ring: The user ring is used to transport data of variable length from the system task to the user task. To transport data of known length, like a single numerical value or even a bitflag, it is more appropriate to use strongly typed struct fields. The following callbacks are not affected by the user task pause state:

• command_callback from register_publisher, unregister_publisher, register_server, unregister_server
• on_subscription_change callbacks of a sfsc_publisher struct
• on_ack callbacks from answer_request and answer_channel_request,
• on_answer callbacks from request and channel_request, both ONLY in the timeout case, the data case is subject to the user ring
• on_service callbacks query_services

Think of the following scenario to see that this behavior (of some things continue running) is actually beneficial (this scenario seems a little out-of-the-sky for now, but as you start getting familiar with the framework, something similar will most likely come to your mind):

1. You provide a non-fire-and-forget (meaning that your answers must be acknowledged by the requestor) server service
2. You receive a request and the servers on_request callback is triggered
3. You make an answer call using answer_request, which requires the reply_topic parameter from the on_request callback
4. According to the documentation, the reply_topic must be valid until the on_ack callback is triggered BUT the parameters of the on_request callback (i.e. reply_topic) are subject to the user ring and thus only valid during the current on_request call UNLESS you decide to pause the user task, what you then do by setting *b_auto_addvance=0 and thus using option 2 (see above)
5. Since on_ack callbacks are still triggered in the user task pause state, on_ack will eventually be triggered and you can leave the pause state from there

Blocking the User Task

An other important question is: Am I allowed to perform a blocking or long taking operation in a callback? As stated above, system and user task are designed to be non-blocking. Consider the following code snippet:

// Note that this code is not runnable since it ignores multiple return values of functions that indicate the operability of the adapter
sfsc_adapter adapter;
start_session_bootstraped(&adapter,host,PORT);
while(1){ //global execution loop
	system_task(&adapter);
	user_task(&adapter);
}

So if your callback now blocks execution, the user task will also block, and therefore, the whole program is blocked, preventing the system task from running. An approach to move the blocking code out of the callback will lead to this:

sfsc_adapter adapter;
start_session_bootstraped(&adapter,host,PORT);
while(1){ //global execution loop
	system_task(&adapter);
	user_task(&adapter);
	blocking_user_code();
}

This is obviously not better than the frist approach, since it will also prevent system_task from being called. You can try to design your function in the system and user tasks manor and split your long taking operation in multiple substeps and use only non-blocking functions yourself. Then, each invocation of your function it will execute only a tiny step of your overall goal. This will result in the following, valid approach:

sfsc_adapter adapter;
start_session_bootstraped(&adapter,host,PORT);
int step_number=0;
while(1){ //global execution loop
	system_task(&adapter);
	user_task(&adapter);
	non_blocking_user_code(step_number);
	step_number++;
}

So to conclude, if your system task and user task run in the same thread, callbacks and other things you do in your global execution loop should not block.

Multi-threading

If you decide to use multi-threading you can set up the system and the user task as different threads. Then, blocking the user task won't interfere with the execution of the system task and is thus allowed. This framework does not include a scheduler, so you have to use your own threading solution. Keep in mind that if you use the a multi-threaded execution model, you might have to add some synchronization (see the sfsc_platform.h section above).

Message drop

Since the system task writes data to the user ring, and the user tasks takes data from the user ring, it can happen that the user ring gets filled faster than it is emptied (especially if the user task is paused for too long!). In this case, newly received data are dropped (according to the ZMTP PUBSUB specification on which SFSC is based).

Further reading

For an in deep discussion and reasoning why this execution model is used, see the white-paper, chapter 4.9.

Configuration

There are several configuration options to adjust the MircoSFSC framework. They can be found in sfsc_adapter_config.h and zmtp_config.h and will influence the RAM (and some even ROM) consumption of a sfsc_adapter struct. The table below list the configuration options, where they can be found, what they are for, and how they change memory consumption. Also, the default values (either a numerical value, or whether the option is defined or not) are listed. Note that all memory sizes are only guide values, the actual values will depend on your platform (e.g. on the pointer size or your memory alignment rules). A xN (where N is a number) in the memory column means that the configured value times N bytes of memory is needed.

Parameter Name	Header	Default	Memory Impact	Description
REPLAYS_PER_TASK	sfsc_adapter_config.h	0	-	The maximum number of micro steps that should be taken per user_task function invocation, 0 for as many as possible. Example: If there are 6 items in the user ring, and REPLAYS_PER_TASK is set to 4, only the first 4 of them will be processed in this user_task call, the next 2 have to wait until the next user_task call.
HEARTBEAT_SEND_RATE_MS	sfsc_adapter_config.h	400	-	The time to wait between sending outgoing heartbeats in milliseconds.
HEARTBEAT_DEADLINE_OUTGOING_MS	sfsc_adapter_config.h	HEARTBEAT_SEND_RATE_MS * 4	-	If it was not possible to send at least one heartbeat in this time (denoted in milliseconds) - most likely due to the system_task function not being called frequent enough - an error will be raised.
HEARTBEAT_DEADLINE_INCOMING_MS	sfsc_adapter_config.h	4000	-	The amount of time in milliseconds in which a heartbeat from the core needs to arrive. If there is no heartbeat in this amount of time, the SFSC session will be treated as terminated.
HAS_STRING_H	sfsc_adapter_config.h	defined	a few bytes of ROM if NOT set	Should be defined if your platform has a memcpy, memset and strlen function. If not defined, inefficient fallback implementations of this functions will be used.
MAX_PUBLISHERS	sfsc_adapter_config.h	6	x4 RAM	Amount of publisher services, a single adapter can operate at the same time. See the register_publisher function documentation for more information.
MAX_SUBSCRIBERS	sfsc_adapter_config.h	12	x4 RAM	Amount of subscriptions to publisher services, a single adapter can operate at the same time. See the register_subscriber function documentation for more information.
MAX_PENDING_ACKS	sfsc_adapter_config.h	6	x48 RAM	Amount of pending acknowledges to transmitted server-service-answers a single adapter can keep track of at the same time. See the answer_request function documentation for more information.
MAX_SERVERS	sfsc_adapter_config.h	6	x4 RAM	Amount of server services, a single adapter can operate at the same time. See the register_server function documentation for more information.
MAX_SIMULTANIOUS_COMMANDS	sfsc_adapter_config.h	6	x24 RAM	Amount of commands (used for creating or deleting something from a SFSC cores service registry) a single adapter can issue at the same time. Needed in the register_publisher, register_server and unregister_publisher, unregister_server functions, see there for more information.
MAX_SIMULTANIOUS_REQUESTS	sfsc_adapter_config.h	6	x40 RAM	Amount pending (not-yet answered) requests a single adapter can make at the same time. See the request function for more information.
REGISTRY_BUFFER_SIZE	sfsc_adapter_config.h	512	x1 RAM	When querying the service registry, received services are stored in a buffer of this size. Must not be greater then ZMTP_IN_BUFFER_SIZE (see below; there is no point in storing a service you can not even receive).
MAX_DELETED_MEMORY	sfsc_adapter_config.h	32	x37 RAM	When querying the service registry, it is necessary to keep track of services in the registry that are marked as deleted (see chapter 4.2 in the white-paper). This value determines, how many delete events can be stored; if you know that there are many services in your service registry, you may want to increase this value. Note however, that this is not a 1:1 relation and you don't need to set this value to the total number of services (if you have 100 services, you don't need to set this value to 100, 32 might be fine). To find the value suited the most for your usecase, you need to experiment a little.
USER_RING_SIZE	sfsc_adapter_config.h	5120	x1 RAM	The size of the user ring. The default value is chosen this high to prevent message drop, for platforms with lower RAM capabilities, a more appropriate value might be 1024.
NO_CURVE	zmtp_config.h	defined	5 KB(!) RAM and ROM if NOT set	Curve encryption is wip. It is recommended that you disable the functions related to CRUVE to speed up compilation and reduce the RAM and ROM footprint of the framework.
ZMTP_IN_BUFFER_SIZE	zmtp_config.h	512	x4 RAM	The size of the ZMTP receive buffer; determines, how big a single ZMTP message can be. If you know, that the services you use need to receive bigger payloads (e.g. because you want to subscribe a publisher whose messages are 1KB in size), you need to adjust this value.
ZMTP_METADATA_BUFFER_SIZE	zmtp_config.h	32	x8 RAM	ZMTP needs a place to store its meta data. You usually don't need to adjust this buffers size, unless you tweak the ZMTP implementation itself.
ENABLE_PRINTS	Your build system	defined	~1KB of ROM	The examples use platform depended mechanics to print something (to the closes thing to a console their is for the platform), to give you some more insights in what is happening. You can disable this for production. Where you set this directive is up to the used build system. NOTE: If you want to use this to see logs on a platform you ported the framework to, you need to implement the functions from examples/shared/console.h for your platform, too.

Using the API

All public API functions are defined in sfsc_adapter.h. You should include this header everywhere you want to use one of the frameworks function. To use most of the functions, you need a pointer to a sfsc_adapter. To declare a sfsc_adapter, you also need to include sfsc_adapter_struct.h. You should only include this header in the compilation unit you declare the adapter and not modify the struct fields of the adapter. The adapter has a stats field, which tells you some information about its current state. To access the stats field of an adapter, use the adapter_stats function. After declaring a adapter, you can use it to start a session. Use the start_session_bootstraped or start_session function with a pointer the the adapter. After the start_session method returns, your SFSC adapter is ready to perform the SFSC handshake, and you should start using the system_task function on the adapter. Once the state of the adapter (accessible through the stats field of the adapter struct) is operational, you can also start invoking the user_task function on the adapter, and use the other API functions with the adapter.

Most functions, including the system_task and user_task functions return error codes. An error code of SFSC_OK indicates, that a function call was successful. A full list of error codes can be found in sfsc_error_codes.h and zmtp_stats.h. All errors of zmtp_stats.h indicate that there was something wrong with data transport: either the network or the ZMTP protocol (on which SFSC is based) somehow failed. These errors are not recoverable, you should initiate a new SFSC session (you can reuse the adapter struct for that, as long as you set all the fields according to sfsc_adapter_DEFAULT_INIT, see section Struct initalization below). Errors of sfsc_error_codes.h are higher level and indicate problems with SFSC itself. They are well documented, and some are even recoverable, so look at their documentation.

Struct initalization

There are some struct types you'll encounter while using the framework. If you want to initialize a struct to its default values (just 0 in almost all cases), you can use the corresponding <struct_name>_DEFAULT_INIT macro. For some structs there is a constant default instance you can use to copy the default values over, and if there is, its name is <struct_name>_default.

Stop using the API

In SFSC, there is no way to gracefully stop an adapter. If you simply stop calling the system_task function on it, the adapter will stop sending heartbeats, and therefor the core will detect that an adapter timed out. Leftover services (services that where registered using that adapter but not unregistered manually) will then be removed from the cores service registry. Note that if the core is configured with a long incoming heartbeat deadline duration, it will take some time for the timeout detection to happen, and in consequence, some time for the leftover services to be removed automatically. Thus, depending on your usecase, you might want to unregister your services manually. An other important function in this context is release_session, which will take the socket handles an adapter used and call your sfsc_socket.h implementation to release the corresponding sockets.