Add the possibility to log arrays

[mallenbach]

In embedded systems, it is sometimes interesting to log the content of a buffer (like a received frame) - usually a buffer of bytes.

@an-dr - do you think it might be interesting to add a function that takes an array of byte and a length as parameters and do the conversion to a string internally to then call ulog_log?

[an-dr]

I print arrays of bytes myself quite often, but it should be properly designed, do you have examples of such logs? From implementation side, it is an interesting task, much more complicated that it might look like from the first glance.

Here are my thoughts and this we questions should answer before implement this:

• It should be an optional feature
• We should come up with an intuitive interface (logtbuf_info(TOPIC, BUFFER, SIZE), logtbuf_info(TOPIC, MESSAGE, BUFFER, SIZE)? Or only without topics?)
• We should print it by chunks using intermediate buffer, otherwise we risk to run out of memory
• It will require changes in ulog_Event and log_print_event
• Probably it will require introduction of print_source type in addition to print_target
• What the output format it should be? How to keep it readable if we print, let say 1024 bytes?

[mallenbach]

From implementation side, it is an interesting task, much more complicated that it might look like from the first glance.

For sure. It is not just printing bytes side by side :-).

It should be an optional feature

Agree

We should come up with an intuitive interface (logtbuf_info(TOPIC, BUFFER, SIZE), logtbuf_info(TOPIC, MESSAGE, BUFFER, SIZE)? Or only without topics?)

I like the fact that we can have a kind of title before printing the byte stream. logtbuf_info(TOPIC, MESSAGE, BUFFER, SIZE) is a good option - MESSAGE can be set to NULL if one does not want to print it.

Do you think that offering both with / without topic option would make it much more complicated to implement?

We should print it by chunks using intermediate buffer, otherwise we risk to run out of memory

It depends the way lower layers are implemented. If one use a DMA, it can be direct. But if using an intermediate ring buffer for instance, printing it by chunks makes sense.

What the output format it should be? How to keep it readable if we print, let say 1024 bytes?

It can printed the way a memory dump in a debugger is. We can make the length configurable at compile time - people will usually chose between 16 and 32-byte long lines. We should also implement it properly aligned to ease

[an-dr]

So with the latest update I have a better vision on the feature

• It should be an optional feature

  • Yes, on by default

• We should come up with an intuitive interface (logtbuf_info(TOPIC, BUFFER, SIZE), logtbuf_info(TOPIC, MESSAGE, BUFFER, SIZE)? Or only without topics?)

  • I introduced ulog(LEVEL, MESSAGE) and ulog_t(LEVEL, TOPIC, MESSAGE), short and generic macros. The interface can be ulog_arr(LEVEL, ARRAY_POINTER, SIZE)

• We should print it by chunks using intermediate buffer, otherwise we risk to run out of memory

• What the output format it should be? How to keep it readable if we print, let say 1024 bytes?

  • I propose to print by chunks of 8, each chunk will be a line. Will work for all situation, not much memory needed, maybe can be even printer with ascii:

     00 01 02 03 04 05 06 07
000  48 49 4A 4B 4C 4D 4E 4F   HIJKLMNO
008  50 51 52 53 54 55 56 57   PQRSTUWV
016  58 59 5A 5B 5C 5D 5E 5F   XYZ[\]^_
024  60 61 62 63 64 65 66 67   `abcdefg

• It will require changes in ulog_Event and log_print_event
• Probably it will require introduction of print_source type in addition to print_target
  • This will require experiments, but it's gonna be fun 😃

[mallenbach]

@an-dr - Based on your previous comment, do we consider that adding this feature is part of the roadmap (and no longer under consideration)? If yes, I´d like to take care of that (in a timely manner this time :-)).

[an-dr]

I would be happy to have it! No rush this time, do it in your pace.

[an-dr]

Reference: https://github.com/mlyszczek/embedlog

It has the memory printing

[mallenbach]

Thanks for the pointer. Is the look and feel similar to what you would like to see added to the library?

[an-dr]

@mallenbach Hi! I spent some time on this... Here is the complete spec

Interface

Macros:

ulog_array_log(LEVEL, TOPIC_NAME, ARR, ARR_SIZE, ...)
ulog_a_log(LEVEL, TOPIC_NAME, ARR, ARR_SIZE,...)

Functions:

void ulog_topic_array_log(ulog_level level, const char *file, int line, const char *topic,
              const uint8_t* array, const size_t array_size, 
              const char *message, ...)
              
void ulog_array_log(ulog_level level, const char *file, int line,
              const uint8_t* array, const size_t array_size, 
              const char *message, ...)

Output

INFO  src/main.cpp:4: Log message:
    Offset | 00 01 02 03 04 05 06 06  08 09 0a 0b 0c 0d 0e 0f | ASCII
-----------|--------------------------------------------------|------------------
0x00000040 | 40 41 42 43 44 45 46 47  48 49 4a 4b 4c 4d 4e 4f | @ABCDEFG HIJKLMNO
0x00000050 | 50 51 52 53 54 55 56 57  58 59 5a 5b 5c 5d 5e 5f | PQRSTUVW XYZ[\]^_
0x00000060 | 60 61 62 63 64 65 66 67  68 69 6a 6b 6c 6d 6e 6f | `abcdefg hijklmno
0x00000070 | 70 71 72 73 74 75 76 77  78 79 7a 7b 7c 7d 7e 7f | pqrstuvw xyz{|}~.

• Non-printable ASCII = .
• Empty arrays: print zeros for all size
• Array is not aligned by 16 - use .: eg

uint8_t arr[] = {40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 4a, 4b, 4c}

vvv

INFO  src/main.cpp:4: Log message:
    Offset | 00 01 02 03 04 05 06 06  08 09 0a 0b 0c 0d 0e 0f | ASCII
-----------|--------------------------------------------------|------------------
0x00000000 | 40 41 42 43 44 45 46 47  48 49 4a 4b 4c  .  .  . | @ABCDEFG HIJKL...

• On empty arrays and NULL print explicitly uint8_t arr[0]; // size = 0:

INFO  src/main.cpp:4: Log message: 
 Empty array

On too big arrays:

INFO  src/main.cpp:4: Log message: 
 4097 byte array (Max printable size: 4096)

Implementation notes

• A buffer for a output string (82 bytes in that case):

0x00000000 | 40 41 42 43 44 45 46 47  48 49 4a 4b 4c  .  .  . | @ABCDEFG HIJKL...

• Read buffers by 16 bytes, then use the 82 bytes buffer for printing a string
• Const header: + 151 bytes:

    Offset | 00 01 02 03 04 05 06 06  08 09 0a 0b 0c 0d 0e 0f | ASCII
-----------|--------------------------------------------------|------------------

• If the feature is disabled all the buffers and unused consr are conditionally excluded by the preprocessor

Build configs

• Enabled by default
• ULOG_BUILD_ARRAY_LOGGING: 1/0 (default: 1)
• ULOG_BUILD_ARRAY_MAX_SIZE: int (default: 4096)