Separate compiler from VM

[hiperiondev]

For embedded applications will be good to separate LD and IL compiler and allow to load precompiled file

[kalamara]

this is actually one of the first things i came across when i considered porting to freeRTOS.
LD and IL compiler are already separated in the sense that you can load a .IL file.
the IL compiler is really just a deserializer, it encodes the IL operands and operations and loads them in a struct in RAM but there is no compilation, the same instruction list that is in the file is also in the memory.
But the real problem is that as it is now, you need a filesystem to load a file from and that is not always available in embedded systems.
It's not so simple, you would need to somehow store the instruction list already binary encoded into a flash memory and load it from there.
maybe make a plc_load_from_memory and use that one instead of plc_load_file.
if you implement it i am interested to see.

[hiperiondev]

Yes, indeed ESP-IDF is a framework over FreeRTOS.
I'll try to make some test with this enhancement

[hiperiondev]

In ESP32 I can use internal flash or external SD. Other uC can use external SD.
You can see an example of internal flash use with LittleFS on: https://github.com/hiperiondev/esp32-berry-lang

[hiperiondev]

At first I think codeline_t code, at rung, should be optional. This consume a lot of memory

[hiperiondev]

I have decoupled compiler and made parsers and code rung optional.
Now compiler can be constructed stand alone.
Not do anything but is the first step :-)

[hiperiondev]

Now parse files ld and il. Do not generate vm file

For construct compiler:
$ Config.sh
$ cmake --build build-compiler

for use:
build-compiler/librelogic-compiler -i input file -o output file

Works with ladder but not parse IL, still don't know why

https://github.com/hiperiondev/librelogic

[hiperiondev]

Found problem, dump rung not create compilable IL: not add % to arguments, ( modifier are separated by space

[hiperiondev]

Finished compiler. Can you test it?

[kalamara]

Finished compiler. Can you test it?

nice! i will try to find some time in the weekend, or monday

[hiperiondev]

I was thinking about a modification of the VM to generate a more compact code and faster execution. When I have a little time I'll try it, but the idea is the following:

// VM INSTR
//        [        INSWORD2 (25)          ]
//              [      INSWORD1 (21)      ]
//                     [   INSWORD0 (16)  ]
// [INSBYTE0][INSBYTE1][INSBYTE2][INSBYTE3]
// [IIIIICNP][RVXOOOOO][BBBBBBBB][TTTTTTTT]
// I: il instruction
// C: conditional
// N: negate
// P: push
// R: return
// V: byte:bit/var
// X: (not used)
// O: operand
// B: byte
// T: bit

#define INSBYTE0(x)  ((x & 0xFF000000) >> 24)
#define INSBYTE1(x)  ((x & 0x00FF0000) >> 16)
#define INSBYTE2(x)  ((x & 0x0000FF00) >> 8)
#define INSBYTE3(x)  (x & 0xFF)
#define INSWORD0(x)  (x & 0xFFFF)
#define INSWORD1(x)  (x & 0x1FFFFF)
#define INSWORD2(x)  (x & 0x1FFFFFF)

#define IL(x)        (x >> 27)
#define OPERAND(x)   (((INSBYTE1(x) & 0xF8) >> 3) + 20)
#define COND(x)      (INSBYTE0(x) & 0x02)
#define NEGATE(x)    (INSBYTE0(x) & 0x04)
#define PUSH(x)      (INSBYTE0(x) & 0x01)
#define RETURN(x)    (INSBYTE1(x) & 0x80)
#define VAR(x)       (INSBYTE1(x) & 0x40)

#define SET_IL(i, v)        (i | (x << 27))
#define SET_OPERAND(i, v)   (i | (x << 16))
#define SET_COND(i)         (i | 0x4000000)
#define SET_NEGATE(i)       (i | 0x2000000)
#define SET_PUSH(i)         (i | 0x1000000)
#define SET_RETURN(i)       (i | 0x800000)
#define SET_VAR(i)          (i | 0x400000)

[hiperiondev]

Boring Sunday, new vm :-)

Here compiler(assembler), decode instruction and vm skeleton
https://github.com/hiperiondev/librelogic_newvm

[kalamara]

interesting, i haven't thought at all about optimizations as i haven't even made any meaningful memory usage or cpu profiling measurements yet. maybe your standalone compiler can be useful for that :)
is it working? i get the general idea but would like to see exactly how much better it performs in numbers.
the goto loop with the labels is pretty cryptic to me. i normally avoid gotos and use switch / return statements instead, it perfroms the same but makes for easier code to understand and maintain i think.

[kalamara]

Finished compiler. Can you test it?

nice! i will try to find some time in the weekend, or monday

i tried it out, it works (if you have the program in the same folder, otherwise it crashes), i like the addition of .vm file format, maybe you could somehow add a flag to compile directly from .ld to .vm.

have you considered GNU getopt for easier command line options handling instead of custom code? no need to reinvent & maintain the wheel i think. this project has had this mistake many times in the past.
overall it's a good addition i think, kudos :)

[hiperiondev]

interesting, i haven't thought at all about optimizations as i haven't even made any meaningful memory usage or cpu profiling measurements yet. maybe your standalone compiler can be useful for that :) is it working? i get the general idea but would like to see exactly how much better it performs in numbers. the goto loop with the labels is pretty cryptic to me. i normally avoid gotos and use switch / return statements instead, it perfroms the same but makes for easier code to understand and maintain i think.

There is a lot of literature and discussion about the advantages of computed goto and threaded code (and much more flame). It really depends on the architecture and the compiler (gcc has flags to optimize this technique). I did several tests and there is not much difference in a PC CPU but there is usually a lot in microcontrollers. Many VMs use it for that reason (Python uses it optionally with the HAVE_COMPUTED_GOTOS flag).
It's not a huge modification but it can have a big impact on embedded architectures.

On the other hand, having an instruction format closer to a real CPU greatly reduces fetching times and op decoding only depends on instructions that are resolved by hardware (and, or, shift) so they are very efficient. The code is much more compact and allow some peephole optimizations

The compiler (actually an assembler) and the instruction decoder work correctly.
The VM is just a skeleton, I have to add the real functionality of the rest of the project to it. But I already have experience in other CPU type VMs, it's not a complex problem.

[hiperiondev]

Finished compiler. Can you test it?

nice! i will try to find some time in the weekend, or monday

i tried it out, it works (if you have the program in the same folder, otherwise it crashes), i like the addition of .vm file format, maybe you could somehow add a flag to compile directly from .ld to .vm.

Is already possible compile from .ld to .vm. You can try it and see how it works.
I can use path for file, not crash for me. Can you give to me an example?

have you considered GNU getopt for easier command line options handling instead of custom code? no need to reinvent & maintain the wheel i think. this project has had this mistake many times in the past. overall it's a good addition i think, kudos :)

Yes, of course. I have used this very simple library for concentrate in functionality.