Information required: Format and encoding of the blueprint

[InputBlackBoxOutput]

Hello there,

I am working on an HDL synthesizer for Redstone circuits and would like to use SCM to minify the gate-level logic. I am not fluent with Java and would appreciate it if you could help me with the following information:

• Blueprint encoding while saving as a gist
• Format for decoded blueprint
• Possible ways to integrate a JSON file having gate-level logic into the code

Thanks for making such a wonderful MOD!

[amadornes]

I'm honestly really glad someone is stepping up to the challenge of writing a netlist to SCM synthesizer. I was contemplating developing an official one later down the line, but it's going to be a lot of work so it's not very high up in my TODO list.

SCM 2 only came out very recently, so the blueprints and redprints (and the respective sharing functionality) from SCM 1 aren't implemented yet. There's been several major architectural and design improvements to the mod which will definitely make your life easier like blueprints now storing all connected tiles instead of just the one you click on, but I haven't settled on a format for storing and sharing blueprints yet.

The format in SCM 1 is a base-64 encoded version of the binary NBT data of the circuit tile. So basically the same that would be written to disc, but base-64 so it doesn't need to be a binary file. It worked fine, but it definitely wasn't an ideal approach.

SCM 2 will almost definitely have a completely new format, most likely based on JSON to allow third party tools like your synthesizer, circuit visualizers/testers and other design tools to work with circuit designs directly.
This is going to take some time to develop, but I'd like to have a working version of blueprints and design sharing by the end of January. I can't 100% promise I'll make that deadline, but that's what I'm aiming for.

Hopefully that helps :)

I'll keep the issue open and post any updates I have here as I work on the new implementation.

[dkranke]

Hello there,

I like the idea of using a HDL for SCM. Not only as an synthesizer, but also to reduce the complexity of circuits updates by building a state machine with it, like the nand2tetris simulator.

SCM2 is still quite far from done and the current beta (0.1.0) doesn't has blueprints, but if the encoding will have any similarity with SCM1 blueprints, they will be most likely also be base64 and zipped nbt-data.

If you want to decode SCM1 blueprints, you need to first base64 decode and then unzip the plain text (not a archive).
Here you will find the nbt-data from a analog memory cell represented in JSON.

I can give you more details if you need them.

EDIT: Just saw @amadornes comment. Apparently the old format will be obsolete, so this comment is obsolete.

[amadornes]

The old format was definitely not meant to be decoded and used outside of SCM, nor to support things like components introduced by 3rd party addons or circuits made up of more than a single tile.
At that time that seemed reasonable enough, but it's not really sustainable (nor was it ever meant to be).

May as well do it right this time around!

Thanks for providing a concrete example, though! It's always good to have old formats documented somewhere in case somebody wants to make a converter once the new one is out.

[dkranke]

I already have a few questions as I'm brainstorming about a new json format.

What would be required or prefered properties for a json format?
Should it be optimized for size or (human) readability?
Should they contain addon and/or version information?
Should all blueprints be editable (even recursive?) or only save the server side representation (behaving like hidden circuits)?
...

[amadornes]

I don't think the first version should necessarily worry too much about either file size or human readability, though that'd of course be nice. As long as the data is represented in a sensible way, I'd say that's enough to cover both of those.

Versioning will definitely be a thing this time around, both for the format as a whole and for component data. Things might change how they work internally at any point, so it's good to have version data to make sure nothing breaks.

Blueprints will remain editable. That said, the old circuit-in-circuit behavior is being replaced by a new mechanic that will make better (and more interesting imo) use of blueprints.
TL;DR: You'll be able to assemble (unmodifiable but recyclable) ASICs from blueprints, which you can use standalone or fit in other circuits. To place a blueprint containing ASICs you'll need those in your inventory. To create an ASIC from a blueprint containing other ASICs, you'll need the blueprints for all of the ASICs involved (recursively) since lore-wise they're etched into just one silicon wafer. ASIC fabrication will take time (more if there's nested designs), resources (lore-wise for doping the silicon), and potentially energy.

There's definitely a lot of stuff to consider, which is why I'm planning on taking as long as I need to figure it out and make something really solid, but that should give you a rough idea of some of my current goals with it.

[dkranke]

This might be a sketch for a possible format (representing the analog memory cell mentioned above), but still requires a lot of work.

It contains the following:

• format version
• meta data for the circuit
• position/placement of (indexed) tiles
• a BOM of used components (refered by index in array, to reduce file size)
• a list of all componentes with their position (tile and index [here: x+7z+49y]) and their attributes

Wires are represented in groups (reducing file-size and resolving connections to other components), colors as indicies and rotations/orientations are enumerated (0-3 clockwise, 4=up, 5=down).

A thumbnail could be interesting for circuit libraries/exchanges.

EDIT: Updated sketch (first version, second version)

[InputBlackBoxOutput]

I did some research and brainstorming on how this can be implemented without getting into the MOD codebase. I will be using Python to tap into Yosys for synthesis from Verilog code and then map the intermediate JSON netlist format used by the tool to the JSON format that SCM2 will use.

I am going to create a web app where one can enter Verilog code or drag and drop blocks (Blockly) which will be processed at the backend to generate a link to the SCM JSON data. This data can then be imported by SCM2 using scm import URL. I also plan to add a viewer for the synthesized cells and nets.

To Do

[] Tap into Yosys and mapping
[] Web app backend using Python Flask
[] Web app frontend using Vanilla JS or ReactJS

Repository: https://github.com/InputBlackBoxOutput/RedstoneHDL

[dkranke]

You can prepare some stuff but my JSON format was just a proposal and according to the Discord there will be a lot of changes.
Hopefully we will have the proper format in a couple of weeks/months. Also even the way how data is imported or exported might get different.

Actually I'm currently in creating a model viewer which would be the perfect addition to your web app.

[dkranke]

How do you plan on creating the verilog interface definition?
I mean a circuit can have none, 1 input, 1 output and/or 1 bundled wire (if bundled wires behave like in SCM1 they use shared I/O) on each tile side to interface with the outside world.

As it depends on the side count of the final circuit, i find that quite hard to model.

[InputBlackBoxOutput]

@dkranke I plan to map each side of a circuit to one input or output and then internally connect all the inputs, outputs and modules using a circular bus made of bundled wire. Not sure if this is the optimal way. I'll look for options when I complete the code regarding placement and routing.