Proposal - create a top level Python driver for the workflow

[oharboe]

I'm sure this has been discussed, but I thought I'd write down my thoughts on a top-level driver.

Today there's a number of siloed tools that have been stitched together with the alpha-release/flow/Makefile. That's great to get things going, but it was never intended to be the ultimate solution.

I would propose that a Python high level flow driver is implemented.

I would argue that Python is the correct choice:

• it is a modern widely used scripting language frequently used in continuous integration
• it has a vast library and infrastructure available to it
• it has excellent development tools: IDE, debugger, etc.
• it has a continuous development infrastructure and reporting(e.g. pytest, mocking, etc.)

This driver should have two APIs:

• the command line. Much along the line of "git" with two levels of arguments "openroad stage
• a Python module that can be imported and then the tools can be accessed

Some example use-cases:

• "openroad synthesize TopLevel" => synthesizes TopLevel.v
• "openroad back-annotate TopLevel" => runs all steps up to back-annotated timing after detailed routing, skips steps alread completed

Some features that become easy and natural to implement:

• reporting. Static HTML pages w/limited interactivity(friendly towards continuous integration servers) with images, etc. There's plenty of Python template libraries to support this, PDF reports.
• a feature to adjust size of die area or timing closure frequence until the required area and timing frequency is found. Requires looping through the tools a number of times, possibly with a binary search.

[tajayi]

As usual, thanks for the insightful feedback. You're right, the Makefile (and this repo) is not intended to be the final incarnation of the flow. It has been somewhat difficult trying to figure out how much resources to pour here, vs the final incarnation.

At the toplevel, the current inclination is to rely on tcl. TCL has traditionally been synonymous with EDA tools. Running a single "OpenROAD" executable will drop you into a tcl shell where you may be able to run commands like synth -f myDesign.v or route -option1 1 -option2 1 or report_area. There will also be a shared database and tighter integration between the tools.
For a batch flow, the user can pass a tcl run file to the program (e.g. OpenROAD -batch -f myFlow.tcl)

It possible/likely that a scripting language like python may be leveraged in the test/integration framework. Many discussions are going on around this. Hopefully once things are more solid we can post some sort of roadmap or plans for interested parties and contributors

[oharboe]

I can see the attraction of Tcl in that it would appeal to EDA users, but I believe tcl would be a mistake for the top-level driver.

The problem with Tcl is that it targets the wrong audience. What's needed for the top level driver implementation is not EDA skills, but strong generic software engineering skills.

Some examples of things that's needed that's trivial in Python but pretty darn hard, if not completely impractical, in Tcl:

1. We need to create unit-tests with mock implementations of each of the tools in the flow. The mock implementations should be tiny and instantaneous. The mocking should not modify or unecessarily complicate the implementation. Python has highly developed mocking libraries for this exact purpose.
2. We need a package management system that allows us to define and pull in the dependencies(a LEF parser? graphic toolkit for reporting?) and isolate the rest of the system from these dependencies. Python solves this using e.g. "tox" and virtualenv.
3. Graphical debugger in the IDE of the developers choice
4. Multiprocess, or distributed computing. Covered by various Python libraries.

Dejagnu(written in Expect, which uses Tcl, runs all the tests for GCC) for instance: it's written in Tcl, but to this date adding multithreading to it has defied the community.

[mithro]

Maybe you could use FuseSoC / EDALize?

[oharboe]

@mithro Definitely merits further study!

[tajayi]

Agreed. I'll bubble up this thread to "management" but EDALize may be a quick and clean way to replace the current Makefile. Will look into it.

[oharboe]

It's interesting to see that Python was the choice of FuseSoC and EDALize, not Tcl. I don't believe that FuseSoC and EDALize users are hardware engineers, but rather software engineers that are putting together an SoC.

I had a quick peek at FuseSoC. I believe it has roots in https://en.wikipedia.org/wiki/LEON the Sweedish/European Open SPARC. It talks about creating a System On Chip, which means configuring all the peripherals and memory map. Kinda like Altera Quartus Platform Designer, I forget what the Xilinx equivalent is called and then there's RISC-V Chisel Diplomacy.

The author is the same for FuseSoC and EDALize, but it looks like EDALize is a better match for taking that batoon from the alpha-release/flow/Makefile.

If I think about what the job of OpenROAD is and what FuseSoC and EDALize is trying to do, I'm skeptical that it's a great match. FuseSoC and EDALize is about creating an open source FPGA workflow alternative for what Altera Quartus Platform Designer is doing

To me OpenROAD should be agnostic about how the Verilog files come about. Creating the SoC Verilog files and HDL IP package management are separate concerns. OpenROAD is about taking Verilog files and creating GDSII files.

[mithro]

EDALize provides a wrapper around EDA tools (FPGA, ASIC, simulation, formal verification, etc) to provide a common top level interface.

Adding @olofk who can talk more about FuseSoC and EDALize.

[olofk]

Hi,

This one had slipped my mind, but I would just like to add some comments to address some confusion about FuseSoC and Edalize.

First of all, FuseSoC and Edalize are from the same author (me) and up until last year Edalize was a part of FuseSoC. FuseSoC still uses Edalize but I saw that it could be useful by itself. The roles of the two tools are that FuseSoC is the package manager and Edalize handles interfacing EDA tools. FuseSoC doesn't create any Verilog (or other RTL), but merely keeps tracks of dependencies between cores (e.g. a SoC might depend on an ethernet MAC which in turn depends on a FIFO implementation which in turn depends on a common utility library). FuseSoC doesn't care about how the RTL was created but keeps information about the cores in a tool-agnostic core description format, currently called CAPI2. This information consists of things like which source files the core has, any defines or parameters that it exposes, if it has any VPI/DPI modules that needs to be compiled and the dependencies on other cores. From a toplevel core, FuseSoC then builds up the dependency tree and collects the information needed to build (or simulate, lint, formally verify) the whole design. This information is then handed over to Edalize. Edalize's job is to create tool-specific project files for the tool that was requested to process the design. There are currently 16 tool flows supported for simulation, FPGA synthesis or linting. In the context of OpenRoad I think it would make a lot of sense to create a backend for the OpenRoad toolchain so that users can take existing designs and tell FuseSoC/Edalize to create the required project files needed to take the design through the rest of the chain.

So this is not a tool that automatically connects cores automatically like the graphical Xilinx/Intel tools mentioned above, but rather allows users to describe their cores in a tool-agnostic way and the relation between them, as well as abstracting away the differences between interfacing dozens of different EDA tools.

Also, it has nothing to do with Leon. I do happen to be from the same city as Leon comes from, but there aren't any deeper connections :)