Skip to content

MetaProgramming

Jump to bottom Edit New page
Li, Xizhi edited this page Dec 25, 2016 · 10 revisions

NPL语言Meta编译器

  • Extend NPL syntax with a LISP like AST structure.
  • Compile *.npl source code into standard lua-compatible code (NOT byte code to support JIT) using AST (Compiler code needs to be written in NPL, please refer to metalua project)
  • Add new NPL syntax to make NPL CAD language more expressive as a DSL (domain specific language)

Our goal is to support syntax like below in NPL cad.

translate(x,y,z){
     for i=1, 4 do
          rotate(i*45){  cube() }
     end
}

The above *.npl code will be compiled to following lua-compatible code via meta programming

push()
   translate(x,y,z)
      for i=1, 4 do
         push()
           rotate(i*45);
           cube()
         pop()
      end
pop()

Introduce a New LISP-like NPL AST syntax (called Func-expression)

symbol ( expressions ) {  
     anything here is subject to the macro of symbol
}

Func-expression is very similar to LISP S-expression, except that it does not break the original lua syntax very much. Yet it allows the same functional programming like in LISP. expressions in above code is just regular lua expression, so Func-expression has two part of inputs, one is regular expressions in brackets (), the others are anything else inside curly brackets {}. At dynamic compile time, code inside {} can be translated in arbitrary ways according to macro definition code usually via simple code replacement or AST parsing.

If no macro is defined for symbol, then above code is equivalent to calling a function of name symbol with expressions. i.e.

symbol ( expressions ){
     statements
}

If a macro is defined for symbol, then above code may compile to a different AST according to the macro. We can dynamically define a new macro in *.npl using the def symbol.

def (symbol_names, expressions){
     -- generate template code here, we can support metalua syntax here ...
}

NPL CAD utilizes above N-expression to create translate|rotate|... syntax with curly brackets, such as

def ({"translate, rotate"}, ... ) {
    push();
        _G[ -{ symbol_name} ](...);
        -{nplp.emit(nplp.ast)}
    pop();
}
def ("scale", p1 ) {
    push();
        scale(p1);
        -{nplp.emit(nplp.getsource())}
    pop();
}

Additional requirement:

  • compiled source code should try to match source code line by line to make debugging possible

Implementation Details

Suppose we have following function expression definition.

def("scale", p1, p2) {
  push()
    scale( +{=params("p1")}, p3)
    +{ emit(); -- emit all code }
  pop()
}

It will translate to following code after compilation using AST.

do
 local f = nplp.FuncExpression:new():init("scale");
 nplp.register(f);

-- generate compiled code
f.CompileCode = function(input)
 local compiledCode = {};
 local f_scope = {
   emit = function(code)  
     if(code) then
       compiledCode[#compiledCode+1] = code;
     else
       compiledCode[#compiledCode+1] = input:GetCodeAsString();
     end
   end,
   -- helper function to get input params
   params = function(name)  return input:GetParams(name)  end, 
   p1 = input:GetParams("p1"),
   p2 = input:GetParams("p2"),
 };

 local function compile(input)
   emit("push()\n")
   emit("scale( ")
   emit(params("p1"))
   emit("p3)\n")
   emit(); -- emit all code
   emit("pop()\n")
 end

 setfenv(compile, f_scope);    
 compile();
 return compiledCode;
end

end
-- base class compile method
function FuncExpression:Compile(ast)
   local lines = self:CompileCode(ast);
   local sourceCode = table.concat(lines);
   if(sourceCode) then
      local final_ast = self:LoadAstFromString(sourceCode, self.filename, self.nLineOffset)
      --recursively define if there is any custom functions in the ast that needs translation. 
      for any custom function in final_ast do
           call inner custom function's compile recursively. 
      end
      return final_ast;
   end
end

-- virtual function: just return as it is. 
function FuncExpression:CompileCode(ast)
   return {ast:tostring()};
end

-- def function implementation
function FuncExpressionDef:CompileCode(ast)
   local mode = self:GetModeFromAst(ast);
   local f = nplp.FuncExpression:new():init(ast:getparam(1));
   f:setmode(mode); -- mode can be strict, line, tokens, etc. 
   nplp.register(f);
   
   local compiledCode = {};
 
 local function compile()
    for traverse ast do 
       if ast is not +{}, 
         generate emit (ast)
       else
         genterate ast to string to compiledCode
       end
    end
 end

 setfenv(compile, f_scope);    
 compiledCode[#compiledCode+1] = {[[return function(ast) 
local compiledCode = {};
 local f_scope = {
   emit = function(code)  
     if(code) then
       compiledCode[#compiledCode+1] = code;
     else
       compiledCode[#compiledCode+1] = input:GetCodeAsString();
     end
   end,
   -- helper function to get input params
   params = function(name)  return input:GetParams(name)  end, 
   p1 = input:GetParams("p1"),
   p2 = input:GetParams("p2"),
 };

 local function compile()
   

]]}
 compile();

 compiledCode[#compiledCode+1] = {[[ 
end

 setfenv(compile, f_scope);    
 compile();
 return compiledCode;
end ]]}
 f.CompileCode = pcall(loadstring(compiledCode))
 return {nil};
end

features

  • [dropped]namespaces: A.B.C(){} 
def("A.B.C")
  • test case with lua syntax: such as A({}){ {} }
  • generate and print errors: def("aaa"){ }
-- [string "?"]:1: ')' expected near '<eof>' <Syntax error>
log("<syntax error>[string "echo('hello'"]:1: ')' expected near '<eof>'")
  • test line numbers
assert(debug.getinfo(1, "nSl").currentline == 1)
  • invoke custom functions without () like A.B.C{}
  • replace code func "" with func("")
  • add ast manipulation functions in side +{}
    • emit(code, linenumber):
      • @param linenumber: if nil, insert at current line. if -1, append to front. if [1,oo], try to emit at given line.
def("linenumber"){
local a=1;
a=a+1;
+{emit("a=a+1")}
+{emit("a=a+2", 4)}
+{emit(nil, 0)}
}

linenumber{ -- assert(line==1)
  a=a+8
  a=a+9 -- assert(line==3)
  a=a+10
}


local a=1;a=a+1;a=a+1;\n
a=a+8\n
a=a+9\n
a=a+2;a=a+10\n
  • helper functions on ast node that can be used inside +{}
def("circle"){
-- mode: unstrict
+{
local lines = ast:tolines();
for i=1, #lines do
   local line = lines[i];
   local radius = line:match("radius[%D]*([%d%.]+)")
   if(radius) then
      ast:replaceLine(i, format("drawingapi.drawcircle(%d)", radius));
   end
end
}
}

circle {  I want a circle with radius 2.0 }
circle {  I want a circle with radius 3.0 }


def("PlusEqual"){
   +{
      nplp.addstatement("+","=", ...})
      emit() 
   }
}

def("CodeSlice"){
   +{ for i=1, 10 do }  -- 
      circle(i);        -- emit("circle(i), 2)")
   +{ end }             --
}

-- support (...)

References:

Music board is very interesting tutorial for 3d printing http://www.thingiverse.com/thing:53235

NPL CAD is written in NPL language and distributed as a plugin in Paracraft.

Download paracraft | NPL Language

Clone this wiki locally