template_engine.txt

    TEMPLATE ENGINE

  "Template engines allows us to parse and generate data
   from logic embedded in text"
                       ~ Baz

  "The job of a template engine is to interpret the
   template, replacing the dynamic pieces with real
   data"

Programming languages are used to solve problems
involving a lot of logic and little literal text
(textual data) while template engines are suited for
text-heavy problems. They usually involve a little
bit of logic, and a great deal of textual data.

  Why do we need template engine?

In web application, we generate HTML pages to be served
(by the web server - how do we create a web server?) to
the browser. These HTML pages are a combination of large
pieces of textual data and dynamic data.

We need a medium to generate the string
containing the mix of the two (static text and dynamic
data).

One way this can be done is to have the static text as
string constants in the application code, and then
introduce the dynamic data by using string substitution.
Check **toy.py**.
Some of the problems this poses are:

    - Frontend and Backend code mixed together so the
      frontend has to edit backend code to make HTML
      changes

    - It would blow-up if multiple pages were involved.

    - It makes the logic hard to see. Increasing cognitive
      load.

The better way to produce HTML pages is with
templates. The file is mostly static HTML with the
dynamic pieces embedded using special notation.
Check **TOY.html**

Comparing **toy.py** and **TOY.html**, the former is
mostly Python code, with HTML embedded in it while the
latter is logic embedded in HTML.

String formatting functions are examples of
mini-languages that revolves around string literal
text and little dynamic part.

    "foo = {foo}".format(foo=7)
               |
            foo = 7

Templates extends the idea of string formatting to
include constructs like conditionals and loops.

To use (HTML) templates in our programs, we need
TEMPLATE ENGINE: a sytem that inputs dynamic data into
static text describing the structure of the page


    IMPLEMENTATION APPROACH

- parsing the template and
- rendering the template
      - managing the dynamic context (source of data)
      - executing the logic elements
      - implementing dot access and filter execution


MODELS OF IMPLEMENTATION
    parsing phase ----------> rendering phase

Interpretation model
    parsing: involves creating a data structure which
             represents the structure of the template
    rendering: walking down the data structure and
               assembling the result text based on the
               instructions found.
    prefereable to use if the templates will be used a
    few times each.
    example of real-world example: Django template engine.

Compilation model
    parsing: generates directly executable code
    renderinng: executing the code produces result.
    example of real-world example: Jinja2 and Mako.

******************************************************
** In this template engine, compilation is used:    **
** templates are compiled to Python code which      **
** produce the result when run.                     **
**                                                  **
** This is an example of technique called:          **
** CODE GENERATION                                  **
******************************************************


    COMPILING TO PYTHON

The parsing phase will convert a template into a Python
function. Each template is converted into a
**render_function**.
   The template in *TOY.html* get converted to the function
   in **render_toy.html**


    WRITING THE ENGINE

The heart of the engine is the Template class, and the
bulk of the work in the engine is parsing the template and
producing the necessary Python code. A helper class,
CodeBuilder, handles the management as the Python code is
being constructed.

CodeBuilder keeps a list of strings that will be the final
Python code, and embedded within it are methods to run the
code. Some of what it does are:
    - keeping track of each line of code that forms the
      desired Python code. A line of code could be
      another CodeBuilder object.
    - managing indentation
The gem lies in the fact that it is oblivious of what is
contained in the Template class. It works without knowing
any implementation detail of the proposed "template".
This shows a clear seperation of concern.

The Template class uses the CodeBuilder's methods
to sequentially generate the final Python code from the
input template.

Early on, a closure function that buffers output string (
strings that should be contained in the final Python code)
is defined. It, the closure function, is used to perform a
micro-optimization by keeping track of a buffer list,
and based on the length of the list, it either
makes a list append or extend call in the compiled Python code.

The parsing begins by splitting the template text into
a number of tokens using regular expression. The text is split
along the tags: comment, variable and control. The tokens are
then looped over and each piece of token goes through its
specific process.

Parsing of control structures is to make sure the nesting
is done properly by pushing to a stack when a control
statement is encountered and poping from it when the end of
the control statement is reached, taking care of the
encountered variables along the way.