How-To Update for Docs (building on #816)

docs/background/community.rst

@@ -36,7 +36,7 @@ want to contribute code, please `fork the code`_ and `submit a pull request`_.
 .. _announcement: https://groups.google.com/forum/#!topic/beeware-users/iUm-EDgypTg
 .. _BeeWare Users: https://groups.google.com/forum/#!forum/beeware-users
 .. _BeeWare Developers: https://groups.google.com/forum/#!forum/beeware-developers
-.. _log them on Github: https://github.com/beeware/batavia/issues
+.. _log them on GitHub: https://github.com/beeware/batavia/issues
 .. _fork the code: https://github.com/beeware/batavia
 .. _submit a pull request: https://github.com/beeware/batavia/pulls
 

docs/how-to/builtins.rst

@@ -1,104 +1,155 @@
 Implementing Python Built-ins in JavaScript
 ===========================================
 
+Python's builtins give Python its trademark magical feel. If you're new to Python,
+please read up on the builtins_.
+
+Most builtins have already been added to the project, but many are do not quite match the original
+implementation exactly. Some may not handle certain types of inputs correctly. In addition, new builtins_
+may arrive with the latest and greatest Python version. This guide should serve as your field manual for
+adding, updating, and navigating our implementations.
+
+Process
+-------
+
+The first thing to do when adding anything to Batavia is to play around a bit with it in a
+`Python REPL <https://www.python.org/shell/>`_.
+Here's an example using ``max()``::
+
+    >> max([1])
+    1
+    >> max((1, 2, 3, 4))
+    4
+    >> max(4)
+    Traceback (most recent call last):
+    File "<stdin>", line 1, in <module>
+    TypeError: 'int' object is not iterable
+
+Your goal is to find out how the function responds to various inputs and outputs.
+You may also want to consult the official documentation of the builtins_.
+Once you're a little familiar, you can start to add your implementation to Batavia.
+
 General Structure
------------------
+*****************
 
-JavaScript versions of Python built-in functions can be found inside the ``batavia/builtins``
-directory in the Batavia code. Each built-in is placed inside its own file.
+`JavaScript <https://www.ecma-international.org/publications/standards/Ecma-262.htm>`_
+versions of Python built-in functions can be found inside the ``batavia/builtins``
+directory in the Batavia code. Each built-in is placed inside its own file. These builtins are
+designed to be used only inside Batavia, as such they need to ensure they are being used in
+a compatible manner.
 
-.. code-block:: javascript
+Each builtin function will receive arguments and keyword arguments and needs to handle them,
+even if the result is throwing an error. Args should be an array, and kwargs should be a
+JavaScript `object <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object>`_.
+The first thing to do is check that both were passed in.
+
+Let's take a look at an example using the ``max()`` builtin.
 
-    // Example: a function that accepts exactly one argument, and no keyword arguments
+Note: ``max()`` already exists in ``batavia/builtins/max.js``, though
+if you can make any improvements or find any bugs to fix, they are welcome.
+
+.. code-block:: javascript
 
-    var <fn> = function(<args>, <kwargs>) {
-        // These builtins are designed to be used only inside Batavia, as such they need to ensure
-        // they are being used in a compatible manner.
+    // Max returns the largest item in an iterable or
+    // the largest of two or more arguments.
 
-        // Batavia will only ever pass two arguments, args and kwargs. If more or fewer arguments
-        // are passed, then Batavia is being used in an incompatible way.
-        // See: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/arguments
+    function max(args, kwargs) {
+        // Always add this code.
         if (arguments.length !== 2) {
             throw new builtins.BataviaError.$pyclass("Batavia calling convention not used.");
         }
 
-        // We are now checking if a kwargs object is passed. If it isn't kwargs will be null. Like
-        // obj.keys() in Python we can use Object.keys(obj) to get the keys of an object. If the
-        // function doesn't need support any kwargs we throw an error.
-        if (kwargs && Object.keys(kwargs).length > 0) {
-            throw new builtins.TypeError.$pyclass("<fn>() doesn't accept keyword arguments.");
-        }
+This code ensures that the function can handle keyword arguments.
+Next, we need to validate the arguments are correct. We can use JavaScript's
+`Object.keys() <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/keys>`_
+to get the keys of an object. If we can't accept certain
+args or kwargs, we will check the Python REPL to see what kind of error should be thrown and throw it.
 
-        // Now we can check if the function has the supported number of arguments. In this case a
-        // single required argument.
-        if (!args || args.length !== 1) {
-            throw new builtins.TypeError.$pyclass("<fn>() expected exactly 1 argument (" + args.length + " given)");
-        }
+.. tabs::
 
-        // If the function only works with a specific object type, add a test
-        var obj = args[0];
-        if (!types.isinstance(obj, types.<type>)) {
-            throw new builtins.TypeError.$pyclass(
-                "<fn>() expects a <type> (" + type_name(obj) + " given)");
-        }
+    .. group-tab:: Python REPL
 
-        // actual code goes here
-        Javascript.Function.Stuff();
-    }
+        .. code-block::
 
-    <fn>.__doc__ = 'docstring from Python 3.4 goes here, for documentation'
+            >> max(a=1)
+            TypeError: max expected 1 arguments, got 0
+            >> max(True)
+            TypeError: 'bool' object is not iterable
+            >> max([])
+            ValueError: max() arg is an empty sequence
 
-    modules.export = <fn>
+    .. group-tab:: Batavia Code
 
+        .. code-block:: javascript
 
-Adding Tests
-------------
+                if (!args || args.length === 0) {
+                    throw new exceptions.TypeError.$pyclass('max expected 1 arguments, got ' + args.length)
+                }
 
-The tests corresponding to Batavia implementations of built-ins are available inside
-``tests/builtins``. The Batavia test infrastructure includes a system to check the compatibility of
-JavaScript implementation of Python with the reference CPython implementation.
+                if (args.length > 1) {
+                    ...
+                } else {
+                    if (!args[0].__iter__) {
+                        throw new exceptions.TypeError.$pyclass("'" + type_name(args[0]) + "' object is not iterable")
+                    }
+                }
 
-It does this by running a test in the Python interpreter, and then running the same code using
-Batavia in the Node.js JavaScript interpreter. It will compare the output in both cases to see if
-they match. Furthermore the test suite will automatically test the builtin against values of all
-data types to check if it gets the same response across both implementations.
+                try {
+                    ...
+                } catch (err) {
+                    if (err instanceof exceptions.StopIteration.$pyclass) {
+                        throw new exceptions.ValueError.$pyclass('max() arg is an empty sequence')
+                    }
+                }
 
-In many cases these tests will not cover everything, so you can add your own. For an example look at
-the ``test_bool.py`` file in ``tests/builtins``. You will see two classes with test cases,
-``BoolTests`` and ``BuiltinBoolFunctionTests``. Both derive from ``TranspileTestCase`` which
-handles running your code in both interpreters and comparing outputs.
+        Useful functions are ``types.isinstance``, which checks for a match against a Batavia type or list,
+        of Batavia types, ``types.isbataviainstance``, which checks for a match against any Batavia instance,
+        ``Object.keys(kwargs)`` for dealing with kwargs, and JavaScript's ``for in``, ``for of``, and
+        ``Array.forEach`` loops for iterating over the JavaScript arrays and objects.
 
-Let's look at some test code that checks if a the Batavia implementation of ``bool`` can handle a
-bool-like class that implements ``__bool__``.
+        Note also the format for errors: ``throw new exceptions.<Error>.$pyclass``.
 
-.. code-block:: Python
+Returning a value
+*****************
 
-    def test_bool_like(self):
-        self.assertCodeExecution("""
-            class BoolLike:
-                def __init__(self, val):
-                    self.val = val
+Builtins_ implement Python functions and should return a Python object.
+Batavia implementations of all Python types are located in ``/batavia/types.js``.
+JavaScript imports use the ``require`` keyword and can be imported inline or at
+the top of the file. Inline imports can be preferable in some cases.
 
-                def __bool__(self):
-                    return self.val == 1
-            print(bool(BoolLike(0)))
-            print(bool(BoolLike(1)))
-            """)
+.. code-block:: javascript
 
-The ``assertCodeExecution`` method will run the code provided to it in both implementations. This
-code needs to generate some output so that the output can be compared, hence the need to print the
-values.
+    ...
 
+    Tuple = require('../types.js').Tuple
+    return new Tuple(my, results, here)
+    }
 
-Process
-----------
+Documentation
+*************
+
+Finally, add the `docstring <https://en.wikipedia.org/wiki/Docstring#Python>`_
+to the function object. In JavaScript, like in Python, functions
+are first-class objects and can have additional properties.
+
+.. code-block:: javascript
+
+    list.__doc__ = 'docstring from Python 3.x goes here, for documentation'
 
-For a given function, run `functionname.__doc__` in the Python 3.4 repl
+    module.exports = list
 
-Copy the docstring into the doc
+Tests
+*****
 
-Run the function in Python 3.4
+No implementation for a project like this is complete without tests. Check out the other sections for
+more details on test structure. Tests are located in ``/tests`` in a similar folder structure to the
+core code, and most test files have already been created. Some things that should almost always be
+tested:
 
-Take a guess at the implementation structure based on the other functions.
+* Write a test or three to ensure your function returns the correct output with some normal inputs.
+* Think of a few weird inputs that could throw off your code (or future code). Test them.
+* If you are throwing an error (excluding ``BataviaError``) anywhere, write a test that tries to throw it.
+* If you accounted for an edge case (look for an ``if`` statement), test it.
+* Check out the `official documentation <https://docs.python.org/3/>`_ for more edge cases.
 
-Copy the style of the other implemented functions
+.. _builtins: https://docs.python.org/3/library/functions.html

docs/how-to/contribute-code.rst

@@ -2,7 +2,7 @@ Contributing to Batavia's code
 ==============================
 
 In the following instructions, we're going to assume you’re familiar with
-Github and making pull requests. We're also going to assume some entry level
+GitHub and making pull requests. We're also going to assume some entry level
 Python and JavaScript; if anything we describe here doesn’t make sense, don’t
 worry - we're more than happy to fill in the gaps. At this point, we don’t know
 what you don’t know!

docs/how-to/contribute-docs.rst

@@ -32,5 +32,6 @@ Create the static files: ::
 
     $ make html
 
-Check for any errors and,if possible, fix them. The output of the file should
-be in the ``_build/html`` folder. Open the file you changed in the browser.
+Check for any errors and, if possible, fix them.
+The output of the file should be in the ``_build/html`` folder.
+Open the file you changed in the browser.

docs/how-to/contribute-tests.rst

@@ -0,0 +1,123 @@
+Implementing Tests in Batavia
+=============================
+
+Basic Test Structure
+--------------------
+
+Batavia's job is to run a browser-compatible Python compiler, which takes valid Python as input and runs it.
+Therefore, tests should test that the output of the Batavia compiler matches the output of CPython::
+
+    print('Hello')  # Code to test
+    Hello               # CPython output
+    Hello               # Batavia output
+    # Outputs match. Test passes!
+
+This test structure is simple and effective. It's used in almost every test we've written.
+
+Adding Tests
+------------
+
+In many cases, existing tests will not cover everything. Feel free to add your own!
+
+The tests corresponding to Batavia implementations of built-ins are available inside
+``tests/builtins``. The Batavia test infrastructure includes a system to check the compatibility of
+JavaScript implementation of Python with the reference CPython implementation.
+
+These tests all derive from ``TranspileTestCase``, which handles running your code in both interpreters
+and comparing outputs. For an example, look at the ``test_bool.py`` file in ``tests/builtins``. You
+will see two classes with test cases, ``BoolTests`` and ``BuiltinBoolFunctionTests``. Both derive
+from ``TranspileTestCase``.
+
+Let's look at some test code that checks if a the Batavia implementation of ``bool`` can handle a
+bool-like class that implements ``__bool__``.
+
+.. code-block:: Python
+
+    def test_bool_like(self):
+        self.assertCodeExecution("""
+            class BoolLike:
+                def __init__(self, val):
+                    self.val = val
+
+                def __bool__(self):
+                    return self.val == 1
+            print(bool(BoolLike(0)))
+            print(bool(BoolLike(1)))
+            """)
+
+The ``assertCodeExecution`` method will run the code provided to it in both implementations. This
+code needs to generate some output so that the output can be compared, hence the need to print the
+values. **Code that is not being printed is not being tested.**
+
+Finally, ``print()`` is an imperfect creature for tests. Some things in Python aren't guaranteed to
+print out in the same order every time, like sets dictionaries. Tests should be structured to compensate,
+for instance by converting to a sorted list. See also the output cleaners section below.
+
+Template
+--------
+
+.. code-block:: python
+
+    def test_<builtin>_<feature/case>(self):
+        # Valid Python code to be tested.
+        code = """
+            print('>>> print(<code>)')
+            print(<code>)
+        """
+        self.assertCodeExecution(code)
+
+This code block provides a printout of the code being run as well as the output of the code,
+which can be very useful for debugging in test cases where more than a few lines of code are being run.
+
+Testing for Errors
+------------------
+
+Since we're testing the compiler, we need to ensure that errors for all of the builtins are thrown correctly.
+We also want to ensure that we're not getting the wrong errors in our tests. Simply include a try/except
+block in your test.
+
+.. code-block:: python
+
+    def test_some_error(self):
+        code = """
+            try:
+                code_that_raises_a_ValueError()
+            except ValueError as err:
+                print(err)
+            print("Test complete!")
+        """
+        self.assertCodeExecution(code)
+
+Remember to catch the specific error you want, and then print the error. Then, print a success message to
+validate that your except block didn't crash as well. **Code that is not being printed is not being tested.**
+
+Output Cleaners
+---------------
+
+In some cases, the test output will vary. ``TranspileTestCase`` will automatically apply some common output
+cleanup for you. Some cases will need more or less cleanup. If you run your Python code directly in the REPL,
+and the output differs from the test case output, you may need to modify what cleanup steps are being run.
+
+As such, ``assertCodeExecution`` accepts optional ``js_cleaner`` and ``py_cleaner`` objects. These can be provided by
+the ``@transform`` decorator, located in ``tests/utils/output_cleaners.py``. Here's an example:
+
+.. code-block:: python
+
+    @transform(float_exp=False)
+    def test_some_floats(self, js_cleaner, py_cleaner):        # + Cleaner objects as arguments
+        code = ...
+        self.assertCodeExecution(code, js_cleaner=js_cleaner, py_cleaner=py_cleaner) # + Cleaner objects again
+
+This code means that the output of floating-point numbers will not be normalized using a regex. Refer to other
+test cases and the docstring for ``@transform`` for more examples.
+
+Node/Python Crashes
+-------------------
+
+If the CPython or JavaScript code crashes outright, UnitTest struggles. For instance,
+``confused END_FINALLY`` in the middle of your test output tends to mean that the JavaScript code threw an
+uncaught exception, causing Node to stop. It's hard for UnitTest to pull the details out of this type of thing
+since that error occurred in Node, not Python.
+
+These types of errors will often appear above the test case as a crash report instead of in the usual section for the
+output of your test's print() statements. Look there for clues.

docs/how-to/development-env.rst

@@ -4,15 +4,15 @@ Setting up your development environment
 The process of setting up a development environment is very similar to
 the :doc:`/tutorial/tutorial-0` process. The biggest difference is that
 instead of using the official BeeWare repository, you'll be using your own
-Github fork.
+GitHub fork.
 
 As with the getting started guide, these instructions will assume that you
 have Python 3 (currently supported versions are 3.5, 3.6, and 3.7).
 
 Batavia codebase
 ----------------
 
-Start by forking Batavia into your own Github repository; then
+Start by forking Batavia into your own GitHub repository; then
 check out your fork to your own computer into a development directory:
 
 .. code-block:: bash