From d12518f59988c2b1fd5a471390b0c980a538741c Mon Sep 17 00:00:00 2001 From: jhermann Date: Fri, 31 Jan 2020 21:50:38 +0100 Subject: [PATCH 1/6] docs/how-to: Shipping Dependency Sets as a Single File --- docs/how-tos.rst | 71 ++++++++++++++++++++++++++++++++++++++++++++++++ docs/index.rst | 1 + 2 files changed, 72 insertions(+) create mode 100644 docs/how-tos.rst diff --git a/docs/how-tos.rst b/docs/how-tos.rst new file mode 100644 index 0000000..dad649d --- /dev/null +++ b/docs/how-tos.rst @@ -0,0 +1,71 @@ +********************** +How-Tos, Tips & Tricks +********************** + +This chapter contains some practical examples of using ``shiv`` in the real world, +to give you a better idea how it can fit into your projects +and their deployment workflows. + + +Shipping Dependency Sets as a Single File +========================================= + +If you have a bunch of Python scripts that are all using the same set of base libraries, +you can distribute those dependencies in the form of a zipapp, +i.e. in a single executable file. +This way, you can have a project for the base libraries +and other projects for the scripts, +including scripts written by your end users. + +The following example uses ``shiv`` itself for demonstration purposes, +you'd use a setuptools project defining the needed dependencies +when applying this ‘for real’. + +So, to create your base library release artifact, call +``shiv`` like this in your project's workdir:: + + PROJECT=shiv + shiv -p '/usr/bin/python3.6 -IS' -o ~/bin/_lib-$PROJECT . + +Note that we do not provide an entry point here, which means this zipapp +drops into the given Python interpreter and is thus usable *as* an +interpeter, but with the dependencies of the project added. + +The ``-IS`` options ensure this zipapp runs isolated (for increased security), +with neither the current working directory +nor the host's site packages in the Python path. + +Now we can exploit this to write a script using the zipapp as its interpreter:: + + cat >script <<'.' + #! /usr/bin/env _lib-shiv + import sys + from pathlib import Path + import shiv + + print('Imported shiv from', + Path(shiv.__file__).parent, + '\nPython path:', + '\n'.join(sys.path), + sep='\n') + . + chmod +x script + ./script + +Calling the script produces the following output:: + + Imported shiv from + /home/user/.shiv/_lib-shiv_8a…e9/site-packages/shiv + + Python path: + /home/user/bin/_lib-shiv + /usr/lib/python36.zip + /usr/lib/python3.6 + /usr/lib/python3.6 + /usr/lib/python3.6/lib-dynload + /home/user/.shiv/_lib-shiv_8a…e9/site-packages + +The underscore prefix in the zipapp name indicates this is not a command +humans would normally use – alternatively you can deploy into e.g. +``/usr/local/lib/python3.6/`` and then use an absolute path instead of +an ``env`` call in the script's shebang. diff --git a/docs/index.rst b/docs/index.rst index 0635ae6..2279766 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -116,6 +116,7 @@ Table of Contents :maxdepth: 2 cli-reference + how-tos history api django From cedc716506cde11d2ea514e28c5db99b8ea5f9f3 Mon Sep 17 00:00:00 2001 From: jhermann Date: Sat, 1 Feb 2020 01:37:54 +0100 Subject: [PATCH 2/6] lib how-to: tip on using a literal list of packages --- docs/how-tos.rst | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/how-tos.rst b/docs/how-tos.rst index dad649d..45da943 100644 --- a/docs/how-tos.rst +++ b/docs/how-tos.rst @@ -31,10 +31,17 @@ Note that we do not provide an entry point here, which means this zipapp drops into the given Python interpreter and is thus usable *as* an interpeter, but with the dependencies of the project added. -The ``-IS`` options ensure this zipapp runs isolated (for increased security), +The ``-IS`` options in ``-p`` ensure this zipapp runs isolated (for increased security), with neither the current working directory nor the host's site packages in the Python path. +.. tip:: + + Replace the ``.`` at the end of the ``shiv`` call + by a literal list of PyPI packages, + or load them from a requirements file (``-r lib-requirements.txt``), + and you don't need an extra project for getting your base set defined. + Now we can exploit this to write a script using the zipapp as its interpreter:: cat >script <<'.' From 15fdc87a4c4cfdbd6d01dedc1bb018dda60d9c87 Mon Sep 17 00:00:00 2001 From: jhermann Date: Sat, 1 Feb 2020 10:08:38 +0100 Subject: [PATCH 3/6] how-to: Enabling Easy Zipapp Installs on Windows --- docs/how-tos.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/how-tos.rst b/docs/how-tos.rst index 45da943..e095f2b 100644 --- a/docs/how-tos.rst +++ b/docs/how-tos.rst @@ -76,3 +76,14 @@ The underscore prefix in the zipapp name indicates this is not a command humans would normally use – alternatively you can deploy into e.g. ``/usr/local/lib/python3.6/`` and then use an absolute path instead of an ``env`` call in the script's shebang. + + +External Resources +================== + +Here are some links to related articles on the web. + +* `Enabling Easy Zipapp Installs on Windows`_ – Describes the required steps to make zipapps really shine on Windows (i.e. make them work pretty much like on POSIX systems). + + +.. _`Enabling Easy Zipapp Installs on Windows`: https://py-generic-project.readthedocs.io/en/latest/installing.html#enabling-easy-zipapp-installs-on-windows From cd27a36e913ed4599d31e1b8dc1260720bf1935f Mon Sep 17 00:00:00 2001 From: jhermann Date: Sat, 1 Feb 2020 10:30:27 +0100 Subject: [PATCH 4/6] docs: shebang - taking advantage of 'env -S' --- docs/cli-reference.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/cli-reference.rst b/docs/cli-reference.rst index c80236a..3fca3f9 100644 --- a/docs/cli-reference.rst +++ b/docs/cli-reference.rst @@ -35,6 +35,18 @@ include the minor version (e.g. ``… python3.6``) – use what fits your circum On Windows, the Python launcher ``py`` knows how to handle shebangs using ``env``, so it's overall the best choice if you target multiple platforms with a pure Python zipapp. +If you have ``coreutils`` with at least version 8.30 on your Linux system, +then you can use env shebangs *and* pass argument to Python, like in this example: + +.. code-block:: shell + + shiv -p '/usr/bin/env -S python3 -I -S' … + +The magic ingredient is the new ``-S`` (split) option, but as of 2020 +you have to target very new stable releases like Debian Buster +or rolling distributions for it to actually work. +Check for the option by calling ``env --help``. + Also note that you can always fix the shebang during installation of a zipapp using this: .. code-block:: shell From 176d0a73f4bf4d2973ddadaa4e972978e90c8221 Mon Sep 17 00:00:00 2001 From: jhermann Date: Sat, 1 Feb 2020 10:35:32 +0100 Subject: [PATCH 5/6] docs: shebang - fix typo --- docs/cli-reference.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cli-reference.rst b/docs/cli-reference.rst index 3fca3f9..6742e1b 100644 --- a/docs/cli-reference.rst +++ b/docs/cli-reference.rst @@ -36,7 +36,7 @@ On Windows, the Python launcher ``py`` knows how to handle shebangs using ``env` so it's overall the best choice if you target multiple platforms with a pure Python zipapp. If you have ``coreutils`` with at least version 8.30 on your Linux system, -then you can use env shebangs *and* pass argument to Python, like in this example: +then you can use env shebangs *and* pass arguments to Python, like in this example: .. code-block:: shell From be7e2af1c431fc3e951e84463b0fcce722876b19 Mon Sep 17 00:00:00 2001 From: jhermann Date: Sun, 2 Feb 2020 09:36:25 +0100 Subject: [PATCH 6/6] how-tos: link to 'Python Alternative to Docker' post --- docs/how-tos.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/how-tos.rst b/docs/how-tos.rst index e095f2b..bd1ade1 100644 --- a/docs/how-tos.rst +++ b/docs/how-tos.rst @@ -83,7 +83,9 @@ External Resources Here are some links to related articles on the web. +* `Python Alternative to Docker`_ – Comparing shiv to Docker as the core tool of a deployment workflow, what are the differences between and advantages of both options. * `Enabling Easy Zipapp Installs on Windows`_ – Describes the required steps to make zipapps really shine on Windows (i.e. make them work pretty much like on POSIX systems). +.. _`Python alternative to Docker`: https://www.mattlayman.com/blog/2019/python-alternative-docker/ .. _`Enabling Easy Zipapp Installs on Windows`: https://py-generic-project.readthedocs.io/en/latest/installing.html#enabling-easy-zipapp-installs-on-windows