Cannot run db migration with the uberjar

[dawranliou]

The current uberjar alias runs the juxt/pack.alpha capsule with the -m option.

According to the doc:

If main is not specified, it will default to clojure.main, meaning it can take options like -m or -r.

What is implied here is that we cannot switch the main namespace once the uberjar is built. For example, we cannot do java -jar <uberjar>.jar -m coast.db create nor java -jar <uberjar>.jar -m coast.migrations migrate because all the args will be passed into server, instead of clojure.main.

[dawranliou]

I propose the following:

• Remove the "-m" "server" option.
• Update README. To run the server with uberjar, use java -jar <uberjar>.jar -m server 1337.
  • (This is what it was before my PR Update shipping instruction #4 😛)

However, this fix also exposes another issue that the the migration files are not located under db/migrations anymore in the uberjar.

[swlkr]

Haha! I accidentally did the right thing!

[swlkr]

For the db/migrations in the uberjar, we could add "db" to the :paths key in deps.edn 🤔

[dawranliou]

I was thinking al;ong the same line when I first saw the problem. After some digging, the problem is that all the paths are joined into one in the uberjar. For example:

The source structure:

db/
  migrations/
    a.clj
    b.clj
resources/
  assets.edn
src/
  server.clj

Becomes this in the uberjar. Note the db directory is no longer there.

migrations/
  a.clj
  b.clj
assets.edn
server.clj

I think we can solve this by:

1. Copy the db/ a build/ directory before the uberjar. Include build/ as an extra-path to build the uberjar.
2. In the coast.migrations/migration-dir, try to load the db/migrations from the resource before loading it on drive.

I'm working on the PRs to make it clearer.

[dawranliou]

The following commits

1. coast-framework/template - f0450ed
2. coast-framework/coast -
   00a85e2e813b5b3141abf3b09329e15db2200c3d

I'm worried that this creates more complexity because the db/migrations folder can be anywhere as long as it is in the path. @swlkr can you share how db migration is usually done in production? If there are better ways, perhaps we don't want to change the behavior. In this case, I will revert #6 to only do what I originally proposed and close coast-framework/coast#82.

I propose the following:
* Remove the "-m" "server" option.
* Update README. To run the server with uberjar, use java -jar <uberjar>.jar -m server 1337.

[swlkr]

I haven't documented this anywhere, I'm still figuring out the best way to deploy small-ish clojure apps to cheap VPS's, which… is not really a design goal of clojure the language but yeah.

So I do two things

1. I use badigeon to build a folder with what I'm assuming is the bytecode, I think this is AOT compiling, not quite sure.

; alias in deps.edn
:build {:extra-paths ["build"]
          :extra-deps {badigeon/badigeon {:git/url "https://github.com/EwenG/badigeon.git"
                                          :sha "dca97f9680a6ea204a2504c4414cafc4ba182a83"
                                          :tag "0.0.6"}}}

; build/package.clj
(ns package
  (:require [badigeon.bundle :refer [bundle make-out-path]]
            [badigeon.compile :as c]))

(defn -main []
  (bundle (make-out-path 'app nil) {:libs-path "jars"})
  (c/compile '[server] {:compile-path "target/classes"
                        :compiler-options {:disable-locals-clearing false
                                           :elide-meta [:doc :file :line :added]
                                           :direct-linking true}}))

The next thing I do is create a post-receive hook on a server somewhere, could be anything, even a separate build server and just git push to it

#!/bin/bash

name=todayinclojure
home_dir="/home/sean"
git_dir="$home_dir/$name/src"
deploy_to="$home_dir/$name"
releases="$home_dir/$name/releases"
tmp_dir="$home_dir/$name/tmp"
target_dir="$home_dir/$name/tmp/target"
timestamp=$(date "+%Y%m%d%H%M%S")

while read oldrev newrev ref
do
    if [[ $ref =~ .*/master$ ]];
    then
      echo "Deploying master branch to $deploy_to and starting service $name"

      # make sure required directories exist
      mkdir -p $git_dir $releases $tmp_dir $target_dir

      # copy latest version of master branch
      git --work-tree=$tmp_dir --git-dir=$git_dir checkout -f
      cd $tmp_dir

      # run migrations and build
      COAST_ENV=prod DB_NAME=$deploy_to/$name.sqlite3 make db/migrate
      COAST_ENV=prod make build

      # save older version of class files
      mkdir -p $releases/$timestamp
      cp -R target $releases/$timestamp/

      # copy the target out of tmp
      cp -R target $deploy_to

      # restart web server
      supervisorctl restart $name
    else
      echo "Ref $ref successfully received. Doing nothing: only the master branch will be deployed."
    fi
done

That bash script is actually todayinclojure.com's post-receive git hook 😅

The Makefile ties everything together:

assets:
	clj -m coast.assets

build: assets
	clj -Abuild -m package

make build is the magical command that gets run in that git hook after the migrations.

So a deploy is just git push ssh://sean@45.76.62.35/~/todayinclojure/src

And since I push the code and not a binary, I don't have to worry about migrations being as a resource or anything, I just copy the folder over and run migrations like I do when developing locally.

Also, please don't hack my server, it uses ssh pub/private keys and everything but I'm pretty sure it's vulnerable to something somehow haha

[dawranliou]

LOL, you don't have to worry me hacking you, but perhaps you'd want to obfuscate the IP.
This is great! Thank you for sharing the script. I'm totally going to try the same.

I just reverted the #6 and closed coast-framework/coast#82, since there are other way to do db migration.