Testing addons locally during development

[frederickjh]

Hi!
In the past when I was working on an addon I would clone the docksal/addons repository in my workarea then in ~/.docksal/addons/ create a link to the folder of the addon I was working on and this worked to test the changes locally. However, this no longer seems to work and it appears that Docksal is either fetching the addon directly from the repository or from a cache.

I have also tested this with the addons that were still linked in ~/.docksal/addons/ and this no longer works with them either.

Any information on how to test Docksal addons locally would be appreciated as I currently have two feature branches ready to test before creating pull requests.

Thanks!
Frederick

[frederickjh]

I just checked and the documentation on addons also says that this should work.

Fin will first look for the addon in the ~/.docksal/addons directory. If it is not located there, then it will attempt to search the Docksal Addons Repository.

[achekulaev]

It's not something Docksal related. It's something related to how you create symlinks or OS specifics.

[frederickjh]

I also copied the files for the pma addon into the ~/.docksal/addons/pma/ folder and that didn't work either, so I am not sure how this is symlink or OS specific. All I know is that it use to work and now it does not with the current version of Docksal.

[achekulaev]

You realize it is working for me with the current version of Docksal right now, right?
So I guess it should be given, that tool developers would need something more than "it's not working" to even start looking into the issue.

Like start with the usual fin config and fin sysinfo to begin with.
Then logically would be to demonstrate that you have addon in place as you claim by showing output of ls -la ~/.docksal/addons immediately followed by fin pma output.
You're programmer, you know the drill.

[achekulaev]

Alternatively you could debug the addon_get_path function yourself to see what might be the culprit :)

[frederickjh]

Further testing with addon in the global docksal addons directory:

$ fin addon install adminer
Downloading addon hook files...
  adminer.pre-install
  adminer.post-install
  adminer.pre-uninstall
Running pre-install hook...

It just downloads the addon and installs it.

With the addon in the project addons directory:

$ fin addon install adminer
Addon already exists in /home/username/docker/testing/pmaaddon/.docksal/addons/adminer
Downloading addon hook files...
  adminer.pre-install
  adminer.post-install
  adminer.pre-uninstall
Running pre-install hook...
Downloading addon main script
  adminer/adminer
Downloading other addon files...
  conf/adminer.yml
  conf/designs.php
Running post-install hook...
  Enabling adminer...
Stopping services...

Wait, fin did see it but to add injury to insult, it downloaded the addon over top of my work. Good thing I committed to git first so I can just check the files back out.

[frederickjh]

So to test:

• linking the addon (adminer) into the project's .docksal/addon/ directory
• fin addon install adminer
• In the git repository for the addons I am working on run:
  • git checkout -- . To revert reversions caused by fin's downloading the addon over the repository.
• fin reset adminer

[achekulaev]

This is the way you re-install or update the addon. fin addon install it again.
The fact it exists already will not stop it because this is how update works.
The fact that global addon with the same name exists will not stop it too.

Not an issue.

[frederickjh]

Currently the addons are linked into the ~/.docksal/addons global addons directory. I have tried copying their folder into it and this also does not work.

CLICK ME to see `ls -la ~/.docksal/addons` output

´´´ $ ls -la ~/.docksal/addons total 12 drwxrwxr-x 3 username username 4096 Mar 11 11:59 ./ drwxrwxr-x 20 username username 4096 Mar 11 12:51 ../ lrwxrwxrwx 1 username username 48 May 8 2018 adminer -> /home/username/workarea/docksal-addons/adminer/ drwxrwxr-x 3 username username 4096 Mar 11 11:59 DISABLED/ lrwxrwxrwx 1 username username 44 Mar 11 10:46 pma -> /home/username/workarea/docksal-addons/pma/ lrwxrwxrwx 1 username username 44 Sep 26 2018 uli -> /home/username/workarea/docksal-addons/uli/ ´´´

CLICK ME to see `fin config` output

$ fin config
---------------------
COMPOSE_PROJECT_NAME_SAFE: pmaaddon
COMPOSE_FILE:
/home/username/.docksal/stacks/volumes-bind.yml
/home/username/.docksal/stacks/stack-default.yml
/home/username/docker/testing/pmaaddon/.docksal/docksal.yml
/home/username/docker/testing/pmaaddon/.docksal/docksal-local.yml
ENV_FILE:
/home/username/docker/testing/pmaaddon/.docksal/docksal.env
/home/username/docker/testing/pmaaddon/.docksal/docksal-local.env

PROJECT_ROOT: /home/username/docker/testing/pmaaddon
DOCROOT: docroot
VIRTUAL_HOST: pmaaddon.dev.example.com
VIRTUAL_HOST_ALIASES: *.pmaaddon.dev.example.com
IP: 192.168.64.100
MYSQL: 192.168.64.100:32778

Docker Compose configuration
---------------------
services:
  cli:
    build:
      args:
        cli_image: docksal/cli:edge-php7.1
      context: /home/username/docker/testing/pmaaddon/.docksal/services/cli
    dns:
    - 192.168.64.100
    - 8.8.8.8
    environment:
      BLACKFIRE_CLIENT_ID: null
      BLACKFIRE_CLIENT_TOKEN: null
      COLUMNS: '159'
      COMPOSER_ALLOW_XDEBUG: '0'
      COMPOSER_DISABLE_XDEBUG_WARN: '0'
      DOCROOT: docroot
      DRUSH_ALLOW_XDEBUG: '0'
      DRUSH_OPTIONS_URI: pmaaddon.dev.example.com
      EDITOR: /bin/nano
      GIT_USER_EMAIL: user@example.com
      GIT_USER_NAME: Me
      HOST_GID: '1000'
      HOST_UID: '1000'
      LANG: en_US.UTF-8
      LANGUAGE: en_US.UTF-8
      LC_ALL: C
      LC_CTYPE: C
      LINES: '42'
      MYSQL_DATABASE: default
      MYSQL_HOST: db
      MYSQL_PASSWORD: mysqlpassword
      MYSQL_ROOT_PASSWORD: rootpassword
      MYSQL_USER: user
      PHP_IDE_CONFIG: null
      SECRET_ACAPI_EMAIL: null
      SECRET_ACAPI_KEY: null
      SECRET_PLATFORMSH_CLI_TOKEN: null
      SECRET_SSH_PRIVATE_KEY: null
      SECRET_TERMINUS_TOKEN: null
      SSH_AUTH_SOCK: /tmp/ssh-AoqBxdXcm0/agent.10264
      VIRTUAL_HOST: pmaaddon.dev.example.com
      XDEBUG_CONFIG: remote_connect_back=0 remote_host=192.168.64.1
      XDEBUG_ENABLED: '0'
    hostname: cli
    image: pmaaddon_cli
    labels:
      io.docksal.shell: bash
      io.docksal.user: docker
    volumes:
    - cli_home:/home/docker:rw
    - /home/username/.composer/auth.json:/home/docker/.composer/auth.json:rw
    - /home/username/.docksal/fishshellconfig:/home/docker/.config/fish:rw
    - /home/username/.docksal/fishshellconfiglink:/home/docker/.config/fishconfiglink:rw
    - /home/username/.docksal/fishermanconfig:/home/docker/.config/fisherman:rw
    - /home/username/.gitconfig:/home/docker/.gitconfig:rw
    - /home/username/.gitignore_global:/home/docker/.gitignore_global:rw
    - /home/username/.ssh/config:/home/docker/.ssh/config:rw
    - /home/username/.ssh/id_rsa-blah.pub:/home/docker/.ssh/id_rsa-bitbucket.pub:rw
    - /home/username/.ssh/id_rsa-blah-blah.pub:/home/docker/.ssh/id_rsa-bossard-hostpoint.pub:rw
    - /home/username/.ssh/id_rsa-blah-blah-blah.pub:/home/docker/.ssh/id_rsa-github.pub:rw
    - /home/username/.ssh/id_rsa-blah-blah-blah-blah.pub:/home/docker/.ssh/id_rsa-gitlab.pub:rw
    - /home/username/.ssh/id_rsa-more-blah.pub:/home/docker/.ssh/id_rsa-keimdigital-hostpoint.pub:rw
    - /home/username/.ssh/id_rsa-blah-too.pub:/home/docker/.ssh/id_rsa.pub:rw
    - /tmp/ssh-AoqBxdXcm0/agent.10264:/tmp/ssh-AoqBxdXcm0/agent.10264:ro
    - project_root:/var/www:rw,nocopy,cached
  db:
    dns:
    - 192.168.64.100
    - 8.8.8.8
    environment:
      MYSQL_ALLOW_EMPTY_PASSWORD: null
      MYSQL_DATABASE: default
      MYSQL_INITDB_SKIP_TZINFO: null
      MYSQL_ONETIME_PASSWORD: null
      MYSQL_PASSWORD: password
      MYSQL_RANDOM_ROOT_PASSWORD: null
      MYSQL_ROOT_PASSWORD: password
      MYSQL_USER: user
    hostname: db
    image: docksal/mysql:5.6-1.4
    ports:
    - 3306/tcp
    volumes:
    - /home/username/docker/testing/pmaaddon/db/import:/docker-entrypoint-initdb.d:ro
    - db_data:/var/lib/mysql:rw
    - project_root:/var/www:ro,nocopy,cached
  pma:
    environment:
      PMA_HOST: db
      PMA_PASSWORD: password
      PMA_USER: root
    hostname: pma
    image: phpmyadmin/phpmyadmin
    labels:
      io.docksal.virtual-host: pma-pmaaddon.dev.example.com
  web:
    depends_on:
      cli:
        condition: service_started
    dns:
    - 192.168.64.100
    - 8.8.8.8
    environment:
      APACHE_BASIC_AUTH_PASS: null
      APACHE_BASIC_AUTH_USER: null
      APACHE_DOCUMENTROOT: /var/www/docroot
      APACHE_FCGI_HOST_PORT: cli:9000
    hostname: web
    image: docksal/apache:2.4-2.3
    labels:
      io.docksal.cert-name: none
      io.docksal.permanent: "false"
      io.docksal.project-root: /home/username/docker/testing/pmaaddon
      io.docksal.virtual-host: pmaaddon.dev.example.com,*.pmaaddon.dev.example.com,pmaaddon.dev.example.com.*
    volumes:
    - project_root:/var/www:ro,nocopy,cached
version: '2.1'
volumes:
  cli_home: {}
  db_data: {}
  docksal_ssh_agent:
    external: true
    name: docksal_ssh_agent
  project_root:
    driver: local
    driver_opts:
      device: /home/username/docker/testing/pmaaddon
      o: bind
      type: none

---------------------

CLICK ME to see `fin sysinfo` output

$ fin sysinfo
███  OS
Linux Ubuntu 18.04
Linux hostname 4.15.0-88-generic #88-Ubuntu SMP Tue Feb 11 20:11:34 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

███  ENVIRONMENT
MODE : Linux Kernel
DOCKER_HOST : 

███  FIN
fin version: 1.93.0

███  DOCKER COMPOSE
EXPECTED VERSION: 1.24.1
docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018

███  DOCKER
EXPECTED VERSION: 19.03.4

Client: Docker Engine - Community
 Version:           19.03.6
 API version:       1.40
 Go version:        go1.12.16
 Git commit:        369ce74a3c
 Built:             Thu Feb 13 01:27:49 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.5
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.12
  Git commit:       633a0ea838
  Built:            Wed Nov 13 07:28:22 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.12
  GitCommit:        35bd7a5f69c13e1563af8a93431411cd9ecf5021
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

███  DOCKSAL: PROJECTS
project             STATUS                    virtual host                                                                                 project root
pmaaddon            Up 32 minutes (healthy)   pmaaddon.dev.example.com,*.pmaaddon.dev.example.com,pmaaddon.dev.example.com.*   /home/username/docker/testing/pmaaddon
docksal             Up 2 hours                docksal.dev.example.com,*.docksal.dev.example.com                                    /home/username/docker/docksal

███  DOCKSAL: VIRTUAL HOSTS
*.docksal.dev.example.com
*.pmaaddon.dev.example.com
docksal.dev.example.com
pma-pmaaddon.dev.example.com
pmaaddon.dev.example.com.*
pmaaddon.dev.example.com

███  DOCKSAL: DNS
curl: (6) Could not resolve host: dns-test.docksal
ERROR: Requesting http://dns-test.docksal failed!

███  DOCKER: RUNNING CONTAINERS
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS                    PORTS                                      NAMES
3b043b3fc115        phpmyadmin/phpmyadmin       "/docker-entrypoint.…"   26 minutes ago      Up 26 minutes             80/tcp                                     pmaaddon_pma_1
b71bf5a145ea        docksal/mysql:5.6-1.4       "docker-entrypoint.s…"   37 minutes ago      Up 32 minutes (healthy)   0.0.0.0:32778->3306/tcp                    pmaaddon_db_1
f0c8411a53ec        docksal/ssh-agent:1.3       "docker-entrypoint.s…"   4 hours ago         Up 4 hours (healthy)                                                 docksal-ssh-agent
8b7359cf2e36        docksal/dns:1.1             "docker-entrypoint.s…"   4 hours ago         Up 4 hours (healthy)      192.168.64.100:53->53/udp                  docksal-dns
05e4faa7cdc0        docksal/vhost-proxy:1.5     "docker-entrypoint.s…"   4 hours ago         Up 4 hours (healthy)      0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   docksal-vhost-proxy
2ba2cb6b75f8        docksal/apache:2.4-2.3      "httpd-foreground"       4 hours ago         Up 32 minutes (healthy)   80/tcp, 443/tcp                            pmaaddon_web_1
4420a42706be        pmaaddon_cli                "/opt/startup.sh sup…"   4 hours ago         Up 26 minutes (healthy)   22/tcp, 3000/tcp, 9000/tcp                 pmaaddon_cli_1
1bf0fc288657        docksal/web:2.1-apache2.4   "httpd-foreground"       2 years ago         Up 2 hours                80/tcp, 443/tcp                            docksal_web_1
7308a6a43b53        docksal/cli:1.3-php7        "/opt/startup.sh sup…"   2 years ago         Up 2 hours (healthy)      22/tcp, 9000/tcp                           docksal_cli_1
639c488e5902        docksal/db:1.1-mysql-5.6    "/entrypoint.sh mysq…"   2 years ago         Up 2 hours                0.0.0.0:32769->3306/tcp                    docksal_db_1

███  DOCKER: NETWORKS
NETWORK ID          NAME                                    DRIVER              SCOPE
b8dc13ea3369        _default                                bridge              local
d146e0c8107c        bridge                                  bridge              local
99a4812a805f        docksal_default                         bridge              local
1d72a0ffa8ab        grav_default                            bridge              local
fb3576c2d9bb        host                                    host                local
d18908688e1b        none                                    null                local
19b7fb348e58        pmaaddon_default                        bridge              local
a6f45e6a2b50        testing_default                         bridge              local

███  HDD Usage
Filesystem                          Size  Used Avail Use% Mounted on
udev                                3.9G     0  3.9G   0% /dev
tmpfs                               797M  2.0M  795M   1% /run
/dev/mapper/blah--vg-root  126G   81G   40G  68% /
tmpfs                               3.9G     0  3.9G   0% /dev/shm
tmpfs                               5.0M     0  5.0M   0% /run/lock
tmpfs                               3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/loop0                          409M  409M     0 100% /snap/netbeans/18
/dev/loop1                           92M   92M     0 100% /snap/core/8592
/dev/loop2                          438M  438M     0 100% /snap/netbeans/24
/dev/loop3                           92M   92M     0 100% /snap/core/8689
/dev/sda1                           472M  281M  167M  63% /boot
tmpfs                               797M     0  797M   0% /run/user/1000

[frederickjh]

This is the way you re-install or update the addon. fin addon install it again.
The fact it exists already will not stop it because this is how update works.
The fact that global addon with the same name exists will not stop it too.

Not an issue.

Sorry, @achekulaev then is what is the proper way to test an addon that you are doing development work on?

[achekulaev]

The way you do it. You keep writing "it does not work" but haven't posted any proof how it does not work.

Show that fin uli does not work

Or I don't understand the issue

[frederickjh]

Maybe I have the order of preference of locations to install an addon from wrong in my head, as I have never see this in the documentation. I always thought fin would look first to see if the addon to install was in the project's addon directory then the Docksal global addon directory before then going and downloading the addon from github.

But if I understand you correctly, if the addon being installed exists on github in the docksal/addons repository then it will download it, regardless if an addon with the same name exists in a project and will overwrite it.

[achekulaev]

Yes, fin addon install will always (re-)download and install addon in your project directory, fin addon install --global will always (re-)download and install addon in your home directory.

[achekulaev]

There was a confirmation to overwrite... Which was commented out... Not sure why... But it was like this for 3 years