Docker Container Platforms

[kanadork]

Hi all. I am a beginner with configuring Docker containers. So far my experience is mostly with installing them and making minor tweaks using Portainer on my home NAS. I saw your post on the Jellyfin website and I love the idea of trackfs, however my NAS is arm64-based and when I follow the instructions to setup the trackfs container, I run into this:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no spe
cific platform was requested                                                                                               
standard_init_linux.go:219: exec user process caused: exec format error  

I have done some searching and there seems to be various different approaches for adapting a container to a different platform, some simple and others very complicated. I am wondering if rewriting for a different platform is even possible with trackfs... and is this worth it for a beginner to try?

[andresch]

Hi Kanadork,

Thanks for looking into trackfs. Let me first warn you: I have no experience with the arm platform or docker images on any platform other than amd64. So please take this response with a grain of salt: From browsing some information I see at least no obvious reason why trackfs should not work on armv8.

For docker the base image is python-alpine and there exists a corresponding arm64v8 base image. So the basics should be there as long as the underlying OS has fuse support.

In addition to docker we would have to look into the trackfs python package and its dependencies. Also there I have no experience with the arm platform. But it should be relatively straight forward to test this with the Dockerfile.dev that I use for local development. And once it works there I can figure out how to publish for the arm platform on pypi.

Unfortunately I have no arm64v8 device available to test this by myself. Let me know if you have time to run a few basic tests with me, and then we'll see how far we get and what we learn from it.

[kanadork]

I would be happy to run some tests on my system in order to help out! I have a python 2.7 package installed already as a dependency for another (containerized) app that I'm running. I could see about setting up the arm64v8/python container that you linked and let you know if I run into any new obstacles.

[andresch]

Ok. Let's first try if you can build trackfs on your machine.

For that you would have to clone/download the trackfs repository in a directory on your NAS and then modify the file Dockerfile.dev:

1. replace line 13: FROM docker:dind with FROM arm64v8/python:3.8-alpine as builder
2. add to the end of line 16: rust cargo

in tools.sh replace in line 35: /usr/bin/trackfs with just trackfs

Then try to build the development environment from commandline with:
docker build -t andresch/testfs -f ./Dockerfile.dev .

With that we see that all binary dependencies of trackfs are supported on the arm64v8 platform.

Then we can look into the python dependencies. For this we launch the dev environment. This uses nearly the same arguments as trackfs, you only need to set in addition the mount point /work to the trackfs root directory, e.g.

docker run --name testfs   \
--device /dev/fuse --cap-add SYS_ADMIN   \
-v /var/run/docker.sock:/var/run/docker.sock  \
-v /path/to/musiclibrary:/src:ro  \
-v /path/to/fuse/mountpoint:/dst:rshared   
-v /path/to/trackfs/sourcdirectory:/work/   
--privileged   
-ti --rm   andresch/testfs

If everything works (which implies that also all python dependencies are available) you should get a bash prompt inside the container.

There you can start trackfs with the command tfs

Once we get at this point, I could see how to setup officially support to arm docker packages.

Curious to see you far you can get without major problems.

[kanadork]

Thank you for these instructions. I made the code changes and then ran the docker build -t andresch/testfs -f ./Dockerfile.dev . command and reached following error:

nektos/act crit platform linux/arm64 is not supported.  Make sure this script is up-to-date and file request 
at https://github.com/nektos/act/issues/new                                                                  
The command '/bin/sh -c curl https://raw.githubusercontent.com/nektos/act/master/install.sh | bash' returned 
a non-zero code: 1  

I guess the nektos/act package also does not support arm64... :(
From what I can tell, this package only depends on docker engine, and I've got docker-ce running just fine. (the hello-world works without issues) Do you think it's possible for us to work around this dependency without a whole lot of headache?

[andresch]

The setup.py error message is strange. I assume you don't get the error during build time (docker build ...) but when running the container (docker run ...), correct?

Can you please do two checks:

• is the root directory of trackfs outside of the container correctly mounted inside the container in /work? (it should contain setup.py)
• does /root/dev inside the container contain symbolic links to all files and directory in /work (including the setup.py)

Also the tfs error is weird. tfs should be defined as an alias in the shell that gets executed as the entrypoint of the container. Are you sure you are connected to the main program of the container and not just to a newly launched shell inside the running container?

[kanadork]

I think I'm misunderstanding the /work/ path usage. I found I had entered the wrong path before, so I corrected it and rebuilt the container. You're right that I was interacting with the wrong shell interface. Below is my most recent attempt trying to enter the tfs command immediately upon creation of the container... the paths still don't look right to me.

(my machine):/volume1/home/(user)/trackfs-master $ docker run --name testfs \                                 
> --device /dev/fuse --cap-add SYS_ADMIN \                                                                         
> -v /var/run/docker.sock:/var/run/docker.sock \                                                                   
> -v /share/Media/music:/src:ro \                                                                                  
> -v /share/trackfs:/dst:rshared \                                                                                 
> -v /volume1/home/(user)/trackfs-master/:/work/ \                                                                  
> --privileged \                                                                                                   
> -ti --rm andresch/testfs                                                                                         
Obtaining file:///root/dev                                                                                         
Requirement already satisfied: mutagen in /usr/local/lib/python3.8/site-packages (from trackfs==0.2.6) (1.45.1)    
Requirement already satisfied: fusepy in /usr/local/lib/python3.8/site-packages (from trackfs==0.2.6) (3.0.1)      
Requirement already satisfied: Lark in /usr/local/lib/python3.8/site-packages (from trackfs==0.2.6) (0.11.3)       
Requirement already satisfied: chardet in /usr/local/lib/python3.8/site-packages (from trackfs==0.2.6) (3.0.4)     
Installing collected packages: trackfs                                                                             
  Running setup.py develop for trackfs                                                                             
Successfully installed trackfs-0.2.6                                                                               
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using ven
/pip.pypa.io/warnings/venv
~/dev # tfs                                                                                                        
fuse: mount failed: No such file or directory                                                                      
Traceback (most recent call last):                                                                                 
  File "/usr/local/bin/trackfs", line 33, in <module>                                                              
    sys.exit(load_entry_point('trackfs', 'console_scripts', 'trackfs')())                                          
  File "/root/dev/trackfs/__init__.py", line 105, in main                                                          
    fuse = FUSE(trackfs, args.mount, foreground=foreground, allow_other=allow_other)                               
  File "/usr/local/lib/python3.8/site-packages/fuse.py", line 711, in __init__                                     
    raise RuntimeError(err)                                                                                        
RuntimeError: 1                                                                                                    
~/dev # /trackfs/tfs                                                                                               
/bin/sh: /trackfs/tfs: not found                                                                                   

Here is the filestructure of the container:

~/dev # ls -la                                                                                                     
total 56                                                                                                           
drwxr-xr-x    1 root     root           298 May  6 14:33 .
drwx------    1 root     root            98 May  6 14:33 ..
lrwxr-xr-x    1 root     root            13 May  6 14:33 .github -> /work/.github
lrwxr-xr-x    1 root     root            16 May  6 14:33 Dockerfile -> /work/Dockerfile
lrwxr-xr-x    1 root     root            20 May  6 14:33 Dockerfile.dev -> /work/Dockerfile.dev
lrwxr-xr-x    1 root     root            16 May  6 14:33 LICENSE.md -> /work/LICENSE.md
lrwxr-xr-x    1 root     root            15 May  6 14:33 README.md -> /work/README.md
lrwxr-xr-x    1 root     root            18 May  6 14:33 README.py.md -> /work/README.py.md
lrwxr-xr-x    1 root     root            13 May  6 14:33 VERSION -> /work/VERSION
lrwxr-xr-x    1 root     root            21 May  6 14:33 dev-launcher.sh -> /work/dev-launcher.sh
lrwxr-xr-x    1 root     root            17 May  6 14:33 launcher.sh -> /work/launcher.sh
lrwxr-xr-x    1 root     root            14 May  6 14:33 setup.py -> /work/setup.py
lrwxr-xr-x    1 root     root            17 May  6 14:33 starttfs.sh -> /work/starttfs.sh
lrwxr-xr-x    1 root     root            14 May  6 14:33 tools.sh -> /work/tools.sh
lrwxr-xr-x    1 root     root            13 May  6 14:33 trackfs -> /work/trackfs
drwxr-xr-x    1 root     root           160 May  6 14:33 trackfs.egg-info
lrwxr-xr-x    1 root     root            10 May  6 14:33 work -> /work/work
                                                   
~/dev # cd trackfs                                                                                                 
/work/trackfs # ls -la                                                                                             
total 60                                                                                                           
drwxrwxrwx    1 1000     users          136 May  4 20:14 .
drwxrwxrwx    1 1000     users          286 May  6 14:15 ..
-rwxrwxrwx    1 1000     users         3909 Feb 28 19:58 __init__.py
-rwxrwxrwx    1 1000     users         4425 Feb 28 19:58 albuminfo.py
-rwxrwxrwx    1 1000     users        12489 Feb 28 19:58 cuesheet.py
-rwxrwxrwx    1 1000     users        14506 Feb 28 19:58 flactracks.py
-rwxrwxrwx    1 1000     users         4813 Feb 28 19:58 fuseops.py
-rwxrwxrwx    1 1000     users         6017 Feb 28 19:58 fusepath.py

[andresch]

This is again weird. The error indicates that the fuse library can't mount the mount-point in /dst.

Let's try to narrow this down:

1. inside the container create a directory /tst and then start trackfs with trackfs --root /src /tst

   1. If that works trackfs works in principle and you should be able to launch another shell inside the running container with docker exec -ti testfs /bin/bash and then see trackfs in action when browsing through the directory-tree under /tst. Then we have to find out why you can't mount fuse on the /dst mount-point.

      1. verify that /dst exsits and is an empty directory
      2. check if /dst is really a mount point with mount | grep /dst
      3. verify that your mountpoint /share/trackfs exists outside of the container

   2. If that doesn't work, something with fuse is broken.

      1. please verify that both your NAS and the container have fuse support by calling fusermount -V inside and outside of the container
      2. check that the binary fuse library is available in /use/lib/libfuse.so;
         1. if not search for libfuse.se inside the container with find / -name libfuse.so -print, set the correct path in dev-launcher.sh line 11 and then terminate and restart the container

[kanadork]

Thank you very much for running through the possible scenarios. From within the testfs container I created a subdirectory like so: ~/dev # mkdir tst and then attempted to start trackfs:

~/dev # trackfs --root  /src /tst
fuse: bad mount point `/tst': No such file or directory
Traceback (most recent call last):
  File "/usr/local/bin/trackfs", line 33, in <module>
    sys.exit(load_entry_point('trackfs', 'console_scripts', 'trackfs')())
  File "/root/dev/trackfs/__init__.py", line 105, in main
    fuse = FUSE(trackfs, args.mount, foreground=foreground, allow_other=allow_other)
  File "/usr/local/lib/python3.8/site-packages/fuse.py", line 711, in __init__
    raise RuntimeError(err)
RuntimeError: 1

I then proceeded to check the existence of the directory /tst in the container:

~/dev # ls -la
total 56
drwxr-xr-x    1 root     root           304 May  8 00:11 .
drwx------    1 root     root            98 May  8 00:10 ..
lrwxr-xr-x    1 root     root            13 May  8 00:08 .github -> /work/.github
lrwxr-xr-x    1 root     root            16 May  8 00:08 Dockerfile -> /work/Dockerfile
lrwxr-xr-x    1 root     root            20 May  8 00:08 Dockerfile.dev -> /work/Dockerfile.dev
lrwxr-xr-x    1 root     root            16 May  8 00:08 LICENSE.md -> /work/LICENSE.md
lrwxr-xr-x    1 root     root            15 May  8 00:08 README.md -> /work/README.md
lrwxr-xr-x    1 root     root            18 May  8 00:08 README.py.md -> /work/README.py.md
lrwxr-xr-x    1 root     root            13 May  8 00:08 VERSION -> /work/VERSION
lrwxr-xr-x    1 root     root            21 May  8 00:08 dev-launcher.sh -> /work/dev-launcher.sh
lrwxr-xr-x    1 root     root            17 May  8 00:08 launcher.sh -> /work/launcher.sh
lrwxr-xr-x    1 root     root            14 May  8 00:08 setup.py -> /work/setup.py
lrwxr-xr-x    1 root     root            17 May  8 00:08 starttfs.sh -> /work/starttfs.sh
lrwxr-xr-x    1 root     root            14 May  8 00:08 tools.sh -> /work/tools.sh
lrwxr-xr-x    1 root     root            13 May  8 00:08 trackfs -> /work/trackfs
drwxr-xr-x    1 root     root           160 May  8 00:08 trackfs.egg-info
drwxr-xr-x    1 root     root             0 May  8 00:11 tst
lrwxr-xr-x    1 root     root            10 May  8 00:08 work -> /work/work

I notice it is not mounted in /work/ like the others.

Finally I checked the usage of fuse both inside the container:

~/dev # fusermount -V
fusermount version: 2.9.9

and outside the container in the NAS environment:

/volume1/home/(user) $ fusermount -V                                                                                                                                                                                                        
-sh: fusermount: not found    

So we can see that something is up with fuse not running outside the container in order to create the virtual mount point which is bound to /dst just as you suspected. So I searched for libfuse.so in my NAS directories and found these:

(user):/usr/lib $ ls -la                                                                                         
total 16724                                                                                                                  
(...)
lrwxrwxrwx    1 root     root            16 Mar 11 15:12 libfuse.so -> libfuse.so.2.9.4*
lrwxrwxrwx    1 root     root            16 Mar 11 15:12 libfuse.so.2 -> libfuse.so.2.9.4*
-rwxr-xr-x    1 root     root        309800 Mar 11 15:12 libfuse.so.2.9.4*
(...)

These paths match the relative paths in dev-launcher.sh : export FUSE_LIBRARY_PATH=/usr/lib/libfuse.so

In conclusion I guess the next obstacle is to get fuse up and running on the system side then?

[andresch]

I believe you have created the tst directory at the wrong location. You've created it in the directly /root/dev/tst rathern than in /tst.
This should not make a huge difference, but then you have to launch trackfs with the correct location, e.g. trackfs --root /src /root/dev/tst.

The missing fusermount command might indeed be an indication that your system doesn't have full fuse support. On the other side the fuse libraries seem to be there (btw: inside dev-launcher.sh we care about the location of the libraries inside the container, not outside) -- which might indicate that also your kernel comes with fuse support, On Debian based systems this would mean that you have the package libfuse but not the package fuse installed. Unfortunately I can't tell you if the missing fuse binaries is a k.o. for being able to run trackfs on your system; if your kernel comes with fuse-support then the fuse libraries and binaries inside the container might be sufficient. Unfortunately I mostly guessing here and can't claim to be an expert of fuse on this operating system / kernel level.

I would recommend to (a) rerun the test with the correct directory location and see if it works and if not (b) try to see if there is any way to get the fuse binaries installed on your NAS OS. But I'm afraid we are entering an area where I'm no longer able to provide you qualified help with the specifics of your NAS OS.

[kanadork]

Hi @andresch, sorry for the delay in getting back to you. I had to carefully retrace my steps in order to make sure I was inputting the correct commands into the container' vs system terminal. I remade the container and created the /tst folder at the top directory properly this time.

I am able to browse the contents of /tst from the container terminal and I see my music listed inside! However the "real" destination folder /share/trackfs is still empty on my NAS. I confirmed that the folder does exist and is reachable from both the terminal and the web UI. Then I checked the mounting of /dst from inside trackfs as you instructed:

bash-5.1# mount | grep /dst
/dev/md2 on /dst type btrfs (rw,relatime,space_cache,subvolid=257,subvol=/base/trackfs)

I'm not sure what this output means, does it look like the container folder /dst set up correctly?

[andresch]

I can't really help on why you can browse the fuse FS in a terminal, but not from other tools. All I can think of are ownership / permissions on the directories/files, but I guess you've checked this already. Everything else, best to my knowledge, would have to be something very specfiic to your NAS setup.

Did you try using a regular directory (e.g. music) within your subvolume /share/trackfs/ as docker mount point for /dst? Did you get the same error message when you then tried to start trackfs within the container? Do you see the music directory in the WebUI?

When you terminate the container non-gracefully then the FUSE mount point is still available in your system. As you don't have the fusermount command on your system. you might try to either launch the container again with the same mount points and then use fusermount -u /dst inside the container or try umount as root user outside of the container with the path of the destination folder outside of the container.

[kanadork]

Hi @andresch, after taking a break from working on my NAS, I am giving it another try from the beginning. I am almost certain that it is a user/access issue that is behind my problems getting the /dst music files to show up outside the container. I notice that when doing the experiment of creating a text file, from within the container, in the /dst folder... the file inherits the root user and it is visible to the NAS applications. However when examining the split flac files in /dst I notice they are assigned to user 999 of group ping and of course that group does not exist on the NAS side.

After making these observations I tried some additional solutions:

1. Using chmod to assign the real folder bound to /dst to 999:ping
2. Creating a new real folder outside of the existing shared subvolume of the NAS
3. Trying step 1 again on the folder created in step 2

Sadly none of these steps had any effect. Some additional observations I have made:

The first run of trackfs -v --root /src /dst prints INFO [MainThread ](albuminfo ) Tags to ignore: "CUE_TRACK.*|COMMENT" and then seemingly hangs. The destination folder is visible in the NAS but not populated. If I open a new terminal session in the container and run trackfs again, the familair error happens:

fuse: mountpoint is not empty
fuse: if you are sure this is safe, use the 'nonempty' mount option
Traceback (most recent call last):
  File "/usr/local/bin/trackfs", line 33, in <module>
    sys.exit(load_entry_point('trackfs', 'console_scripts', 'trackfs')())
  File "/root/dev/trackfs/__init__.py", line 105, in main
    fuse = FUSE(trackfs, args.mount, foreground=foreground, allow_other=allow_other)
  File "/usr/local/lib/python3.8/site-packages/fuse.py", line 711, in __init__
    raise RuntimeError(err)
RuntimeError: 1

I can then correct this with umount /dst in the container and then calling trackfs once more. Based on this I imagine the above "error" is in fact normal behavior as the container is trying to mount to the same place twice.

From reading your linked document I decided to verify the states of the volumes in the container and noticed that all of them are Propagation: rprivate. As I understand it, flagging /dst as :rshared would correct this, but then we're back to the original issue of not being able to create the container.

Thank you for your earlier help and I hope maybe some of these observations might reveal something new...

[kanadork]

Another update from me: I was finally able to get the mounting working properly on my system!

I followed example 19.4 from this page. I guess it must have been some issue specific to my setup because after manually creating the mounts by precisely following those steps, I can now access everything from Jellyfin. I have run into a further issue where the scraper does not seem to be cooperating with the new mount, but I will keep playing with the feature... Thanks again for your help.

[andresch]

This is actually great news. Just for me to understand: The problem was the way you did mount the filesystem outside of the container where trackfs tries to create fuse mountpoint for /dst, correct?

Looks like it is then also worth looking into a docker image for arm7. Can't promise how fast I will learn how to do it. Would you be willing to help me with testing when I find a way to publish such a container on docker hub?

[kanadork]

The problem was the way you did mount the filesystem outside of the container where trackfs tries to create fuse mountpoint for /dst, correct?

Yes that seems to be the case

Would you be willing to help me with testing when I find a way to publish such a container on docker hub?

Absolutely!

[kanadork]

Further update from me regarding usage with Jellyfin...
At this point I can see the /dst contents from my NAS and I can even play music files using the native player. However I can't seem get solve the scraping issue nor can I play the same music files from within the Jellyfin web UI. I created a new library specifically to index my /dst folder and it still struggles to match tags for all but 5-6 albums, none of which were originally FLAC+CUE examples.
At this point I can confidently say that trackfs itself is working as intended on my system. I just need to figure out why jellyfin fails to "see" the contents of the media files mounted in /dst.