Debian 13 Compatibility

[bentallea]

Here are the following issues I experienced when install MANS on a fresh Debian install:
The following packages need to be installed before you can run ansible:

• ansible
• git
• curl
• wget
• python3-full
• python3-pip
• python3-setuptools
• python3-dev
• build-essential
• autoconf
• automake
• pkg-config
• libuuid1
• libblkid-dev
• libselinux1-dev
• libssl-dev
• zlib1g-dev

You must make a Python virtual environment with the following modules installed and source activate the venv environment:

• github3.py>=1.0.0a3
• requests==2.32.5
  This does not include the requirements.txt from both muffins-awesome-nas-stack and mergerfs-cache-mover installation, which also need to be installed into the venv.

In manage_linux/main.yml, the following packages need to be removed:

• mlocate
• software-properties-common

In manage_linux/main.yml, the following packages need to be added:

• plocate

Your snapraid-sync-runner is hard configured to run at Sunday at 03:00 regardless of the setting in vars.yml, and refused to move any files on my system, for did it create the parity data during snapraid sync, nor did it create any btrfs snapshots. Unfortunately, as I suffered a critical error involved with the snapraid sync, I have deleted the OS that ran the stack and no further logs are available.

[tigattack]

I ran MANS against a fresh Debian 13 install last weekend, so this is good timing.

I didn't experience the need for all those runtime dependencies in the first list (except the obvious requirement of ansible). A lot of them look like requirements for building various components of MANS, the dependencies for which are installed during execution. I think something else has gone wrong there; you, as the user, should not have to install most of those yourself, and indeed this was my experience.

As you stated, the dependencies in the second list are needed on the Ansible control node. Good spot re. requests not being included, though. I'll put in a PR to resolve this.

I'll also put in a PR shortly to replace mlocate with plocate and remove software-properties-common. I'm unsure why the latter was originally required, but I don't think it is anymore as the playbook completed fine on Debian 13 without it. The only thing I know it provides is the add-apt-repository binary, which shouldn't be needed here.

[tigattack]

FWIW, I did run into a lot of issues with clashes between Debian 13's Python dependencies and those of the various Python-based tools installed by MANS.

Some of this will take a little more time & thought to resolve, but one thing I was going to look at was having Ansible create a Python virtual environment for mergerfs-cache-mover. This only addresses one of the Python-related issues, but it's a simple starting point and should be easy enough to implement.

All the other Python issues were build requirements, IIRC. A virtual environment could be the right option there too, but it will depend on the specifics of each tool's build processes.

Lmk if you have any thoughts @monstermuffin.

Edit: actually, one of the packages mentioned in the first list of the issue here is a missing dependency - I needed to add python3-setuptools to build btrfs-progs.

[bentallea]

mans-auto.txt

here is the ssh capture of me installing the mans stack in a debian VM with 9x 16gb disks, 7x8tb disks, and 2x 4gb disks. here were the instructions I followed:

sudo apt update

sudo apt install ansible git curl wget python3-full python3-pip python3-setuptools python3-dev build-essential autoconf automake pkg-config libuuid1 libblkid-dev libselinux1-dev libssl-dev zlib1g-dev 

python3 -m venv venv

source venv/bin/activate

python3 -m pip install github3.py>=1.0.0a3 requests==2.32.5 #based on testing, this is an ansible dependency, rather than a MANS dependency

git clone https://github.com/monstermuffin/muffins-awesome-nas-stack.git

cd muffins-awesome-nas-stack

cp inventories/inventory.example.yml inventories/inventory.yml

cp vars_example.yml vars.yml

<change inventory.yml for localhost connection>

<configure vars.yml for all disks and content files, and change main mountpoint to /mnt/storage>

<make required change for manage_linux/defaults/main.yml>

pip install -r requirements.txt

ansible-galaxy install -r requirements.yml

ansible-playbook playbook.yml -K

now, I will run the mans stack manually fixing bugs as I go.

sudo apt install git curl ansible

git curl https://github.com/monstermuffin/muffins-awesome-nas-stack.git

cd muffins-awesome-nas-stack

cp inventories/inventory.example.yml inventories/inventory.yml

cp vars_example.yml vars.yml

<edit inventory for local connection>

<configure vars.yml for all disks and content files, and change main mountpoint to /mnt/storage>

<make required change for manage_linux/defaults/main.yml>

pip install -r requirements.txt

sudo apt install python3-pip # pip not already installed

pip install -r requirements.txt # receive instructions to create venv

cd

python3 -m venv venv # receive instructions to install venv

sudo apt install python3-venv

python3 -m venv venv

source venv/bin/activate

pip install -r requirements.txt # finally able to install requirements

ansible-galaxy install -r requirements.yml

ansible-playbook playbook.yml -K # fails at TASK [install_btrfs : Compile btrfs-progs], setuptools missing

sudo apt install -y python3-setuptools

ansible-playbook playbook.yml -K

mans-manual.txt

virtual machines are preserved for future testing
also, apologies for accidentally marking issue closed. could someone please reopen the issue?

[tigattack]

Thanks a lot @bentallea. FYI I edited your comment to repair the code blocks. Inline code use single backticks, code blocks require 3 backticks on the lines above and below the block :)

I'm struggling to track down where the dependency for Python requests on the control node is coming from. If requests isn't present (pip uninstall requests), where does it fail, and what's the error? I would test this myself, but requests is a system dependency on the machine I'm using right now, so I can't remove it.

[bentallea]

It could have been something solved with a recent update, and I did not experience the requests issue during my reconstruction process. If I remember correctly, I think it was a dependency of ansible to install fastfetch from github.

see the attached file. also, passwordless sudo does not work.

mans-auto2.txt

[bentallea]

And the log just reinstalling requests.

mans-auto3.txt

[monstermuffin]

Hey chaps,

Thanks for looking at this. Big ol' thank you (as per) to @tigattack for starting to tackle some bits.

I will review your PRs and merge soon, I am also going through a reorganisation of my Infra right now and that has taken precedent over looking at this. In all honesty I never considered Trixie compatibility but I am moving a lot of stuff to Trixie myself now so this is good timing.

A lot of things in MANS are probably legacy/left over from the initial testing/trying stuff out. As the role(s) is quite vast and does a lot of things, there is defiantly some debt in these files so this exercise, especially with eyes that are not mine is extremely helpful and I'm grateful :)

[tigattack]

Np, happy to help :)

I think most of the easy stuff is already addressed in the draft PR, but there's not much point merging as it will still fail on trixie without more fixes.

The more difficult work will be figuring out all the Python stuff. There are two main causes of the Python issues:

• Recent versions of Python (I want to say >=3.12 but not sure) are very strict about not messing with libraries at a system level, so basically all cases of pip install (both explicit via Ansible and implicit via build processes for various other installs MANS is doing) will break.

  • As I write this, I seem to recall discussing this with you before, so it's possible we already addressed this, at least in part? Maybe this was actually already the case in bookworm? Maybe not as much of an issue as I thought, but definitely worth looking into.
    Memory is failing me, who knows — As it's been a week or so since I worked through all the issues to build on Trixie, naturally I have forgotten 99.9% of the entire process 🙃 I don't recall if I actually ran into this specific issue, or if it was just the following one.

• The big one: Trixie appears to have quite a significantly different set of Python system dependencies to Bookworm, which causes tons of conflicts (again with both explicit & implicit usage of pip install)

  • The first solution that comes to mind (and probably the simplest/most viable?) is building a venv for each distinct Python-related install. This would be a bit annoying, but I don't have any better ideas right now...

    • venvs could be created in /tmp for one-time stuf, but should be set up in a relevant path for lasting stuff (i.e. cache mover).

Edit: community.general.easyinstall could be useful.