fixing bugs for running lobster jobs in el9

[rgoldouz]

Ben’s recommendations were:

Run Lobster for Python 3

I installed the lobster-python3 branch.

Use the correct CCTools

export PATH="/afs/crc.nd.edu/group/ccl/software/x86_64/redhat9/cctools/current/bin:$PATH"

Run the correct factory

--runos rh9-wq-prot-11

With these changes, my jobs were submitted successfully. I also had to make a few fixes to the Lobster package itself:

SCRAM directory detection

Update sandbox.py:L68:

def _get_cmssw_arch(self, dirname):
candidates = glob.glob('{}/.SCRAM/slc*'.format(dirname))
if len(candidates) != 1:
candidates = glob.glob('{}/.SCRAM/el*'.format(dirname))
if len(candidates) != 1:
raise AttributeError("Can't determine SCRAM arch!")
return os.path.basename(candidates[0])

CVMFS source

Modify wrapper.sh:L129:

slc=$(sed -n 's/.[Rr]elease ([0-9][0-9])./\1/p' /etc/redhat-release)
arch=$(echo sandbox-${LOBSTER_CMSSW_VERSION}-el${slc}_.tar.bz2 | grep -oe "el${slc}_[^.]*")

Python command

On EL9, python is not defined. I replaced calls to python with python3 in:

wrapper.sh:L164

source.py:L363

Logging formatter fix

In task.py:L57, update to:

chevron = '>' * int(record.levelno / logging.DEBUG + 1)