Enable ARM build, although some tests still fail

[adamnovak]

Changelog Entry

To be copied to the draft changelog by merger:

• vg can now build on the ARM architecture and pass unit (but not CLI) tests

Description

This pulls in #3023 and #2867, and a couple fixes to tie it all together, and lets vg build on ARM. I tested it on a little ARM board I have.

rock64@rock64:~/workspace/vg$ bin/vg version
vg version v1.30.0-71-gd1151be6e "Carentino"
Compiled with g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 on Linux
Linked against libstd++ 20200808
Built by rock64@rock64
rock64@rock64:~/workspace/vg$ arch
aarch64

We're not quite ready for the Raspberry Pi yet, and I don't know if it passes any tests on ARM, but it builds.

I'm not sure what's wrong with the SSW headers being out of date and not having the new argument, on Gitlab. I had to rm libssw.a to get the build to work.

[adamnovak]

This is the problem:

anovak@octagon vg]$ ag "bool Align.const char. query,"
deps/ssw/src/ssw_cpp.h
139:  bool Align(const char* query, const Filter& filter, Alignment* alignment, const int32_t maskLen) const;
156:  bool Align(const char* query, const char* ref, const int& ref_len,

deps/gssw/src/ssw_cpp.h
125:  bool Align(const char* query, const Filter& filter, Alignment* alignment) const;
139:  bool Align(const char* query, const char* ref, const int& ref_len, 

deps/vcflib/src/ssw_cpp.hpp
134:  bool Align(const char* query, const Filter& filter, Alignment* alignment) const;
148:  bool Align(const char* query, const char* ref, const int& ref_len,

We also need to upgrade vcflib's SSW, and also make sure we keep GSSW and SSW's ssw_cpp.h files separate.

We probably should drop SSW as a vg dependency and just pull it in from inside vcflib, like we do for htslib.

[adamnovak]

We do indeed pass unit tests:

===============================================================================
All tests passed (8252337 assertions in 500 test cases)

rock64@rock64:~/workspace/vg$ arch
aarch64

[adamnovak]

Here's a gizpped ARM64 vg static binary for testing.

vg.gz

[adamnovak]

The vcflib version won't interfere because I already fixed that in vcflib/vcflib#181.

I'm dropping the SSW headers from GSSW, since we don't actually use them, in vgteam/gssw#29

[adamnovak]

@mr-c, as an Apt Expert, do you have any idea why Arm Travis decides to remove one flavor of libcurl in favor of another here: https://travis-ci.org/github/vgteam/vg/jobs/756614100#L133

Or, moreover, why that results in having held broken packages and not being able to install any remaining vg dependencies here: https://travis-ci.org/github/vgteam/vg/jobs/756614100#L1190

Are we doing something wrong? Or is Travis giving us a bad packaging state to start with? Or can apt just not safely swap libcurl flavors?

I think a different version of libcurl is installed by default on amd64 versus on ARM. The RDF packages we need use the one that amd64 has by default, but on ARM it ends up needing to swap, and somehow forgets to uninstall the packages that relied on the old libcurl (which I don't know what they are).

[mr-c]

Can you try without the --no-upgrade to apt?

[adamnovak]

I'll give it a shot.

[adamnovak]

No, that doesn't seem to have worked.

[mr-c]

@adamnovak try switching to Ubuntu Focal?

[adamnovak]

I got a weird test failure:

txm --jobs 1 /builds/vgteam/vg/doc/wiki/Construction-Examples.md 
TAP version 13
1..1
not ok 1 Construct an hs37d5 graph: program exited with error
  ---
  program: |
    bash -eo pipefail
  exit status: 1
  stderr: |
    Restricting to 1 from 1 to end
    index file ./wiki-data/mock-hs37d5/data/hs37d5.fa.fai not found, generating...
    Restricting to 2 from 1 to end
    [vg::Constructor] Error: "2" not found in fasta file
    Restricting to 3 from 1 to end
    Restricting to 4 from 1 to end
    Restricting to 5 from 1 to end
    Restricting to 6 from 1 to end
    Restricting to 7 from 1 to end
    Restricting to 8 from 1 to end
    Restricting to 9 from 1 to end
    Restricting to 10 from 1 to end
    Restricting to 11 from 1 to end
    Restricting to 12 from 1 to end
    Restricting to 13 from 1 to end
    Restricting to 14 from 1 to end
    Restricting to 16 from 1 to end
    Restricting to 17 from 1 to end
    Restricting to 15 from 1 to end
    Restricting to 19 from 1 to end
    Restricting to 18 from 1 to end
    Restricting to 20 from 1 to end
    Restricting to 21 from 1 to end
    Restricting to 22 from 1 to end
    Restricting to X from 1 to end
    Restricting to Y from 1 to end
  stdout: ''
  check location: |
    lines 16-56
  program location: |
    line 1
  ---
# 0/1 passed
# FAILED 1

I don't think this is ARM related; I think we might be racing ourselves to make the fai or something. I will commit it and see if the problem goes away.

[adamnovak]

OK, while vg itself passes its internal unit tests, not all of the bash-tap tests (make test) of the command line tool are passing yet: https://ucsc-ci.com/vgteam/vg/-/jobs/74784#L7109

I'm going to merge this anyway, with the arm build/test set to only run on branches starting with "arm". We can work on fixing the failing tests in additional PRs.