Pindel does not install with HTSLib - undefined reference to `hts_drand48()'

[jst43]

I've installed htslib from git as:

git clone https://github.com/samtools/htslib.git
cd htslib/
autoheader
autoreconf
./configure
make
make install

which ran without errors. Now when I try to install pindel as:

git clone https://github.com/genome/pindel.git
cd pindel/
./INSTALL ../htslib/

I get the following error message:

path is now: /mnt/raid/Resources/Software/htslib/
make -C src pindel
make[1]: Entering directory '/mnt/raid/Resources/Software/pindel/src'
g++  -L/mnt/raid/Resources/Software/htslib/ -Wl,-rpath /mnt/raid/Resources/Software/htslib/ pindel.o reader.o reporter.o searcher.o parameter.o refreader.o control_state.o search_deletions_nt.o search_inversions.o search_inversions_nt.o bam2depth.o search_tandem_duplications.o search_tandem_duplications_nt.o output_sorter.o farend_searcher.o search_variant.o searchshortinsertions.o searchdeletions.o output_file_data.o bddata.o shifted_vector.o read_buffer.o line_reader.o ifstream_line_reader.o gz_line_reader.o pindel_read_reader.o user_defined_settings.o fn_parameters.o logstream.o search_MEI_util.o search_MEI.o assembly.o genotyping.o -O3 -fopenmp -lhts -lm -lz -o pindel
reader.o: In function `ks_shuffle_uint32_t(unsigned long, unsigned int*)':
/mnt/raid/Resources/Software/pindel/src/reader.cpp:67: undefined reference to `hts_drand48()'
collect2: error: ld returned 1 exit status
Makefile:36: recipe for target 'pindel' failed
make[1]: *** [pindel] Error 1
make[1]: Leaving directory '/mnt/raid/Resources/Software/pindel/src'
Makefile:12: recipe for target 'pindel' failed
make: *** [pindel] Error 2

mv: cannot stat 'src/pindel': No such file or directory
mv: cannot stat 'src/pindel2vcf': No such file or directory
mv: cannot stat 'src/sam2pindel': No such file or directory
mv: cannot stat 'src/pindel2vcf4tcga': No such file or directory

Pindel successfully compiled. The pindel executable can be found in this directory.

You can test pindel with the example files in the demo directory; example commands are given in the RUNME in that directory.

For help and background information, see the pindel wiki and its FAQ on gmt.genome.wustl.edu/packages/pindel, as well as
the FAQ file in the Pindel root directory (the same directory as this INSTALL script is located)
or contact us on kaiye@xjtu.edu.cn

[jmarshall]

This hts_drand48 malarky in ksort.h is a recent addition to htslib by @daviesrob.

What platform are you on? Has htslib been installed previously? Is it possible that you have compiled the Pindel .cpp files against headers in your ../htslib directory but this link command is picking up an older libhts.a?

[jst43]

I'm running Ubuntu 16.04 LTS, I recently installed samtools 1.3.1 which comes with htslib-1.3.1, but I was unable to install Pindel using the path to that htslib directory (I think the error was the same one as in this issue.

I have two libhts.a files; one for the htslib that was installed with samtools and one for the htslib I downloaded from git. I don't know how I would check which copy of libhts.a is being picked up in the installation.

[sarmadym]

I am having the exact same issue on RHEL7

[jmarshall]

Check which one is being pulled in by adding -Wl,-map,pindel.map to the g++ -L … command, or more quickly by checking whether there are any libhts.{a,so*} files in /mnt/raid/Resources/Software/htslib (i.e., the directory that is the argument to -L).

Alternatively point Pindel's ./INSTALL at the location where you installed this htslib rather than your htslib source directory tree, though Pindel's homegrown configury is somewhat rustic and this may or may not work well.

PR #75 has the gist of a workaround, but it is a terrible hack rather than the right solution.

[daviesrob]

Yes, it should be picking up hts_drand48() from libhts. You could try using nm to see if the library you expect to have it actually does:

$ nm -a ../htslib/libhts.a | grep hts_drand48
                 U hts_drand48
                 U hts_drand48
00000000000001dc T hts_drand48
$ nm -a ../htslib/libhts.so | grep hts_drand48
00000000000357e8 T hts_drand48

[jst43]

@jmarshall In the /mnt/raid/Resources/Software/htslib/ directory I have libhts.a, libhts.so, and libhts.so.2
@daviesrob Here's the outputs of the nm commands I ran:

$ nm ../htslib/libhts.a | grep hts_drand48
                 U hts_drand48
                 U hts_drand48
0000000000000150 T hts_drand48
$ nm ../htslib/libhts.so | grep hts_drand48
000000000002e480 T hts_drand48

[daviesrob]

Gah! The problem is that it's missing the extern "C" { } wrapper around the declaration of hts_drand48() in ksort.h. It works fine if you're using C, but not with C++ due to name mangling.

Applying this patch should fix it for you:

diff --git a/htslib/ksort.h b/htslib/ksort.h
index f50aa71..cf99884 100644
--- a/htslib/ksort.h
+++ b/htslib/ksort.h
@@ -65,6 +65,10 @@
 #include <stdlib.h>
 #include <string.h>
 
+#ifdef __cplusplus
+extern "C" {
+#endif
+
 // Use our own drand48() symbol (used by ks_shuffle) to avoid portability
 // problems on Windows.  Don't include htslib/hts_os.h for this as it
 // may not get on with older attempts to fix this in code that includes
@@ -289,4 +293,8 @@ typedef const char *ksstr_t;
 #define KSORT_INIT_GENERIC(type_t) KSORT_INIT(type_t, type_t, ks_lt_generic)
 #define KSORT_INIT_STR KSORT_INIT(str, ksstr_t, ks_lt_str)
 
+#ifdef __cplusplus
+}
+#endif
+
 #endif

I'll apply this to htslib and check it in.

[jst43]

Can confirm it's working for me now, thanks!

[jmarshall]

D'oh!

There's an extent to which you only want extern "C" around the new extern declaration, not the bulk of ksort.h.

[jst43]

@jmarshall so should the } be placed after
extern double hts_drand48(void);
instead of at the end of the file?

[daviesrob]

@jmarshall True, but I don't think it really matters, does it? And putting the closing bracket at the end means it's more likely to catch any similar problems in the future (not that I think there will be any, as I don't think we'd want to declare anything else in there).

[oeco28]

I am still having a problem with this. I tried installing pindel with htslib_1.8.
I can see that ksort.h has a
extern "C" {
...
statement around the declaration of drand48() so I am not sure what the problem might be.
I run within the pindel folder the command:
./INSTALL /path_to_htslib/

and I get the following error:

make -C src pindel
make[1]: Entering directory '/home/repository_software/pindel/src'
g++ -I/home/repository_software/htslib-1.8 -Wall -g -c -O3 -fopenmp pindel.cpp -o pindel.o
pindel.cpp: In function ‘void MergeInterChr(ControlState&, UserDefinedSettings*)’:
pindel.cpp:1557:66: error: call of overloaded ‘abs(unsigned int)’ is ambiguous
if (abs(All[index_a].FirstPos - All[index_b].FirstPos) < 10 && abs(All[index_a].SecondPos - All[index_b].SecondPos) < 10 && All[index_a].NumSupport + All[index_b].NumSupport >= cutoff) {
^
In file included from /usr/include/c++/7/cstdlib:75:0,
from /usr/include/c++/7/bits/stl_algo.h:59,
from /usr/include/c++/7/algorithm:62,
from pindel.cpp:23:
/usr/include/stdlib.h:837:12: note: candidate: int abs(int)
extern int abs (int __x) __THROW attribute ((const)) __wur;
^~~
In file included from /usr/include/c++/7/cstdlib:77:0,
from /usr/include/c++/7/bits/stl_algo.h:59,
from /usr/include/c++/7/algorithm:62,
from pindel.cpp:23:
/usr/include/c++/7/bits/std_abs.h:56:3: note: candidate: long int std::abs(long int)
abs(long __i) { return __builtin_labs(__i); }
^~~
/usr/include/c++/7/bits/std_abs.h:61:3: note: candidate: long long int std::abs(long long int)
abs(long long __x) { return __builtin_llabs (__x); }
^~~
/usr/include/c++/7/bits/std_abs.h:70:3: note: candidate: constexpr double std::abs(double)
abs(double __x)
^~~
/usr/include/c++/7/bits/std_abs.h:74:3: note: candidate: constexpr float std::abs(float)
abs(float __x)
^~~
/usr/include/c++/7/bits/std_abs.h:78:3: note: candidate: constexpr long double std::abs(long double)
abs(long double __x)
^~~
/usr/include/c++/7/bits/std_abs.h:84:3: note: candidate: constexpr __int128 std::abs(__int128)
abs(__GLIBCXX_TYPE_INT_N_0 __x) { return __x >= 0 ? __x : -__x; }
^~~
/usr/include/c++/7/bits/std_abs.h:102:3: note: candidate: constexpr __float128 std::abs(__float128)
abs(__float128 __x)
^~~
pindel.cpp:1557:127: error: call of overloaded ‘abs(unsigned int)’ is ambiguous
(abs(All[index_a].FirstPos - All[index_b].FirstPos) < 10 && abs(All[index_a].SecondPos - All[index_b].SecondPos) < 10 && All[index_a].NumSupport + All[index_b].NumSupport >= cutoff) {
^
In file included from /usr/include/c++/7/cstdlib:75:0,
from /usr/include/c++/7/bits/stl_algo.h:59,
from /usr/include/c++/7/algorithm:62,
from pindel.cpp:23:
/usr/include/stdlib.h:837:12: note: candidate: int abs(int)
extern int abs (int __x) __THROW attribute ((const)) __wur;
^~~
In file included from /usr/include/c++/7/cstdlib:77:0,
from /usr/include/c++/7/bits/stl_algo.h:59,
from /usr/include/c++/7/algorithm:62,
from pindel.cpp:23:
/usr/include/c++/7/bits/std_abs.h:56:3: note: candidate: long int std::abs(long int)
abs(long __i) { return __builtin_labs(__i); }
^~~
/usr/include/c++/7/bits/std_abs.h:61:3: note: candidate: long long int std::abs(long long int)
abs(long long __x) { return __builtin_llabs (__x); }
^~~
/usr/include/c++/7/bits/std_abs.h:70:3: note: candidate: constexpr double std::abs(double)
abs(double __x)
^~~
/usr/include/c++/7/bits/std_abs.h:74:3: note: candidate: constexpr float std::abs(float)
abs(float __x)
^~~
/usr/include/c++/7/bits/std_abs.h:78:3: note: candidate: constexpr long double std::abs(long double)
abs(long double __x)
^~~
/usr/include/c++/7/bits/std_abs.h:84:3: note: candidate: constexpr __int128 std::abs(__int128)
abs(__GLIBCXX_TYPE_INT_N_0 __x) { return __x >= 0 ? __x : -__x; }
^~~
/usr/include/c++/7/bits/std_abs.h:102:3: note: candidate: constexpr __float128 std::abs(__float128)
abs(__float128 __x)
^~~
Makefile:47: recipe for target 'pindel.o' failed
make[1]: *** [pindel.o] Error 1
make[1]: Leaving directory '/home/repository_software/pindel/src'
Makefile:12: recipe for target 'pindel' failed
make: *** [pindel] Error 2

mv: cannot stat 'src/pindel': No such file or directory
mv: cannot stat 'src/pindel2vcf': No such file or directory
mv: cannot stat 'src/sam2pindel': No such file or directory
mv: cannot stat 'src/pindel2vcf4tcga': No such file or directory

Pindel successfully compiled. The pindel executable can be found in this directory.

You can test pindel with the example files in the demo directory; example commands are given in the RUNME in that directory.

For help and background information, see the pindel wiki and its FAQ on gmt.genome.wustl.edu/packages/pindel, as well as
the FAQ file in the Pindel root directory (the same directory as this INSTALL script is located)
or contact us on kaiye@xjtu.edu.cn

I also tried installing it with samtools_1.3 but I am not having much luck either. Have you seen this happen?

thank you,
Omar

[jmarshall]

The abs problem is fixed by applying #64.