Issues with threading

[jetrz]

Hello, I am calling BPCells::svds(mat, k=50, threads=n_threads), but the threads seem to be all sleeping? Would greatly appreciate help on how to debug this, thank you :)

Misc:

• mat is of dimensions 10513465 cells x 19486 genes
• From iostat, %idle is very high (close to 100) and %iowait is very low (close to 0)
• CPU usage is still well under 50% despite having 8 threads.
• I've tried forcing OpenBLAS and OpenMP threads to = 1, and verified that Sys.getenv("NSLOTS") = 8.

[jetrz]

This is the relevant section of code:

mat <- open_matrix_anndata_hdf5(mat_path)
nc <- ncol(mat)
idx <- sample(seq_len(nc), size = floor(subset * nc), replace = FALSE)
cat("Before subsetting:", dim(mat), "\n")
mat <- mat[, idx, drop = FALSE]   # subset columns
cat("After subsetting:", dim(mat), "\n")

print("Running PCA...")
svd <- BPCells::svds(mat, k=50, threads=n_threads)
pca <- multiply_cols(svd$v, svd$d)

[jetrz]

Oh wait, I just realised its most likely because I'm using AnnDataMatrixH5 format instead of the DirMatrix right? Sorry, that's very careless of me. Converting to DirMatrix now and then will test again.