WSL 2 uses half the number of cores on AMD Threadripper 3990X

[AAlMutairi]

Environment

Windows build number: Microsoft Windows [Version 10.0.19041.329]
Your Distribution version: Ubuntu: 20.04
WSL 2

Steps to reproduce

I am using AMD threadripper 3990x in my PC. when I use the command lscpu I get the following

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   48 bits physical, 48 bits virtual
CPU(s):                          64
On-line CPU(s) list:             0-63
Thread(s) per core:              2
Core(s) per socket:              32
Socket(s):                       1
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           49
Model name:                      AMD Ryzen Threadripper 3990X 64-Core Processor
.
.
.

Also when I use the command nproc, I get 64.

However, using both openmpi and mpich to run parallel job, mpi uses only 32 cores (half real cores). For this test I used the following code (copied from: https://mpitutorial.com/tutorials/mpi-hello-world/)

#include <mpi.h>
#include <stdio.h>

int main(int argc, char** argv) {
    // Initialize the MPI environment
    MPI_Init(NULL, NULL);

    // Get the number of processes
    int world_size;
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);

    // Get the rank of the process
    int world_rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);

    // Get the name of the processor
    char processor_name[MPI_MAX_PROCESSOR_NAME];
    int name_len;
    MPI_Get_processor_name(processor_name, &name_len);

    // Print off a hello world message
    printf("Hello world from processor %s, rank %d out of %d processors\n",
           processor_name, world_rank, world_size);

    // Finalize the MPI environment.
    MPI_Finalize();
}

Expected behavior

.
.
.
Hello world from processor Ubuntu, rank 10 out of 64 processors
Hello world from processor Ubuntu, rank 18 out of 64 processors
Hello world from processor Ubuntu, rank 23 out of 64 processors
.
.
.

Actual behavior

.
.
.
Hello world from processor Ubuntu, rank 10 out of 32 processors
Hello world from processor Ubuntu, rank 18 out of 32 processors
Hello world from processor Ubuntu, rank 23 out of 32 processors
.
.
.

[AAlMutairi]

Not sure if it is relevant but I am experiencing the same issue in hyper-V too.

[WSLUser]

It's the kernel config. Look at https://github.com/microsoft/WSL2-Linux-Kernel/tree/master/Microsoft. In the config for x86_64 you will see it's set to 64. This is standard from the Linux kernel. What you can do is update the config to match the number of cores you have. Ideally WSL would do a check on the number of CPU cores and update the config appropriately in .wslconfig. For now this is a manual process.

[AAlMutairi]

@WSLUser , thanks for the answer. Just to confirm, you meant updating config-wsl since I couldn't find .wslconfig. if this is the case, I believe the part of interest is the following:

CONFIG_NR_CPUS_RANGE_BEGIN=2
CONFIG_NR_CPUS_RANGE_END=512
CONFIG_NR_CPUS_DEFAULT=64
CONFIG_NR_CPUS=256

Are you suggesting that despite the range, WSL 2 uses the default value as a max value?

My apologies if I misunderstood your suggestion.

[WSLUser]

You have to create .wslconfig. https://docs.microsoft.com/en-us/windows/wsl/release-notes#build-18945

[AAlMutairi]

@WSLUser thank you again for the suggestion and sorry for the misunderstanding on my part. I tried your method to change the number of processors. It works when I decrease the number of processors but unfortunately, it doesn't work passed the 64 processor (which is equivalent to 32 physical processors). it seems to still limit me to half of number of physical cores (64/2 = 32).

[sanastasiou]

@WSLUser Does this work also for multiple CPUs? i.e. Dual Xeon setup?

[WSLUser]

Not sure. @craigloewen-msft would probably know better. In your case it appears the kernel config itself needs updating. You should be able to override the original value in .wslconfig as well. You should see the option in the release notes. And yes (sorry I didn't answer before), it's using the default value. So you'll overwrite it. I don't recommend going above 256.

[AAlMutairi]

@WSLUser Thanks for all the help, I guess I will wait for the kernel to be updated.
@benhillis @therealkenc, would you be able to let us know if such fix to the kernel will be added to the next build?
@sanastasiou Did you have the chance to try the .wslconfig method?

[sanastasiou]

@AAlMutairi not yet,not sure if it applies to dual cpu setups as well. If it does, I'll try.

[AAlMutairi]

Any updates or fixes to test?

[mozram]

It affect compiling also when running make -j. Only half of CPUs used whereas WSL1 does not have this issue. Ryzen 2600, Ubuntu 20.04 WSL2

[sanastasiou]

This basically blocks any usage of WSL 2, even if I check out my repo there, I lose 50% of my processing power.. That's simply a no go.

[AAlMutairi]

@mozram, it is surprising that it was working for you in WSL 1. Unfortunately for me, Both don't work for me.