Nano custom alphabet encoding not giving correct result. Help?

[gogo199432]

Hey, so I'm trying to use the custom alphabet functionality of this lib since I'm trying to implement a Nano cryptocurrency client in CSharp. However for some reason I get the wrong encoding at the end, and since I'm too stupid to understand your code, I was hoping you could help me out?

Links for the nano stuff I'm using for reference:
https://docs.nano.org/integration-guides/the-basics/#seed (here the public address section)
https://docs.nano.org/integration-guides/key-management/

I get the byte array with this function:

  public static byte[] ConvertHexToByteArray(string hex)
  {
            return Enumerable.Range(0, hex.Length / 2).Select(x => Convert.ToByte(hex.Substring(x * 2, 2), 16))
                .ToArray();
  }

I have a test that checks for the string->byte array conversion so it should be okay. I'm trying to encode the public key to public address.
Key: 5b65b0e8173ee0802c2c3e6c9080d1a16b06de1176c938a924f58670904e82c4
Address: 1anrzcuwe64rwxzcco8dkhpyxpi8kd7zsjc1oeimpc3ppca4mrjt

Here is my code:

public byte[] PublicKey { get; private set; }

public string PublicAddress { get; private set; }

private static readonly CustomBase32Encoding Encoder = new CustomBase32Encoding("13456789abcdefghijkmnopqrstuwxyz", null);

public void GeneratePublicAddress()
{
    
    var blake2BConfig = new Blake2BConfig
    {
        OutputSizeInBytes = 5
    };
    var hasher = Blake2B.Create(blake2BConfig);
    hasher.Update(PublicKey, 0, 5);
    var digestHash = hasher.Finish();

   // The important stuff here:
    var base32Pub = Encoder.GetString(PublicKey);
    
    PublicAddress = "nano_"+base32Pub+Encoder.GetString(digestHash);
}

Thanks in advance :)