NVME: Invalid argument (os error 22)

[johnramsden]

We are getting this - it is not properly getting logged, we have a logging problem as well. This was only found by inserting a print after the receive of the write_response.

OxCache/oxcache/src/server.rs

Line 295 in cb3435a

return Err(std::io::Error::new(std::io::ErrorKind::Other, format!("write location error: {}", e)));

I believe this corresponds to us attempting to write data larger than the MDTS. We likely need to chunk accordingly in a loop for writes.

I've been testing with a block interface so I'm not sure how zone append will behave here or if it will be the same.

[johnramsden]

Related #23

[johnramsden]

For zone append, we can use a reader writer lock and read lock if MDTS is larger than what we're trying to write. And if we're trying to write larger than the MDTS we write lock and loop over append until complete.

We should assert that all the locations are sequential.

[johnramsden]

Added a couple tests and determined some new info:

---

ZNS append fails beginning at 512K (tested with increasing powers of two)

1e8eb37

See oxcache/src/bin/znsappendtest.rs

Wrote 65536 bytes
Wrote 131072 bytes
Wrote 262144 bytes
Failed to write 524288 bytes
Error: Custom { kind: Other, error: Errno(Errno { code: 22, description: Some("Invalid argument") }) }

Verified, this is NOT scheduler. (mq also fails, and we are on 24.04, noop should be safe)

---

Also relevant, tested what throughput we can achieve with looping 256K, and it is sufficient to reach sheet max (1200MB)

efe3670

oxcache/src/bin/znsappendtest.rs

Elapsed: 92.310 seconds
Wrote 105.176 GiB (1.139 GiB/s)

---