Why ffcv is faster than native pytorch data loading?

[gongbudaizhe]

Good work !

I am just curious why ffcv is faster than native pytorch data loading? Once users understand the mechanism under the hood, they are more likely to adopt it : )

[GuillaumeLeclerc]

Hello!

Thank you very much. There are many reasons but on the top of my head:

• Single file vs many smaller files (less stress on the file system)
• Using the knowledge about which samples will be needed in the future to start reading the data ahead of time and absorb the latency of slow storage
• The use of threads instead of processes remove the need of any inter-process communication (shared memory etc...) and reduces memory consumption
• Compilation of the CPU augmentations transforms in machine code at the start of training (JIT).
• Copying the next batches while the compute is happening instead of simply asynchronously (but still on the critical path)
• Ability to use to move augmentations to CPU or GPU depending on where is your bottleneck.
• The use of pre-resized datasets
• In the case of systems with a lot of RAM, leveraging operating system level caching to implicitly store the entire dataset in RAM
• Being able to use alternative or a mix of different encoding for the data. JPEG can be slow to decode, if you have enough storage/RAM but not enough compute using uncompressed images might be faster, and FFCV enables that simply by switching a flag.

[h2th3k]

A quick question, as you probably mentioned in the repo, first epoch is pretty slow. My experience is using resnet18, after preprocessing the Imagenet Dataset, still the first epoch will take 1 h30 mins on A100-40G which is way more than the benchmark documented in the repo. I wonder where it could go wrong? Thanks. @GuillaumeLeclerc