Train/Inference Random Model using this Library.

[xuann6]

Hi there,

I am trying to re-produce the result of your experiment, and the provided code seems only contains the function of error analysis. On the other hand, I found a function "create_trainer_engine", which seems used to train the model.

I am confused if the pack of code is only used for error analysis or it can be extended to generate the accuracy results of your paper. I understand this might need some efforts but I am trying to figure out if this can work. Thanks!

In addition, it seems that only Hopper arch from Nvidia (GH100) supports FP8 operation at the moment, is there any other options if I would like to train/inference the model using FP8?

Thanks again for reading the message.

[akuzmin-qc]

Yes, our repository contains the code to reproduce the inference results with the quantized model (see image_net.py), please refer to the README file for further instructions:
https://github.com/Qualcomm-AI-research/FP8-quantization/blob/main/README.md

[Ther-nullptr]

So does your code support FP8 training of QAT? I see that you have written some STE functions, but not finished yet.

[Ther-nullptr]

class FP8QuantizerFunc(Function):
    @staticmethod
    def forward(ctx, x_float, bias, num_exponent_bits):
        return quantize_to_fp8_ste_MM(x_float, bias, num_exponent_bits)

    @staticmethod
    def backward(ctx, grad_output):
        return grad_output, None, None

The STE is defined but never used. Is this STE work?

[akuzmin-qc]

The codebase currently supports only inference, and there are no training pipelines. However the FP8 quantizer implementation itself supports learning and can be used as a part of QAT pipeline.

[xuann6]

I see, thanks for the answer! That is helpful to me, wish you a nice day.