How to Calculate QSUR in practice？

[RanchiZhao]

Hi! I'm looking to calculate QSUR as a priori indicator of quantization difficulty. Given a model's weights and target quantization bits(WxAy), what's the process to compute this metric?

[BrotherHappy]

Under the assumption that the samples follow a Gaussian distribution,
you can estimate the parameters using the Maximum Likelihood Estimation (MLE) method.
The corresponding NumPy implementation is as follows:

import numpy as np

# Assume X is an N × d numpy array
def estimate_gaussian(X):
    N, d = X.shape
    mu = np.mean(X, axis=0)  # Compute the mean vector of shape (d,)
    X_centered = X - mu
    Sigma = (X_centered.T @ X_centered) / N  # Compute the covariance matrix using MLE
    return mu, Sigma

[lidh15]

so you mean the normalized QSUR shown in the paper is the QSUR of some Gaussian data with the estimation of mu and Sigma, instead of the real data?

Under the assumption that the samples follow a Gaussian distribution, you can estimate the parameters using the Maximum Likelihood Estimation (MLE) method. The corresponding NumPy implementation is as follows:

import numpy as np

Assume X is an N × d numpy array

def estimate_gaussian(X):
N, d = X.shape
mu = np.mean(X, axis=0) # Compute the mean vector of shape (d,)
X_centered = X - mu
Sigma = (X_centered.T @ X_centered) / N # Compute the covariance matrix using MLE
return mu, Sigma

[thuiaaa]

same problem, actually , we can calculate the QSUR follow the paper theoretically, but as the hidden channel increase, it seem to be really close to 0, looking for more communicate with this

[thuiaaa]

while actually I think maybe it doesn't make any sense for a real tensor

[BrotherHappy]

What is actually being calculated is: the nth root of QSUR. This can be achieved through log transformation to avoid numerical issues.

[lidh15]

What is actually being calculated is: the nth root of QSUR. This can be achieved through log transformation to avoid numerical issues.

but anyway the codes posted in April is not computing it
would you please show some codes how to calculate the nth root of QSUR