issue in the function computing NDCG

[rguo12]

Hi,

I just find there is an issue in the following function

def ndcg(y_pred, y_true, top):
   assert y_pred.shape[0] == y_true.shape[0]
   top = min(top, y_pred.shape[0])

   first_k_docs = sorted(zip(y_true, y_pred), key=cmp_to_key(doc_comparator))
   first_k_docs = np.array(first_k_docs)[:top,0]

   top_k_idxs = np.argsort(y_true)[::-1][:top]
   top_k_docs = y_true[top_k_idxs]

   dcg = cumulative_gain(first_k_docs)
   idcg = cumulative_gain(top_k_docs)

   return dcg / idcg if idcg > 0 else 1.

how can ndcg=1 if idcg == 0? If idcg == 0 you should just ignore that query. This definitely makes the NDCG look higher than it is expected to be.

Best,

Ruocheng Guo