Reproducing CoT-MAE results on NQ

[zhengmq2010]

Hi, I want to reproduce the results on NQ, but I don't see hyperparameter setting in paper or repo. Do I just need to follow the script msmarco/eval_msmarco.sh and modify the augments to reproduce? Can you provide more instruction? It would be great if you can provide the model weight of retrievers on two stages when finetuning on NQ. Thanks in advance!

[ma787639046]

Sorry for the late reply. The issue raised in our organization repo does not give me a notification.

Following Condenser, we use DPR to train and test on NQ.

Two-stage pipelines are used:

Stage 1: Train with BM25 negatives.

Stage 2: Train with BM25 negatives + hard negatives mined from CoT-MAE stage 1 retriever.

You can also refer to here(https://github.com/texttron/tevatron/blob/main/examples/coCondenser-nq/README.md) for pipeline instructions, but we use the hard negatives mined from CoT-MAE stage 1 retriever, rather than the negatives provided by coCondenser.