data collator for PO and SFT each

[naajeehxe]

When running run_script.sh for DPO or SFT, which data collator should I use?

The train_dpo_lora.py file is implemented with DataCollatorForSupervisedDataset, but I get a warning message from dpo_trainer.py suggesting that I should use DPODataCollatorWithPadding.

CSR/train_csr/dpo_trainer.py

Line 683 in 353de42

if not self.use_dpo_data_collator:

Could you help me reproduce your work?

Additionally, I noticed that the DPODataCollatorWithPadding implementation does not accept arguments like max_length or max_prompt_length as input, but your code attempts to pass these arguments.

I would appreciate it if you could assist me with these issues.