support transformers multi-modal grpo

[hjh0119]

Summary

• Support transformers-standard multimodal message format (content: List[Dict]) in template pipeline
• Fix _check_max_length to only truncate input_ids/labels, keeping multimodal tensors intact
• Clean Arrow-serialized None keys from content blocks to prevent Jinja template misparsing
• Add multi_modal_data / mm_processor_kwargs pass-through in vLLM sampling pipeline
• Forward encode() kwargs in LazyDataset for proper add_generation_prompt support
• Add CLEVRProcessor, MultiModalAccuracyReward, multimodal GRPO demo

Changed files

• template/base.py — _build_mm_messages supports both List[Dict] and legacy str content;
  _apply_chat_template strips null keys; _check_max_length only truncates sequence fields
• sampler/vllm_sampler/vllm_engine.py — sample() accepts multi_modal_data dict directly
• sampler/vllm_sampler/vllm_sampler.py — extract multi_modal_data from message content blocks
  for vLLM; add mm_processor_kwargs forwarding
• dataset/lazy_dataset.py — forward encode() kwargs to batch_encode

[gemini-code-assist]

Code Review

This pull request introduces multimodal GRPO training capabilities, featuring a new training script for the Qwen3.5 VL model on the CLEVR dataset. Core framework enhancements include a new MultiModalAccuracyReward, a CLEVRProcessor, and a refactored vLLM sampler that better supports multimodal data and standard message formats. Review feedback identifies a logic error in the training loop where metrics are reset prematurely, as well as opportunities to optimize reward function instantiation and fix a potential typo in the reward computation.