What is the benifit of DualPipeV?

[AriaAstesia]

I wonder the benfit brought by V-shape. Assuming the batch size and the number of PP stages are the same, DualPipeV requires either twice the devices, twice the micro-batch size, or twice the micro-batch number. The latter two options take twice the time, while the first approach appears to be the same as the original one?

[shifangx]

The benfit brought by DualPipeV:
(1) DualPipeV is more concise than DualPipeV.
Conciseness is a very important factor to consider when choosing a solution.
The more concise it is, the easier it is for everyone to understand and the easier it is to integrate with other algorithms.
For example, less concise may lead to some misunderstanding, such as discussed in #22.
And also, more concise is useful when combining DualPipeV with other features, such as FSDP(Fully Sharded Data Parallel).

(2) DualPipeV can help us runing the same size model with the smallest possible scale.
This is quite useful for debugging and troubleshooting.