About inference engine JetEngine

[STAR-CV]

Hello, great work! I'd like to ask about my self-developed model with a block diffusion architecture similar to sdar. Which parts can be modified to enable the use of accelerated inference engines JetEngine?
I would be most grateful for any suggestions.

[Labman42]

if your model's prefilling and decoding method is the same with SDAR, you just need to write your own model file (reference the files in the jetengine/models)

[STAR-CV]

Thank you so much for your prompt response. Should any further issues arise, I will be sure to reach out to you again.Thank you so much for your outstanding work.

[STAR-CV]

Hello, I'd like to inquire about multi-node inference code. By invoking the LLM configuration, we've successfully implemented single-node multi-GPU inference. @Labman42