[Possible Feature] ELLA for SD1.5

[iwr-redmond]

I have come across code that integrates Tencent's criminally underused ELLA with Diffusers. Daniel Sandner's article from last year has some good examples of what ELLA can achieve with his Photomatix SD1.5 model.

The code is contained in a third-party fork, with the model code being here and the pipeline code being here. The pipeline integrates Loras, IP Adapters, and PAG, which I think is more than the reference code was capable of (the ComfyUI implementation supporting Loras and ControlNets).

There are some issues that the pipeline author has flagged, suggesting that implementation may be of moderate complexity. Note that despite being almost entirely relegated to ComfyUI, the original ELLA reference code was implemented using Diffusers.

I reckon ELLA would make a good pair with HiDiffusion.

[iwr-redmond]

It is possible that the just-released LLM4Gen would serve a similar purpose without as much adaptation. The code appears to combine CLIP and LLM together, rather than requiring separate prompts like ELLA. While the demo code shows nothing about Loras, Controlnets, etc, the author says that it should be fine.