Fine tuning ditto on one avatar image

[adityaVR]

If we fine tune ditto on single avatar image and then generate real time talking head for that avatar from audio , will the resulting model be lighter and will it give faster response ? If not , what are the best ways to achieve low latency with ditto for known /one avatar image scenario?
Does ditto skip it's identity resemblance /identity extraction step if we train it directly on a single avatar image that will also be used at inference time ?
Can we pre compute the identity features (and store apperance features in memory) and reuse them to make process faster ?

[RobinROAR]

If we fine tune ditto on single avatar image and then generate real time talking head for that avatar from audio , will the resulting model be lighter and will it give faster response ? If not , what are the best ways to achieve low latency with ditto for known /one avatar image scenario? Does ditto skip it's identity resemblance /identity extraction step if we train it directly on a single avatar image that will also be used at inference time ? Can we pre compute the identity features (and store apperance features in memory) and reuse them to make process faster ?

Fine-tuning Ditto on a single avatar will not reduce inference latency or improve response speed, because the model's parameters and architecture remain unchanged. Since the characteristic of Diffusion-based models is training-free arbitrary driving of single images, for fixed avatar scenarios, you can choose more lightweight methods such as GANs.