Viso Master Ultimate Mod (Lots of Features - Updated -V8.2- Clustering+Frame Enhancer+Progress Bar Update 07/09/2025)

[FlowDownTheRiver]

Hi there everybody!
I wanted to have so many cool features,but I have seen whether the development is paused or some other forks are doing some background work. So I wanted to contribute in my own terms which is not pushing a PR, but I wanted to work freely.

So I added some features and I borrowed some from another repo.
Feature list :
V1 :

1. Batch Processing.
   This can be run manually by pressing the batch process button after you set your parameter.(enable swapping,restore faces etc.)
   Important Note here : When Running batch processing set the face similarity slider down to a value as all images have different faces,so "the first image or video detected with the face dictates" how similar the other faces in the other sources and based upon that it triggers the face detection. If the value is high this can lead to faces not getting swapped for the rest of the batch. Be aware of this.
   My previous code was for images only here: Process All Images with current settings (Mod) #94 but now it have the intelligent mode for file types.So images are processed like images,but video or gif file types are pressing the record button so to say.
2. Clipboard monitoring.
   First you set your options like selecting an embedding,source faces,enable swap etc.
   When clipboard monitoring is enabled any image or video is copied to the target file list.
3. Clear Target files after Processing.
   This automatically clears the target file list after batch processing.
   This can be used in combination with clipboard monitoring.
4. Auto process new target files .
   This gives the ability to trigger automatic batch processing.
   This can be used with clipboard monitoring,clear target files after processing.
   So for example :
   clipboard monitoring : on
   clear target files after processing : on
   auto process new target files : on
   You copy some image,video or mixed file type. This will add files to target list ,it will automatically process files with the settings you have and clear the target file list all automatically.
   Note : It has a hidden feature like queue as well. So when the program is processing files and you copy new items to clipboard,it will process the current operation first and then add new files to target list and process these new files. Pretty nice feature to have.
5. Output to target location.
   This gives the ability to ignore your static output folder and outputs the batch processing files into the directory of target's folder. So you now have the ability to "work in place" . This also can be used with all of the combinations above like clipboard monitoring etc.

Note : With these options above you can now process images or videos even "WITHOUT LOOKING" at the program interface. Just copy files and they will be automatically processed. Cool option!

6. Open output directory button.(Located next to search embeddings)
   This is an intelligent button.How? If you enable "output to target location" this will output to the target directory,and this button is intelligent to know that your last output directory is not static but changing. So it opens this dynamic output location. If "output to target location" is off,this will open your static output directory that you set from the program .

6.5) Extra little features I added is clear source and target list,multi select clearing.

Note : Above the features I have done,below the features I borrowed.

Features that I borrowed which is still being in development here : https://github.com/asdf31jsa/VisoMaster-Experimental . So thanks to the official developer .
7) Texture transfer for the faces.
8) Texture transfer masking.
9) Xseg mouth masking as a secondary Xseg.
10) Interpolation types and block shift. (not sure what these are intended to do.)
11) Auto restore ( these are not working ) Edit : It works but very subtle effect,not that impactful in my tests.
12) This repo had the bug for DFM models are not scaled correctly so I personally fixed it. All DFM models are correctly scaling to 512 now without any bugs.

@asdf31jsa Thanks to your work .
@axel-devs I have seen your work, but did not have the chance to test it. So if any of you guys want to go from here,please be my guest. Do not expect any PR from me :)

V2 :
This version has everything V1 has.
New Features :

1. Sort embeddings.
   You can now sort embeddings in 3 modes. No sorting (default sorting), A-Z (alphabetically from A to Z) , Z-A (alphabetically from Z to A). This also supports loading embeddings list from a file in any kind of order. Simply set a mode and load your file.

2. This is more of a bug fix where V1 had an issue. Sometimes the program was giving Qthread error and crashing when batch processing with clipboard monitoring + clear target files + auto process target files is on. So to reproduce that, I almost non-stop copied new files to clipboard when the program is still processing files and at some point it was crashing. Now that is fixed ! (hopefully,because I have made very long tests after the code got no more error.)

V3 :
This version has everything V2 has.

1. File operations for image files are saved as "jpg" instead of "png" as it saves huge space. (It was requested)

V4 :
This version has everything V3 has.

1. HyperSwap models support is added. Added Hyperswap256 from FaceFusionLabs #103 (thanks to @jcuzens work) but his work was not complete as the models needed to be downloaded manually. But in my implementation everything is downloaded automatically when you select the model.

Note : I have tested what I can and vs V3 there shall not be anything broken.

V5 :
This version has everything V4 has.

1. I have added 2 new options. Process all target images for each source or each embedding for the batch processing.
   (Note you can use both options enabled). The feature was requested in here : Viso Master Ultimate Mod (Lots of Features - Updated -V8.2- Clustering+Frame Enhancer+Progress Bar Update 07/09/2025) #96 (comment)
   I have done my tests with multiple source,target and embeddings for both image and video files. Hopefully nothing is broken :)
   Second note : The process bar is doing its calculation based on target image count and it doesn't multiple it with total embedding count or source image count. So it was not designed to show this kind of processing at the first place. I will not update it,as it is only cosmetics,not functionality,I really don't have time for that,sorry!)

V6 :
This may not be a proper release as some features were broken starting from the app version 1,such as face make up options. Why it was broken? When you try to implement,add new features including borrowing code from other repos things tend to break apart. So that happened,but since I personally do not use make up options that slipped from my attention. This was the most challenging coding so far as I spend around 5-6 hours and still have some problems. By knowing that...
This was brought to my attention here : #96 (comment)

This version has everything V5 has. (hopefully)...

1. Face make up features fixed.

NOTE : THIS IS THE FINAL RELEASE. I will not be pushing any more updates or take requests. There can be bugs and there are bugs I know that exists but however this is it.

I appreciate all the great people here and there for taking up the journey together...

Community Contribution Edition : (developped by @goodjobching . Thanks to his work on this)
V6.1:
This version was released in this post : #96 (comment)
His release notes are :

1. Fix: In v6, batch-saving EMBs for photos could output the same face each time (likely related to my environment or character-encoding settings, not the project’s logic). This has been fixed.

2. Directory grouping: On save, a subfolder is created under the output root based on the EMB label (auto-created if missing).
   Example: EMB named test1 → saved to output_root/test1/

3. New feature: Added “Batch Save (Current Frame)” next to single-save: applies all EMBs to the currently displayed frame and saves each (works for both videos and images). (While processing, it’s recommended to wait until the batch completes before performing other actions.)

4. Filename de-dup: Preserves the original naming while prepending YYYY-MM-DD-hh-mm-ss and appending a 6-character alphanumeric suffix to avoid overwriting across frames of the same source.

---

Here I want to add some extra notes to be clear. I have tested this mod personally but didn't test every single feature to check anything is broken or not. However I just assume it works as V6 + his work.
Here EMB = embeddings.

The difference of "Batch Save(Frame) button" from "Batch Process button" is, this button saves the current selected single image or single frame with the settings you set in the UI by doing a loop over all the embeddings while Batch Process doing the looping over all target images or videos,not a single one. Also it creates a subfolder with the name of the embeedding and saves the result inside it with a protection of adding date and a suffix for not overwriting the previous results.

It is great to see a version shared by other people especially under this post! Totally loved it.

V7 :
This version has everything V6.1 has.
According to the user complaints here in this post : #96 (comment) I decided to make this.

Based on https://github.com/starrlord/embeddingtool, I have made an Advanced Embedding Editor.
Do not get fooled I said "based on", it has really big code change and advanced features and this is implemented into Visomaster. Credits to @starrlord for making the base.

Features of "Advanced Embedding Editor" : (located next to search embeddings section in the UI)

1. You can load a single json file or multiple files to combine them together. It is intelligent enough to know if there are common embeddings it keeps 1 unique record.
2. You can use menu buttons to load or drag & drop to load single or multiple files. If you already have a file open it will present you a pop-up menu to use different modes for loading.
3. Speaking of which,you have multiple loading and saving options.
   a) Load file or Files
   b) Load additive (This will give you the option to load file but it will add embeddings from the new file to the current one)
   c) Save the whole list
   d) Save Selected
4. Sorting from "A-Z", "Z-A", original, manual. You can select embeddings drag & drop to different placements for manual sorting.
5. You can copy embeddings and paste them even across the files.
6. Lots of shortcuts including redo,undo operations.

Shortcuts :

UI :

This has taken huge time of mine,I said V6 was my last push but hey I couldn't let somebody else to make the last push with V6.1, so this is a huge update :)

V7.2 :
This version has everything V7 has.

1. Bug fixes for "view face compare", "view face mask" , "frame enhancement" , "face make up" . Video face alignment problem also fixed.
   I tried to test what I can and everything seems to work.

V8 :
This version has everything V7.2 has.

1. Bug fix for "Frame enhancers". It was outputting 1x even though the model enhancing with its native resolution. This bug is coming from way back in the first version. Now it produces the expected output.
2. Clustering Feature : This was added in version 6.1 when Batch Single Frame was introduced but it was forced.So Outputs were categorized in subfolders with embeddings name. Now it is optional. Settings -> Batch Processing -> Cluster Output by Source Name. This is also supported with other Batch Processing options enabled via Clipboard monitoring and manual batch processing with clicking batch processing buttons. In short it does the subfolder thing or not. :) It was requested in here : Viso Master Ultimate Mod (Lots of Features - Updated -V8.2- Clustering+Frame Enhancer+Progress Bar Update 07/09/2025) #96 (comment) and I also got annoyed by the fact that it was forced to have.
3. Progress bar now evaluates and makes the math for the source + embeddings + the target images/videos. So you will have a correct representation for the progress bar.

V8.2 :
This version has everything V8 has.

1. Missing feature of manual saving video with "record button" now accounts for "clustering" setting. Discovered in here : Viso Master Ultimate Mod (Lots of Features - Updated -V8.2- Clustering+Frame Enhancer+Progress Bar Update 07/09/2025) #96 (comment)

app.zip

app_v2.zip

app_V3.zip

app_v4.zip

app_v5.zip

app_v6.zip

app-6.1.zip

app_v7.zip

app_v7.2.zip

app_v8.zip

app_v8.2.zip

How to apply the mod :
Download "app" folder,backup your previous app folder. Replace this folder with the old one. (you can either delete the old one,or overwrite)

--- IMPORTANT INFORMATION for "CONDA" environment users. ---

I have investigated the error what users are getting about embeddings. The root cause of the error is your launch script :

call conda activate visomaster
call app/ui/core/convert_ui_to_py.bat <--- This line causes the error,it overwrites the modded file
SET APP_ROOT=%~dp0
SET APP_ROOT=%APP_ROOT:~0,-1%
SET DEPENDENCIES=%APP_ROOT%\dependencies
echo %DEPENDENCIES%
SET PATH=%DEPENDENCIES%;%PATH%
python main.py
pause

Instead do this. Make a copy of the start.bat file and remove that line,save the bat file and now copy the modded "app" directory in the main folder of the app(note if you already copied but launched the bat script,it overwrites the modified file,so you need to re-copy the modded "app" folder again) :

call conda activate visomaster
SET APP_ROOT=%~dp0
SET APP_ROOT=%APP_ROOT:~0,-1%
SET DEPENDENCIES=%APP_ROOT%\dependencies
echo %DEPENDENCIES%
SET PATH=%DEPENDENCIES%;%PATH%
python main.py
pause

Short instruction :

1. Modify your start.bat file and remove "call app/ui/core/convert_ui_to_py.bat" line
2. Copy the mod file to the main folder.
3. Start visomaster with your "new" start.bat file.

And All credits goes to official developers and contributers...

Future Ideas section :

Not because I will work on these but...

Gender-age detection.

• Full scanning all embeddings for gender detection and tag(assign) embeddings with gender. (these will not be scanned next time to save time). Right click option to change tags to fix auto detection errors.

• Auto assign selected male and female embeddings across all frames.

• Search embeddings options to limit batch process. for example you search for embbedings to limit your list and batch processing will account for only these faces. search can be modified to have comma for multi search items to widen the range here as well.

• Ah and right click option to rename embeddings.Sometimes we make typo errors. (done with advanced embedding editor)

• Flagging embeddings to be excluded for batch processing shall be nice as well.

• Speaker diarization,audio driven lips with expressions. Here different facial expressions can be saved as presets or assigned to letters,so lips can be moved by matching the talk. Easier said than done.

These are concepts if somebody wants to work on these...I am not stuck about ideas,but I don't have time to work on. I write these for those who wants to work on the project to push it further and to guide...

[JinJomes]

I'd suggest you create a fork instead of just zipping up your app folder (could work as a download over on the discord, maybe).

Gotta say though - this is a bit on-the-nose:

10. Interpolation types and block shift. (not sure what these are intended to do.)
11. Auto restore ( these are not working )

Both are working - and explained pretty clearly - on their original repo. If you're going to pad out your file admin mod with someone else's features, it's not a good look to imply they don't work. Common courtesy, n'that.

[FlowDownTheRiver]

I'd suggest you create a fork instead of just zipping up your app folder (could work as a download over on the discord, maybe).

Gotta say though - this is a bit on-the-nose:

10. Interpolation types and block shift. (not sure what these are intended to do.)
11. Auto restore ( these are not working )

Both are working - and explained pretty clearly - on their original repo. If you're going to pad out your file admin mod with someone else's features, it's not a good look to imply they don't work. Common courtesy, n'that.

And this is why I don't want to clone and push PR. When you do a lot of changes and check changes from other repos, you get responsible and people expect to do more stuff from you. I just want to put what I have done and what others have done which works BTW without taking responsibility to deal with developer requests etc. This is intended for people to try out, not for proper way of development which I am aware of. Also about the experimental build repo, I already read what was written there. Anyway, I did my part with all do respect to other devs. Feel free to use it or not...

[axel-devs]

The other user suggested you make a fork-

I'd suggest you create a fork instead of just zipping up your app folder (could work as a download over on the discord, maybe).

But you replied with another explanation for why you don't want to make a pull request-

And this is why I don't want to clone and push PR. When you do a lot of changes and check changes from other repos, you get responsible and people expect to do more stuff from you.

They were very specific with their recommendation, and even included a tip about the Discord. The only reason I (and likely most people) won't be using this is purely because it's not in a fork. I do not believe you've added any malicious code, but for security minded users or anyone wanting an easier view of your changes—forking the project is the best option. It's significantly better for seeing an overview of what to expect with your mod outside of your own claims. Transparency is extremely important for most users. Creating this post as a GitHub Issue certainly doesn't help either.

Still, this part is totally fair-

I just want to put what I have done and what others have done which works BTW without taking responsibility to deal with developer requests etc. This is intended for people to try out, not for proper way of development which I am aware of.

I get what you're saying, but creating a fork to publish your changes does not mean you owe anyone anything. You could push the fork then never touch it again, and nobody would care lol. I don't say that to be harsh or criticize, it's just the reality of sharing smaller projects like your mod on GitHub; if you don't care no one else will. Also, just because you make a fork does not mean you have to also make a PR for it.

Regarding the actual mod, if you wanted this to be "Ultimate", you would definitely need to include my Job Manager or something equal/better. Not to fluff my own work, but it's essentially a requirement for a ton of users. I personally do not have a problem with you copying/pasting any of my code from my mod over to yours, but others might feel differently.

I think another huge reason you should make a fork is because you've included someone else's code. Idk, just a bit odd to re-distribute that in a random .zip under a GitHub Issue. Either way, cool to see more people finding an interest in working on VisoMaster.

[FlowDownTheRiver]

The other user suggested you make a fork-

I'd suggest you create a fork instead of just zipping up your app folder (could work as a download over on the discord, maybe).

But you replied with another explanation for why you don't want to make a pull request-

And this is why I don't want to clone and push PR. When you do a lot of changes and check changes from other repos, you get responsible and people expect to do more stuff from you.

They were very specific with their recommendation, and even included a tip about the Discord. The only reason I (and likely most people) won't be using this is purely because it's not in a fork. I do not believe you've added any malicious code, but for security minded users or anyone wanting an easier view of your changes—forking the project is the best option. It's significantly better for seeing an overview of what to expect with your mod outside of your own claims. Transparency is extremely important for most users. Creating this post as a GitHub Issue certainly doesn't help either.

Still, this part is totally fair-

I just want to put what I have done and what others have done which works BTW without taking responsibility to deal with developer requests etc. This is intended for people to try out, not for proper way of development which I am aware of.

I get what you're saying, but creating a fork to publish your changes does not mean you owe anyone anything. You could push the fork then never touch it again, and nobody would care lol. I don't say that to be harsh or criticize, it's just the reality of sharing smaller projects like your mod on GitHub; if you don't care no one else will. Also, just because you make a fork does not mean you have to also make a PR for it.

Regarding the actual mod, if you wanted this to be "Ultimate", you would definitely need to include my Job Manager or something equal/better. Not to fluff my own work, but it's essentially a requirement for a ton of users. I personally do not have a problem with you copying/pasting any of my code from my mod over to yours, but others might feel differently.

I think another huge reason you should make a fork is because you've included someone else's code. Idk, just a bit odd to re-distribute that in a random .zip under a GitHub Issue. Either way, cool to see more people finding an interest in working on VisoMaster.

These are great advises and proper way of doing it. I am not using Discord for around 2 years and I rarely login there. So I am not aware about specific instructions. No,there is no malicious code in it. I did it for myself but for the sake of sharing,in whatever form it is,I just wanted to contribute to people who has done so much work already and add some more features. Obviously "Ultimate" title is for people not to miss the mod to distinguish a bit better from other issues,not because it is mega,super,crazy damn good etc. :) But I love these features that I added...

I get what you saying and you are right. However for the sake of delivering stuff I ignored the common path.

Just as an example : I made another mod for another repo here on the issues tab : lllyasviel/FramePack#138 and tried to help people and it served its purpose. As long as people are happy with it, I am glad to be helpful .

So maybe in the future I follow the community guidelines better. Again thanks for your advises.

[Beeboy-1234]

Could you please update the new models? I'm using version 0.1.6 and I still have problems with people's faces. When I see only half of their faces, they still look like jelly. Problems with faces looking up, looking down, looking left, looking right, the new face doesn't fit perfectly with the old face. I saw news that there's a new model called Hyperswap_256. I don't know if it will help. Please update it so that it can be used with Visomaster 0.1.6. I would really appreciate it. Right now, I still have to use an editing program to help fix the problematic videos, which takes a lot of time. Please help.

[FlowDownTheRiver]

Could you please update the new models? I'm using version 0.1.6 and I still have problems with people's faces. When I see only half of their faces, they still look like jelly. Problems with faces looking up, looking down, looking left, looking right, the new face doesn't fit perfectly with the old face. I saw news that there's a new model called Hyperswap_256. I don't know if it will help. Please update it so that it can be used with Visomaster 0.1.6. I would really appreciate it. Right now, I still have to use an editing program to help fix the problematic videos, which takes a lot of time. Please help.

It doesn't have to do with the new model. What you should do is to determine your settings better. For example your face detection model makes a change, enable edit on the main page makes a change, face restoration parameters for eyes,mouth also plays a role. Also under face detection model there is face landmarks models option ,so you can enable different landmark models to see the effect. On the main page enable face adjustment mode to scale the landmarks to fit better or not. So there are lots of stuff that plays role. Unfortunately there isn't a magical setting to work with all scenes and all videos. If you are using the mod I posted,you also have texture recovery with masking options. While this can be good at some scenes,sometimes it messes up if the faces moves around. So I would suggest disabling this for videos mostly as it is not predictable with some other settings.

Besides the suggestions, I already worked on Hyperswap models to be implemented on Visomaster. Model loading,menus,download was ok until I hit a road block about the architecture compatibility. It was giving error with the keys. So after spending 8 hours or so I have given up 2 different implementations and deleted everything. I rage quit. So I am not sure if I try that again. No promises,no expectations.

[Beeboy-1234]

I have tried adjusting all the values ​​in the program, whether it is Swapper model, face mask, Occlusion Mask, DFL Xseg Mask, Restore type, landmark detect model. I have tried adjusting all these values. I have been trying it for about 6-7 months, but I still have a problem with the jelly face. If a person's face is only half visible, or when looking down, looking up, turning left or right, I still have a problem. So I think the cause might be that the model in the program may not be trained enough. There should be an update to new models that fix the defects in creating accurate face replacement. Another problem is the mouth. When using a spoon to eat food, the mouth is messy. We have to use a face mask, but it will not make the mouth look realistic like the image we want. We have to use the mouth of the original clip. It cannot imitate the mouth of the image we want to use. Currently, I have to use Premiere Pro to help cut and fix the problematic parts. Normally, the video clip used is about 10 minutes, which wastes time fixing the defects that the Visomaster 0.1.6 program has not fixed. Waiting for the developer of the new model training program should help make it better. Since you have made improvements, I wonder Have you encountered the same problem as me? And how did you solve it to minimize it or not solve it at all? Because it will waste a lot of time because the frame rate will be high for a 10 minute clip. So I would like you to recommend or update a new model. Which I think it should be better trained than the old model. Please help me. Thank you.

[JinJomes]

Hyperswap models got converted over to Viso couple few days back over on the discord. Pretty average, tbh.

[FlowDownTheRiver]

I have tried adjusting all the values ​​in the program, whether it is Swapper model, face mask, Occlusion Mask, DFL Xseg Mask, Restore type, landmark detect model. I have tried adjusting all these values. I have been trying it for about 6-7 months, but I still have a problem with the jelly face. If a person's face is only half visible, or when looking down, looking up, turning left or right, I still have a problem. So I think the cause might be that the model in the program may not be trained enough. There should be an update to new models that fix the defects in creating accurate face replacement. Another problem is the mouth. When using a spoon to eat food, the mouth is messy. We have to use a face mask, but it will not make the mouth look realistic like the image we want. We have to use the mouth of the original clip. It cannot imitate the mouth of the image we want to use. Currently, I have to use Premiere Pro to help cut and fix the problematic parts. Normally, the video clip used is about 10 minutes, which wastes time fixing the defects that the Visomaster 0.1.6 program has not fixed. Waiting for the developer of the new model training program should help make it better. Since you have made improvements, I wonder Have you encountered the same problem as me? And how did you solve it to minimize it or not solve it at all? Because it will waste a lot of time because the frame rate will be high for a 10 minute clip. So I would like you to recommend or update a new model. Which I think it should be better trained than the old model. Please help me. Thank you.

I totally understand your frustration on this one. Don't get me wrong, but I have made an educated guess about Hyperswap models will not fix the problems that you are facing. Why? Infact you are right about lack of training data,but also angles. Lots of models were trained to a certain angle because the landmark detection and facial features are correctly placed until it reaches a certain point of angle. The training datasets are diverse but lacking angles . So because of these limitations and they do not want to pollute the datasets with not correctly assigned facial points and they may not want to use from above or from below data as much. Half face in your case becomes "Jelly" as model tries to put the landmarks to an uncertain point and it may skew from frame to frame which causes that problem. You are on point but that was the reason I commented "It doesn't have to do with the new model." . I hardly believe unless the developers of the models for different architectures overcomes these issues, they will be like what you want them to be.

So what can we do? I made those suggestions in my previous comment because I didn't know if you tried those settings which shall help eliminate some of the problems but not all. In other words I didn't know how experienced you are on the settings or options you are given. Even if you try all you may not get what you want because of the limitations of the tech limitations I tried to explain above.

You can examine the cause of the problem on the "Jelly" face by enabling landmark detection and render the landmarks over the face if they move to an absurd placement where the face becomes "Jelly" . Let's say you can move the points manually but does the model is able to generate correct representation of the face with the corrected landmarks? That is the question we need to investigate.

So in summary, I hardly believe Hyperswap models will handle these problems but will see.

Note : I didn't try these models as of now.

[Beeboy-1234]

This means that AI programs cannot fix the shortcomings we encounter, even if it is an AI program in the future, right? No matter how many rounds of training, right? The values ​​that I use to fix it, for now, I set it as a basic standard to solve the problems that I told you in the previous post: In the Swapper Model, I use Instylewwapper256VersionB. This part is that I have tried other models in the visomaster0.1.6 program. This one should be the best. Then turn on Occlusion Mask to fix the mask joints. Turn on XSeg Mask. Next, in the Restore Type section, select GPEN-256, which should be the best out of all the ones I've tried. Then turn on Enable Face Expression Restorer, turn on Noremalize Lips, turn on Retargeting Lips. The last part is the Settings category, set providers Priority to tensorRT-Engine, turn on Auto Rotation, turn on Enable Landmark Detection, and select Landmark Detect Model as 3d68. Use all of these to faceswap all clips. Then look at all the clips to see which parts have problems, such as the scene where a spoon is used to eat rice. Then change the Bottom Border value to about 50-60 to fix the mouth and spoon, but it will only replace the mouth of the original clip. It will not be the mouth from the picture that we want to put in, making the mouth and spoon when eating not the mouth of the real person in the picture. This makes the face change and not look like the person we used in the picture. This is the problem. The face is not the same as the person we put in the picture. The next part is overlapping images or giant snake faces, similar to jelly or ghost faces, or looking up and down, turning left and right. If I encounter problems in some scenes, I have to adjust the Landmark Detect Model value to choose 68, 98, or 106. Mostly, it will be around this value, but it can fix some of it, not all of it. It wastes time cutting out damaged clips. All of the Premiere Pro programs that I have mentioned are the ones that I have tried to adjust while using this program since October 2024 until now, 29/9/2025. I still encounter the problems that I have told you about. So I hope that the development of new AI model training will help fix these shortcomings. This is my own understanding that Detailed model training makes AI smarter and can fix these flaws. In my settings, is there anything you think can be improved? Please advise. Thank you.

[FlowDownTheRiver]

About face restorers I can say,it really depends on the face size of the source image on the scene. Outside of the context of Visomaster, when gpen was first introduced in the official repo I have made my own tests and it was lacking tiled processing while gfpgan was supporting it which was giving error. So I contacted the official developer and he implemented that. There was also masking problems with the bounding box and I tested with different values and those reports I provided was implemented in the official repo. I love those kind of developers which respects people's opinion who wants to contribute input. Long story short I have a very long history about testing these face restoration models gfpgan,gpen,codeformer and so on.

So by taking this as a background and from my experiences over the years, Gpen tries to preserve the identity while trying to restore the face but vs Gfpgan its restoration capabilities lacks behind if the face in the original source is very low quality or very low resolution. So Gfpgan is more creative and restores better buts lacks the identity preservation,Gpen on the other hand if there is enough information like face is definitive enough preserves identity and does its job. Codeformer on the other hand has a "w" value which is not exposed in most of the apps goes in between these. So identity preservation or more restoration is represented with this value.

This leads to the question of what to use and when?

We have multi restoration mode.
So for the first model it depends on your preference and about the condition of the source image(resolution etc). Besides the resolution in context of face swapping inswapper model goes with 128 native resolution when doing the swapping so it is pretty pixelated outside of the fact that of the original source image. So if the source face is bigger it gets upscaled with these pixelated data and this leads to the problem of enlarged pixels goes to face restoration. This may sound complicated but this is a priority issue. The developer needs to see this whether to upscale the face before getting enlarged or not. So this is pretty much out of the scope of this conversation but I just wanted to give a bit of taste how tricky things shall go on the development process.

Anyway if you chose what looks the best with your first face restoration model you can add a secondary restorer in the form of
Gpen 1024,or gpen 2048 models. Gpen 1024 adds details,sharpness to the already restored face. Note that be careful about this one. If the resolution of the restored face is too low you will not benefit from this model. The results will suffer as the face resolution is not fitting the model's resolution of 1024 pixels. So in this case 512 + face resolutions shall be candidate for Gpen 1024 as they shall be definitive enough to add details.

2048 model on the other hand doesn't add as much sharpness but it adds clarity and crispness. It adds a bit of saturation. You can observe this if you use this model on black and white photos faces gets colored a bit because of training data. So you can use this model to have the absolute final touch. Ofc you need to have enough resolution for the faces.

Note that I don't write detailed information like this anywhere but you asked very nicely I wanted to inform a bit so maybe other people shall benefit as well. Beyond this the program is large enough and I can't possibly cover all use cases or answer all of the raising questions as it can go pretty wild and long...

Anyway I hope this gives you some insights and use cases... I know these as I am standing here as an end user asking for features for so many years and got so many developer friends and I got pretty nice returns as results with open minded developers. So if I hit a hard wall like some "devs" are very strict with the rules and they do not want to listen, I develop my own software with the features I want. Anyway I am on the both side,as an end user and as a developer with enough understanding about the stuff I would say. So I understand you pretty good.

Hopefully this message will be useful to others as well.

[Beeboy-1234]

The original face that I use When I get the image, such as the capture, the file size may not be large. I will use the Topaz photo AI pro program to upscale it 4 times and edit the image to be beautiful from the program. It can be called a clear and beautiful image. I don't know if I used this method correctly. Then I took the original image that I got and used it with the video clip that I want to change the face. In the Face Restorer section, I turned it on and changed to GPEN-256 because when compared to the original value that the program gave, which is GFPGAN-v1.4, I think the image in the clip that came out of GPEN-256 will be better, clearer, and the face can be seen more clearly. But there is a slight disadvantage that it makes the lips more colorful, such as red lips, they will be too red. But other programs such as CodeFormer, I tried it and GPEN-256 looked better from testing it in many clips. As for GPEN-512, GPEN-1024, GPEN-2048, I tried it and I still think that the image is no different when looking at it from the screen. Plus, it is slow and takes a long time to process, so I decided to use GPEN-256. It looks the best. I don't know if taking the original image that was captured, maybe not very large, such as 128 or 256, then upscaling and editing the image with the Topaz photo AI program, and then using the original image with the clip, will it affect the face swapping? And is what I did right? Please advise. As for the previous part, I wrote the standard settings that I tried many times until it became my standard every time I swapped faces. Is there anything that needs to be adjusted or fixed? Please advise. I will be able to adjust and fix it as a standard when I swap faces to get the best results. Use it as a standard so that I don't have to fix it often with clips that often have problems with looking down, looking up, turning left and right, or eating rice using a spoon and mouth that has to be close to the spoon while eating, things like that. Please advise. Thank you very much.

[FlowDownTheRiver]

Unfortunately you are missing something that I haven't written before. When you generate videos with face swapping with all of these models there is this thing called frame coherence. Which means frame consistency for the previous frame to the next frame.

So there is a limitation. None of these models know the next frame it will not adjust itself for the upcoming data. This means each frame is calculated without knowing what will happen next or what has happened before. This will get you flickering,not consistently aligned eyeballs and stuff. This also means since each frame is calculated independently ignoring motion blur,turning faces whatever you will never get perfect results.

To achieve what you are asking for there needs to be a frame coherent face swapping model with all of the extreme angles in the dataset near perfectly trained. So this does not exist.

The other thing is we do the masking over the swapped model . So swapping model doesn't know how to occlude objects. That is why you are having trouble with masking. In other words your success is depending of the masking models' capabilities whether it is xseg,bisenet segmentation masking etc. So if you want to have the original swapped mouth act there and you still want to have the spoon you are out of your luck with the current implementation or face swapping model capabilities which doesn't have an inbuilt occlude capabilities.

Long story short,you are asking for advises which the tech currently has not reached yet.

Your best bet can be manual workflow in post editing software. You take 2 passes ,one with masking applied,one with masking not applied. Create a difference mask there and blend. This will take huge time and professional workflow. It can be achieved but I am not here to tell you how you can do it step by step.

You can work with After Effects,Premiere Pro, Davinci Resolve etc.

Anyway,I will not write anything else or do not ask anymore questions,as this is not the right place.

Hope you find a way and understand what is going behind the scenes. Note that I haven't included the parts I already answered related to your questions as some of them were already discussed. No more questions are taken or will be answered as this is going out of scope.