Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to continue encoding even when some frames do not have a face (enhancement) #18

Open
Barnacules opened this issue Aug 22, 2023 · 0 comments

Comments

@Barnacules
Copy link

I would like to request a switch that allows the program to continue even with videos that have some frames where a face isn't detected. It should just continue with the .WAV playback as usual when nobody is shown on screen and then sync the lips back as they jump cut back from another scene. This would save form having to cut up videos into multiple little chunks to run them and then re-edit them back together.

Current Behavior
If the passed in video file to interfence_realersgan_video.py contains any frames without a face in them, it will fail to continue.

Wanted Behavior
Continue with a warning to the user that some frames a face wasn't' detected and the audio will just continue to play during those segments without any lip syncing until a face is detected again.

Implementation
You could add a switch --ignore-missing-faces to allow this behavior while preserving the existing behavior of just failing outright if no faces are detected in a single frame in the target video

I think this change should be somewhat easy to make since you would just have to add another parameter to the queue, you're putting the extracted frames into indicating if it should be fed to the lip sync algorithm or if it should just be placed in the final encoded video untouched. If things like ESRGAN are being utilized to upscale the video or smooth thing then just apply those to the frames without doing the face detection or altering the image with the new lip positions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant