Changelog:
- add dw pose estimator from https://github.com/IDEA-Research/DWPose
- add onnxruntime-gpu install, (update env for dw_pose)
- add dw_pose model downloader
- add controlnet preview, kudos to #kytr.ai, idea - https://discord.com/channels/973802253204996116/1124468917671309352
- add temporalnet sdxl - v1 (3-channel)
- add prores thanks to #sozeditit https://discord.com/channels/973802253204996116/1149027955998195742
- make width_height accept 1 number to resize the frame to that size keeping the aspect ratio
- add cc masked template for content-aware scheduling
- add reverse frames extraction
- move looped image to video init cell as video source mode
- fix settings not being loaded via the button
- fix bug when cc_masked_diffusion == 0
- add a message on missing audio during video export / mute exception
- go back to root dir after running rife
- add gdown install
- add deflicker for win
- add experimental deflicker from https://video.stackexchange.com/questions/23384/remove-flickering-due-to-artificial-light-with-ffmpeg
- fix linear and none blend modes for video export
- detect init_video fps and pass it down to video export with respect to the nth frame
- do not reload already loaded controlnets
- rename upscaler model path variable
- make mask use image folder correctly as a mask source
- fix gui for non controlnet-modes (getattr error in gui)
- fix video_out not defined
- fix dwpose 'final_boxes' error for frames with no people
- fix xformers version
- fix flow preview error for less than 10 frames
- fix pillow errors (UnidentifiedImageError: cannot identify image file)
- fix timm import error (isDirectory error)
- deprecate v2_depth model (use depth controlnet instead)
- fix pytorch dependencies error
- fix zoe depth error
- move installers to github repo
DW Pose
to select dw_pose, go to gui -> controlnet -> pose_detector -> dw_pose
Download new install.bat or install via !pip install onnxruntime-gpu gdown manually
Width_height max size
You can now specify a single number as width_height settings, and it will define the max size of the frame, fitting the output inside that size, keeping the aspect ratio. For example, if you have a 1920x1080 video, width_height=1280 will downscale that video to 1280x720
Controlnet Preview
Added controlnet annotations (detections) preview. Thanks to #kytr.ai for the idea.
To enable, check only_preview_controlnet in Do the Run cell. It will take the 1st frame in frame_range from your video and generate controlnet previews for it.
Prores
add prores codec thanks to #sozeditit https://discord.com/channels/973802253204996116/1149027955998195742
Create video cell -> output_format -> prores_mov
It has better quality than h264_mp4, and a smaller size than qtrle_mov
Video Init Update
Moved looped image init to video init settings. To use init image as video init, select video_source -> looped_init_image
Added reverse option
Extract frames in reverse. For example, if a person moves into the frame, it's easier to reverse the video so that the video starts with the person inside of the frame.
Detect fps
fps will now be detected from your init_video, divided by extract nth frame value and used in your video export if, you set your video export fps to -1.
Example: your video has 24fps. you extract every 2nd frame. The suggested output fps will be 24/2 = 12fps. If you set your video export fps to -1, it will use predicted fps = 12.