Live Portrait in Comfy UI similar to Hedra AI but its FREE

Live Portrait in Comfy UI similar to Hedra AI but its FREE - Talking Avatar in Local PC

Рет қаралды 25,527

Ай бұрын

Install on your local PC/laptop: LINKS
Github (for Comfy UI) - github.com/kijai/ComfyUI-Live...
My workflow with assets -openart.ai/workflows/seal_har...
Made with Stable Diffusion and Comfy UI
#hedraai #ai #comfyui
More details about Live Portrait:
LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control is an innovative approach for creating lifelike video animations from a single source image. Here are the key details:
Objective: LivePortrait synthesizes realistic videos by using a single source image as an appearance reference and deriving motion (facial expressions and head pose) from a driving video, audio, text, or other sources.
Framework: Unlike mainstream diffusion-based methods, LivePortrait leverages an implicit-keypoint-based framework. This approach balances computational efficiency and controllability.
Training Data: The model is trained on an extensive dataset of approximately 69 million high-quality frames. A mixed image-video training strategy enhances generalization.
Enhancements:
Compact Implicit Keypoints: These keypoints effectively represent blendshapes, contributing to better generation quality.
Stitching and Retargeting Modules: These modules, implemented with a small MLP, enhance controllability without significant computational overhead.
Performance: LivePortrait achieves remarkable generation speed (12.8ms on an RTX 4090 GPU with PyTorch) even when compared to diffusion-based methods.

Пікірлер: 20

@sir_wesley86 10 күн бұрын

this would be so cool for our NPC portraits in game

@Lincoln_Vail 25 күн бұрын

Looking for Leonardo to add this asap

@leandrogoethals6599 23 күн бұрын

How much vram do u require?

@AlejandroGuerrero 21 күн бұрын

Is there any tool (like this to run locally) to upload an mp3 voiceover and generate the mouth and eye movements to later use in this process? Thanks!

@AI_Made_Simple. 21 күн бұрын

Capcut

@heysoygiovanni6616 13 күн бұрын

good

@Avalon19511 22 күн бұрын

Unfortunately unlike Hedra it does not really animate the entire head, characters with short hair look better but anyone with long hair it looks odd.

@Omsip123 22 күн бұрын

Yeah, as soon as the head rotates, it gets weird. But who knows, soon this might be solved

@user-ex6pk6fh4p 26 күн бұрын

I have (IMPORT FAILED) ComfyUI-LivePortraitKJ everytime. Tried reinstall but it's still failed.

@AI_Made_Simple. 26 күн бұрын

What is the error in console?normally import failed is related to dependencies

@user-ex6pk6fh4p 26 күн бұрын

@@AI_Made_Simple. I found out that it's because I didn't install insightfacem but still i have an issue when installing it "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.40.33807\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Iinsightface/thirdparty/face3d/mesh/cython -IS:\ComfyUI_windows_portable\ComfyUI_windows_portable\python_embeded\Lib\site-packages umpy\core\include -IS:\ComfyUI_windows_portable\ComfyUI_windows_portable\python_embeded\include -IS:\ComfyUI_windows_portable\ComfyUI_windows_portable\python_embeded\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.40.33807\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" /EHsc /Tpinsightface/thirdparty/face3d/mesh/cython/mesh_core.cpp /Fobuild\temp.win-amd64-cpython-311\Release\insightface/thirdparty/face3d/mesh/cython/mesh_core.obj mesh_core.cpp insightface/thirdparty/face3d/mesh/cython/mesh_core.cpp(147): warning C4244: =: преобразование "int" в "float", возможна потеря данных insightface/thirdparty/face3d/mesh/cython/mesh_core.cpp(147): warning C4244: =: преобразование "int" в "float", возможна потеря данных insightface/thirdparty/face3d/mesh/cython/mesh_core.cpp(210): warning C4244: =: преобразование "int" в "float", возможна потеря данных insightface/thirdparty/face3d/mesh/cython/mesh_core.cpp(210): warning C4244: =: преобразование "int" в "float", возможна потеря данных insightface/thirdparty/face3d/mesh/cython/mesh_core.cpp(294): warning C4244: =: преобразование "int" в "float", возможна потеря данных insightface/thirdparty/face3d/mesh/cython/mesh_core.cpp(294): warning C4244: =: преобразование "int" в "float", возможна потеря данных "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.40.33807\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Iinsightface/thirdparty/face3d/mesh/cython -IS:\ComfyUI_windows_portable\ComfyUI_windows_portable\python_embeded\Lib\site-packages umpy\core\include -IS:\ComfyUI_windows_portable\ComfyUI_windows_portable\python_embeded\include -IS:\ComfyUI_windows_portable\ComfyUI_windows_portable\python_embeded\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.40.33807\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" /EHsc /Tpinsightface/thirdparty/face3d/mesh/cython/mesh_core_cython.cpp /Fobuild\temp.win-amd64-cpython-311\Release\insightface/thirdparty/face3d/mesh/cython/mesh_core_cython.obj mesh_core_cython.cpp insightface/thirdparty/face3d/mesh/cython/mesh_core_cython.cpp(36): fatal error C1083: Не удается открыть файл включение: Python.h: No such file or directory, error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.40.33807\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for insightface Failed to build insightface ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (insightface)

@colinfransch6054 24 күн бұрын

Thank you. Everything works great except only on the cpu. I have tried to reinstall onx-gpu within the ComfyUI env but still resorts to cpu.

@AI_Made_Simple. 19 күн бұрын

Did you try installing prebuilt version of insightface... github.com/Gourieff/comfyui-reactor-node#i-for-windows-users-if-you-still-cannot-build-insightface-for-some-reasons-or-just-dont-want-to-install-visual-studio-or-vs-c-build-tools---do-the-following

@user-ex6pk6fh4p 19 күн бұрын

@@AI_Made_Simple. thanks, i'll try it