## 2024/08/02
Animals Singing Dance Monkey 🎤
🎉 We are excited to announce the release of a new version featuring animals mode, along with several other updates. Special thanks to the dedicated efforts of the LivePortrait team. 💪 ### Updates on Animals mode We are pleased to announce the release of the animals mode, which is fine-tuned on approximately 230K frames of various animals (mostly cats and dogs). The trained weights have been updated in the `liveportrait_animals` subdirectory, available on [HuggingFace](https://huggingface.co/KwaiVGI/LivePortrait/tree/main/) or [Google Drive](https://drive.google.com/drive/u/0/folders/1UtKgzKjFAOmZkhNK-OYT0caJ_w2XAnib). You should [download the weights](https://github.com/KwaiVGI/LivePortrait?tab=readme-ov-file#2-download-pretrained-weights) before running. There are two ways to run this mode. > Please note that we have not trained the stitching and retargeting modules for the animals model due to several technical issues. _This may be addressed in future updates._ Therefore, we recommend **disabling stitching by setting the `--no_flag_stitching`** option when running the model. Additionally, `paste-back` is also not recommended. Before launching, ensure you have installed `transformers==4.22.0`, `pillow>=10.2.0`, which are already updated in [`requirements_base.txt`](../../../requirements_base.txt). We have chosen [XPose](https://github.com/IDEA-Research/X-Pose) as the keypoints detector for animals. This relies on `transformers` and requires building an OP named `MultiScaleDeformableAttention` by ```bash cd src/utils/dependencies/XPose/models/UniPose/ops python setup.py build install cd - # equal to cd ../../../../../../../ ``` You can run the model using the script `inference_animals.py`: ```bash python inference_animals.py -s assets/examples/source/s39.jpg -d assets/examples/driving/wink.pkl --no_flag_stitching --driving_multiplier 1.75 ``` Alternatively, we recommend using Gradio. Simply launch it by running: ```bash python app_animals.py # --server_port 8889 --server_name "0.0.0.0" --share ``` > [!WARNING] > [XPose](https://github.com/IDEA-Research/X-Pose) is only for Non-commercial Scientific Research Purposes, you should remove and replace it with other detectors if you use it for commercial purposes. ### Updates on Humans Mode - **Driving Options**: We have introduced an `expression-friendly` driving option to **reduce head wobbling**, now set as the default. While it may be less effective with large head poses, you can also select the `pose-friendly` option, which is the same as the previous version. This can be set using `--driving_option` or selected in the Gradio interface. Additionally, we added a `--driving_multiplier` option to adjust driving intensity, with a default value of 1, which can also be set in the Gradio interface. - **Retargeting Video in Gradio**: We have implemented a video retargeting feature. You can specify a `target lip-open ratio` to adjust the mouth movement in the source video. For instance, setting it to 0 will close the mouth in the source video 🤐. ### Others - [**Poe supports LivePortrait**](https://poe.com/LivePortrait). Check out the news on [X](https://x.com/poe_platform/status/1816136105781256260). - [ComfyUI-LivePortraitKJ](https://github.com/kijai/ComfyUI-LivePortraitKJ) (1.1K 🌟) now includes MediaPipe as an alternative to InsightFace, ensuring the license remains under MIT and Apache 2.0. **Below are some screenshots of the new features and improvements:** | ![The Gradio Interface of Animals Mode](../animals-mode-gradio-2024-08-02.jpg) | |:---:| | **The Gradio Interface of Animals Mode** | | ![Driving Options and Multiplier](../driving-option-multiplier-2024-08-02.jpg) | |:---:| | **Driving Options and Multiplier** | | ![The Feature of Retargeting Video](../retargeting-video-2024-08-02.jpg) | |:---:| | **The Feature of Retargeting Video** |