OpenSourceJoker/LivePortrait
3
0
Fork 0
mirror of https://github.com/KwaiVGI/LivePortrait.git synced 2024-12-22 04:12:38 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

feat: update env, doc, one-installer (#283)

Browse Source 
* chore: remove transformers for macOS

* chore: specify torch cuda version

* chore: update install instruction

* chore: update install instruction

* chore: update readme

* doc: update

---------

Co-authored-by: zzzweakman <1819489045@qq.com>
This commit is contained in:
Jianzhu Guo 2024-08-05 18:04:26 +08:00 committed by GitHub
parent 357226b2e7
commit 6a7efe2742
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
6 changed files with 61 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
assets/docs/changelog/2024-08-02.md
View File

@ -13,14 +13,14 @@
</table>
🎉 We are excited to announce the release of a new version featuring animals mode, along with several other updates. Special thanks to the dedicated efforts of the LivePortrait team. 💪
🎉 We are excited to announce the release of a new version featuring animals mode, along with several other updates. Special thanks to the dedicated efforts of the LivePortrait team. 💪 We also provided an one-click installer for Windows users, checkout the details [here](./2024-08-05.md).
### Updates on Animals mode
We are pleased to announce the release of the animals mode, which is fine-tuned on approximately 230K frames of various animals (mostly cats and dogs). The trained weights have been updated in the `liveportrait_animals` subdirectory, available on [HuggingFace](https://huggingface.co/KwaiVGI/LivePortrait/tree/main/) or [Google Drive](https://drive.google.com/drive/u/0/folders/1UtKgzKjFAOmZkhNK-OYT0caJ_w2XAnib). You should [download the weights](https://github.com/KwaiVGI/LivePortrait?tab=readme-ov-file#2-download-pretrained-weights) before running. There are two ways to run this mode.
> Please note that we have not trained the stitching and retargeting modules for the animals model due to several technical issues. _This may be addressed in future updates._ Therefore, we recommend **disabling stitching by setting the `--no_flag_stitching`** option when running the model. Additionally, `paste-back` is also not recommended.
Before launching, ensure you have installed `transformers==4.22.0`, `pillow>=10.2.0`, which are already updated in [`requirements_base.txt`](../../../requirements_base.txt). We have chosen [XPose](https://github.com/IDEA-Research/X-Pose) as the keypoints detector for animals. This relies on `transformers` and requires building an OP named `MultiScaleDeformableAttention` by
Before launching, ensure you have installed `transformers==4.22.0`, `pillow>=10.2.0`, which are already updated in [`requirements_base.txt`](../../../requirements_base.txt). We have chosen [X-Pose](https://github.com/IDEA-Research/X-Pose) as the keypoints detector for animals. This relies on `transformers` and requires building an OP named `MultiScaleDeformableAttention` by
```bash
cd src/utils/dependencies/XPose/models/UniPose/ops
python setup.py build install
@ -38,7 +38,7 @@ python app_animals.py # --server_port 8889 --server_name "0.0.0.0" --share
```
> [!WARNING]
> [XPose](https://github.com/IDEA-Research/X-Pose) is only for Non-commercial Scientific Research Purposes, you should remove and replace it with other detectors if you use it for commercial purposes.
> [X-Pose](https://github.com/IDEA-Research/X-Pose) is only for Non-commercial Scientific Research Purposes, you should remove and replace it with other detectors if you use it for commercial purposes.
### Updates on Humans mode
@ -50,6 +50,7 @@ python app_animals.py # --server_port 8889 --server_name "0.0.0.0" --share
- [**Poe supports LivePortrait**](https://poe.com/LivePortrait). Check out the news on [X](https://x.com/poe_platform/status/1816136105781256260).
- [ComfyUI-LivePortraitKJ](https://github.com/kijai/ComfyUI-LivePortraitKJ) (1.1K 🌟) now includes MediaPipe as an alternative to InsightFace, ensuring the license remains under MIT and Apache 2.0.
- [ComfyUI-AdvancedLivePortrait](https://github.com/PowerHouseMan/ComfyUI-AdvancedLivePortrait) features real-time portrait pose/expression editing and animation, and is registered with ComfyUI-Manager.

18
assets/docs/changelog/2024-08-05.md Normal file
View File

@ -0,0 +1,18 @@
## One-click Windows Installer
### Download the installer from HuggingFace
```bash
# !pip install -U "huggingface_hub[cli]"
huggingface-cli download cleardusk/LivePortrait-Windows LivePortrait-Windows-v20240805.zip
```
If you cannot access to Huggingface, you can use [hf-mirror](https://hf-mirror.com/) to download:
```bash
# !pip install -U "huggingface_hub[cli]"
export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download cleardusk/LivePortrait-Windows LivePortrait-Windows-v20240805.zip
```
Alternatively, you can manually download it from the [HuggingFace](https://huggingface.co/cleardusk/LivePortrait-Windows/blob/main/LivePortrait-Windows-v20240805.zip) page.
Then, simply unzip the package `LivePortrait-Windows-v20240805.zip` and double-click `run_windows_human.bat` for the Humans mode, or `run_windows_animal.bat` for the **Animals mode**.

40
readme.md
View File

@ -37,8 +37,8 @@
</p>
## 🔥 Updates
- `2024/08/05`: 📦 Windows users download the [one-click installer](https://huggingface.co/cleardusk/LivePortrait-Windows/blob/main/LivePortrait-Windows-v20240805.zip) for Humans mode and **Animals mode** now! For details, see [**here**](./assets/docs/changelog/2024-08-05.md).
- **`2024/08/02`**: 😸 We released a version of the **Animals model**, along with several other updates and improvements. Check out the details [**here**](./assets/docs/changelog/2024-08-02.md)!
- **`2024/07/25`**: 📦 Windows users can now download the package from [HuggingFace](https://huggingface.co/cleardusk/LivePortrait-Windows/tree/main) or [BaiduYun](https://pan.baidu.com/s/1FWsWqKe0eNfXrwjEhhCqlw?pwd=86q2). Simply unzip and double-click `run_windows.bat` to enjoy!
- **`2024/07/24`**: 🎨 We support pose editing for source portraits in the Gradio interface. We’ve also lowered the default detection threshold to increase recall. [Have fun](assets/docs/changelog/2024-07-24.md)!
@ -57,6 +57,10 @@ We are actively updating and improving this repository. If you find any bugs or
## Getting Started 🏁
### 1. Clone the code and prepare the environment 🛠️
> [!Note]
> Make sure your system has [`git`](https://git-scm.com/), [`conda`](https://anaconda.org/anaconda/conda), and [`FFmpeg`](https://ffmpeg.org/download.html) installed. For details on FFmpeg installation, see [**how to install FFmpeg**](assets/docs/how-to-install-ffmpeg.md).
```bash
git clone https://github.com/KwaiVGI/LivePortrait
cd LivePortrait
@ -64,16 +68,37 @@ cd LivePortrait
# create env using conda
conda create -n LivePortrait python=3.9
conda activate LivePortrait
```
# for Linux and Windows users
#### For Linux or Windows Users
The [X-Pose](https://github.com/IDEA-Research/X-Pose) dependency has **strict limitations** on the CUDA version. To check your current CUDA version, run the following command:
```bash
nvcc -V # example versions: 11.1, 11.8, 12.1, etc.
```
We provide installation commands for `torch` corresponding to three common CUDA versions. If your version is not listed, please visit [PyTorch Official Website](https://pytorch.org/get-started/previous-versions/) to find the installation command for your CUDA version.
```bash
# for Linux and Windows users (choose one based on your CUDA version):
# for CUDA 11.1
pip install torch==1.10.1+cu111 torchvision==0.11.2 torchaudio==0.10.1 -f https://download.pytorch.org/whl/cu111/torch_stable.html
# for CUDA 11.8
pip install torch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 --index-url https://download.pytorch.org/whl/cu118
# for CUDA 12.1
pip install torch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 --index-url https://download.pytorch.org/whl/cu121
# ...
```
Finally, install the remaining dependencies:
```bash
pip install -r requirements.txt
```
#### For macOS with Apple Silicon Users
The [X-Pose](https://github.com/IDEA-Research/X-Pose) dependency does not support macOS, so you can skip its installation. While Humans mode works as usual, Animals mode is not supported. Use the provided requirements file for macOS with Apple Silicon:
```bash
# for macOS with Apple Silicon users
pip install -r requirements_macOS.txt
```
> [!Note]
> Make sure your system has [`git`](https://git-scm.com/), [`conda`](https://anaconda.org/anaconda/conda), and [`FFmpeg`](https://ffmpeg.org/download.html) installed. For details on FFmpeg installation, see [**how to install FFmpeg**](assets/docs/how-to-install-ffmpeg.md).
### 2. Download pretrained weights 📥
The easiest way to download the pretrained weights is from HuggingFace:
@ -124,7 +149,7 @@ python inference.py -h
```
#### Fast hands-on (animals) 🐱🐶
Animals mode is ONLY tested on Linux with NVIDIA GPU.
Animals mode is ONLY tested on Linux and Windows with NVIDIA GPU.
You need to build an OP named `MultiScaleDeformableAttention` first, which is used by [X-Pose](https://github.com/IDEA-Research/X-Pose), a general keypoint detection framework.
```bash
@ -207,6 +232,7 @@ The results are [**here**](./assets/docs/speed.md).
Discover the invaluable resources contributed by our community to enhance your LivePortrait experience:
- [ComfyUI-LivePortraitKJ](https://github.com/kijai/ComfyUI-LivePortraitKJ) by [@kijai](https://github.com/kijai)
- [ComfyUI-AdvancedLivePortrait](https://github.com/PowerHouseMan/ComfyUI-AdvancedLivePortrait) by [@PowerHouseMan](https://github.com/PowerHouseMan).
- [comfyui-liveportrait](https://github.com/shadowcz007/comfyui-liveportrait) by [@shadowcz007](https://github.com/shadowcz007)
- [LivePortrait In ComfyUI](https://www.youtube.com/watch?v=aFcS31OWMjE) by [@Benji](https://www.youtube.com/@TheFutureThinker)
- [LivePortrait hands-on tutorial](https://www.youtube.com/watch?v=uyjSTAOY7yI) by [@AI Search](https://www.youtube.com/@theAIsearch)
@ -216,7 +242,7 @@ Discover the invaluable resources contributed by our community to enhance your L
And many more amazing contributions from our community!
## Acknowledgements 💐
We would like to thank the contributors of [FOMM](https://github.com/AliaksandrSiarohin/first-order-model), [Open Facevid2vid](https://github.com/zhanglonghao1992/One-Shot_Free-View_Neural_Talking_Head_Synthesis), [SPADE](https://github.com/NVlabs/SPADE), [InsightFace](https://github.com/deepinsight/insightface) repositories, for their open research and contributions.
We would like to thank the contributors of [FOMM](https://github.com/AliaksandrSiarohin/first-order-model), [Open Facevid2vid](https://github.com/zhanglonghao1992/One-Shot_Free-View_Neural_Talking_Head_Synthesis), [SPADE](https://github.com/NVlabs/SPADE), [InsightFace](https://github.com/deepinsight/insightface) and [X-Pose](https://github.com/IDEA-Research/X-Pose) repositories, for their open research and contributions.
## Citation 💖
If you find LivePortrait useful for your research, welcome to 🌟 this repo and cite our work using the following BibTeX:

2
requirements.txt
View File

@ -1,2 +1,4 @@
-r requirements_base.txt
onnxruntime-gpu==1.18.0
transformers==4.22.0

6
requirements_base.txt
View File

@ -1,6 +1,3 @@
torch==2.3.0
torchvision==0.18.0
torchaudio==2.3.0
numpy==1.26.4
pyyaml==6.0.1
opencv-python==4.10.0.84
@ -18,5 +15,4 @@ imageio-ffmpeg==0.5.1
tyro==0.8.5
gradio==4.37.1
pykalman==0.9.7
transformers==4.22.0
pillow>=10.2.0
pillow>=10.2.0

3
requirements_macOS.txt
View File

@ -1,4 +1,7 @@
-r requirements_base.txt
--extra-index-url https://download.pytorch.org/whl/cpu
torch==2.3.0
torchvision==0.18.0
torchaudio==2.3.0
onnxruntime-silicon==1.16.3