欢迎访问我的网站,希望内容对您有用,感兴趣的可以加入免费知识星球。

嘴型融合模型SadTalker

人工智能 迷途小书童 1年前 (2023-04-01) 1647次浏览 2个评论

环境

  • windows 10 64bit
  • SadTalker
  • torch 1.12.1+cu113

视频看这里

Youtube

Bilibili

安装配置

创建一个全新的 python 虚拟环境

conda create -n sadtalker python=3.8
conda activate sadtalker

然后,拉取源代码,并且安装对应的依赖

git clone https://github.com/Winfredy/SadTalker.git
cd SadTalker

# 安装gpu版的torch
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113

# 安装dlib-bin,它比dlib更快
pip install dlib-bin

# 安装其它依赖
pip install -r requirements.txt 

# 安装gfpgan,如果不使用gfpgan增强,可以不安装
git clone https://github.com/TencentARC/GFPGAN.git gfpgan_src
cd gfpgan_src
python setup.py install

测试

找一段音频 test.mp3 和视频 test.mp4,进行测试

python.exe .\inference.py --driven_audio test.mp3 --source_image test.mp4 --result_dir outputs --enhancer RestoreFormer

其中,--source_image 参数可以是视频,也可以是图片, --result_dir 参数指定最后合成后的视频存放位置,--enhancer 指定视频增强的方法,可以是 gpfgan 或是 RestoreFormer

如果你的显卡够好,可以调整 --batch_size 参数,默认值是2,值越大,处理的越快。

还有部分参数,可以根据自己的实际情况添加。

FAQ

Traceback (most recent call last):
  File ".\inference.py", line 110, in <module>
    main(args)
  File ".\inference.py", line 71, in main
    batch = get_data(first_coeff_path, audio_path, device)
  File "D:\BaiduNetdiskDownload\SadTalker\SadTalker\generate_batch.py", line 68, in get_data
    orig_mel = audio.melspectrogram(wav).T
  File "D:\BaiduNetdiskDownload\SadTalker\SadTalker\utils\audio.py", line 47, in melspectrogram
    S = _amp_to_db(_linear_to_mel(np.abs(D))) - hp.ref_level_db
  File "D:\BaiduNetdiskDownload\SadTalker\SadTalker\utils\audio.py", line 95, in _linear_to_mel
    _mel_basis = _build_mel_basis()
  File "D:\BaiduNetdiskDownload\SadTalker\SadTalker\utils\audio.py", line 100, in _build_mel_basis
    return librosa.filters.mel(hp.sample_rate, hp.n_fft, n_mels=hp.num_mels,
TypeError: mel() takes 0 positional arguments but 2 positional arguments (and 3 keyword-only arguments) were given

这是由于 librosa 的版本高了,按照 requirements.txt,安装 0.6.0 版本即可

下载地址

包括了源码和模型

链接:https://pan.quark.cn/s/076390b82acb

参考资料

喜欢 (0)

您必须 登录 才能发表评论!

(2)个小伙伴在吐槽
  1. 你好,我的电脑配置是: 腾讯云 NVIDIA GV100GL [Tesla V100 SXM2 32GB] Ubuntu 22.04.3 LTS Python 3.8.17 我按照官方文档 https://github.com/OpenTalker/SadTalker#linux 尝试安装SadTalker,遇到这一步 pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113 时报错: ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. accelerate 0.21.0 requires pyyaml, which is not installed. aiohttp 3.8.4 requires attrs>=17.3.0, which is not installed. altair 5.0.1 requires jinja2, which is not installed. altair 5.0.1 requires jsonschema>=3.0, which is not installed. datasets 2.13.0 requires pyyaml>=5.1, which is not installed. gradio 3.40.1 requires jinja2<4.0, which is not installed. gradio 3.40.1 requires pyyaml=5.0, which is not installed. huggingface-hub 0.16.4 requires pyyaml>=5.1, which is not installed. modelscope 1.8.1 requires attrs, which is not installed. modelscope 1.8.1 requires pyyaml, which is not installed. modelscope 1.8.1 requires simplejson>=3.3.0, which is not installed. oss2 2.18.1 requires six, which is not installed. peft 0.4.0 requires pyyaml, which is not installed. pydeck 0.8.1b0 requires jinja2>=2.10.1, which is not installed. streamlit 1.24.1 requires blinker=1.0.0, which is not installed. streamlit 1.24.1 requires click=7.0, which is not installed. transformers 4.31.0 requires pyyaml>=5.1, which is not installed. uvicorn 0.23.2 requires click>=7.0, which is not installed. modelscope 1.8.1 requires numpy=1.13.0, but you have torch 1.12.1+cu113 which is incompatible. streamlit 1.24.1 requires pillow=6.2.0, but you have pillow 10.0.0 which is incompatible. xformers 0.0.20 requires torch==2.0.1, but you have torch 1.12.1+cu113 which is incompatible. 请问应该如何解决呢?谢谢!
    snakeninny2023-08-23 10:53