:arrow_forward: Demo

:arrow_forward: AI-generated Elon Musk

Avatarify

Photorealistic avatars for video-conferencing apps. Democratized.

Based on First Order Motion Model.

Created by: GitHub community.

News

• 11 July 2020. Added Docker support. Now you can run Avatarify from Docker on Linux. Thanks to mikaelhg and mintmaker for contribution!
• 22 May 2020. Added Google Colab mode. Now you can run Avatarify on any computer without GPU!
• 7 May 2020. Added remote GPU support for all platforms (based on mynameisfiber's solution). Demo. Deployment instructions.
• 24 April 2020. Added Windows installation tutorial.
• 17 April 2020. Created Slack community. Please join via invitation link.
• 15 April 2020. Added StyleGAN-generated avatars. Just press Q and now you drive a person that never existed. Every time you push the button – new avatar is sampled.
• 13 April 2020. Added Windows support (kudos to 9of9).

Table of Contents

• Requirements
• Install
  • Download network weights
  • Linux
  • Mac
  • Windows
  • Remote GPU
  • Docker
• Setup avatars
• Run
  • Linux
  • Mac
  • Windows
• Controls
• Driving your avatar
• Configure video meeting app
• Skype
• Zoom
• Teams
• Slack
• Uninstall
• Contribution
• FAQ
• Troubleshooting

Requirements

You can run Avatarify in two modes: locally and remotely.

To run Avatarify locally you need a CUDA-enabled (NVIDIA) video card. Otherwise it will fallback to the central processor and run very slowly. These are performance metrics for some hardware:

• GeForce GTX 1080 Ti: 33 frames per second
• GeForce GTX 1070: 15 frames per second
• GeForce GTX 950: 9 frames per second

You can also run Avatarify remotely on Google Colab (easy) or on a dedicated server with a GPU (harder). There are no special PC requirements for this mode, only a stable internet connection.

Of course, you also need a webcam!

Install

Download network weights

Download model's weights from Dropbox, Yandex.Disk or Google Drive [228 MB, md5sum 8a45a24037871c045fbb8a6a8aa95ebc]

Linux

Linux uses v4l2loopback to create virtual camera.

1. Download Miniconda Python 3.7 and install using command:

bash Miniconda3-latest-Linux-x86_64.sh

1. Clone avatarify and install its dependencies (sudo privelege is required):

git clone https://github.com/alievk/avatarify.git
cd avatarify
bash scripts/install.sh

1. Download network weights and place vox-adv-cpk.pth.tar file in the avatarify directory (don't unpack it).

Mac

(!) Note: we found out that in versions after v4.6.8 (March 23, 2020) Zoom disabled support for virtual cameras on Mac. To use Avatarify in Zoom you can choose from 2 options: - Install Zoom v4.6.8 which is the last version that supports virtual cameras - Use latest version of Zoom, but disable library validation by running this command in terminal:

codesign --remove-signature /Applications/zoom.us.app

We will use CamTwist to create virtual camera for Mac.

1. Install Miniconda Python 3.7 or use Homebrew Cask: brew cask install miniconda.
2. Download and unpack the repository or use git:

git clone https://github.com/alievk/avatarify.git
cd avatarify
bash scripts/install_mac.sh

1. Download and install CamTwist from here. It's easy.

Windows

:arrow_forward: Video tutorial

This guide is tested for Windows 10.

1. Install Miniconda Python 3.7.
2. Install Git.
3. Press Windows button and type "miniconda". Run suggested Anaconda Prompt.
4. Download and install Avatarify (please copy-paste these commands and don't change them):

git clone https://github.com/alievk/avatarify.git
cd avatarify
scripts\install_windows.bat

1. Download network weights and place vox-adv-cpk.pth.tar file in the avatarify directory (don't unpack it).
2. Run run_windows.bat. If installation was successful, two windows "cam" and "avatarify" will appear. Leave these windows open for the next installation steps.
3. Install OBS Studio for capturing Avatarify output.
4. Install VirtualCam plugin. Choose Install and register only 1 virtual camera.
5. Run OBS Studio.
6. In the Sources section, press on Add button ("+" sign), select Windows Capture and press OK. In the appeared window, choose "[python.exe]: avatarify" in Window drop-down menu and press OK. Then select Edit -> Transform -> Fit to screen.
7. In OBS Studio, go to Tools -> VirtualCam. Check AutoStart, set Buffered Frames to 0 and press Start.
8. Now OBS-Camera camera should be available in Zoom (or other videoconferencing software).

The steps 10-11 are required only once during setup.

Remote GPU

You can offload the heavy work to Google Colab or a server with a GPU and use your laptop just to communicate the video stream. The server and client software are native and dockerized available.

Docker

Docker images are only availabe on Linux.

1. Install Docker following the Documentation. Then run this step to make docker available for your user.
2. For using the gpu (hardly recommended): Install nvidia drivers and nvidia docker.
3. Clone avatarify and install its dependencies (v4l2loopback kernel module):

git clone https://github.com/alievk/avatarify.git
cd avatarify
bash scripts/install_docker.sh

1. Build the Dockerfile:

cd avatarify
docker build -t avatarify .

Setup avatars

Avatarify comes with a standard set of avatars of famous people, but you can extend this set simply copying your avatars into avatars folder.

Follow these advices for better visual quality: * Make square crop of your avatar picture. * Crop avatar's face so that it's not too close not too far. Use standard avatars as reference. * Prefer pictures with uniform background. It will diminish visual artifacts.

Run

Your web cam must be plugged-in.

Note: run your video-conferencing app only after Avatarify is started.

Linux

The run script will create virtual camera /dev/video9. You can change these settings in scripts/settings.sh.

You can use command v4l2-ctl --list-devices to list all devices in your system.

Run:

bash run.sh

If you haven't installed a GPU add the --no-gpus flag. In order to use Docker add the --docker flag.

cam and avatarify windows will pop-up. The cam window is for controlling your face position and avatarify is for the avatar animation preview. Please follow these recommendations to drive your avatars.

Mac

Note: On Mac Avatarify runs only with Google Colab or a dedicated server with GPU.

Please find where you downloaded avatarify and substitute path /path/to/avatarify below.

1. To run Avatarify please follow instructions for Google Colab or a dedicated server.
2. Go to CamTwist.
3. Choose Desktop+ and press Select.
4. In the Settings section choose Confine to Application Window and select python (avatarify) from the drop-down menu.

cam and avatarify windows will pop-up. The cam window is for controlling your face position and avatarify is for the avatar animation preview. Please follow these recommendations to drive your avatars.

Windows

1. In Anaconda Prompt:

cd C:\path\to\avatarify
run_windows.bat

1. Run OBS Studio. It should automaitcally start streaming video from Avatarify to OBS-Camera.

cam and avatarify windows will pop-up. The cam window is for controlling your face position and avatarify is for the avatar animation preview. Please follow these recommendations to drive your avatars.

Note: To reduce video latency, in OBS Studio right click on the preview window and uncheck Enable Preview.

Controls

Keys	Controls
1-9	These will immediately switch between the first 9 avatars.
Q	Turns on StyleGAN-generated avatar. Every time you push the button – new avatar is sampled.
0	Toggles avatar display on and off.
A/D	Previous/next avatar in folder.
W/S	Zoom camera in/out.
U/H/J/K	Translate camera. H - left, K - right, U - up, J - Down by 5 pixels. Add Shift to adjust by 1 pixel.
Shift-Z	Reset camera zoom and translation
Z/C	Adjust avatar target overlay opacity.
X	Reset reference frame.
F	Toggle reference frame search mode.
R	Mirror reference window.
T	Mirror output window.
L	Reload avatars.
I	Show FPS
ESC	Quit

Driving your avatar

These are the main principles for driving your avatar:

• Align your face in the camera window as closely as possible in proportion and position to the target avatar. Use zoom in/out function (W/S keys) and camera left, right, up, down translation (U/H/J/K keys). When you have aligned, hit 'X' to use this frame as reference to drive the rest of the animation
• Use the overlay function (Z/C keys) to match your and avatar's face expressions as close as possible

Alternatively, you can hit 'F' for the software to attempt to find a better reference frame itself. This will slow down the framerate, but while this is happening, you can keep moving your head around: the preview window will flash green when it finds your facial pose is a closer match to the avatar than the one it is currently using. You will see two numbers displayed as well: the first number is how closely you are currently aligned to the avatar, and the second number is how closely the reference frame is aligned.

You want to get the first number as small as possible - around 10 is usually a good alignment. When you are done, press 'F' again to exit reference frame search mode.

You don't need to be exact, and some other configurations can yield better results still, but it's usually