Creating eye-tracking with Blender’s CATS plugin
Learn how to add eye-tracking to your 3D avatars
The mission
If you think having blendshapes and visemes makes your avatar come alive, wait until you see how cool eye tracking looks. Set it up easily in Blender with the phenomenal CATS plugin.
Resources
- Blender2.8x or newer
- Maya or your preferred 3D software
- The latest compatible Unity version with VRChat Avatar SDK
Installing the plugin
First things first, let's start by installing the plugin. I'll provide a brief explanation to ensure a smooth process.
It's quite simple: download the latest version of the CATS plugin using this GitHub link:
https://github.com/GiveMeAllYourCats/cats-blender-plugin
You'll receive a ZIP file. No need to extract its contents; just relocate the file to a folder that's more organized than your Downloads.
Now, let's fire up Blender.
Click on the "Edit" button located at the top left of the screen, then select "Preferences”.
Within “Preferences”, navigate to “Add-ons”. You'll notice an "Install" button positioned at the top right of the window. Click on it and then locate the ZIP file we downloaded earlier.
Give it a few moments; you'll soon observe that you've successfully added the plugin.
Before closing the window, remember to check and activate the plugin.
Requirements and setting up the bones
Right now, you should have your avatar rigged and ready. Remember that eye tracking is completely optional and you should have prepared your avatar for it before hand.
You can get creative with the uses, but for me, I separated the pupils for the actual eyes of the character. It gives a really good overall result.
Like I mentioned, is your avatar fully rigged? If not, take a look at this guide to simplify the process.
Guide: “How to Rig Your Avatar for the Metaverse using Maya LT 3D”
Assuming everything is set, let's proceed to create a new set of bones for the eyes.
Start by creating a set of bones originating from the head.
This fresh set of bones should extend up to eye level and then have an additional bone that goes up to the pupil.
While not mandatory, it provides a clearer understanding of the desired structure.
Now that you've configured it, mirror the new bones to ensure you have a set for each eye.
Keep in mind the importance of accurate naming for the bones. Label them as Eye_L and Eye_R. This step will prevent any issues when using Blender and the CATS plugin.
With this in place, the next task is to skin the new bones to the pupils, allowing them to move in sync.
Everything is now set up correctly. Export the avatar as an FBX file and open Blender!
Using the plugin
In Blender, import the model we exported earlier. However, avoid using Blender's native import tool; instead, opt for CATS' dedicated import feature.
Unsure where to find it? It's conveniently located on the right side.
Click on the arrow to expand the menu, and you'll find CATS listed at the bottom.
Within the "Model" section, you'll encounter a prominent "Import Model" button. Give it a click, locate your avatar, and you're all set.
If your character appears chaotic upon import, hit the "Fix Model" button. With any luck, this should rectify the issue.
Navigate to the Eye Tracking section and ensure that the bones are correctly positioned.
From this point onward, everything is quite straightforward.
Simply click on "Create Eye Tracking" at the bottom of the submenu.
The plugin will initiate various corrections and operations that will lead to certain bone adjustments, ensuring that everything is set up accurately.
You can utilize the “Testing” tab to personally assess how the Eye Tracking is shaping up thus far.
Now, you can easily export the character.
Setting it up in Unity
This final step involves ensuring that everything functions smoothly within Unity! For setting up the avatar in VRChat, follow our guide: "Create and Upload a VRChat Avatar with Blendshapes Visemes."
The only additional aspect to consider is configuring the new eyes. Utilizing the Eye Look menu, you'll find an extensive submenu below, allowing you to choose both eye bones.
Once you've chosen the bones, you can experiment with each of the five different options to align them with their descriptions. Keep in mind that the value you set represents the maximum movement.
Consequently, the eyes won't exceed that range of motion.
