Setting up the Unity Project
It's possible that you can use older and newer versions of Unity, but for the sake of not complicating things, I recommend you use Unity 2018.3.14f1, since it's the same I use.
As always, create the Unity Project with the correct version. No need for Android versions. Now you should be on the empty scene. Before creating anything, let's install a couple of things first.
The Unity Recorder allow us to record videos, Gifs and images directly from the program, so no need to screenshot anything. Just Unity.
To install this, you need to go to Window > Package manager
The package manager will open up, and it'll take a couple of seconds to load everything.
Unity Post Processing Effects
Same as the Recorder, go to Window > Package manager and search for Post Processing
Now you only need to install it, there's no need to do anything else for now.
This is more of a personal preference, but you can search online for diferent shaders that you can use on your avatars to make them look cooler. The one we use is Toony Colors Pro 2, although there are a lot of free online alternatives that give similar results.
This is the fun in this part of the avatar process, experimentation.
Setting up the scene
It's time to set up the scene. At least the basics so you can always start with something.
I'll leave a UnityPackage down below with everything I'm about to do.
For a good representation of our characters, we use 3-point lighting method. It is a traditional way of lighting up a scene and it uses, as the name suggests, 3 different points of light to illuminate the avatar. Only the avatar, not the scenary around it.
Use this link to get the basic set-up of lights.
This is the basic setup we use for the avatars. Regarding the rest of the scenary I use the standard directional light that comes with every Unity scene and I create any new light that I see necessary to achieve the result I want.
You might be asking yourself "how can I make one set of lights affect the avatar and not the scenary or viceversa?"
We use the layer system Unity has by default. For this test I use a cube, but its exactly the same when you have your avatar posed.
First you need to create the layer your avatar will be on.
We have the canvas ready with the frame but how can I see the final result?
You need to set up the game window to display the correct resolution for it to work.
If the canvas is active everything will set up correctly and you will see the frame in place.
Poses for your avatar
Now it's time to pose your avatar!
I hope you used Mixamo because if you did, this part will be a lot easier.
Just use the FBX file you used to create the VRM (with the rig and the visemes) and upload it again in Mixamo. This time it won't ask you to put the bones in place, because it already has them!
Anyway, find a pose that you like.
Once you have what you need, download the avatar and put it on the scene. But that's not all!
You can still manually edit the pose by manually moving the bones to get a better, exagerated and more expressive. Also you can move around any dynamic bones you have manually to get much better results.
Think how your avatar would be like, what kind of personality would have and how it would act. If you have a nice base pose you can get a much better result if you edit the pose afterwards. Look at this example:
Play with the poses so you can represent how cool your avatar is! There is no limit!
Color and composition
Color is one of the most important factors. You need to understand what colors your avatar have and how to complement them accordignly to make it pop-up in the scene.
Having a basic understanding of the theory of colors will help you tons in this one.
To give you an example, I'll use the Cool Banana render that I made because it shows perfectly what I'm trying to explain.
The principal color of Cool Banana is, of course, yellow, which complementary is purple.
Of course this is not a Law, but rules. You should not follow these strictly and can bend it around how you please, so for Cool Banana I used both purple and tones of Dark blue.
By using these rules, you will ensure that no avatar will blend with the background and they'll be the first thing you will see. Not only that, it'll make sure complicated poses are easily readable.
What? You don't know what color is which and you can't figure it out? Don't worry, I have a solution! Adobe Color Wheel!
You can select the colors you want and with a click of a button, it will give you your complementary colors. What's even better, is that you can actually upload an image and it'll tell you what are the 5 more prominent colors, so you can then, select a few to color the rest of the background.
One of the most useful tools to have, and with practice, you won't even need it in the future!
Now it's time to set up a scenary for our avatar.
This is really up to you, if you have ideas where your avatar will be posed, make it a reality. Play with the primitives Unity provides to make a scenario or use Blender or Maya to create an even more detailed background.
Play with perspective, at the end of the day, its just an image, so you can stretch and deform everything to make it look cool. Look at this example of out Avatar nº26 Udon.
Looks pretty normal. But look how the scene is set up.
You can use another shader material to give a different look to you avatar. The material is one of the most importants parts to change on your avatar since it will radically change how it looks. See the diference down below.
Now that you have the background and the avatar in there, it's time to give the final touches, that is, post processing. Go where you camera is, and add a new component called "Post Processing Volume"
Now it's time to take the picture.
If you have done the steps of how to configure the recordings from before, just open the recording window and hit "Start Recording". It'll automatically render the first frame of the camera and it'll save it on your PC.
Be sure to turn off the frame from the canvas since it'll be already added when you mint your Avatar in CryptoAvatars.
Thats all I have! Experiment, and look at the renders we and other creators make for inspiration for your own work.
The avatar cards are one of the most important pieces on the whole process since it's the first thing any potential buyer is going to see.
Be creative and have fun!
Preparing the model
This way of doing facial animation should be considered when starting and through the process of creating the character.
Once we have the mesh of our character mesh ready, we can start getting some references from other games and other pieces of media to choose the extreme poses they'll be able to perform with facial animation. I'm going to choose one of my favourite 3d animated movies as an example: Wreck it Ralph. Let's use Vanellope since the proportions for the example is closer to hers more than Ralph.
So now we have some references, let's plan in how many parts we need to divide the whole puzzle to make it work... We need to separate:
VERY IMPORTANT: after duplicating and separating the meshes from the original one, you can see the cuts where the normals broke.
We can use Maya's Transter Attributes to copy the normals from the base mesh.
See? Much better! None of the cuts are visible :)
Texturing the model
Let's start texturing the eyes. The principle which all of this system is based on is an offset the UVs of the mesh or the material. Once we have the eyes in place let's proceed with the eyelids and make them blink!
As you can see, we need to leave some extra space on the sides of the eye so we only get one iris for each eye, even if we push the UVs to the limits.
When painting the eye, I use the Base Iris we already have to draw the eye itself, but the point will be to leave only the eyelids and the red part as opaque, and leave as transparent the center so the eyeballs can offset under this new layer on top. Let's draw the rest of the frames and then we'll organize them in a texture atlas!
This is the result of the 4 frame animation.
Let's now organize them so we can have more variations with other expressions in the future! We can use a 4x4 grid, giving us 4 animations of 4 frames each, 2 animations of 8 frames, or however you like to organize it! Thanks to Photoshop guides we can organize them in a regular way, with 1/4 of the total width of the texture.
The red parts are going to be painted white so we can change the skin tone inside Unity, and the insides of the eyes are going to be removed so they're transparent, looking like this when exporting the texture:
The checker pattern is the transparency bit.
Let's import our character into Unity now and try how our system works. Assign all the materials to the correct mesh. Make sure the Eyelids' Material is set on "Fade" so it lets the underneath colours go through.
Animating the model
The next part is where different workflows can be used to make the eyes, eyelids and facial parts animate we setup before.
The optimal way to do this would be to setup a script in Unity that receives the offset of different facial parts with a setup of bones that control those offset from Maya, so the Animator in Maya can see what they're doing. It's a more complex topic and I hope I can make a blog post in the future about that.
We can also setup an animator that controls each part separately to modify the offset of the materials (this is the one we're using in this example since it's a very simple workflow, and can be compatible with Mixamo animations), but it's not really optimized.
We have to be mindful of the type of curves we're creating for a sprite sheet kind of texture (eyelids in this case), since if it was linear it would look super strange panning from one frame to the next. We need it to snap from frame to frame, hence we'll be using a "constant" type of curve.
Here you can see the difference between using "constant" and using "linear"
There are many workflows to achieve the same result, and in this case we might get carried away about what should we use, either blend shapes (morph targets) or bones for a facial rig, but there are always more options available to you, and the games from Nintendo make a really good use of "now forgotten" techniques that were more common in the past.
Hope you learned something new today, and that it was useful. See you around! :)
Setting up UVs
How we did it
When we made our avatars we made the UVs in a particular. Each UV isle is one color, so that way in Substance Painter.
You've got your avatar ready, right? If you are in doubt, you should follow this few guidelines to make sure you won't have any problems:
If you think your UV maps are done, then let's go to the next step: PAINT!
Painting or re-painting
Our avatars are quite simple in terms of painting, usually using a combination of flat colors, maybe some gradients, patterns to mix things up and letters in some cases. So, how can we make those?
We will use Substance Painter for almost all of that.
First, you need to import you character. Use whichever program you know (Blender, Maya, 3D Max...) to export an FBX file.
Then in Substance, create a new proyect. Use the template which fits with your proyect and use the select button to find your fbx file.
Now you should see your character on the left side and the UV map on the right side of the screen.
On the top right corner of the UV map view you can select "Base color" so that way you can only see the colors without any material added.
Now we will go on painting.
It will create 2 squares, one representing the color and the black one is the mask.
Click on the color and right under the Layers tab, scroll through the propeties until you find Base Color. You can select any color you want. Using the dropper tool you can even select colors that are on another place outside the program.
If you have the color you want to use, you will see nothing has changed. Now its time to use the mask we created before.
Click on the mask black square, and go to the left of your screen. You will see a couple of icons. Select the Polygon Fill one. Right above it, select UV Chunk Fill.
Now you can begin to paint.
Select whichever UV isle you want and it will get the color you said.
Repainting on avatar
If you are repainting one of our avatars you can actually add the base texture if you want to preserve one zone like the original.
On the top left, new → import resources. Click on the Add Resources and find the texture you want to add.
Make sure to mark it as a texture.
And import it to your current session.
Now that it's imported it should appear on the texture section on the shelf (Located on the bottom).
You can drag and drop the texture into the base color on any base layer and with the help of a black mask, you can select which zones to use.
Once you have your final results it's time to export.
Go to New → Export Textures
Click on the name of the material and deselect or select the maps you have used or want. For our avatar we only used Base Color, so we won't use the other ones.
On global settings you can change the directory output, the file type and the padding.
This last one is important. Padding is the amount of pixels over a UV isle in which the program will draw over the limit. If that doesn't make any sense here's a comparasion between a texture with no padding at all (right) and 10 pixels of padding (left).
By default it's set to infinite, which is alright, but I usually set it to "Dilation + Default background color" and I set the pixels to a minimum of 8. That's why it was important to space out UV Isles.
Once all the parameters are correct, hit the export button and done!
We need to set the avatar in T-pose for an easier workflow.
Before import hands
Delete one hand
Match and Combine both objects
It's time to adjust the hand and match the vertex with the mesh of the avatar.
Finally, if we are not convinced with the final result, we also can move the vertex/edges/faces of the object before export. Just make sure that the avatar hands are exactly where you want to, with the scale as you want,...
Once we followed this steps, now we have to export into a fbx and upload it to mixamo.
Make again mirror of the avatar
To make this faster than usually, I use Blender 2.9.
Navigating he Viewport
First of all, import the fbx of the avatar, just before you want to upload.
We want to select one side (right or left) and delete the other one.
Make sure that the middle vertex are in the position 0 in X
Make again mirror of the avatar
When you have been uploaded your avatar to mixamo (using the Auto-rig), choose the animation of typing to get sure that each finger works fine. Also, we have some settings at the right, we will use one whose Tittle says "Character Arm-Space".
Of course, the scale of the hands, topology, etc can not match perfectly with all the avatars, but we can modify at least, the mesh of the hands or avatar before combine together and make again the UV's.
However, normaly change the mesh of an avatar rigged could be tiresome. But following different tips and rigging with mixamo gives you a lot of time that you can spend in other things.
So the first thing you need to know to create different animations in Blender, is what are the actions? How to create one? How they work? and How do you delete them?
An Action in Blnder is a tool to record and contain the data. As everything else in Blender, Actions are data-blocks. So when you animate an object by changing its location with keyframes, the animation is saved to the Action. That way you can create as many animations as actions you want.
Each Animation could be a different action on the character or object. For example run, iddle, shoot....
Then to view the action menu, change the content of the page to the Dope Sheet and then you can change it to the Action Editor.
Changing the window to dope sheet
Menu to change to the Action Editor
When you have the editor in view, you can start creating actions, but before you do that, you must have a couple of things in mind:
Once you have created and armature (Top corner Add→Armature), you are ready to create actions. With the Armature selected, press the New button, change the name of the animation and don't forget to press the shield (If you do not press the shield, Blender will not save the new name, and you will lose the changes you do with the action).
With the armature Selected create a new action
When you finish with the action, close it by pressing the X button at the right, so you access to the first menu on the editor and create a new one from scratch. An Important note if you want to delete the action because you don't like it, don't need it or if you duplicate it by mistake, DON'T press the X button alone (as stated before this closes the action but it doesn't delete it) and DON'T press the delete key nor the backspace one this won´t delete it either.
The ONLY way to delete it is to press Shift + the X button on the right, then close and reopen the file to refresh the motor, if you do not close the program you will still see the actions that you deleted.
From Maya To Blender
Now let's say, that you want to work in another software like Maya, This time the change is easy, but you have to follow a series of steps.
Once you import to blender; the fbx will create an armature in the scene with the animations in it. Then all you have to do is :
Exporting and Viewing
When you export the model, check a few options in the presets:
Web with the different animations to select
Getting your VRM ready
If you bought your avatar from CryptoAvatars.io or you already have a VRM file, congrats! You don't have to do anything here, you can go to next section.
This part of the guide is only for people who want to use one of our +100 free avatars or any fbx model for their meetings.
Get your FBX ready, because you are going to turn it into a VRM file. For this part, you will need Unity 2019.4LTS and the last avaible version of the VRM plugin for Unity, which you can download right here: https://github.com/vrm-c/UniVRM/releases
Once you have both, just create a new Unity Project and drag and drop the UnityPackage containing the VRM plugin.
Get you FBX and texture into the project too, and create a new material. Make sure everything is correct. Things that you should be on the look out are:
Now everything is ready, is time to export.
Select your avatar and go to the top left of your screen, to VRM → UniVRM 0.58 (or whatever version you are using) → export humanoid.
Set the language to english, if you haven't already, and add a title, author and version of the avatar.
Now click export and save it wherever you want.
Nice! You now have a VRM file that you can use on the next steps.
Basic VMagicMirror settings
Once you start it up you will see a green screen and another window in Japanese. We will set the language to English, of course.
Now is time to load in our VRM file. Use the "Load File on PC" button to find your VRM file.
The avatar should appear out of frame. Right below the load button there is and Adjust Size by VRM. Click it and should make everything a bit more on focus.
It is very likely that the avatar arms might look broken, we are going to change that now.
Go into the settings menu and into the Motion subsection and you will see Arm menu, and there are 2 options you want to look "Waist width" and "strength to keep upper arm to body [%]"
You can tinker with those 2 options until you get a desireable result.
More VMagicMirror settings
The basic stuff is ready to go, but, if you want to change anything else, these other settings might help you tune what you want.
Changing the position of the camera
Open the settings window and find the "Layout" menu. Here you'll find all the Camera settings you need.
By checking "Free Camera Mode" you'll be able to move the camera all you want inside the View Window, where your Avatar is. Clicking with the middle mouse button allows you to move the camera and the right click button rotates the camera around. Using the scroll wheel up and down you can zoom in and out. You can also change the Field Of View (FOV) just below the "Free Camera Mode".
Changing the position of the devices around your avatar
On the same "Layout" menu, you will see the "Devices" submenu, and just check "Free Layout".
Once that's done return back to the screen where your avatar is, and you will find gizmos in each of the devices your avatar interacts with.
Use the gizmos to reposition the devices to your liking. On the top left you can change if you want to change the position, the rotation and the scale of the devices, as well as changing if you want the gizmos to use the device as reference of coordinates by selecting "Local" or use global coordinates with "World".
Finally, if you want to go to the Default camera, just click the button on the bottom and the camera should reset itself inmediately.
Turning off all devices
Now, if you don't want to have any device and just want your avatar to stand up and talk, you can disable all the devices on the same Layout menu. Just scroll down until you see the Device Visibility and there you can see different options you can turn on and off. I would suggest that if you turn off all the devices, you also select the "Always-Hands-Down Mode" in the Motion Menu.
Changing the background
You can change the background color if you dont wan't the bright green color of the chroma by going into the "Window" menu and using the Background Edit color option. Now is up to you how you want the background to look.
You can also make the background completely transparent by going into the "Basic settings" just above the Background submenu and toggling the "Transparent Window" option.
Setting up some animations
Saving and loading configurations
Do you have your place set up and want to save it for other time? You can! Just head back to VMagicMirror Menu, not the settings, the menu where you can load your avatar, and on the bottom right, you will see different options under the name "Setting Management".
There you can Save, Load and Reset to default if you want.
Saving creates a .VMM file with all the information inside about how everything is set up. Once you have that file saved on a folder, you only need to click on load and everything will set up correctly.
Now is time to set up Zoom. This one is easy and fast. Since VMagicMirror is a program, you must use the "Share screen" option instead of a webcam. Find the window your avatar is in, and just click share.
Be mindful, if it ever says that the screen sharing has stopped, because you minimized the screen or any other reason, you can always restart it by clicking Resume Screen Sharing on the menu toolbar that appears on the top or botton of your screen. This will make that your VMagicMirror screen is always sharing to other people even when you interact with other programs.
Other Options: 3teneFREE
Is a free, VRM compatible with similar characteristics to VMagicMirror. One of the differences is being able to highly customize the background, with the possibility of even adding other 3D models to the scene.
If that still doesn't work, it's possible that the blendshapes are not correctly set up for the VMagicMirror.
We are going to fix that, and we are going to need Unity 2019.4 LTS and the lastest VRM plugin for Unity which you can download it here.
Just download the UnityPackage file and drag and drop into Unity to install it. Now do the same with your VRM. Drag and drop it into Unity.
When it finishes loading all the file, it will appear a bunch of folders, referencing the VRM.
One of these folders is called "Blendshapes". Enter it, and it will appear a bunch of different names. For the sake of simplicity, the only ones that we will take a look are:
For example, if you are on A, you need to find "vrc_v_aa" since thats the one that matches.
Same for every other:
Drop it into the scene, click on him, go to VRM → UniVRM 0.58 (or whatever version you are using) → export humanoid and just click export, since it already has all the information inside.
My arm is not bending correctly
Unfortunately, VMagicMirror is still fairly early in development and this kind of things can happen. Just the way some of the IK are set up makes it impossible from our end to fix these kind of problems.
If changing the arm parameters like I said before doesn't make the result better, you might want to try other program for a more permanent solution.