Saturday, 21 November 2015

TUTORIAL: Principles of Blender Rigging and Weighting - Part VIII

This is the eight and final part of my series of tutorials on rigging and weighting in Blender for objects destined for use in Opensim. You'll find links to Parts I-VII on my main tutorials page. In final installment I'll be talking about tri- or quad-weighting meshes -- why one might wish to do so and what sort of strategies tend to be the most productive.

As you might recall from the end of Part VII, these are terms I use to indicate the number of bone weights involved for an area of vertices and not the face shape of the mesh's polygons. A "tri-weighted" area of a mesh is one where each vertex is assigned to three different bone groups, and a "quad-weighted" one has four.

With the default SL avatar the vast majority of the vertices are weighted to either one or two bones. It might even be all of them since I can't think of any vertex with three weights. Towards the extremities of the armature that makes perfect sense and looks fairly natural in-world (if one ignores the fact that the human body is far more articulated than an avatar).

Using the arm as an example, you'll see that all the vertices of the hand are fully weighted to only the mWrist bone, the majority of the forearm is fully weighted to the mElbow bone, and the upper arms is weighted to the mShoulder bone. Between each, there is a very small region of vertices that have weights for the two bones that meet at the nearby joint.

The same is true of the legs (and the ankle area has no intermediary transition zone at all) as well as the head and neck (when thinking about an armature, the "head and neck" are an extremity because they're the end of a bone chain extending from root).

When you compare the general movement of these areas of the avatar to the way your own body moves, they're not too dissimilar. Your body looks far smoother but, overall, it's not too bad.

Things get ugly in the main part of the body. When you look at the way the shoulder area (mShoulder, mCollar) and torso (mPelvis, mTorso, mChest) move. By looking at the weights, you'll see that the bone group assignments are just like those of the extremities: areas where only a single bone has influence, and very rapid transitions between them.

Because these overlapping regions are very narrow, the mesh movement at those transition zones is fairly extreme resulting in significant texture stretching and very angular, unnatural looking bends.

 Nowhere is this more evident than in the transition from chest to torso to pelvis when your avatar is bending forward or twisting side to side. Not only is the mesh shape utterly in-human, but the textures are stretched to obscene extremes (both for skin and any wearable clothes).

You'll see similar issues and effects with movement that involves the collar and/or shoulder, and some extremely odd -- to put it mildly -- bending and stretching in the area of butt and groin.



Most of these issues are due in large part to the very low poly count of the model which provides minimal topology in the area for weighting transitions to be made. Even so, a basic examination of your own body will tell you that when you move those areas, it's never done in isolation. Sure, some of the in-world ugliness stems from poorly made animations, but at least as much of it comes from the combination of three key factors:
  • limited armature (3 bones to cover the entire back instead of having ten times that many as the human body does) which we can't alter
  • limited geometry of every few vertices available to weight with, which we can change by using mesh
  • limited bone influence by never having a region be influenced by more than two bones and often by only one bone which is also something we can change by using a reweighted mesh
This isn't just about the avatar body, though. The exact same issues arise when making clothing. If you match the weights of the body you'll perpetuate those shortcomings, and two of them aren't entirely necessary.

The strategy for minimizing these problem areas will work for revising or improving a mesh body, and will also work for any mesh clothing you make that's intended to be worn by the normal avatar. You can "double up" on the improvements when using mesh clothing made to match an enhanced mesh body.

Geometry and Topology

The first part of our strategy to have things move more naturally is to address the mesh itself and ideally will involve both geometry and topology. For people new to mesh, those might seem like synonymous terms but there's a subtle distinction (or at least in the way I use them there is). In general:
  • geometry is the pure vertex count...more vertices means more geometry
  • topology looks at the distribution or arrangement of those vertices
For our purposes, if we just wanted to "improve" the geometry we'd be tempted to do something like a simple subsurf modifier which would give us more than 3x as many vertices to work with (Ruth is ~8k verts and ~16k tris and a single subsurf makes her ~26k verts and ~52k triangles). I can already hear the screams...but bear with me.

The interesting thing if you do this is that Ruth doesn't actually become all the much easier to weight. No matter how much work you do to them those transition areas are still pretty ugly -- although better -- and you've almost quadrupled the face count. This is because Ruth's topology -- the way her vertices are arranged to model the shape -- isn't all that great and isn't improved significantly by a simple subsurf.

In comparison, it's possible to create a model with better topology that isn't even double Ruth's geometry and is much easier to work with for weighting. The accompanying picture is Ruth (pink) with 1 level of subsurf standing beside my model (yellow) that has a completely different topology and lower geometry (~14k verts and 13k most-quad faces  that becomes 26k faces after triangulation). Ruth is using the default SL weights, mine uses a mixture of tri-weighting and quad-weighting. Both are in the identical pose.


Admittedly I could spend a day or two improving subsurfed Ruth's weights and get a closer approximation (and my very first mesh avi was done that way) but by by restricting myself to dual-weighting and only working with the SL topology, the results were never satisfactory. The key to improvement was adding geometry only where it was beneficial and ensuring that the topology of the mesh was arranged to support the natural zones of movement for the body.

It's well beyond the scope of this tutorial to go into that side of things in any depth -- it would be a lengthy course of its own -- but if you're making clothing to fit Ruth it's worth bearing these things in mind:
  • Areas of the clothing that will cover parts of Ruth that are static (areas that have 100% weight to a bone) are areas the require only the minimum possible geometry to give the shaping you want
  • Areas of the clothing that near joint locations and will need to bend and stretch a lot are areas where you need extra geometry, and ideally geometry where the loops follow the contour of the pivot zone of the bend
  • Even though a Ruth joint will be bending in a fairly ugly way, if you leave a little clearance for your clothing to bend differently (or use an alpha mask) Ruth won't poke through it so you can give the illusion of a far smoother bending of the body.
The vast majority of the improvement is achieved through the weighting, but before I talk about that -- which is the meat of this tutorial --  let me first address any moaning and wailing about increased geometry.

Yes, an increased poly count will result in an avatar or clothing item that takes longer to load and is a little harder for graphics cards to manage. If you export Ruth (just basic Ruth without the subsurf) as a .dae, her file is slightly under 1MB in size. My alternate model is fractionally over 3MB. In case you're curious, Ruth with subsurf is a little under 5MB. Now let's think about all three of these for a moment.

The original model was designed about 15 years ago to be suitable for the typical internet speeds and graphics capabilities of that time. Hopefully it won't come as a surprise that those specs are slightly different than today's standards. I did a bit of googling and came up with the following fun facts about what was fairly commonplace at the time that Ruth (and the rest of the Second Life stuff) was first being designed.
  • approximately 85% of internet use in North America was still being made using dial-up connections (max 56kbps) and the best available "high speed" connection for commercial users was 1Mbps (which would cost you a small fortune)...only ~5 million Americans were using broadband of any type and only 7 States didn't even yet have access to it. I'm sure numbers in Europe were similar (with obvious regional variations, much as Canada -- where I live -- was a bit ahead of the curve)
  • On the graphics side of things, nVidea had only recently released the GeForce 256 which was the first card to have its own GPU and sported an astounding 256KB of DDR RAM! If you drove a Porsche at the time you might have upgraded it to 512KB.
  • And to look at? Why not one of those insanely expensive new 1280x1024 CRT monitors...
  • Driving the whole thing? Probably one of those fancy new Intel P3 1GHz chips running Win98 (or WinMe). If you were wealthy you might have had as much as 256MB of RAM but it was costing you about $100/64MB to buy it (but only 5 years earlier that same $100 would only buy you 4MB).
Now let's look at those file sizes. Remember that the file only needs to be sent once to each viewer that enters the region. Ruth is hard-coded into the viewer and doesn't need to be sent, but if you wear mesh Ruth -- even if identical -- you'd have to send that data.

According to Akamai Technologies' data, the global average download speed in 2014 was slightly under 4Mbps and the majority of developed nations had speeds in the range of 10Mbs but for convenience let's use 8Mbps since that's the same as 1MBps. At that speed, the Ruth mesh might take a second to deliver to most users, my alternate mesh might take as much as 3 seconds, and even the painfully-high poly subsurfed Ruth would be there in 5 (and as I demonstrated, that's overkill).

You get the point, though..it's probably not pushing people's capabilities or the original design's envelope too much to add a little extra geometry in appropriate places if you're making your own avatar.

Tri-Weighting - Bone Chains

Now let's get to the good stuff, and let's focus just on the main torso for now.

I said earlier that by examining the way your own body moves you'll notice that the major problem area of the avatar's torso is one where your natural movement is never really in isolation. When you bend over to the side, your entire spinal column -- all 30+ vertebrae -- rotates in a smooth, continuous way, as does all the stuff between the bones and skin (and it's actually muscle doing the moving and the bones just go along for the ride), and on the outside your skin stretches and moves along with everything else. It's not three distinct sticks moving separately...it's a whole continuous region moving as one.

To mimic this effect, we need motion of the mChest bone to have a "ripple-down" effect on the mesh that extends far lower than the SL weighting provides. Similarly, when the mTorso bone moves, it should affect the body quite a bit more than the base mesh's weighting does. Further examination of your own body will tell you that the rib cage tends to move as more of a block than the mid abdomen, and that in the region or your hips your pelvic bone is preventing the sides and back from moving and stretching as much as the belly. Your back doesn't move much at all compared to your front, either.

Way back at the beginning of this series we played a bit with a two-bone armature and a cylinder. In doing so we learned about smoothing the transition between two bones and it was quite easy to achieve because our cylinder's geometry was detailed enough to give us enough vertices to play with, and it's topology -- a series of evenly spaced concentric rings -- was ideal.

Your own torso is not all that different. Even without tri-weighting you could simply add some geometry from about the base of the breasts to the crest of the hips, fix the topology, and then make those 2 "joints" (chest-torso and torso-pelvis) have a smoother transition. We don't need any extra geometry above that because the rib cage moves more as a block, and the lower portion of the pelvis wouldn't benefit much from it either (for movement...for anatomy is a different story).

I tried that approach at first with my own avatar and wasn't satisfied with the results because that middle area that was heavily weighted to the mTorso bone would move as a block, and a real body doesn't. After a considerable amount of experimentation I discovered that a far more realistic motion was achieved with weighting as shown here:

The total geometry difference between this model and Ruth is less than double
the total tri count, and a not-inconsiderable amount of that increase comes from
having modeled toes and a few other anatomical improvements..the actual geometry
change for this area alone is less than a 50% increase

As you can see, almost the entire trunk of the body has weights for three bones (ignore the shoulder region for now). The mTorso bone has nowhere that is fully weighted to it...it is always being influenced by both the mChest and mPelvis bones which is more or less exactly the way that part of your body reacts to movement. You'll also notice that the mTorso and mHip bones interact a little across the hip crest and outer butt.

My mesh has just sufficient geometry as well as the topology required to provide loops of vertices that flow smoothly through each of those transition zones. When I make clothing, I give it similar consideration.

This area we've been looking at is only one of the two occasions where tri-weighting can be extremely effective: a case where you have 3 bones in a chain. The other situation -- a "tri-weighted Y-joint" requires a different approach which we've actually already seen Part VI but I'll reiterate again, later in this one.

For tri-weighted chains, the method of doing the weighting is one that takes a while to become comfortable with so hopefully I can give you some decent advice:
  • I find it essential to have auto-normalize enabled to keep the sum of weights equal to 1.0
     
  • Work with the model in T-pose and then only rotate one of the three bones you're working with for that tri-weighted region. When moving a bone, consider whether another of the tri-weighted bones moves with it and also influences the mesh. For example, rotating the mTorso bone is also rotating the mChest, so the only weight that counteracts this movement will be coming from the mPelvis (and mHip) bone. When you rotate the mChest, both mTorso and mPelvis weights will be counteracting the motion of the mesh.
     
  • Any time you either add or subtract weight (rather than blur) think about the "math" being done by the auto-normalize. If you're adding weight to a bone, it proportionally reduces the weight of both of the other bones. If you subtract weight, both other bones will have weight added.
     
  • Auto-normalize does proportional changes. Let's imagine a vertex that has weights of 0.1 for mChest, 0.8 for mTorso, and 0.1 for mPelvis. If you add 0.1 to the mChest weight using your brush, it will rise to 0.2 and the other two bones, combined, need to lose 0.1. They don't each lose 0.05 though, because it's proportional. Instead, the mChest will lose a bit under 0.09 and mPelvis will lose a little over 0.01. It might help to think about relative percentages. It might sound complicated but believe me, it's the way you'll want it to work and makes things much, much easier to balance!
     
  • Always try to work with the "outside" bones. For example, if you rotate the mChest bone you will want to do your weight adjustments by brushing with either the mChest or mPelvis bones and avoid using the mTorso bone. Then zero the chest bone's rotation, rotate the mTorso, then do your adjustments with mChest and mPelvis again.

    In this scenario, the only times you'd paint with mTorso weights is for a region that is affected by mTorso, mPelvis and mHip (and not mChest). You'll also potentially paint with it in the upper chest area when working on movement of mesh that has weights for the collar.

    What you'll find is if you balance the two "outer" bones like this, the "middle" bone will automatically end up with the right weights by virtue of the auto-normalization.
  • When you're smoothing out little bumps and jitters in weights you'll find that you should mostly be working with a bone that has weights in that region that are either very high (red) or very low (blue), and only rarely with a bone that has mid-range (green) weights.
     
  • I would characterize tri-weighting as being about 50% math/anatomy and about 50% "skill." I think there's a real art to it that can only be developed over time and practice. The more you do it, the better you'll gain an "intuitive feel" for the process. Eventually it almost becomes second nature.
     
  • This should probably go without saying, but before making any major weight changes in an area you'll want to save a back-up copy of the file to revert to if you find that it isn't doesn't end up doing what you thought it would. That will happen a lot when you're first learning.
Even though the example I'm using here is for an avatar body, the exact same strategy applies equally to clothing, even if the underlying mesh body that will wear the clothing isn't tri-weighted. This works nicely on the torso of a dress for Ruth where you leave enough clearance between her body and the dress (or alpha-mask any poke-through).

Tri-Weighting - Y-Joints

The other time you'll want to consider tri-weighting is one we looked at in some depth in Part VI when we did the weighting for the skirt portion of the dress. It involves an overlap of weights for two bones that both attach to and pivot at (or near) the same joint position of a third bone. This happens in two places on the body:
  • on the "cross-over" part of a skirt or dress, or the crotch area of saggy pants
  • the shoulder region
For the former, you will find that you almost certainly do not want to do tri-weighting of the underlying body. If you do, you'll find the groin behaves very strangely as the legs move (trust me, or try it yourself if you'd like a chuckle and don't mind wasting some time). For something that "bridges" the gap between the legs, though, it often produces a much more natural looking result.

I gave detailed instructions for that type of weighting in the previous tutorials so there's no need to repeat it here.

The shoulder region (of the human body) is an extremely complex one to model and weight (and animate) to look natural. It's a complicated  area that the armature simply can't do justice to with its Linden Lab structure, and can be thought of as a mirrored pair of Y-joints where on each side there is an interaction between the incoming arm chain bones mShoulder and mCollar with the main mChest bone "trunk" of the body.

Your real life body has some additional influences where the motion of left shoulder affects the right side of the back, and the base of the neck, and even extends (by way of skin stretching) quite a long way down the sides. The collar and shoulder musculature involved are also tied into the pectorals and lats, but only when moving the joint in certain ways; and...well....the purpose here isn't to do an advanced course on anatomy so let's just leave it at the shoulder being a complete mess of interconnected stuff.

To be fair, the 3D animation world has long since resigned itself to the knowledge that a simple armature and set of weights is never going to be sufficient to make shoulder movement look natural so this is definitely not a shortcoming that is unique to our SL-based avatars. Tri-weighting this area can make a rather drastic improvement over Ruth's very strange weights but when you approach it you will need to think of it as a chain, rather than as a Y. The chain is
mChest <> mCollar <> mShoulder
where you ignore the neck and will have to accept some odd creasing/shaping on the top of the shoulder.

I would suggest that you don't try to do any cross-over interaction between the collar bones on the back (or upper chest at the base of the neck) because from experience I can tell you that it will never look right in motion -- or at least I wasn't able to achieve natural-looking mesh responses after devoting 50+ hours of dedicated weighting work in that region. What I was able to do, however, was fix the top of the shoulders (and make minor improvements in the area in general) by....quad-weighting.
In case you're curious, the 3D animation world does have a couple of solutions for the shoulders that work extremely well and can be found (at least in part) in most modeling and animation software. They tend not to be suitable for real-time rendering, but an application like Opensim is a relatively rare one compared to the bread-and-butter work of this software (CGI, frame-rendered animation, still renders, etc. where there is no need for instant on-screen response).

The three most common methods are:
- "shape key counter-morphing" where one or more shape keys are defined and assigned to the necessary vertices, then animations also include the required counter-morph slider settings
- "bi-lateral weighting" where different weights can be assigned to positive and negative transformations of the same bone (if the bone translates in a positive direction from its rest pose it has one set of weights and if it translates in a negative direction it has a different set of weights)
- "vertex target hooking" which works a bit like a cross between the shape key method and a gravity well object to retard mesh motion away from its rest position

Blender fully supports the first, doesn't support the second (yet), and has partial support of the third. For real-time environments the third method is the one most likely to make its way into the mainstream in the not-too-distant future (but not in SL/Opensim).

Quad-Weighting

The quad-weighting method isn't for the faint of heart. It takes a lot longer to balance and smooth the weights because in many areas you can't isolate the effects to the motion of a single bone. Instead, you have to carefully balance it all as you paint.

This is the one time where turning auto-normalize off can be helpful, provided you remember to then later normalize all bones and re-check that everything is moving the way you want it to. Before attempting this method you should also be fairly comfortable with tri-weighting. If you are...
  • Turn auto-normalize on if it isn't already
  • Begin by tri-weighting the region(s) based on nearest appropriate trio of bones. For the outer half of the top of the shoulder this will be the mChest-mCollar-mShoulder bones. For the inner half it will be the mChest, mCollar and mNeck. You may also have a small area where you want mHead, mNeck and mChest
  • Start adding weight for the fourth bone until the general response is roughly appropriate
  • Turn auto-normalize off
  • Move each bone in isolation, using only blur or subtract to balance
  • Any time you subtract, be sure to subtract roughly proportionally to the existing weights for each bone that needs to be reduced (so if you need to subtract neck and chest, subtract them both one at a time and in approximate proportion to their pre-existing weight balance)
  • Once it's more or less correct, click the "Normalize All" toolshelf button and ensure that you you have the options set for "All Groups" and disable the "lock active" toggle.
  • Check the bones again, repeating the process until you no longer need to subtract weights after normalizing and can simply blur
  • Enable auto-normalize again
  • Do one final check of all bones involved, blurring as required (or very, very small additions and subtractions)
  • SL/Opensim doesn't support weighting 5 bones to a vertex so try to avoid it if possible. It won't cause an error on upload but I'm fairly sure that the lowest weight is dropped and the others are normalized again after the drop which can make a mess of all that work you did
Even once it looks pretty good in Blender you'll want to upload and watch the area closely in-world as you play a series of animations that move it through its typical range of motion. Be sure not to base your evaluation on just a single animation or two...there are plenty of animations that are poorly made, particularly for shoulder movement, so your weights may look great for some and horrible for others. There's nothing you can do about this other than make your own custom animations.

In this picture you see the upper body, rear, where most of the shoulder region and areas in the upper body are quad-weighted. This produces a much smoother and more natural reaction in the body for motion of the shoulders and neck. Don't be surprised, when doing this from scratch, if it takes many hours of work on just this one region to achieve a decent balance of weights.

Less Is Often More

I should close out this tutorial -- and final installment of the series -- by saying that one thing I've learned is that less is often more. I will always try to do the majority of my work with only dual-weighting and save any tri-weighted regions for the areas where it's really required. I only considering quad-weighting for special cases and only if it's absolutely necessary. If you do it on your body you'll also have to do it on any piece of clothing that goes over it...

When I do have to tri- or quad-weight, I'll use one of a few prepared weight source meshes I've created that are already weighted to match my body. These source meshes are very high poly (200k+) and perfectly symmetrical (which made them much easier to weight when I created them). I began creating them at low poly, weighted them, then subsurfed and applied, then tweaked the resulting weights until they were best possible (including upload them, cut in pieces, to test on my body).

Using these stored source weight meshes, I generally have extremely close-to-final results after weight-transfer to my target mesh and will only have to do a few minor tweaks. It's worth putting some time and effort into making these since it will save you many hours of weighting work on future items.

I hope this series has been helpful as a starting point for your own work. Don't get frustrated if your first few attempts aren't as good as you'd like them to be...the art of weighting is one that takes time and practice to become comfortable with, and even longer to master (I'm not even sure that I've mastered it yet, for that matter).

Good luck!