<< Home
Mental Ray Tesselation Tutorial

If you have ever had problems with Mental Ray not tessellating how you want I hope this clears things up a bit. I didn't notice a problem until I started rendering things larger and with trim surfaces. After some research I found that mental ray has it's own tessellation section and not only can you make things look better, but you can have much more control over speed.

If you don't know much about tessellationg yet you may want to do some reading first because I am assuming that you know how to use the advanced tessellation section for individual surfaces using maya.

As of yet Maya has not integrating the tessellation options into the attribute editor not is it called tessellation. It is called the mental ray Approximation Editor. There are a few down sides to the approximation editor. You can not display render tessellation so you have to make educated guesses to know what will happen when you make a change. The other downside is that you can't change the surface approximation of more than one object at the same time Even if they are almost identical. The upside is that you have more control.

You CAN change the tessellation of many objects at once, but unless your scene is made of almost all the same type of surface then you will want to avoid doing that unless you just don't have time to assign approximations to each surface. In case you are like me and sometimes Don't have time to approximate each surface here is how you can set approximations for all surfaces at once:

 

Overriding ALL surfaces:

1. Open up the render globals, make sure mental ray is set as the render choice.

2. Look down for the overrides section shown below and click the checkered box to the right. This will override the entire scene and change the approximation for all the surfaces in the scene to whatever you choose. I will go into that later. There are some good presets here for quick starting points for tessellation. Those are explained in detail near the middle of the page.

If you haven't noticed this box before it also can override caustics and global illumination options too.

The override section will override All surfaces to be tessellate in a certain way except for the displacement nodes: note though, that a surface with displacement actually gets tessellated twice. First he surface has a calculation, then the displacement map is added to it. Not having complete control over the two for an individual object would, in some cases, just be silly.

So, we have The approximation Editor. Find it here:

 

Windows>Rendering Editors>mental ray>Approximation Editor

This is what the window looks like with nothing selected. Notice two new approximation attributes: trim curve approx and subdivision approx. This will come in handy.

When you select an object you will notice the create and assign buttons highlight. For your first object you will have to hit Create to make a new attribute. (I will explain those later). you can select another object, that is similar, and assign the same surface approximation. To use this editor you simply keep in on screen, select each object one by one, and create and assign surface approximations. I would recommend naming your surface approximations well or you will forget which is which.

When an approximation attribute is selected you can click edit to change it's attributes.

Remember Trimmed surfaces should use the trim curve approximation method. And a surface with displacement should have the displacement approximation added to it. Surface Approximation only effects NURBS surfaces, however the displacement attribute can effect all types of surfaces.

 

OK, now for the harder parts. What do all these options mean in the surface approximation nodes? I'll do my best to explain them.

Approximation Presets:

These are killer starting points for you to use that you may, or may not, have to tweak to get your desired surface. This is what they do:

 

Regular Grid: This is the default. You have control over all approximation attributes.

Parametric Grid: This setting is probably what you would use for patch models. It subdivides every patch into a set number of polygons(triangles). You will want to make sure that the surfaces are built as close to 1to1 UandV as possible; meaning that for any U length the corresponding V length is about the same. You can fix those problems by rebuilding your surfaces, remember that you will lose any stitching when you do that.

Regular Grid: If your UV spacing is not great you might want to use this method. This gives the entire surface a fixed number of triangles and distributes them evenly. That means even if you have 3 isoparms tightly packed in a row the triangles will still be approximately the same size over the entire surface. Note: if you use the method of squeezing entire rows of CV's together to make parting lines in your objects then don't use this surface approximation. You either have to turn up your grid incredibly high or your lines will disappear.

Angle Detailed: This approximation preset places more triangles where the surface has higher curvature. For objects with flat areas and curved areas this method might be best. This changes as each surface changes, so you could assign this same preset to many different surfaces, even if they aren't that similar.

Pixel Area: surfaces closer to the camera will tessellate more than surfaces farther away. This also depends on the actual render size because there will be many more pixels in a larger render. Keep that in mind. I haven't tested this yet in an animation, but I have a feeling that it may not be great for animating objects that will be both close and far in an animation because you may get flickering pixels. Someone please correct me if I'm wrong. It may reuse the initial tessellation. Anyhow, It is great for print images and other things that don't move in relation to the camera for sure.

 

Someone brought to my attention that Alias has made it very difficult to change a surface approximation node after it is created. In fact, from the render globals you can't change it. It seems that you must create another node every time a change is made. DON'T Do that. Here is how to find the node:

Open the outliner
Make sure the display>DAG objects only checkbox is unchecked.
scroll down until you see some mental ray nodes.
one of them should be the approximation node you are looking for.
If you know the name already you can type it in at the top to display it
click on it, and open the attribute editor
You can change the settings there without making a new node

Thanks Karl, for bringing that little Maya mistake to my attention.


Some people may want to stop there, at least until they get the hang of it. More advanced users should learn about the rest of the features.

Approximations Method:

This is where the computer figures out when and where to subdivide, and how many times.

 

Parametric: This method subdivides based on number of UV coordinates. You can calculate triangles by using this formula: Triangles = U * V * degree*degree*2. Notice that degree is degree of your surface. Unless you pay much attention most of you will have surfaces with third degree surfaces and if you are smart a flat surface will be a first degree surface. If you had a curved surface with the subdivisions set to U=2 and V=3 then the formula would look like this: 2*4*3*3*2 which will give you 144 triangles. If you had a flat surface and left it with a degree three it would still have 144 triangles. Make it degree 1 and you reduces the number of triangles to calculate to only 16. Moral is use degree1 if you can.

Regular Parametric: This also subdivides a surface using U and V, but with a much simpler calculation. The triangles are roughly the same over the entire surface because they are using the entire surface instead of each individual patch. To find triangles here just do this: triangles = U*V. Simple.

Length/Distance/Angle: This method is adaptive. The surface is tessellated until each attributes minimum value is met. There is a box called View Dependent here. If this is on the value states is in pixels of your final render, not in Unit values. Keep in mind your final render size if you use that.

 

Length: This means that a triangle will be subdivided until no triangle has an edge longer than your specified length. Smaller numbers mean more triangles.

Distance: This method will subdivide triangles until your specified distance from the camera.

Angle: This value is expressed in degrees. A normal value is 45, and a value of 0 is ignored. Smaller values cause the number of triangles to increase. How it works is it calculates the angle of the normals of neighboring triangles and subdivides until the angle is less than your specified angle.

Approximations Styles:

This is how the triangles will actually be arranged using the method you choose.

 

Grid: This can sometimes produce more triangles than necessary. Imagine putting an isoparm across your surface to divide the surface any place that needed more detail in both U and V direction. This is basically how that works. If you have an detailed portion of a plane in the center then the four corners of that surface will have good tessellation, the center would have good tessellation, but the rest of the surface would have unnecessary tessellation.

Tree: The subdivides each triangle only as necessary. The rest of the triangles are very uniform, but still in a grid-like patterns.

Deluanay: This only works with NURBS surfaces. The later found "Max triangles," and "grading" attributes directly effect this style. Use those two attributes to fine tune this style. The deluanay style tries to make the largest angles possible of all three angles of the triangle thus reducing those long this triangles. All three sides are approximately the same tenth.

Fine: This creates an enormously large amount of triangles, but looks great. Your machine doesn't have enough memory? No problem. Mental Ray will subdivide each surface so the the memory load isn't too great. It then subdivides the triangles as well. So, your final result may be 9 surfaces with lots of little triangles on them, but your memory load will not suffer to much. This still will take the longest to render because of the amount of calculation, but shouldn't cause your machine to crash any more than maya already does.

 

UV Subdivisions:

This is available with the Parametric and Regular Parametric methods. This tells the computer how many times each surface, or patch respectively will be subdivided in their respective U and V directions. Remember Parametric method is the patch value, and Regular Parametric is the entire surface UV value.

 

Min, Max Subdivisions:

Available when using Length/Distance/Angle method. It controls the min and max number of times a triangle is subdivided. I wouldn't use a max value too much higher than 3 if you could help it. Each subdivision increases the number of triangles by four times each time you raise the number.

 

Grading:

This applies only to the delaunay style. I just noticed a bug as well. If you select an approximation method, then change the approximation style, the settings don't change unless you change approximation methods and then switch back. You will see what I mean if you do this: Select the spatial approximation method with grid style. Switch it to delaunay and you notice no difference. Grading only works with Delaunay style. Now change approximation methods and switch back to Spatial again. Notice that the change now takes effect. You can now select grading as a method.

Grading uses a gradation to smooth out the effect of changing from dense triangles to not so dense triangles. I wouldn't use a value past 15 or 20. It really doesn't have much good effect after there. You'll have to experiment with that to find values that work for your surface.

 

Sharp:

This is mostly pertaining to displacement. It uses a value from 0 - 1. 0 being rounded edges and 1 being sharp. Use higher values when trying to create sharp displacements. Use a small number if your map has too much contrast for the effect you wanted.

 

That's it. That explains all, or at least what I know, of the attributes there.

Once you have created several approximation methods you can easily reuse them for each surface to fix the tessellation. Here is the actual tutorial part. It's real simple, and I don't use displacement in this exercise. This issue probably comes up a lot if you us mental ray to render with trim surfaces. You can download the scene at the bottom of the page if you want to follow along.

The image below is a typical trimmed surface. I used the round tool to round the edges of a square, and pokes some holes in it. The square is flat on all sides in this case just because I wanted something quick to show.

As you can see some of the surfaces are clearly similar. The 8 rounds on the corners, the 12 edges, the three circular fillets connecting the holes, the 6 flat sides of the cube, the cylinder, and the part of the sphere at the top. That leaves only 6 approximations we actually have to make. First things first, I have not made the flat sides of the cube 1st degree surfaces. So I will rebuild them to be such. That will dramatically cut down on the tessellation. I did one test render to first make sure that this was a case where Maya tessellation was insufficient. This is what you may see. (click to take a closer look)

The main problems are the corners. If this were the only angle we were going to see this object at, then we could eliminate a few things like the lower back corner, bottom, the two back sides, and the back and lower back edges. This would also cut down on render time. There would be a problem with bounced light if you wanted to use final gather, shown, or global illumination. If you weren't trying to fool anyone into thinking it was a photo though, no one would notice.

Note: If you want to use the scene remember, to make rendering faster rebuild the six flat surfaces to be 1st degree.

OK, so let's start fixing this stuff. Pull up the approximation editor: Windows>Rendering Editors>Mental Ray>Approximation Editor. You can just leave that up for the rest of the tutorial. It doesn't take that much screen space.

I'm going to start with one of the corners first because they seem to be the worst in the test rendered image.

First, select the object you want to tessellate, and click create.
A box should have popped up that has all of the attributes at the top of the page that I explained.

For a round object that has a trim on it we should set a trim approximation.
Note: that when you use the round tool all surfaces it creates are trim surfaces. If it is an object as simple as this and you are going to have a complex scene later on it is well worth it to either trim convert these surfaces in the rebuild surfaces NURBS options or simply rebuild them manually. You could do that by selecting the trim edge and birailing or lofting, and using the boundary surface for the 8 corners.

Notice that there are no presets for the trim curve approximation. Since this is a static object that won't be bouncing around I will use Regular Parametric because this is a rather simple object that can easily be calculated in my head.

There should be the same number of tessellation per span of the corner as the corresponding edge of the rounded edges of the box. That way there will be no visible holes. Let's figure that out. Find out which attribute, U or V, is the curved side of the edges. This should be easy because one should have say 3 spans, and one should have 1 span. In my case here it is 7u and 1v. So I know that U is the curved edge.

I am going to use 4 U subdivisions and 1 V subdivision. for the edges. Because of this I should use 4,4 for the corners. The corners share all three edges of the NURBS surface with the edge of the rounded edges of the cube. As you should know NURBS surfaces can not have 3 sides. If you made a boundary then all points of one side are collected to a single point, so it only looks as if there are three sides. The same thing is happened to a sphere. Two sides are actually the top and bottom sections brought to a single point. The other two edges meet together. Let's look to see if we had any problems with the tessellation in those areas now.

I just found another bug in this program. If you name a tessellation to make it easier to find it does not refresh. You have to close the window completely and reopen it. Maya actually gives you an error, * node does not exist. If that happens, just close the window and open it again.

Now that I named my surface approximations for the two objects I can assign them easier to the other similar objects.
The render finished and I found that I was wrong. It's fairly easy to tell from the image below. I reversed the numbers for U and V I would think. Just to make sure I will reverse the numbers to see what happens.


Here it is with reversed U and V parameters.

As you can see most of the problem is fixed, so I can safely say that the numbers were switched. The lines are mostly gone, but the tessellation still doesn't quite match up. It will take some more tweaking. Until I have used this a lot I will always have to tweak that, but less and less as I get better at making an initial guess.

My second guess was 1,16 (U,V). The number of the corner still stayed the same at 8,8 just to see if the tessellation would line up. Sure enough, it lined up fine. You might ask, "Why did Andy (that's me) double the V parameter, but not the U parameter. That surface is really only degree 3 in one direction The other direction needs no further subdivisions because it has no bend. The corners, however, bend in both U and V direction which is why both U and V need to be subdivided. Remember that flat surfaces need no subdivisions. That can save lots of render time; especially if you make one object and then decide to make an array of a thousand of them, or make it part of a particle system. If you were rendering only this object then you would more than likely use a preset to save yourself time. The amount of extra time spent rendering would not amount to the time fooling around with each separate approximation.

Now that the edges and corners are fairly smooth I will worry about the divot on the top. You can notice a circle of evil black dots and lines around the bottom half of the round there. This should be easy to get rid of. My first idea was to try out the edge approximation I made on the round in the middle. You can see in the render below that it is missing. This is because the surface direction is the opposite. The easiest solution is just to switch surface directions and see if that works. I also tried the corner tessellation on the little half circle in the middle. That, I think, will work out fine. You can see the edge looks very smooth. You can also see that from this angle the top flat surface's inner ring seems to have some slightly rough tessellation. We will deal with that after the curved surfaces.

Swapping the U and V directions worked and solved that problem, but what was I thinking? The edge approximation is built for a degree 3, degree 1 surface. This surface is a 3,3 degree surface. That is why this next render looks like it does.

This surface clearly needs its own tessellation unless the corner tessellation works.

Sure enough, the corner tessellation worked fine. This keeps the file size slightly smaller (not significantly), but more importantly saves me time. The other curved surfaces are either end of the little tunnel through the box. These surfaces are 3,3 degree surfaces, similar to the one we just fixed. I would first try and see if that tessellation works good.

OK, here is where using FG (final gather) can be a bad idea to "quickly" light a scene. I often do that to achieve a simple yet fairly nice shadowed lighting scheme. The problem is that I can not figure out if the object is tessellating correctly, or if there are just final gather errors on the lower right edge in the image above to the left. I will have to temporarily raise the FG rays to find out if everything is right. Remember to only render the region, not the whole image in cases like these. The image to the right has 500 rays instead of only 75. I usually use between 500 and 1000 rays for a high quality image. You should use higher than that for supreme quality, but the ever so slight smooshy look that is left with 1000 rays merely adds more realism to the image in most cases. In most cases you can't see the smudges with 1000 rays (you will notice that the larger your image is, the lesser rays you will need to achieve a good looking image with FG).

I took the liberty of adding the edge tessellation to the cylinder through the middle, since a cylinder has a 3,1 degree surface. In some cases, remember, that you may have to swap UV directions to make that work.

Now, the image should look great like the one below. (by the way that is with 500 FG rays, not the initial 75) I also changed the floor to red. The one in the sample file uses Lambert1. Don't change that shader. Apply a new shader if you want to make changes to the floor.


The only surfaces left to do are the flat surfaces. I would only recommend doing that if speed was important to you. The automatic tessellation setting are working fine in this instance and I have already rebuilt the surfaces here to degree1. I did not do that in the sample file. You must do that yourself. Remember the floor is also a rendered surface and also is tessellated. That's all there is to it.

If you notice anything that I missed please let me know, and I will add it with mention to you if you like. If I am just dead wrong about something then please let me know too. I do not want to mislead anyone. Please send questions, comments or suggestions about any of the tutorials to:

tutorials@andycg.com

 

Sample file: Maya5 binary file