I received a request recently to have a post about how to manage large scenes. I thought that was a great idea for a topic so here goes.
Poly Count
Once you force yourself to follow good modeling habits then poly count almost stops being an issue except on rare occasions. The most important thing you can do on a high poly count scene is to instance objects whenever possible. This just makes sense. If you can use the exact same object in other areas in your scene then you should instance it. Not only will it be easy to update one model and have all the instances update as well, but it saves a huge amount of memory when rendering and also makes your file size a lot smaller. There is a downside to instances though. When you start to have a lot of them. Like around 5000 objects and up. Its great for getting your shot to render and saving the file, but working in the file becomes really slow. You have to make a trade off. If you have some RAM to spare with all your instances then often it will help to grab a bunch of the more simple objects (for example a bunch of grass blades) and break their instances and attach them into one single mesh. You now have one much larger poly count object which will take up more memory at render time, but you have sped up how interactive max is while your working considerably. Always do this with simpler geometry though. You have to make sure that what your attaching won't be so huge that you run out of memory again.
Another great way to deal with huge scenes is to have separate scene files for the background, midground and foreground. Then composite renders together to create the final seamless shot. I worked on a project once (which unfortunately I still can't talk about about and probably won't ever be able to), but it involved a huge jungle campsite. This cinematic went for about 3 minutes straight without a single (visible) cut and all took place in first person. The scene for this was so expansive and huge because the main character (you) run from one end of the camp to the other so it became nearly impossible to cheat things unless they were really far away. We ended up needing 5 separate environments that all fit together like a puzzle that were rendered separately and composited back together. If you gotta do it then you gotta do it.
Textures
Textures have always been the trickiest to balance. I always want the highest possible res textures I can get up close to camera and as things go back I'll make them smaller and smaller. But even with a lot of planning this can be difficult. Especially when you have a lot of different types of surfaces and maps that are close to camera. Once the texture is loaded into memory then max is going to treat them all the same, so you want to feed the highest quality image you can with the smallest file size. Most of the time for me this is just a simple jpg. As long as you don't have visible artifacting in the compression then your fine in most cases. I have also been known to use png textures with indexed colors. This keeps the file sizes lower and if you do it carefully you get almost no visual drop in quality. I stick with 8 bit textures for almost everything, except for objects that need displaced. For those maps I use 16 bit. Just so were all clear about this...You can NOT take an 8 bit texture, convert it to 16 bit in Photoshop and use that. displacement maps need to be generated in 16 bit or higher. In the end the best thing to do is to just be very picky about which textures are important to be high res and which won't be.
Displacement
I have used displacement in Vray, Mental Ray and scanline. Scanline was always predictable, but you aren't able to render millions of polygons since the renderer isn't designed to deal with that. That said I still use max's own displacement for some things. The displacment world space modifier is great. You can create very detailed and still fairly optimized meshes with it. Its not usable for deforming meshes though. I have a long history with Vray, and will admit that I was very let down by Mental rays displacement. Its better than using max's modifiers, but at least back in max2008 there was a lot of memory flushing problems. I found that I couldn't get more than a couple heavily displaced objects in my scene before I was out of RAM. This was a big problem for me since I was creating a big rocky landscape at the time. In the end we had to customize every single shots displacement settings just to get it to render. I was never happy with the level of displacement we had to settle for. I haven't used MR again for a while so I'm not sure if its any better in max 2009 or 2010. Vray on the other hand has never let me down when it comes to displacement. You can load it up all over your scene and it will render. Maybe not fast, but it will finish. Vray handles dynamic memory flushing very well. Vray also has a couple options for how to calculate displacement. So if one way isn't working for you, you can easily switch to another method. I like to use 2d style displacement in Vray because its very fast and predictable, but it does reach a point where it requires to much memory. So in those cases I switch over to 3d displacement which will still take a texture map to drive it, but the calculations are handled differently and you can use a lot more displacement in your scene at the expense of render time.
Lighting
Its been a while since I've had problems with optimizing lighting. So much of what I do now is all HDR lit and doesn't have more than a few extra lights....If any. I'm currently rendering an interior 3d scene for a commercial lit with only an HDR outside the room. No lights at all. But there are still many projects that will require lots of lights. Memory wise its better to raytrace your shadows, but that is also slower. Shadow maps are great, but you have to be careful balancing them. You can run out of RAM really quick. Never use an omni light unless you have to. Omni's project six shadow maps where a spot light only projects one. So you can save yourself lots of ram by choosing the right light for the job. With shadow maps there are three parameters that are important to optimize. First is the bias. This one is kinda worthless most of the time. It doesn't really affect memory or rendertime but it will affect the look of the shadow. Lower values are best. I usually go with .01 for everything unless an issue comes up. More important are the Size and Sample Range. Size is going to determine the detail of the shadow. The higher the value the sharper the shadow. The sample rangle will blur the shadow and soften out any jagged edges. I've seen people balance these two numbers wrong a lot. The size is what is going to use up the most RAM, but the samples are going to cause the bigger render hit. If you want a soft shadow, don't turn your map size way up and then crank the sample range up to blur it out and make it looks soft. Instead keep your samples at the default 4 and adjust your map size until its as close as you can get it to the final result you want. Then slowly turn up the sample range. I have never needed to go above 16 for the sample range. I've never gone above 4096 for a shadow map size either. Just like with modeling. Instance your lights as often as you can.
Rendering
Know your renderer inside out. Setup a simple scene that has a little of everything in it and then start spinning values and really get a feel for what they do. It can be as dramatic as knowing one value that needs to change to take a render from a couple hours to a couple minutes, IF you know the right thing to adjust. So get to know what everything is really doing and how it works. With large scenes your probably going to have longer render times just because theres more in them. Be really smart about your test renders. Find VERY low quality settings that will render extremely fast. Even if the render quality is very poor. This is where I start, and remain until almost the end of a project. Every once in a while I'll crank settings up just to make sure something isn't messed up, but keep everything low for as long as possible. Most of the time you really just need to be able to see what is happening in the shot and all the noise and mess can be dealt with later once you know everything is working. Use region renders as much as possible. Once things are coming together and working I'll usually select one shot that best represents the sequence and push that one through to make it final. This shot will become the master shot. This way only one shot on the render farm is going to take any time and everyone else can keep working with fast renders until the look, and which passes will be needed, are all figured out. This way you also learn what settings are likely going to be for final shots and how long they will take to render. This will help you predict how much time your going to need to render all the final shots.
I think that just about does it. :)
Tuesday, March 3, 2009
Thursday, January 8, 2009
Catching up
Time is flying by. We finished up that previous photo real project I was mentioning in my last post. It was a commercial for LG brand televisions. The commercial featured a bunch of robots made up of parts from inside the television. They come to life late at night in the living room of a modern looking home and have themselves a little rave party. Then the owners of the house come home and they all scurry back into the television and hide. The designs for the robots were pretty cool and were VERY highly detailed. The commercial featured live action background plates with CG robots. This project was animated in Maya, with all the lighting and rendering done in Max/Vray. It was refreshing to be back in software that I knew very well. The project went very smoothly once all the characters were approved. All the characters were highly reflective and had a large range of material types on them. Everything from chrome and etched metals to glass, plastics, translucent bits and glowing LED lights. I learned a fair amount about eliminating grain in Vray on this project. Rendertimes were fantastic considering the complexity. We were using full 360 HDR light rigs with GI, glossy reflections, glossy refractions, translucency, 3d motion blur and DOF all rendered in camera. The longest frame times were just under 4 hours for shots where the character completely filled the frame. Most shots were 45 minutes or less for final renders. The commercial is being shown over seas, unfortunately. I'm not sure when or if it will be shown here in the US.
I'm currently working on a commercial for Sobe which will be shown durring the super bowl. This project is pretty intense. We're nearing the end though. I can't give any details about it yet, but it is being animated in Maya and rendered out of Lightwave. I was a little worried jumping back into Lightwave after being out of it on the LG spot for a couple months. Suprisingly, it all came back to me pretty quickly. We're about a week away from delivery and unless there are some last minute "gotcha's" we should deliver right on time. Now that I said that I'm sure to get a big list of notes ;) .
There is a pretty good chance on the next project I'm on I'll be modeling the main character. Should be interesting considering I'm more of an environment/lighting guy. I'm learning that Blur's artists are a lot more specialized than here at DD (at least in commercials). A generalist here really does do some of everything.
I'm currently working on a commercial for Sobe which will be shown durring the super bowl. This project is pretty intense. We're nearing the end though. I can't give any details about it yet, but it is being animated in Maya and rendered out of Lightwave. I was a little worried jumping back into Lightwave after being out of it on the LG spot for a couple months. Suprisingly, it all came back to me pretty quickly. We're about a week away from delivery and unless there are some last minute "gotcha's" we should deliver right on time. Now that I said that I'm sure to get a big list of notes ;) .
There is a pretty good chance on the next project I'm on I'll be modeling the main character. Should be interesting considering I'm more of an environment/lighting guy. I'm learning that Blur's artists are a lot more specialized than here at DD (at least in commercials). A generalist here really does do some of everything.
Wednesday, November 19, 2008
Good example
I recently came across a great example of a situation that can happen when compositing that I thought I'd point out.
http://mymentalray.com/forum/showthread.php?t=1491
Basically, the example shows a limitation when working with RawLighting passes. The compositing math for this example is being done correctly, but because of the anti-aliasing you cannot composite these passes back together again and get the same image that you would get out of your 3d package in one pass. In some cases you can get away with having this problem in your comp and not see it. This thread shows a great example of the problem really showing its ugly head.
There is a work around that would allow you to create a raw lighting pass (if you wanted one) that would multiply over the diffuse pass and create the indentical output from your 3d package. To do this you need to render a lighting pass (textures and lighting together) and a diffuse pass. You could then divide the lighting pass by the diffuse pass which will give you a RawLighting pass. The difference between this RawLighting pass and if you rendered it from your 3d package is that this method will take the anti-aliased color differences into account. If you then color correct this and multiply it back over your diffuse pass you will get the exact same output as your 3d package would do.
Why would you want to do this? It is less efficient. Your basically rendering extra passes and using extra nodes in order to create the pass you just rendered anyways. You might consider doing this if you wanted to tweak the diffuse pass and have it automatically update down the chain reguardless of lighting changes. You might also want to work this way just cause its easier to actually see your lighting seperate from your textures. For lighting passes this workflow is probably more trouble than it is worth. But it does work.
Anyways, I thought the example was interesting and good to know about.
http://mymentalray.com/forum/showthread.php?t=1491
Basically, the example shows a limitation when working with RawLighting passes. The compositing math for this example is being done correctly, but because of the anti-aliasing you cannot composite these passes back together again and get the same image that you would get out of your 3d package in one pass. In some cases you can get away with having this problem in your comp and not see it. This thread shows a great example of the problem really showing its ugly head.
There is a work around that would allow you to create a raw lighting pass (if you wanted one) that would multiply over the diffuse pass and create the indentical output from your 3d package. To do this you need to render a lighting pass (textures and lighting together) and a diffuse pass. You could then divide the lighting pass by the diffuse pass which will give you a RawLighting pass. The difference between this RawLighting pass and if you rendered it from your 3d package is that this method will take the anti-aliased color differences into account. If you then color correct this and multiply it back over your diffuse pass you will get the exact same output as your 3d package would do.
Why would you want to do this? It is less efficient. Your basically rendering extra passes and using extra nodes in order to create the pass you just rendered anyways. You might consider doing this if you wanted to tweak the diffuse pass and have it automatically update down the chain reguardless of lighting changes. You might also want to work this way just cause its easier to actually see your lighting seperate from your textures. For lighting passes this workflow is probably more trouble than it is worth. But it does work.
Anyways, I thought the example was interesting and good to know about.
Friday, October 31, 2008
Long time
I had a feeling this might happen. No updates in a long time. Things have eased up a bit today and I found some time to give a little udpate.
Gnomon has released my latest training DVD on Multi-pass Compositing. My first project at Digital Domain is on TV and before pretty much every movie now at the theaters. My first project was actually 3 commercials of which I worked on two. The first one is called "Mecha Mesquito" and is a commercial for the Honda Fit model car. The commercial features a bunch of cars modified to look like mosquitos complete with wings and legs made of car parts. They watch the Honda Fit drive by and begin chasing after it in the hopes of sucking the gas tank empty. At the end they become entranced by a building in the shape of a bug zapper and fly into it. The project was done mainly in Lightwave, with just some minor stuff in other packages. There were two other commercials in this campaign. The other I worked on is called "Defense Mechanism" and features the Honda Fit again but in a city with an underwater feel. All the other cars are fish like. The last I did not work on but had lots of bats flying out of the cars trunk.
After finishing that project I went on "overhead" which was a totally new experience for me. Basically you come to work and wait for someone to find work for you. I started to learn Maya in that time, but two days in they found a new task for me. The VFX supervisor on the film "Curious Case of Benjamin Buttons" came to me and asked if I could render the opening shot of the movie in 3ds Max using Vray. I can't give any details about the shot, but this was an experimental pipeline involving converting animation from another package into max on a VERY grand scale. Unfortunately, after two weeks of trying to make it work we ran into memory problems that just could not be overcome. So the shot remained in the orginal software package it was created in and I was put back on overhead. I was pretty bummed cause the shot is very cool and I really loved working on it. I got to see the final shot (rendered in another package) and it turned out fantastic. A part of me still wants to know what it would have turned out looking like using Max/Vray though. It had been looking really good before the memory cap.
After a couple more days on overhead I was put on another commercial project which again I can't talk about. This project is very cool though. I'm really enjoying it. I came to DD wanting to push myself to create completely photo realistic work and this project is turning out to be the closest I've ever come to that goal. Again on this project I'm involved with creating a pipeline for it which has been quite interesting. I've done quite a bit of scripting over the years in Max, but mainly creating tools that only I needed to depend on. I had a very nerve racking but also exciting experience on this project where we were going to be in serious trouble if we didn't have a particular tool created and I was the one to do it. It was a bit challenging to write this tool and the moment that another artists tried it for the first time on a shot......it worked. The few quite seconds before the tool was run for the first time in production was quite an intense moment. Seeing it work flawlessly that first time made my day and since it was the major stubbling point on the project and we had gotten past it made me feel like this project was well on its way to success. At the moment I'm wrapping up the last of the character modeling. Thats right....for those of you reading this who know me...I'm not a character modeler, but that is in fact what I did on this project. I modeled and textured 3 characters actually. They weren't all that difficult though. Mostly hard surface modeling. But it was something different for me. I've modeled only environments for the past 5 years.
Next week we begin rendering shots. I can't wait to see the shots start coming together.
Tim J
Gnomon has released my latest training DVD on Multi-pass Compositing. My first project at Digital Domain is on TV and before pretty much every movie now at the theaters. My first project was actually 3 commercials of which I worked on two. The first one is called "Mecha Mesquito" and is a commercial for the Honda Fit model car. The commercial features a bunch of cars modified to look like mosquitos complete with wings and legs made of car parts. They watch the Honda Fit drive by and begin chasing after it in the hopes of sucking the gas tank empty. At the end they become entranced by a building in the shape of a bug zapper and fly into it. The project was done mainly in Lightwave, with just some minor stuff in other packages. There were two other commercials in this campaign. The other I worked on is called "Defense Mechanism" and features the Honda Fit again but in a city with an underwater feel. All the other cars are fish like. The last I did not work on but had lots of bats flying out of the cars trunk.
After finishing that project I went on "overhead" which was a totally new experience for me. Basically you come to work and wait for someone to find work for you. I started to learn Maya in that time, but two days in they found a new task for me. The VFX supervisor on the film "Curious Case of Benjamin Buttons" came to me and asked if I could render the opening shot of the movie in 3ds Max using Vray. I can't give any details about the shot, but this was an experimental pipeline involving converting animation from another package into max on a VERY grand scale. Unfortunately, after two weeks of trying to make it work we ran into memory problems that just could not be overcome. So the shot remained in the orginal software package it was created in and I was put back on overhead. I was pretty bummed cause the shot is very cool and I really loved working on it. I got to see the final shot (rendered in another package) and it turned out fantastic. A part of me still wants to know what it would have turned out looking like using Max/Vray though. It had been looking really good before the memory cap.
After a couple more days on overhead I was put on another commercial project which again I can't talk about. This project is very cool though. I'm really enjoying it. I came to DD wanting to push myself to create completely photo realistic work and this project is turning out to be the closest I've ever come to that goal. Again on this project I'm involved with creating a pipeline for it which has been quite interesting. I've done quite a bit of scripting over the years in Max, but mainly creating tools that only I needed to depend on. I had a very nerve racking but also exciting experience on this project where we were going to be in serious trouble if we didn't have a particular tool created and I was the one to do it. It was a bit challenging to write this tool and the moment that another artists tried it for the first time on a shot......it worked. The few quite seconds before the tool was run for the first time in production was quite an intense moment. Seeing it work flawlessly that first time made my day and since it was the major stubbling point on the project and we had gotten past it made me feel like this project was well on its way to success. At the moment I'm wrapping up the last of the character modeling. Thats right....for those of you reading this who know me...I'm not a character modeler, but that is in fact what I did on this project. I modeled and textured 3 characters actually. They weren't all that difficult though. Mostly hard surface modeling. But it was something different for me. I've modeled only environments for the past 5 years.
Next week we begin rendering shots. I can't wait to see the shots start coming together.
Tim J
Friday, August 15, 2008
Gnomon release
The Gnomon Workshop has posted in their "coming soon" section news about the release of my next training DVD. It is called "Multi-pass compositing" and deals with rendering passes out of 3ds max using several different renderers and compositing in Digital Fusion. The DVD mainly focuses on the compositing side of things. I really tried to include as much information as I could. I started off with some basics, but then I got into the "meat and potatoes," if you will, about how to work with passes. I feel that over the years I've come across many compositing work flows and issues in production and I made sure to include examples and explanations on how to deal with as many of them as I could on this DVD. I feel really good about how this DVD turned out. Hopefully if anyone reading this checks it out they'll enjoy it too. :)
www.thegnomonworkshop.com
www.thegnomonworkshop.com
One week and counting
I've just completed my first week at Digital Domain. So far its been exciting, frustrating and overwhelming at times. I've been bombarded with a ton of new software to learn, both proprietary and off the shelf. The amount of new software to learn has been the overwhelming part. Each day its gotten easier, but its still a lot to try to soak in all at once. At times its been frustrating because for the past 5 years at Blur I've known the pipeline and tools inside out. Even helped to create some of them. But now I find myself the struggling new guy all over again. So far that has been both exciting and frustrating as I mentioned before. Knowing one pipeline inside out and then being dropped into a new pipeline and software that is completely foreign has been the hard part. Your faced with wanting to produce work at the same level and speed that your used to doing, but you know no matter what that there has to be a period of time in which to learn and your going to be slower until you gain experience with the new tools and pipeline. I'm excited about learning new software and work flows. I'm also excited to get to work on different types of projects and make new friends.
Wednesday, July 30, 2008
Project Challenge: Hawx
So as I mentioned in my previous post. Hawx is finished. So its time to give a little breakdown on it. :)
Hawx is a 30 second television spot for the game. The final output resolution was 1920x1080 HD at 30 fps. We had a total of 17 shots. We had 1 animatic artist, 2 vehicle modelers, 2 riggers, 2 animators, 2 scene assemblers and 2 FX artists on the project. The project was for the most part rendered in Mental Ray. I ended up needing to render a few passes in Vray though. More on that later.
The biggest problem as usual ended up being memory management. The city street shots were pretty heavy on the poly count and textures, but nothing above and beyond the normal range. What made it difficult to render was that we needed to have the characters render in the scene in order to get proper reflections and light interaction on the environment as them move through it. Since the final output res was 1080p we had some very high res texture maps. It only takes a few of those before your in memory troubles (This project was in 32bit so we had the 3 gig RAM limit). It came down to carefully optimizing each shot so that it would be as high quality as possible but still render.
Another issue we came across was one that we originally thought would save us time. In this spot we were re-using some characters and vehicles from a previous project we did. The only difference was that the materials needed to be changed over to Mental Ray from Brazil. So we didn't need to worry about new character models or rigs. Unfortunately, in the end it came out as a wash. The models were over a year old and weren't up to date with our current pipeline needs as far as our tools are concerned. So the animators and riggers fought against that enough that it probably didn't save us any time at all in the long run and caused headaches for them. Sorry guys :(
Scene assembly was pretty straight forward. Most of the shots took place in the air so there was no interaction with the environment to worry about. Renders were pretty fast even at final quality. I think the longest render times I had were about 1.5 hours per frame at final quality. That was for some closeup shots of the pilot in the cockpit.
The aerial shots turned out to be simpler than I was originally anticipating. For the most part we got away with some satellite photos mapped onto a rough landscape model. Then we sprinkled in some models of the taller buildings, but left the shorter ones as just part of the aerial photo texture. This map ended up being HUGE. I broke it into two maps cause it was just too big. Each of the two texture maps was 19,000 pixels square. I could just barely get these two maps to render by themselves on the landscape model. So I spent some time looking into ways of dealing with extremely large texture maps. Which lead me to tiled .map and .exr files. In the end I ended up not needed to use them and just rendered as normal, but what you can do with them is really impressive so I plan to start making use of them more often in my work.
The beauty of a tiled .map or .exr is that Mental Ray can look at these texture maps and know if they are tiled or not. When I say "tiled" I don't mean you take a texture map and repeat it a number of times. Tiled in this case means that within the texture map are smaller defined areas or "tiles." When these types of texture maps are used in your scene and Mental Ray finds them it does not load the whole texture map into memory...Instead it only loads the tile it needs in order to render the current bucket. Then when its finished rendering that bucket it dumps the memory and loads the next tile it needs. So you have almost no Ram overhead, and it seemed in my case to actually initialize and render the scene faster. It is also possible to have these .map or .exr files store multiple resolution texture maps inside them. This way if Mental Ray knows that an object is far away it can load a lower resolution version instead of the full res one. The downside to this is that it requires a lot of hard drive space. These texture maps get huge. Mine were around 400 megs each. Another downside is that it takes some manual setup to create the tiled texture maps. For that I found some dos based executables on the OpenEXR website which allow you to convert texture maps to tiled texture maps. But its not a simple "save as" menu, you have to actually type out some basic code in a certain format. Totally worth the trouble though if you have a lot of highres maps you need to render. I found that .map files worked just fine in Max 2008, however, exr's did not work properly. The memory was not being handled right in mental ray. In max 2009 the exrs worked beautifully. FYI
In the end the project went very smoothly. The clients were great, the project delivered on time and on budget. There were hardly any long work days and nobody pulled any all nighters.
Subscribe to:
Posts (Atom)