I received a request recently to have a post about how to manage large scenes. I thought that was a great idea for a topic so here goes.
Poly Count
Once you force yourself to follow good modeling habits then poly count almost stops being an issue except on rare occasions. The most important thing you can do on a high poly count scene is to instance objects whenever possible. This just makes sense. If you can use the exact same object in other areas in your scene then you should instance it. Not only will it be easy to update one model and have all the instances update as well, but it saves a huge amount of memory when rendering and also makes your file size a lot smaller. There is a downside to instances though. When you start to have a lot of them. Like around 5000 objects and up. Its great for getting your shot to render and saving the file, but working in the file becomes really slow. You have to make a trade off. If you have some RAM to spare with all your instances then often it will help to grab a bunch of the more simple objects (for example a bunch of grass blades) and break their instances and attach them into one single mesh. You now have one much larger poly count object which will take up more memory at render time, but you have sped up how interactive max is while your working considerably. Always do this with simpler geometry though. You have to make sure that what your attaching won't be so huge that you run out of memory again.
Another great way to deal with huge scenes is to have separate scene files for the background, midground and foreground. Then composite renders together to create the final seamless shot. I worked on a project once (which unfortunately I still can't talk about about and probably won't ever be able to), but it involved a huge jungle campsite. This cinematic went for about 3 minutes straight without a single (visible) cut and all took place in first person. The scene for this was so expansive and huge because the main character (you) run from one end of the camp to the other so it became nearly impossible to cheat things unless they were really far away. We ended up needing 5 separate environments that all fit together like a puzzle that were rendered separately and composited back together. If you gotta do it then you gotta do it.
Textures
Textures have always been the trickiest to balance. I always want the highest possible res textures I can get up close to camera and as things go back I'll make them smaller and smaller. But even with a lot of planning this can be difficult. Especially when you have a lot of different types of surfaces and maps that are close to camera. Once the texture is loaded into memory then max is going to treat them all the same, so you want to feed the highest quality image you can with the smallest file size. Most of the time for me this is just a simple jpg. As long as you don't have visible artifacting in the compression then your fine in most cases. I have also been known to use png textures with indexed colors. This keeps the file sizes lower and if you do it carefully you get almost no visual drop in quality. I stick with 8 bit textures for almost everything, except for objects that need displaced. For those maps I use 16 bit. Just so were all clear about this...You can NOT take an 8 bit texture, convert it to 16 bit in Photoshop and use that. displacement maps need to be generated in 16 bit or higher. In the end the best thing to do is to just be very picky about which textures are important to be high res and which won't be.
Displacement
I have used displacement in Vray, Mental Ray and scanline. Scanline was always predictable, but you aren't able to render millions of polygons since the renderer isn't designed to deal with that. That said I still use max's own displacement for some things. The displacment world space modifier is great. You can create very detailed and still fairly optimized meshes with it. Its not usable for deforming meshes though. I have a long history with Vray, and will admit that I was very let down by Mental rays displacement. Its better than using max's modifiers, but at least back in max2008 there was a lot of memory flushing problems. I found that I couldn't get more than a couple heavily displaced objects in my scene before I was out of RAM. This was a big problem for me since I was creating a big rocky landscape at the time. In the end we had to customize every single shots displacement settings just to get it to render. I was never happy with the level of displacement we had to settle for. I haven't used MR again for a while so I'm not sure if its any better in max 2009 or 2010. Vray on the other hand has never let me down when it comes to displacement. You can load it up all over your scene and it will render. Maybe not fast, but it will finish. Vray handles dynamic memory flushing very well. Vray also has a couple options for how to calculate displacement. So if one way isn't working for you, you can easily switch to another method. I like to use 2d style displacement in Vray because its very fast and predictable, but it does reach a point where it requires to much memory. So in those cases I switch over to 3d displacement which will still take a texture map to drive it, but the calculations are handled differently and you can use a lot more displacement in your scene at the expense of render time.
Lighting
Its been a while since I've had problems with optimizing lighting. So much of what I do now is all HDR lit and doesn't have more than a few extra lights....If any. I'm currently rendering an interior 3d scene for a commercial lit with only an HDR outside the room. No lights at all. But there are still many projects that will require lots of lights. Memory wise its better to raytrace your shadows, but that is also slower. Shadow maps are great, but you have to be careful balancing them. You can run out of RAM really quick. Never use an omni light unless you have to. Omni's project six shadow maps where a spot light only projects one. So you can save yourself lots of ram by choosing the right light for the job. With shadow maps there are three parameters that are important to optimize. First is the bias. This one is kinda worthless most of the time. It doesn't really affect memory or rendertime but it will affect the look of the shadow. Lower values are best. I usually go with .01 for everything unless an issue comes up. More important are the Size and Sample Range. Size is going to determine the detail of the shadow. The higher the value the sharper the shadow. The sample rangle will blur the shadow and soften out any jagged edges. I've seen people balance these two numbers wrong a lot. The size is what is going to use up the most RAM, but the samples are going to cause the bigger render hit. If you want a soft shadow, don't turn your map size way up and then crank the sample range up to blur it out and make it looks soft. Instead keep your samples at the default 4 and adjust your map size until its as close as you can get it to the final result you want. Then slowly turn up the sample range. I have never needed to go above 16 for the sample range. I've never gone above 4096 for a shadow map size either. Just like with modeling. Instance your lights as often as you can.
Rendering
Know your renderer inside out. Setup a simple scene that has a little of everything in it and then start spinning values and really get a feel for what they do. It can be as dramatic as knowing one value that needs to change to take a render from a couple hours to a couple minutes, IF you know the right thing to adjust. So get to know what everything is really doing and how it works. With large scenes your probably going to have longer render times just because theres more in them. Be really smart about your test renders. Find VERY low quality settings that will render extremely fast. Even if the render quality is very poor. This is where I start, and remain until almost the end of a project. Every once in a while I'll crank settings up just to make sure something isn't messed up, but keep everything low for as long as possible. Most of the time you really just need to be able to see what is happening in the shot and all the noise and mess can be dealt with later once you know everything is working. Use region renders as much as possible. Once things are coming together and working I'll usually select one shot that best represents the sequence and push that one through to make it final. This shot will become the master shot. This way only one shot on the render farm is going to take any time and everyone else can keep working with fast renders until the look, and which passes will be needed, are all figured out. This way you also learn what settings are likely going to be for final shots and how long they will take to render. This will help you predict how much time your going to need to render all the final shots.
I think that just about does it. :)
