Very jagged shadows in HW render

[blashfordschnell]

hi all,

for some reason my depth map shadows render absolutely terribly when rendered in hardware. is this my video card to blame (GeForceFX5700LE) or is it something else? i have messed with the depthmap attributes endlessly, and they change it a little bit but do not address the main problem, which as i see it, is that there is no AA on the shadows. see image:

this is driving me crazy. any help would be greatly appreciated.

Thanks!

[failure]

Biger shadow map resolution or smaller ground plane.

[blashfordschnell]

i understand what you are saying, but i don't think that the Dmap res is the problem. maybe this will illustrate it better: here are two renders, HW and SW, both at production value defaults. it is not even close. now i understand that the HW render can't do as nice a job as SW, but it is like it's not even trying..:

[joojaa]

hardware shadows wont blur!

[failure]

It is a resolution problem - you have probably bumped into your gpu's texture resolution limit.

See, when you use dmap shadows, a black and white image ( called depth map ) is rendered from the light's point of view - marking all areas that light cannot "see" as black - therefore in shadow. This image is then mapped onto geometry as a shadow pass.

When such image is created maya first estimate light's viewing frustum and then renders all the geometry that is inside that frustum. In case of directional light is is practically the whole scene.

So if you have a tiny cone on a large ground plane, a very small number of depth map pixels will be used for actual shadow, rest will be "wasted" on the large non-shadowed plane area. Let's see; if the shadow in 1x1 unit in size and is cast on 30x30 unit ground plane and dmap size is 1024x1024 the shadow will be only 34x34 pixels in size. If you blow it up on sd resolution in sure looks jagged. Of course, in practice it's not exactly like this because renderer tries to optimize the shadow frustums, but still this is the root the of problem

With software rendering the depth map resolution can be cranked sky high ( still inefficient in cases like yours though ) but with hardware rendering your graphics card accelerated texturing facility is used to render and apply depth maps. All cards have upper limit on hw texture size ( about 2048x2048 on consumer grade cards ). So no matter what you specify in the dmap size - in actual hw rendering it won't exceed hardware limits.

Only acceptable solution in this case is to optimize your scene:
- use spot lights because they "see" a lot smaller scene area than directional light, and you have control over how much they "see"
- create smaller ground plane near the cone ( in size just enough to hold the shadow ) and position it slightly above large ground plane. Now turn off shadow recieving and casting for the large plane.
- and look into your gpu specs ( or run diag software ) to find out texture resolution limit of your card. It's useful to know such things when you need to do hw rendering.

[blashfordschnell]

hey failure,

thanks for the good explanation. reading it helped me understand dMaps a bit better, and it helped me understand the natural limitations and how to work with them.

here is a small shot of the scene with which i was having shadow problems. having tweaked the positioning and settings of the spot, and added an ambient light to lighten the shadows, i have it at a decent result. the shadows are still a little jagged, but the results are acceptable. i also figured out that if you dupe a light a few times and shift the dupes just a little bit to the sides of the original, and lessen all of their intensities, you can sort of 'fake' a simple blur. this shot does not have that tho:

there is one other problem i have been having, however.. have you guys ever had the problem where you are HW rendering a scene with a lot of stuff in it, and it renders good once, and then the next time you try the same shot the lighting is off and some objects are lighted and others not lighted? i feels like a memory deficiency of some sort, but i am packing 1GB of RAM and a 128 MB card.

thanks again guys, i appreciate your help. take it easy.

[failure]

It's very likely a hardware limitation again - this time on number of hardware lights.

Maya does all of it's hardware rendering ( hw renderer and everything in viewports and editors ) using rendring api called openGL. OpenGL has, by it's design, a limitation on number of lights in can render per frame. And it's 8. It's a bit silly but openGL is rather old and when it was first standardized 8 light limit probably sounded like a good idea.

When OpenGL was introduced not too many cards had a feature called "hardware T&L" ( t and l stands for transform and lighting ). This is a commonplace today and it enbles your card to do all of the vertex transformation and lighting claculations on hardware level at virtually no time cost.

In order to do lighting part of T&L a program need to upload lights data into the card before the frame is rendered. Now, number of lights that card can handle is limited ( to something like 8 or 16, probably more on newer card ) The limit will eventualy die with domination of pixel shader cards - but that's another story.

So if you want your hw rendering to be executed correctly on any hardware, you should respect the OpenGL limit of 8 lights. Even if your gpu supports less than 8 hardware lights OpenGL guarantees you that it will be rendered correctly but in that case needed lights will be calculated by the cpu - meaning a bit of the software rendering calculation - which is slower.

Most of todays gpus handle more than 8 lights and OpenGL can accesss them thru OpenGL extensions - but if you force a large number of light that exceeds hardware limit rachable thru GL extension you can expect all kinds of funky random behaviours - depending on your card, drivers, number of requested vs. possible hw lights, the logic maya hw uses to prioritize the lights etc.

Again, look into specs and try to respect the limits. If you absolutely need a large ammount of lights try to render in multiple passes each using 8 ( or whatever the hw limit of your gpu is ) lights. Then composite the passes using additive function ( screen )

However it may be that this in not the problem in your case. Put the number of lights under control and see if it helps.