Edge Anti-aliasing / Compositing Problem

[mattiasa]

Hi. I want to visualize objects inside a metallic tank by making half the tank transparent in after-effects.
A simpler but equivalent model: I make a nurbs-plane with a red ball hidden under it. The nurbs-plane
is split into two parts with Detach. The two parts are not moved away from each other. In Maya workview and when I render the masterlayer in Maya7 / mentalray, no red shows through where the plane is split - the split plane looks like one solid plane.
I set up three render-layers: the ball, each half of the nurbs-plane. Then I render 100 frames in
mentalray and take the three layers into after-effects. Here I reduce opacity of one
nurbsplane-layer from 100% to 0% between frame 40 and frame 60, so that the ball comes into sight.
Problem is: in aftereffects, black line between the two nurbs-planes shows, before opacity goes down. I want the nurbs-planes to look seamless, like one single plane, before opacity goes down on the one half.
I think the black line comes from a combination of
1)edge antialiasing in maya/mr
2)the way opacity is calculated in aftereffects
I render straight (not premultiplied) tiff-s. I have tried lots of things to get it right.
Best so far is: the two nurbs-layers in maya must both be rendered with black background at the
edge where they will meet (so the red ball, that lies right below the seam, must be on its own render
layer). Then take the three layers into aftereffects. Now is a black seam visible - bad!!. One the
top layer, set blending mode to Add Alpha. Black seam disappears - but not where the red ball lies
below, here a blackish red shines through. Set opacity of ball to 0%. Finally - seam gone. Then
when the ball is supposed to show, opacity of the disappearing nurbs-plane goes down while opacity
of the ball goes up. Finally looks right.

But is there an easier way? The most difficult with the above procedure is to get alle the meeting
edges to render against black in maya. Is it possible to turn off antialiasing on the edges of a
render layer, or on a per-edge (geometry) basis in mentalray?

Kind regards.

[joojaa]

Its not straight versus premulted problem, its just that since you discard date on the way there it don't align anymore. Personally i would NEVER try to do it thisway, if i did i wouldnt aa (aa is signal reconstruction of a oversampled data) at all but rather dunmp the higher order samples in and let the reconstruction happen in comp.

The main problem is that the reconstruction actually assumes something of the nature of your sampled image, however a 1 pixel wide box filter might align perfectly (but numerical imprecision and post value noise might prove otherwise), under special circumstances.

I would rather render a layer for the parted half on top of the full image instead

[mattiasa]

Thanks. I solved it by manipulating transparency in Maya, not in AfterEffects. Its ok, but then I have to use lambert material - because specular reflections etc will still show on a transparent object (with blinn or some other shiny material).

Next time I will try your solution of two planes on top of each other.

Just out of curiosity I had to test the split plane with Box filter that you described. I have the detached nurbs-plane with a red ball below. I render with mentalray (within Maya), to .tif, Straight, no premult. Three renderlayers as before.
In mentalray, first I set Quality Presets = Production. Then I modify:

In order to turn aa (anti-aliasing) off:
Min sample level = -1
Max sample level = 0
(Will this turn anti-aliasing off? how else do you turn anti-aliasing off?)

Filter = Box
Filter width = filter height = 1

I render and take it into aftereffects. Its the same as before: the seam shows (layer blend = normal). When I go layer blend=add alpha, the seam disappears, only not where the ball lies below - here a reddish seam shows.

Did I do this right, or are there any rendering-parameters that should be set differently?
How do you turn off anti-aliasing in mr, and --dump the higher order samples in--?

Just eager to learn some more.

Kind regards