Visibility Testing?

[l-spiro]

I would really like to prune invisible objects from my exports.

This is my code for testing visibility:

CODE
// Is a DAG node visible?
bool MPI_CharExport::isVisible( MFnDagNode &mfnDag, MStatus &msStatus ) const {
    if ( mfnDag.isIntermediateObject() ) { return false; }

    MPlug visPlug    = mfnDag.findPlug( "visibility", &msStatus );
    if ( MStatus::kFailure == msStatus ) {
        MGlobal::displayError( "MPlug::findPlug" );
        return false;
    }
    else {
        bool bVisible;
        msStatus    = visPlug.getValue( bVisible );
        if ( MStatus::kFailure == msStatus ) {
            MGlobal::displayError( "MPlug::getValue" );
            return false;
        }
        return bVisible;
    }
}

// ================================
// Then using it like this:
// ================================
    for (; !idDags.isDone(); idDags.next() ) {
        // Attach a function set to the object.
        MFnMesh fmNode( idDags.item() );
        // Skip history items.
        if ( !fmNode.isIntermediateObject() ) {
            MDagPath dpPath;
            if ( MStatus::kFailure == idDags.getPath( dpPath ) ) {
                MGlobal::displayError( "MDagPath::getPath" );
                sStatus = MS::kFailure;
                break;
            }
            MFnDagNode dnVisTester( dpPath );
            MStatus sVisStatus;
            if ( isVisible( dnVisTester, sVisStatus ) && MStatus::kSuccess == sVisStatus ) {

                writeString( fmNode.name(), oStream );

                if ( !exportPolies( idDags.item(), oStream ) ) {
                    sStatus = MS::kFailure;
                    break;
                }
                MPI_SkinData sdSkin;
                if ( !sdSkin.extractSkin( dpPath, fmNode.name().asChar() ) || !sdSkin.exportSkin( oStream ) ) {
                    sStatus = MS::kFailure;
                    break;
                }
            }
        }

    }

This matches the standard way I see in all tutorials and even in the examples provided with the Maya SDK.

I am using Maya 9. Is there a different way to do it with Maya 9?
My results now are that I get all kinds of geometry in my exported file that I do not see in Maya 9 (visibility tests always pass).
For example, in Maya 9, this car I downloaded is just a car. But in my exported file there is also a driver sitting in the seat!

Thank you,
L. Spiro

[joojaa]

Visibility is a tricky thing, because you can mean so many things with it. What maya considers visible and what 2 separate user do can be wildly different. For example view ports can isolate certain parts of the scene. because the user now only sees those are the rest invisible? (theres no right answer, on the other hand you could say yes because the user might not be aware of them, on the other hand they MIGHT be terribly aware that this should not be hidden but is just a view port filter, simply because he has 2 view ports up)

You can mean, whets visible in this panel, whats in this layer etc. etc.

So you should define what YOU mean by visible, is it what maya renders or is it something else.

Well you are missing one of many fickle details here!
[li] you neglect to test whether or not the direct descendant is visible or not. Because you dont recurse you dont necceserily get to see that very effectively. But suffice to say if ANY of the parent nodes is hidden so is the child. This is a very important part of the deal imagine
CODE
xform - xfrom2 +- Shape1
               |
               +- Shape2

now if xform is hidden so is, xform2, Shape1 and Shape2! Even tough they themselves dont set visibility*. In other words maya recurses the dag and terminates on each ending condition.[/li][li]isVisible does test the node but not the settings users are in. For example you neglect to consider whether or not the object is in a visible layer[/li][li]you need also test lodVisibility attribute if exists, also the overrideEnabled and .overrideVisibility pair, also chek if they are in bb mode etc.[/li][li]Then theres the rendering flags wich can make things invisible tough they are visible in the gui [/li][li]Then theres the view port filters wich can be set things invisble for that view only[/li][li]Drawing overrides can make things invisble tough they do show up in render.
[/li][li]Its possible to add a shader that makes the object invisible.[/li][li]It is possible to hide things behind the far and near clip plane[/li]So simply what PART of invisible do you mean. What maya considers render invisible or what happens to visible to the user? Or what. So in maya theres several different things you could call invisible thet dont combine to one cear answer thats tghe true one.

At the end of the day you cant really be sure.

*alas its not EXACTLY that easy a user MIGHT override otherwise with a viewport filter.

[l-spiro]

My definition of what is visible is straight-forward: what I can see in my viewport and what renders is what is visible.

I frequently download models for testing purposes. Using the car as an example, there are no special settings on the viewport. It also renders with no driver. I would never know there was a driver there at all if it did not magically appear in my exported file.
And after traversing all the hierarchies I could find in all of the editor windows in Maya 9, I was never even able to find this mystical driver anywhere in the Maya project.

Whatever the definition of visible, I can assuredly say that this driver is not within it.

Wouldn't traversing the list using MItDag::kDepthFirst fix the descendant problem?

I really do not know how to test for any of the things you mentioned. I do not even know how to walk backwards up the tree to see if parents are hidden, but that may be the solution here.

P. S.: How can I find this driver in my Maya project?

L. Spiro

[joojaa]

Well your definition has a few caveats.

QUOTE
renders is what is

In that case you need to also check for the primary vivibility attribute. Then if object had a shader attached to it.
QUOTE(L. Spiro @ 05/26/09, 05:24 PM) [snapback]308368[/snapback]
Wouldn't traversing the list using MItDag::kDepthFirst fix the descendant problem?

No, not really. You still only make local decisions. ask for the value of the child you would need to store the date form item to item, and know when not to store it.
QUOTE
I do not even know how to walk backwards up the tree to see if parents are hidden, but that may be the solution here.

No point,, recurse yourself or you could use MItDag:: prune if something is not visible

Also your skipping of histry objects can cayuse a second very weird and extremely rare occurrence where the child of that shape is visible.

[l-spiro]

** UPDATE **
Nevermind; I got it. I can now avoid exporting invisible objects (finally!).
I use pruning, and my isVisible() test does work, but if you see any conceptual errors or gotchas that I missed do feel free to point them out.

I really can not prune invisible objects.

ATTEMPT #1:
I tried prune() using the same visibility test that I showed earlier.
CODE
    for (; !idDags.isDone(); idDags.next() ) {
        // Attach a function set to the object.
        MFnMesh fmNode( idDags.item() );
        // Skip history items.
        if ( !fmNode.isIntermediateObject() ) {
            MDagPath dpPath;
            if ( MStatus::kFailure == idDags.getPath( dpPath ) ) {
                MGlobal::displayError( "MDagPath::getPath" );
                sStatus = MS::kFailure;
                break;
            }
            MFnDagNode dnVisTester( dpPath );
            MStatus sVisStatus;
            if ( isVisible( dnVisTester, sVisStatus ) && MStatus::kSuccess == sVisStatus ) {

                ...
            }
            // ********* NOW I PRUNE *********
            else {
                idDags.prune();
            }
        }

    }

The result is exactly the same. This means either prune does not work or isVisible() always returns true.

ATTEMPT #2:
So I tried to modify isVisible() (but without using prune()).
CODE
// Is a DAG node visible?
bool MPI_CharExport::isVisible( MFnDagNode &mfnDag, MStatus &msStatus ) const {
    if ( mfnDag.isIntermediateObject() ) { return false; }

    MPlug visPlug    = mfnDag.findPlug( "visibility", &msStatus );
    if ( MStatus::kFailure == msStatus ) {
        MGlobal::displayError( "MPlug::findPlug" );
        return false;
    }
    else {
        bool bVisible;
        msStatus    = visPlug.getValue( bVisible );
        if ( MStatus::kFailure == msStatus ) {
            MGlobal::displayError( "MPlug::getValue" );
            return false;
        }
        if ( !bVisible ) { return false; }
        if ( mfnDag.parentCount() == 0 ) { return true; }

        // If all parents are invisible, we are also invisible.
        // If any parents are not, we are not.
        unsigned int uiParents = mfnDag.parentCount();
        MStatus msThisStatus;
        for ( unsigned int I = 0; I < uiParents; ++I ) {
            //mfnDag.parent( I, &msThisStatus ).
            MFnDagNode dnParent( mfnDag.parent( I, &msThisStatus ), &msStatus );
            if ( MStatus::kFailure == msThisStatus || MStatus::kFailure == msStatus ) { return false; }
            if ( isVisible( dnParent, msStatus ) ) {
                if ( MStatus::kFailure == msStatus ) { return false; }
                return true;
            }
        }
        // No visible parent found.
        return false;
    }
}

Apparently I do not know how to recurse backwards up the nodes, because this always returns false.

Can you point me in the right direction?

L. Spiro

P. S.: Where does MGlobal::displayError print? I have never seen any error messages in any windows in Maya, although I print some before and after my export begins and ends (respectively).

[joojaa]

QUOTE
This means either prune does not work or isVisible() always returns true.

It probably always returns true. Your still only checking for just 20% of all invisible. You don't check layers at all, Wich would probably in reality account for 60-80% of all visisbility cases. So are there any display layers in the scene?
QUOTE
Where does MGlobal::displayError print? I have never seen any error messages in any windows in Maya,

Well what windows did you check for? Output window and script editor (note no they re NOT the same thing, the other is maya standard out and th other is whats in maya window) are both good choices for sanity cheeking. But then theres the slight thing of checking if you have errors suppressed.