Maya And Boost

[paolog]

I have always used Boost (see www.boost.org) in my C++ coding and find it indispensable. Now, however, my supervisor (who is familiar with Maya but not with Boost) is suggesting I ought to dispense with it, which I am loath to do.

In particular, I make liberal use of Boost's smart pointer library, which automatically handles memory allocated on the heap for you, so that you do not need to work out when to delete it. The library works, has been thoroughly tested and is used worldwide in many applications without any problems.

I am creating a C++ plug-in. Attributes that are arrays are created using MObject. Arrays internal to the node are managed using Boost's smart pointers. However, the creator() function has the following syntax:

void* MyClass::creator(void)
{
return new MyClass();
}

Questions:

1. Is this correct?
2. What is responsible for managing this memory?
3. Is a delete needed anywhere? The constructor is currently empty.

The behaviour I am seeing is that the node is created correctly, but causes a segmentation fault after having been called a seemingly arbitrary number of times in Linux and causes Maya to hang. The node persists from call to call. In Windows, there seems to be no problem.

My supervisor suggests that, as Maya uses its own scheme for memory management, Boost's smart pointers might somehow conflict with this and cause a memory leak. I find this difficult to believe, but if it is the case, then so be it.

Incidentally, a plug-in that he and I worked on once before used ordinary C++ management and works without any problems. I could replace all the smart pointers with bare pointers, but as it is a possible memory leak I am trying to fix, this is likely to make things worse or obscure the problem.

Does anyone know for sure whether or not Boost is incompatible with Maya? I have asked a similar question on the Boost user's mailing list.

Alternatively, am I making a basic error in my plug-in that is easily fixed?

Thanks for any feedback.

[joojaa]

Wrong forum so i moved you here!

QUOTE
My supervisor suggests that, as Maya uses its own scheme for memory management, Boost's smart pointers might somehow conflict with this and cause a memory leak. I find this difficult to believe, but if it is the case, then so be it.

Well inside your plugin your free to use memory as you wish. However the fundamental rule about maya api is that maya owns everything and you own none of it. So basically you never really get to allocate memory anyway, maya does it for you and incidentally frees it for you too. As long as you use a data block as your intended. (if you don't then maya wont save the info).

You can do whatever you please within your node but you cannot really communicate maya without using mayas data block. So if you have some persistent data you should be using datablock, this way a maya user also has access to the data.

Also maya auto releses stuff it no longer needs, the only thing you need to release is whatever you built for yourself. (wich should be limitted to a runtime only accelerator scheme) This is also why you cant store pointers to data becasue they get shuffled about and the mdatablock is the smart pointer.

The automatic freeing is possibly dangerous here, because if maya unloads your plugin mid run and you dont give your smart pointers time to release they might find themselves being in a state where they are no longer gotten time to compute but are not released, no matter how smart they are. Because maya terminates the plug also, not you. So in asense youd need to release theese pointers manually anyway unless you are ina habit of clearing your poinetrs between every run in compute.

So set just put everything in datablocks this way you have neither have to worry about allocation of memory arrays, manging it, nor freeing it.

[paolog]

OK, thank you. That's very helpful. Thanks for moving my message to the right forum too.

When you say "datablocks" do you mean that all the data should be of type MObject? Or is "datablock" a concept or data type for use in the code?

Here's a summary of the current interface between Maya and the plug-in:

In the MEL script:

1. In the procedure called from Maya, load the plug-in if it has not already been loaded, and create the UI for my program.
2. When the user clicks a certain button in the UI for my program, create a node for the plug-in.
3. When the user clicks another button in the UI, call the node multiple times, once for each frame in the playback range.
4. Each call to the node sets values of the input attributes of the plug-in and then requests values of the output attributes.
5. The node is not deleted anywhere in the script (and so gets deleted when a new scene is created or Maya is shut down, I presume).

All fairly sound so far, I think.

In the plug-in:

1. The class contains public, static MObjects to store the input and output attribute values to be passed in and out.
2. For each of these there is a corresponding private, non-static member variable in the class. Some are smart pointers (for example, boost::shared_array x), and one is a valarray of doubles. Aside from arrays, I have an object of type std::pair<double, std::valarray >.
3. In the constructor, I allocate memory on the heap for arrays using smart pointers (for example, x.reset(new double[10])), and also set the size of the valarray (using resize()).
4. In the destructor, I resize the valarray to 0 to deallocate the memory for its contents.
5. The creator() function returns a new instance of the class. Standard stuff, I think. I take it from what you say that Maya takes responsibility for managing this memory.
6. The initialize() function creates all the attributes (all the MObject variables) and sets up the dependencies between inputs and outputs. Again, standard stuff.

7. The readData() function copies the values of all the input attributes into the private (non-MObject) member variables of the class. For example (error-checking omitted):

   MStatus stat = MS::kSuccess;
   MDataHandle handle;

   handle = data.inputValue(maya_my_var, &stat); //maya_my_var is of type MObject
   my_var = handle.asInt(); //my_var is of type int

8. The compute() function calls readData, then uses the private member variables to compute the results, then calls a function to copy the results into the output attributes (pretty much the reverse of step 7), and finally calls setClean on the plug passed to compute(). For example:

   handle = data.outputValue(maya_result, &stat); //maya_result is of type MObject
   handle.set(result); //result is of type double (say)

9. There are also the initializePlugin() and uninitializePlugin() functions, of course.

What I would expect to happen is that the memory pointed to by the smart pointers and used by the valarray is deallocated when the destructor is called, but clearly this is not how it works.

So what are the incorrect steps above? Presumably I need to copy the data in and out (steps 7 and 8), but should not be using smart pointers... is that right? Should I allocate memory on the stack instead? I could also unwind my std::pair<double, std::valarray > object into a double and a valarray, or even a double and an ordinary array if need be (the internal computation uses a valarray but I could always make it return an ordinary array).

Could you explain how I use a datablock to achieve what I need to?

Many thanks for your feedback.

[joojaa]

just one question.

Are the double pairs really intended to be flushed when you exit maya? Because maya only stores the memory it is taking care of. Anyway in many ways the maya memory management is much better than the one you get here. Because maya not only stores the memory it also arranges it for fast computation. So when maya detects it needs your node it moves the entire data into a continious fast to pick up range. And then when it no longer needs it puts it away, now your smart pointers don't do this.

Also remember the smart pointers do really rely on you to release them here. So you dont handle the memory indices and stuff but still. In many way the datas you get form maya are very similliar.

[paolog]

All of the memory I allocate in the node is used just to compute the results, which are then displayed by my script. However, the plug-in is called many times, and I need the memory to be there still so that I can do the compuation at each call.

When my Maya application (or Maya itself) is closed down, all of the memory allocated by the plug-in can be deallocated.

So, is it safe for me to use bare pointers instead of smart pointers, allocate memory in the constructor or initialize and then leave it up to Maya to deallocate it? That is, should I not deallocate the memory in the destructor or anywhere else?

[joojaa]

QUOTE(Paul G @ 04/18/07, 05:49 PM) [snapback]264343[/snapback]
So, is it safe for me to use bare pointers instead of smart pointers, allocate memory in the constructor or initialize and then leave it up to Maya to deallocate it? That is, should I not deallocate the memory in the destructor or anywhere else?

No, in general when you program with maya you don't store pointers at all, you just ask for a handle to a mdatablock which returns you a usable data handle which acts as a temporary pointer to your data. You then flush the pointer and ask it again when you come to next cycle of compute. This si akin to using smart pointers, maya just calls them datahandles.

(remember you have no data at all you have just pointers that point int a space that does not stay constant)

However the smart pointers dont really work like you expect, you do need to unallocated them when you no longer use them, they will offcourse in normal apps get terminated on exit and if you run the same strand of code later thet will do flush eventually. But maya wont, it just terminates al code that the smart pointers are using for possible cleanup.

[paolog]

Thanks, joojaa. I'll look into using datablocks instead of smart pointers.

[paolog]

[quote name='Joojaa' date='04/18/07, 09:15 PM' post='264374']

No, in general when you program with maya you don't store pointers at all, you just ask for a handle
to a mdatablock which returns you a usable data handle which acts as a temporary pointer to your
data. You then flush the pointer and ask it again when you come to next cycle of compute. This si akin
to using smart pointers, maya just calls them datahandles.

I'm still not quite clear about this. Am I able to use my own classes, or do I have to unwrap everything and use Maya's data structures, which are more limited?

At present, in readData, I read the values of the input attributes and store these in the member variables of MyNode. Now, from what I understand, these variables do not necessarily persist from one call to compute() to the next, so I should not allocate memory dynamically in the usual way or use smart pointers for variables internal to the node, but use handles and MDataBlocks instead. Could you give me (or direct me to) an example, please?

[joojaa]

QUOTE(Paul G @ 04/23/07, 12:44 PM) [snapback]264637[/snapback]
I'm still not quite clear about this. Am I able to use my own classes, or do I have to unwrap everything and use Maya's data structures, which are more limited?

Yes you can use whatever you want but you must do manual cleanup. Remeber smart pointers or not they must be deallocated. When you do standalone code your constantly asking things with your smart pointers that gives them time to cleanup and manage themselves. But maya doesn't give them any extra time once it shuts down all expressly unallocated stuffs get leaked. In a way your asumption of that the pointers clean up after themselves is only partially true they only clean up on delete. So at no point should you ever assume anything about code inside maya. And its termination time cleanup must happen correctly because your code ending doesn't mean maya ends. In fact your code ends a million times per second inside maya in sense. So you allways start with clean slate.

Anyway Maya data structures can be allmost as complex as you like. (well they cant have back pointers) , and theres nothing that states you couldn't define a tree structure with maya data.

read :

http://www.highend3d.com/boards/index.php?showtopic=236530

Then read mayas manual.

But basically if you new something in the creator you must delete it in the destructor. Something you cerate in compute must be destryed in compute or in destrctor (in wich case the only benefit you get form smart pointers in not to keep track of what you need to destray here just destroy the smart pointrer instead but make no mistake it needs to be destroyed,o rin case of shared pointers all of them), again remember compute gets called alot by maya. Also remember maya allready hasdata in datastructires so dont try to implement what you allready have. (like voxeltrees or bsp trees), thet would be double bookkeeping.

[paolog]

OK, I have now eliminated smart pointers from my code. Fortunately the sizes of all of my arrays are known at compile time, so I have been able to declare the arrays in my header files as (for example) int a[SIZE], where SIZE is some enumerated-type value.

I also noticed that I was using std::cout, which I remember can cause conflicts. I've changed all my std::couts to std::cerr.

I noticed this because of the stack trace I was getting, that suggested it could be to do with string output. I am still getting the same stack trace, even after making the above changes. I get this when control returns from my stand-alone function. Does it suggest anything in particular?

maya encountered a fatal error

Signal: 11 (Unknown Signal)
Stack trace:
/lib/libpthread.so.0 [0x45a155d9]
/lib/libc.so.6 [0x45a81d78]
/lib/libc.so.6 [0x45ac87dd]
malloc
XtMalloc
/usr/aw/maya7.0/lib/libXm.so.3 [0x4566b879]
/usr/aw/maya7.0/lib/libXm.so.3 [0x4566ce02]
_XmTextShowPosition
_XmTextSetCursorPosition
XmTextSetInsertionPosition
Tfield::append(Tstring const&)
TcommandWindow::appendToHistoryField(Tstring const&)
TcommandWindow::display(Tstring const&)
TcompositeCommandReporter::display(Tstring const&)
sophia_message
f_sprint
node_exec
fc_while
node_exec
f_function_entry_node
node_exec
f_function_entry_node
node_exec
sophia_call_executable
SophiaExecutable::evaluate(void*)
TcommandEngine::executeCommand(Tstring const&, bool, bool, TmelCmdResult*, unsigned,
int)
TsubstituteScriptAction::substituteAndExecute(Tevent const&, Tstring*, Tstring*,
Tstring*, Tstring*, Tstring*)
TcontrolScriptAction::doIt(Tevent const&)
Tcontrol::doAction(Tevent const&)
Tbutton::activateXtCB(void*)

(By the way, joojaa, could I ask you to run your replies through a spellcheck before you send them, please? I find them quite difficult to read sometimes. Thanks.)

[joojaa]

QUOTE(Paul G @ 04/23/07, 05:19 PM) [snapback]264668[/snapback]
(By the way, joojaa, could I ask you to run your replies through a spellcheck before you send them, please? I find them quite difficult to read sometimes. Thanks.)

i do run them but you are not on the right wavelength anyway

just a question why why would you bother with all this if your own external implementation intend to use arrays? whats wrong with one of the following?

MColorArray
MDagPathArray
MDoubleArray
MFloatArray
MFloatPointArray
MFloatVectorArray
MFnArrayAttrsData
MFnDoubleArrayData
MFnIntArrayData
MFnPointArrayData
MFnStringArrayData
MFnUInt64ArrayData
MFnVectorArrayData
MIntArray
MMatrixArray
MObjectArray
MPlugArray
MPointArray
MRenderLineArray
MStringArray
MTimeArray
MTrimBoundaryArray
MUint64Array
MUintArray
MVectorArray

So why would you try to implement something way beyond your skill set in first place? especially when theres a much more natural way. Anyway if you cant bother to read the api manuals yourself the entire point is moot. Because no one can help a person who's not willing to help himself? So either post your source, because your not demonstrating any progress here, and its jus ta matter of time till i label you as a lost cause.

Anyway i don't think your problem has to do anything with the smart pointers. Just something really fundamental

PS. Amway im beginning to see why the computer engineering majors were first introduced to scheme as a language to learn the vocabulary of programming to reset all the previous bad habits

Anyway im looking into your stack trace.

[paolog]

QUOTE(Joojaa @ 04/23/07, 05:13 PM) [snapback]264680[/snapback]
just a question why why would you bother with all this if your own external implementation intend to use arrays? whats wrong with one of the following?

MColorArray

Because I am unware of the existence of these and don't see why I would need to use them anyway. Surely Maya can cope with a declaration like float arr[10]? Do I really need to use MFloatArray arr[10] instead in my C++ code? Wouldn't this just be a wrapper around a float array in any case?

QUOTE(Joojaa @ 04/23/07, 05:13 PM) [snapback]264680[/snapback]
So why would you try to implement something way beyond your skill set in first place? especially when theres a much more natural way. Anyway if you cant bother to read the api manuals yourself the entire point is moot. Because no one can help a person who's not willing to help himself? So either post your source, because your not demonstrating any progress here, and its jus ta matter of time till i label you as a lost cause.

I value your help but don't appreciate having assumptions made about me, thank you.

To answer your questions:

• Q: Why implement this? A: Because my employer asked me to. If there is a more natural way, then please let me know what it is.
• Q: Why aren't I reading the manuals? A: I am reading the manuals, but I haven't found the answer I require there; hence I'm posting a question here in the hope that I'll get the answer more quickly this way. So I hope you'll see that I am willing to help myself.

I can't post the source as it is company confidential. That is why I have only posted disguised snippets or descriptions of what I am doing. Here's an outline of my source that might be helpful. This is meant for illustration only and is not intended to be compilable.

CODE
enum { NUMRESULTS = 10; }

class MyClass
{
public:    //members are public so that I need not provide access functions in this illustrative example
    int value;
    std::string str;
};

class MyNode
{
private:
    MObject maya_value;
    MObject maya_str;

    MyClass obj1;
};

MStatus MyNode::readData(MDataBlock& data)
{
    MStatus stat = MS::kSuccess;
    MDataHandle handle;

    handle = data.inputValue(maya_value, &stat);
    value = handle.asInt();

    handle = data.inputValue(maya_str, &stat);
    str = maya_str.asString();
}

MStatus MyNode::compute(const MPlug& plug, MDataBlock& data)
{
    MStatus stat = MS::kSuccess;

    if ((plug == output1) || (plug == output2))
    {

        if (readData(data) != MS::kSuccess)
            return MS::kFailure;

        double result[NUMRESULTS];
        bool success = my_stand_alone_function(obj1, result);

        if (success)
        {
            for (unsigned int i = 0; i != NUMRESULTS; ++i)
            {
                std::cerr << "Result " << i << " = " << result[i] << std::endl;
            }
        }
    }
    else
    {
        return MS::kUnknownParameter;
    }

    data.setClean(plug);
    return stat;
}

class MyModel
{
    //various simple types here, and also a variable of type std::pair<double, std::valarray >
};

bool my_stand_alone_function(const MyClass& obj, double* const arr)
{
    MyModel model;
    simplex simplex_method(&model);
    //compute stuff here
    //set elements of arr here
}

simplex makes copies of MyModel objects, so I MyModel has operator= defined that makes a deep copy.

QUOTE(Joojaa @ 04/23/07, 05:13 PM) [snapback]264680[/snapback]
Anyway i don't think your problem has to do anything with the smart pointers. Just something really fundamental

Right, that's exactly what I am thinking now, because I've got rid of the smart pointers.

Note that all my arrays are now fixed-sized arrays declared in header files. All instances of std::cout have been replaced with std::cerr, and all instances of printf have been removed (as the stack trace looks like it might have something to do with strings).

QUOTE(Joojaa @ 04/23/07, 05:13 PM) [snapback]264680[/snapback]
PS. Amway im beginning to see why the computer engineering majors were first introduced to scheme as a language to learn the vocabulary of programming to reset all the previous bad habits

Please don't post smart-alecky comments like these. They don't help. Be assured that my computing qualifications, experience and skills are well above average. It's just I have a lot of gaps in my knowledge of Maya, which is still fairly new to me.

QUOTE(Joojaa @ 04/23/07, 05:13 PM) [snapback]264680[/snapback]
Anyway im looking into your stack trace.

Thank you for your help. By the way, this posting was much more readable - thanks.

[paolog]

PS: I'm researching the stack trace too.

[joojaa]

QUOTE(Paul G @ 04/24/07, 12:37 PM) [snapback]264726[/snapback]
* Q: Why implement this? A: Because my employer asked me to. If there is a more natural way, then please let me know what it is.

Well in that case you should really not be helped for free, tough we do it on highend all the time. Newer mind im always free on higehend Tough probably shouldnt. As my normal consultancy fee is 200 euros a hour FYI. As of now ive spent maybe 15-25 minutes thinking of your problem maybe all in all.

QUOTE
I can't post the source as it is company confidential. That is why I have only posted disguised snippets or descriptions of what I am doing. Here's an outline of my source that might be helpful. This is meant for illustration only and is not intended to be compilable.

yes, thats normal however you are in sense asking the forum to develop this for you so you might as well make a code thats sufficiently generic that we can fix it and build upon it the above is probably sufficent to root out the idea but. That don't mean you need to put your employers secrets in that code. Life aint free even highend end isnt free the rice you pay is divulging what you do (so we can steal it).

Ok just a tip for the future enclose the code inside code tags Anyway you exemplify why you cant just take coders of the shelf and put them in front of the maya api because its about how maya works not how you like to code. But hey now I have to go back to my work. SO ill be back later unless Bateman has time to get it first.

PS how can you be unknown of the existence of the classes if you have read the manual? also why should anybody be interested in your problems if you aren't interested in doing the work for your code?

some over course which you dont need but seem to want anyway:

Anyway embedding your own gui is not very maya like (and this should not be done no matter how great i sounds), in way maya makes this very hard for you since it tries to force you to use mel for your gui. theres stuff in www.secretlevel.com on how to embed mfc stuff into maya.

from their representation:

Dont fight Maya; some things are easier to do with MEL interfaces
Bucking the Maya UI paradigm leads to less productive, confused Artists (i.e. proliferation of pop-up windows)

http://216.239.59.104/search?q=cache:DhxEX...lient=firefox-a

so what your basically doing is a geometry/value cache node of some kind.

[paolog]

Update:

• I'm changing std::string to MString in my node class
• I had an array-out-of-bounds error, which was causing Maya to crash in Windows too. I've fixed that, but it's still crashing in Windows too - I'm looking for any others I might have overlooked.

At least I've got consistent behaviour in the two operating systems now.

[joojaa]

QUOTE(Paul G @ 04/24/07, 01:55 PM) [snapback]264736[/snapback]
* I had an array-out-of-bounds error, which was causing Maya to crash in Windows too. I've fixed that, but it's still crashing in Windows too - I'm looking for any others I might have overlooked.

Hmm... well that would off course crash anypogram. Just out of curiosity you are offcourse you running this into a debugger? At least then you'd know how far it works.

[joojaa]

When you try to split a secondary thread if so how do communicate with maya? I mean the entire api isn't completely thread safe, so you'd best to wait for commands to execute in the main thread. Se the manual more closely on this. Mainly this means you need to call the api calls in your main thread or trough MGlobal::executeCommandOnIdle()

Tough because its so easy to find in the manual you have considered this already.