hello
our company will invest in the very near future in a 100 computer renderfarm. now, i am recently new at the company, and my area of expertise has been till now mainly telecomunications so i am kindda new in this 3D field and i have a few questions. if you can't answer, please could you redirect me to some places that hold PRACTICAL info on this subject? thanks
so, here it goes:
1. if we want to use mental ray as the render soft, and we want to use linux (i really don't want to use windows), do we have to have maya installed on each computer on the renderfarm, or only on the main node? if we don't what software has to be used?
2. we will have a renderfarm and a file server. now the renderfarm main node will take the to-be-rendered files from the fileserver, and after completing the job, it will put them back. how is this achieved?
3. do the computers in the renderfarm have to have large capacity hdd-s? according to my logic (which may be naive at least) during the rendering process there are huge files swaped, but what does that exactly mean in figures?
there are many more to go, but enough for one post. if there is anyone who is willing to share from his/her experience in the matter (that is linux based maya network rendering and renderfarm) i kindly ask that person to help me. it seems the practical advice is what i need most and find hardest.
best wishes.
Should just need mental ray installed on each system. You will probably need some 3rd party software to distribute the rendering to each node though. Shouldn't be too hard to come by. mental images probably sells something; or may even have it built into the cost of mray.
Seems reasonable enough to me. Essentially you'll feed your scene data files into the dispatcher; and he'll dole out to each render slave as he sees fit. The render slaves will write the final image output to an nfs mounted volume off of your file server.
Really an 18 gig drive is plenty for a render slave; especially when your storing the final product on an NFS mounted volume.
As for storage needs; depending on the final product you may need 100gb or you may need 6tb. you would know that better than I.
Hello
I've setup a a fiew render farm... Irix and Linux.
You can have both setups. It is a lot simpler to have it on one main node when you want to update.. the problem is network trafic. If you install on the main node, make sure your nextwork is up to it. I've seen this problem many times
Well you can buy software to do this (render dispatchers). Highend has a list of available software. I tryed to get some of the software and ended up writing my own dispatcher.
For the price you can get a programer to make what ever you want. Most software you will find will do the job... Support is where I have a problem with most of the software out there. Dispatcher software is not the biggest selling software on the market. (PS thank you for making some guys). Most of the time it is writen and support by the same guy.. that is also doing other things... This is why I suggest getting a programmer to do a custom just for you. Still ther is some very good software out there with a good team and support... you just have to find them...
Thats the hard part.
3.. the main system disk... fist of all.... make it a fast single disk.. or no disk at all.
Two field of thought.
No1 you can have a disk on the system. You can install maya locally for this you need a disk 
. You can also install project and files that are needed for render. This is a good option if you have a slow nextwork. This solution give you the possibility to have a big SWAP partition. Always make SWAP the exact amout of physical ram you have... or double. If your renders getin to swap generaly things will SLOWDOWN... so put as much ram as you can in your farm.
No2 disk less solution. This very cool for fast network setups.
Boot through a nextwork image that is copied to ram or the local HD. (have a lot of ram I MEEN A LOT of ram solution). With this solution.. you can add and remove render nodes at will with and simple setup.
I have a lot more on render farm setups... but I this is enough for today... best of luck
//oogy
- Well you can buy software to do this (render dispatchers).<<
What about SPIDER for Network Rendering? It used to cost near $10,000 US and now it's free since StationX is closing it's doors and offer the code. I have contacted them in hopes of knowing if they have or will allow porting to Linux. Presently it's for Win2000 and NT.
http://www.stationx.com/index.htm
Also there is NXN Alienbrain Studio for asset management. They support Win2000/NT/XP, Linux and OSX but it is not free. You can download a demo of the software to test it in a studio setting.
you will also need to buy a "mentalRay standalone" server-client software for rendering on your farm nodes (unless you have maya5.0 on all of them which would be rather expensive I recon. If you do its a matter of writing a simple shell script which will render each frame for a maya scene on a different machine and write the output to your farm using the "mayarender_with_mr" command. If you have the standalone variant that using the mental ray to translate a maya file into mi (mental ray default scene file) and than feeding thementalrayrender command this mi file. Its really no rocket science to even write an error savy script which will restart a render on a frame where it crashed. Maya for unix has a command line dispacher program called "dispatcher" but i'm not at my work machine so I can't test if that accepts any mray specific variables.
first of all thank you all for your kind responses.
now, let me see if i got all this straight. the idea is: someone has a file he's been working on, the file is ready to render. the file server holds all these files and it also runs a batch server, or dispatcher, or whatever is called that manages the process of keeping the to-be-rendered files in the queue. file1 is then sent to the renderfarm's main node, that does all the thinking, how to send the files to which computer, what is the optimal load balance etc. after the rendering is complete, the ready file is then deposited on an exported NFS volume on the fileserver.
am i close?
then the mentalray server side should reside on the fileserver, or the renderfarm node?
is there any other mentalray compliant render software besides mentalray? one of the posts said that it would be possible to have the renderfarm clients diskless, if given high-speed network and plenty of ram (gigabit ethernet and 2 GB RAM was what we thought of), but is that possible with mentalray?
thank you again.
best wishes
As was asked... the preference was Linux... Spider doesn't suport linux yet... It's not a solution for the NOW.
NXN looks nice....
When the team over at NXN called me about their software it looks very promising. Offers several handy tools for administration of data and promises to improve on network security.
I have noticed one thing in this field is that like all businesses you shouldn't worry so much about the theives getting past your Firewall but the theives you may already have working for you.
Internal stuff... is always a killer...
Keep me posted on the NXN stuff...
Thanks
As was asked... the preference was Linux... Spider doesn't suport linux yet... It's not a solution for the NOW
Yes it does. Only the render manager has to be a windows box. There is a few pages in the spider manual on using linux. I know a few studios that had also previously used it with a linux farm back when it was still commercial(2-3 years ago).
If you read the original post... the render server in linux as are the clients..
Yes it is possible....
It's not the renderer the problem... it the scene files. and what is inside the file.... example.
If you are sub'ds it will use more ram then polys
If you use mocap it will take more ram then key frame animation.
Many things to think of....
Yes it is possible to do this with Mentalray
Beaker,
Are you saying for certain that SPIDER can run on linux? The file download is an "exe" file and I believed that can't execute on a linux box. If it has to run on a Windows box to serve a Linux render farm I'm confused. I like SPIDER in that it's now free.
This would be the perfect candidate for an open source project. How many times have in-house render queuers been written? Probably dozens of times. And most of them are never going to be sold, so there's no reason to keep them closed source.
It could be just like the way Apache was developed...
Andrew Klaassen
I'll answer for him since it seemed apparent to me what he was saying.
What he's saying is that out of 100 machines; 1 of them runs windows. That one that runs windows has the spider render que setup on it.
He then doles out work to the 99 linux render slaves out there.
In short; windows render masters running spider to distribute; and
linux slaves to do the actual work.
spider shouldn't have to run on any of the slaves.
Russel,
I guess that option (Spider on Windows Server to handled files to Linux Render Farm) is great for some one like me who has Windows installed on a partition while Linux is installed on another. The only issue I have with this is that it again makes the user feel like they are still handcuffed to Gate's Windows Empire and not totally free for Linux.
I received this email from Spider Network early this afternoon:
"Hello,
We've gotten numerous requests regarding whether or not we would consider releasing the source code to Spider as open source. We currently have no plans to do that.
We are hoping that by providing the software free of chage that some of the larger facilities that are making use of Spider will be willing to contract with Spider Networks to provide bug fixes and to create maintainance versions. These maintenance versions would be to add support for future versions of OSes and for the various rendering programs Spider supports.
We would also be willing to provide source code to a facility wishing to pay a license fee and sign an NDA.
Regarding Linux support, there is a render daemon for Linux that will work in conjunction with the Spider service running on Windows. Jobs must be submitted on Windows and can render on the Linux farm. The daemon as a free download has yet to be provided but will be as soon as it's available."<<
It seems there are few options at the moment for Linux users to run these programs. The one program that seems is made for and supported on Linux is NXN Alienbrain Studio. The good thing is it gives you 30 days to try the software and also comes with a demo project to get comfortable with the program.
Moogy1,
I have to head out downtown for a client meeting and to kick my web designer in the rear (website still not finished..grrrr). Anyway, I'll upload the render que manager (GRender) for Linux later. It's free and you can find the link at http://grender.sourceforge.net/
I searched SourceForge.net and this was the only one I could find that supports both Maya and Mental Ray. Also helps your studio that it costs nothing.
Cheers 
One rendermanager that I suggest you check out is RUSH. It works with Maya + MR, plus numerous other renderers, ie: anything that has a commandline support, plus some others(Maya, MR, Shake, Prman, BMRT, Afterfx, Brazil, Houdini, Lightwave, Softimage, Arnold), plus support for every platform(Win, Mac, Linux, SGI). I find it a very strong setup for large farms. I used it on a previous job and we were able to throw maya and shake renders at it, usually with 100-200 jobs going a day. Only downside would be the price($150 a node, plus 10% discount for 100 nodes). I find it worth it though for handling that many nodes.
http://seriss.com/rush/
Blue Sky used it for Ice Age on their Alpha farm.
What about SGE "Sun ONE Grid Engine".
SGE is a very powerfull piece of software.
I have just finnished setup here at http://www.km-animation.dk/, and we are very eksited about it. We use it with the MayaRender and the MentalRayMayaRender
and its completly free http://gridengine.sunsource.net/. And is published under the * SUN INDUSTRY STANDARDS SOURCE LICENSE
thank you again for all the posts.
after some thinking i reached to the conclusion that given the circumstances the best solution seems to be a Redhat + Oscar (http://oscar.sf.nethttp://oscar.sf.net) combination. The advantages of Oscar seem obviuos: easy administration, c3, lam, mpich, pbs, pvm, systemimager support included. An alternative that looks much more appealing to me is Debian + FAI (http://www.informatik.uni-koeln.de/faihttp://www.informatik.uni-koeln.de/fai) + openMosix (http://openmosix.sf.nethttp://openmosix.sf.net), since i've used this succesfully in the creation of 30+ computer clusters. The only fear i have though is that Maya - which is "officially" for Redhat - would create problems when running on debian (although i installed Maya 4.5 and 5.0 on debian without any problems). I guess this remains to be seen after some testing.
openMosix seems to be a very good add-on to the traditional renderfarm as it allows the nodes to migrate secondary apps of the nodes in the heaviest load to lighter load nodes to keep renderspeed up. however, I would like to know if anyone ever tried using openmosix as a distributor of images to be rendered using something like Maya Renderer or Mental Ray.
thank you.
best wishes.