Faster way to create particles from TXT?

[mattwright]

Hi

I have written a script that takes a text file as input, and for each text file line, adds a particle (using emit) to a previously created paritcle system. Then, I use the particle -e command to edit the particle color to information that is also in that particles text file line -

so a line of the text file is just:

[x pos], [y pos], [z pos],[r color],[gcolor],[b color]

The problem is, that the text files I am importing in are extremely big - 26mb - some over 1million lines of particle information...

When I cut down the size of the text file to just a few thousand lines, the MEL script works great - and doesn't take too long..

However when using one of the larger files, it just takes for ever to import the particle cloud... infact, I have never waited long enough for one to fully import (even after 3 hours, i dont reckon it is even close to being done).

Does anyone have any suggestions on how to speed this up -

again - the basics of the script..

fopen and fgetline to read the lines of the text file..

a simple loop that creates a particle (using emit), and then edits the particle to set its rgbPP.

I have also got the script to disable both the undo queue, and history.

[joojaa]

Dont emit each particle separately... But all at a time. Prefeerably so that you use setAttr to push all or most of the data at once, since its even better tahn emit in many ways.

[mattwright]

Thanks for the info

I am pretty new to scripting, and really have no idea how to do all the paricles at once!!

Right now, the loop looks like:

emit -object points
-position $xpos $ypos $zpos; // set x y z position

particle -e -or $x -at rgbPP -vv $rnormal $gnormal $bnormal pointsShape //set particle color

The data in the txt file for the RGB is in 0-255 range, so there is a couple of lines in there to also normalise the RGB data.

I would love to know how to do this in all one shot.

I also cannot work out a way to use setAttr to set rgbPP

Cheers!

Matt

[joojaa]

one way is to collect the data into a string and then evel that string later!

[mattwright]

Hmm... interesting...

I tried the following:

instead of doing an emit on each loop, I created a string ($posstring) - and then each loop, I added the X Y Z co-ords to that string:

$posstring = ($posstring +$buffer[0] +$buffer[1] +$buffer[2])

I setup a simple text counter, that would display how fast this loop went... It works great for the first 5 thousand lines imported... however as it has to start adding information to what is now becoming a REALLY long string, it slows down something crazy...

After about 6000 lines, it seems to be faster having the emit in the loop, rather than generating the string, and then eval'ing the string..

Any ideas??

Matt

[joojaa]

DOH! Well offcourse it is your memory is being filled by the string and once you it the max you start t swap things.

Solution generate a string with say 3000 lines worth of particles emit and flush string and genrate a nother 3000 lines worth of particles and emit those 3000 particles and so on and so on.

the idea is not to have a emit for each and every particle in exsistece.

[mattwright]

Ah, not a bad idea that Only thing is, I would have no idea how to do it

Interestingly enough, I did a test, with a 3000 particle text file, and the speed difference between emit everytime and using eval really wasnt that great... Unless my MEL code is a mess of course...

[joojaa]

well probably the difference is somewhere sooner than at 3000 units or something. But in reality id just write the data directy to the data array instead of emitting. Because its faster. look at the particleShapes node reference and setAttrs ref page for info on how to do this.