Pdc Format ! Help! Help! Help!

[amitc]

PDC is driving me nuts.
1) I am trying to find out how PDC file should be wriiten
2) Then use the PDC to drive particles

The PDC format documentation is confusing.
It does not tell what is specification for BIG_endian or little endian (is 0 big endian and 1 little endian or vice versa or something else)
It does not tell what size the arrays will be for each type. A table specifying attributes and size would help.

Anyways Here what I have done so far.
Create a particle object with 4 particles

CODE
particle -n p -ll 0 0 0 -ur 3 0 0 -grs 1 -c 1;

Export them as PDC
CODE
dynExport -f pdc

[reyrey]

Funny, I just went through this process the other night.

I'm writing some plug-ins for particle-flow and XSI to import\export PDC cache files. I too find that even on my pc the PDC cache output function is using Big Endian byte order and you must do a swap on all non-char data types which was a bit frustrating at first. I am getting a version of 1 and an Endianness of 1 btw.

Agreed, dynExport always includes particleId. And I do not think you will be able to only export a single position axis by default. You could however, add a custom attribute that you assign the current Y position of the particle to and export that attribute instead of position, I would think. But, at that point, you might as well just write your own export function out of maya instead of using PDC. Unless you want to manipulate the data externally and feed it back in through the cache mechanism...

I am not seeing an additional 4 bytes before the next attribute record and my PDC parsing function seems to be working fine (I'm writing it in C++ but that shouldn't matter).

There is definitely very minimal information on the PDC file format out there besides the single page in the help, it took a bit of patience but I did get it going.

Post some code( your swapper and/or your vector array handling seems the likely culprit ) and I'll take a look if you'd like.

Good luck,

Todd Kuennl
Rapid-TD

QUOTE(amitc @ 10/06/05, 03:38 PM)
PDC is driving me nuts.
1) I am trying to find out how PDC file should be wriiten
2) Then use the PDC to drive particles

The PDC format documentation is confusing.
It does not tell what is specification for BIG_endian or little endian (is 0 big endian and 1 little endian or vice versa or something else)
It does not tell what size the arrays will be for each type. A table specifying attributes and size would help.

Anyways Here what I have done so far.
Create a particle object with 4 particles
CODE
particle -n p -ll 0 0 0 -ur 3 0 0 -grs 1 -c 1;

Export them as PDC
CODE
dynExport -f pdc  -mnf 0 -mxf 10  -atr position p;

Why does this write 2 attributes "position" and "particleid" when I am just aking it to write one?
Is there a way to just just write PositionY only instead of all x y and z?

So I get a file of size 190 bytes of which 28 is standard header so far good.
After the header  4+8+8=12 bytes which tells the length of "position" string and position written as chars and  integer defining type as 5 ie vector array.
I presume vector array is 3 doubles. (beats me how it differs from double array)
There are 4 particles so there should be 4 particles*3 doubles*8 bytes =24 bytes.
Before the next attribute(particle id) there is an extra of 4 bytes? dont know what they mean. It looks like its an an integer but should be at the start of the vecotor array or end?

I have written a barebone PDC reader in C.
I am finding some inconsitency with the documentation.
There are extra 4 bytes in every attribute record. Not sure where they come from and where they should be (beginging or end).

It seems the PDC is written in BIG-endian. I am working on windows/intel and linux/intel
I have to swap integers to read correctly. The double go bonkers as I am not swapping them correctly.
Is there a way to write pdc from Maya in little endian for testing?

Well ultimately I want to drive theses particles from external data. Lets say only in position Y (to create a height field)

Any help from you folks would be great
Thanks
[snapback]216643[/snapback]

[amitc]

Ok I have found out the problem above with 4 missing bytes. I was making a mistake in my C code.

Todd thanks very much for your response.
My integer swap code is working fine but the doubles are not swapping correctly here is the C code

CODE
void
swapDouble( char * theDouble )
{

[amitc]

QUOTE(amitc @ 10/06/05, 08:00 PM)
doubles are not swapping correctly here is the C code
CODE
void
swapDouble( char * theDouble )
{

[reyrey]

Yep. (nbytes - 1) - i

Mine is a bit different, I am using std::swap, it does require you to include to compile though.

void ByteSwap(unsigned char * b, int n)
{
register int i = 0;
register int j = n-1;
while (i<j)
{
std::swap(b[i], b[j]);
i++, j--;
}
}

Then I define a macro that casts to unsigned char, gets the sizeof() and passes it to ByteSwap.

Glad you got it working What if you pipe your data into a softbody?

Todd Kuehnl
Rapid-TD

QUOTE(amitc @ 10/06/05, 08:21 PM)
Looks like this was the bug
CODE
this should
theDouble[8-i] = swap;
theDouble[7-i] = swap;
like this

Yes  I am also getting version to be 1 and endian to be 1 (big endian)
Hopefully  it means 0 would be little endian
[snapback]216661[/snapback]

[amitc]

Todd still my doubles are not swapping correctly.
something is wrong in my swapDouble code.
where is algorithm.h located on linux?

[reyrey]

It is part of the STL, did you try just without the .h? Are you using GCC? Check this link out:

http://www.dinkumware.com/manuals/reader.a...h=algorith.html

Good luck,

Todd

QUOTE(amitc @ 10/06/05, 09:47 PM)
Todd still my doubles are not swapping correctly.
something is wrong in my swapDouble code.
where is algorithm.h located on linux?
[snapback]216663[/snapback]

[amitc]

QUOTE(toddk @ 10/06/05, 10:00 PM)
It is part of the STL, did you try just without the .h? Are you using GCC? Check this link out:

Yes, yes

IT seems our code is same. But my doubels do not swap correctly.
How do you swap your doubles.
making a call like this ByteSwap( b,8) ????

Is double 8 bytes??
Is vector array 3 doubles???

Appreciate your help
Thanks
--Amit

[reyrey]

I define a macro like this:

#define ByteSwap2(x) ByteSwap((unsigned char *) &x,sizeof(x))

the just do:

ByteSwap2(value);

The macro handles the size automatically for you.

Todd Kuehnl
Rapid-TD

QUOTE(amitc @ 10/07/05, 10:00 AM)
Yes, yes

IT seems our code is same. But my doubels do not swap correctly.
How do you swap your doubles.
making a call like this  ByteSwap( b,8) ????

Is double 8 bytes??
Is vector array 3 doubles???

Appreciate your help
Thanks
--Amit
[snapback]216743[/snapback]

[amitc]

Todd
does your program read doubles and swaps them correctly?
I am having trouble swapping it seems the doubles are not written in IEEE standard binary format.

Again the integer swaps fine the trouble is doubles
--Amit

[reyrey]

Yes, my code seems to be swapping them correctly. Perhaps you could send me that section of your code and I could compare it with what I am doing. You can PM me if you'd prefer to send it via e-mail.

-Todd

QUOTE(amitc @ 10/07/05, 01:27 PM)
Todd
does your program read doubles and swaps them correctly?
I am having trouble swapping it seems the doubles are not written in IEEE standard binary format.

Again the integer swaps fine the trouble is doubles
--Amit
[snapback]216768[/snapback]

[amitc]

Okay I have got the pdc format worked out.
Now I can read and write pdc with my c program .

I am still not able to use my pdc files along with particles.

Let say I have 5 particles in a grid along x axis.
Something like

CODE
particle -ll -2 0 0 -ur 2 0 0 -grs 1 -c 1 -n myParticles;

Now I want to move these particles at everyframe in y axis.
Using my C program I create a series of pdc files with position attribute and particle ID only, but it does not work. I have turned off memory cache and I am pointing to the correct cache directory.
How should this be done?

Does particleID attribute needs to be written in PDC file? Can I skip it?

All I am changing is y position. Can I just write out y position instead of writting all x,y,z ?

Please help!
Thanks
--Amit

[failure]

let your c program dump the pdc files created by maya

[amitc]

failure I am not understanding what you are suggesting. Here is what I am trying to do.

When I create pdc files(using my c program) and write them in big-endian byte order they work correctly.

but if I write them in little_endian byte order they dont work. (I think my c program is generating correct pdc files)

I have my data in little endian binary.
So I'll just slap a pdc header in front of the data to make it work with maya.
My data comes from custom simulation programs.

Other thing I am trying to do is.
Add a custom per particle attribute say ppy.
I'll read in my data to ppy. This ppy attribute would change the Y position of each particle differently.

So far I have not been successful in creating a runtime experssion to change Y position (per particle) based on my custom attribute.

Any thoughts?

[failure]

If maya doesn't like little endian then just convert the data into big endian. Really no big deal. It's one line of c code.

I just suggested that you expand your app so it can read and pretty-print content of any pdc file. That way you can create cache files from within maya, inspect them with your app and create all your pdc files in that fashion.

[amitc]

QUOTE(failure @ 10/12/05, 02:19 AM)
If maya doesn't like little endian then just convert the data into big endian. Really no big deal. It's one line of c code.
I already have this working. Thats how I found out which byteorder works.
My thinking was if the pdc documentation mentions byte order then it can read it.

QUOTE(failure @ 10/12/05, 02:19 AM)
I just suggested that you expand your app so it can read and pretty-print content of any pdc file. That way you can create cache files from within maya, inspect them with your app and create all your pdc files in that fashion.
I am already doing this. It took me a while to deconstruct the pdc format.

When I read the some pdc files I am getting a ghostframes attribute at the end of the file. It does not contain any data the files ends after it. I could not find documentation for it. Not a big deal since the files ends there.

Thanks for your comments failure.

I found the PDC format documentation lacking few things I am going to sum them here just in case someone needs it
**********************************************
the following 3 numbers could be anything. They are there for version control and expansion purposes
1 Integer indicating the file format version number.
2 Integers holding extra bit information that various file format version might decide to use.

for big_endian the number should be 1
for little_endian it should be zero but maya does not read little_endian pdc files for some reason
1 Integer holding bit information about whether the values stored in the file are BIG_ENDIAN or LITTLE_ENDIAN.

A little more description about the data types
0 ---> Integer (just one int=4 bytes - usually per object attribute)

1 ---> Integer Array ( N ints=N*4bytes - where N is the number of particles, usually per particle attribute)

2 ---> Double (just one double=8bytes - usually per object attribute)

3 ---> Double Array ( N doubles=N*8bytes - where N is the number of particles, usually per particle attribute)

4 ---> Vector (3 doubles=3*8bytes - usually per object attribute)

5 ---> Vector Array ( N vectors=N*3 doubles=N*3*8bytes - where N is the number of particles, usually per particle attribute)

---

Other thing to aware about when writing a program to read pdc files is
there could be word alignment problems which are dependent on compilers and language you choose for writing your code. I hit this using C. I used memcpy to get correct data rather than moving pointer to correct adderss(which always gave wrong results).
In a lame way what it means is when you read data from memory by providing a memory address or pointer you might get incorrect results. As compilers will hand you the data from a valid address rather than from where you asked from.

Please feel free to move this to faq if it seems useful or correct me if I missed out something.

--Amit