Clamping Vertex Color Values

[andy-davis]

Thanks to Mr Kovach, I've managed to get a mutliple-pass mental ray vertex baking tool up and running.
Having some trouble with the color values though.
Because I'm adding the light values to a base color, sometimes the RGB values are greater than 1.
Is there a way to clamp the values in a color set to between 0 and 1?

I tried using the polyGeoSampler command
(polyGeoSampler -cmn 0.0 0.0 0.0 -cmx 1.0 1.0 1.0;)
but this changes all of the vertex colours to weird values rather than clamping just the out of range vertex colours.

Any ideas?

[rgkovach]

there are two ways you can do this in maya 2008 and higher

when you create a vertex color set, you can choose to clamp it:

CODE
polyColorSet -clamped;

also, the vertexBakeSet can clamp the Min and Max values and also how the new colors get mixed with the existing values (overwrite, add, subtract, multiply, average, divide).

out of curisoity, are trying to bake into the same vertex color set multiple times?

also, you should check out RenderHeads vertex color tool called VertexChameleon...

[rgkovach]

Here is another script you will find useful as you venture down the path of vertex color sets.

In maya 2008 (not sure about 09), clamped Color sets do not get deleted correctly out of the scene.
For some reason, the clamped attribute is not removed, so your scene ends up with vertex color set "ghosts".

asking Maya for a list of colorsets using
CODE
polyColorSet -query -allColorSets;
is not 100% accurate!
instead use this (works off of shapeNodes):

CODE
{
    string $rkSelShape[]=ls -sl -l;
    string $multiAttr[] = listAttr -multi ($rkSelShape[0]+".colorSet");

      for ( $attr in $multiAttr )
      {
       if ( gmatch $attr "*.colorName" )
        {
              string $name = getAttr ($rkSelShape[0]+"." + $attr );

              string $tokens[];
              tokenize $attr "[]" $tokens;

              print ( "  Color Set Index " + $tokens[1] + " is named " + "'" + $name + "'\n" );
            }
      }
    print "Done!\n";
}