Hi,
I'm writing a script to automate render layer setup plus recognize alpha maps in transparency channel.
Here is the video on how to do this manually by Tales of Nalstone:
https://www.youtube.com/watch?v=Bcqfd6jeSzo
This is what end up so far, if you can help, please give me some hints or advice:
//Remove Material overide, still dont know how to do this part
//select models
editRenderLayerGlobals -currentRenderLayer defaultRenderLayer1; // change it to AO layer or Depth layer name later
string $transforms[] = `ls -tr`; //lists all the transform nodes in the scene
string $polyMeshes[] = `filterExpand -sm 12 $transforms`; //filters out all the non-polymesh nodes
select -r $polyMeshes; //selects all the polymeshes
//select shader from selected models
$objList = `ls -sl -fl`;
hyperShade -smn "";
$matList = `ls -sl`;
//create layerShader for each shader
for($i=0; $i<size($matList); $i++){
string $newShader = `shadingNode -as layeredShader -name ($matList[$i] + "_layerShader")`;
//Change layerShader Compositing Flag to Layer Texture
setAttr ($newShader + ".compositingFlag") 1;
//Plug the surfaceShader1 to color of layerShader
connectAttr -force surfaceShader1.outColor ($newShader + ".inputs[0].color");
//check surfaceShader name again in actual scenes
//assign back layer shader to the models
string $objShader = `hyperShade -objects $matList[$i]`;
hyperShade -assign $newShader;
}
//I dont know why but a defaultRenderLayer1_layerShader shader was created every time I run the code
catchQuiet(`delete defaultRenderLayer1_layerShader`);
// plug transparent texture into transparency of the new layer shaders
// I cannot do this in the first for loop, but the below one not working neither
for($i=0; $i<size($matList); $i++){
string $newShader2 = $matList[$i] + "_layerShader";
// find name or path of "transparency" texture
string $tranTex[] = `listConnections -type "file" ($matList[$i])`;
connectAttr -force ($tranTex[$i] + ".outTransparency") ($newShader2 + ".inputs[0].transparency");
// if nothing connected, assign surfaceShader1 to the model
if (!`size $tranTex`)
{
hyperShade -objects $newShader2;
hyperShade -assign surfaceShader1;
}else{
connectAttr -force ($tranTex[$i] + ".outTransparency") ($newShader2 + ".inputs[0].transparency");
}
}