No, it can be any format of image, maybe except gif, not sure about that. Do you know how to make alpha channel in psd? Do you know how they work? When you have it done, you have to do some shading stuff with hypershade and thats it.
Example:

this is a grass texture.

this is it's alpha a.k.a transparency channel.
Both combined and making a shader in maya and applying the shader to a simple poly plane, you would get to see only the parts of the image that are white in alpha.
But i assume that you want to put one texture on top of another and make one of these a bit transparent for better blending. One guy has already asked this question and you will find answers here http://www.highend3d.com/boards/index.php?showtopic=232605 . The only difference is that he is using a bit different object and different image.
If you have any questions, post again.