I can hardly imagine your setup without actually seeing the image, but if you do have a clean outline, why don't you just draw a rotoshape around it (outbounds), invert it, and feed it as garbagematte?
If you also have transparent areas inside the character, you should cover them with another rotoshape that would go into holdoutmatte.