Ha, funny how it works that way, you have a problem for a while, can't figure it out, then you post it, then five mintues later you figure it out.
If anyone has the same problem it seem to be cause by having the solver disbaled when creating the cloth object. Silly eh?
TNG