what version of maya are you using ?
first feature of maya toon wasn't accept aliasing... or was it motionblur ???
normaly actual versions are okay with everything.
But antialiasing should be a global problem, not a outline it is a filter on top of everything.
If you have a to thin outline you will always have a problem.
try to increase to antialiasing sample in your render global
try to resample your outline in its attribute editor to
thats all that i see...
cheers