Hi!
First of all, I'm no programmer, I'm a digital designer, and I'm trying to develop
a shader as good as the one from Guilty Gear's the game.
So far I've achieved a ground level of understanding this method, yet I think I'm on the right way. If u watch carefully this video u'll notice that there are 3 and 3 only things that matter to be controlled, which are: threshold, normal vector and light vector, being the normal vector the most important of them all! I've stumbled upon some pretty good scripts already (which i've not tested yet due to xmas time), the only thing that is concerning me a lot is how to apply the vertex painting to the math.
In the video, it is explained that there is an offset to the threshold, (since the threshold is the 90º normal vector of the vertex pointing to the light vector, if in threshold number, it's lit, if not, it's dark). The offset behaves like an extension of the threshold permiting that areas of the object be more easly lightned, an others harderly (in a scale that I imagine to be [0-1], 0 being totally shaded, and 1 with a fair angle of light capturing). If only I could make the vertex colors to behave like this offset I mentioned, or at least help the control of the vetices to "attract" or "repel" light beyond the already altered normal vectors ... it woud be wonderful!
Here are my current and all literature about the whole thing:
Video explaining the Vertex control procedure in an old 3d program ("Softimage"): youtube.com/watch?v=iH3p8N7qbv8
github from this old program's plugin:
github.com/Gotetz/UserNormalTranslator
github from a newer plugin made to run with Maya:
github.com/yiyangchenbuffalo/maya_normaltranslatortools
Some other intelligent stuff about it:
en.m.wikibooks.org/wiki/GLSL_Programming/Unity/Toon_Shading
rbwhitaker.wikidot.com/toon-shader
PLEASE! WHO SHOULD EVER SOLVE THIS OUT WILL HELP ME A LOT TO DEVELOP COOL STUFF!!!!
Again: I'm no programmer, just a cg artist.