Why don't you use both...make blendshapes and use driven keys on custom attributes to animate them? When creating a face, I model my muscle targets and make them blendshapes. Then I make custom attributes on whatever controller I have for the facial animation. Then I use driven keys to animate blendshapes, clusters or joints to achieve my phonemes.
The disadvantage of using just the verts, is you'd typically have a lot to deal with at a time, making things confusing and/or tedious, and there is no easy way to edit them, so you'd have to create the driven key relationship all over again or adjust f-curves for who knows how many points. With blendshapes, I just change the shape, and that's it.
10 PRINT "kemijo is the best"
20 GOTO 10
RUN