Well it probably has a lot to do with what you're gettingAttr on.
If you're gettingAttr on a top-level node, then maya just has to look at the animation curves at that time point, and the operation is likely to be more or less transparent.
But if you're gettingAttr on, say, an object pointConstrained to a character's hand, then maya has to resolve almost the entire scene to give you the information you want. I can see how a couple of calls like that could quickly slow a scene to a crawl.
Worst case, would be gettingAttr on something driven by a dynamics effect like a rigidbody. To give an accurate result (and I suspect maya would try to do that) the program would have to replay the entire course of the animation, complete with dynamics, for each out-of-frame call to getAttr. In such a situation a frame cache, either manual or automated, would really be the only reasonable solution.
Oh and by the way, I didn't know about that cache node - so I'm extra glad I stumbled in here. That rocks. Thanks!