Whats in the furstrum, ie wat is visible to the camera at a time. That is if you dont have raytrace on, in wich case they all need to be accounted for. Even so the primary scanline does not account for them. Basicaly this is a trivial optimization for the computers to do. Basicaly it does not speed your rendering up. IT might however speed up the export to smething like sequantial rib/mi a bit but not much (taht is the computer does things much faster than you do so if it takes you 15 minutes to key this youe allready lost the benefit for sure)
Basicaly your shaders, light and amount of geometry is all that makes sense to optimize. (make sure obects are one sided! if they dont need the second side thisway backfaces will be culled)
So no doing this manualy does not make any sense. In reality only way to speed up stuff is to have more then one computer. Even so a BADLY done scene might take 20 mintuest to accomplish a well done allmost same result scane by 1:100, but thgen its more about inept users than anything else.