Rotating the left and right eye view is is not the only way to generate the stereo images. The left and right eye views can be sheared instead. The left and eyes remain oriented along the direction of view, but each eyes view is sheared along z so that the two frustums converge at the fusion distance.
Although shearing each eye's view instead of rotating is less physically accurate, sheared stereo views can be easier for viewers to achieve stereo fusion. This is because the two eye views have the same orientation and lighting.
For objects that are far from the eye, the differences between the two approaches are small.