The optical sensors are just a small part of the human (and animal in general) vision system. A much bigger component is our innate (evolutionarily acquired) understanding of basic mechanics, simple agent theory, and object recognition.
When we look at the road, we recognize stuff in the images we get as objects, and then most of the work is done by us applying basic logic in terms of those objects - that car is off the side of the road so it's stationary; that color change is due to a police light, not a change in the composition of objects; that small blob is a normal-size far-away car, not a small and near car; that thing on the road is a shadow, not a car, since I can tell that the overpass is casting it and it aligns with other shadows.
All of these things are not relying on optics for interpreting the received image (though effects such as parallax do play a role as well, it is actually quite minimal), they are interpreting the image at a slightly higher level of abstraction by applying some assumptions and heuristics that evolution has "found".
Without these assumptions, there simply isn't enough information in an image, even with the best possible camera, to interpret the needed details.
> "A much bigger component is our innate (evolutionarily acquired) understanding of basic mechanics, simple agent theory, and object recognition. ... they are interpreting the image at a slightly higher level of abstraction by applying some assumptions and heuristics that evolution has "found"."
Of course, and all this is exactly what self-driving AIs are attempting to implement. Things like object recognition and understanding basic physics are already well-solved problems. Higher-level problem-solving and reasoning about / predicting behaviour of the objects you can see is harder, but (presumably) AI will get there some day.
Putting all of these together amounts to building AGI. While I do believe that we will have that one day, I have a very hard time imagining as the quickest path to self-driving.
Basically my contention is that vision-only is being touted as the more focused path to self-driving, when in fact vision-only clearly requires a big portion at least of an AGI. I think it's pretty clear this currently means this is not a realistic path to self-driving, while other paths to self-driving using more specialized sensors seem more likely to bear fruit in the near term.
When we look at the road, we recognize stuff in the images we get as objects, and then most of the work is done by us applying basic logic in terms of those objects - that car is off the side of the road so it's stationary; that color change is due to a police light, not a change in the composition of objects; that small blob is a normal-size far-away car, not a small and near car; that thing on the road is a shadow, not a car, since I can tell that the overpass is casting it and it aligns with other shadows.
All of these things are not relying on optics for interpreting the received image (though effects such as parallax do play a role as well, it is actually quite minimal), they are interpreting the image at a slightly higher level of abstraction by applying some assumptions and heuristics that evolution has "found".
Without these assumptions, there simply isn't enough information in an image, even with the best possible camera, to interpret the needed details.