I didn't find his answers particularly convincing. His answer focused on costs mainly, and how "the best part is no part". We have already seen multiple accidents caused by camera's limitations [1] which would not have happened if Tesla used Lidars.
Cameras have poor dynamic range and can be easily blinded by bright surfaces. While it is true that humans do fine with only eyes, our eyes are significantly better than cameras.
More importantly, expectations are higher when an automated system is driving the car. It is not sufficient if, in aggregate, self-driving cars have fewer accidents. If you lose a loved one in an accident where the accident could have been easily avoided if a human was driving, then you're not going to be mollified to hear that in aggregate, fewer people are being killed by self-driving cars! You'd be outraged to hear such a justification! The expectation therefore is that in each individual injury accident a human clearly could not have handled the situation any better. Self-driving cars have to be significantly better than humans to be accepted by society, and that means it has to have better-than-human levels of vision (which lidars provide).
The dynamic range is the reason Tesla know counts photons rather than use traditional camera processing. They basically remove the concept of exposure entirely and simply pass the sensor photon counts to the neural net.
This approach not only simpler as it removes photo processing/encoding but the result is that the NN can operate with a very high dynamic range similar to the human eye and in many cases can be sensitive on the single-photon level.
> They basically remove the concept of exposure entirely and simply pass the sensor photon counts to the neural net.
That sentence does not make sense. There's no such thing as a count without a corresponding interval that count occurred over. That interval is the exposure.
You can of course do lots of (very) short exposures to avoid sensor saturation. That's "just" a movie at a very high frame rate. And then you can post-process this in lots of exciting ways, align the frames, average them, etc, etc.
Yeah that's fair. A CCD sensor basically converts individual photons to electrical charges. What Tesla has said they've done is thrown away all the traditional image signal processing & post-processing which often includes a lot of exposure-related averaging.
You're right though that we don't typically use real-time neural networks that operate based upon spike rate, so an interval needs to be chosen for photon counting which could be considered a kind of exposure and it is critical that the interval be short enough to avoid saturation.
Lol this doesn't make any sense. The dynamic range of a fully sunlit California highway during noon in the summer (I.e. the brightness reading of the darket vs the brightest spot) is wayyyy higher than any existing sensor. You cannot ignore exposure, you have to choose which part of the scene you want within the brightness range that your camera sensor can capture. You will have areas of the scene that clip, in other words areas of the scene that are pure black or white with no data.
You can do bracketed exposures, but that's literally the opposite of ignoring exposure.
Just keep the duration low so that you never saturate the sensor even in bright sunlight and let the NN do the summations.
At a fundamental level it is somewhat akin to bracketing except all that HDR processing/frame matching is performed within the NN rather than a traditional image processing stack.
The NN is better at this anyway since it must already be performing camera/pose motion tracking to correlate what it's seeing from frame to frame.
Counting photons won't keep a camera from being "jammed." Unless you are using a physically perfect polarizing filter, such that each pixel on the sensor only receives photons from the exact angular window, traced back through the lenses, you have a camera that can ultimately be "jammed."
The human eye isn't so great on those terms. But humans can raise their hand to block the sun if it's straight at our eyes.
The crash you referenced occurred in 2016 when they were using radar on the cars and I don't believe they were yet using raw photon counts nor did the NN have any voxel-based memory as it does now.
The big limits of LiDAR are cost, more than anything. There have been dozens of public driving trials where from a functionality level the answer has been positive (apart from traffic lights, the bastards), but nobody wants to buy a solution with a six figure BOM, before integration.
how do you count photons continuously? what... this makes no sense, if you pass "the photon count" you just did a exposure... also how does a photo diode count photons?
That video is from 2020, but Tesla didn't remove radar until 2021. Meaning that the crash occurred with radar still active, which I feel just backs up what Karpathy was saying.
Mr. Rajkumar of Carnegie Mellon, who reviewed the video and data at the request of The Times, said Autopilot might have failed to brake for the Explorer because the Tesla’s cameras were facing the sun or were confused by the truck ahead of the Explorer. The Tesla was also equipped with a radar sensor, but it appears not to have helped.
“A radar would have detected the pickup truck, and it would have prevented the collision,” Mr. Rajkumar said in an email. “So the radar outputs were likely not being used.”
Tesla later said that during the crash, Autopilot’s camera could not distinguish between the white truck and the bright sky. Tesla has never publicly explained why the radar did not prevent the accident.
I have some experience with LiDAR, they fail easily if a water droplet is on the cover or if signs are too bright. It’s a whole different technology challenge.
Why is LiDAR still expensive enough for the cost to be a problem for anyone? Why have large numbers not driven these down to commodity devices at this point? Or maybe similar tech at another frequency like spread spectrum microwave with phased array semiconductor antennae?
> Why is LiDAR still expensive enough for the cost to be a problem for anyone?
Because economy of scale is not the only reason things are expensive. Lidar with 2cm distance resolution has to detect light and measure time to less than 60 picoseconds. It requires sophisticated, expensive sensors like avalanche diodes and complex and extremely fast circuitry.
Consider what a single pixel of a lidar device is doing. It's operating 8 orders of magnitude faster than a normal camera pixel. It's detecting an incredibly small amount of light; the spot of a weak laser on a questionable surface from tens of meters away. Its doing that in the presence of light that is tens or hundreds of thousands of times more powerful than the reflection it's actually looking for. Much of that reflection is even at the same wavelength of the laser!
Even the mechanics of the device are critical. Pixels are gathered sequentially, unlike in a camera, so any vibration makes the data uselessly fuzzy. Vibration in a camera is correlated extremely tightly between millions of pixels. In lidar its correlated with dozens to hundreds. Any high frequency vibration has large negative impacts on the data.
> Or maybe similar tech at another frequency like spread spectrum microwave with phased array semiconductor antennae?
...that's just radar. Literally. Microwaves are 300 Mhz to 300 GHz. Automotive radar is like, 80 GHz, +/=5 GHz (mostly). It uses phased arrays. It uses highly integrated devices. They don't use semiconductor antennas, because the area on a silicon die is far, far to valuable to use for an antenna, but the antennas are incredibly cheap. Radar and lidar are just pretty different.
The numbers aren’t that large. Nowhere near the volume of optical components sold, which benefit from continual optimization that the billions consumer electronics sold each year undergo.
The problem with self-driving is that it is based on data, but the environment may change. See e.g. the case where Tesla thinks that firetrucks are roads.
So if fashion changes, pedestrians may suddenly look like road too, as just an example.
Another problem is that state-of-the-art classification networks have an accuracy in the 90% range. Given that a car has to take hundreds of decisions in a single ride, then even if the accuracy was 99%, you see that error rate simply gets too high.
> state-of-the-art classification networks have an accuracy in the 90% range.
If you're referring to ImageNet SOTA, is has 20000 different classes, including 120 different dog breeds [1]. This is a vastly different task than reliably detecting pedestrians where Tesla can actively curate a dataset of hard examples (from their fleet), whereas ImageNet is fixed, sometimes with low quality labels and as few as a couple of hundred examples. Tesla can also pick a point on the ROC curve to give higher recall but more false positives (which is important for VRUs specifically). Another big factor is that Tesla is using video, not still images, which makes predictions even more robust.
And that's just for pedestrians, Tesla are also using a general ViDAR (visual LiDAR) which is trained to detect obstacles that do not have a specific class. The ViDAR again operates on image sequences, not a single image, and can thus pick out structure from motion.
> While it is true that humans do fine with only eyes, our eyes are significantly better than cameras.
They also have better failure modes and a really sophisticated error management system. They are susceptible to optical illusions, though.
> It is not sufficient if, in aggregate, self-driving cars have fewer accidents.
This is the incorrect analysis anyways. This was always going to be true because a large portion of accidents are single vehicle accidents where the driver was at fault for the crash. Usually due to speeding, alcohol, youth, or a combination of them.
If they didn't have fewer accidents then something is very very wrong with the entire idea. Which may very well be the outcome here. Looking at multi-vehicle accidents where there was no fault of the driver who died, it's not clear that an automated system driving the car would have saved them.
Roads are built right next to cliffs and bodies of water. Semi trucks can completely destroy your vehicle in an instant. Large accidents on snowy or foggy highways happen. Drunk drivers exist and sometimes literally do come out of nowhere, a pickup truck moving at 60mph has enough energy to knock a firetruck onto it's side if you hit it side-on and freeway ramps dump out right onto residential streets. Parts fail, floormats get stuck, people don't wear their seatbelts, and you can get a license to ride on a motorcycle if you want.
It's a guess based on the research I've done, but my expectation is around 20% of fatal accidents can in some way be prevented by automation. You'd honestly prevent more fatalities by putting an ignition interlock on everyone's vehicle or building real barriers between traffic and pedestrians.
> a large portion of accidents are single vehicle accidents where the driver was at fault for the crash. Usually due to speeding, alcohol, youth, or a combination of them.
Also plenty of suicides in that group, which confuses the stats.
We really need SDCs to have fewer accidents than human drivers, excluding the suicides.
Not to mention waymo works well with LiDAR, cameras and radars. If you’re argument is it’s too hard to deal with that much data, it’s definitely the wrong answer.
> While it is true that humans do fine with only eyes
We do not. Humans are terrible at driving. Traffic accidents are one of the leading causes of death in the developed world. Billions of dollars of property damage occur every year because humans are not up to the task. A self driving system that is as safe as an average human driver would be an absolute failure.
> Traffic accidents are one of the leading causes of death in the developed world.
Perhaps leading cause of premature death or leading cause of accidental death or leading cause in demographics who are otherwise unlikely to die, but they are nowhere close to the top of the overall list.
Humans are amazing at driving. We typically go millions of miles in a lifetime without causing any fatal vehicle crashes and can generally handle unknown situations just fine.
> Traffic accidents are one of the leading causes of death in the developed world.
Not even close, really. A bit under 1%. You are more likely to die from an overdose, or suicide. And much, much, much more likely to die from cancer or heart disease.
And that is without getting into the trade-offs. Cars at least have a significant utility value, which is not true of suicide, opiate addiction, cancer, or heart disease. We should try to reduce traffic deaths, but we should not lose perspective.
I was not precise enough in the wording. Leading cause of accidental death. Obviously it is not beating out old age or heart disease. Doesn't change the fact that a self driving system with the record of a typical human would be considered unacceptable.
I agree on that point. For self driving to be accepted, it has to be at least as good as a good driver. Drunks, oldsters, youngsters, and the like all push the average down. Many normal people are actually pretty good drivers statistically, and those are (perhaps not coincidentally) the ones most likely to be in a position to afford a fancy new self-driving car.
You're right to note the advantages of lidar and (narrow range of) contrast problem for cameras (they arent eyes.) This is why the Uber human driver shouldn't have trusted the machine at night, in particular.
But you still have to address his system argument, which was that adding geegaws that added little would actually increase overall risks along the supply chain (plus maintenance) while distracting the team and adding more risk that way, for very little apparent (but only apparent) gain. The team does believe that they'll get to better than human driving, and do that without lidar.
"But you still have to address his system argument"
Do you? It's his argument that he needs to substantiate... it's not my burden to confirm his conjecture. And even looking at it on its face... it's clearly self-serving bs. It doesn't seem to be a problem for any other car company, so I'm a bit confused as to why it's such a problem for Tesla. Of course, the obvious answer is that Tesla is cheap and doesn't want to pay to have a team that would have sufficient bandwidth to do what every other car company and self-driving system is doing...
You still have to address the argument he makes that the net effect on risk of keeping the lidar was in fact negative. You're just gainsaying it. That's not a contribution to any discussion.
So you're again gainsaying his explanation, but gainsaying says nothing that helps anyone else make a decision. How are the rest of us who aren't psychic to know he's lying?
I'm not gainsaying his explanation. You don't seem to understand that a function of his explanation is what Tesla is willing to invest and their ability to recoup in a sale of the car.
Circling, circling, circling - but no specific analysis, citations, or criticisms of Tesla's stated logic. Gainsaying isn't a positive contribution to a conversation.
That's especially amusing considering how if you view Go as a visual problem, it's beyond trivial compared to what is required to safely operate a motor vehicle (even just visually).
There's a new article showing up just today at HN detailing more issues securing supply lines (for libraries and much more) re software; it's a deep problem, not a non-existant problem. But it doesn't have to be malicious actors at work. SpaceX lost a rocket because a supplier's product wasn't built to spec. SpaceX makes that part themselves now, and highly favors vertical integration, which just means making your own sh*t. Just maintaining documentation is a bear. There's another article on HN today detailing an Air Astana near crash because of "improvement" maintenance changes (with poor documentaion) that went very badly wrong.
> It is not sufficient if, in aggregate, self-driving cars have fewer accidents
It would be sufficient if it would be the case. With actual proof.
Reality is that in limited abstract situations, self driving card maybe have some advantages. But, that is all that we can claim. And when self driving fails, somehow human is always the cause.
I disagree, but mainly because of the way humans perceive risk.
From a public standpoint I don’t think it’s sufficient because there’s inherent trust lacking in an automated system. With ape-driven systems we have a certain amount of trust because we can more accurately intuit what the other ape is reasonably thinking. This is not the case with autonomous driving which leads to a wider amount of uncertainty. Not unlike how we are intuitively less trusting of someone who is legitimately “crazy” even if statistically we don’ can’t say they are shown to be more dangerous.
You’re right, but that still misses the point. More frequently updated software doesn’t make people intuitively trust it more. For some, it can do the opposite by making them question why it needs to be updated so much to begin with.
Humans don’t generally measure risks statistically. They do so emotionally and with lots of cognitive biases. You don’t alleviate that with more and more facts, unfortunately.
Sorry if I was unclear. Software is more unpredictable in my view, both because of fleet updates and it being so fundamentally different than humans. (On the whole, I realize AI and ML could work similar to some human processes.)
Having seen their AI Day, I supposed this was all about a unified, pseudo-visual voxel representation – and especially about generating scenarios. Apparently these have become a crucial part of the system and generating a broader variety of sensor data would be a considerable liability.
Cameras have poor dynamic range and can be easily blinded by bright surfaces. While it is true that humans do fine with only eyes, our eyes are significantly better than cameras.
More importantly, expectations are higher when an automated system is driving the car. It is not sufficient if, in aggregate, self-driving cars have fewer accidents. If you lose a loved one in an accident where the accident could have been easily avoided if a human was driving, then you're not going to be mollified to hear that in aggregate, fewer people are being killed by self-driving cars! You'd be outraged to hear such a justification! The expectation therefore is that in each individual injury accident a human clearly could not have handled the situation any better. Self-driving cars have to be significantly better than humans to be accepted by society, and that means it has to have better-than-human levels of vision (which lidars provide).
[1] https://www.youtube.com/watch?v=X3hrKnv0dPQ