Tesla staged 2016 self-driving demo, says senior Autopilot engineer

graylshaped

Ars Legatus Legionis
67,723
Subscriptor++
I'm reminded of the joke/story about the not-from-the US couple who arrive for graduate school a few days early, but are allowed into their graduate housing. Since the campus food service isn't up and running, they go to a typical American grocery store to buy a few days of food, using the pictures on the cans to guide them.

They bring back 2 cans of Crisco.

(While Crisco cans often feature a beautiful cherry pie and apparently do so currently, at the time Crisco was pushing using it for fried chicken, and that was the can art I most remember from childhood, like this:)
Herb-batter-fried-chicken-recipe-1985-750x1021.jpg

My sophomore year as an undergrad, I got to the dorms the Sunday they opened after Christmas, driving carefully through heavy snow. It got heavier. Classes were canceled for the first week. The 25% of us who had been able to get to the dorms for that week could have had a much better party if the cafeterias weren't on skeleton crews, but we managed.
 
Upvote
0 (2 / -2)

Random_stranger

Ars Praefectus
5,217
Subscriptor
Tesla removing shit doesn't surprise me anymore. What does surprise me is that the performance upgrades only upgraded the rear brakes? Wtf?

My guess is that they also upgraded the front brakes and are still shipping with Brembos on the front, just not the rear. Rear brakes get a lot less attention, and maybe they were hoping to slip it in without anyone noticing? Save a quick $100-200?
 
Upvote
6 (6 / 0)

ERIFNOMI

Ars Legatus Legionis
17,198
My guess is that they also upgraded the front brakes and are still shipping with Brembos on the front, just not the rear. Rear brakes get a lot less attention, and maybe they were hoping to slip it in without anyone noticing? Save a quick $100-200?
It's possible I misunderstood. I read it as they were previously only upgrading the rears. Mostly because the argument is that the site still says the brakes are upgraded but now they're not. If the fronts are still upgraded, that's honestly fine. The rear brakes do next to nothing anyway. My car has massive 6 piston Brembos on the front but the rears are standard.

Silently making the change and trying to hide it behind fake covers is still fucked though.
 
Upvote
9 (9 / 0)
Post content hidden for low score. Show…

ranthog

Ars Legatus Legionis
15,240
My guess is that they also upgraded the front brakes and are still shipping with Brembos on the front, just not the rear. Rear brakes get a lot less attention, and maybe they were hoping to slip it in without anyone noticing? Save a quick $100-200?
That has been their MO since the start of the pandemic. They've stripped a lot of features out of the cars after people ordered the cars expecting those features.

How much of a silent price hike this is is questionable, since it is hard to know how you'd structure pricing them out if they weren't just standard features on the vehicle. But I got to imagine a conservative estimate is at least $2k to $3k on most of their models.
 
Upvote
7 (7 / 0)
Oh also in this same deposition? Ashok Elluswamy, the head of autopilot team, doesn't know what an operational design domain is.


View: https://twitter.com/moodyhikmet/status/1614743058092019712


Jesus. I'm basically a beginning programmer and most of what I know about self driving comes from Ars Technica and a long time ago elektrik.

How on earth could he not know those terms? I didn't know ODD by that term but obviously know it's use case and I knew the rest by the jargon presented.

So either an idiot is in charge or perjury is taking place. Maybe both.
 
Upvote
9 (10 / -1)
Literally Autopilot and FSD are marketed as Level 2 automation, while Honda and Mercedes-Benz offers vehicles at Level 3.

Ford, via BlueCruise, offers hands free lane changes. GM, via Super Cruise, offers does the same and interstate changes.

Tesla's "offerings" aren't novel or superior.
The GMC Sierra also offers TOWING while on SuperCruise with compensation for trailer sway. Competitors have leapfrogged Autopilot at this point.
 
Upvote
13 (13 / 0)

ranthog

Ars Legatus Legionis
15,240
This all happened years ago. Why is this brought up now? Did something leak? Nowhere in the article does it state why it's currently being reported.

Musk and Tesla being exposed is certainly not surprising.
It was difficult to verify the claims made in the video were not easy back then . The video may not be considered deceptive if there were specific routes FSD could reliable drive. At that point pretty much everything about the program was not public knowledge, and Tesla wasn't promising the results right away.

Since then we've had the FSD beta program on the road and in public for anyone to see just how non-functional the system is. It is also many years since the systems should have been ready based on Tesla's promises to the users to deliver.
 
Upvote
9 (9 / 0)

AFRider

Smack-Fu Master, in training
90
Subscriptor
I can't tell you what model year the first Corolla I drove was, but the most recent one was a 2022 according to my receipt. I also distinctly remember seeing the acronym "LTA" appear on the display in both cars when I hit the button to toggle the feature on/off. I'm pretty sure the feature I used was in fact LTA and was meant to keep the car centered.

You make a great point about having trouble communicating what this system is actually supposed to be doing. This problem has also been illustrated by you, me, and ranthog having difficulty determining what this lane keeping feature in this specific car was even called. I think this supports the statement in my original post that we need enforced standards for ADAS in passenger cars. It's OK that cruise control is just cruise control because at this point we all know what that means, but the minutia of different types of lane keeping and other automation are just too difficult to communicate succinctly to a driver who isn't already familiar with the system.

I had a relative who recently bought a new Subaru Forester. The dealership asked them to come back after driving it for a few weeks for orientation, the theory being that it is too overwhelming to absorb everything at once. They love it, and use all the driving assist features.

And to save someone else from looking it up:

Operational Design Domain

Operational design domain means a description of the specific domain or domains in which an automated driving system is designed to properly operate, including types of roadways, ranges of speed, weather, time of day, and environmental conditions.
 
Last edited:
Upvote
8 (8 / 0)

aexcorp

Ars Praefectus
3,314
Subscriptor
That's a much safer and smarter way to do it, yes.


Except when it doesn't and people get hurt or killed.
I have not compared AP against the others on the market, but my experience is decidedly mixed. It does work pretty well most of the time, but there are a few things that are quite annoying with it:
(1) phantom braking still does happen at times, despite the various OTA updates in the last year or two, and it's rather irritating and can be dangerous in terms of getting rear-ended. I know it would be the other driver's fault but who cares; I don't want to have to deal with insurance, repairs, etc. especially considering that repairs on Teslas can take some time.
(2) AP brakes much harder than seems necessary during routine driving. It's unpleasant for those in the car and I can't see any reason why it doesn't just cruise and start to regen brake a bit earlier when brake lights show up or the car in front starts to decelerate. My only theory is that this approach somehow regens more energy, but I have no clue.
(3) In odd opposition to #2, AP also takes a long time to decide to accelerate when cars ahead are doing the same, leaving sometimes large gaps that other drivers love to abuse. This is true even when setting the distance to minimum (which is now 2 cars after Tesla disabled radar even on cars like mine that have the sensors). I sometimes find myself having to press the accelerator to compensate for the laziness of AP.
 
Upvote
6 (6 / 0)

iollmann

Ars Scholae Palatinae
1,254
This video was probably the single biggest factor in the decision to add the FSD package when I bought my car, and the subject of FSD is the single biggest factor in why I'll never buy another Tesla.

I don't regret buying the car.

I don't regret buying Autopilot

I regret falling for the scam that was FSD, and I'm hoping some day to see a refund, or Elon in prison. Or both!
Engineering 101: Never, never, never buy a product for an unreleased feature. Wait for the feature to arrive or buy the lesser model without the feature. There are lots and lots of reasons why things that may seem easy aren’t, and as many or more reasons why even if it is easy, you will never get it because it is unprofitable.
 
Upvote
15 (15 / 0)

sleuth_diplomacy

Seniorius Lurkius
15
Subscriptor
I had a relative who recently bought a new Subaru Forester. The dealership asked them to come back after driving it for a few weeks for orientation, the theory being that it is too overwhelming to absorb everything at once. They love it, and use all the driving assist features.

And to save someone else from looking it up:

Operational Design Domain

Operational design domain means a description of the specific domain or domains in which an automated driving system is designed to properly operate, including types of roadways, ranges of speed, weather, time of day, and environmental conditions.
Your description of that Subaru dealership's practice sounds really good. Making this more widespread would be a good step towards solving the problem of communicating the limits of ADAS. The problem is that not every dealer or manufacturer currently does this, and it doesn't do anything to educate drivers who acquire those same vehicles used instead of new. If ADAS features and capabilities were standardized, education on them could be incorporated into standard driver's ed for new drivers, and everyone who already has a license would at least know that whatever knowledge they gained from a previous car would transfer over and/or be able to look up documents telling them what to expect.

This is starting to sound more like training for pilots, which might not be such a bad thing.
 
Upvote
4 (4 / 0)

co-lee

Ars Tribunus Angusticlavius
6,123
This all happened years ago. Why is this brought up now? Did something leak? Nowhere in the article does it state why it's currently being reported.

Musk and Tesla being exposed is certainly not surprising.
Elluswamy's deposition was just leaked.

The news hook is not that FSD doesn't drive by itself. The hook is that the head of Tesla's automated driving program testified under oath that the much-shared video was deliberately deceptive (er, represented aspirations rather than current capabilities). And that they put out the video and sold the package knowing they were promising capabilities they couldn't deliver. That testimony was not previously public.

(ETA: critical "not" in the last sentence)
 
Last edited:
Upvote
21 (21 / 0)

co-lee

Ars Tribunus Angusticlavius
6,123
Your description of that Subaru dealership's practice sounds really good. Making this more widespread would be a good step towards solving the problem of communicating the limits of ADAS. The problem is that not every dealer or manufacturer currently does this, and it doesn't do anything to educate drivers who acquire those same vehicles used instead of new. If ADAS features and capabilities were standardized, education on them could be incorporated into standard driver's ed for new drivers, and everyone who already has a license would at least know that whatever knowledge they gained from a previous car would transfer over and/or be able to look up documents telling them what to expect.

This is starting to sound more like training for pilots, which might not be such a bad thing.
You can expect that kind of service from a high-end luxury brand like Subaru.
Econobox manufacturers like Tesla can't afford that level of service given their tiny margins.
It's a tradeoff: pay Subaru prices and get support or pay those dirt cheap Tesla prices and you're on your own.
Different people will make different decisions given that tradeoff.


/s as needed
 
Upvote
11 (11 / 0)
This all happened years ago. Why is this brought up now? Did something leak? Nowhere in the article does it state why it's currently being reported.

Musk and Tesla being exposed is certainly not surprising.
It’s becoming relevant again now that the FSD beta is open to everyone, there have been several high-profile crashes, and multiple investigations in the news. If 5 years ago, something I said sounds like fraud, but it’s minor and more ephemeral, and all of a sudden it’s given new context (say, an investigation into fraud), it becomes newsworthy again. People reporting on these things are learning that context. Hence it is once again news (and bigger news than it was at the time, given the what it says about those issues today).
 
Upvote
12 (12 / 0)

ranthog

Ars Legatus Legionis
15,240
It’s becoming relevant again now that the FSD beta is open to everyone, there have been several high-profile crashes, and multiple investigations in the news. If 5 years ago, something I said sounds like fraud, but it’s minor and more ephemeral, and all of a sudden it’s given new context (say, an investigation into fraud), it becomes newsworthy again. People reporting on these things are learning that context. Hence it is once again news (and bigger news than it was at the time, given the what it says about those issues today).
Or in the case of the article today, there was damning testimoney from the person in charge of FSD development that the video was both staged and deceptive in nature. That is extremely damning form an insider.

It is sort of like being able to find a smoking gun.
 
Upvote
15 (15 / 0)
Or in the case of the article today, there was damning testimoney from the person in charge of FSD development that the video was both staged and deceptive in nature. That is extremely damning form an insider.

It is sort of like being able to find a smoking gun.
It’s my understanding that the deposition is old, and was not widely reported at the time. That’s what I’m talking about when I say the relevance and newsworthiness has grown with proper context. I may be wrong in that, but I’m pretty sure the deposition happened 2 or 3 years ago.
 
Upvote
7 (8 / -1)

BradTheGeek

Ars Tribunus Militum
1,878
Subscriptor
Head of the team says this, under oath, "The intent of the video was not to accurately portray what was available for customers in 2016."

and still a couple of the usual customers are carrying water for this shitpile failure. I wish I could be surprised.

"If I polish his shoes enough, the lord may bless me with a dogecoin!"
 
Upvote
8 (8 / 0)

DCStone

Ars Tribunus Militum
2,735
It’s my understanding that the deposition is old, and was not widely reported at the time. That’s what I’m talking about when I say the relevance and newsworthiness has grown with proper context. I may be wrong in that, but I’m pretty sure the deposition happened 2 or 3 years ago.
According to the Reuters piece linked in the Fine Article, the deposition of the Tesla engineer was taken in July. The Reuters piece is dated Jan. 17th 2023.
 
Upvote
14 (14 / 0)

nimble

Ars Scholae Palatinae
847
Subscriptor
It’s my understanding that the deposition is old, and was not widely reported at the time. That’s what I’m talking about when I say the relevance and newsworthiness has grown with proper context. I may be wrong in that, but I’m pretty sure the deposition happened 2 or 3 years ago.

Per the Reuters article, the deposition happened in July last year.
Elluswamy was deposed in a lawsuit against Tesla over a 2018 crash in Mountain View, California, that killed Apple engineer Walter Huang.

Andrew McDevitt, the lawyer who represents Huang’s wife and who questioned Elluswamy’s in July, told Reuters it was “obviously misleading to feature that video without any disclaimer or asterisk.”

Apparently depositions only become public if used as evidence at trial (unless leaked illegally):
https://depositionacademy.com/do-depositions-public-record
 
Upvote
15 (15 / 0)

Uragan

Ars Legatus Legionis
11,185
Elluswamy's deposition was just leaked.

The news hook is not that FSD doesn't drive by itself. The hook is that the head of Tesla's automated driving program testified under oath that the much-shared video was deliberately deceptive (er, represented aspirations rather than current capabilities). And that they put out the video and sold the package knowing they were promising capabilities they couldn't deliver. That testimony was not previously public.

(ETA: critical "not" in the last sentence)
Was his deposition actually "leaked"?
 
Upvote
4 (4 / 0)

Longmile149

Ars Scholae Palatinae
2,587
Wasn't it shown by someone that in fact, only a SINGLE camera was responsible for environmental awareness in FSD mode? I seem to recall in the report that they disabled or covered up all the other sensors and cameras leaving only the front facing camera and the car still worked in FSD mode and no errors or warnings were reported due to all these other systems not working, and the FSD did not behave any differently than before.

So in line with their argument that the demo wasn't meant to portray existing functionality but potential functionality, perhaps the presence of all those extra sensors and cameras did not mean that they were actually being used, but could potentially be used. , but are now being cost-reduced out of the design as they really were not being used :rolleyes:. I think the interior driver monitoring cameras are now active in the latest FSD, but that's obviously mainly to protect Tesla from lawsuits and liability in case of driver inattention..
Do you have any links to back that up?

It wouldn’t surprise me, but that’s a tough pill to swallow unsourced even when we’re talking about Tesla.
 
Upvote
7 (7 / 0)
They're not wrong that some 2020+ cars' driver assist is frighteningly terrible but I suspect they probably tried it on like, one car and have extrapolated that to "every car."

Little story. I rented a 2020 or 2021 Corolla when traveling last year and I was terrified by its lane keeping. Its cruise control was nice and smoother and smarter than my 2015 Impreza, but dear god it would drift fucking frighteningly close to the lines. I had to yank back control every time. The roads were pristine and new with clear lane lines and no "false lines" from things like old lane lines or crack repairs. The weather was nice, minimal overcast during the summer, so very even lighting conditions. It was essentially ideal conditions and my wife and I were scared shitless. It may have never drifted out of the lane, I can't know for sure, but damn it was pushing well beyond the comfort zone.

It may be anecdotal but the conditions felt so ideal I'm kind of blown away it's even on the market. It was legitimately scary. Also one reason I feel it's not just my experience is because there's only one fucking camera and no real stereo vision. The radar obviates the need for a second camera for cruise control but it doesn't do dick for lane keeping. I really don't get it.

Oh also I tried the same feature on a Civic through Zipcar and that thing did a fine job. Maybe Toyota "hydrogen is a good idea" Motor Corporation doesn't give a shit like they used to.
That's because the 2021 Corolla has Lane Tracing Assist, not lane centering. Other companies call the same thing Lane Keeping Assist.

https://www.enstoyota.ca/toyota-corolla-safety-features/

Driving Reimagined By Lane Tracing Assist​

Lane tracing assist is an extension of the lane departure alert and is utilized when the car is in DRCC mode. The point of the assist is to ensure the vehicle stays within its marked line while travelling at a set speed. However, drivers should be aware that lane tracing assist is not meant to be a substitute for driving and is not an autonomous driving function. Your hands need to remain on the wheel at all times and your eyes on the road, but lane tracing assist and DRCC are both great backups that are designed to function as safety nets while on the road.

It's there to prevent you drifting out of the lane, not drive for you. You were supposed to continue driving and it was there to keep you in the lane if you got momentarily distracted.
 
Upvote
8 (8 / 0)
This needs to be split into two separate topics.

1) I believe Tesla committed knowing and deliberate fraud about FSD and Autopilot. I think that their plan was to deliberately exaggerate the capabilities of these systems and to have the public confuse these two systems. The latter is evident in how they put the excessively staged / faked video about FSD on the Autopilot page of their website. They put in weasel words to try and prevent lawsuits, but they also put in the video the line about the driver only being there for legal purposes. In other words, their implicit message was "Ignore the disclaimers. They are only there because our lawyers made us. This totally legitimate video shows you what our systems can really do." They need to pay dearly for this fraud and the people they have killed with it. In no way does Huang's actions reduce or remove Tesla's culpability.

2) Huang knew Autopilot behaved dangerously at a certain location and that it took manual intervention to prevent an accident, yet he apparently used Autopilot at this location without properly monitoring it. The only way his behavior makes any sort of sense is if Autopilot version N behaved dangerously, Autopilot version N+1 fixed it, and Autopilot version N+2 broke it again. I've not seen any evidence of this.

Let me give you an exaggerated example of my thinking on this. Suppose you make a gun with a safety that fails 10% of the time, and yet you advertise the safety is utterly reliable. Now suppose I take that gun, load it, put on the safety, point it at my head and pull the trigger. The mess on the wall that results is due to both your gun's faulty and falsely advertised safety mechanism, and my violating the basic rules of gun safety. The accident would not have happened without both, and neither excuses the other.

See the thing is humans suck at monitoring a system that mostly gets it right with occasional failures. Humans get bored and think about other things then miss some critical detail. This is and has been known for decades.

And yes people will lose focus even when their lives are on the line. There's a famous story of nitroglycerin workers falling asleep while controlling the mix. https://www.wattylercountrypark.org.uk/safety

Search up Vigilance Task or Vigilance Problem, there's plenty of research in this area.
 
Upvote
14 (14 / 0)

numerobis

Ars Tribunus Angusticlavius
50,241
Subscriptor
Jesus. I'm basically a beginning programmer and most of what I know about self driving comes from Ars Technica and a long time ago elektrik.

How on earth could he not know those terms? I didn't know ODD by that term but obviously know it's use case and I knew the rest by the jargon presented.

So either an idiot is in charge or perjury is taking place. Maybe both.
I would bet their lawyer advised claiming ignorance if there was any ambiguity possible.
 
Upvote
9 (9 / 0)

sd70mac

Ars Tribunus Militum
2,610
Subscriptor
Unfortunately without LIDAR or RADAR, it may be tough to get around the corner without killing anyone.
Frankly, I’m amazed that it’s even possible at all with two cameras on a rotating mount, two microphones, and stability sensors. (I’m talking about the MK I human.)
 
Upvote
9 (9 / 0)
"The person in the driver's seat is only there for legal reasons. He is not doing anything. The car is driving itself."
#TFW you realize, a few years on, that the reason that person is there for legal reasons is so Elon's crappy AI can dump all the legal liability on the human after it crashes the car.
 
Upvote
11 (11 / 0)

numerobis

Ars Tribunus Angusticlavius
50,241
Subscriptor
Upvote
11 (11 / 0)

sd70mac

Ars Tribunus Militum
2,610
Subscriptor
On an article about how Tesla faked a demo, you're going to believe Musk when he predicts a million FSD drivers by the end of the year?
We can take it with salt, but absent alternative sources, we don’t have much information to go on. That number is also probably a little lower than the actual total, given that it is only for North America. We must remember that many people paid the extra $7500 to get their car sooner, as orders with FSD were prioritized for delivery during periods of high demand, and rolling it into a loan increased the payments only modestly, around $100 per month I calculated.
 
Upvote
6 (6 / 0)
So the Tesla fans' view is:

1. If a fault occurs, it's the fault of the driver.
2. If there is a fault in Tesla's system, see #1.
Isn't the liability here such that if FSD/Autopilot/whatever it is this quarter is in control and then the driver takes over and there's still an accident that the FSD/Tesla is legally off the hook since the driver was in control at the time of the accident? Perverse incentives there...
 
Upvote
1 (1 / 0)

markgo

Ars Praefectus
3,779
Subscriptor++
Wasn't it shown by someone that in fact, only a SINGLE camera was responsible for environmental awareness in FSD mode? I seem to recall in the report that they disabled or covered up all the other sensors and cameras leaving only the front facing camera and the car still worked in FSD mode and no errors or warnings were reported due to all these other systems not working, and the FSD did not behave any differently than before.

So in line with their argument that the demo wasn't meant to portray existing functionality but potential functionality, perhaps the presence of all those extra sensors and cameras did not mean that they were actually being used, but could potentially be used. , but are now being cost-reduced out of the design as they really were not being used :rolleyes:. I think the interior driver monitoring cameras are now active in the latest FSD, but that's obviously mainly to protect Tesla from lawsuits and liability in case of driver inattention..

1) Radar equipped systems appear to perform much better at eliminating high speed phantom braking events. It was used for this, now disabled even on models with the hardware.

2) Sensor fusion is a thing. Every other self driving system is doing it. Elon is maniacally directing his team to do it with cameras only, even when other systems (radar for emergency braking, ultrasonic for parking distance) are reliable and reasonably cheap.
 
Upvote
14 (14 / 0)
This all happened years ago. Why is this brought up now? Did something leak? Nowhere in the article does it state why it's currently being reported.

The video, which now has been proven to be faked and to not show actual FSD but instead a preprogrammed drive, is STILL being used by Tesla to advertise FSD on their website.

https://www.tesla.com/autopilot
 
Upvote
18 (18 / 0)

Longmile149

Ars Scholae Palatinae
2,587
Do you have any links to back that up?

It wouldn’t surprise me, but that’s a tough pill to swallow unsourced even when we’re talking about Tesla.
Oh my god

I went looking for it and I was not prepared for how incredibly, unbelievably horrifying this is.


View: https://m.youtube.com/watch?v=77cbSy-rBEQ&feature=youtu.be


It’s like 12 minutes long and the driver is a true believer. He doesn’t seem to understand at all the implications of what he’s demonstrating.

He disables most of the car’s sensors and it still allows him to engage both FSD and Autopilot.

This video is wild.

At one point he’s got the radar covered with aluminum foil and the car is intermittently reporting a vehicle inches in front of him and it still fires up the FSD and takes off because the cameras don’t see anything.

They aren’t doing any sensor fusion or cross-checking. There’s damn near no failsafe, no error checking, and no sanity checks. There were two times in the video that his FSD wouldn’t engage, and one of those times all he had to do was hit the button again and it started right up.

For almost the entire video he’s got all the side and rear cameras covered and he talks a couple times about how cool it is that the car remembers there’s stuff there as it continues to visualize him moving past it even though he knows the car can’t see anything there.

Watch this.
 
Upvote
18 (18 / 0)