Steering failures are Tesla’s new federal safety worry

DRJlaw

Ars Praefectus
5,828
Subscriptor
It would appear that Tesla is, in fact, different because the power steering is electric and not hydraulic.

Nope. There are many other modern passenger automobiles with electric power steering. If this is somehow related to Autopilot/FSD software rather than the basic electromotive unit, Tesla will have found a very much different way to mess up an otherwise standardized feature. Or it could be an issue with the basic electromotive unit and sensing that's not much different. Thus, why it should be investigated rather than handwaved away based upon claims that Telsas are better per vehicle mile driven or have fewer recalls.

Look into the issue, find the issue, fix the issue. Just like normal, everyday automakers are supposed to do.
 
Upvote
15 (15 / 0)

Eigenvogel

Ars Scholae Palatinae
1,416
It would appear that Tesla is, in fact, different because the power steering is electric and not hydraulic. I am relieved that they do not run their entire steering array by wire (seriously, very relieved) but it being entirely shut down by a power steering issue is far from ideal.
My guess is it's not literally completely shut down, but the steering ratio, overall weight, and tire size of the cars is such that steering without power assistance is very difficult. Alignment comes into this too -- more caster will make cars return to center better but makes steering harder because of the "jacking effect," where you're literally lifting the front end of the car as you turn the wheel off center.
 
Upvote
10 (10 / 0)

sryan2k1

Ars Legatus Legionis
46,537
Subscriptor++
I have rebooted my 2018 Tesla while driving. Scary, but it continued to work just fine. Of course, I was not dealing with hardware issues at the time, but software issues. Can't imagine this would be good to try when faced with a power steering issue.


I'm partial to "Gleeks". I realize it doesn't riff off of their new name, but being tweeted at is about as pleasant a prospect as being gleeked on is to me at this point.
No, you rebooted the infotainment computer. Not the car parts of the car.
 
Upvote
-6 (0 / -6)

cerata

Ars Centurion
239
Subscriptor
Mean Time Before Failure exists for a reason. The expected value should be non-zero, but it should also be extremely low. Nearly zero.
"Failing gracefully" also exists for a reason. If power steering fails, even temporarily, a 2-ton mass traveling at highway speeds ought to remain manually steerable.

Other human drivers should also practice basic safety & courtesy and maintain sufficient distance. The driver of a car affected in this way should be able to switch on hazard lights and come to a full stop in the middle of the highway, without risking a collision or verbal/physical abuse. But that's its own problem, and the main reason I don't drive¹.

Safety-critical electronic/software design is way beyond my pay grade, so perhaps this is something Tesla engineers couldn't have reasonably foreseen,and perhaps not enough of these anecdotes were reported to Tesla for them to be identified as a possible systemic issue. But naïvely, I think power steering firmware could and should be simple enough to exhaustively test, and resilient even to cosmic ray bit-flips. At the least, it ought to unaffected by console updates/reboots.

¹ I don't mean any disrespect to people who need to drive, whether due to work or where they live. My partner's visual disability prevents them from driving, so car-essential suburbs literally aren't an option for us.
 
Upvote
3 (4 / -1)

iim

Ars Tribunus Militum
1,710
What is the expected rate of failure for power steering?
Very low, much more reliable than hydraulic steering.

Electric steering is one electrified part of the car that whole auto industry had already decided they were going to change over to before Tesla was a thing.

The set up that Tesla uses is almost identical to to the electric steering of almost any ICE vehicle.

The one drawback of electric steering, if there is one, as they tend to not give much indication when they are about to fail, and they just work until they don’t.

The important thing is the wheel is still mechanically linked to the steering wheel so it will turn even without power steering. It’s just going to take a lot of effort so don’t plan on any slow speed U-turns unless physically fit.
 
Upvote
7 (7 / 0)

iim

Ars Tribunus Militum
1,710
Question - aren't Tesla's steer-by-wire? In which case the steering wheel itself basically functions the way a steering wheel controller or joystick plugged into a PC's USB port does, rather than the way that a steering wheel in a hydraulic (or electrically assisted power-steering) car does.
Last I checked, they still have the front wheels mechanically linked to the steering wheel.

Steer by wire has been a thing for sometime now anyone with rear wheel steering already has it on the back wheels.
 
Upvote
1 (1 / 0)
I have rebooted my 2018 Tesla while driving. Scary, but it continued to work just fine. Of course, I was not dealing with hardware issues at the time, but software issues. Can't imagine this would be good to try when faced with a power steering issue.


I'm partial to "Gleeks". I realize it doesn't riff off of their new name, but being tweeted at is about as pleasant a prospect as being gleeked on is to me at this point.
He said "Turn it off" not "reboot the IVI system".
 
Upvote
0 (0 / 0)

Xenocrates

Ars Tribunus Militum
2,503
Subscriptor++
Kamitchell, you said "Power steering should not be affected at all by a software failure." I admire that desire but reality is very different.

Consider that in electric power steering (EPS) the amount of power steering assist is a software controlled function with input from a few sensors (steering angle and torque at least, possibly also some vehicle gyro or other sensors). Old fashioned hydraulic steering can be controlled directly by a 3-way hydraulic valve that moved (rotated) directly by the steering column. How does a steering column directly control an electric steering motor like that? It doesn't.

Think about something as simple as the on-center "dead band." Vehicle steering needs a small zone on-center that includes no assist or force in either direction, no active electric motor assist (possibly a holding force but that causes other trouble). That dead band is carefully tuned in software because steering designs vary so much (steering geometry, sensor responsiveness/accuracy/stability, etcetera). You can't just pick or 'hardwire' the same dead band for every car. Similar for the assist profiles and how the system detects failures in sensors or other physical causes (e.g. hitting a curb may change steering angle faster than humanly possible, therefore be detectable in software). There are other software detections if the system shorts to voltage supply or ground (throw a diagnostic code and shut off the assist).
Now, I can see your point. However, dead-band can be pretty effectively tuned in with good old fashioned analogue circuits if you wanted to. It's just easier and more reliable in a safety rated software solution these days. The sensors can be read by multiple systems, so the diagnostic software doesn't need to have anything to do with the actual assist solution.

But I think the biggest thing to focus on is the fact that the power steering software was:
1) Updated OTA, without a specific technician intervention to trigger the update to the PS software/firmware.
2) Written so poorly as to totally crash out and fully kill the assist, without setting a persistent diagnostic flag for some form of failure such that it was predictable and repairable.
3) Deployed to a nation-wide fleet, rather than tested exhaustively in a limited scope encompassing applicable vehicle configurations.

NONE of that should be the case with safety critical software. Safety critical software should only be updated from a known working configuration at explicit intervention from a qualified individual. Safety critical software should, if it fails, do so in a fail-safe fashion and generally, should set alarms to have whatever defect triggered the failure fixed immediately. Safety critical software should be tested against all expected conditions, both hardware configuration and inputs, to determine potential failures before it is deployed.

This sort of stuff is very expensive to develop for properly, and a limited number of qualified people know how to do it right. I'm not qualified to write safety critical software outside of relying on pre-existing hardware and software guardrails in PLCs for safety rated functions. I don't know that anyone qualified to write that kind of software actually works at Tesla, but I can say for sure that the folks managing Tesla's software safety programs are NOT qualified to do so, given the persistent issues and FSD beta.
 
Upvote
5 (5 / 0)

Numfuddle

Ars Tribunus Militum
2,641
Subscriptor
"Failing gracefully" also exists for a reason. If power steering fails, even temporarily, a 2-ton mass traveling at highway speeds ought to remain manually steerable.

Other human drivers should also practice basic safety & courtesy and maintain sufficient distance. The driver of a car affected in this way should be able to switch on hazard lights and come to a full stop in the middle of the highway, without risking a collision or verbal/physical abuse. But that's its own problem, and the main reason I don't drive¹.

Safety-critical electronic/software design is way beyond my pay grade, so perhaps this is something Tesla engineers couldn't have reasonably foreseen,and perhaps not enough of these anecdotes were reported to Tesla for them to be identified as a possible systemic issue. But naïvely, I think power steering firmware could and should be simple enough to exhaustively test, and resilient even to cosmic ray bit-flips. At the least, it ought to unaffected by console updates/reboots.

¹ I don't mean any disrespect to people who need to drive, whether due to work or where they live. My partner's visual disability prevents them from driving, so car-essential suburbs literally aren't an option for us.
I can only speak for EU regulations and mostly from experience with German OEMs but it should be comparable.

EU OEMs specified their own SW development process workflow - derived from SPICE - called Automotive SPICE. and basically every supplier is required to use it. For safety critical systems they also mandate an ISO 26262 derived workflow called ASIL that specifies different safety/integrity levels based on impact/criticality of a failure - Level A being the lowest/least critical and Level D being the highest/most critical.

ASIL D is basically: "if this system fails someone may die" so unsurprisingly steering would be classified as ASIl level D.

ASIL prescribes workflows, analysis steps, testing steps, etc. depending on the ASIl level. If developing a system in accordance with a certain ASIl level is not possible or unfeasible you can "de-compose" the system into multiple redundant parts of an ASIL level one tier lower, e.g if you can't design a system as ASIL D you can dicide it into multiple redundant modules of level ASIL C. How this whole safety analysis/classification and de-composition is done is also governed by the respective standards.

One input into the whole process is a failure mode and effects analysis (FMEA) which is used to identify failure modes and their effects and which is also used to sort them into classes depeneding on a number of criteria. Most importantly impact, How severe is the outcome if a system fails?

Steering failure would be a "10" on the FMEA safety level. Impact is: "people can die or be severely harmed if this system fails"

It has to be ensured that ASIL level D systems never fail completely. Either you can prove that the system can never fail in a certain way (hard to impossible) or you have to design ways in which such systems can gracefully degrade into a safe or fail over state. These failure mitigation steps have to be proven to work and ensured by the specified workflow and analysis steps mandated by the standard.

A power steering failure that degrades into "un-assisted steering" would be OK if the car can still be controlled without the assist system active. A power steering failure that degrades into "steering is now stuck in a certain position and the vehicle is no longer controllable" is not an acceptable outcome. This failure mode should also have been identified in the FMEA step of the safety analysis and mitigation/prevention should have been specified and impkemented that prevents this failure mode from ever occuring.

This is why no one does true steer by wire. You'd either have to prove that your implementation of a steer by wire system can never fail (impossible) or you'd have to define a fail-over into a safe state if the steer by wire system fails i.e a conventional steering rack.

This is just a brief glimpse into what you have to do to design and implement a safety critical system but Tesla truly screwed up here if a simple SW update can break their steering completely.
 
Upvote
14 (14 / 0)

Penforhire

Ars Tribunus Angusticlavius
6,515
Subscriptor
Xenocrates, to your point about OTA fixes, depending on the sensor/software configuration there is a mechanical precise on-center that has to be set during major changes or retrofits (make sure wheels are pointed straight ahead and record that position as a reference). It may not be necessary if the changes just carry forward a factory setting but this is certainly needed if the sensor (or entire steering column) is ever replaced.

You're right that analog circuits could 'hard wire' behaviors but that is less flexible than software. It may exist but I don't know of anyone's EPS system that uses that approach (every year now I get more out of touch with state of the art for EPS). Software is useful for things like error detection and linearizing sensor signals anyway so its extension to system control is almost free.

Numfuddle, unless something has changed, steering is not required to have ASIL-D level of reliability despite how citical it is. That is proven by the field issues noted in this very article. It is required to have fault detection (ASIL-B) not fault tolerance (ASIL-D) so the system can turn off. If these systems were ASIL-D the driver would see a trouble light telling them to get service while the system detected the fault and changed to a parallel (alternate) control.

One of the easy ways to envision ASIL-D for just a steering sensor (or almost any other function) is to provide three independent signals and compare them constantly and use a "minority report" approach where you throw out an errant signal (compared to the other two). It seems like this should be more commonly used but it isn't. Three sensing elements (fault tolerant) are notably more expensive than two (fault detecting) and the system has other issues limiting ASIL level. Do you add a second electric (EPS) motor in case the first fails? More interesting at the sensor level, do you provide separate supply voltage and ground signals to each of the sensing elements I mentioned? If so, the wiring harness and connectors just got bigger/heavier/more expensive. If not, a single power supply failure (say a single connector pin that frets or otherwise opens up) ruins all your ASIL-D behavior for that sensor.
 
Upvote
1 (2 / -1)

DRJlaw

Ars Praefectus
5,828
Subscriptor
Upvote
7 (7 / 0)

ardent

Ars Legatus Legionis
12,466
My guess is it's not literally completely shut down, but the steering ratio, overall weight, and tire size of the cars is such that steering without power assistance is very difficult. Alignment comes into this too -- more caster will make cars return to center better but makes steering harder because of the "jacking effect," where you're literally lifting the front end of the car as you turn the wheel off center.
I own a vehicle that doesn't have power steering. I assure you that if you try to turn the wheel with the vehicle stopped it requires a lot of upper body strength to move the linkage. But once the vehicle is moving it's...not great, ever, but usually fine. It's a debate for automotive engineers whether the turning radius on old vehicles sucked because of not having power steering, or if not having power steering forced the turning radius to suck.

Nope. There are many other modern passenger automobiles with electric power steering. If this is somehow related to Autopilot/FSD software rather than the basic electromotive unit, Tesla will have found a very much different way to mess up an otherwise standardized feature. Or it could be an issue with the basic electromotive unit and sensing that's not much different. Thus, why it should be investigated rather than handwaved away based upon claims that Telsas are better per vehicle mile driven or have fewer recalls.

Look into the issue, find the issue, fix the issue. Just like normal, everyday automakers are supposed to do.
That..."article"? wasn't terribly helpful. Looking for actual journalism on the subject it appears many cars manufactured after 2010 have at least partial electric assist and it is still pretty controversial despite being pretty proved out.

https://www.roadandtrack.com/car-cu...zda MX-5 Miata,that they use electric assist.
It seems Tesla is simply exceptional. Exceptionally bad, again.
 
Upvote
1 (2 / -1)

Numfuddle

Ars Tribunus Militum
2,641
Subscriptor
Numfuddle, unless something has changed, steering is not required to have ASIL-D level of reliability despite how citical it is. That is proven by the field issues noted in this very article. It is required to have fault detection (ASIL-B) not fault tolerance (ASIL-D) so the system can turn off. If these systems were ASIL-D the driver would see a trouble light telling them to get service while the system detected the fault and changed to a parallel (alternate) control.
Yes but the steering assist systems and the systems interacting with the steering (distance control, lane assist etc.) are required to have ASIL-D for the most part. I apologise that I wasn't making myself clear there.

edit: this includes electric power steering
 
Upvote
3 (3 / 0)

Penforhire

Ars Tribunus Angusticlavius
6,515
Subscriptor
DRJLaw, that NHTSA link is a good one for a general overview of EPS considerations but it does not say steering systems must be ASIL-D. See Table 22 on page 46, comparing the different possible ASIL goals or section 9.1 on page 59 for more description.

The link to Renasas is a sales pitch, not a requirement. They, like other sensor suppliers, offer sensors that can contribute to or ease development of an ASIL-D system. Big surprise that they want to sell extra capability ($$), right?

The IEEE notes the risk is rated ASIL-D. Totally agree. Kind of a no-brainer along with braking function. Those can kill you faster than or more easily that most other systems (even acceleration isn't quite as critical). But they are not saying current consumer systems are meeting ASIL-D.

More practically, I can prove most systems are not ASIL-D simply because they fail without warning us first. True ASIL-D has to be fault tolerant. These steering systems, like the Tesla steering sold today, are not. Everybody wants ASIL-D (highest system safety). Few applications are paying to accomplish it, certainly not (yet) in passenger cars.
 
Upvote
3 (3 / 0)

techchimp

Wise, Aged Ars Veteran
119
Not a week goes by that I don't feel vindicated in my defensive-driving decision to treat all Teslas on the road as potentially liable to try to kill me at a moment's notice for no reason at all.

I thought the problem was going to be a series of seemingly inexplicable rightward lurches.
I almost got hit by a self driving waymo yesterday.. it swerved to avoid a piece of paper that was being blown by the wind. The guy in the car grabbed the wheel to keep it from side-swiping me.
 
Upvote
0 (0 / 0)

steelcobra

Ars Tribunus Angusticlavius
9,891
Nope. There are many other modern passenger automobiles with electric power steering. If this is somehow related to Autopilot/FSD software rather than the basic electromotive unit, Tesla will have found a very much different way to mess up an otherwise standardized feature. Or it could be an issue with the basic electromotive unit and sensing that's not much different. Thus, why it should be investigated rather than handwaved away based upon claims that Telsas are better per vehicle mile driven or have fewer recalls.

Look into the issue, find the issue, fix the issue. Just like normal, everyday automakers are supposed to do.
Recalls require that the company admit to there being a problem as a starting point, though.

And since Teslas are a mix of "hope and pray the hardware is good with the iteration you got" with "Software-defined vehicle" it can be hard to know the exact cause.
 
Upvote
0 (0 / 0)

steelcobra

Ars Tribunus Angusticlavius
9,891
Xenocrates, to your point about OTA fixes, depending on the sensor/software configuration there is a mechanical precise on-center that has to be set during major changes or retrofits (make sure wheels are pointed straight ahead and record that position as a reference). It may not be necessary if the changes just carry forward a factory setting but this is certainly needed if the sensor (or entire steering column) is ever replaced.

You're right that analog circuits could 'hard wire' behaviors but that is less flexible than software. It may exist but I don't know of anyone's EPS system that uses that approach (every year now I get more out of touch with state of the art for EPS). Software is useful for things like error detection and linearizing sensor signals anyway so its extension to system control is almost free.

Numfuddle, unless something has changed, steering is not required to have ASIL-D level of reliability despite how citical it is. That is proven by the field issues noted in this very article. It is required to have fault detection (ASIL-B) not fault tolerance (ASIL-D) so the system can turn off. If these systems were ASIL-D the driver would see a trouble light telling them to get service while the system detected the fault and changed to a parallel (alternate) control.

One of the easy ways to envision ASIL-D for just a steering sensor (or almost any other function) is to provide three independent signals and compare them constantly and use a "minority report" approach where you throw out an errant signal (compared to the other two). It seems like this should be more commonly used but it isn't. Three sensing elements (fault tolerant) are notably more expensive than two (fault detecting) and the system has other issues limiting ASIL level. Do you add a second electric (EPS) motor in case the first fails? More interesting at the sensor level, do you provide separate supply voltage and ground signals to each of the sensing elements I mentioned? If so, the wiring harness and connectors just got bigger/heavier/more expensive. If not, a single power supply failure (say a single connector pin that frets or otherwise opens up) ruins all your ASIL-D behavior for that sensor.
Keep in mind that Musk's attitude is that Tesla should reinvent the wheel on everything they do and ignore the decades of experience other companies have in making a car safe to operate.
 
Upvote
2 (2 / 0)

Eigenvogel

Ars Scholae Palatinae
1,416
I own a vehicle that doesn't have power steering. I assure you that if you try to turn the wheel with the vehicle stopped it requires a lot of upper body strength to move the linkage. But once the vehicle is moving it's...not great, ever, but usually fine. It's a debate for automotive engineers whether the turning radius on old vehicles sucked because of not having power steering, or if not having power steering forced the turning radius to suck.
I don't think it's affected the radius much -- that's set more by fender well clearances and allowable CV joint angles.

But without power steering there are some handling trade offs. You might need to crank the wheel more turns to go from lock to lock, or the self-centering might be weaker due alignment settings meant to reduce steering effort. You might even need a bigger steering wheel for leverage, which leads to interior design and driving position trade offs.

Modern cars get to avoid those trade offs and the result is some of them have very heavy steering, even while moving, if the power assist fails.
 
Upvote
4 (4 / 0)

ardent

Ars Legatus Legionis
12,466
I don't think it's affected the radius much -- that's set more by fender well clearances and allowable CV joint angles.

But without power steering there are some handling trade offs. You might need to crank the wheel more turns to go from lock to lock, or the self-centering might be weaker due alignment settings meant to reduce steering effort. You might even need a bigger steering wheel for leverage, which leads to interior design and driving position trade offs.

Modern cars get to avoid those trade offs and the result is some of them have very heavy steering, even while moving, if the power assist fails.
I'm still inclined to believe that gearing decisions would have driven turning radius realities, but I'm not really interested into reading old auto engineering books to find out.
 
Upvote
0 (0 / 0)

Eigenvogel

Ars Scholae Palatinae
1,416
I'm still inclined to believe that gearing decisions would have driven turning radius realities, but I'm not really interested into reading old auto engineering books to find out.
Turning radius depends on how long the vehicle is and how sharp an angle the front wheels can make; gearing just controls how much you have to turn the steering wheel to get there.
 
Upvote
6 (6 / 0)

cerata

Ars Centurion
239
Subscriptor
One of the easy ways to envision ASIL-D for just a steering sensor (or almost any other function) is to provide three independent signals and compare them constantly and use a "minority report" approach where you throw out an errant signal (compared to the other two). It seems like this should be more commonly used but it isn't. Three sensing elements (fault tolerant) are notably more expensive than two (fault detecting) and the system has other issues limiting ASIL level. Do you add a second electric (EPS) motor in case the first fails? More interesting at the sensor level, do you provide separate supply voltage and ground signals to each of the sensing elements I mentioned? If so, the wiring harness and connectors just got bigger/heavier/more expensive. If not, a single power supply failure (say a single connector pin that frets or otherwise opens up) ruins all your ASIL-D behavior for that sensor.
Would three sets of steering sensors really increase the overall build-cost of the car by an appreciable amount? I could see duplicate power connectors (preferably routing the cables along different paths) doing so.

Thanks for the insights, everyone.
 
Upvote
0 (0 / 0)

Penforhire

Ars Tribunus Angusticlavius
6,515
Subscriptor
Unfortunately yes, even just a third sensing element is significant to the manufacturer’s BOM. I can’t speak exact dollars because even my historical knowledge is proprietary but GM et al really care about a few extra dollars here and there.

Keep in mind adding a sensing element costs a bit more than just the extra Hall effect element and (possibly) magnet (for one common technology, many arrangements won’t add another magnet). Complete sensor has to be reworked - housing, internal support circuitry (PCB and other interconnections), harness and usually the connector (adding a 3rd output), and, usually, magnetic flux concentrators. In-process and final electrical checks have to be refitted to check that 3rd signal.
 
Upvote
2 (2 / 0)

TylerH

Ars Praefectus
5,108
Subscriptor
Mush is nothing like Jobs. Jobs was abrasive but he knew how to delegate, he knew how to trust his subordinates to develop and execute on plans and products. Jobs did have the extra benefit of being right more often than he was wrong when making product decisions, but his sometimes abrasive feedback style was not the whole story. Mush is incapable of choosing to delegate and trust his subordinates, hence why SpaceX has to dedicate energy to distract him away from critical work so he won't meddle
They both are notorious for micro managing things
 
Upvote
0 (0 / 0)