Did you recently take on (or consider) a loan of 84 months or longer on a car purchase?
A reporter would like to speak with you about your experience; please reach out to PR@Edmunds.com by 7/22 for details.
A reporter would like to speak with you about your experience; please reach out to PR@Edmunds.com by 7/22 for details.
Options
Comments
It is no surprise these kinds of problems occur across ALL the various manufacturers . . . If you start with an imperfect-unreliable-time-multiplexed-latency-prone-software microprocessor approach, you end with an imperfect-unreliable-time-multiplexed-latency-prone-software microprocessor approach . . . The problems will persist . . .
Good Will toward Toyota and ALL the other car makers -
Gregg
As a person who changes out computer motherboards all the time, I've seen many computers boards that have had capacitors fail. The problem is not specific nor limited to one computer manufacturer either. Most of the capacitors however are manufactured by the same company because on just about every board replaced, these caps appears to be produced by the same company. Depending on which capacitor blows and what position it is on a computer board, it cause various issues with a PC. So what would the difference be with a car computer board? In a PC, I've seen caps blow that cause fans to rev up and in other cases, reboot consistantly, power up with nothing displayed, or not come on at all. A board that doesn't have the proper power being regulated can cause many issues, and it would seem to me, makes no differences when it comes to controlling a vehicle. Given the variables of heat and moisture, mix it with dirt, salt, and dust crossed with electricity, there is no telling what could actually happen.
I guess the only true way to figure this issue out is to put an actual car in the lab environment and start shorting out some caps and.or resistors to see what happens.
I'm an aerospace systems engineer and have a colleague who has a program that dynamically simulates simple models of the hardware/software interactions and automatically exposes the opportunities for software data integrity corruptions. This is the only automated method known to me to insure highly deterministic performance. It can be done manually, through brute force checks, but is only valid for that exact design implementation. Changing one line of code means it all must be done again from scratch.
I was going to say but I think I'm going to read more about this 1 Mil there giving out and send the info to them.
Good luck to all including Toyota engineers.
Thats just my take on it...That is why I only restore pre 1975 cars
A fix - if the problem can not be found - is to add a circuit that requires the throttle control to be undated in x amount of time - it it is not the gas is let off - so instead of crashing full speed into something you get rear ended by another vehicle and your auto explodes like the Pinto did.
This defect would require a lot of previous state conditions to occur. Since it is rare - I would expect that this is in fact what it is. Bottom line it is a software problem that can not be detected through normal testing and design verification analysis will be the only way to catch it. But untimately it is simple. Not enough stack allocated or the stack not being protected or protected using faulty methodology.
There is also the possibility that there are stack overwrite failsafes in the software so that when a stack overwrite is detected either too late or causes a state that can not be recovered from - since the watchdog timer fires - the stack overflow detect occurs again - and this just ends up in a unrecoverable loop.
Say a stray signal out of the cruise control spikes the acceleration control. A double-parallel circuit is reporting that the cruise control button is in "Rest" position on both channels, then the Comparison circuit takes note of that every 5 ms. When the spike comes without a matching "On" position from both positive and negative reads of the cruise control button then the comparison circuit rejects the artificial spike as a false command. Stretch the idea in a different angle and say the cruise control button is sticky and does not sit 100% at rest or in the On position. The accelerator comparison has to be designed to monitor all components that affect acceleration including the cruise control obviously. The first time there is a 5 ms inconsistency in the function of the cruise control button the system is alerted because either A) "Rest" is Off while "On" is not On, or,
This type of self-monitoring circuit design won regulator's praise when it proved flawless. And I've mentioned in previous remarks that the main problem you get with these systems is the inability to operate because everything is NOT perfectly within tolerance, that slightly worn cruise control button brings an error code. Our system included a cache of every function of the comparison circuit for the prior 15 seconds before any fault. When you pull up the error readout you can view in sequence both channels of activity like a Flight Data Recorder, and then zero in on the trigger event, the worn cruise control button whose tolerance exceeded the 15ms threshold in its actuation for instance, that gave a single false-positive and disallowed acceleration and averted machine-based fatalities.
All this is aside from mechanically sticky accelerator levers, but according to the Chicago Tribune this morning, eight people who were repaired mechanically for this problem have reported that they've already had unintended acceleration again, and they believe it's computer issues and NOT mechanical. You certainly can imagine people who've already been through this life threatening ordeal paying excruciating attention to the effects of this problem right?!
IMO, Toyota and all other manufacturers must have such a fail-safe on electronic accelerator controls and that's going to mean replacing every out-of-tolerance switch in the world that fails the comparison circuit installations which will dictate acceptable engineered tolerances of all sensors and switches. They might be better off engineering a throttle-body mod and spring loaded cable as a genuine mechanical solution for older models and possibly less expensive.?. These systems simply must default off one way or another.
This can be fixed by moving the accelerator pedal further away from the gas pedal.
The Chicago Tribune this morning reports that seven (I falsely stated eight previously) of the users who already got mechanically 'fixed' for unintended acceleration have experienced this again:
"7 Drivers allege recall did not fix acceleration problems"
Having previously gone through this frightening experience you can assume these people were paying attention to the pedal, yet they believe it's computer caused!
My arguments on this thread point to a fundamental problem with the fail-safe/comparison-circuit design as you articulate... I think there's no other way for unintended acceleration to occur, outside of a raw mechanical failure like a stuck down-pedal or stuck throttle valve, a faulty fail-safe is imbedded.
The fact that by sticking a mechanical limiting device in the peddel appears to help may keep the signal below the FF threshold.
Could be a simple "home sense" issue.
Enough preaching, I guess. Keep reading though....
As microchips get smaller and smaller and the electromagnetic spectrum all around us gets busier and busier, the electronics components, in fact, behave as though they are tiny antennas. Commands get interrupted and software gets corrupted. Hardening is the Answer. Simply cover the chips with very thin gold leaf. The cost per unit part may be less than $25 (of course, the labor will be more).
It is narcissistic to think we can possibly know everything that is going on all the time with these sophisticated components. This is another "Titanic type" case study in which we do not understand the performance of materials in their fielded environment Technology has gotten ahead of the root science here. It may take decades before we fully comprehend what has been going on. We must be willing to know that we do not know everything right now.
Innovative uses of intelligent databases will prevent all future problems like this and, most importantly, save lives. I can show anyone who is interested how to do this. The technology is ten years old. The only thing that is needed is the will to do it. You do not need to understand all of the science now, just be willing to share and communicate experiences directly between engineers and customers through modified CRM programs.
The automotive industry can learn a lot about the dedication to quality found in the aerospace industry.
I look forward to the discussion.
http://www.popularmechanics.com/automotive/how_to/4347704.html
After all, the alternator still charged the battery. This makes it "good" by most historical criteria. However, since it now provides power to sophisticated electronics, the requirements on its output are much more demanding.
If voltage spikes on the power line make it all the way into the silicon, random switching will occur, in which case the software can't be trusted; literally anything can happen. Anything that depends on the software - such as a monitor to determine the state of a switch - may or may not work at all, and may or may not have accurate historical data. This is the essence of "illegal state". This is why, under any circumstances, a hardware device should be available to force a reset of the computer if it fails to respond appropriately in a specified timeframe. The forcing should be hardware, not software or firmware. Cycling power would be the safest way to do it., I think.
In older cars, there was no credible mechanism for unintended acceleration.
Today, such a credible mechanism does exist; there is no mechanical connection between the accelerator and the throttle.
Also, last Sep, there was a 911 call where a passenger very calmly told the operator what the driver was doing to try to make the Lexus stop. The call ended when the Lexus, driven by an off-duty cop, crashed at high speed.
There certainly does appear to be a problem. While it is Toyota on the hot seat right now, it seems unlikely to be strictly a Toyota problem. As someone else mentioned here, the automotive industry does not have to go through the certification procedures that the aircraft industry has to go through, and if we are going to do "drive by wire", the auto industry SHOULD have to pass similar certifications.
Full battery voltage is applied to the rotor coil until the alternator output rises to a predetermined value, let's say 14.3 volts, for instance. Once the alternator output rises to that value the battery voltage to the rotor coil is switched off(***). Due to the now decaying magnetic field in the rotor the alternator output begins to fall. Once the alternator output voltage has fallen to ~12.5 volts the full battery voltage is again applied to the rotor coil.
The above sequence is repeated at a frequency dependent on the overall time constant of the battery/alternator circuit, typically a Khz or more.
*** This is the important point. If the battery source voltage were simply switched off, like happens with the A/C electromagnetic clutch coil, the rotor voltage would almost instantly rise to the point of arcing across the battery voltage switching device, old style mechanical relay or today's solid state "switch".
But within the alternator's voltage regulator, in parallel with the rotor coil, is a fairly high wattage resistor. This resistor acts as a voltage "damper", soaking up, damping, the surge voltage that the rotor coil would otherwise generate.
Should that damping resistor fail the result would be a much more rapid PWM rate along with HUGE voltage spiking on the 12 volt battery circuit.
The frequency of the problem makes analysis difficult. Mechanical failures are usually not as intermittent. Electronic failures, or anomalies require a trigger, something to aggravate the symptom. Insufficient wiring/ shielding/ grounding is an area that would be suspect. As space becomes tighter, more and more connections are packed into tighter spaces. Some of these lines can create magnetic or frequency interference depending on routing. I think some detailed investigation of the engine control/ body harnesses might be well spent. A failure in the pedal assembly could cause an acceleration signal to be continuously triggered, but it does not explain some of the other accounts. There is always a possibility of a problem with multiple causes...
Should the brake light switch fail there is always the backup CC "off" switch.
The Toyota/Denso DBW system already includes dual sensors, both at the gas pedal and at the throttle plate. According to the documentation the ECU, microprocessor, is supposed to be constantly monitoring both sets of sensors. This "monitor" software constantly checks that the two sensors are tracking each other correctly, with the correct range, and the voltages are of separate enough values to discern they are not shorted together.
As Dr. Gilbert demonstrated, the Toyota/Denso "shorted together" monitoring voltage "window" is out of tolerance with reality. The nominal separation voltage is 0.80 volts and in order to detect a short the signals must be within 0.02 volts. 0.40 volts would have been more appropriate IMMHO.
The out of order execution results in "setting" a variable(X) within the firmware code that indicates that the CC is in the "Set/Accel" position. With that variable(X) "set" "true" the car would begin to accelerate indefinitely.
Step on the brakes and the appropriate firmware executes which would normally disable CC but doesn't reach that execution point since it first checks another variable(Y) that indicates CC is not engaged. There is more than one possibility for "Y", CC active, not being set "true". First, the aforementioned external event, or there is the possibility of a code fault, oversight, and the "Y" variable will not be set until the "set/accel" button/function is released. Obviously this latter event is now just NOT going to happen.
Suppose a person happens to "tap" the brakes, left foot braker, just at the very same instant (at the nanosecond level) CC set/accel is depressed. The "brake-on" firmware routine executes first but there is no "X" variable to clear/reset. Immediately thereafter the CC set/accel routine executes.
And now suppose the driver NEVER again releases and re-applies braking.
I understand what you're saying, but you're thinking about monitoring for faults in switches and sensors while assuming that the software is written correctly.
What I'm imagining is something like this (purely theoretical since I have no idea how Toyota's software is actually structured): Suppose there's a code module that handles cruise control inputs, by monitoring and interpreting the state of the cruise switches. Then, separetely, suppose there's a code module that handles controlling the throttle position by sending commands to the throttle assembly. In this scenario, all the sanity checking for the cruise hardware (monitoring switch states and such) would be handled by the cruise code module.
Now, suppose there is a shared space for data passing between these modules. Let's say there are two variables, call them cruise_state and cruise_speed. If the driver sets the vehicle to cruise at 65 MPH, the cruise code module would set cruise_state to 1 and cruise_speed to 65. The throttle code module would periodically check these variables to see whether it needs to move the throttle in response to a cruise command (or cancel) from the driver. Since the cruise inputs have already been sanity-checked by the cruise code, the throttle code trusts that the data in those two variables accurately represents the driver's inputs and does no further checking. Now suppose some other routine overruns a buffer and overwrites cruise_state and cruise_speed with all 1's. The throttle code does its next check of the cruise variables, finds nonzero cruise_state and cruise_speed == 255, and we're off to the races.
This is just a toy example; I'm sure there are a lot more complexities and differences in the real system. I haven't done this kind of embedded control programming so I don't know what the standard techniques are, but I've done enough low-level C stuff to know that all kinds of interesting things can happen that are hard to duplicate because they only happen under rare conditions.
I guess this is one of those problems like the old "an engineer, a physicist and a programmer in a broken-down car" type jokes -- everybody's input is based on the field they're familiar with. The discussion about electronics and interference has been interesting to me because I don't know much about it.
-Andrew L
Most PCs won't work at even moderate extremes of temperature whereas automotive applications have an extremely large temperature range, especially in underhood applications. Comparing PCs to Automotive really isn't a fair comparison.
there is a section of code that allows the throttle to be 'opened' more than the gas pedel sensor value says it should be. There are potientially many causes of this but they all cause the microprocessor to make an overriding decision to send a higher value than the 'throttle request' value would indicate. The real problem here is lack of experienced and failsafe programming methods. This is a boundry value problem and a value that the designers never applied to the software. It occurs infrequently enough that a novice control engineer, i.e. virtually every far east 'programmer' would miss. The code needs to written and any code that causes the throttle control value to mathmatically exceed the throttle request must be removed. It is probably in the IDLE subroutines or in the actual throttle position sensor if they were stupid enough to make that a feedback loop. I encountered this problem before when making a marine engine electonicly controlled. They probably use a carbon film resistor for the sensors. Real bad idea. I use wire wound sensors, no spikes or opens. I am going to the junk yard tommorow to frankenstein a brain, gas pedal and throttle body.
By the by, an alternator spike or em would not effect capasitor secured power to a microprocessor. If that were the case the micros would have gone haywire before they even left the assembly line. Get real people. Get an emulator and a logic analyzer.
Even if the RFI/EMI "spike" entered the control module via an input or output signal or ground wire..?? And what about common mode grounding problems..?? The "ground" voltage at the sensor or servomotor being significantly different, due to a 100 AMP surge current to the 12 volt ABS pumpmotor...?
Until you can reliably replicate the failure an emulator and/or logic analyzer will be of no help.
Intel 4004....You must be about my age, the first I4004 I bought was in ~1972.
What we are seeing is driver error and mass hysteria. Of course the media is piling on, just like they did to Audi 20+ years ago.
And really folks, now that the government and the unions own GM is it suprising to see Toyota executives being beat up in front of Congress? It's called Chicago style politics.
"Testing conducted by VRTC determined that the brake pedal force required to stop a subject vehicle with a wide open throttle was significantly greater than when the vehicle is operating with a closed throttle.
Significant brake pedal force in excess of 150 pounds was required to stop the
vehicle, compared to 30 pounds required when the vehicle is operating normally.
Stopping distances increased from less than 200 feet to more than 1,000 feet."
This is from a NHTSA report that can be found at
http://pressroom.toyota.com/pr/tms/toyota/document/NHTSA_Filing.pdf
They also tested for EMI and found no problems.
"The Vehicle Research and Test Center obtained a Lexus ES350 for testing. The
vehicle was fully instrumented to monitor and acquire data relating to yaw rate,
speed, acceleration, deceleration, brake pedal effort, brake line hydraulic
pressure, brake pad temperature, engine vacuum, brake booster vacuum, throttle
plate position, and accelerator pedal position. Multiple electrical signals were
introduced into the electrical system to test the robustness of the electronics
against single point failures due to electrical interference. The system proved to
have multiple redundancies and showed no vulnerabilities to electrical signal
activities. Magnetic fields were introduced in proximity to the throttle body and
accelerator pedal potentiometers and did result in an increase in engine
revolutions per minute (RPM) of up to approximately 1,000 RPM, similar to a
cold-idle engine RPM level. Mechanical interferences at the throttle body caused
the engine to shut down
But if we intend to drive by wire, we have to bear the cost. There should be a zero percent probability of a car running away with the passengers and being unstoppable.
If that goal is to be achieved, then we need the reviews and certifications.
If that goal is not to be achieved, we shouldn't be doing drive by wire. It is easy enough to say that "the genuine incidents of a car running away are so rare that it doesn't matter", but if it happens to you or your family you might think otherwise.