Crowbar redundancy

[Mimir]

Problem: if a MOSFET fails on your EUC when in use, you faceplant. And MOSFET failures are most commonly under load, which often means high speed, which means a significant safety hazard.

Solution: Stick two control boards in there, so that if one board stops working the other can take over.

New problem: MOSFETs tend to fail short- so the second control board won't be able to drive the motor, it'll just be driving (at least one phase) current into a dead MOSFET.

And this is, as far as I know, basically where things are- more reliable wheels just have higher-specced FETs, more in parallel, better cooling, more conservative limits, etc. (Where you do see dual-board redundancy, it seems mostly meant to protect against things like gyro failures, software bugs, etc, or in some cases (like on the Ninebot One S[12]) two redundant battery packs too, providing some protection against BMS issues and bad cells.)).

 

So, is this the best that can be done? I think there's a better way.

Shown is a simplified driver bridge for one phase of a motor.

In normal operation, board B does almost nothing (it's possible board B could be a separate, smaller board with only the motor driver components, to save cost). However, it communicates with board A over I2C or something, and detects when M1 or M2 fail. Periodically, it tests itself and monitors board A.

If M1 or M2 fails open-circuit, board A deactivates itself and board B uses M3 and M4 to replace M1 and M2, maintaining a stable ride.

If M1 or M2 fail short-circuit, though, board A *hopefully* deactivates itself (it doesn't actually need to do this, which adds a bit more reliability), and board B *also* deactivates itself (turning all FETs off). Then board B turns M3 on if M2 failed, or M4 on if M1 failed, until it detects its current draw fall to zero. By doing so, it shorts the battery pack through (without loss of generality) M1, F1, and M4. Since M1 is acting like a short, this means F1 and M4 together see almost the entire battery pack short circuit current (approx. 1 gazillion amps) which very rapidly blows F1. We have now reduced this case to the open-circuit case- M1 and M2 are totally disconnected from the motor, and the ride can continue. (In practice, this condition occurring would trigger a controlled shutdown of the wheel (beeps, tiltback after a small delay, followed by refusal to power back on until repaired.)

 

 

This solution provides redundancy against an open or short failure of any one MOSFET. If F1 is placed at the end of the motor cables near the motor, and two redundant motor cables are used, separated physically, it also protects against melting motor cables.

 

If two fuses are used in a Y configuration, and if board A performs the same checks on board B as board B does to A, a short-circuit failure from M3 or M4 (while board A operates normally) can also be handled. However, since M3 and M4 pass no current during normal operation (and are likely on a separate, very cool, heatsink), the chance of them failing under normal operation should be minimal.

 

It's important that board B check M3 and M4 during the wheel's POST, to ensure that they're still working (since otherwise a failure of them would be silent, not noticeable, until you need redundancy and don't have it.)

If two redundant battery packs are used, board A and B can coordinate in normal operation to share the load between them (and blow the fuse to isolate a malfunctioning board and its battery in the case of a failure), or, possibly, a separate board could handle balancing load between the two packs and isolating a misbehaving pack.

 

Some problems remain. Notably, the fuse needs to blow, and blow fast; at a huge overcurrent (10x rated capacity) a fuse blows in something like 25ms, which might be a bit of a jolt to the rider but shouldn't cause a faceplant. Also, M3 might fail when the battery's shorted through it- just when it's needed the most- to prevent this, some sort of relay might work, but finding relays with sufficiently high current ratings and low on resistance in reasonably small packages is difficult.

 

Thoughts?

[Keith]

1 hour ago, Mimir said:

F1 and M4 together see almost the entire battery pack short circuit current (approx. 1 gazillion amps) which very rapidly blows F1. 

Very Interesting idea - why doesn’t the above blow M4 - it cannot be too healthy for it can it?

i suspect just preventing a faceplant and gracefully stopping would not be enough for most people. I would expect them to want to at least get home with a suitable “redundancy compromised” alert?

[Mimir]

44 minutes ago, Keith said:

Very Interesting idea - why doesn’t the above blow M4 - it cannot be too healthy for it can it?

i suspect just preventing a faceplant and gracefully stopping would not be enough for most people. I would expect them to want to at least get home with a suitable “redundancy compromised” alert?

Yeah, the stress on the B FETs is why I suggested a full graceful shutdown.

Another option would be a SCR in parallel with the B FETs explicitly for blowing that fuse; that might end up the safest option, but I don't know what the cost impact would be.

[Hunka Hunka Burning Love]

Paging @esaj!!!  Where’s our resident non-expert electronicsologist?  There’s some electronics talk that needs to be analysed, counter-graphed, and responded to in like!  

[esaj]

20 hours ago, Mimir said:

Problem: if a MOSFET fails on your EUC when in use, you faceplant. And MOSFET failures are most commonly under load, which often means high speed, which means a significant safety hazard.

Solution: Stick two control boards in there, so that if one board stops working the other can take over.

New problem: MOSFETs tend to fail short- so the second control board won't be able to drive the motor, it'll just be driving (at least one phase) current into a dead MOSFET.

And this is, as far as I know, basically where things are- more reliable wheels just have higher-specced FETs, more in parallel, better cooling, more conservative limits, etc. (Where you do see dual-board redundancy, it seems mostly meant to protect against things like gyro failures, software bugs, etc, or in some cases (like on the Ninebot One S[12]) two redundant battery packs too, providing some protection against BMS issues and bad cells.)).

 

So, is this the best that can be done? I think there's a better way.

Shown is a simplified driver bridge for one phase of a motor.

In normal operation, board B does almost nothing (it's possible board B could be a separate, smaller board with only the motor driver components, to save cost). However, it communicates with board A over I2C or something, and detects when M1 or M2 fail. Periodically, it tests itself and monitors board A.

If M1 or M2 fails open-circuit, board A deactivates itself and board B uses M3 and M4 to replace M1 and M2, maintaining a stable ride.

If M1 or M2 fail short-circuit, though, board A *hopefully* deactivates itself (it doesn't actually need to do this, which adds a bit more reliability), and board B *also* deactivates itself (turning all FETs off). Then board B turns M3 on if M2 failed, or M4 on if M1 failed, until it detects its current draw fall to zero. By doing so, it shorts the battery pack through (without loss of generality) M1, F1, and M4. Since M1 is acting like a short, this means F1 and M4 together see almost the entire battery pack short circuit current (approx. 1 gazillion amps) which very rapidly blows F1. We have now reduced this case to the open-circuit case- M1 and M2 are totally disconnected from the motor, and the ride can continue. (In practice, this condition occurring would trigger a controlled shutdown of the wheel (beeps, tiltback after a small delay, followed by refusal to power back on until repaired.)

Like @Keith suspected, the "1 gazillion amps" will likely become a problem.  Take something like IRF4110's, used in Gotways,  https://www.infineon.com/dgdl/irfb4110pbf.pdf?fileId=5546d462533600a401535615a9571e0b the pulse duration must be really, really short (you state 25ms later on). @Hunka Hunka Burning Love wanted graphs, so here goes:

The first graph, Figure 8: Maximum Safe Operating Area already shows the crux of the problem: The safe operating area is limited by the pulse width, current and voltage drop over the mosfet. Assuming that the M1 failed at short-circuit would still present the maximum resistance of the datasheet (4.5 milliohms), the fuse resistance would be , say, 20 milliohms and the M4 is at the "typical" internal resistance of 3.7 milliohms, you get a total series resistance (leaving out battery connectors and such) of 28.2 milliohms. Round that up to nice even 30 milliohms, since this is just an example.

Next let's just assume that the current draw is sagging the voltage battery, and make it 75V over the two mosfets and the fuse (again, just for example). The current running through here is 75V / 0.03 ohms = 2500A! Well, the graph already shows that the mosfet cannot take this, the datasheet theoretical maximum pulsed current is 670A (and that's likely hard to reach in real world scenarios). But still, if we assume that the current is much, much lower, how does the voltage over the mosfet affect things? 

Lumping the resistances of the shorted M1 and the fuse, let's say they're 25 milliohms in total, and M4 is 3.7 milliohms, the voltage over M4 would still be:

75V * 0.0037ohm / (0.025 + 0.0037ohm) = 9.67V

Looking at Fig 8, with around 10V Vds, the maximum the mosfet can safely withstand is about 100A for 1 millisecond before it reaches the critical temperature of 175 Celsius in 25 Celsius ambient temperature, and about 40A for 10 milliseconds. For 25 milliseconds (assuming the fuse blows that fast), the value would probably be maybe 10A? If the broken M1 and the fuse have lower resistances, the Vds for M4 will be even higher.

Figure 13 would come into play if M4 could withstand the pulse in general, just to check whether it can handle the heat (the theoretical thermal resistance is actually lower for short duration pulses). Figure 14 is for avalanche ratings, and would come into play if the voltage over the mosfet went (briefly) above the rated maximum.

Based on this, I don't think the idea of blowing the fuse by shorting over it with the mosfets only is a good idea. Don't know about the SCR? Or if more mosfets could be used in-between instead of a fuse to isolate the power feed from M1 & M2.

 

 

[Mimir]

3 hours ago, esaj said:

Like @Keith suspected, the "1 gazillion amps" will likely become a problem.  Take something like IRF4110's, used in Gotways,  https://www.infineon.com/dgdl/irfb4110pbf.pdf?fileId=5546d462533600a401535615a9571e0b the pulse duration must be really, really short (you state 25ms later on). @Hunka Hunka Burning Love wanted graphs, so here goes:

The first graph, Figure 8: Maximum Safe Operating Area already shows the crux of the problem: The safe operating area is limited by the pulse width, current and voltage drop over the mosfet. Assuming that the M1 failed at short-circuit would still present the maximum resistance of the datasheet (4.5 milliohms), the fuse resistance would be , say, 20 milliohms and the M4 is at the "typical" internal resistance of 3.7 milliohms, you get a total series resistance (leaving out battery connectors and such) of 28.2 milliohms. Round that up to nice even 30 milliohms, since this is just an example.

Next let's just assume that the current draw is sagging the voltage battery, and make it 75V over the two mosfets and the fuse (again, just for example). The current running through here is 75V / 0.03 ohms = 2500A! Well, the graph already shows that the mosfet cannot take this, the datasheet theoretical maximum pulsed current is 670A (and that's likely hard to reach in real world scenarios). But still, if we assume that the current is much, much lower, how does the voltage over the mosfet affect things? 

Lumping the resistances of the shorted M1 and the fuse, let's say they're 25 milliohms in total, and M4 is 3.7 milliohms, the voltage over M4 would still be:

75V * 0.0037ohm / (0.025 + 0.0037ohm) = 9.67V

Looking at Fig 8, with around 10V Vds, the maximum the mosfet can safely withstand is about 100A for 1 millisecond before it reaches the critical temperature of 175 Celsius in 25 Celsius ambient temperature, and about 40A for 10 milliseconds. For 25 milliseconds (assuming the fuse blows that fast), the value would probably be maybe 10A? If the broken M1 and the fuse have lower resistances, the Vds for M4 will be even higher.

Figure 13 would come into play if M4 could withstand the pulse in general, just to check whether it can handle the heat (the theoretical thermal resistance is actually lower for short duration pulses). Figure 14 is for avalanche ratings, and would come into play if the voltage over the mosfet went (briefly) above the rated maximum.

Based on this, I don't think the idea of blowing the fuse by shorting over it with the mosfets only is a good idea. Don't know about the SCR? Or if more mosfets could be used in-between instead of a fuse to isolate the power feed from M1 & M2.

 

 

Hm- that makes sense. Using MOSFETs instead of a fuse would ~double power dissipation (I assume the motor driver FETs aren't spending too much time in the linear region, right? (i.e. their power dissipation should be ~Rds(on)*duty cycle*(A^2)- the same as an additional series MOSFET), which complicates thermal problems to the point where it doesn't really make sense.

The key, for me, is that the isolation needs to be as close to zero-cost in normal operation as possible- this rules out *series* MOSFETs. Now, you can have a parallel MOSFET that blows a fuse, since it doesn't actually have to be powered on until needed; you can, similarly, have a relay, SSR, SCR... to blow a fuse, you can switch current directly if you have a sufficiently low-resistance, high-current relay (but remember you need to fit three of those in a wheel!)

It's tricky, but it should at least be doable. Having dedicated fuse-blowing FETs or a SCR seems like perhaps the best option.

[esaj]

58 minutes ago, Mimir said:

Hm- that makes sense. Using MOSFETs instead of a fuse would ~double power dissipation (I assume the motor driver FETs aren't spending too much time in the linear region, right? (i.e. their power dissipation should be ~Rds(on)*duty cycle*(A^2)- the same as an additional series MOSFET), which complicates thermal problems to the point where it doesn't really make sense.

The thing to remember here is that I'm just a hobbyist on electronics, but... I think the linear region does play a role there   (EDIT: further testing says... not that much).

The wheels seem to use something like around 6-8kHz PWM-frequency, I ran a quick simulation with an IRFB4110-mosfet with a pulsed voltage source driving the gate at 10kHz PWM-frequency (0.1ms period), 10ns rise and fall time & 50% duty cycle, 84V with 0.6ohm internal resistance (30 milliohm per cell for 20S), and 1ohm resistor as load:

See the power dissipation spikes at the mosfet... the actual average power dissipated in the mosfet in this case is about 5.8W, somewhere around 10.5W when conducting, and of course near zero when it isn't conducting (LTSpice says around 0.7mW due to leakage current), which isn't that bad, but still needs cooling, and still the spikes during the linear-region transition reach over 1kW.

EDIT: Turning the PWM duty cycle to 95% and the average is about the same as conduction losses, so the high spikes don't matter that much in the long run, at least as long as the PWM-frequency is kept low and the spikes don't go to "insane" values, which alone could damage the mosfets. Going up to higher PWM-frequencies, the switching losses do start to add up.

No matter how low-impedance gate driver you have (here a more or less "perfect" voltage source), the mosfets will always have some time-delay in turn-on and turn-off (the gate needs to "charge & discharge", plus I don't know what sort of other physical effects are at play):

If trying to use a really high frequency to drive the fet, the maximum frequency is limited by the rise- and fall-times (and delays), and the switching losses would become so high that they alone would fry it in the long run. Guess that's why they keep it so "low".

But, if using a mosfet as a simple "off-switch", ie. only turned on/off to enable/cut the power, this transition is "one time", so it bears even less meaning. If the fuse is replaced with a mosfet (or two in series, one anti-parallel to prevent the body diodes from conducting), the linear region is of no concern anyway (but cost-wise, the mosfet itself and additional circuitry for gate drive might be).

 

Quote

The key, for me, is that the isolation needs to be as close to zero-cost in normal operation as possible- this rules out *series* MOSFETs. Now, you can have a parallel MOSFET that blows a fuse, since it doesn't actually have to be powered on until needed; you can, similarly, have a relay, SSR, SCR... to blow a fuse, you can switch current directly if you have a sufficiently low-resistance, high-current relay (but remember you need to fit three of those in a wheel!)

Depends... other characteristics of the "safety" mosfet shouldn't play that much role (to my knowledge), just have the V_DS(max) high enough and internal resistance as low as possible, so that the fet itself doesn't overheat. As the motor coils and half-bridge mosfets are already taking the "brunt" of the dissipation with their normal R_DS(on) + switching losses in transition, that shouldn't need to be like super-low (1milliohm or under) still? Could be wrong though, I didn't make any calculations, but if the internal resistance is in single milliohms, likely not that big of a problem?

 

Quote

It's tricky, but it should at least be doable. Having dedicated fuse-blowing FETs or a SCR seems like perhaps the best option.

I'm not a huge fan of having fuses in the motor-phases, but yeah, not saying it couldn't work. Mostly I'd be worried about them blowing under "normal" circumstances, like taking off or fast acceleration up-hill, which can momentarily cause very high currents.

[Mimir]

17 hours ago, esaj said:

Depends... other characteristics of the "safety" mosfet shouldn't play that much role (to my knowledge), just have the V_DS(max) high enough and internal resistance as low as possible, so that the fet itself doesn't overheat. As the motor coils and half-bridge mosfets are already taking the "brunt" of the dissipation with their normal R_DS(on) + switching losses in transition, that shouldn't need to be like super-low (1milliohm or under) still? Could be wrong though, I didn't make any calculations, but if the internal resistance is in single milliohms, likely not that big of a problem?

 

I'm not a huge fan of having fuses in the motor-phases, but yeah, not saying it couldn't work. Mostly I'd be worried about them blowing under "normal" circumstances, like taking off or fast acceleration up-hill, which can momentarily cause very high currents.

I think you might be right, here, then- adding an extra FET (well, you'd need two, actually) in there would work for isolation. You'd end up needing an extra 6 MOSFETs (plus another ~12 or so for the redundant ones!), but ... sacrificing room in the shell for another few battery cells seems worth it, and the transistors themselves are a lot cheaper than hospital bills.