Concept: Adaptive Safety Margin and Speed Limit

[supercurio]

TLDR

Processing accelerometer to identify rough terrain allows to gradually adjust beeps, alarms and tilt-back to the riding conditions in order to provide consistent safety guarantees.
It will prevent avoidable crashes, currently due to the safety margin being static: calibrated and suitable only for smooth roads.

Inspiration

An idea I had after watching Chooch V12 HT first ride crash, and first referenced in this reply to @Tasku

Quick info and analysis about the crash:

1. Speed reported at 37 mph / 60 mph
   It is (near) the max for V12 HT
2. Tire pressure: 40 PSI
   Goal: protect the rim of this demo wheel sent by eWheels
3. A bump follows a whole, which make the wheel bounce up in the air
4. We can observe the wheel trailing behind the rider instead of staying below the rider
   Cause: The wheel lost forward speed by hitting the bump which converted the energy into the upwards bounce.
5. In the air, the rider moves forward faster than the wheel which lacks traction
6. Eventually, the rider lands ahead of the wheel instead of on top, hence crashing.

My conclusion is that in step 3, the wheel doesn't have enough instantaneous torque in order to maintain the same speed as the rider during the brief moment it touches the ground.
The causes for this crash are

• Major: a riding speed too high for the environment given the wheel speed and torque headroom.
• Minor: high tire pressure which makes the wheel bounce back before the stabilisation algorithm / controller / motor have a chance 

There was no apparent beeps or tiltback triggered.

Problem to solve

Alarms/Beeps and tiltback are the typical feedback mechanism the wheel has to indicate that it is near its maximum capabilities.

With this information from the wheel knowing its status, the rider can modulate speed and reduce it before getting in a situation where the vehicle would not be able to provide the power needed to keep the wheel stabilized, hence introducing an over-lean and inevitable crash.

From Chooch's video we learn that alarm & tiltback thresholds calibrated for safe riding on good roads are inadequate on bumpy off-road trails

Despite that fact might seem obvious to everyone, in practice many riders push their wheel near the limits - wether they realize it or not. So these limits must be set according to the real world conditions.

Solution description

Adaptive safety margin and speed limit leverages information every self-balancing vehicle already has access to: accelerometer data in order to dynamically adjust the beeps/alerts and tilt-back thresholds.

Example

If the wheel identifies that the wheel is rolling on a bumpy off-road terrain, it should increase the safety margins as well as 

Two methods proposed to identify terrain

• A fairly simple algorithm could process the vertical acceleration read by the sensor, using filtering and thresholds to identify how "bumpy" is the ride. The bumpy index would go up quickly and go back down slowly in order to provide a form of smoothing over time.
• A neural net could run on a local micro-controller in real time and do recognition on which type of road surface is currently ridden, like asphalt, gravel, forest single track, skate park, jumping.
  The model should infer more parameters like: confidence and intensity of shock/bumps/vibrations in order to set the safety margin.

Suspension and tire pressure

Elements affecting the amount of shocks and vibrations:

• Higher tire pressure: increases
• Suspension: reduces
• Suspension tuning: increases or reduces

Fortunately, tire pressure or suspension behavior, are taken into account automatically by this approach. It's a useful trait since they affect the safety margin requirements as Chooch describes in the crash video.

Bonus: extra features

Based on machine learning

If we have an AI model inferring riding type, we can adjust the wheel pedals, and throttle response accordingly.

Sensor fusion between accelerometer and phase current variations

The amplitude of oscillations of motor phase current should also tell about how irregular the surface is, since each branch, rock, hole would create a negative or positive current spike in order to maintain flat pedals and self-balancing.

Combining accelerometer data with phase current can give a more rich representation of the ground surface in multiple dimensions, in a way that can give a sense of its texture.

With barometer sensor

By using a barometer sensor (not available in current EUC generations), the wheel could also identify very precisely inclines and adjust its behavior in real-time when climbing, riding downhill.

Example uses:

• Adjust power limits when climbing to avoid overheating
• Adjust pedals when climbing to avoid dipping too much in soft modes
• Adjust pedal behavior descending to avoid dipping too much backwards which makes braking using power pads difficult in softer modes
• Tune self-balancing behavior to maximize grip when climbing or descending on rocky terrains

Mobile phone implementations 

As suggested by @rolis on the EUC-Technical group, phone implementations could also be explored.

Mobile app alternative

Provided the phone would be securely held in a pocket and not moving around erratically in a backpack, it should be possible to infer a few things about the riding conditions from the movement of the device.

In turn, alarms could be emitted by the mobile app to a bluetooth speaker or standalone alerting device both for:

• adaptive safety margin, as offset on top of inverter load (Inmotion) and safety margin (Kingsong) provided over bluetooth data, and possibly estimated PWM for Begode and Leaperkim wheels.
• speed alarms, as offset on top of the real wheel speed limit (Kingsong) or offset on top of estimated max speed limit (based on voltage) for other wheels.

Mobile apps could run with the idea and might implement this concept quicker than OEM in their firmware, although there is the challenge of the lower quality of the data captured.

Prototyping

A mobile phone strapped securely to a wheel case would be useful to:

• build and test a proof of concept
• gather accelerometer data in order to train machine learning models

This would be more like an intermediary step towards a wheel implementation, a convenience, helping to iterate fast initially, and enable crowdsourcing for accelerometer data collection in diverse contexts.

Implementation recommendations 

In order to avoid dangerous bugs, the minimum safety margin (calibrated for smooth asphalt) must always be the minimal safety margin and maximum speed limit the absolute maximum as well.
These parameters must not only be calculated or inferred by a neural net, but always enforced some minimum and maximum values.

In some cases, some expert riders would prefer disable the adaptive margins and limit at their own risk. Typically for race scenarios, or if the implementation has issues which were not fixed yet in a firmware update.

If using neural net inference, riders should be able to record and contribute training data which can be sent to the OEM via a mobile app.

I believe that a low complexity implementation, without machine learning or barometric data would already provide value.

Limitations

• The solution proposed does not help with sporadic potholes on otherwise smooth roads
• Tiltback might not be effective on rough terrains (but alarms still are)

Early feedback

I described this idea with @Cecily Inmotion who shared it with Inmotion Engineers. They answered:

Quote

your advice is very useful. They will try your advice

So maybe we'll have an OEM implementing soon, at least in testing? 
I'm confident that these adaptive limits are required to make this domain of EUC safety more adaptive instead of relying solely on rider experience.

I would like any EUC maker to implement these ideas, so there won't be any patents filed or any attempts to keep control.

 

Now, what do you guys think. Does it makes sense to you too? Would you like to have this feature? Can you think of more improvements or drawbacks?

Edited April 12, 2022 by supercurio
Added mention of sensor fusion with motor current, added Mobile section

[mrelwood]

What a delightfully ambitious suggestion! Not surprised that it came from you though. 

The part that concerns me is the suddenly changing speed limit, and hence a tilt-back threshold that changes rapidly. They need to be careful with the implementation not to make the tilt-back start suddenly in an environment where keeping balance is already challenging.

 Currently the implementation of the tilt-back has seemed to be such that it’s intensity depends on how far beyond the threshold you’re at. An instantly changing threshold would make the tilt-back too violent unless there’s a careful specific condition for the tilt-back that’s stated by the variable limiter.

 How much do you think the speed threshold should decrease when the bad surface detector is at it’s maximum?

[Tasku]

Not sure what to think about. Some can ride without falling and when they detect possibly hard terrain, they slow on their own. (edit. The solution here in terms of cars was that people were trained and granted with a licence to drive car. If they mess stuff up the licence were taken away, easy solutions.)
Do we always need algorythms and software solutions, well generally speaking, automated cars are safer at the moment than those driven by the "average" human. But in general safety improvement is a good thing for sure.
Could over engineer solution cause new found problems? Yes.

I will leave this robot piano video that catches fire to think about. Worth the watch when thinking of "what could go wrong".

edit2. Also the frequency and data would be somewhat related to analyze the data to make your solution work. But the best solution could scan the road infront and  also light it so you get good data. Again would need signal prosessing right? 6:21 in the video. Just the EUC are not equipped with such. 
Maybe you could indeed detect type of road and limit the max speed just by analysing and limiting the next 2-5s of ride performance. Sounds very interesting project.

edit3. I still think this will not stop Chooch from falling. But I love to be wrong and get positive surprise out of these kind of projects!

 

Edited April 12, 2022 by Tasku
edit

[Funky]

Then again, i bet most of us don't ride at max allowed speeds? Right? Right? If wheel goes 40 you don't ride 40.. Ride 35 max. <(Random numbers..)

And most of us would not be riding at max speed, on bumpy roads.. Use the brain, or eat dirt.

EUC's should have 30% or even more safety margin. If you ride 40 (this case it's max speed) euc wont allow go faster. But in reality max speed that euc could do, would be around 60+ I wonder how big "safety" margins we got anyways?

I especially don't ride my 18xl faster than 40kph.. Because i know it's "max" speed is 50ish.. And i don't wanna feel pedal tilt back. Even if i would ride it's max speed 50kph, i wonder how safe it would be? Even on flat straight road.. On dirt roads with lose gravel/bumps i'm riding like a snail ~15-20kph.  And i don't want a kiss from the ground.

Btw amazing post. I'm also all for safety. I bet no one wants to eat dirt.  

To me euc are magic wheel, that rides on dreams/hopes. And if i cross some threshold, those dreams/hopes can pop like a balloon and everything goes to hell. < So better not to cross them. 

[mrelwood]

29 minutes ago, Tasku said:

Not sure what to think about. Some can ride without falling and when they detect possibly hard terrain, they slow on their own. (edit. The solution here in terms of cars was that people were trained and granted with a licence to drive car. If they mess stuff up the licence were taken away, easy solutions.)
Do we always need algorythms and software solutions, well generally speaking, automated cars are safer at the moment than those driven by the "average" human. But in general safety improvement is a good thing for sure.

Look at it this way: Chooch is a very experienced off-road rider, and he has ridden in terrains like that at very high speeds countless times. Despite his experiences, he made an error of over inflating the tire as well as riding as if it were a suspension wheel. If an experienced rider can mess up like that, just think of how badly us mortals can misjudge a situation. I think the idea is definitely sound and well based.

29 minutes ago, Tasku said:

I will leave this robot piano video that catches fire to think about. Worth the watch when thinking of "what could go wrong".

The piano didn’t actually catch fire, and it was just a smoke machine used to make a joke… But I do appreciate the thought. The feature can be made badly.

I would also say that an inhibiting fear of a badly implemented new feature is a sure way to prevent all advancement in tech.

[Tasku]

20 minutes ago, mrelwood said:

The piano didn’t actually catch fire, and it was just a smoke machine used to make a joke… But I do appreciate the thought. The feature can be made badly.

I would also say that an inhibiting fear of a badly implemented new feature is a sure way to prevent all advancement in tech.

Well I think, that just by analysing the signal data we could maybe even give suggestion on tire pressure. Too high or too low. But then again, just by changing out of stock tyre.. and here we go again 

I tend to weight good and bad. Often if you leave thought on the side of danger can make others think that was the final word. Not the case. I think this idea shows potential too.

I also think that analysing the "signal" data and profiling could help improve safety. This could indeed be a good feature and improvement. I just try to say that I am not as far into this as Mr. @supercurio is.

The best part of the video was the signal analysis that I assume would one way or another be the process that determs the profile that you can safely use. Other than that the video is what it is, a sidetrack.

(edit: Bit confused if this would be app of it's own or addon possibility? Would be nice if this solution would work within the one bluetooth connection.. could it work it EUC world, as addon maybe=?)

EUC world has "safety margin" but I assume this data analysis is to one day have somewhat similar function. Maybe you could talk to @Sebawhat his plans are with it? 

 

Edited April 12, 2022 by Tasku
edit

[Tawpie]

How about use your GPS coordinates to look up the vibration data from other riders and cripple or unleash your wheel in advance? It wouldn't know the line you're going to choose but it might have a general idea.

I'm not saying this is a terrible idea, because it has a lot of merit. And I prefer to buy wheels that do look after my safety rather just say "here's a gun, go shoot it". Something like this could be a great addition.

Nevertheless, and it could be that it's because my roots are in hardware and my present is in software (yes, I'm a hardware dweeb, gone soft)... but

                   I. don't. want. any. more. software. in. my. wheels.

Particularly "smart" software, that was trained on an inherently incomplete and a priori not-representative-enough dataset. Software is very tricky stuff—as it gets more complicated and less deterministic, it rapidly becomes impossible to test. I mean, it's not a slam dunk to even get a power-up sequence right—something you didn't expect (and probably couldn't test anyway) happens and a hardware issue cascades into a software issue that exposes another software issue in the power-up sequence that turns into one of the most awe inspiring 4k videos we've seen in a long time.

When you ride a 16" wheel with a rock hard tire on an imperfect surface near its specified performance limit, why are you disappointed when you eat it? (and what does the manual say is max safe speed on a V12HT?) Seems to me that the wetware*** needs to play a larger role than either the hardware or the software... because when the wetware fails and an ill advised decision is the result, there's nothing the smartest AI in the world can do about it. Short of figuring out that the rider is an actual safety hazard and screaming "no forking way I'm going anywhere, the operator is clueless" the next time they (try to) power it up. Biometrics baby, we know how to do that.

And a gentle reminder. We pretty regularly pillory KS for crippling their wheels in software, designate them as weak and slow and undertorqued. How happy will we be when the wheel decides the trail is too bumpy to go faster than 10 kph? Or refuses to accelerate because we just went through a bunch of patches in the asphalt or dropped off a curb... or haven't gone 2 meters yet? You get three guesses. And you already know the first two don't count.

I think I could live with giving me, the operator, a choice of 'terrains' and set fixed operating parameters from there (ack. sounds like FM). Forest road? Single track (green/blue/black—up or down) Freshly paved street? Fairly smooth dirt road with a few oddball dips and bumps? NYC so-called-paved-street? Cobblestones? Rock garden? I'm not sure what the differences in parameters would be...

Even that won't work 100% though. Wetware knows how to lie to the wheel.

Sure, someday the wheel might identify the terrain for me on the fly and do a bang up job—but how much more safety critical software do you really want in your wheel?

 

***wetware: n. completely made up word that I didn't make up (but wish I did). generally refers to the jelloy stuff protected by your helmet

Edited April 13, 2022 by Tawpie

[mrelwood]

 

9 hours ago, Tasku said:

I tend to weight good and bad. Often if you leave thought on the side of danger can make others think that was the final word. Not the case. I think this idea shows potential too.

I also think that analysing the "signal" data and profiling could help improve safety. This could indeed be a good feature and improvement. I just try to say that I am not as far into this as Mr. @supercurio is.

I hear you. 

 

9 hours ago, Tasku said:

(edit: Bit confused if this would be app of it's own or addon possibility? Would be nice if this solution would work within the one bluetooth connection.. could it work it EUC world, as addon maybe=?)

EUC world has "safety margin" but I assume this data analysis is to one day have somewhat similar function. Maybe you could talk to @Sebawhat his plans are with it?

The current firmware on all wheels deny any changes to the speed limits while riding. So a phone app can’t do what is being suggested. It can be a firmware feature only. Luckily Inmotion got interested in this right away!

1 hour ago, Tawpie said:

How happy will we be when the wheel decides the trail is too bumpy to go faster than 10 kph? Or refuses to accelerate because we just went through a bunch of patches in the asphalt or dropped off a curb...

The limits must of course be sensible. I would think that it wouldn’t decrease the top speed to below 20mph under any circumstances. Maybe even higher than that.

 But jumping down a curb at speed is a valid concern. Although… what’s the highest speed anyone would do that?

[Freestyler]

Nice write up Francis!

I doubt they'll be able to implement something as sophisticated as your suggestion, but maybe something good will come out of this!

 

The stm32 micro-controller that Begode (and Inmotion I think) uses has specs in the realm of 72mhz, 64-128kb of flash storage, 20kb ram etc.

Kingsgong uses gd32 micro-controllers which are drop-in replacements for stm32s with more performance afaik and I remember Enaon saying that while the wheel is moving, the CPU is 60% utilized. (not confirmed).

 

I'm in the process of adding dynamic tiltback in Begode wheels myself, but in a more basic form.

I have validated already that I can dynamically adjust the tiltback speed while moving. (from within firmware, because you can not send bluetooth commands while moving as MrElwood mentioned)

I can already change the PWM alarm, so the the only thing missing is tying tiltback speed and PWM together.

I was thinking something along the lines of: When PWM exceeds 70%, set tiltback to the current speed, and clear it when under 70%.

Of course it will need some tuning to ensure it is not jerky.

Edited April 13, 2022 by Freestyler

[Lefteris]

14 hours ago, Freestyler said:

Of course it will need some tuning to ensure it is not jerky

A new era of custom firmware is upon us, as long as it is safe and not exploited by removing all limits by some Neanderthals. :-/ 

You all know the negative results :'(