Jump to content

!!! Firmware or hardware problems with KS18L, wheels locking up while trolleying


esaj

Recommended Posts

4 hours ago, Mimir said:

Wheel didn't lock up, it just powered the motor off entirely- so, yeah- and the LED ring displayed two non-consecutive red lights (not a part of the normal pattern)

Yeah. Okay. Thats what is supposed to happen. That is a lockup fail safe mechanism that KS uses to circumvent the catastrophic outcomes of lockup. It shuts down the wheel if it senses an impending lockup. ;)👍

Link to comment
Share on other sites

On 1/25/2019 at 9:51 PM, Mimir said:

Funny, I had the same thing happen to me Wednesday. Firmware 1.11 on my XL. While trolleying through a store.

Concerningly, I had noticed earlier that the store seemed to have a lot of static- got a few nasty shocks off shelves beforehand, but didn't think about the wheel...

 

Works fine for now, but if I suddenly stop posting and there's a news story about an EUC fire in LA, you know what happened.

The lock-up problem still not solved then and the fitting of anti-static foil in all new XLs and/or upgrading old Ls with it is basically pointless... (and curious at the same time)

BTW @Seba - you were right about it from the beginning...

  • Like 1
Link to comment
Share on other sites

12 hours ago, hal2000 said:

The lock-up problem still not solved then and the fitting of anti-static foil in all new XLs and/or upgrading old Ls with it is basically pointless... (and curious at the same time)

BTW @Seba - you were right about it from the beginning...

I don't like to be right in such situations :( But I'm happy that King Song FW developers, working on 1.11 version, followed the way I described in the following post:

 

At least you don't have to disassemble your wheel on every lock up.

  • Like 1
  • Upvote 1
Link to comment
Share on other sites

I don't see how @Seba can state he is right about this being a HW issue that can lock up while riding. So far I have not heard of any lock ups while riding, only trollying. And, the longer time goes on, the less likely it appears to be a HW issue. I have over 1300 miles on mine after the first 50 where I had 2 trolly lock ups, and no controller replacement.

 

The FW seems to have fixed it for me, though I am still (very slightly)  cautious. Seems to be a good wheel...so far. 

  • Like 1
Link to comment
Share on other sites

1 hour ago, Circuitmage said:

I don't see how @Seba can state he is right about this being a HW issue that can lock up while riding. So far I have not heard of any lock ups while riding, only trollying. And, the longer time goes on, the less likely it appears to be a HW issue. I have over 1300 miles on mine after the first 50 where I had 2 trolly lock ups, and no controller replacement.

As you may have noticed, both @Rama Douglas and @Mimir reported that their KS-18XL got locked up. Two XL, so two new wheels. Apparently the issue may be still present in new wheels. And no, 1.11 firmware didn't fixed anything. It just included some workaround I've been suggesting to the King Song engineers some time ago. Workaround that resets the complete hardware in case of lock up. If it would be problem with firmware, no hardware reset would be ever needed. And if it would be a firmware problem, every wheel would be affected. But it's not the case. For example mine KS-18L, from first batch - 5000 km and no single lock.

There can be many causes of lockup, but symptoms points me to hardware problem with clock generator. It's not a sofware bug if STM32F103 suddenly stops clocking its core and peripherials. And we should thank God that no lockup occured during normal ride. Maybe board temperature plays a role, so when wheel is under load and MOSFETs are heating the controller, problem doesn't appear. But I will still claim that it is hardware problem - with assembly, design or material, don't know.

But it's just my personal opinion, backed by several years of STM32 hardware & software develompent expertise.

  • Like 1
Link to comment
Share on other sites

I have one of the first batches of 18L in the US.   I don't do much tolleying (a minute or two in office or from car to house).... I decided to try out a longer trolley around my house.

I got the crazy death shakes after just a couple minutes, I'm on Firmware 1.11.

I was able to reproduce it pretty easily to get a video:

 

  • Like 3
Link to comment
Share on other sites

7 minutes ago, that0n3guy said:

I have one of the first batches of 18L in the US.   I don't do much tolleying (a minute or two in office or from car to house).... I decided to try out a longer trolley around my house.

I got the crazy death shakes after just a couple minutes, I'm on Firmware 1.11.

I was able to reproduce it pretty easily to get a video:

 

WOW! I didn't know King Song makes jack hammers.  Cool! 

Not really:mellow:

Link to comment
Share on other sites

1 hour ago, that0n3guy said:

I was able to reproduce it pretty easily to get a video:

 

So this means you can trigger this on demand?

If that is the case then this should be of interest to Kingsong. As a trigger on demand is first step in finding the cause and then a remedy.

Link to comment
Share on other sites

@Seba Ok, up front I'm not trying to argue with you, these are some of my detailed thoughts on this (being a still concerned rider)...

If it is a HW clock issue affecting the STM32F103 , then wouldn't that also mean it could not run the FW code fix? It's only a few seconds between the lock up and the reboot now (which I have not seen mine do since the FW fix). If the crystal is dead, then it could not run again. If it's an intermittent clocking issue, then it SHOULD crop up while riding (at approximately the same rate it initially did before the FW fix).
 I have designed and debugged embedded code one a few things myself. I don't know all the technical details on this unit...but it sounds like you are familiar with the clocking requirements. From what I can see on these wheels (Both my KS16s and 18L) and my experience with control systems,  their PID routine at stop and slow speeds appears very jumpy and possibly prone to error. Both of my new KS's had catastrophic (I mean catastrophic as in not able to ride) issues at slow speeds. This makes perfect sense to me as you want a wheel that is very responsive and reactive, however this can also cause control problems at slow speeds. Both riding and PID computing the wheel at fast speeds is easier, compared to stopped or slow speeds.

In other words, when designing a robot that moves, getting it to go "the right way" initially is more difficult than keeping it on it's track or modifying that momentum(speeding up or slowing down). At stop, you have forward and reverse to deal with...which I am assuming is causing the FW error in the PID controller when trollying. Please , correct me if I am wrong.

And (now for some maths). I was going to try and estimate the frequency of failures over a number of units, but 1) I don't have the failure rate information and 2) I want to take this from an isolated HW issue from your POV. If MY unit even failed 10% of the time while riding compared to trollying, then I would have  had 5 lock ups while riding (10% of 1300 / ever 25 miles) so far this year.

And, again, the last example stated above is still a trolly lock up, not a riding lock up.

 

With all of that said, I still see possible HW failure. My 16S was fixed with a controller replacement, and there are probably lots of opportunities for failure on these devices.

Edited by Circuitmage
  • Like 1
Link to comment
Share on other sites

How much motor control is done by the peripheral in the chip, and how much is in firmware? From what I've seen, that family just has some PWM controllers with useful tweaks.

 

In other words- suppose the firmware hangs unexpectedly. Do the last-programmed PWM values stay there? If so, this would explain the original lockup issue- each phase of the motor basically becomes an inductor, connected across the battery (but being PWMed still.)

 

If that's the case, it's possible that a pretty big range of root causes are responsible. ESD crashing the CPU. A bug in the code handling the lift sensor, causing an infinite loop- suppose this happens in a timer ISR (the firmware has to poll the lift sensor, since it's not a discrete trigger) when timer interrupts are masked.

The firmware fix may, then, have just been enabling a watchdog timer. If the CPU hangs, it gets a surprise reset a bit later, and the initialization code resets the motor phase PWM outputs to full off. This would explain the symptoms *I* observed- the wheel moved freely when turned by hand, so the motor wasn't being driven. The weird lights? Just whatever random values the CPU happened to send out on what I assume is a clocked shift register, SPI-style, for the ring lights.

 

The mystery here is why it's only ever when trolleying...

 

  • Like 2
Link to comment
Share on other sites

4 hours ago, Unventor said:

So this means you can trigger this on demand?

If that is the case then this should be of interest to Kingsong. As a trigger on demand is first step in finding the cause and then a remedy.

I just tried to trigger it again, nothing, but last time it was warmed up.  I can try again another day.   It's kinda nerve racking thinking it could lockup while riding very slow.

Link to comment
Share on other sites

8 hours ago, Circuitmage said:

@Seba Ok, up front I'm not trying to argue with you, these are some of my detailed thoughts on this (being a still concerned rider)...

The problem is that I'm also very concerned, because brand new KS-18XL waits in my garage for first ride... :)

8 hours ago, Circuitmage said:

If it is a HW clock issue affecting the STM32F103 , then wouldn't that also mean it could not run the FW code fix? It's only a few seconds between the lock up and the reboot now (which I have not seen mine do since the FW fix). If the crystal is dead, then it could not run again. If it's an intermittent clocking issue, then it SHOULD crop up while riding (at approximately the same rate it initially did before the FW fix).

There are more than one oscillator on board of STM32 microcontroller. Every time you reset the hardware, it boots first from internal RC oscillator. Now the formware can setup other clock source and usually do, configuring crystal oscillator and PLL. When PLL will lock (it's used to obtain arbitrary, higher clock frequency from fixed frequency crystal) software will switch from RC slow clock to fast PLL clock. 

STM32 is equipped with two safety measures - watchdog and clock security system (CSS). Both are independent from system clock an may run in absence of main clock. Watchdog can reset the complete system in case of software halt, CSS can automatically switch to internal oscillator allowing firmware to be executed. It can also disable PWM generation for the time of reconfiguration needed.

5 hours ago, Mimir said:

How much motor control is done by the peripheral in the chip, and how much is in firmware?

STM32F103 is equipped with special timers that can be used to generate PWM. It would be unwise not to use this hardware :)
 

5 hours ago, Mimir said:

The firmware fix may, then, have just been enabling a watchdog timer. If the CPU hangs, it gets a surprise reset a bit later, and the initialization code resets the motor phase PWM outputs to full off. This would explain the symptoms *I* observed- the wheel moved freely when turned by hand, so the motor wasn't being driven. The weird lights? Just whatever random values the CPU happened to send out on what I assume is a clocked shift register, SPI-style, for the ring lights.

It's exactly as you write. Seems that KS engineers followed easier, "watchdog way", but it's OK. For a short time of lock up PWM controller outputs still drive the MOSFETs, but there is current limiting in the controller board. Side light strips are using very popular, addresable WS2812 LED chips. There is only one serial data wire, no clock. So in case of any glitches in control they may show weird lights, but this may occur also in perfectly working wheel.

  • Like 1
Link to comment
Share on other sites

Good comments on the clocking. That all sounds familiar. :)

Still, I have over 1300 miles on my 18L with no lock ups and no controller changes since the two initial lock ups...pretty happy so far. And, if you factor in ride time versus trolly time...it's looking really good. I usually ride for about 15 mins a day, and trolly for about 1 minute. On weekends I can ride for an hour with less trolly time. So if it is a timing glitch...still seems like it is only while trollying. Just put your armor on and watch it the first 50-100 miles. :)

 

  • Like 2
Link to comment
Share on other sites

1 hour ago, Thai-lad said:

The fact that Kingsong still haven't a clue as to what's actually causing the problem rules out purchasing this wheel for me.  Unfortunate, as it seems a well designed product in all other respects.

The problem doesn't appear during normal ride and is rather incidental. Having workaround in 1.11 firmware, KS engineers stopped to dig into the problem deeper, satisfied that user complains stopped. But you're right. This should be fixed.

Edited by Seba
Link to comment
Share on other sites

4 hours ago, Seba said:

The problem doesn't appear during normal ride and is rather incidental. Having workaround in 1.11 firmware, KS engineers stopped to dig into the problem deeper, satisfied that user complains stopped. But you're right. This should be fixed.

The problem *hasn't yet* appeared during a normal ride, *that we know of*.

If the wings kept falling off planes, and nobody knew why, but so far it's only happened on the ground... not really that reassuring.

If KS knows the cause, they know whether it could occur while riding. And they need to tell us. If not...

 

Suppose it's thermal- issue won't happen when the wheel's hot. You grab your wheel on a cold day, and start your ride. Before the board heats up enough... oops! watchdog reset and faceplant.

  • Like 2
Link to comment
Share on other sites

Right, so...guess what.....at 6am today I get ready to leave, lean my 18L up against a wall after turning it on...and when I go back to grab it , it starts shaking so violently that it moves out of my hand and falls, with the wheel spinning and jerking in both directions. I was able to get it back up and hit the power button and it did shut off, but yeah...still something going on with these.

I blame this thread for bringing it back up...my wheel heard you.

Again, this is a trolly mode/ at still failure. It seemed to be just when I started to move it again after it was perfectly stopped leaning against the wall. This still points to a PID routine FW issue to me, but I will be wearing my armor more as I ride in the future.

This morning I didn't even want to deal with this and just left without riding. :(

 

  • Like 1
Link to comment
Share on other sites

Guys: I emailed Jason (e-wheels) about it. He had me redo the calibration and didn't seem too worried. Sounds like it's static electricity based issue which only occurs during trolleying. So, now, I simply turn off my wheel and trolley it with no juice. Everything else is performing lovely. Stop worrying guys. Just gear up and hope for the best, as we always used to before knowing of the issue at hand. Kingsong knows about it and will hopefully come up with a solution. Until then, I recommend trolleying with it off and putting the worries away and enjoy this awesome machine! Law of Attraction?!? 😎

Edited by Rama Douglas
  • Like 1
Link to comment
Share on other sites

12 hours ago, Circuitmage said:

Right, so...guess what.....at 6am today I get ready to leave, lean my 18L up against a wall after turning it on...and when I go back to grab it , it starts shaking so violently that it moves out of my hand and falls, with the wheel spinning and jerking in both directions. I was able to get it back up and hit the power button and it did shut off, but yeah...still something going on with these.

I blame this thread for bringing it back up...my wheel heard you.

Again, this is a trolly mode/ at still failure. It seemed to be just when I started to move it again after it was perfectly stopped leaning against the wall. This still points to a PID routine FW issue to me, but I will be wearing my armor more as I ride in the future.

This morning I didn't even want to deal with this and just left without riding. :(

 

Did you download the latest firmware? It contains a fail-safe shutdown to prevent this problem. Essentially, the wheel detects the impending doom and plays opossum until the danger passes. You then hit the power button and wheel turns back on and all is fine. 

I have heard all the explanations about static electricity being the cause. Perhaps, but I not convinced that the issue is unrelated to the lift sensor. 

The wheel lock-up has never happened to me. The failsafe emergency shut down during trolleying has happened to my son when he was trolleying the 18L through a building on UMD's campus.  

Several months ago, we had finally decided to give the lift sensor a try. It shut down while traveling the following day. I turned off the lift sensor and several months later the 18L has not shut down again; with no apparent change in usage or trolley frequency🤔🤔🤔

After six weeks of riding, shut down has not happened to my 18XL. I don't expect that it will because the lift sensor is off. 

I have been riding 18L/XL since their release and I have had zero lockups,  and one shut down. I believe that it is because I keep the lift sensor off. 

  • Like 3
Link to comment
Share on other sites

13 hours ago, Circuitmage said:

Right, so...guess what.....at 6am today I get ready to leave, lean my 18L up against a wall after turning it on...and when I go back to grab it , it starts shaking so violently that it moves out of my hand and falls, with the wheel spinning and jerking in both directions. I was able to get it back up and hit the power button and it did shut off, but yeah...still something going on with these.

I blame this thread for bringing it back up...my wheel heard you.

Again, this is a trolly mode/ at still failure. It seemed to be just when I started to move it again after it was perfectly stopped leaning against the wall. This still points to a PID routine FW issue to me, but I will be wearing my armor more as I ride in the future.

This morning I didn't even want to deal with this and just left without riding. :(

 

So was your trolley handle extended?  Have you had the anti-static tape installed?  What firmware version are you running?  Is your lift sensor disabled?

Edited by Thai-lad
Link to comment
Share on other sites

1 hour ago, Lutalo said:

Not sure why. There is no case history of any lock-up issues when riding at any speed. 

If his trolley handle wasn't extended and his lift sensor either disabled (or not engaged when he moved the wheel from it's resting position) then what Circuitmage experienced could just as easily have happened with him standing on it.

Link to comment
Share on other sites

3 hours ago, Thai-lad said:

If his trolley handle wasn't extended and his lift sensor either disabled (or not engaged when he moved the wheel from it's resting position) then what Circuitmage experienced could just as easily have happened with him standing on it.

Again. No case history. Not one instance wheel lockup while riding. Tranquilo 

  • Like 2
  • Upvote 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...