Gotway/Kingsong protocol reverse-engineering

[Jason McNeil]

Over the years had many [failed] experiments with different sleep reduction regimens, before concluding it's not for me; quite certain I don't have the BHLHE41 gene... Judging by your prolific output on the forums, you're already an Uberman! 10Hz is pretty good, probably enough for trends but there are situations, like over-acceleration crash investigation, where higher frequency might be desired. Is it a case of whatever the module on the board is providing? 

[esaj]

10Hz is pretty good, probably enough for trends but there are situations, like over-acceleration crash investigation, where higher frequency might be desired. Is it a case of whatever the module on the board is providing?

Unfortunately, the messages are received every 200ms, so it's only 5Hz, and yes, the rate is decided by the software on the mainboard, at least I think it cannot be changed.

Took a bit longer test ride a little while back, now that it wasn't completely dark     Also found out that the UI-vibration alerts don't get updated to the warning-system, so looks like I have at least one bug to fix before release   Other than that, the app played nicely... I stopped to use the iPhone inclinometer to measure some hills, and turned off the GW & app on those occasions, and it connected back on first try! Not sure if I was just having bad luck with the connections last night, but also might be because I used release-build this time instead of debug.

A couple of graphs from the recordings I took:

Strong braking (power-braking, well, as good as I can power brake on the Gotway, haven't really practiced much):

This is on slight downhill slope (a few degrees, this one I actually didn't measure), I first accelerate to around 20km/h, then do a power braking (twice in a row). Regenerative power peaks hit over 3kW.

 

Braking down a gravel hill in the woods (I was already moving when I started recording), slight bump upwards, then starts steep going to 25 degrees, then evens out to 10 degrees at end and coming to stop at level:

 

Going up the same hill, start from level (the first spike is just accelerating on getting going), starts at around 10 degrees, then steepens up to 25 degrees and then levels out after the "bump":

 

 

[Jason McNeil]

Very cool, do you think there's some micro-averaging going on in the controller? The stats from eLogger are much more chaotic than from the App.

That FW controller can take loads of power, do you know what the rated MAX is? Graph 2 is pretty revealing, nearly 8 secs of 20-25A charge current (/4 = 5A per parallel) (can you imagine a situation like a 300m continuous descent!) that really can't be doing the pack any good,   

[esaj]

Very cool, do you think there's some micro-averaging going on in the controller? The stats from eLogger are much more chaotic than from the App.

That FW controller can take loads of power, do you know what the rated MAX is? Graph 2 is pretty revealing, nearly 8 secs of 20-25A charge current (/4 = 5A per parallel) (can you imagine a situation like a 300m continuous descent!) that really can't be doing the pack any good,   

Could be averaging, also the current measuring could maybe be off by some factor... probably the mosfets are the limiting factor in the controller, if someone knows the type, the datasheets could be checked for max current.

[esaj]

Well that was a couple of hours wasted over stupid issue with the vibration-alert settings not updating when values are changed in UI after connecting to the wheel... ("The OnSharedPreferenceChangeListener gets garbage collected in your case if you use an anonymous class.")  On the side, I also fixed a couple of other issues (one was somewhat severe, the UI updating thread stopped with dead PipedInput/Outputstream after device got disconnected and was automatically reconnected, others were more minor). I haven't really tested this (except with disconnect/reconnecting/lift-tests/changing vibration values on-the-fly and turning warnings on/off while they're "playing"), do tell if you have any issues:

Download here

It's a basic Android apk, should be no developer-mode needed or anything, but your phone must be allowed to install software outside Google Play/whatever they're usually got from. If you've installed the Gotway-app from outside Google Play, you should be good to go, otherwise: 

From your smartphone or tablet running Android 4.0 or higher, go to Settings, scroll down to Security, and select Unknown sources. Selecting this option will allow you to install apps outside of the Google Play store. Depending on your device, you can also choose to be warned before installing harmful apps. This can be enabled by selecting the Verify apps option in the Security settings.
On devices running an earlier version of Android, go to Settings, open the Applications option, select Unknown sources, and click OK on the popup alert.

Not sure on how old Android it will work, tested with 4.2.2 and 4.4.2. Not sure if it will work in Lollipop (Android 5.x)

Edited September 13, 2015 by esaj

[Tilmann]

Hi @esaj,

from all I can tell from stationary testing, your app works like a charm! Tried it on my Oppo Find 7 with Lollipop. BT always connects on the first try (which never happens with the GW or KS apps). Cosmetics: right menu buttons are slightly cut off (resolution 2560 x 1440); terminating the app is followed by a pop-up message "Unfortunately, Wheelmetrics Proto has stopped" - I guess, the phone is really upset to shut down such a fine piece of software .

EDIT: phone vibrates at the set alarm levels, but the Pebble watch does not.

BTW: Went on the "Skate by Night" tour today - 22km through Berlin. Six Ninebot Ones were participating: an older "E"-model could not keep up due to low batt and sent its owner on the tube, 4 NB1s arrived more or less on their last electrons, and one arrived with 50% charge left: that was a "C+"-model (weaker motor: 450 Watt) with the 1RadWerkstatt 388Wh mod.

Edited September 13, 2015 by Tilmann
Added Pebble detail

[esaj]

BT always connects on the first try (which never happens with the GW or KS apps).

That's good, I was having some issues at times with the connections, but mostly they seem to work ok. I suspect disconnecting & reconnecting "fast" may have something to do with it, the chip on the wheel seems to only accept one connection at a time, and after disconnecting & reconnecting might refuse a new one if it thinks the old one is still alive. But that's just a guess.

Cosmetics: right menu buttons are slightly cut off (resolution 2560 x 1440);

It seems to happen on all resolutions (both on my tablet and vee's 4" phone)... guess the graph-element could be too wide and pushes them outside the screen. The UI-editor in Android Studio is pretty useless as the UI is built from "fragments" (kind of like larger pieces of UI you can throw around, nothing to do with OpenGL fragment-shaders) and it can't render them together correctly anyway   Will have to see what I can do about it.

terminating the app is followed by a pop-up message "Unfortunately, Wheelmetrics Proto has stopped" - I guess, the phone is really upset to shut down such a fine piece of software .

Had that problem earlier, but thought I fixed it... it has probably something to do with the order the background services are put down (probably one of them crashes during shutdown if some other part it needs goes down first)   Now that you mention it, I'm not sure if I ever shut the app down while it was connected to the wheel, is this when it occurred? Or maybe I did, I think at least usually I shut down the wheel first... Can you check are there any services left alive? If not, it's not that critical, but of course annoying.

EDIT: phone vibrates at the set alarm levels, but the Pebble watch does not.

No idea how the Pebble <-> Android -connection works, I'm just telling the OS that I want to vibrate with some pattern (or stop vibrating).

BTW: Went on the "Skate by Night" tour today - 22km through Berlin. Six Ninebot Ones were participating: an older "E"-model could not keep up due to low batt and sent its owner on the tube, 4 NB1s arrived more or less on their last electrons, and one arrived with 50% charge left: that was a "C+"-model (weaker motor: 450 Watt) with the 1RadWerkstatt 388Wh mod.

Nice, not only are the cells larger, but they likely won't drop as fast because they probably have higher discharge ratings also vs. the originals  

Edited September 13, 2015 by esaj

[Tilmann]

 

Had that problem earlier, but thought I fixed it... it has probably something to do with the order the background services are put down (probably one of them crashes during shutdown if some other part it needs goes down first)   Now that you mention it, I'm not sure if I ever shut the app down while it was connected to the wheel, is this when it occurred? Or maybe I did, I think at least usually I shut down the wheel first... Can you check are there any services left alive? If not, it's not that critical, but of course annoying.

When I look in "Application Management" after terminating the app, it's still listed with a consumption of 3.00MB. Would love to give you more info, but I have no idea how... 

[esaj]

When I look in "Application Management" after terminating the app, it's still listed with a consumption of 3.00MB. Would love to give you more info, but I have no idea how... 

At least on my 4.4.2 -tablet, the app is listed under Settings -> Apps -> Running with text underneath saying "X process and Y services". Tapping on that, it lists the process itself and its memory consumption + the services and their names with buttons to stop them. If you see them, please tell me what is still running, that might give some hint what to look for.

EDIT: now that I tested it again on my tablet, I do not get the error message closing it, but it doesn't close properly (couple of services are left behind), although they don't do anything or really consume memory, I'll get on fixing that when I got the time. Probably broke the shutdown when I was changing things trying to get the vibration-settings to work earlier  

Edited September 13, 2015 by esaj

[Tilmann]

At least on my 4.2.2 -tablet, the app is listed under Settings -> Apps -> Running with text underneath saying "X process and Y services". Tapping on that, it lists the process itself and its memory consumption + the services and their names with buttons to stop them. If you see them, please tell me what is still running, that might give some hint what to look for.

Sorry, no such luck. I just can't find any info regarding running processes and services. And going through the "Developer Options" I could not identify any switch, that might deliver such additional detail.

I also installed the app on a Lenovo YOGA Tablet Pro 2 (Model 1380L) with android 4.4.2 (same 2560 x 1440 resolution): 

As you can see, it cuts the buttons on the right, too, but here without using the entire screen real estate.

Problematic here: the silly tablet does not have an android menu button. So I can't invoke the BT connection dialogue.

Terminating the app here does not produce the "Unfortunately, ..." message. 

[esaj]

Sorry, no such luck. I just can't find any info regarding running processes and services. And going through the "Developer Options" I could not identify any switch, that might deliver such additional detail.

I also installed the app on a Lenovo YOGA Tablet Pro 2 (Model 1380L) with android 4.4.2 (same 2560 x 1440 resolution): 

As you can see, it cuts the buttons on the right, too, but here without using the entire screen real estate.

Problematic here: the silly tablet does not have an android menu button. So I can't invoke the BT connection dialogue.

Terminating the app here does not produce the "Unfortunately, ..." message. 

I still think the graph-element takes up too much space and pushes the buttons off-screen... they then get cut at the margins. Have to see what I can do about that.

The missing menu-button is weird, it looks otherwise similar like on my Lenovo A-7600F, except on mine the menu-button appears on the lower right corner:

I still don't know that much about Android or it's quirks, guess I should get a book on the subject. Hate reading long documentations from screen, plus usually the books explain the basics and more common things better than the original documentation, for more specific problems StackOverflow and similar are usually the best sources...

 

 

Edited September 14, 2015 by esaj

[esaj]

Sorry, no such luck. I just can't find any info regarding running processes and services. And going through the "Developer Options" I could not identify any switch, that might deliver such additional detail.

I also installed the app on a Lenovo YOGA Tablet Pro 2 (Model 1380L) with android 4.4.2 (same 2560 x 1440 resolution): 

As you can see, it cuts the buttons on the right, too, but here without using the entire screen real estate.

Problematic here: the silly tablet does not have an android menu button. So I can't invoke the BT connection dialogue.

Terminating the app here does not produce the "Unfortunately, ..." message. 

The app should now close properly,show the buttons a bit more properly (though they still cut out a bit for some reason ) and I replaced the menu entirely with a separate button. Download (it's the same link as before though) & instructions here:

 

[esaj]

The first version of the app did not work with King Song, but I got a data capture from a stationary King Song, and investigated the reason.

Here's a part of the data:

00 18 5A 5A 5A 5A 55 AA 18 4E 00 00 00 00 00 00 00 10 F9 66 00 01 FF F8 00 18 5A 5A 5A 5A 55 AA 18 4B 00 00 00 00 00 00 00 10 F9 66 00 01 FF F8 00 18 5A 5A 5A 5A 55 AA 18 4F 00 00 00 00 00 00 00 0F F9 69 00 01 FF F8

So, it looks quite similar to the Gotway-data (and of course needs to be to work with the Gotway-app). But, there are subtle differences:

 Gotway:
 04 18 5A 5A 5A 5A 55 AA 19 A7 FF FF 00 00 00 01 FF E0 F8 BD 00 01 FF F8 00 18 5A 5A 5A 5A 55 AA 00 09 1A 9D 00 00 00 00 00 00 00 00 00 00 00 00 
|  <---   header  --->  |Volt |Spd  |   Trip    |Crnt |Temp |    Unknown   | <---  header  ---> | Odometer  |  Unknown, padding? Always zeroes  |

King Song:
 00 18 5A 5A 5A 5A 55 AA 18 4B 00 00 00 00 00 00 00 0E F9 C0 00 01 FF F8 00 18 5A 5A 5A 5A 55 AA 18 4B 00 00 00 00 00 00 00 0F F9 C4 00 01 FF F8

If you look at what I have marked as the header on Gotway, the first byte is different in King Song. But it doesn't stop at that. Also after the "second header", the data is different... it's actually a repeat of the first part on King Song (unless the wheel where the data is from has been ridden for 407764.992 kilometers ). So actually, King Song doesn't send odometer data, it just repeats the voltage / speed / trip / current / temp -data:

King Song:
 00 18 5A 5A 5A 5A 55 AA 18 4B 00 00 00 00 00 00 00 0E F9 C0 00 01 FF F8 
 00 18 5A 5A 5A 5A 55 AA 18 4B 00 00 00 00 00 00 00 0F F9 C4 00 01 FF F8
 00 18 5A 5A 5A 5A 55 AA 18 4B 00 00 00 00 00 00 00 0F F9 C5 00 01 FF F8 
 00 18 5A 5A 5A 5A 55 AA 18 4A 00 00 00 00 00 00 00 0F F9 C7 00 01 FF F8
|  <---   header  --->  |Volt |Spd  |   Trip    |Crnt |Temp |    Unknown |

And to be more precise, the Gotway-app uses only the 18 5A 5A 5A 5A 55 -part to distinguish the start/end of data, and that's why the King Song protocol works with that also (I think Gotway-app might also try to read every second data as odometer, and show weird figures, anyone seen this happen with Gotway-app + King Song -wheel?).

The reason I used the 0x04 "extra byte" before the first header was to distinguish the voltage/etc. -data from the odometer. Normally, this isn't needed, as the data starts from the voltage/etc -portion, and you can rely that the every second time it's that, and every second time it's the odometer, but due to supporting more protocols and just in case the data would start from the "middle" for whatever reason, I wanted to be sure it can pick it the "right way around" and not try to read the voltage/etc. -portion as odometer data and vice versa.

The current version of the app has autodetection-logic to "measure" which protocol (currently supports Gotway & King Song) the data matches and pick the correct way to read it based on that (so it will not expect King Song to send odometer-data). The idea is that the app can be extended to support multiple different wheels, and it can autodetect the type of wheel without need for the user to tell it which wheel it is. The autodetection also uses part of the data at the beginning for protocol matching, and that can and will lead to a situation where the data starts from the "middle" after autodetection is done and it's being fed to the actual decoder.

 

[RolluS]

Hello,

I own a Microworks controller board which should have serial data enabled, and the seller send gotway app for it.

I hadn't the Bluetooth module so I've bought a HC-05 module, but unfortunately GW app neither than Wheelmetrics can grab data (BT connection is OK).

I guess the BT module UART is misconfigured. Please, could someone get his board UART parameters?

 

 AT+UART?

http://www.instructables.com/id/AT-command-mode-of-HC-05-Bluetooth-module/step5/AT-commands/

I think the connection parameters may be my problem.

For the AT commands, you just connect to the HC-05 with a terminal via a RS232-TTL adapter (FTDI, Arduino...).
 

Thanks for the help guys

[RolluS]

No body? Please

@esaj

[Hunka Hunka Burning Love]

Does this help any?

http://www.robotshop.com/media/files/pdf/rb-ite-12-bluetooth_hc05.pdf

http://www.linotux.ch/arduino/HC-0305_serial_module_AT_commamd_set_201104_revised.pdf

https://arduino-info.wikispaces.com/BlueTooth-HC05-HC06-Modules-How-To

Edited June 24, 2016 by HunkaHunkaBurningLove

[RolluS]

Yep, thank you. 

As we can see in these doc, we can set baud rate, parity, stop. 
For my problem I guess these parameters does not match between my controller and my bluetooth module. That's why I'am asking  if someone could get his board UART parameters?

 

[Hunka Hunka Burning Love]

Oh I see.  You need to find the baud, parity, and stop settings which the Microworks BT module is programmed at to communicate with the control board.  It is a problem to try a few different settings like 9600,0,0 or 38400,0,0 and vary the stop and parity settings or is it too much a guessing game?

Did you try contacting the Microworks people who sold you the board?  Maybe @electric_vehicle_lover can check his board as he has a few of these Microworks controllers.

Edited June 25, 2016 by HunkaHunkaBurningLove

[RolluS]

Exactly this!

I've already asked @electric_vehicle_lover and unfortunatly he can't check. I guess he will need the information for his led project BTW.

I'll ask the seller.

[electric_vehicle_lover]

Hey!!! I just shared pictures of communication with the 30B4 and I can read the values like the capers, current, etc - please see the firmware message.

[JumpMaster]

Hi all,

I've made an Android and Pebble App for KingSong which I think should work with GotAway too. I decompiled the KongSong app and took the parts to decipher the data.

APK and PBW can be downloaded from here.

https://github.com/JumpMaster/WheelLogAndroid/releases/tag/0.1

Source code is also on GitHub.

Let me know what you think.

Edited June 27, 2016 by JumpMaster

[electric_vehicle_lover]

Great work @JumpMaster :-) -- thanks for sharing.

@esaj / @JumpMaster I want to read the value with an Arduino, can you please tell me a correct sequence of data that I should look to be able read speed and current? - I want to control an RGB led bar, making it RED while braking (when current > 0 and inverted) and making an sequence that is faster with the speed value.

My Arduino code for now is this:

// send data only when you receive data:
  if (Serial.available() > 0)
  {
    // read the incoming byte
    data = Serial.read();
    
    switch (state)
    {
      // start by looking for the START sequence of bytes: 0x18 0x5a 0x5a 0x5a 0x5a 0x55 0xaa
      case 0:
      if (data == 0x18)
      {
        state++;
      }
      else state = 0; 
      break;
      
      case 1:
      if (data == 0x5a)
      {
        state++;
      }
      else state = 0;
      break;
    
      case 2:
      if (data == 0x5a)
      {
        state++;
      }
      else state = 0;
      break;

      case 3:
      if (data == 0x5a)
      {
        state++;
      }
      else state = 0;
      break;
      
      case 4:
      if (data == 0x5a)
      {
        state++;
      }
      else state = 0;
      break;
      
      case 5:
      if (data == 0x55)
      {
        state++;
      }
      else state = 0;
      break;
      
      case 6:
      if (data == 0xaa)
      {
        state++;
      }
      else state = 0;
      break;
     
      // next 2 bytes are voltage
      case 7:   
      state++;
      break;
     
      case 8:   
      state++;
      break;

      // next 2 bytes are speed
      case 9:   
      state++;
      speed_temp = (data << 8);
      break;
       
      case 10:   
      state++;
      speed_temp |= data;
      speed = speed_temp;
      Serial.print (speed);
      break;      

      // next 4 bytes are trip distance
      case 11:   
      state++;
      break;
     
      case 12:   
      state++;
      break; 
      
      case 13:   
      state++;
      break;
     
      case 14:   
      state++;
      break;

      // next 2 bytes are current
      case 15:   
      state++;
      current_temp = (data << 8);
      break;
     
      case 16:   
      state = 0;
      current_temp |= data;
      current = current_temp;
      Serial.print ("-");
      Serial.print (current);
      Serial.print ("\n");
      break;
      
      default:
      state = 0;
      break;
    }

It seems to work when I send the data from PC terminal but not from the board (but I verified with oscilloscope that TX is well connected).

[electric_vehicle_lover]

@esaj do you know why the calc values of  Temperature = signed 16bit int, real value = value  / 340.0 + 36.53  (Celsius) ??

[esaj]

9 hours ago, electric_vehicle_lover said:

@esaj do you know why the calc values of  Temperature = signed 16bit int, real value = value  / 340.0 + 36.53  (Celsius) ??

My best guess is that if they're using the MPU6050 as the gyroscope/accelerometer, they're reading the temperature from there. The equation is exactly the same as how you read temperature data from the MPU6050:

The equation comes from this:  https://www.invensense.com/wp-content/uploads/2015/02/MPU-6000-Register-Map1.pdf

Page 30:

The scale factor and offset for the temperature sensor are found in the Electrical Specifications table (Section 6.4 of the MPU-6000/MPU-6050 Product Specification document). The temperature in degrees C for a given register value may be computed as: Temperature in degrees C = (TEMP_OUT Register Value as a signed quantity)/340 + 36.53 Please note that the math in the above equation is in decimal.

Edited August 2, 2016 by esaj

[qba]

@esaj, it seems that packet starts from 55 AA, and ends with 5A 5A 5A 5A. This conclusion is from terminal app I have run on my iphone (first received bytes were always 55 AA and last 4x 5A). That simplifies many things, doesn't it? Anyway, the "5" and "a" are often used in serial protocol as they are "0101" and "1010" respectively. This helps to distinguish noise from useful bitstream.