Help Needed: Android (or windows) Dashboard for Realtime display of SFOC5 controller's serial output, etc

Let's restart this thread at this point, since I haven't been able to figure out how to do anything, and haven't been able to conscript anyone to do it (including the controller creator).

So this post:
https://endless-sphere.com/forums/viewtopic.php?f=30&t=95633&p=1472376#p1472376
is the new start of the thread.

Basically I need help (really, someone to write) making an app or a program or something, that can be used to deal with data in and out of the SFOC5 controller's serial port, to display it's realtime data output as a dashboard, and input settings into it, without having to go thru the complicated mess presently required.

Is intended to be open-source and usable for anyone that ends up with an SFOC5 by Incememed, whose thread for the controller itself is here:
https://endless-sphere.com/forums/viewtopic.php?f=2&t=30680&p=1400439#p1400439
By itself it has no display/control app, just a bunch of general-purpose programs to do various parts of teh necessary functionality, which makes the controller unnecessarily difficult to setup, troubleshoot, experiment with, and use.

I'd like to fix that for everyone.













This is starting for the SFOC5 over here;
https://endless-sphere.com/forums/viewtopic.php?f=2&t=30680&p=1400439#p1400439
but if it's built right it could be used for any controller that can output realtime statistics via serial, including the Cycle Analyst.

I started a project on the MIT App Inventor here
http://ai2.appinventor.mit.edu/#5746969786056704
becuase i know nothing at all about android programming, so it's probably my best shot at figuring it out with minimal help.

(I know the concepts of programming in general, and decades ago did stuff in BASIC, assembly, machine code, etc, and started to try to learn C...but I'm terrible at all of those, so I don't expect to be any better at this).


Anyhow, for anyone that wants to help, the idea is that there is a serial data stream that has to be parsed, converted to human readable units, and some of this data will be displayed on screen realtime, on an old tablet I have here (a samsung galaxy sgh-1487, android 4.1.2). Or generically, on any android device with the right hardware, if that can be done later.

Specifics on the stream and conversion factors are on page 28 of the manual, shared here:
https://drive.google.com/open?id=1hUZNHZ-GG1cUyFpnZPxqK4348mGH2Pd0
along with the rest of the documentation:
https://drive.google.com/drive/folders/1SrqjgC2lSxXCAzWI8V--YkTYgAr2jAxa


If it's possible I'd like to also log all or select data from the stream to a file on the android device, or a card in the reader (that is part of the external USB port device that plugs into the tablet's port).


My first hitch is finding a way for the android device to read from the serial port of a generic clone-prolific-chipset USB-serial adapter (as thats the one I have that works with the SFOC5's port; others I have don't for one reason or another).

I dont' even see USB as an option for connectivity in the MIT stuff, so....

Until I can do that part, I haven't started building any other part of the app as it all depends on being able to do that.



If all else fails, maybe I can find a BASIC program that will run in Windows10, that can poke/peek (or whatever) at a serial port to read streaming in data like a terminal program, or even just read a capture file as it's being written by a separate termnal program (like realterm), and parse and display that data on screen as text in human readable format.

Nice project! Why don't you use a bluetooth module to send the data to the smartphone? There are many code examples how to send / receive serial data via bluetooth with android. This turorial shows how to conncet the module in principle. The BlueTerm app can only show ascii data, there are other apps in the playstore, that can show and log hex data also.
https://opensourceebikefirmware.bitbucket.io/windows_instructions/index6.html

regards
stancecoke

Bluetooth is a good idea, the Android probably supports that better than a physical serial connection.

FWIW, it's a tablet rather than a smartphone (has no phone capability, just wifi/BT).

I've seen the BT seems to be used for just about all android connection stuff. It's the only option even listed in the MIT App Inventor...so I may be forced into it. Was just hoping to use the hardware I already have. :/

However, using BT means waiting for parts, and buying stuff, spending money for just a beta test (the controller has to go back to incememed at the end of the test). If I'm going to spend any money on the testing, I'd rather save it to potentially buy the controller at the end of the test (if it is cheap enough and can be improved enough for my uses, which it most likely can).

Besides...I'm kind of (self-trained?) to not spend money wherever I can avoid it, and to reuse whatever I already have if I can (even if it's the hard way :lol: ).

Sure, the BT'd still be potentially useful for other things like CA logging, etc., but I can already do that to a laptop via USB-serial if I needed the data (generally don't), and I don't need to see CA stats on a different screen than the CA already has. (if I do, there's already a project here on ES to do that, which I wish could be adapted to my purposes as it would probably greatly simplify what I'd have to learn and do).

Probably just have to bite the bullet and get a common, well known and well-supported (and cheap, which probably negates the last one) serial-BT module.


That said, I'm also working on how I might convert an existing captured log file on a PC into a table of human readable data, perhaps with columns for the various different readouts.

It's more for curiosity, to see how the controller behaves over a ride, just to glance down the columns and see how (for instance) current and battery voltage change, or temperature rises, etc. Don't need to graph it as I can see a trend via numbers about as quick as in a graph.

ideas for the app:

Take in the data live and convert it then and there (and log it, too), and display it on screen as it changes. Have peaks (and minimums) left displayed above and below the currently-incoming values.

Let's restart this thread at this point, since I haven't been able to figure out how to do anything, and haven't been able to conscript anyone to do it (including the controller creator).



What is needed is one application, or perhaps two, depending on what the eventual programmer of this thing can do, that does the following:

--Accepts the realtime stream of serial hexadecimal data from the SFOC5's output (probably via BT, for android stuff, can be direct serial/USB if it's a Windows laptop program)

--Parses that stream into its' component segments of data per the SFOC5 manual's explanation of which bits are about what

--Logs that data in human readable form to a file for later reference / troubleshooting, date/time stamped.

--"Simultaneously" displays that data in GUI human readable form on a dashboard screen (preferably configurable by the user for what is displayed; it's not all important)

--one part of that display is a hex representation of a binary "code" list, that basically is a set of flags telling the user which error codes or battery conditions / etc presently exist, so that could be a separate top row of stuff on the display; it's more "critical" information than the other realtime data, as far as momentary conditions go. (presently this is also communicated on a separate 4-LED module as a binary representation of a hex number, and you either have to just know what the codes mean, or stop riding every time one comes up to look it up in the chart in the manual--I am terrible at remembering these things, so I have to look them up--would be much simpler to take the bytes from teh stream and put this up on the dashboard as human-readable information).

--Ideally be able to display that as a larger central value for each readout that's the current number, with peaks (and minimums) left displayed much smaller above and below the current values. Perhaps in a shade of text color that's more "grayed out" than the current value, if smaller is a problem.

--Separate screen reads the static Stats from the unit at the push of a button (onscreen), to display what the last set of detailed data was, in GUI human readable form.

--Logs that data adding to a separate file in human readable form from the above, date/time stamped, also for reference troubleshooting.

Separately from the above, but could go in the same program:

--A Setup screen that allows all parameters to be chosen / entered by the user (isntead of the spreadsheet presently required)

--the setup screen would also ahve a button to send the settings to the SFOC5 (instead of having to copy/paste the spreadsheet send field(s) to a terminal program (which has to be configured every time first), and then a cable connected, and then the send button pressed).

--be able to send the firmware updates to it via the serial port (presently must be loaded into a program called DS30loadergui.exe, various settings configured, write button pressed, and then the serial port connected). Perhaps this could leverage the commandline version of the program that I presume the gui version is based around? Don't know how this works under the hood.

I could live without the last requirement, as firmware updates are rare, but settings changes are needed often during tuning, and other experimentation / setup, so that part would be kinda critical to the program.

I could also live without the logging, if that's an issue, but it could be very handy for troubleshooting. The realtime display (with min/max history) is fairly necessary though.


At least one small piece of hardware has to be made to do this, and that's a:

-- BT-serial adapter
--with it's own DC-DC converter to power it off the ebike's traction battery
-- and provide 5v for the SFOC's MCU during settings sending and stat reading, but only upon pressing the onscreen button in the program to do this

I can probably build the hardware from modules available online, but don't know how to have it only enable the 5v output to SFOC5's serial input on remote command. I think either RTS or CTS or something like that could be used, but I cant' remember which comes from the remote end, or if they're already used by the SFOC serial, and if there's some alternative signal line on the BT-serial adapter taht could be remotely triggered by the program.




I think I covered everything...but whoever is willing to help would need to discuss it here, and probably read the current SFOC5 manual (attached). Specifics on the stream and conversion factors are on page 28, IIRC.
View attachment Manual SFOC5 050419.pdf


Please ask any specific questions you have, and I'll get you answers if I don't have them.

There are of course countless different ways this could be implemented. Personally I would build a web application. The benefits are:

• Cross platform
• No requirement for heavy toolchains or development environment setup
• Easy to use final application, no software installation necessary
• Instant feedback allows fast iteration, easy to debug
• Relatively simple for anyone to modify
• Lots of high level libraries available e.g. for math functions or graphs

Many options are out there for communicating with a web app. A while ago that would have involved something like using websockets with a daemon running on the PC which transferred data between the websocket and a serial port. In Android that would involve a custom app (I.e. using the Apache Cordova framework) and a USB serial device driver plugin that exposed the serial port through a custom JavaScript API to the web app.

Another option is a WiFi chip like the ESP8266 or ESP32 that can create a WiFi hotspot and host a web server itself.

There are several native web APIs available now that can support connecting to external devices like this. The obvious one would be Web Serial. Unfortunately that's not supported in any browser yet although it looks like Chrome is almost ready to release it. The other APIs are Web USB and Web Bluetooth. WebUSB is supported in Chrome on all desktop platforms and Chrome for Android. It's also supported in Opera and the upcoming Edge 75. Web Bluetooth is a little better supported and as well as the previously mentioned browsers it also works on the Android browser, Samsung Internet, Opera Mobile and can work on iOS with the WebBLE browser.

I don't have any experience yet with WebUSB so I'll focus on Web Bluetooth. Web Bluetooth only supports BLE (Bluetooth Low Energy) so common bluetooth serial modules like the HC-05 won't work. If I was creating a product from scratch, I would use a Nordic nRF52 series chip with a custom firmware. This is exactly what I've done with my BLE energy monitor project. For this application I think it will be simpler to use a commonly available Bluetooth LE module that won't require any custom firmware or hardware. Fortunately these have been available for some time: http://jnhuamao.cn/bluetooth.asp?id=1. They are well documented and are supported with firmware updates (click on the images on their website).

The HM-10 module is the most common and can be found cheaply on eBay and AliExpress. Although beware of cheaper copies with different or older firmware. This module can be frequently found soldered to a base board which contains level shifter circuitry, a linear regulator (5V > 3.3V) and an LED as well as breaking out the important pins. The only potential issue is that BLE does not support a native serial protocol. These modules implement a custom protocol but it involves reading or writing bytes from a "GATT Characteristic". Initially Bluetooth LE only supported a maximum 20 byte payload in each characteristic. This means that it would require several reads or writes to receive or send a large serial string. Probably not an issue but depends on their implementation. The newer modules such as HM-18 and HM-19 support larger MTUs (and thus payloads) but they're not so widely available, especially soldered to a base module.

Instead of worrying about a power supply from the module, I would simply have the Bluetooth module be powered by the controller's 5V output.

This is the module I'd recommend buying: https://www.aliexpress.com/item/1911389178.html If you're familiar with Chinese shipping methods you will know that AliExpress Standard Shipping > China Post Registered Air Mail > Yanwen Economic Air Mail. I will buy one as well.

If you don't already have Bluetooth 4.0+ support on your PC I would recommend buying one of these: https://www.ebay.com/itm/Mini-USB-Bluetooth-Adapter-V-4-0-Dual-Mode-Wireless-Dongle-CSR-4-0-Win7-8-XP-L/252848510670 or if it's a laptop you could upgrade the WiFi card to one that includes Bluetooth 4.0+ like the Intel 7260.

First, thank you for this.

flangefrog said:

Personally I would build a web application.

Click to expand...

Makes perfect sense (I didn't list it as a possibility cuz I didnt think it could be, with having to talk to the serial port or BT, and reading/writing local data...and I have no idea how to do web stuff besides ancient html text stuff).

Many options are out there for communicating with a web app. A while ago that would have involved something like using websockets with a daemon running on the PC which transferred data between the websocket and a serial port. In Android that would involve a custom app (I.e. using the Apache Cordova framework) and a USB serial device driver plugin that exposed the serial port through a custom JavaScript API to the web app.

Another option is a WiFi chip like the ESP8266 or ESP32 that can create a WiFi hotspot and host a web server itself.

Click to expand...

Lost me there.

That would definitely have to be done by someone other than me. I understand a tiny bit about wifi hotspots, pretty much nothing about web servers, and zero about setting up chips to do these things.

(I tried at one time to use some "hackware" to turn a wifi router into something like that so I could make it monitor a camera/etc., all stuff already done by others successfully, but I couldn't get it to happen even with the same version of the same hardware and software...and that was way back when I had relatively current info about then-present technology).

It sounds a lot more complex than a "simple' BASIC or whatever program on a PC to just read a port, convert the incoming stream, and stick it on a screen, and log the values to a file. (even that is beyond me in current OS environments...I think I could still do it on an Apple ][ or Vic20.... :lol: )


Also lost me on all the stuff after that, about the hardware. I might be able to look these up and figure something out...but, the idea is to have something very simple for anyone to use, that could be wired up from common cheap modules (cuz I am not gonna get into a business of making these things, I just want to get a cheap and easy open-source setup figured out for SFOC users to display/program/setup their SFOCs). If it takes a bunch of expertise to get it built and setup, it's easier for most users to just stick with the complicated mess they've already got to deal with. :/

Instead of worrying about a power supply from the module, I would simply have the Bluetooth module be powered by the controller's 5V output.

Click to expand...

I don't think that's possible, because in order to send settings to it, or read the stats from it, it has to start out unpowered (switched off at the status panel, that cuts the MCU/driver power/etc) and then be powered by 5v from the serial port to boot it up and get it to send it's stats, and then watch for the incoming settings string starting with 1234 and ending with F00F, IIRC.

If it was an ultra low power unit the controller might be able to power something, but it's not even designed to power hall sensors (it's sensorless only), so I don't know that it has enough current capability. ANd it would only work for reading the realtime serial output data for the riding dashboard (because of the limitation above, coudln't be used for the other stuff needed).

I'll have to check with Incememed about the "surplus" power availability, if any.

If you don't already have Bluetooth 4.0+ support on your PC I would recommend buying one of these: https://www.ebay.com/itm/Mini-USB-Bluetooth-Adapter-V-4-0-Dual-Mode-Wireless-Dongle-CSR-4-0-Win7-8-XP-L/252848510670 or if it's a laptop you could upgrade the WiFi card to one that includes Bluetooth 4.0+ like the Intel 7260.

Click to expand...

I think the laptop has BT; dunno what version it's an old Vista unit by Gateway. Have to check.

I have two old tablets, one android, one ipad, neither of which can be upgraded to a modern OS, I think they both have BT. I have an old samsung android phone, with BT, but I know it can't be upgraded either. I may have a nwer samsung android phone with BT, from a year or two ago, with no SIM, but have to see if it works.

Ok, well, the SFOC5 does have 20mA of "spare" capacity on the 5v line. But this also has to power the throttle for most bikes; I don['t know how much that takes.

(on mine, IIRC, I'm powering my throttles from the Cycle Analyst).

I forgot to ask if the SFOC can provide 5v power to the device that's waking it up to send it settings and/or read it's stats, so I'll report back on that.

From the SFOC thread:

Only when being powered up can it enter programming mode, and only if its RX pin is pulled high (i.e. another device is signalling it wants to communicate).

Click to expand...

So the dashboard comms device will have to have it's own 5v power supply, provided most easily by a DC-DC from the traction pack.



*If* programming or reading the stats was not necessary, and *only* the realtime serial output was going to be used to display/log data on the dashboard, *and* the comms device needed no more than 20mA minus however much the throttle takes, *then* the comms device could be powered by the SFOC5.

The WiFi chips were just mentioned as as another option, I didn't go into this further as among other things it would be more complicated and require custom firmware and more hardware.

amberwolf said:

Ok, well, the SFOC5 does have 20mA of "spare" capacity on the 5v line. But this also has to power the throttle for most bikes; I don['t know how much that takes.

Click to expand...

That would probably be enough. The Bluetooth LE modules only take up to 8mA and an analog hall throttle takes less than 5mA.

amberwolf said:

I think the laptop has BT; dunno what version it's an old Vista unit by Gateway. Have to check.

I have two old tablets, one android, one ipad, neither of which can be upgraded to a modern OS, I think they both have BT. I have an old samsung android phone, with BT, but I know it can't be upgraded either. I may have a nwer samsung android phone with BT, from a year or two ago, with no SIM, but have to see if it works.

Click to expand...

If the laptop runs Vista I doubt it has BT 4.0. And Web Bluetooth actually requires Windows 10. Depending on the age the iPad or Android devices are more likely to have it but I would check the specs at https://www.gsmarena.com

If none of the devices support BT 4.0 it might be better to go with another solution, either with BT 2.0 or WiFi or simply go with a wired USB-serial option for PC only.

amberwolf said:

So the dashboard comms device will have to have it's own 5v power supply, provided most easily by a DC-DC from the traction pack.

Click to expand...

I don't read that post as requiring a separate 5V supply, just that the RX line has to be pulled high which could be done even if the module was powered off the controller.

The WiFi chips were just mentioned as as another option, I didn't go into this further as among other things it would be more complicated and require custom firmware and more hardware.

Click to expand...

I'd be all for them if they're easy to setup for this purpose once the firwmare/etc is written, and still cheap, and someone can write taht firmware/etc. I just have no real idea how to do anythign with them.

I expect that they'd be more universally useful, not just for different display devices, but also for *other projects* like this that needt o get serial data to a dashboard.

If the laptop runs Vista I doubt it has BT 4.0. And Web Bluetooth actually requires Windows 10. Depending on the age the iPad or Android devices are more likely to have it but I would check the specs at https://www.gsmarena.com

If none of the devices support BT 4.0 it might be better to go with another solution, either with BT 2.0 or WiFi or simply go with a wired USB-serial option for PC only.

Click to expand...

I have a win10 laptop, but it's too large to stick on the handlebars as a display, and it's also not well-built so it wouldn't survive very long used on the trike. (used it originally when testing teh SFOC for logging/etc on test rides, but over time vibration made ports that aren't even connected at the time stop working, like the VGA).

Don't think the old one could handle win10 well enough to be useful...it's already so slow/etc even with a fresh wipe/restore and only basic os/drivers on it that it can be frustrating to use on occasion.

Personally, I think using some solution that will work with both old and modern hardware is better, like I guess BT2, or Wifi (which seems fairly universal), is probably best, as it makes this project more universally useful. If someone has to buy new devices for display that are more expensive, and thus riskier to stick on the handlebars/dashboard/etc, it's less likely to be used (or useful) for those that can't afford it (like me) or that just don't want to.

I just don't know what's possible, or what's better / easier to create.

I don't read that post as requiring a separate 5V supply, just that the RX line has to be pulled high which could be done even if the module was powered off the controller.

Click to expand...

Except that AFAIK to get 5v you have to turn on the "status led panel switch" that sends 12v to the 5v regulator to power that, so until it's "on" you don't get 5v (AFAIK--I have to test that). But if this is already on, and the MCU is getting 5v already, you can't give it 5v to turn it on cuz it's already on, so it won't go into programming mode even if you raise the RX line.

Again, I'll have to test that, but AFAICR this is how it works, cuz I don't think I've been able to program it when the status panel switch was turned on--it had to be off to do that. [

I received the HM-10 Bluetooth LE modules a while ago and I had some time to play with them today. I made a web app which works with Kunteng controllers and displays the data received and allows you to change all the settings. No circuitry is required apart from connecting the HM-10 module to the right connectors on the controller and no installation or configuration of any app, just go to the website on a device which supports Web Bluetooth. I haven't yet added offline support but that's easy. https://kinsatech.nz/kunteng