Handheld Serial Programming: USB or Wireless

Greetings –

Many folks want to program their controllers via handheld devices as a matter of convenience. Without barging into other good threads on programming and their apps, why not focus on the specific challenges that at present inhibit this bridge of technology.

Simplistically, the goals should be scoped to explore and resolve the interface between common Chinese controllers using off-the-shelf serial communication technology:

• USB
• Bluetooth

USB Issues:

• Presently hardware exists to mate a USB cable between computer and controller (natch).
• Typical handheld devices (such as Android) do not allow for serial communication via USB without rebuilding the kernel. This is fairly easy to accomplish if you are adept at programming, otherwise not. In addition, this is an Inheritably problematic solution to deploy.
• Aside from the mobile O/S inhibitions, it seems easiest to fix in terms of hardware & cabling; nothing new to purchase.

Bluetooth Issues:

• Handheld programming application does not exist, though could be adapted with some effort.
• Requires a BT module, possibly a 5V -> 3.3V regulator to power it. BT boards appear to range from $7 to $15.
• Without a more sophisticated module, the hardwired commit-switch is easier to implement.
• Regardless, this is probably the best alternative for the DIY hobbyist if the mobile application existed.

Thoughts and ideas welcomed. I’d like to build a list of BT candidate boards and see if we can get one side of the equation to connect.

With potential, KF

No bytes?

I have decided to conduct a simple experiment using an Arduino (clone likely) Bluetooth Module.

The Plan:

1. Facilitate communication between the Desktop and the BT device first
2. Follow up with flashing the Controller - if possible - using the existing programming applications.
3. Create a test application for Android to communicate with BT device
4. Work with existing contributors (if willing) to port the Python version of the Controller programming app over Android and validate.

Work begins today. Purchased off of eBay:



I expect this to be a direct replacemenet for this:



Specs vary between clones, but we expect it to be minimally compatible for this application.

Features of purchased module:

• Wireless Serial 4 Pin Bluetooth RF Transceiver Module RS232.
• 100% brand new and high quality.
• Allows your device to both send or receive, the TTL data via Bluetooth technology, without connecting a serial cable to your computer.
• Works with any USB Bluetooth adapters.

References for this programming effort:

• ArduinoBT Specs
  Control an Arduino from your Android device using a cheap bluetooth module
  Scripting Layer for Android
  IronPython

Concerns:

• Clone BT Module may present unforeseen compatibility issues.
• Learning Python. I may elect to try a .NET test app first, and then port that over to IronPython to buffer the learning curve.

Comments:

• Most of the BT modules advertise up to 30-foot/9m range. For the desktop, this would greatly simplify my process by eliminating the need for the USB cable, and that in itself makes this attempt worthy.
• Estimated delivery of BT Module: Mon. Mar. 11 - Tue. Mar. 19

Updates as they happen. 8)
Cheers, KF

.....

You make a good point.

In my search for a BT module I came across an interesting alternative:


4-Channel BT Relay

• Wireless Relay Board
• 8 Channel USB/Wireless Relay Module (Xbee,Bluetooth,WIFI Extend ) + cell phone remote control

This certainly opens up possibilities beyond the scope of this thread, and I instantly imagined how one could possibly remove Cruise Control and the 3-Way Current Control from the handlebar. That’s getting a little bit ahead, cart-before-the-horse, though certainly an excellent option once we tackle the chief convenience of flashing new params in the field.

~KF

.....

Uses RF. Plus we’re limited to 64 char data packet when we’d rather have streaming (unlimited). We want to use a Tablet or SmartPhone or other device, and I can’t see how to do bridge that without another RF device. BT is already built into most portables and mobile O/S.

~KF

.....

.....

kingfish. I used those bluetooth rf board with arduino in a uni master project to interface an arduino with a computer. not quite as easy as it should have been at the computer end since the java library was not well documented. but worked eventually. used bluetoothdesktop library by xtrapixel.ch which was based on something else by francis lee. hope that give you some (google based) ammo when you come to trouble shooting should you need to.

xadmx, hehe – no worries; I’m debating making a 2nd pot of java myself after last night… :wink:
The only person I know that has Controller flashing software is anpaza on this thread:

• XPD: open-source keywin e-bike lab replacement

I have not formally asked him for contribution; my focus is simply to make the hardware connection work - and then ask as a feature request to port the app over to Android.

For my part, I have d/l’d Android SDK, IronPython and Mono for Android, and will attempt to you my Samsung Android Charge as the test victim (willing or not). With IronPython it’s possible to port the app over to .NET which could make it available for Windows Phone et al. Lots of potential.

For distribution, it’s likely this will cost a tiny sum to get it code-signed and blessed. I’m not there yet but it has occurred to me we could possibly use Kickstarter. But first, we have to crawl before we can walk. As it is… the solution is still baking in the oven (hasn’t arrived yet).

Whereswally606, Thank you for the lead; is this it? bluetoothDesktop

Mucho thanks for the support, KF 8)

KF, fully support your effort in this area. Having a BT serial port module for the Android is beyond awesome!! Having a BT serial port module for the PC is fully awesome!

C'est ca, tres bien KF. BluetoothDesktop is the one. not javadoc'd but good nonetheless.

Thanks BigMoose

I’m looking at the source code now in Visual Studio (with the VS Python plugin); it’s Open Source, so apparently we can do what we want – as long as we keep it Open Source: That sounds like a reasonable agenda. :wink: Makes one think what code-signing would cost for OS or NfP. Anyways…

Python Programmers: There are text strings embedded in the code – English default. And then there are two localizations in Deutsche (German) and Russian, both are compiled. I haven’t yet figured out how they map. Standard coding practice dictates all strings belong in a resource file; I don’t know how to do that in Python. Always something to learn… The actual code itself appears humanly readable, looks closer to the C-language than to Java.

Using the Android Virtual Device Manager, I can select several targets from Android 2.1 to 4.0. In addition Device Definitions range from the Google Nexus S (480x800), to the 10.1" WXG” Tablet (1280x800), and back down to itty bitty 2.7” QVGA (240x320). I might have to do a poll to acquire the LCD… or most popular. Two years ago my phone (Samsung Droid Charge) was top of the line at 480x800; this appears to be the standard now. What do you think – poll or just make a stab in the middle?

ADDENDUM:
Forgot to add links to smartphone resolutions...
http://smartphoneresolution.com/ <- is that like basic or what?
smartphone screenresolution on flickr


Knee-deep, KF

Pardon the slight OT; it might actually be relevant to the thread (I intended it to be, but not sure it ended up that way).

xadmx said:

i have been thinking along the same lines, but having a touch screen on the bike using something like the stm32 development board with a basic display.

Click to expand...

Do you mean the STM32 Primer (nee Circle), or rather the newer Primer2? I have the original one, intended to learn programming to use it to create an ebike control computer...but I am horrible at learning how to code. And now, the Cycle Analyst v3 actually does a fair bit of what I wanted it to do...but not all of it and not itn eh way I wanted to do it (graphically).

Somewhere on LadyAda's forums at Adafruit, there is a thread under my membername about that controller and some of the specifications I started to write for it, when someone was volunteering to help with it (but dropped out before even starting), though those specs were still very incomplete when I gave up. (I thought I'd also posted it here on ES but can't find it)


I wanted to use some sort of wireless communications for data offloading and programming, but not knowing how I might do that I think the specs just called out USB for transfer (or maybe a memory card, I forget).


I'm afraid I don't have the technical knowhow for any wireless methods to help much directly with this thread, but if opinions are needed, I'd say that having BT would be a very useful thing, for programming adapters and methodology, though I expect USB is a lot easier.

AW, I think your point is completely valid especially when we consider the scope of the larger digital dashboard type experience.

The native issue with Smartphones and USB is that we are blocked by design from using the combination whilst still a phone. Plugging the USB cable in serves two functions: Facilitate charging, and file access to the embedded memory card: The phone asks which mode I want to use, and if I elect file transfer between desktop then I cannot access phone applications until disconnected. (As a matter of practice, when not in use my phone is always connected via USB, but only to a charging hub that is disconnected from any other device other than wall AC).

It’s possible to root the phone and rebuild the O/S to allow the dual-functionality, but at present I could be violating Federal Law… not that I haven’t done that before (or am doing that now), but I kinda like to keep my phone the way that it is. :wink:

The 2nd avenue is using BT, and here we are not blocked. At present the Windows 7 BT Service cannot find my Android phone; that may or not be by design. Maybe Win8 may resolve the issue, but I’m not going to upgrade for that reason alone. One solution is to d/l drivers off the Internet that swear they can fix the problem… although I want source code, I want to know why the issues exist, and fix them without risky inheritance. Thanks to Whereswally606, I’ve been digging into the bluetoothDesktop Library – and other related efforts in the .NET realm and have discovered that there are better BT implementations out there. I have yet to test them, though perhaps that will change today.

The issues so far are:

• Enabling BT on Windows 7 in a manner that can discover my Android device (presently blocked)
• Enabling File Transfer between Desktop and Android conveniently without breaking warrantee or fooling about with modes. BT promises to allow that and retain security; we’ll find out. FT between devices is important as a first step before flashing the Controller.
• Deciding on the best language for Android (and other smart devices including iPhone and WinPhone). I don’t know enough about Mono yet to assume it will do all, though at present it has my focus.
• One app or two? The original concept was to develop a mobile application to configure and flash the controller via BT. Now though it seems prudent to have this from the desktop as well: Is it one app that does all, or two? Architecturally, we have the following responsibilities:
  • Using a visual interface, set parameters and store the configuration in a local file system
  • Open existing configurations, read the contents, and correctly set the interface for further activity
  • Compile the configuration into an acceptable binary format for consumption
  • Select a COM port and communicate bidirectional via Serial to the Controller, and successfully flash the unit
  • Report Status and errors if any
  • Don’t crash ever
  • Support localization (means other languages). I have a plan on how to do that – though the implementation is dependent upon the programming language.

Through this discovery it appears obvious we should seamlessly share configurations to and from Desktop-and-Mobile, or Mobile-and-Mobile; that would complete the circle in my mind. I am positive there will be security issues to resolve.

Anyways, that’s my digest for this morning having slept upon the rest.

Hmmm, empty mug. Time for another pot.
One less hour, KF

I am an android neophyte. Luke tried 4 years ago to get me out of the horse an buggy days, I resisted ... I finally got my first smart phone 2 weeks ago. Galaxy S3. I picked up a BT earpiece to take telecons at work... love it... tried to get the windows 7 machine to recognize it to play music through... It found the earpiece but it wouldn't show up in the audio picks. Google search rambled something about windows left the BT stack out of windows 7 although it is in XP... said to buy a BT/USB dongle, install the software stack, and now your built in BT should work. I'll have to try it.

I would start with a "simple" application like making the programmer. I have found that when the job gets too big, it gets lost in the development cycle. There are so many off growths possible for other applications if that "simple" BT controller programmer could be made to work! I'll bet if you develop and put a "simple" framework out there, development and applications will just explode.

That said, I don't know if you are willing to slow down this much, but when you figure out how to write, compile and load an app into the android; I was wondering if you would go slow enough so that a neophyte like me might figure out what language, what compiler,.... and all that jazz. You know the basic framework of what it takes to write and load an app.

^^Yes, I can do that. I know my responsibilities: Documents must flow like The Spice from Arrakis

I too worry about how to load the app into Android; if it goes to the Marketplace then I’ll have to go through vetting. We're not there yet.

Downloaded both Mono for Android and MonoTouch. Since Mono claims to be CSharp-based I might try just a standard WPF port of Python source so I completely understand the binary construction and Flashing issues (read: a prototype for study); like you said – a programmer app. Plus I can isolate the strings into a resource file where they need to be for localization; kill two birds in one.

Mono has an migration analyzer tool that checks for gotchas; I can run that and lurk for issues. The VS plugin for Mono though (not the Mono for Android) leaves something to be desired; underwhelming and cryptic.

Anyways, I have my stash of snacks lined up for a gloomy day -> meaning it’s a good day to write code (as are most in the PNW) :wink:

Cheerios, KF

Made good progress yesterday; I have the lion’s share of English strings isolated into a resource file and the elementary classes defined for production. My hope is to compile a binary by EOD.

Project Notes for Today:
The following is a list of issues uncovered in this investigation; there is no fault here – just a “comprehension” on how it’s been accomplished in the past and present, and I am trying to work through them with creative license.

It seems basic, though the XPD application differs from the other Chinese versions in a couple of documented ways – and that’s fine. Example:

• File Open & Save
• In the Chinese apps, we can find and load an ASV file using the typical OpenFileDialog() method, and Save works quite similarly.
• With XPD, it’s a little different in that the default configuration templates are located next to the application, and customized versions are stored in the User profile under an “xpd” directory. I appreciate that for the logical sense, although it makes it more challenging to import a previous version.
• Solution: I’d like to do something in-between where we still have our template location, our custom location, plus I still want to hunt and search in other locations – including network drives. This sets us up to load files off the desktop from a mobile device (and vice versa); we may or not use this specific process... I don’t know if it will work, or if we need an “import” or “file transfer” feature. Regardless, I am adding the Open File & Save File Dialogs back into the traditional mix. Hopefully this will work in Mobile mode.

• ASV File Format:
  At present there are two known versions of this format: Chinese interpretation and the XPD, the latter of which is well-documented on why the differences exist. I hope you guys won’t hate me for this but… I want to introduce a new format which is altogether different. First, allow me to propose a truce:
• As a high-bar, I would like to try and import both ASV versions. For that matter, it’s possible they can be exported as well… I’ll know when I get there.
• The new format will use a completely different extension so that previous applications won’t be able to consume the data. Anyone that has ever worked with me before knows my penchant for creating self-evolving formats (SEF): The promise of SEF is that we have inheritance with version control, and translation templates. A good protocol starts with simple common elements, thus the Chinese ASV would become Version 1, and the XPD ASV would become 1.1 because it is almost identical. The version I propose would have at least one more field titled “Comments” for reasons that are self-described:
  • Personally, I have a boat load of ASV configurations and the only way I can differentiate them is in the Filename which is lame. I want a comments field where after three pots of coffee I can wax poetically about the features.
  • The other advantage to this format is to add metadata such as “Author” or “Owner” and really any other user-defined tags of personal value. None of the extra data will affect the binary production for flashing.
• The file format of the new proposed configuration would be XML which is universally acceptable for storing data. Actually we can go much deeper and create collections of configurations having like properties, example:
  • I have a 2WD ebike with a Front Controller and a Rear Controller, each is uniquely configured. What’s to stop me from creating various configurations say for Commuting, Cross-Country, Race Day in Winter, and so forth – all stored under one file name (or library if you will). This keeps organization simplified in the folder.
  I haven’t gone hogwild here – these are just proposals. It all started with “How am I going to load my ASV file for testing?” …and well, coffee had a lot to do with inspiration :wink:

There is more of this line of thought developing as I walk through the Open Source and comprehend the full utility provided by XPD developer Anpaza. One of my goals is to make it so we don’t have to recompile the code to include new controller templates or languages. But more on that when I get there.

Still waiting on the BT module.
Baby-stepping, KF

KF, I just ordered my bluetooth/RS232 module.

I was wondering... as a first step could you make a "terminal module" for the Android? Something that put the modules output on the screen and let you type in a phrase and send it to the module?

Would that be a logical step on the way to the programmer? ... or a needless detour?

You mean like a ring-down, "Hello World" first contact? Sure - that could be accomplished. My BT module hasn't arrived, although it would be a good first step - after solving Device Recognition.

Which module did you order and when will it arrive?
~KF

I got this one http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=290859124600&ssPageName=ADME:L:OC:US:3160



4 to 10 days shipping from today.

I will bet the device recognition sequence is not trivial... I naively hope it's all handled in "the stack"

I was thinking something like add a TTL to RS232 adapter to the module. Hook it to a pc and open hyperterminal. Have the module linked to the Android, and have the Android have a "simple" hyperterminal on it. Type on the Android, it shows up in hyperterminal on the PC. Type on the PC hyperterimal and it shows up in a text box on the Android...

To me that would be a super taking off point for all sorts of Android interaction with "stuff" I have an idea how many hours it will take to get what seems "simple" going! Nothing is "trivial" until you have done it and have it working... then you see all the shortcuts you could have taken!

Yeah, um… I think BT Discovery may not go easy either. However we have two good libs to toy with, plus there’s the not-so-fully-implemented CE Stack – but that requires some C++ to fix and I don’t want to get down into the mud for that. Other folks have solved the BT issue so I have more hope than fear.

Digging around… I found a bit of code that made me LOL; going to post it in Harmless Jokes.
Enjoy, KF

Status:
This project is going slower than I initially thought – though we’re making good progress. I can easily imagine at least another week of solid effort – mainly because I am working with unknown technologies. Don’t expect a port to Android a week from now; more likely a really good buff of the Desktop WPF prototype… although that is an interim step towards Mono deployment.

New & Exciting:

• The Bluetooth Module arrived on time on Monday. I haven’t had time to play with it yet. There was zero documentation; not a good sign. It means I have to go prospecting for setup.
• Serial Capture: As of today I have setup two virtual serial ports connected through Null-modem emulation, and have jury-rigged serial capture from the XPD application. In English:
  • Virtual Ports enabled using com0com, accepted defaults for COM6 & 8 except using Port class & baud rate emulation. Took a little bit of arm twisting to trick this out.
  • Serial Port Capture: Tried using Tera Term which I use for lurking via modem on Caller-ID though it was too limiting. Realterm Serial Capture though did the trick, and it allows either ASCII-BIN or HEX dump to file.
  • Using Hexplorer to investigate the resulting binary and I confirm that the begin and end bytes match the Python code.
    This step is important for validating my code output against a known trusted quantity.
• Prototype Features: Today I can read in an ASV file, validate through a series of tests, and load it onto the stack in preparation for Flashing. There’s no user interface per se; this is all down in the trenches straight code. Before we flash, we need to create a binary representation of the ASV settings. The visual context of those settings is different than the textual contents of the ASV file: Some mathematics are required to display these settings in a more convenient and comprehensible manner. Helper functions are required to present and retain changes, and that aspect of coding was no small feat.
• Organization of the application and data is about 60% completed. With the Python code, any changes to content require recompilation. The code I am developing departs from that model so that new controller boards of published families can be added without knowing how to write code. Allow me to explain briefly what I am talking about:
  • There are two classes of Chinese Controllers: The EB2XX models and the EB3XX. They share much in common, but their settings are slightly different. These models are grouped into “Families” called Infineon2 and Infineon3 (for obvious reasons). For developers out there – I am using the same conventions as in the Python application. If a new series of controllers is introduced - say EB412, then we’ll need to write a new class for that family – however the application is architected in such a way that we do not have to redesign the code.
  • Each Family has a series of models: EB206, EB209, EB212… EB224, etc. For each model there are very specific unique mathematical calculations that transform the setting value of certain parameters into screen numbers, and screen numbers back to setting values. These maths are nicely built into Python in tight shorthand. However if we want our cake and eat it too, these have to be exposed if we want to add new models. I grabbed some really nice fast code that can interpret formatted strings into math Postfix tokens that in turn evaluates the formula and produces a result. The short story is that is works.

What’s left:

• We’re just now at the point to where We (as in the royal we, meaning me, myself and I – integrating all of those multiple personalities, or at least the ones that matter for this project, but I digress…) can compile a binary and validate it against a known quantity – today.
• Develop a rudimentary user interface to test the math codes as previously described.
• Add Serial Communication over USB, and flash a controller.
• Add in Bluetooth as an alternate device, and whatever else it takes to support secure air-born communication with a remote device.
• Find a remote BT serial device and flash the controller.
• Find my Android phone and exchange files over BT.
• Begin Phase II: Porting the code over to Mono for Android.

A humble bow to Anpaza:
Where possible, I am keeping the naming conventions the same between Python and my code. I humbly thank Anpaza about once an hour for writing this tremendous piece of work; I can’t begin to imagine how many hours he spent sorting out the details and the formulas. Python is a little bit of a challenge for me cos I’ve only been looking at it for maybe a week now, but he’s done a fine job with it where I can follow along without a question. – and that’s a mark of perfection in itself. 8)

If humility and thanks could be transformed into bricks and mortar, it would build a rather large shrine to house my marble & bronze appreciation for Anpaza. :wink:

Been at this more than a week straight; think I’ll have a beer tonight. St. Pat’s on Sunday; that’ll be my day off.

Cheers, KF

Enjoy the Guinness. Thanks for proceeding with this KF its a massive step forward for making our bikes commuters in the week and opening them up for the trails at the weekend. It will also make fine tuning the power output so easy that those of us who want to go further by putting in just the right amount of pedaling for us can do. Also think of this One bike, many riders each with individual riding power requirement able to change them via presaved settings on a phone. The future is bright but Orange was bought out in the UK and is now EE Ha Ha Ha (maybe that was funnier in my head)

Not to unfairly question the basis of your design, but why not just add a screen & joypad to the arduino and plug it directly into the programming plug of the controller? Run that arduino on batteries, and encase it in a project box.

The arduino has sufficient EEPROM to remember the variables you last sent to the controller
A screen will allow you to see the variables via a menu system
Variable limits can be hard programmed into the arduino code
A joypad will allow you to navigate and set the variables.
Because the arduino will pump out native serial communication - no usb port required, just wire a harness with a reset button

A small arduino will fit in a mints tin 8)