Linux and Windows Toolchains for use with Open Firmware EBike code

gfmoore

100 W
Joined
May 1, 2021
Messages
172
Location
Stoke-on-Trent, UK
I do not want to pollute the Re: TSDZ2 EBike wireless standard (like Specialized Turbo Levo) - OpenSource (Casainho) thread https://endless-sphere.com/forums/viewtopic.php?f=28&t=106346&p=1654673#p1654673 with technical details that are not relevant to what he is actually doing, but I did want to provide any resources and tips for anyone who wants to set up a toolchain for working with his and anyone else's code in order to understand and if possible further develop the marvellous work these guys do, and it truly is amazing.

I hope this is acceptable.

Regards
Gordon Moore - yes, but it's not "him"... :wink:
 
The first thing I want to do is to be able to compile and build his repos without any warnings or errors. This C development stuff for embedded devices is a bit new to me, so if you spot any glaring errors please feel free to advise and correct.

I started off with a totally clean Linux (Ubuntu 20.04 minimal) system running on a Core i3 Intel NUC device.

I will put my steps here as soon as possible and will attempt to keep them updated here.

I will also look at doing the same with Windows 10 and using the Windows Subsystem for Linux (wsl2) that also uses Ubuntu 20.04.

Finally I will look at pure windows.

So please keep looking here. (I will upload relevant documentation to the Open Firmware EBike repo as soon as possible too.)

edit: Setting up pure Ubuntu 20.04 toolchain.

1) I took the Ubuntu 20.04 distribution and installed a minimal version on my PC. (Intel Core i3 NUC)
(As an aside: I work headless from my Dell Core i7 laptop and use x11vnc - man I can't tell you how many problems I had
getting that to work properly (took 5 installs as I kept breaking my boot up), but eventually I did. If you need the tech details
give me a shout :) )

2) As always
Code:
sudo apt clean
sudo apt update
sudo apt upgrade

Also allow Ubuntu to do it's automatic updating ... there is a Software Updater app in the Ubuntu menu.

3) (For me I added x11vnc server so I could work remotely from the machine on my laptop)


(I am now following the Developer steps from the repo (@Ranana did this I think?)
4)
Code:
sudo usermod -a -G dialout USER_NAME    --reboot
I don't know what this does?

5) Using the Ubuntu software application I installed Add VSCode (I use the Insiders version, but the stable version is available too)

As a note when you get messages about updating VS Code there are a number of options. I created a script:
updateVSCode
Code:
sudo apt-get update
sudo apt-get install code
Change the file permissions chmod a+x updateVSCode and run as ./updateVSCode. (Choose your own security permissions)
There may be easier, better ways of doing this, let us know.

Check that VSCode runs.

$ code . I think does it or from the Ubuntu menu if you use a GUI. (Gnome)

6) Install some extensions to VS Code

install C/C++ extension (to edit C/C++ source files)
install Cortex-Debug extension (to be able to debug ARM microcontroller)
install Task Manager extension (to call makefile and other commands) – Zhang Shen

7) Install git and the Github extension - for cloning if this is how you like to work (I assume you can just download the repo as a zip and unzip into a directory?)

Code:
sudo apt-get install git
git config --global user.name "Gordon Moore"             <-- use your own name!!!
git config --global user.email g@gordon.etc"              <-- not certain this is required?

Install the GitHub Pull Requests and Issues extension
see https://code.visualstudio.com/docs/editor/github

These next steps weren't straightforward. Ubuntu uses some kind of security protection - a keyring. If you automatically login then this will popup as you attempt to connect VS Code with github - the password is your Ubuntu password. It's a right pain.

Click on github icon in left trim stack (vertical bar)
You’ll then see a 1 on people icon – requesting a github signin, just do what it says I guess including the (gnome?) keyring. (Do you have a github account?)

Click on git source control icon to see if we are in business

It will attempt to cnnect to github, gets some authentication code, hopefully returns you to VS Code (and then the keyring pops up - wish I could stop this, but because I autologon...
(Use seahorse (called password and keys now) and blank out the password in the default keyring or just delete it? - it didn't work.)

Probably not a bad idea to do a reload of VS Code (or even a full reboot - though this is Linux not Windows :wink: )
Open the command palette ( Ctrl + Shift + P ) and search for and execute the command: >Reload Window.

8) Back to Dev instructions on repo:

Code:
sudo apt-get install openocd
sudo apt-get install gdb-multiarch
sudo apt-get install gcc-arm-none-eabi
sudo apt-get install binutils-arm-none-eabi             //not sure this is needed as it didn't do anything?
sudo apt-get install libnewlib-arm-none-eabi          //ditto?

9) I found I needed to install "make"
Code:
sudo apt install make
(might need full gcc path at some later point? so perhaps
sudo apt-get install build-essential 
would be better? (advice?)

10) I needed to install nrfutil. The instructions on the Nordic site didn't work. Now the way I did it might not be the best way as it needed some adjustments to the $PATH environment. The following method installed nrfutil into my local bin directory, not the global one /bin ? I would be grateful for any insights on this as I'm sure there must be a better way.

Check that Python 3 is installed - it should be
Code:
python3 --version

Now install the pip python installer (pypi?)
Code:
sudo apt install python3-pip

//Now install nrfutil
Code:
pip3 install nrfutil

//Now you need to adjust the environment $PATH as it installs into the local ~/.local/bin

For me I went into the .bashrc and added some lines I got from the ~/.profile file

Code:
sudo nano .bashrc

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/.local/bin" ] ; then
    PATH="$HOME/.local/bin:$PATH"
fi

activate it

Code:
 .  .bashrc

As I said there may be a better way?
(Note I also added it to the /etc/profile in the same way - I think this is better?????)

Nearly there...

11) So now in VS code and if you clone repos

Create a directory where you want your cloned/downloaded repo to go (I use ~/ebike) i.e. mkdir ebike

Click on the Explorer icon on the left trim stack (vertical menu bar).
It should have an option to clone?
Clone the repo: https://github.com/OpenSourceEBike/TSDZ2_wireless into th ebike directory

It's up to you whether you open a folder or open a workspace. There is a workspace file there so I used that.

File Menu Open workspace
Navigate to the ebike/TSDZ2_wireless/EBike_wireless_TSDZ2/firmware folder
A message for the workspace file should come up - click on it.
Voila, your files await.

12) To clean and build
On the left trim stack menu, select the task icon (it may be hidden in the ... icon depending on your available screen resolution.)

You should see the menu:
Click on the green arrow for clean and then on the green arrow for build. (Not sure what the difference is between build and build development - it will be in tasks.json)

Your code should now compile and run nrfutil and create your zip. (But currently there are some issues - see next post.

Please let us know how it goes and what issues and I'll update this asap.

Gordon

13 May 2021
 
Assuming I have a clean working toolchain for Linux, I cloned the repo using VS Code and open the workspace on TSDZ2_wireless>EBike_wireless_TSDZ2>firmware and noted a number of problems.

1) 1. Change c_cpp_properties.json to locate the sdk_config.h file. I used the nearest one in the include folder. I assume that they are all the same, but haven't checked.

Code:
             "forcedInclude": [
                "${workspaceFolder}/include/sdk_config.h"
            ],

Whilst here I also changed the
Code:
"intelliSenseMode": "gcc-arm"
to stop intellisense changed messages

2) I also had to provide a reference to the nRF5_SDK_16.0.0 folder. I assume this comes for Nordic? There are quite a few of these dotted around. Again I am assuming that no changes have been made to any of these?

Code:
c_cpp_properties.json 
            "includePath": [
                "${workspaceFolder}/**",
                "${workspaceFolder}/../../common_firmware/include/**",
               "${workspaceFolder}/../../common_firmware/nRF5_SDK_16.0.0/**"    //<------
            ],
            "forcedInclude": [
                "${workspaceFolder}/include/sdk_config.h"
            ],
by the way the /** means recursive all wildcard I think.

3. In boards.h comment out custom_board.h as it cannot be found and I assume is not used. I assume it would be better if there was one really in case anyone want to do further development.

4. There were lots of issues in the problems tab (at the bottom of VSCode)

4a) unused rt_vars in eeprom_init_variables() in eeprom.c I commented out line 272 - is this correct?

4b) Lots of enumerations (warnings) that hadn't been included in switch statements:

I added the following dummy switch statements in state.c: around lines 369, 461 and 583

Code:
switch (g_motor_init_state) {
...
      case MOTOR_INIT_READY:
          process_frame = 1;
        break;

      //add unhandled enumerations
      case MOTOR_INIT_GET_MOTOR_ALIVE:
      case MOTOR_INIT_SET_CONFIGURATIONS:
      case MOTOR_INIT_GET_MOTOR_FIRMWARE_VERSION:
      case MOTOR_INIT_GOT_MOTOR_FIRMWARE_VERSION:
      case MOTOR_INIT_OFF:
      case MOTOR_INIT_ERROR_ALIVE:
      case MOTOR_INIT_ERROR_GET_FIRMWARE_VERSION:
      case MOTOR_INIT_RECEIVED_MOTOR_FIRMWARE_VERSION:
      case MOTOR_INIT_ERROR_FIRMWARE_VERSION:
      case MOTOR_INIT_ERROR_SET_CONFIGURATIONS:
        break;
}

...
          g_tsdz2_firmware_version.minor = p_rx_buffer[5];
          g_tsdz2_firmware_version.patch = p_rx_buffer[6];
          g_motor_init_state = MOTOR_INIT_GOT_MOTOR_FIRMWARE_VERSION;
          break;
          
        //add unhandled enumerations
        case FRAME_TYPE_CONFIGURATIONS:
        case FRAME_TYPE_ALIVE:
          break; 

...

static void motor_init(void) {
...
    switch (g_motor_init_state) {
    
//these enumerations are not handled?
        case MOTOR_INIT_ERROR_ALIVE:
        case MOTOR_INIT_ERROR_GET_FIRMWARE_VERSION:
        case MOTOR_INIT_RECEIVED_MOTOR_FIRMWARE_VERSION:
        case MOTOR_INIT_ERROR_FIRMWARE_VERSION:
        case MOTOR_INIT_ERROR_SET_CONFIGURATIONS:
          break; 

    }
  }
}

}

Not totally sure if these should be on the inner switch or the outer switch?

5) I am left with some what I assume are cast errors, but this is now beyond my understanding:

The error message (for one of the three) is:
Code:
./main.c: In function 'services_init':
./main.c:1007:43: warning: assignment to 'ble_tsdz2_write_handler_t' {aka 'void (*)(const unsigned char *, short unsigned int)'} from incompatible pointer type 'void (*)(uint8_t *, uint16_t)' {aka 'void (*)(unsigned char *, short unsigned int)'} [-Wincompatible-pointer-types]
 1007 |   init_tsdz2.tsdz2_write_handler_periodic = tsdz2_write_handler_periodic;

Looking at the code fragments I've tried to extract we have:

Code:
ble_services.h
typedef struct
{
  ble_tsdz2_write_handler_t tsdz2_write_handler_periodic;
  ble_tsdz2_write_handler_t tsdz2_write_handler_configurations;
  ble_tsdz2_write_handler_t tsdz2_write_handler_short;
} ble_tsdz2_init_t;

main.c
742  static void tsdz2_write_handler_periodic(uint8_t *p_data, uint16_t len)
       {
764  }


982  void services_init(void)
       {
987    ble_tsdz2_init_t init_tsdz2 = {0};

1005   memset(&init_tsdz2, 0, sizeof(init_tsdz2));

1007   init_tsdz2.tsdz2_write_handler_periodic = tsdz2_write_handler_periodic;

1013 }

I haven't yet figured our what is going on here, but I think that the parameters to the functions do not have the correct type. This may or may not be important, but improper type errors may be a problem down the line. Strict type checking is essential in some languages (Java), but not others (JavaScript). With my limited experience of C and pointers I suggest this could be an issue at some point? Any takers on what to do?

Gordon
 
Final couple of issues:

6)
Code:
Preparing: _build/TSDZ2_wireless.bin
DONE TSDZ2_wireless
cd ./../../EBike_wireless_bootloader/firmware && make -f ./Makefile
make[1]: Entering directory '/home/gordon/ebike/TSDZ2_wireless/EBike_wireless_bootloader/firmware'
Makefile:346: Cannot find include folder: ./config
DONE nrf52840_xxaa

So looking at the Makefile NOTE this is the EBike_wireless_bootloader/firmware/Makefile !!!
Code:
7   Proj_DIR := .

177 $(PROJ_DIR)/config \

373 SDK_CONFIG_FILE := $(PROJ_DIR)/config/sdk_config.h

I think the \ is just a concatenation operator for all the directories that the make searches.

For line 177 since the /config directory doesn't exist I removed it. (pity couldn't just comment it out with #)

For line 373 I changed config to include, since that was the closest directory to find the sdk_config.h

so
Code:
373 SDK_CONFIG_FILE := $(PROJ_DIR)/include/sdk_config.h

7) In the main.c in the TSDZ2_wireless/EBike_wireless_bootloader/firmware
Code:
Compiling file: main.c
./main.c: In function 'read_pin':
./main.c:164:28: warning: variable 'temp' set but not used [-Wunused-but-set-variable]
  164 |   static volatile uint32_t temp;
      |                            ^~~~
Compiling file: nrf_dfu_svci.c

so at line 162 in this bottloader main.c
Code:
/* This function is specifically to avoid the follwing issue:
[173] GPIO: Writes to LATCH register take several CPU cycles to take effect */
static __attribute__((optimize("O0"))) bool read_pin(uint32_t pin)
{
  nrf_gpio_cfg_input(pin, GPIO_PIN_CNF_PULL_Pullup);

  static volatile uint32_t temp;
  temp = NRF_P0->IN;
  temp = NRF_P0->IN;
  temp = NRF_P0->IN;

  if (nrf_gpio_pin_read(pin))
    return true;
  else
    return false;
}

Interesting bit of code. So I like how these three attempts at reading the Pin 0? is just to cause a bit of a delay.

Since I must be an obsessive compulsive! I added a
Code:
  temp = NRF_P0->IN;
  temp = NRF_P0->IN;
  temp = NRF_P0->IN;
  
  if (temp == 0) ;

just to get rid of the warning. :lol:
I guess there could be a cleaner refactor...

So that's it. Apart from issue 5 which I don't yet know how to fix, the TSDZ2_wireless/EBike_wireless_TSDZ2/firmware will now build without errors or warnings, However, I cannot guarantee the correctness of my fixes. I guess I should compare the contents of the hex/bin files with the real ones.

Gordon
 
I do not know why the 5 and how to solve, but I know the code works for me like that.
 
So moving on to TSDZ2_wireless/EBike_wireless_remote/firmware

Opened workspace, clean, build

Code:
nRF5_SDK_16.0.0/components/toolchain/gcc/Makefile.common:113: GM ./nRF5_SDK_16.0.0/components/toolchain/gcc/Makefile.posix 
make: -gcc: Command not found
Cannot find: '-gcc'.
Please set values in: "/home/gordon/ebike/TSDZ2_wireless/EBike_wireless_remote/firmware/nRF5_SDK_16.0.0/components/toolchain/gcc/Makefile.posix"
according to the actual configuration of your system.
nRF5_SDK_16.0.0/components/toolchain/gcc/Makefile.common:130: *** Cannot continue. Stop.
The terminal process "/bin/bash '-c', 'make'" failed to launch (exit code: 2).

This took me ages to figure out, because I knew that gcc existed and worked (use the terminal).

Eventually I figured out that the solution had been given me:
Code:
Please set values in: "/home/gordon/ebike/TSDZ2_wireless/EBike_wireless_remote/firmware/nRF5_SDK_16.0.0/components/toolchain/gcc/Makefile.posix

When I checked I couldn't find that file, though I had used locate and it had existed, but in other directories, I had even opened it and looked at it (from the terminal, not VS Code). Moved it from a similar location on common_firmware and bingo! A clean build, no problems or errors.

So kudos to Casainho.
:thumb:

Gordon

My next step is to get all this implemented in the Windows 10, wsl2 and Ubuntu set up.

@beemac, if you're watching, I sense you have a pure windows setup already now? Care to explain how you achieved it? Even in general terms, as I don't think I'm going to bother with a pure one.
 
(Sorry I've left some markdown formatting in so I can update the repositories)

# Instructions for setting up toolchain using Windows 10, Windows Subsystem for Linux (wsl2) and Ubuntu 20.04 LTS

This is a work in progress

### Step 1: Set up and configure Windows Subsystem for Linux v2 (wsl2) on Windows 10

Here is a detailed explanation [link to wsl2 article] (https://code.visualstudio.com/blogs/2019/09/03/wsl2) of what to do and check.

You will then need to download Ubuntu 20.02 LTS from the windows store.

So you can get into a bash shell for Ubuntu in various ways, but I just click on start menu and start typing Ubuntu ... or add it to start menu or ...

You should get a bash shell.

### Step 2: Update Ubuntu

Code:
sudo apt clean
sudo apt update
sudo apt upgrade

### Step 3: Install Visual Studio Code for Windows (VS Code)

Here we install VS Code as a normal windows app, not a Linux app - that doesn't work because wsl2 does not yet support GUIs (afaik) (Well I initially tried and it didn't work.)

### Step 4: Install the VS Code WSL extension

VS Code - Extensions: search Remote WSL and install - you may need to reboot VS Code (CTRL SHIFT P Developer: Reload Window (I'm using an Insiders version of VS Code)

This is where the magic happens. This extension allows Vs Code to access the Linux subsystem rather than the windows workspace (if that's the right word?)

If you look at the very bottom left you will see a green icon


If you click on it you will get a popup window, click on "new WSL window" (or why not try the other options... :) )
You don't need to keep the original VS Code is open. It will remember that you are on wsl2 next time you open VS Code. To go back to Windows workspace click on green icon and select as appropriate.

Note how the green icon at the bottom left changes to "WSL:Ubuntu:20.04"
VS Code is now looking at the Linux workspace.

### Step 5: Install VS Code extensions

Install C/C++ Extension Pack - accept the message which installs Linux stuff
Install Cortex-debug in VSCode
Install Task manager extension (Zhang Shen??) (This is cool. It allows shortcuts to clean, build and other things you want to do - the json is in .vscode > task.json if you want to add your own.


### Step 6: Install Linux pre-requisites as per documentation

[link to documentation](development-flash_and_debug_firmware.md)

But in brief:

Code:
sudo usermod -a -G dialout USER_NAME so for me sudo usermod -a -G dialout gordon

sudo apt-get install openocd
sudo apt-get install gdb-multiarch
sudo apt-get install gcc-arm-none-eabi
sudo apt-get install binutils-arm-none-eabi
sudo apt-get install libnewlib-arm-none-eabi
sudo apt-get install srecord


Note: I already had a toolchain installed for compiling C programs, that is gcc etc, but I can't remember what is needed. You may need to do this (Certainly you will need make

Code:
sudo apt install make

Perhaps doing
Code:
sudo apt-get install build-essential
may be better?

(I think it would be good to close down stuff and do a total reboot - it's up to you ;) )

### Step 6A: Install nrfutil

I needed to install nrfutil. The instructions on the Nordic site didn't work. Now the way I did it might not be the best way as it needed some adjustments to the $PATH environment. The following method installed nrfutil into my local bin directory, not the global one /bin ? I would be grateful for any insights on this as I'm sure there must be a better way.

Check that Python 3 is installed - it should be, if not you'll have to install it (sudo apt install python3 probably)

Code:
python3 –version

Now install the pip python installer (PyPi?)

Code:
sudo apt install python3-pip

//Now install nrfutil

Code:
pip3 install nrfutil

Now you need to adjust the environment $PATH as it installs into the local ~/.local/bin
For me I went into the .bashrc and added some lines to the bottom. I got the lines from the ~/.profile file

Code:
sudo nano .bashrc

Code:
…
# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/.local/bin" ] ; then
    PATH="$HOME/.local/bin:$PATH"
fi

activate it
Code:
 .  .bashrc

As I said there may be a better way?
(Note I also added it to the /etc/profile in the same way - I think this is better?????)

Nearly there.

### Step 7: Clone a repo

For me I created a directory (folder = windows nomenclature!) called ebike.

Now in VS Code if you click on the folders/directory icon at top left of the vertical bar (it's got a ui name but I forget - stack I think?) it will (should) show an option to open a folder or clone a repository. Choose clone and enter the location of the github repo you want.

Select the ebike directory and the repo will be cloned in a folder underneath this location.

e.g. https://github.com/OpenSourceEBike/TSDZ2_wireless

(This also contains the EBike_wireless_remote code, though there is also a higher level ebike_wireless_remote - I'm awaiting direction on which to use.)

### Step 8: Open the workspace you require

(This was new to me, but the repo TSDZ2_wireless uses a file called workspace.code-workspace that organises the projects - at least that's how I understand it. Opening a workspace allows you to open the project you are interested in (more easily perhaps?) )

[link to article on VS Code workspace](https://code.visualstudio.com/docs/editor/workspaces)

### Step 9: Clean and build

You can now use the icon on the left stack (probably at the bottom) (or in the hidden items indicated by ...) called Task Manager.

From the Task Manager select clean and the run icon, the build and the run icon.

Please see forum article https://endless-sphere.com/forums/viewtopic.php?f=28&t=111757&p=1655841#p1655841 for details on getting any application code to build if it doesn't work.

### Step 10: What else?

### Step 99: Do something cool!
 
Interesting find:

Windows and wsl2 Ubuntu on core i7 32GB ram (and old one) is twice as fast at compiling as pure Ubuntu on Core i3 and 4GB ram (also old one). :D
 
Back
Top