So I ran another test this morning. Lifted the rear wheel and ran the motor very slowly. No problem it seemed to turn fine. Then I disconnected the red phase. No turning wheel until I spun it by hand. The same with the blue and green phase. Reconnected all again and the wheel would turn again from a dead stop so it's definitely getting signal from all phases. Tested it when I was on the bike and it's still jerky on take-off.
Could this mean fets in the controller are blown? I did a long ride and I'm still seeing +- 10% less than full power with wide open throttle.
I'm trying to figure out what to do next. Either get the hub to a motor specialist and have the halls and phase wires replaced, or try have the controller repaired/replaced.
Edit, forgot I tested it with a lyen 9 fet controller and the start up was still rough. So all issues point to the motor...