What is happening is you get drag from the magnets in the direct drive motor. Its called cogging, and gets stronger if any phase wire is shorted, or regen braking is engaged.
Your cogging sounds about normal to me, not exceptionally strong, or weak. It varies by motor brand, and even by individual motors sometimes. One motor I had could even fail to rotate when you pushed the bike at first, but ran fine. That was an extreme example.
One thing about cogging, it does increase the faster you spin the motor, so it resists a lot more when decending a mountain at 30 mph, than it does if you are just riding along at 5 mph, like you do if something broke, or you just ran out of battery.
To get good coast when you need it is easy, just tickle the throttle a tiny bit, less than 1/4 throttle, and the motor will then turn easy, without using up your battery. At low throttle, the motor will turn a certain rpm, but if you are already pedaling or coasting that speed, it will not use more than a few watts. if it draws 50w that sounds like a lot, but its not. One amp hour of 48v would last one hour at 50w draw. So a few min coasting, or pedaling down hills at low throttle will not blow your range.
So when you go down a big hill, and want to coast faster, open the throttle a crack. When you are screaming down a hill and would not like to hit 50 mph, leave the throttle off and it will slow you some.