Since you won't tell us anything about the specifics, we can only guess and speculate.
The problem with your question is that it's answer will depend on the speed and load, etc, you're driving the motor at, because the amount of power you get out of a motor depends partly on those, and some motors are better at some speed ranges than others, and some are better at some loads than others, and so on.
So any answer we give you will only apply to the originally-designed usage range for the motor in question, not to the application you're using it in. It might be that there is a much better choice for your application, if only we knew what it was....
greenmountainski said:
It has to be a hub motor.
If you're intending to put the props on the hubmotor directly, you should keep in mind that typical hubmotors are not designed to spin fast enough to do what you want. A few hundred RPM maximum, usually 200-300rpm.
I did a quick google, and see that small prop-driven-aircraft typically use 2600-3000rpm. I don't know what a helicopter main rotor uses, that was inconsistent. If it's a lot lower, in the hubmotor range, then you could probably use one, but:
The tolerances for hubmotors can be very large, and runouts on axles/etc (and thus bearing mounts, rotors, etc.), don't really matter on a wheel, but they will matter a great deal for high RPM with a prop load, and may matter with a lower RPM and a much higher prop load.
It's possible that they are not even mechanically strong enough, depending on how you intend to mount the props to them. The only feasible way I can imagine would be to make a complete collar that fastens around the rotor, with it's own shaft for handling the load (at least on the "top" end) that is just spun by the motor and does not depend at all on the integrity of the motor to prevent problems, or the motor's own bearings. If you used a "pipe" axle in the motor, in place of the original, it could pass all the wiring thru that, so you could then put another pipe around it, to place the prop bearings on, which will need to be thrust bearings and rotational bearings (unlike the hubmotor's, which only need to be rotational bearings).
The power losses in the motors from running them at far far higher than their design RPM is likely to make them pretty inefficient, and require some form of active cooling.
You'll also need to use a massively higher voltage than they were meant for to get that RPM, and have a controller capable of that kind of ERPM. I expect that's going to be fairly expensive, if one even exists.
I also suspect you're going to be spending more money on re-machining the parts, replacing bearings, etc., to be able to do what you're after, enough to have instead designed it to use a motor that was intended for driving propellers.
If you're driving the propshaft via an RPM-multiplying transmission of some kind, it's going to add more weight in addition to the heavy hubmotors.
greenmountainski said:
My mechanical engineer who I am paying says we need to start at the motors which are already on the market, rather than build a fuselage first and then find the right motors after.
That's probably the best way.
But you're starting with the wrong type of motor for a prop application, whether high or low RPM.
He thinks building motors is out of the question, because of technical difficulties.
That's probably true, unless you either really know how to make them already, or can use simulation software to get you close, then build prototypes until you get to where you want to be.