You don't necessarily need a ton of wires and stuff. I have an Ariel Rider X-Class with a 48V battery and 52V battery. Either battery can handle the full max discharge my controller is set to. I just have one fully potted ideal diode in front of the 48V battery:

And an ammeter in front and behind it that I sealed in transparent heat shrink after calibrating:

And then just a Y cable from two XT-60 connectors to an XT90S antispark connector to my motor. You could fit it all in the controller compartment above the bottom bracket (I zip tie it to a frame tube for better cooling, though).
I figure the only way I can mess up is if I drained the batteries completely, unplugged the 52V battery, charged only the 48V battery, then plugged everything together without checking the voltages. I don't really have any plans to ever unplug them, though.
My batteries have on/off switches, but there's plenty of ideal diodes with back to back FETs that let you have a tiny on/off switch if you want:
The on/off switch just controls power to the MOSFETs, so carries barely any current. It doesn't have to handle the full discharge amps.
That said, just a typical marine boat rotary battery selector switch rated for 60V 300A is only the size of the ideal diode alone:

So Glenn's suggestion isn't huge either.