Actually, charging all of them in parallel is probably better than using a 48v charger. Since the batteries are in parallel, the voltage on each battery will be equal. They will tend to share the current very evenly unless one battery shorts out or something.
It's going to be a hassle to rewire them for charging every time, and be careful not to mix up the connections.
A really good way to charge them would be to buy 4 separate 12v chargers. You can leave the batteries in series, and have wires coming out from each one to a plug (or plugs). This way the batteries will tend to stay balanced and you won't need to mess with the wiring to charge. To do it this way, you need to make sure the chargers are isolated, but that's easy to test and easy to fix if they are not.
I would not recommend a 48v charger unless you have some kind of battery balancing circuit. Over time, they tend to get out of balance, and the weaker cell gets fried.