Go to your local Home Depot or paint store and get a gallon of Jasco prepaint treatment. It is a high butyl soap and has some phosphoric acid in it to remove oxidation. Your problem is that the surface (the top coat of paint) is very oxidized so anything that contacts it will react and stain. Use the Jasco and your problems will go away. There will be no need for hard scrubbing.
If you have a situation where the staining is in the paint, itself, you can use dilute phosphoric acid applied with a hand-held spray bottle but be careful, the acid mixture will burn your skin.
I do this kind of stuff for a living, painting and cleaning all manner of things, residential siding, process refrigerant tanks, expensive art frescoes, pipe arrays, interior of mfg. plants, hardwood interior trim, I get my hands on all types of surfaces for renewal and painting.