Why rub on the paint at all? Let the cleaner/chemical loosen it. As mentioned try your solvents and soaps in a hidden area first to make sure it will not melt the paint. You have a textured paint and rubbing it will change the texture.

Do you have any idea what this stain is? Petroleum or organic based? Tree sap, bug juice, food, paint, oil/grease?

Try cooking oil & peanut butter they can loosen things and make them slide off. They will leave a residue and feed the paint and make it look different. A bit of simple green Will remove that.

Some brake clean removes paint, others do not. Experiment with that if needed.

Bristle brush is not much different than sandpaper.