I spent 25 years living in the country, surrounded by Confederate flags, and then Trump signs. It was much easier to discuss the weather or how many foxes or bear you saw lately. But you're in a tough spot.
I think deep down you probably already know the answer. Snarky comments won't help you or anyone else. Nor will you change anyone's mind. If they see you (at the pool or whatever) hanging out with POC, I think some will be more inclined to keep their statements to themselves. But I wouldn't count on it.
Good luck !