That's a passive aggressive thing to say to you, and unkind.

When people are rude or talking on subjects you prefer to avoid, just ignore them. Don't look at them, seem bored and distracted and leave the room, or go home as though you remembered something you had to do.

If you only look at people and pay attention them when they are behaving themselves, it quickly trains them to be kind and friendly to you.

This technique works quickly if you are consistent. Don't give anyone being nosy or religious any attention at all, and it takes away their motivation to harass you.