No, but it can be devastating if your spouse finds out, says Vaughan. "The person may suddenly feel as though she doesn't know her partner." If you've made a strong emotional connection with someone else, with or without sex, it can be very painful for your spouse.