Most of us have at least one passive-aggressive person in our life. Maybe it’s a mother who nitpicks and criticizes everything we do, or a co-worker who uses sarcasm to cut us down, or a micromanaging boss who drops hints, but never tells us directly that we’re not doing a good job. Just being around a passive-aggressive person can harm one’s mental health.

How can you deal with passive-aggressive people?

There are a few things you should know about passive aggression: First, it is a form of anger. Your passive-aggressive mother, co-worker, and/or boss are deeply angry people. They’re just as angry as a person who screams or throws things, but they have a different way of showing it. Passive-aggressive people are often terrified of confrontation, so they couch their anger with smiles. Some may not be self-aware enough to realize they’re angry, but their anger, bitterness, or frustration lies just under the surface.

As a marriage and family therapist in Santa Monica, I treat many people with passive-aggressive anger issues. I sometimes see their partners as well. It’s those individuals, who have to deal with a passive-aggressive person, day after day, who often need the most help, one, because the passive-aggressiveness is hurting them, and, two, because they’re likely enabling the behavior.

It takes two people to support a passive-aggressive relationship. To stop the cycle, try these five steps:

1. Hold them accountable.

When you fail to hold a passive-aggressive person accountable for their actions, you unintentionally perpetuate their behavior. If you’re a people-pleaser, this is especially devastating: You want to make everyone happy, and you don’t like confrontation or conflict, so you absorb all kinds of subtle emotional abuse. Stop blaming yourself or making excuses for others; you are not responsible for the damaging way a passive-aggressive person shows their anger.

2. Stop apologizing.

Unless you did something wrong, don’t apologize. Especially don’t apologize if they refuse to be direct and tell you what they feel you’ve done wrong. If your boss says, “Leaving early again today?” every time you go home before 5:30, but is never straightforward about wanting you to work later, don’t apologize or make an excuse. Be upfront and ask if you need to stay late. It may be that he or she really does need you at work later, but it also may be that the boss just wants you to feel guilty because that makes them feel more in control.