We study how the powerful perceive power abuse, and how negative experience related to it influences the appropriateness judgments of the powerless. We create an environment conducive to unfair exploitation in a repeated Public Goods game where one player (punisher) is given a further ability to costlessly subtract money from others (victims). Punishers who abuse their power rationalize their behavior by believing that free-riding, while forcing others to contribute, is not inappropriate. More importantly, victims of such abuse also start to believe that punishers' free-riding and punishment are justifiable. Our findings demonstrate the capacity of humans to exculpate abusive behavior.