Office Lingo: Agree to Disagree

One of the phrases of Office Lingo I hate the most is "agree to disagree." People use it all the time and think they sound cool when they say it. It makes no sense to me when I break it down though. Earlier today, I said something that most people would disagree with. The person I was talking to said "I would agree to disagree." So in other words, he disagreed. Why couldn't he just say "I disagree?" What a waste of syllables and at the expense of sounding like a moron. If he disagrees, then he's not agreeing with me. By agreeing to disagree, it means he's agreeing with me to disagree with me, which is a false statement because we're disagreeing with each other.