Over the past few years I have become very versed in optimizing WordPress for speed. Most generally I can get just about any WordPress site to load in under one second.
Recently I was working for a client that had a site that was just dragging. We tried everything, disabling plugins, caching, I optimized all of the theme files, used a cdn, re-installed wordpress, hell we even tried a new host and dns server. While everything done had helped, we were still running over 10 seconds to load the home page! This was absolutely UN-acceptable.
The culprit: Post tags.
At some point in time it appears that someone used some sort of auto-tagging software to tag post. This particular site had over 6k tags. Once I removed all of the tags with 0, or 1 post associated we were left with about 2k, and the load times significantly decreased to less than 5 seconds.
The owner is still working to clean out the old tags, but I am convinced this will put them right back where they need to be performance wise.
Just another reason to not overuse tags.