Check your PerformanceFu and it will show you the memory usage of Addons, which can help alot.
Some of the mods you are running are pretty hefty, but nothing jumps out at me... other than some of your libs. What addons do you use that require Earth, Chronos, Sea, MCom, and Khaos?
Auctioneer, AtlasLoot, CensusPlus, are big ones. SCT can be a hog if you turn too much on. DurabilityStatus and DurabilityFu prbably do the same thing?
Give us some more info and we'll see what we can see. Big question though: Did this start happening after patch 2.1? If so... it's Blizzard's fault.