I keep seeing people mention that this is normal since you run with embedded libraries, but I haven't seen anyone mention the alternative to running it embedded.
You can in fact download and run Ace2 as a separate addon/library. It would basically load all of the libraries themselves, and your addon list would show them as their own independant entry. This would also be the case for the memory consumption list. Might be an option if you really wanted to segregate the memory usage more accurately and see how much of it is Ace, and how much of it is Bartender itself (or other addons)
Honestly, once you get over 40 to 50% of your addons being built with Ace, it might be smarter to run Ace2 non-embedded anyway. Makes more sense to me from a code standpoint and a debug/testing standpoint. The downside being you have to update the library separately as if it was an addon itself. If you don't update it, and another addon updates with newer embedded, you are right back at square one as if you didn't have Ace non-embedded. The addon that loaded the newer version will overwrite the Ace library install, and that addon will show the memory consumption again...
And actually, I believe you can select not to download embeds with each addon with the ace updater (correct me if I am wrong). This would basically download the addon without any of the embedded libraries, assuming you have Ace already being downloaded and updated. Check out
http://wiki.wowace.com/index.php/Ace2_Memory_Concerns as previously linked, and the last section briefly explains how this works.