Table Structure

[Ailae]

Hello all.

Looking for some thoughts about table structure. I'm making an AtlasLoot-type addon which obviously has tables upon tables with items.

At the moment, the structure of a module-table look like this (for example):

local t = {
    ["Throne of the Tides"] = {
        ["Order"] = "Lady Naz'jar:Commander Ulthok:Mindbender Ghur'sha:Ozumat:Trash",
        ["Lady Naz'jar"] = {
            ["Order"] = "Normal:Heroic",
            ["Normal"] = "55202,55198,55195,55201,55203",
            ["Heroic"] = "56267,56269,56268,56270,56266",
        },
        ...
    ],
}

(I know it's not localized etc, but not worrying about that right now.)

I use the field Order to make sure stuff is ordered properly since you can't trust the table (when used as a dictionary) to have the same structure. I guess my question is, is this a inefficient way to do it? When retrieving the data I do for match in string.gmatch(table.Order, "[^:]+") do.

Would a structure like this be more efficient/memory-friendly?

local t = {
    ["Throne of the Tides"] = {
        [1] = {
            label = "Lady Naz'jar",
            data = {
                [1] = { 
                    label = "Normal", 
                    items = "55202,55198,55195,55201,55203", 
                },
                [2] = { 
                    label = "Heroic", 
                    items = "56267,56269,56268,56270,56266",
                },
            },
        },
    },
}

Guessing this might be less efficient perhaps, due to it containing more nested tables. I suppose I could embed the label in the item-string as well and just getting that one out when "unpacking" stuff. Something like [1] = "Normal:55202,55198,55195,55201,55203".

Or is either of these ways completely bonkers? It's hard to draw any conclusions by looking at how much memory the addon uses in total, if it's "too much" or not.

Thankful for any input.

[Xubera]

I was under the impression that one style wouldnt really matter as long as you werent creating garbage, like endlessly creating new tables each time. If they are static they are holding memory, not causing cpu cycles (which is what would hender performance, i think...)

I would use the 2nd method myself, as everytime someone called it, it would take less commands such as :gmatch(), therefore using less cycles when the tables are active.

Though take my words like a pound of salt (not a grain) as I have not taken the time to research or claim that I know what really goes on in there. These are just the ideas that come to mind when I think of it

[Rilgamon]

I'd look at the ways you want to access the data.
Then construct it so that you can get to the data you need with as few
lookups as possible.

[yssaril]

The first is more efficient. lua is very efficient in loading/creating long strings but is slow when parsing tables at load time so you want to reduce the number of tables initially. With a bit of code you can then expand your strings into tables at runtime ondemand where the table creation time is minimal since you only create what you need right then and there

[Ailae]

I see.

Taking what Rilgamon said into account, the 2nd alternative (albeit slightly tweaked) feels more manageable from an upkeep-perspective. Since I'll do more than just drops from mobs (stuff like factions and tiers) the number of sublabels ("Normal"/"Heroic" in the example above) can vary.

So right now, I'm thinking of doing like this:

local t = {
    ["Throne of the Tides"] = {
        [1] = {
            label = "Lady Naz'jar",
            data = {
                [1] = "Normal:55202,55198,55195,55201,55203" 
                [2] = "Heroic:56267,56269,56268,56270,56266",
            },
        },
        ...
    },
}

I suppose I could cram all labels and items into just the one string but that feels like a maintenance-nightmare. :P The upside by doing this way rather than example 1 is that should be easier to localize as well as not having everything fail if the string in table.Order doesn't match the key used later.

The way it is used is basically this:
I register the table t with a name, this is then added to a list of Categories. So the example above would be added as "Dungeons" or similar. The next level contains the name of the dungeon(s), which is sorted alphabetically and then used as sub-Categories to the main Category. The next level is the key for the items and are added as a sub-list to a sub-Category (sorted by index, since bosses are in a specific order usually).

All this gets put into 3 dropdowns, dependent on each other. When an item is clicked in the sub-list dropdown, a page with the items appear where the data-label (Normal/Heroic) is a header and the items following are anchored to this label.

[Xinhuan]

If you really want to be more efficient and use less memory, then you want to split your data into 2 or more separate tables.

That is if you have a structure that is

local t = {
  [1] = {label="abc", data="def"},
  [2] = {label="abcd", data="defg"},
  [3] = {label="abcde", data="defgh"},
}

Consider this instead:

local labels = {
  [1] = "abc",
  [2] = "abcd",
  [3] = "abcde",
}
local data = {
  [1] = "def",
  [2] = "defg",
  [3] = "defgh",
}

Not only do you have less tables, your code will often be faster as you don't have to access tables inside tables inside tables. The downside is that using this method of data storage will make sorting the data across the multiple tables more difficult and a custom sort function will be needed.

If the data is also manually compiled by hand, you will also have to make sure the data is synced across the multiple tables.

[Ailae]

Interesting. The data is manually gathered so using a structure like that feel like it would make maintenance more tedious, even if it's the most efficient. I'm not -that- worried about memory and stuff, at least not yet. With about half the items added and the browser more or less done, it still only uses roughly 80kb of memory (with all item-tables loaded directly and not only when used). I guess I just needed a sanity-check and some insights to good practices.

Thanks for all replies.

[Xinhuan]

Well, a table access is magnitudes more expensive than assignment or if-then-else statements.

Also, every new empty table is about 80 bytes of memory. While memory isn't the biggest concern, if you have A LOT of data, you should consider rearranging the data into separate tables. It will make your code maintenance easier (because your code will be easier to read without tables in tables) even if your data maintenance becomes harder.

[Ailae]

I understand that looking up a single level table is faster. But I don't have that much data, nor do I intend to, I'll basically just have Cataclysm stuff in it. I've reworked everything to the concept I outlined a few posts back, but gonna think about what you said regarding separating it into different tables. Just need a "clever" way to assemble it all together I guess.

In the long run I realize that the gain probably is so small (due to size of the data) that it doesn't make any sense to worry too much about it. But it's interesting never the less, really appreciate the feedback. Right now I'm mostly thinking of caching the item-strings so that if I choose to display those items again I can bypass the whole string.(g)match thingamajig. But in all probability I'm over-complicating it in relation to the size.

At the moment I have all the data I'm going to need (bar patches) for 5-man dungeons, reputations and Justice/Valor items (excluding tiers) and at load it's about 67kb with the browser (which I could probably clean up quite a bit). It's barely a drop in the ocean.

On a sort of related note, there's not really any point in assigning upvalues for variables used in for-loops is there? I think I've read somewhere that they get recycled instantly after the loop or something anyway.

[Xubera]

In the long run I realize that the gain probably is so small (due to size of the data) that it doesn't make any sense to worry too much about it. But it's interesting never the less, really appreciate the feedback.

A single grain of rice can tip the scale. One man may be the difference between victory and defeat.

If everyone made all their addons with no regard, then it may not make a difference when its only their running, but get enough running together and the garbage collector gets busy.

In all seriousness though, I don't know much about optimization or memory efficient methods, so I won't give advice I cant give. But you don't have to mention you plan on not caring about sloppy code in every post.

Still interested if what comes from this project of yours Like a building list of items, that continues to grow!

[SDPhantom]

Originally Posted by Ailae local t = {     ["Throne of the Tides"] = {         ["Order"] = "Lady Naz'jar:Commander Ulthok:Mindbender Ghur'sha:Ozumat:Trash",         ["Lady Naz'jar"] = {             ["Order"] = "Normal:Heroic",             ["Normal"] = "55202,55198,55195,55201,55203",             ["Heroic"] = "56267,56269,56268,56270,56266",         },         ...     ], }

It's usually a good idea to structure your data in a way that's the easiest to access without using too much CPU power.
I would advise something like this.

local t={
    ["Throne of the Tides"]={
        ["Normal"]={
--          Order List
            "Lady Naz'jar";
            "Commander Ulthok";
            "Mindbender Ghur'sha";
            "Ozumat";
            "Trash";
 
            ["Lady Naz'jar"]={55202,55198,55195,55201,55203};
            ["Commander Ulthok"]={};
            ["Mindbender Ghur'sha"]={};
            ["Ozumat"]={};
            ["Trash"]={};
        };
        ["Heroic"]={
--          Order List
            "Lady Naz'jar";
            "Commander Ulthok";
            "Mindbender Ghur'sha";
            "Ozumat";
            "Trash";
 
            ["Lady Naz'jar"]={56267,56269,56268,56270,56266};
            ["Commander Ulthok"]={};
            ["Mindbender Ghur'sha"]={};
            ["Ozumat"]={};
            ["Trash"]={};
        };
    };
    ...
};

The order lists will get incremental numerical indicies and can be used in a for loop using ipairs(). The item table can then be accessed using the value as the table's index.

[Xinhuan]

Don't use ipairs().

It is proven slower in every case compared to

for k = 1, #t do
    local v = t[k]
    -- Do stuff with k and v
end

because ipairs() is a function call, and every iteration calls an iterator function that is returned by ipairs() to increment the counter. So for a table of 200 in size, that's 201 unnecessary function calls.

[Ailae]

Originally Posted by Xubera A single grain of rice can tip the scale. One man may be the difference between victory and defeat. If everyone made all their addons with no regard, then it may not make a difference when its only their running, but get enough running together and the garbage collector gets busy. In all seriousness though, I don't know much about optimization or memory efficient methods, so I won't give advice I cant give. But you don't have to mention you plan on not caring about sloppy code in every post. Still interested if what comes from this project of yours Like a building list of items, that continues to grow!

I think you misunderstood me if you think I don't care about sloppy code. All I meant was that the difference would probably be negligible for this amount of data, so I can sacrifice some efficiency for more easily (to me) managed code. If I really didn't care I wouldn't be here asking questions, would I? I do this for fun in my spare time, but I try to do the best I can. I tried searching for any previous discussions on the topic but couldn't find anything substantial.

If you want to you can look at it here:
https://github.com/ailae/Shinies

But I guess it's more or less quite the mess still. Was my first try with dropdowns and scrolls and whatnot. But it works! :P

[Xubera]

oi oi, i hate the drop down menus too ><