How to subdivide a large addon?

[acapela]

i am the author/maintainer of the Aloft nameplate addon (http://www.wowinterface.com/download...o.php?id=10864).

some time back, my users asked if there was a way to slice Aloft up into a suite of separate addons that they could then selectively disable (and recover runtime memory in the process).

so, for the past 10 days or so, i have been trying to accomodate that. i am up to about 15 separate addons in the Aloft suite (the parent addon, and 14 children, each with its own specific job to do). it looks like i will end up with about 25-30 such addons by the time i am done.

now my users are complaining about "too many modules" .

one of them suggested looking at BigWigs, claiming that it offered some sort of "after-market" mechanism for bundling "plugins" together into a single directory, or separating them into individual addons, at the user's discretion. but in looking at the most recent version of BigWigs, it looks like a plain old modular addon to me (several folders, one per addon, each with a TOC and LUA, the embeds bundled with the "parent", tied together with dependencies specified in the TOCs). no batch file tooling in sight.

as an alternative, i could offer each of my modules as a separately versioned/released artifact, but combined with the lack of a viable automated updater (WAU is dead, i am a Vista user and manual intervention is required for use of the WoWInterface updater, and the Curse updater has never worked for me), this could result in quite a burden on the user (worse than just a delete/copy/paste on a set of 30 folders).

i want to be able to manage a set of 30 very closely related addons, such that they can be released/distributed in a single archive, operated on as file-system artifacts as a group (even if some custom "batch file" tooling is required), but still allow the user to enable/disable them as separate WoW addons. i would like to be able to do this without having to get into binary installers and the like (i have always felt that that is what WAU and similar applications are for, but we don't seem to have a viable one of those available at the moment).

can anyone suggest a solution? some addon out there that has solved this problem?

thanks.

[Seerah]

I agree that having 30 separate addon modules in the addons directory is too many. But look at how Prat does it. Prat has its modules all bundled together, but they are still each LoD.

[acapela]

Originally Posted by Seerah I agree that having 30 separate addon modules in the addons directory is too many. But look at how Prat does it. Prat has its modules all bundled together, but they are still each LoD.

yes, thanks for the pointer. looks like that could do the trick. i will start dissecting it and see how much new code it will require.

can anyone suggest any other methods (i.e., besides the Prat approach)?

[Psoewish]

Originally Posted by acapela can anyone suggest any other methods (i.e., besides the Prat approach)?

Even if I would know another method, I wouldn't suggest it!
I personally love the Prat approach as it's the cleanest of them all imo, so I suggest you just go with that.

I hate it when an addon suddenly makes 20 folders in my addon directory

[slizen]

I don't know how Prat handles it's module system, but isn't it impossible to make a module LoD without giving it an own toc?
Doing it without it's own toc and folder the game would still have to load the file, and then decide whether to run it or not.

[acapela]

Originally Posted by slizen I don't know how Prat handles it's module system, but isn't it impossible to make a module LoD without giving it an own toc? Doing it without it's own toc and folder the game would still have to load the file, and then decide whether to run it or not.

yeah, i am still going around with myself on this. i just don't have enough experience, i am afraid.

this LOD without separate TOC effect is what Prat apparently does. and indeed, playing with disabling modules using Prat, it looks like a substantial memory savings results. i am not clear whether this is data structure memory that is not being used, or if in fact the LUA itself is not being loaded when a module is disabled. i am still groping my way towards comprehension of this mechanism :-).

so, i just need to keep playing with it.

[Taffu]

SLDataText uses the Ace3 module system in this way, where all the modules are LoD through the module system but they do not require a TOC and load through the core. Quite a few Ace3-compatible AddOns do this as well, such as Prat and Mapster. Take a look at any of those AddOns and how they impliment the module system provided by Ace3 to create the modules to load with the core and LoD in-game.

[Psoewish]

Another thing you could try is contact the author of a modular addon like Prat, and ask for a few pointers on how to make your addon work in the same fashion.

I'm sure they'd be happy to give you those.

[acapela]

Originally Posted by Taffu SLDataText uses the Ace3 module system in this way, where all the modules are LoD through the module system but they do not require a TOC and load through the core. Quite a few Ace3-compatible AddOns do this as well, such as Prat and Mapster. Take a look at any of those AddOns and how they impliment the module system provided by Ace3 to create the modules to load with the core and LoD in-game.

alas, Aloft is an Ace2 addon. porting to Ace3 as a prerequisite to this is something i am prepared to do, but that will just add a lot of time/effort to the whole thing. i would need to know if Ace3 is a basic requirement to achieving this, so i can warn my users about it.

if this can be done with Ace2 (and i am hoping to find an Ace2 addon that made an attempt at this, perhaps even Prat 2.0), then that would be a quicker/easier way to deliver this resource mitigation requirement to my users.

ultimately, i would like the resource mitigation issue to be decoupled from the Ace3 issue, so that each can be pursued separately.

[Slakah]

Prat works by encapsulating the module in a function call, the same thing is done by many mods using the Blizzard Interface Options menu by on executing the code when the options are shown.

So for example you could do this in your main addon:

local modules = {}
function Aloft:AddModule(name, func)
	modules[name] = func
end

function Aloft:LoadModule(name)
	modules[name]()
end

and for your module

function Aloft:AddModule(<module name>, function()
	
--Module code here

end)

to my knowledge, thats a very simplified version of what Prat does.

Also looking at Mapster's and SLDataText's code the modules are executed all the time, unless I'm missing something of course.

[acapela]

Originally Posted by Slakah Prat works by encapsulating the module in a function call, the same thing is done by many mods using the Blizzard Interface Options menu by on executing the code when the options are shown.

yeah, i could see that much from inspection. i don't know if there is anything more involved, i have not rummaged the rest of Prat 3.0 and looked at the rest of the module-load workflow.

as far as this practice of "encapsulating modules in functions" goes, what i don't understand (due to lack of experience with LUA) is whether/how this saves memory, and if so, why or how much.

so:

a) i don't know how this Prat-style variation on "conditional compilation" works in WoW Client LUA environment
b) assuming all LUA is compiled and resides in memory, whether ever used or not, i don't know how much overhead passive "compiled" LUA represents (i.e. LUA that is never "loaded"/executed) compared to "active" LUA (i.e. LUA that ends up being "loaded"/executed)

if "passive" LUA ("included" via TOC/XML but encapsulated in a function that is never invoked) has a hefty footprint (simply from being scanned and compiled at startup), then the only way to give my users what they want (reduced latent overhead from functionality they do not use) is to give them what they don't want (lots of atomic modules packaged as separate addons that they can physically disable). a quandary.

my sense from disabling Prat modules is that "passive" LUA ("included" via TOC/XML and "visible" at compile time, but never executed and therefore somehow not "visible" at runtime) has a fairly small footprint.

i don't understand why, of course. i want to dig around on the web (or see if someone can walk me through it here in this thread) and understand the underlying physical mechanism a little better, before i settle on a specific approach and start fooling with my codebase.

[break19]

Originally Posted by acapela -snip- as far as this practice of "encapsulating modules in functions" goes, what i don't understand (due to lack of experience with LUA) is whether/how this saves memory, and if so, why or how much. -snip-

Actually that's a moot point, since unless functions are called/events are processed there is minimal memory footprint, and basically no cpu usage. So, the only thing you're saving by making module lua files is static memory.

The way sylvanaar explained his prat2/3 approach when he designed it was, there's one file with a bunch of "load this module" functions based on whether or not they're supposed to be loaded. If they are supposed to be, then that code then loads the module's lua file. If not, then the only resident memory is caused by the "module loader" lua file, since it never loads that particular module.

In someways it is a bit of a tradeoff, since having separate addons will save perhaps 10k-20k of static memory usage. But seriously, is 10-20k really worth the extra aggravation to users?

break19

[acapela]

Originally Posted by break19 Actually that's a moot point, since unless functions are called/events are processed there is minimal memory footprint, and basically no cpu usage.

yes! thank you! this seems to be the takeaway point here.

however it does it, LUA as a language system doesn't seem to use much in the way of static resources just holding compiled code.

so, now i need to figure out whether anything about Ace2 precludes this. looks like Prat-2.0 is an old-fashioned static LUA addon. i will just experiment, see if i can get a piece of existing Aloft code to operate this way.

[Seerah]

Just update it to Ace3 while you're rewriting it into modules - it will save you work doing it all at once. Ace2 is pretty much on life support now.

[acapela]

Originally Posted by Seerah Just update it to Ace3 while you're rewriting it into modules - it will save you work doing it all at once. Ace2 is pretty much on life support now.

tried porting to Ace3 already... it was very invasive, and i ended up with an addon that loaded alright (no LUA errors) but had an endless list of defects all over the place that would have taken me a lot of time to weed through.

i want to a) basically do one thing at once, and b) have some more experience before i try to tackle Ace3 again. yes, it will happen, just not as part of a refactoring that has another purpose. a matter of priorities here.

[Psoewish]

Wouldn't it be easier in the end to do the converting to ACE3 first, and then tackle the modular loading problem?

Well I don't know how this stuff works, but to me it sounds like converting to ACE3 is a huge change. So if you first make it modular, and then convert to ACE3, wouldn't you have to weed through the problems of modular loading again?

I think it would be easier in the end then if you would make it ACE3 first, and then tackle the modular problem straight in the ACE3 thingy.

That's just what I think it's like though, I could be incredibly wrong :P

[acapela]

Originally Posted by Psoewish Wouldn't it be easier in the end to do the converting to ACE3 first, and then tackle the modular loading problem? Well I don't know how this stuff works, but to me it sounds like converting to ACE3 is a huge change. So if you first make it modular, and then convert to ACE3, wouldn't you have to weed through the problems of modular loading again? I think it would be easier in the end then if you would make it ACE3 first, and then tackle the modular problem straight in the ACE3 thingy. That's just what I think it's like though, I could be incredibly wrong :P

in theory i would agree with you, and it could be in the end that the direction i am headed could result in more work.

i could, in fact, just branch everything several times and work on multiple things simultaneously... but i have done things that way before (not by choice), and IMO it causes more problems and eats up more time than it is worth.

i could also tell you all sorts of war-stories about teams that went ahead and jumped into a comprehensive/monolithic refactoring of a legacy codebase instead of doing things incrementally, preserving testability the whole way through...

users report bugs along the way, and production releases need to continue to flow, and etc. yes, this is just an addon for a game, but i am just one person, no-one to delegate things to, and i can't escape a "production support" mentality about even something like this (too many years of indoctrination ).

so, that means one major refactoring at a time, done in priority order (and slicing up Aloft in some form is more important to the users that an Ace3 port)... unless it proves to be infeasible.

[acapela]

alright, i have a couple of Aloft's modules loading at runtime, Prat-style (encapsulated in functions).

there are some Aloft data lifecycle issues i have had to work through (but that was to be expected), and i don't have enough functionality in the aggregate loading this way yet to try to quantify how much memory is used before/after loading, but it is functioning.

Ace2 did not end up imposing any restrictions.

thanks so much to everyone who contributed here for all your help, i greatly appreciate it. i would have just been groping in the dark otherwise. if anyone thinks of anything else, feel free to chip in... i am sure i am doing any number of things wrong/ineffectively .

[Psoewish]

I'm glad to see it's all coming along. :]
After all this is one of my standard wow addons, so less memory is never a bad thing ^_^