General CombatLog Parser

[Miles]

Note: Total rewrite of the start post. Mod is uploaded to the Beta interface section, will update when it's available / posted.

Download

Complete documentation is on http://digibites.net/wiki/mcp/start. The following are excepts taken from it:

Miles Combat Parser

Miles Combat Parser is a generic parser module for combatlog events. The basics are to extract information from the text of events, to something that programs can easily understand. For example, "You hit Target for 400." will be translated to a "Melee" event, with { "You", "Target", 300 } as parameters.

The need for a generic parser

A lot of mods, but mostly recap, damagemeters and combatlogs, listen to such events. Currently, every one of those tries to match the text events into something they can understand. Except for the complication of parsing all the events (~1k lines for a nearly complete ruleset), this parsing also consumes quite a lot of cpu resources. Instead of everyone parses their own, I've decided to create a easy to understand and extend mod to do all this, so mods can concentrate on doing what they do the best, whether it is to display it, record stats or something else.

Design goals

Because of the nature of event processing, and other mods depending on it, I have set a few goals for MCP:

* Speed - Text processing is realtime, compared to waiting for stuff to load / be done, parsing has to be done *right now*, and without introducing lag or delays. Goal in this addon is to have a worst case delay of 0.2ms / 5000 msg/sec, an overhead of 10x above a plain string.find. Benchmark by replaying a Ragnaros and ZG fights shows the parser reaches 16k msg/sec - totally unnoticible in game, or 1 sec lag when you see the whole Ragnaros fight at once after a uber-lagspike :P
* Complete - I aim for a 100% completeness of all possible combatlog strings, including the exotic ones I never seen before, like "%s loses %d health for swimming in slime."
* Universal - It's *very* easy to write addons that listen to events fired by the parser, an example is posted for watching HoT's in SpellTimers. Integration with Recap, MilesCombatLog, and some other parsing-heavy mods will be added, hopefully starting a chain of events leading to exchangeble parsers who parses, and addons which processes the events, instead of regex everything everywhere.
* Standard compliant - Should there ever be a parsing output standard, it will be compliant with it.

Events in MCP

Events in MCP are catagorized into 11 main groups, namely:

Heal = "Heal", -- caster, target, spell, amount, crit, ishot
Aura = "Aura", -- target, aura, fade, harmful, replaced
Melee = "Melee", -- caster, target, amount, crit, school
Spell = "Spell", -- caster, target, spell, amount, crit, school, isdot
Cast = "Cast", -- caster, spell, begin
Dot = "Spell", -- caster, target, spell, amount, crit, school, isdot
Miss = "Miss", -- caster, target, spell, type (Caster Spell Target Resist, example Ragnaros Lava Burst You Resist)
Reflect = "Reflect", -- caster, target, damage, type
Interrupt = "Interrupt", -- who, target, spell
Env = "Environment", -- who damage type
Death = "Death", -- who killer

It's represents the 11 basic types of messages you can get in WoW.

Listening to Events

Subscribing to MCP is quite simple, for example, if you want to print the "Miss" type messages in ChatFrame1, you can do it with a few lines.

function FormatMiss(v)
	-- caster, target, spell, type 
	-- (format: Caster Spell Target Resist, expample: Ragnaros Lava Burst You Resist)

	if (not v[3]) then v[3] = "Attacks" end
	
	ChatFrame1:AddMessage(string.format("%s %s %s %s", v[1], v[3], v[2], v[4]));
end

MilesCombatParser.addListener("Miss", "MissPrinter", FormatMiss);

When a "Miss" occures, your function will be called with the parameters, "caster, target, spell, type" in a table. Optionally, you can declare a second parameter, function bla(v, t) will be supplied the type of the event too.

--

This mod will enable much easier processing of combat information, enabling a whole new world of possibilities. Next thing on my todo is to integrate it further in MilesSpellTimers (not released yet), a mod used for starting timers at OnCast events, and for timing Healing over Time events with MCP.


Renew from Kupo on Kupo, with a 35 hp tick strength, 10 seconds left.


No more 'how many tick do I have on pain?' :P

[JoshBorke]

will it support data-syncing?

[Miles]

No, it's only a parser for combat logs.

Syncing of that is not recommended, as it may exceeds 300 lines per second in raids - flooding the server with that much data will get you kicked.

However, you are always welcome to write a mod to sync the totals like in damagemeters, or broadcast parts of the events only, maybe xxx died messages in battlegrounds + tloc.

[JoshBorke]

good point :-)

awesome mod btw.

[Miles]

Little snippet of code that helps @ AQ20 final boss:

         MCP.addListener("Aura", "MST-AuraWatch", function(v)
            -- param order: target, aura, fade, harmful, replaced
            if (v[1] == "Ossirian the Unscarred" and not v[3] and string.find(v[2], "Weak", 1, true)) then
               MSpellTimers.Obj:AddQueue({ timeout = 45, color = "ffff4040", name = v[2]}, v[1]);
               SendChatMessage("Ossirian gained weakness: " .. v[2], "RAID");
            end
            if (v[1] == "Ossirian the Unscarred" and v[3] and string.find(v[2], "Weak", 1, true)) then
               SendChatMessage(" ** Ossirian lost weakness: << " .. v[2] .. " >> **", "RAID");
            end
         end);

Modify MSpellTimers line to whatever you wish, like starting a 45s countdown on CT Timer mod, as that isn't available (or quite finished) yet.

[Thaistalyn]

Miles thanks for the addon!

Just a quick general questions:

I will probably use this addon on to "detect" certain mob calls and write CTBoss Mod addons. Since most of the new bosses do not have documented abilites I want to be able to parse my combat log for their abilites and the events that trigger the ability. (I'm trying to find the event for "Spawn of Fankriss")

Your example of Ossiran is a good one. You knew what text to search for/on based on observation of the log I'm guessing?

My example is Buru the Gorger:

When he targets a player he emotes "Buru the Gorger sets eyes on <target>!" In this case I guessed CHAT_MSG_MONSTER_EMOTE as the ONEVENT trigger for my logic to run.

So, after all that my question to you is : Does your addon report the trigger event as well as the event variables?

Using your parser it may be easier for me to guess the event category and create a listener function for future boss alerts. Where would I add this listener? Would it be easier to create my own custom addon or add it to your MCParser.lua

[Miles]

Hmm... calling on emote is a common one for bosses.

The easiest way to get it into the parser:
Register the parser to watch for the CHAT_MSG_MONSTER_EMOTE too.
Add CHAT_MSG_MONSTER_EMOTE on a 'e' branch on the main tree, with the rule r = "%s"; o = { 1 }; t = "Emote"

After that, the parser will also fire Emote messages, and listeners will be notified with the text of the emote.

--
I can't promise a time, but I'll get that in the first release, as it's quite a useful addition for bossmods. Watching the emote and yelling out on ventrillo sucks at Buru, wiped us twice last night because someone was sleeping and we lost too many people for the 'nuuuuuuuuke' phase

Good thing that the Ossirian is under control now, saves a lot of gold and frustration - got him on the second time last raid, with the first being a "where the hell is the f*cking crystal" wipe.

--
Ossirian: yeah, I noticed that he gains a "Arcane" / "Fire" / etc .. " Weakness" debuff, and I wanted to time that damn thing - that boss is a really tricky one when you can only guess. So I added a aura event listener, filtered on name. First try was without text filter for weakness so it started a timer for every debuff , but with that and the raid spam + chat timestamp I pinpointed the duration and finalized the thing.

Usually, bossmods watches for emotes, or some other events (life drain debuff @ hakkar), and announces the raid or schedules a timer/event. Another useful event is the 'blabla begins to cast x' / 'blabla casts x' event, in the "Cast" event group - the crystals @ Ossirian does that too, but it's a bit overkill to watch them both.

[rophy]

Hi, a universal combat log parser is a great idea.

I'm making a CombatStats-like addon, which can be found at http://ui.worldofwar.net/ui.php?id=2069

Currently I'm trying to implement cross-language support on my addon, so I think you might be interested on my current process.

Bascially I used the pattern converter of Recap, and improved it so that it actually works on all languages.

So far I've tested my addon on english, german, french, tradition chinese, and it seems to be working well.

I think you might be interested on the information about cross-language support so I posted here, feel free to email me at [email protected], I'm very interested to help on developing a universal combat log parser.

[Miles]

The parser itself is language insensitive, but the ruleset is - for performance reasons, I don't try every single of the MISS messags (absorb, block, deflect, miss, evade, resist, parry), but do a substring test first, an operation that clocks 2M ops/s here, instead of ~150k op/s of string.find without the plain flag. It's still HEALEDCRITOTHEROTHER and other GlobalStrings based, but the text filters killed the other locale support.

When someone using another locale wants to add support for other locales, they can - but for now, it's english only. I will be releasing a updated ruleset, but with lower performance, when I have the time. What I won't do is localize it myself, as that requires a lot of testing and verification. Afaik, only the french locale bugs on the international ruleset, because the DOT's ordering is different.

The parser itself is flexible enough to parse even WTS / WTB messages, by add a rule and registering the event - the order, amount of parameters to capture, event type, everything is customizable.

[rophy]

hmm... from what I saw in your source code, the "rule set" is something like:

%s's %s hits %s for %d %s damage -> src=1, dst=3, atk=2, dmg=4, 5=sch

If this was what you meant, then we're talking about the same thing,

Basically you don't need to localize the source, destination, attack etc at all,
you only need to parse the GlobalStrings.lua patterns and convert them when the game is loaded, since the conversion is once only, it won't hit the performance.

So from my knowledges you don't need to localize rulesets for every languages, just convert them.