Trying to make addon to export WoW Statistics

[doublebrutal]

I currently manually input all my Achievements > Statistics into spreadsheets once every few months and doing it on multiple characters is a massive chore.

I have programming background in vb6, vb.net, java, html, css. I followed the simple starter guides to create an addon and launch a popup window in game.

Then I hit a wall. My gut feeling is WoW has broken, or no API for what I am trying to do. I would love to be proven wrong though!

I launched wow with -console and exported their UI code. I searched through the BlizzardInterfaceCode (specifically the files Blizzard_AchievementUI and Blizzard_APIDocumentation) to no avail.

If you are in game and press Y to bring up Achievements, then click on Statistics at the bottom right, this is the information I would love to export as Comma Seperated Values, or be able to work with the information within the addon to display account wide statistics (aware this would require logging into each alt at least once).

I mainly started this venture as I wanted to consolidate all my battleground and arena statistics to see my win rates per character, my kill death ratio per character & overall. I currently do this by manually inputting every single statistics from the Player vs PLayer section into a spread sheet, which then runs a tonne of formulas and graphs etc to show me that my hunter is 3 kills per death, my dk is 1 kill per death, my warrior is my top flag carrier, etc...

I even tried some getStatistic API stuff within game and noticed this seems to give a STRING value such as false593 with 593 being the value of the specific statistic i found on an api website (but that website is out of date, it still shows the api calls for the things in the Statistics>Character>Combat that got remove a couple patches ago).

Having hit deadends with looking at WoW's code and not finding an addon to do this, as a last resort I figured i'd come ask some experts:

Can this be done? I am willing to put in some work to make the addon but with no working API and no real lua experience im very stuck.

[Fizzlemizz]

See the GamePedia page on SavedVariables and Saving variables between game sessions

This is the ONLY method/format/place(s) addons can read/write data to the disk. The data is read only once when a character logs in and written only once when the character logs out/reloads.

[doublebrutal]

I had a read, that stuff wasn't much help at this point in time - I need to find a way to read that very tidy table of Statistics in the achievement pane.

One thing I just played with was screenshotting a chunk of the Statistics panel in game, then cropped the statistics table into a jpg and saved it. I then went to a website to convert JPG to .xlsx and it correctly translated the picture of the table from in game into a spreadsheet. This doesn't help with an addon, and is still a very manual process and quite time consuming (the conversion is slow).

It baffles me that a program can translate a jpg image of a table into a spreadsheet, but we can't simply export a clean table of data from within the game... :\

My input to website converter:


The result:


Also an example of an addon with data exporting is Reflex - It records all bgs and arenas played.. then lets you export the data as comma seperated values. This has been great for shadowlands season 1 and season 2 for exporting pvp data to spreadsheet to work with... but obviously is limited to the time frame it's been installed, and limited to pvp.

What I want to achieve is far simpler... a dump of the Statistics in the Achievement>Statistics section - pvp at a minimum, but all of it because why not the process will be the same.

[Fizzlemizz]

Originally Posted by doublebrutal It baffles me that a program can translate a jpg image of a table into a spreadsheet, but we can't simply export a clean table of data from within the game... :\

Because, complicated

A very basic addon to print the raw stats when you first open the achievement addon

local f = CreateFrame("Frame")
f:RegisterEvent("ADDON_LOADED")
f:SetScript("OnEvent", function(self, event, ...)
    local addonName = ...
    if addonName == "Blizzard_AchievementUI" then
        self:UnregisterAllEvents()
        local categories = STAT_FUNCTIONS.categoryAccessor()
        for _, cat in ipairs(categories) do
            local catName = GetCategoryInfo(cat)
            local total, completed, incompleted = GetCategoryNumAchievements(cat)
            print(catName, "total", total, "completed", completed, "incompleted", incompleted)
            for i=1, total do
                local quantity, skip, id = GetStatistic(cat, i)
                if not skip then
                    print(GetAchievementInfo(cat, i))
                end
            end
        end
 
    end
end)

you can paste into this website to download as an addon to install.

You can get the information on functions like GetAchievementInfo and more at GamePedia.

Using the information about SavedVariables, you could save then information to disk as a table instead of using print(). From there you can start refining or extending the saved information to achieve the desired result (a result you can read from lua table format to CSV or whatever), but there is no easy "GiveMeWhatIWant" function.

[Fizzlemizz]

I guess the other alternative would be to do a "scroll 'n scrape" of the actual stat rows. The entries in the "value" key will be formatted for display (like icons for gold/silver/copper etc.) so they will need to be broken down for "saving" but something like:

As above, the print() would need to be converted to a Save process.

SLASH_doublebrutalStats1 = "/dbs"
local achievs = {}
SlashCmdList.doublebrutalStats = function(msg)
    if not AchievementFrame or not AchievementFrameStatsContainer:IsVisible() then
        print("Can't see Achievement Frame!")
        return
    end
    if strupper(msg) == "CLR" then
        wipe(achievs)
        return
    end
    if strupper(msg) == "PRT" then
        for k, v in pairs(achievs) do
            print(k, v.text, v.value)
        end
        return
    end
    local count = 0
    while true do
        count = count + 1
        local button = _G["AchievementFrameStatsContainerButton"..count]
        if not button then break end
        if button.text:GetText() then
            if achievs[button.id] then
                achievs[button.id].text = button.text:GetText()
                achievs[button.id].value = button.value:GetText()
            else
                achievs[button.id] = { text=button.text:GetText(), value=button.value:GetText() }
            end
        end
    end
end

/dbs <scroll> /dbs to scrape the infomration from the list
/dbs prt to print the scraped information
/dbs clr to clear the scraped information

[SDPhantom]

Originally Posted by Fizzlemizz Originally Posted by doublebrutal It baffles me that a program can translate a jpg image of a table into a spreadsheet, but we can't simply export a clean table of data from within the game... :\ Because, complicated

It's not that the process itself is complicated, but the reason behind it. Ultimately, Blizzard doesn't want to allow us direct access to the user's filesystem. There are many reasons for this ranging from user security to bot prevention.

Lua itself was designed to be an embedded language, meaning it only has the bare minimum in terms of built-in functions. It's up to the host program to provide the API to expand its functionality. Similarly, the host's developer can decide to remove any of Lua's built-in functions. Many of which can be extremely dangerous on an end user's system if 3rd party scripts (such as addons) are allowed.



As mentioned previously, the only way to save and load data off disk is to use the SavedVar system.

Another method is to show a multi-line EditBox with pre-formatted text the user can copy/paste. Note Ctrl-C/Ctrl-V does work in WoW for this.

[doublebrutal]

Hey guys, thanks for those 2 ideas, going to go test them now!

[doublebrutal]

So the first code output a messy version of what I am after into the chatbox in game, but is far bigger than i can scroll through in the chatbox to copy/paste. I am currently trying to make it output to savedvariables but not really getting anywhere. When I open my savedvariable for my addon in the WTF folder, it is a single line of code:

EXPORTSTATSDB = nil

An example of the results returned from that code by fizzle is:

7399 Challenge mode dungeons completed 0 false nil nil nil 1 136243 false false nil true

I'm guessing it's hitting one of these nils and just stopping instead of outputting to savedvariables. Would it be easier to have the output come up in a window like /simc or /amr for a simple copy paste export?

I tried using message instead of print but it only outputs the very first value of 7399 in the message window.

[doublebrutal]

I also tried the second suggestion of scraping the statistics panel - this is a tidier output format but unfortunately doesn't behave as expected. I opened the Player vs Player Rated Arenas section. Typed /dbs. Scrolled to bottom of that section. Typed /dbs prt. This output a mash of the arena section and other sections as well into chat. It also is too long for chat box for any kind of copy pasting to be useful.

[doublebrutal]

Just tried another approach using the addon CopyAnything

This sadly only lets me point at a single cell of a table and copy the text, so one by one, the name of each stat, then the value of each stat, much much worse than even my old way of manual data entry xD

[Fizzlemizz]

The first one using Saved Variables (assuming EXPORTSTATSDB as the SV declared in the .toc) might look something like (saves the data under each character name/realm sub key) (it's just a crude example so treat accoringly):

You have to open the achievements frame for it to work.

local f = CreateFrame("Frame")
f:RegisterEvent("ADDON_LOADED")
f:SetScript("OnEvent", function(self, event, ...)
    local addonName = ...
    if addonName == "Blizzard_AchievementUI" then
        self:UnregisterAllEvents()
        EXPORTSTATSDB  = EXPORTSTATSDB or {} -- create SV table iuf required
        local player = UnitName("player") -- create sub-table for current polayer info
        player = player.."-"..GetRealmName()
        EXPORTSTATSDB[player] = EXPORTSTATSDB[player] or {}
        local SVKey = EXPORTSTATSDB[player]
        local categories = STAT_FUNCTIONS.categoryAccessor()
        wipe(SVKey) -- clear the tasble tyo update with latest ionformation
        for catid, cat in ipairs(categories) do
            local catName = GetCategoryInfo(cat)
            local total, completed, incompleted = GetCategoryNumAchievements(cat)
            SVKey[catName] = {} -- create category sub-sub table
            SVKey[catName].total = total
            SVKey[catName].completed =completed
            SVKey[catName].incompleted = incompleted
            SVKey[catName].Stats = {} -- create sub-sub-sub table for category stats
            local Stats = SVKey[catName].Stats
            for i=1, total do
                local quantity, skip, cid = GetStatistic(cat, i)
                if not skip then
                    local id, name, points, completed, month, day, year, description, flags, icon, rewardText, isGuild, wasEarnedByMe, earnedBy, isStatistic = GetAchievementInfo(cat, i)
                    Stats[id] = {}
                    local Stat = Stats[id]
                    Stat.name = name
                    Stat.points = points
                    Stat.completed = completed
                    Stat.month = month
                    Stat.day = day
                    Stat.year = year
                    Stat.description = description
                    Stat.flags = flags
                    Stat.icon = icon
                    Stat.rewardText = rewardText
                    Stat.isGuild = isGuild
                    Stat.wasEarnedByMe = wasEarnedByMe
                    Stat.earnedBy = earnedBy
                    Stat.quantity = quantity
                    Stat.isStatistic = isStatistic
                    local numCriteria = GetAchievementNumCriteria(id)
                    if numCriteria > 0 then
                        Stat.criteria = {}
                        Stat.numCriteria = numCriteria
                        for c = 1, numCriteria do
                            local criteriaString, criteriaType, completed, quantityNumber, reqQuantity, charName, flags, assetID, quantity = GetAchievementCriteriaInfo(id, c)
                            tinsert(Stat.criteria, {
                                criteriaString=criteriaString,
                                criteriaType=criteriaType,
                                completed=completed,
                                quantityNumber=quantityNumber,
                                reqQuantity=reqQuantity,
                                charName=charName,
                                flags=flags,
                                assetID=assetID,
                                quantity=quantity,
                            })
                        end
                    end
                end
            end
        end
    end
end)

[doublebrutal]

OK that SavedVariables output is a thing of beauty I can definitely work with this!!! Thank you so much for the help!!!

UPDATE: Never mind, I think I will just have to data entry, even this beautifully formatted SavedVariables file didn't save a single value for any of the statistics ):

Example output:

[14785] = {
["rewardText"] = "",
["wasEarnedByMe"] = false,
["isGuild"] = false,
["flags"] = 1,
["description"] = "",
["name"] = "Total deaths in PvP arenas",
["completed"] = false,
["points"] = 0,
["icon"] = 136243,
["isStatistic"] = true,

It only gives the true or false of the statistic - a problem i encountered a few weeks ago when i first tried making scripts/macros using getStatistic()

[Fizzlemizz]

Don't be too quick with the thanks, it's by no means complete as each stat can have different value sets and this doesn't get those .

The information should be buried somwehere in the Achivement UI code.

[doublebrutal]

https://www.wowhead.com/forums/topic...s-macro-248202

I found this resource just now which is much closer to what I am trying to do

UPDATE: Using this method, I created my own list of PvP achievement IDs, which 'printed' to chat log at 12.29pm.. which i thought would be saved in ElvUI chatlog at 12.29pm.. nope, it only saved player chat stuff and the achi stuff was skipped ):

[doublebrutal]

Originally Posted by Fizzlemizz Don't be too quick with the thanks, it's by no means complete as each stat can have different value sets and this doesn't get those . The information should be buried somwehere in the Achivement UI code.

Yeah it looked so clean at first but as soon as i went to use it I realised it didn't catch a single value of any statistic ):

With all the crazy websites like DataForAzeroth i thought BLizzard's API was gonna be way better than this, was not expecting the statistics panel to not have API calls.

[Fizzlemizz]

Originally Posted by doublebrutal i thought BLizzard's API was gonna be way better than this...

Define better.

Updated the previous code to add some numbers. I suggest you run it, /reload and check it against what you see in the in-game stats. for that character.
Probably better to delete the SavedVariables files(s) before starting the game so you only have one characters worth of information.

[elcius]

I think this is something you should be doing with the web api instead of in-game
https://develop.battle.net/documenta...t/profile-apis

[doublebrutal]

Things like Duels won and Duels lost are correct, along with Total deaths from players. Gonna spend some time now figuring out what all the incorrect value have in common.

[doublebrutal]

Oooh never mind I see it's somehow combining oddly seperated data to one correct value:

[105] = {
["rewardText"] = "",
["numCriteria"] = 2,
["criteria"] = {
{
["criteriaString"] = "Warsong Gulch Victories",
["charName"] = "Doubleßrutal",
["quantityNumber"] = 186,
["completed"] = true,
["quantity"] = "186",
["reqQuantity"] = 1,
["assetID"] = 489,
["criteriaType"] = 1,
["flags"] = 0,
}, -- [1]
{
["criteriaString"] = "Warsong Gulch Victories",
["charName"] = "Doubleßrutal",
["quantityNumber"] = 41,
["completed"] = true,
["quantity"] = "41",
["reqQuantity"] = 1,
["assetID"] = 2106,
["criteriaType"] = 1,
["flags"] = 0,
}, -- [2]
},
["wasEarnedByMe"] = false,
["isGuild"] = false,
["description"] = "Warsong Gulch victories",
["flags"] = 1,
["quantity"] = "227",
["name"] = "Warsong Gulch victories",
["completed"] = false,
["points"] = 0,
["icon"] = 136243,
["isStatistic"] = true,

Now I am reading it properly it seems like you have done it!

The 227 is correct, and the same pattern of output actually worked for Warsong Gulch battles too!

Good job on getting through Blizzard's Spaghetti code on this one!

[doublebrutal]

So to follow up on one thing that works for now, I dumped all my toons into the SavedVariables file. I then copied it's entirety to Excel. I then made 2 copies of a table of PvP statistics, with a column for each character.

In one copy of this table, I linked each stat to it's value in the raw paste data (this way I could easily select the correct value of the 3 in situations like warsong gulch above. The other copy of the same table simply linked to this table (=SHEET3!B3245) for example.

This way I have a permanent list of all the stats in raw format which for example:

                Char name                  Char name                      etc...
Arenas played   ["quantity"] = "8582",     ["quantity"] = "1328",

I then wrote a VBScript to remove non numeric characters and put it on a button on the copied sheet.

When I paste the data in sheet 3, sheet 2 pulls the values I want for each character accurately, then sheet 1 is literally just linked to sheet 2 cells so it updates identically. Then on sheet 1 I push the button, and it cleans the whole sheet up to show:

                   Char name            Char name          etc...
Arenas played      8582                 1328

Now this will work for me personally for the time being but it is far from future proof.

In patch 9.2 when new arenas are added, the links between sheet 2 and the raw data will no longer be accurate, as there will be more lines of code and it will all go out of whack until I manually rejoin the data, but this shouldn't happen super often so I think I can live with it.

I spent some time thinking how to turn the savedvariable data into usable info for an addon but since things like Warsong gulch has 3 values called "[quantity=""] trying to search the strings and pick the max of the 3 seemed like a nightmare.

Thanks again for the help at least now for personal use I can spend the next 2 weeks in excel nerding out over 17 years of WoW PvP data