Thread Tools Display Modes
Prev Previous Post   Next Post Next
10-25-12, 09:41 AM   #1
myrroddin
A Pyroguard Emberseer
 
myrroddin's Avatar
AddOn Author - Click to view addons
Join Date: Oct 2008
Posts: 1,240
website scraping help please?

I was talking to the staff at wowdb.com, and they gave permission for me to scrape their mailbox data. wowhead.com does not track mailboxes, if people wondered.

My problem is that I have no idea how to scrape the data because I have never done any scraping. I looked at DataTools on Curse, and I got headaches.

If someone wants to teach me, I would like to learn how DT makes an addon, populates it with data from wowhead or wowdb or the game's files, etc. On the other hand, if someone wants to present me with "here you go", I'll take that happily as well. The format I want the data is something like the following example, as I will need to make corrections like mailboxes in the wrong location, put icons on the correct floors, etc.

The LocationMapper line for the mailbox link above starts on line 7067, and I understand it doesn't follow the table format I need. If I could even break it down into something readable and then copy/paste, that's fine too.

Lua Code:
  1. -- HandyNotes_PostService.lua
  2. local HPS = LibStub("AceAddon-3.0"):NewAddon("HandyNotes_PostService")
  3.  
  4. do
  5.     function HPS:ParseData()
  6.         local mailboxes = HPS:Data()
  7.         -- blah blah
  8.     end
  9. end
  10.  
  11. function HPS:OnIntialize()
  12.     -- blah
  13. end
  14.  
  15. -- PostService_Mailboxes.lua
  16. local HPS = LibStub("AceAddon-3.0"):GetAddon("HandyNotes_PostService")
  17.  
  18. function HPS:Data()
  19.     local mailboxes = {
  20.         ["StormwindCity"] = { -- Astrolabe's mapFile name, but it could be Blizzard's mapID and I can convert
  21.             [0] = "1|62207440|" -- [mapFloor] = "factionNum|coord|factionNum|coord|" where Alliance = 1, Horde = 2, Other = 3
  22.         }
  23.     }
  24.     return mailboxes
  25. end
  Reply With Quote
 

WoWInterface » Developer Discussions » General Authoring Discussion » website scraping help please?


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off