Wowhead Parsing with Python
This is just a little Python script I whipped up to parse out data from Wowhead pages. Originally it was just for use with Daily To-Do's, gathering quest data, but I thought I may as well share it.
It parses out the Json embedded in Wowhead pages' source, and turns it into nice Lua tables for use in addons or whatever else.
Code:
import json, urllib, re
from types import *
encoder = json.JSONEncoder()
url = 'http://www.wowhead.com/achievements'
p = urllib.urlopen(url)
p = p.read()
json_re = re.compile("new Listview\({template: '\w*', id: '\w*',.*data: (\[.*\])")
match = json_re.search(p)
f = open("parse.lua","w")
if match and f:
data = match.group(1)
data = data.strip()
# parse with json
datadic = json.loads(data)
f.write("parseTable = {\n")
for category in datadic:
f.write("{\n")
for key in category:
if type(category[key]) is IntType:
f.write("\t{0} = {1},\n".format(key,category[key]))
elif type(category[key]) is ListType:
f.write("\t{0} = {1},\n".format(key,str(category[key]).replace("[","{").replace("]","}")))
else:
f.write("\t{0} = \"{1}\",\n".format(key,category[key].replace("\"","\\\"")))
f.write("},\n")
f.write("}")
f.closed
print "Complete and written to 'parse.lua'!"
print "Output:"
f = open("parse.lua","r")
print f.read()
f.closed
else:
print "Error"
You can set the url variable to any Wowhead page that has Json embedded into it and it should magically come out into a nice Lua table you can use in your addons or whatever else you might need it for.
Not sure why I chose to use Python, seeing as prior to this script I had neither used or really seen it before, but whatever.
I'm sure this script is a horrible excuse for programming, but it works I guess.
Hopefully someone might find it useful.
Feel free to leave feedback or something.
NOTE: Wowhead imposes an artificial limit of 200 results, so not all results will be shown if there is more than 200. |
