Extract "wow-update-12759.mpq"... meh!
Trying to extract the DBC and LUA/XML files but getting weird patch-like content in the files. How does the new patch system work?
|
I've been trying to figure this out with no luck too. Apparently Tekkub found a way to extract build 12694, which uses the same patch format. I couldn't find any mention of what tools he used though. The actual patch files look like bsdiff binary diffs with a proprietary blizzard header prepended to them. I had trouble applying any of the patches though, even with the extra header trimmed out.
____ Edit: On second look at what Tekkub committed, he didn't extract patch 12694 properly. He has the changes between builds 12644 (pre-streaming client) and 12635 (base streaming client); None of the changes from wow-update-12694.mpq are included though. |
Yeah, I noticed that too. I've been opening update exes before (they are mpq too) and this file structure seems familiar, it's only hard to find out how we can now extract the up-to-date FrameXML folder when it appears that each new build will have it's own "patch" MPQ containing diff files and not the whole patched file, hmm...
Perhaps someone at Wowhead may help? They do extract DBC, BLP icons and similar and they are also subjected to the same difference patching system. |
I just used MPQ extractor.
You can find it on www.devklog.net The last time I tried it was a few weeks ago and got a folder worth of files extracted. Not sure if that's what you're looking for or if even if it extracts it right. Wait.... I might of tried this on a previous build. I'll try to remember to repeat it when I get home. |
Try it yourself Bluespacecow, use the MPQEditor, open "wow-update-12759.MPQ" and extract the "Interface/FrameXML" folder. Open up the files inside and be surprised. ;)
|
Quote:
I note that at http://www.zezula.net/en/mpq/download.html Is a beta version for extracting wow-update-12694.MPQ Is the problem extracting the files the problem ? Or the format they are in once extracted. Try the Beta Version at that site above and extract with that if you haven't already. Zezula lists that MPQ (wow-update-12694.MPQ) as a partial MPQ file so wow-update-12759.mpq may be as well |
Even with the beta version of that MPQEditor, it still extracts patch files. Its file properties dialog can tell you if a packed file is a patch, but that's it. I even tried using the multi-MPQ mode with build 12694 loaded over top of 12635. Still no luck though.
|
The problem is that like in this case, I want the latest FrameXML files. I naturally open "wow-update-12759.MPQ" as it's the update file downloaded with the launcher. Inside I see the various localizations and the "base" that is common. I open enUS/Interface/FrameXML/ and see several files. I open for example "ActionButton.lua" and see this (binary data):
Code:
PTCHB`@AMD5_({†œv"*aV*Ͷ0*0XFRMBSD0˜BˆBSDIFF40AAA„e *Edit* 1. Extracted "locale-enUS.MPQ" and it's FrameXML 2. Extracted "wow-update-12694.MPQ" and it's FrameXML 3. Extracted "wow-update-12759.MPQ" and it's FrameXML I downloaded bsdiff and bspatch (Binary diff/patch utility version 4.3) and when I try to patch a file from #1 by using a file from #2 I get "corrupt file", seems like they also added some custom data to their patching files, hmm... |
I think the BSDIFF40 part in the PTCH files is invalid.
For example the bsdiff part of ActionButton.lua from wow-update-12759.mpq says the resulting file would be 146789209322029069 Bytes long (thats 130 Petabyte) :rolleyes: As far as I understand the format the data in bsdiff files should be compressed with bzip2, but there's clear text in the .lua diffs. I didn't find any file format beginning with 'PTCH', so maybe that's a Blizzard invention. |
Borlox, that's what I also found out.
There is some clear text indeed, the data that is to be inserted/replaced, e.g. is all there. Problem is figuring out where to put, replace, append, delete, e.g. I tought wowhead or mmochamp figured this out as they do read DBC files right, so they need to figure out how to patch their DBC with the updates so they can for example scan Spell.DBC for changes. ;) Asking Wowhead mods on IRC atm, let's see if there is a way. |
Quote:
|
at least on windows the addonkit from blizzard still works but you need 100% downloaded/patched and need to close the laucher/wow before you start extracting
|
I think the Addon Kit is skipping the wow-update MPQs too. I can't find any of those changes in its output. As an easy test case, check for <FrameXML/Util.lua>, which should be there since patch 12694.
|
Addon Kit only extracted from my retail, not beta. Also I think it also only extracts from the locale file and not including the patches.
|
Found some (if not all) of the Interface files that were patched in this build in the Data\Cache\enUS (or whatever locale you use) directory in the build's patch MPQ. From what I can tell, though, any totally new files aren't in there at all. The new spell flyout files aren't in there, for example, but are in another MPQ in text form rather than as a patch (since they're new).
Looks like it may take a bit more work than before to extract things that can be committed to a revision control repo or diffed before a new kit is released that can cope with the new scheme. Nothing a little scripting can't handle, but it's still a change to adapt to. |
Adys from MMO-Champion and Ladik the author for MPQ Editor (not sure of the nick entirely) to integrate this feature into the editor itself most likely, patch the files as they are extracted from the updates -very neat ey?
Adys told me that this would not be available until several weeks later so we just hang in there, someone is working on fixing this issue we have. ;) |
I currently work on "Open MPQ in patch mode" feature of MPQ Editor. The desired functionalty is that you open a base MPQ, then add list of patch MPQs (currently, there are wow-update-12694.MPQ and wow-update-12759.MPQ), and it should seamlessly extract patched file already. I hope first experimental version might be available this week.
|
Quote:
Something like this could work well: Code:
MPQEditor.exe /multiple "enUS\locale-enUS.MPQ" "wow-update-12694.MPQ" "wow-update-12759.MPQ" |
Quote:
However, this opens the multiple MPQs in multi-mode. I guess it's time for big redesign of the entire command line syntax, because it has been there for like 6 years and too many things have changed :-) Quote:
|
Awesome, did you write it from scratch (the patching func) or did you borrow some ideas from the bsdiff sources? Just curious. I guess you wrote from scratch as you are probably super good at programming. :P
|
Quote:
|
Quote:
|
That's great news, thank you Ladik!
|
What is this MPQ community you're talking about?
|
Quote:
|
That they are very secretive? Any forum you can redirect me to? Or are you its only member?
|
Quote:
|
Quote:
|
Quote:
|
We should throw a release party :banana:
|
Quote:
|
Ladik, you're the man. ;)
|
Quote:
|
Quote:
|
Quote:
|
No point in worrying nor stressing him, he will release his tool when he feels it is stable enough for a public alpha/beta release. ;)
|
Ladik can your tool be used to extract the latest .blp files from the mpq?
or is it only lua & xml? |
It will support any kind of files in the MPQ, don't worry.
|
Vladinator thanx for the heads-up m8
|
Ok, seeing how eager you all are :p, I made experimental version of MPQ Editor that supports opening multiple MPQs in patch mode. You can download it here:
removed This link will only be valid for a few days, I'll remove it as soon as I get something more tested. Do not use it for anything else that extracting a file from patched MPQ. Good news: Each step of the patching process is verified by MD5, both before patching and after patching. So once you manage to extract something without an error message, you can be sure that it's correct. Bad news: I hadn't much time to test it yet. This build is by no means finished. Usage: Best by command line, like this: MPQEditor.exe /patch "locale-enUS.MPQ" "wow-update-12694.MPQ" "wow-update-12759.MPQ" In case you are unable to extract a patched file, please, send me an e-mail to zezula-at-volny-dot-cz. Please, don't forget to include: - The name of the primary MPQ - List of patches you entered - Full name of the extracted file More tomorrow, it's late here and I'll go something more relaxing for rest of the (already ending) day. |
Quote:
Code:
MPQEditor.exe /patch "locale-enGB.MPQ" "wow-update-12694.MPQ" "wow-update-12759.MPQ" |
For those interested, now that Ladik has released MPQEdit with patch support. Here is a Python library to read PatchFiles (PTCH).
http://www.pastethat.com/9pGwm Released under BSD. |
Sorry, guys, but I had to remove the link. You know our rules! :)
|
Quote:
|
Adys, I think Jerome is a cool name. ;)
Could you post a really short sample? |
Quote:
oldFile = open("old/testfile.lua") f = PatchFile(patchFile) print f.apply(oldFile.read()) oldFile.close() |
Quote:
MPQ editor is an external program , not an addon. |
If you try Adys' python script, append this at the end to actually use his library as a application:
Code:
if len(sys.argv) < 3: I think it's correct, tough I tried to patch ActionButton.lua from local-enUS.mpq with the wow-update-12694.mpq patch and getting MD5 errors. |
Quote:
|
Quote:
For all others who e-mailed me: Thanks for bug reports, I'll fix them problems today, and hopefully a more stable version should be ready this evening. |
Who is Jerome?
|
Edit: Found the error, the right base files are in enUS\patch-enUS.mpq, not in locale-enUS.mpq :rolleyes:
Quote:
Code:
>python patch.py data\locale-enUS\Interface\FrameXML\ActionButton.lua Data\update-12694\Interface\FrameXML\ActionButton.lua Data\patched-12694\Interface\FrameXML\ActionButton.lua "MpqEditor.exe enUs\locale-enUS.mpq wow-update-12694.mpq" opens the file without any error message, but it's empty. The same happens when both update mpqs are used. Thanks for all the work you've done! I'm really looking forward to a working version :) |
Quote:
Thanks for the reports guys. Below is an updated PTCH parser: http://dpaste.com/231608/ Also here is my blizpatch.py in case anyone wants it. Mind you you'll need to change imports to make it work. http://dpaste.com/231609/ |
A new BETA build of MPQ Editor, available on my website, should be now able to handle all patches properly. I fixed several bugs there. Please, let me know if you find a file that you are unable to extract, or it doesn't look well after extracting.
This version should be far more stable that the one I published yesterday late in the night (note for myself: Never ever release anything when it's close to midnight). As usually, there is new BETA version of StormLib available, which also contains algorithms for uncompressing and applying Blizzard patches. Note: The original download link for the experimental version is longer valid. |
Quote:
Thanks a lot, Ladik and Adys, your work is much appreciated. |
Quote:
|
Extracted the files.
But for some odd reason there are alot 1kb .blp files and some larger that can't be converted to .png Don't know if this is a bug, or just the patched data |
Is there a way to open a patch archive for multiple mpqs?
I'm trying to hack support for the wow-update-*.mpqs into the WoWModelViewer. My first idea was to open all the patch archives for each mpq, but that doesn't work, as they may be opened just one time. Does anybody have a better idea, or is anyone already working on this? |
Quote:
|
Quote:
|
All times are GMT -6. The time now is 06:55 PM. |
vBulletin © 2024, Jelsoft Enterprises Ltd
© 2004 - 2022 MMOUI