Thread Tools Display Modes
08-19-19, 01:57 PM   #1
Odjur
A Murloc Raider
AddOn Author - Click to view addons
Join Date: Feb 2014
Posts: 7
GetAuctionItemInfo removing items from auction house page

I am playing around with developing a personal auction house API, but I am running into one issue that I don't know how to approach solving. My current goal is to do a GetAll search and then record the information for each of the items. The problem is that calling GetAuctionItemInfo seems to be able to update the list, removing auctions. Here's a high level explanation of what my issue is:
  • I do a GetAll search
  • I call GetAuctionItemInfo on auction 1
  • It returns none, some, or all of the information
  • If it returns some of the information, I have to call GetAuctionItemInfo on it again

The problem is that in that last step, if the auction was removed from the list, then I might be calling GetAuctionItemInfo on a new auction, and there doesn't seem to be a way to guarantee that it is the same.

Maybe I'm just misunderstanding something? If anyone could give some guidance for this I would greatly appreciate it, thanks!
  Reply With Quote
08-19-19, 03:43 PM   #2
myrroddin
A Pyroguard Emberseer
 
myrroddin's Avatar
AddOn Author - Click to view addons
Join Date: Oct 2008
Posts: 1,240
Are you registering for this event? https://wow.gamepedia.com/AUCTION_ITEM_LIST_UPDATE
  Reply With Quote
08-19-19, 04:00 PM   #3
Odjur
A Murloc Raider
AddOn Author - Click to view addons
Join Date: Feb 2014
Posts: 7
I wasn't but I think that was because of my assumption that GetAuctionItemInfo could update any auction in the list, not just the auction that it was called on, which would make it unclear whether the item being removed would affect the current index or not. In hindsight that seems like the less likely case. Thanks for the help!

Last edited by Odjur : 08-19-19 at 04:12 PM.
  Reply With Quote
08-22-19, 04:23 PM   #4
Odjur
A Murloc Raider
AddOn Author - Click to view addons
Join Date: Feb 2014
Posts: 7
Ok, so it seems that I was right the first time. After testing it appears that GetAuctionItemInfo can update the list, removing any item, not just the one being scanned. This means that even if I am registering AUCTION_ITEM_LIST_UPDATE and the list updates, I don't know whether the removed item is before, after, or at my current index.

Is it possible to stop GetAuctionItemInfo from updating the list? I can't seem to find documentation for the function. I'm really not sure how to proceed, so any help would again be appreciated, thanks.
  Reply With Quote
08-22-19, 05:28 PM   #5
elcius
A Cliff Giant
AddOn Author - Click to view addons
Join Date: Sep 2011
Posts: 75
Not exactly sure what the problem you're having is, but the most efficient way to handle a GetAll response is to ignore the event. just scan and keep track of your position in the results with periodic updates.

Lua Code:
  1. local ScanPos;
  2. local TotalAuctions = 1;
  3. local Scanning = false;
  4. local LastScanTime = 0;
  5.  
  6. Scanner = {};
  7. function Scanner:Ready()
  8.     return AuctionFrame and AuctionFrame:IsShown() and select(2,CanSendAuctionQuery());
  9. end
  10. function Scanner:CanStart()
  11.     return (not Scanning) and self:Ready(), LastScanTime, 900;
  12. end
  13. function Scanner:Start()
  14.     Scanning = true;
  15. end
  16. function Scanner:OnFinished(incomplete)
  17.     Scanning, ScanPos, TotalAuctions = false;
  18.     AuctionFrameBrowse:RegisterEvent('AUCTION_ITEM_LIST_UPDATE');
  19. end
  20.  
  21. C_Timer.NewTicker(0.1,function()
  22.     if not Scanning then return end
  23.    
  24.     local ready = Scanner:Ready();
  25.     --[[
  26.     if not TotalAuctions then -- first grab the auctionhouse size, this can be skipped
  27.         if not ready then return end
  28.         AuctionFrameBrowse:UnregisterEvent('AUCTION_ITEM_LIST_UPDATE');
  29.         EventHandler:Register('AUCTION_ITEM_LIST_UPDATE',function()
  30.             TotalAuctions = select(2,GetNumAuctionItems("list"))
  31.             --print('total:',TotalAuctions);
  32.             return true;
  33.         end);
  34.         QueryAuctionItems("",nil,nil,0,0,0,0,0,0,false);
  35.         return;
  36.     end
  37.     --]]
  38.     if not ScanPos then -- init scan
  39.         if not ready then return end
  40.         --print('Downloading');
  41.         ScanPos = 0;
  42.         LastScanTime = time();
  43.         QueryAuctionItems("",nil,nil,0,0,0,0,0,0,true); -- GetAll
  44.         return;
  45.     end
  46.    
  47.     if CanSendAuctionQuery() then -- scan complete
  48.         Scanner:OnFinished();
  49.         return;
  50.     end
  51.    
  52.     -- continue parsing auction data.
  53.     local n, total = GetNumAuctionItems("list");
  54.     if n ~= total or n <= ScanPos then
  55.         return -- bad data block
  56.     end
  57.     local GetAuctionItemInfo = GetAuctionItemInfo;
  58.     for i = ScanPos+1, n do
  59.         --this is where you call your auction (single) result handler
  60.         --Database:AddAuction(GetAuctionItemInfo('list', i));
  61.     end
  62.    
  63.     print(n..'/'..TotalAuctions..': +'..(n-ScanPos),'('..floor((n/TotalAuctions)*100)..'%)');
  64.     ScanPos = n;
  65. end);
  66. --[[
  67. EventHandler:Register('AUCTION_HOUSE_CLOSED',function()
  68.     if ScanPos then -- Scan interrupted
  69.         Scanner:OnFinished(true);
  70.     end
  71. end);
  72. --]]
  Reply With Quote
08-22-19, 06:06 PM   #6
MooreaTv
An Aku'mai Servant
AddOn Author - Click to view addons
Join Date: May 2019
Posts: 38
I use a mix of the events and timer to wake up in
https://github.com/mooreatv/MoLib/bl...er/MoLibAH.lua

but after the first event (you do want to wait for the first AUCTION_ITEM_LIST_UPDATE because otherwise GetNumAuctionItems is 0) you can probably just indeed brute force it until you get all the items
(be nice to not hang the UI though)

I found I had to 'pre fetch' or pre ask for more items for them to get filled up, I'm not sure what's the best size but it is > 50 (for instance 500 is much faster overall than 50 which is the NUM_AUCTION_ITEMS_PER_PAGE default)

@elcius- from your (nice!) snippet, it seems you are actually trying to fetch all of them at once, does that work fine on a 60k+ result ?

Last edited by MooreaTv : 08-22-19 at 06:09 PM.
  Reply With Quote
08-22-19, 06:50 PM   #7
elcius
A Cliff Giant
AddOn Author - Click to view addons
Join Date: Sep 2011
Posts: 75
it's a common misunderstanding that GetAll delivers the auctions all at once, that's why most people still listen for the event, or even worse they wait until everything as arrived and then lock up the game trying to process them all at once.
In actuality auctions get streamed over 1024 at a time, until near the end of the results.
results that have already arrived are immediately available in the results list, and thus can be processed while waiting for the rest, once more results arrive the result list will just expand.

Last edited by elcius : 08-22-19 at 06:54 PM.
  Reply With Quote
08-22-19, 08:43 PM   #8
Odjur
A Murloc Raider
AddOn Author - Click to view addons
Join Date: Feb 2014
Posts: 7
I really appreciate all of the replies. I wrote a small test script to hopefully demonstrate my issue, and put it in an addon called "Test". The following is what you need to do, and after doing it the number of auctions should be different.
  • /test getall
  • Wait for the getall search to complete
  • /test count
  • Wait ~30 seconds (my guess is that in this time auctions expire / sell)
  • /test scan
  • Wait for the scan to finish
  • /test count

Code:
 -- Search the entire auction house at once
local function GetAll()
	if select(2, CanSendAuctionQuery()) then
		AuctionFrameBrowse:UnregisterEvent("AUCTION_ITEM_LIST_UPDATE")
		
		-- QueryAuctionItems(name, minLevel, maxLevel, page, isUsable, qualityIndex, getAll, exactMatch, filterData)
		QueryAuctionItems("", nil, nil, 0, false, 0, true, false, nil)
	end
end

 -- Display the number of items on the current auction house page
local function Count()
	print(GetNumAuctionItems("list") .. " items remaining")
end

local start
local finish

 -- Call GetAuctionItemInfo, GetAuctionItemTimeLeft and GetAuctionItemLink on each auction
local function Scan()
	finish = math.min(GetNumAuctionItems("list"), start + 1000)
	
	for index = start, finish do
		local info = GetAuctionItemInfo("list", index)
		local timeLeft = GetAuctionItemTimeLeft("list", index)
		local link = GetAuctionItemLink("list", index)
	end
	
	if finish < GetNumAuctionItems("list") then
		start = finish
		C_Timer.After(0, Scan)
	else
		print("Finished")
	end
end

local eventFrame = CreateFrame("FRAME")
eventFrame:RegisterEvent("ADDON_LOADED")
eventFrame:SetScript("OnEvent", function(_, event, addon)
	if event == "ADDON_LOADED" and addon == "Test" then
		SlashCmdList["Test"] = function(message)
			if string.lower(message) == "getall" then
				GetAll()
			elseif string.lower(message) == "count" then
				Count()
			elseif string.lower(message) == "scan" then
				start = 1
				Scan()
			end
		end
		
		SLASH_Test1 = "/Test"
	end
end)
  Reply With Quote
08-22-19, 09:26 PM   #9
elcius
A Cliff Giant
AddOn Author - Click to view addons
Join Date: Sep 2011
Posts: 75
I see, some auctions do get removed, most likely due to the fact character names of the posters get resolved when querying GetAuctionItemInfo, which in turn causes the auction list to refresh/update when a name is resolved.

I recommend you just process the auctions as soon as you get access to them, so as to ensure none are expired, like in my example.

Alternatively you can probably just scan the list in reverse from n to 1. not ideal since you need the final count, and should really avoid waiting for the download to finish before you start processing the auctions.

Another possible solution would be to just call GetAuctionItemInfo on all the auctions once, this will resolve all the names of the players, then do another pass after all the names are loaded, the second pass shouldn't drop any auctions.
  Reply With Quote
08-22-19, 11:14 PM   #10
Odjur
A Murloc Raider
AddOn Author - Click to view addons
Join Date: Feb 2014
Posts: 7
Thanks for the suggestions. It does not seem to be the sellers names being retrieved that causes the update, as each subsequent pass removes more listings. I still believe that it is expired/selling auctions that is causing this update.

I also have to mention that it appears that GetAuctionItemTimeLeft is the only function that can remove auctions from the page, as long as AUCTION_ITEM_LIST_UPDATE is unregistered from AuctionFrameBrowse. I'm not sure what the implications of this are for non-getall scans, but my solution will likely be to simply exclude it and chalk it up as a bug. If anyone comes up with a better solution, please let me know, thanks!
  Reply With Quote
08-23-19, 12:02 AM   #11
MooreaTv
An Aku'mai Servant
AddOn Author - Click to view addons
Join Date: May 2019
Posts: 38
We had some strangeness and I have an explanation:

It appears experimentally that if you wait enough (probably for auctions to expire) one of the GetAuctionItem* api becomes actually blocking (I suspect because it's reordering the result list) and causing reentrance (using coroutines)

This throws quite a wrench into an implementation expecting linear executing and only 1 timer running at a time

gory details are on https://github.com/mooreatv/MoLib/issues/16
  Reply With Quote

WoWInterface » Developer Discussions » Lua/XML Help » GetAuctionItemInfo removing items from auction house page

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off