Long update times Auction API

API Discussion
Hey,

I'm trying to write a little page to see what auctions are present and their status.

But, the problem I'm experiencing is that the JSON file that I get from the API is only updated once every 5-90 minutes (varies greatly)

Any ideas on how to solve this?

Thanks in advance!
I'm not sure I understand exactly what is the problem you're trying to solve?

I don't think you can solve the fact that the auctions aren't updated as frequently as you would like or that they aren't updated at the same frequency.

Is your problem keeping track of how frequently you should be checking the status of a particular auction and/or what status to display based on the results?

Some options:

- Keep track of it on a per auction basis and adjust based on historical averages. So if the most recent data is older than the average update time for the last 3 checks, maybe display it as "down."

- Check each of them at a really long interval that should guarantee that they have all been updated (e.g. once every 4 hours or something).

- Check as frequently as you like, but display how "old" the data is for each auction on your web page. You could also display a confidence level if the data is older than X (e.g. display yellow if older than an hour, or red if older than 2 hours).
12/04/2018 04:46 AMPosted by Dawg6
I'm not sure I understand exactly what is the problem you're trying to solve?

I don't think you can solve the fact that the auctions aren't updated as frequently as you would like or that they aren't updated at the same frequency.

Is your problem keeping track of how frequently you should be checking the status of a particular auction and/or what status to display based on the results?

Some options:

- Keep track of it on a per auction basis and adjust based on historical averages. So if the most recent data is older than the average update time for the last 3 checks, maybe display it as "down."

- Check each of them at a really long interval that should guarantee that they have all been updated (e.g. once every 4 hours or something).

- Check as frequently as you like, but display how "old" the data is for each auction on your web page. You could also display a confidence level if the data is older than X (e.g. display yellow if older than an hour, or red if older than 2 hours).


Hey,

The issue is indeed the infrequent updates of the auctions. Is this known behaviour or is something off?
I don't think the battle.net APIs were ever intended to be a "real time" API.

I've seen some other posts that talked about how the update frequency seems to vary between US and EU, so I suspect it's fairly well known limitation.

Here's some other threads:

https://us.battle.net/forums/en/bnet/topic/20753766332
https://us.battle.net/forums/en/bnet/topic/18301006762?page=2#post-25
I actually did some testing on this a few days ago and my results are similar to your assumptions. It seems a few of the Taiwanese and Korean realms literally update once every 4 minutes. I assume those realms are the highest population realms.

Additionally most US realms seem to update anywhere from 34 minutes to 1 hour on average. EU realms typically seem more stable at around 30 minutes average update time. With a few updating a bit closer to 20 minute update.

Not including China, there are 556 Realms in total, however because we have connected realms that share auction houses there are really only 260 unique AH File Dumps between the US/EU/KR/TW regions.

If you update all 260 Realms at the same time, I discovered that once every 2 minutes there will be about 7 new realms that have been updated. This is because different realms update at different times so your dealing with all 260 auction house files, you literally have new data coming in at all times.

However one helpful way to think of these files is how there generated. There basically a summary of all the auctions for a particular group of realms. This means that if an auction house has 1000 items listed, the file dump will contain 1000 entries. Now if 100 new auctions are generated on the realms, the next time the file updates it will contain 1100 entries and so on and so forth.

Basically you can think of it as, new entries are added to the top of the files and old entries are at the bottom, during an update, new entries get added and any expired auctions since the last update get removed.

That's not 110% accurate representation of what happens but by thinking about them in this manner I've found it can really help formulate a strategy when programming your way through the data.

Basically just because an Update occur's doesn't mean that there is an entirely new file with all new auctions. There might not be any changes to the file from one update to the next (Unlikely though).

Anyway I'll stop there but hopefully that help's jog the mind a little bit when trying to understand how the files update and what happens to them when they do update.

EDIT: Also note that we cannot and do not control when these files update. Were at the mercy of Blizzard and we just have to find ways to work around certain limitations.
12/04/2018 10:51 AMPosted by Whims
Additionally most US realms seem to update anywhere from 34 minutes to 1 hour on average. EU realms typically seem more stable at around 30 minutes average update time. With a few updating a bit closer to 20 minute update.

US realms each update every 119 minutes. EU has two update groups, one group of realms updates every 55 minutes, the other, every 70 minutes. It's been like this for a while. You can find more info at https://theunderminejournal.com/dataintervals.php (Ignore the 20-minute intervals for TW/KR, that's just the floor I set. Blizzard updates them more frequently than that.)

Blizzard has one script that runs once per minute on the US that updates the snapshot for the oldest realm. They sometimes run another copy of the script concurrently if we whine about it enough, but that never lasts for very long. Somehow they figured out how to have 2 scripts run pretty consistently for the EU. (They used to have even more than that over there.)

I describe an algorithm to fetch the freshest auction data here: https://us.battle.net/forums/en/bnet/topic/20766817044#post-4
12/04/2018 12:08 PMPosted by erorus
US realms each update every 119 minutes. EU has two update groups, one group of realms updates every 55 minutes, the other, every 70 minutes. It's been like this for a while. You can find more info at https://theunderminejournal.com/dataintervals.php (Ignore the 20-minute intervals for TW/KR, that's just the floor I set. Blizzard updates them more frequently than that.)


Thanks for that information Erorus, my data set was a really small test but TUJ has been at this for quite a long time, it's nice to see how they did the Data Intervals, very same train of thought I am going by in writing my own scripts at the moment.

Join the Conversation

Return to Forum