Application points and daily request limits

Web & Mobile Team
We've received a few questions about how the daily request limit throttler works and I wanted to explain the point system we use for the API.

First of all, every request we receive is associated with an application. Even when an unauthenticated request comes in, we create a temporary fake application with an id generated from the IP address of the request. Each application falls under an application level or type and each of those levels/types has an associated point value.

Instead of throttling applications by the number of requests made, we look at the cost of the requests made. For example, requesting realm status information is very cheap, therefore has a reduced request cost compared to looking up a guild, which is more expensive.

It doesn't make any sense to say what the exact cost of requests are because we regularly evaluate and tweak the cost of requests based on hardware, software and application usage. What I can do is give you some rough guidelines to use when making lots of requests to the API.

  • Requests that hit the data APIs and the realm status cost the least. These represent the baseline cost of a request.
  • Characters are expensive to request, but we employ considerable caching to reduce this cost. They can be considered 3x the cost of the baseline.
  • Guilds are expensive, especially fetching membership information. We cache as much as possible, but they still run about 3x the cost of the baseline.
  • When you use HTTP last modified headers and get a cache hit (that is to say no information needs to be returned), the cost of that request is reduced to the baseline.
  • Getting the list of AH dumps for a realm is cheap, so we'll consider it the same as the baseline.
  • Downloading one of the generated AH dump files has no cost as it is generated offline and served as a static file.


When an application, either anonymous or registered, spends more points in a day than it has available, based on the application type/level, the request is denied.
Reply Quote
90 Undead Mage
13810
Does requesting the optional fields for a character incur a higher cost?

What does it cost when requesting a character, only to get a "Character not found." error?

If someone were to go and use a Class C block of IPs to request data, would each IP address in the block be allowed the ~3,000 requests per day?
Reply Quote
85 Goblin Warrior
710
08/05/2011 03:36 PMPosted by Straton
Downloading one of the generated AH dump files has no cost as it is generated offline and served as a static file.


Hmm, interesting. How likely is it that the dump file URLs would change? If it's "free" to just grab the dump, it might be a reasonable strategy to simply grab the dump once an hour on the assumption it's probably changed, and if necessary, do a local checksum to see.

First of all, every request we receive is associated with an application. Even when an unauthenticated request comes in, we create a temporary fake application with an id generated from the IP address of the request. Each application falls under an application level or type and each of those levels/types has an associated point value.


This is probably going to be difficult to update for IPv6 :( As a net nerd, I wonder if this would be a (further?) disincentive to push ahead with IPv6 on us.battle.net. Take it as a comment, not a question :-)

EDIT: Also, gzip isn't mentioned here, is it preferable from your side for gzip to be enabled or not? The bandwidth is kind of irrelevant on my side, and could well be so on yours, in which case the CPU cost of gzip might start to matter.
Edited by Cheevos on 8/5/2011 11:49 PM PDT
Reply Quote
Web & Mobile Team
Hmm, interesting. How likely is it that the dump file URLs would change? If it's "free" to just grab the dump, it might be a reasonable strategy to simply grab the dump once an hour on the assumption it's probably changed, and if necessary, do a local checksum to see.


Likely, but not because we want to make things difficult for developers. We'd like to store a past set of N dumps for each realm and given that the times that a realm is process may not always be the same, the file names will differ over time.
Reply Quote
Web & Mobile Team
This is probably going to be difficult to update for IPv6 :( As a net nerd, I wonder if this would be a (further?) disincentive to push ahead with IPv6 on us.battle.net. Take it as a comment, not a question :-)


The system already supports ipv6.
Reply Quote
90 Undead Mage
13810
Is ipv6 active for browsing battle.net sites?
Reply Quote
90 Night Elf Priest
12995
it might be a reasonable strategy to simply grab the dump once an hour on the assumption it's probably changed, and if necessary, do a local checksum to see.


Best option would probably be to hit the api address periodically sending the last-modified headers. When a dump file is updated, the headers should have an updated timestamp for the last time it was modified. This should be enough to alert you whether or not the dump file has been updated.

From there, you also have a field in the response that has a element of 'lastmodified' with the epoch timestamp when that file was last modified. If greater than your last pull, then it's updated and you should grab it again. Grabbing the actual file may be free, but this will allow you to increase your frequency of your pulls, without ever pulling the same file more than once.

Reply Quote
85 Tauren Druid
3265
Does requesting the optional fields for a character incur a higher cost?

What does it cost when requesting a character, only to get a "Character not found." error?

If someone were to go and use a Class C block of IPs to request data, would each IP address in the block be allowed the ~3,000 requests per day?


I'm also interested in the answers to these - particularly the 404 question. My database currently has a ton of invalid characters that I'm trying to cull, so in the short term I expect to be generating a lot of 404s.
Reply Quote
Web & Mobile Team
08/06/2011 10:00 AMPosted by Chaud
Is ipv6 active for browsing battle.net sites?


No, I was just stating that when we do make the transition (at some point, I honestly don't know when) then we'll be in ok shape.
Reply Quote
Web & Mobile Team
Does requesting the optional fields for a character incur a higher cost?

What does it cost when requesting a character, only to get a "Character not found." error?

If someone were to go and use a Class C block of IPs to request data, would each IP address in the block be allowed the ~3,000 requests per day?


I'm also interested in the answers to these - particularly the 404 question. My database currently has a ton of invalid characters that I'm trying to cull, so in the short term I expect to be generating a lot of 404s.


If you think about it, a cache miss is one of the more expensive conditions that we have to deal with. It wouldn't make any sense to make cache his less expensive than a cache hit because you are doing several times more work.
Reply Quote
85 Goblin Warrior
710
it might be a reasonable strategy to simply grab the dump once an hour on the assumption it's probably changed, and if necessary, do a local checksum to see.


Best option would probably be to hit the api address periodically sending the last-modified headers. When a dump file is updated, the headers should have an updated timestamp for the last time it was modified. This should be enough to alert you whether or not the dump file has been updated.


Yeah, this is what I'm doing, but the cache hit on the API address counts towards my quota :-) I don't think I'm anywhere near close enough to 3k a day for it to matter, though, particularly since I haven't scheduled the AH data fetch. I'm deciding whether I want to offer an AH watcher to my guild that can send an SMS if a rare item pops up under a threshold price.
Reply Quote
Web & Mobile Team
Yeah, this is what I'm doing, but the cache hit on the API address counts towards my quota :-) I don't think I'm anywhere near close enough to 3k a day for it to matter, though, particularly since I haven't scheduled the AH data fetch. I'm deciding whether I want to offer an AH watcher to my guild that can send an SMS if a rare item pops up under a threshold price.


I want to stress that the previously mentioned cap of 3k a day is a soft cap with a hard limit of 10k. What that means is that we can assure you that you'll be able to make at least 3k requests (remember, for the sake of argument this request is considered a character request) a day, but we don't start turning down requests until around 10k. That hard cap may shrink or grow depending on the state of the API servers and other factors.

If you have been running an anonymous application and going past 3k character requests, this is why. We are just trying to allow developers and applications to get as much bang for their buck.
Reply Quote
1 Troll Rogue
0
Thanks for the detailed quota information, it's very useful.

Are the quota limits shared across all sites (us.battle.net, eu.battle.net) or is there a separate quota for each?
Reply Quote
Web & Mobile Team
08/07/2011 08:07 AMPosted by Ujournalh
Are the quota limits shared across all sites (us.battle.net, eu.battle.net) or is there a separate quota for each?


The daily request limit is measured separately for each region.
Reply Quote
80 Troll Hunter
2370
How do I know if I reached the maximum limits for the day? My WoWTools at facebook sudenlly stopped to work this morning :/ - In the server, I cant access the Blizzard Website. (xserver.energylabs.com.br)
Reply Quote
Web & Mobile Team
08/07/2011 09:34 AMPosted by Tesker
How do I know if I reached the maximum limits for the day? My WoWTools at facebook sudenlly stopped to work this morning :/ - In the server, I cant access the Blizzard Website. (xserver.energylabs.com.br)


You'll receive JSON responses indicating that you've been throttled. If you can't even connect (as in open a socket) then you may have bigger issues to deal with.
Reply Quote
90 Night Elf Hunter
11760
so my update queue on my site pulls full char info for 1 char every 25 seconds .. and im regestered should i have an issue?
Reply Quote
90 Night Elf Hunter
11760
08/08/2011 08:01 AMPosted by Gibblets
Do the math yourself and find out?

briliant deduction... thus alas no help in it ...
At present i can pull 2542 characters in 24 hours this can be increased and decreased .. im wondering about multiple repetitive hits resulting in a ban
or if there is a time between pulls they want us to observe
Reply Quote
Web & Mobile Team
08/08/2011 08:01 AMPosted by Gibblets
Do the math yourself and find out?


Chill.

briliant deduction... thus alas no help in it ...
At present i can pull 2542 characters in 24 hours this can be increased and decreased .. im wondering about multiple repetitive hits resulting in a ban
or if there is a time between pulls they want us to observe


Nah, you would be fine.
Edited by Straton on 8/8/2011 8:33 AM PDT
Reply Quote

Please report any Code of Conduct violations, including:

Threats of violence. We take these seriously and will alert the proper authorities.

Posts containing personal information about other players. This includes physical addresses, e-mail addresses, phone numbers, and inappropriate photos and/or videos.

Harassing or discriminatory language. This will not be tolerated.

Forums Code of Conduct

Report Post # written by

Reason
Explain (256 characters max)
Submit Cancel

Reported!

[Close]