Comunity Platform API Feedback

10 Human Paladin
0
There does not apear to be a dedicated thread for general Comunity Platform API feedback, so I thought I would start one. Only Grotako’s thread is more general in nature http://us.battle.net/wow/en/forum/topic/2516991705 , but that is limited to the Character, Guild and Arena API’s.

Now for some feedback. Let’s take a look at the URL buildup for the different API’s.

Realm Status API

http://{region}.battle.net/api/wow/realm/status?realm={realm_name}
| |_________________| |_____| |__________| |________________|
| | | | |
scheme[1] domain[2] api[3] api-path[4] parameter[5]

Character API

http://{region}.battle.net/api/wow/character/{realm}/{name}?fields={CSV}
| |_________________| |_____| |______________________| |__________|
| | | | |
scheme[1] domain[2] api[3] api-path[4] parameter[5]

Guild API

http://{region}.battle.net/api/wow/guild/{realm}/{name}?fields={CSV}
| |_________________| |_____| |__________________| |__________|
| | | | |
scheme[1] domain[2] api[3] api-path[4] parameter[5]


Arena API

http://{region}.battle.net/api/wow/character/{realm}/{size}/{name}?fields={CSV}
| |_________________| |_____| |_____________________________| |__________|
| | | | |
scheme[1] domain[2] api[3] api-path[4] parameter[5]


As we can see part [1] [2] and [3] of the URL buildup are concise. Part [3] and [4] make up the path of the URL, with part [4] having a different makeup depending on the API being called. Part [5] consists of a query.

Consistancy

The API’s mentioned here can be split into two groups when looking at URL buildup; the Realm Status API and the other API’s. The Realm Status API does not contain any variable information in it’s path [4]. The other API’s do contain variables in their paths. What at first seems like an inconsistency is actually a well thought out design. The goal here was to have the base URL (part [1] through [4]) contain the bare minimum of information necessary for each individual API to return a default payload. Additional parameters [5] can be added to each of the API URL’s to narrow down or to expand upon the information returned by the targeted API.

What I do find confusing however is the difference in the query-string [5] buildup between the Realm Status API and the other API’s. The Realm Status API has a traditional build up where the realm parameter can be repeated thus:

?realm={realm_name}&realm={realm_name}&realm={realm_name}

Whereas the Other API’s consist of one parameter (fields) which can hold comma separated values, like so:

?fields={CSV}

Why not also have the Realm Status API reflect parameter usage in the same manner and have realms be a single parameter with comma separated values?

?realms={CSV}

Validation

The API payload is a JSON format and since there is not yet a JSON schema language approved by a standards body, validating the payload is not a given. The old armory had an XML payload and no schema attached to it (at least not to the outside world), but the hierarchy and makeup of the payload did change over time. Keeping up with those changes without schema validation was a challenge.

To not have history repeat itself I would like to suggest the use of Orderly ( http://orderly-json.org/ ) or a similar library to compile JSONSchema’s. (I really love the idea of schema’s that are defined in the same language that they are validating, just like XML) Those schema’s can then be made public through the API URL like so:

http://{region}.battle.net/api/wow/realm/status/schema
http://{region}.battle.net/api/wow/character/schema
http://{region}.battle.net/api/wow/guild/schema
http://{region}.battle.net/api/wow/character/schema


In keeping with the design of the current API’s, the above mentioned URL’s would return the latest version of a schema. Older schema’s can be returned with an aditional parameter:

?version={version}

This alone would help tremendous amounts towards building fault-tolerant systems that consume the Comunity Platform API’s. The addition of the schema version in the payload would be an added bonus, but certainly not a necessity.

{
"schema":"v1.2",
"realms":[
{
"type":"pvp",
"queue":false,
"status":true,
"population":"medium",
"name":"Dunemaul",
"slug":"dunemaul"
}
]
}


EDIT: Many changes.
Edited by Blabbermouth on 5/28/2011 1:21 AM PDT
Reply Quote
Web & Mobile Team
05/28/2011 12:52 AMPosted by Blabbermouth
Why not also have the Realm Status API reflect parameter usage in the same manner and have realms be a single parameter with comma separated values?


You can:

"http://us.battle.net/api/wow/realm/status?realms=Medivh,Lightbringer"
Edited by Straton on 5/28/2011 10:10 AM PDT
Reply Quote
Web & Mobile Team
05/28/2011 12:52 AMPosted by Blabbermouth
The API payload is a JSON format and since there is not yet a JSON schema language approved by a standards body, validating the payload is not a given. The old armory had an XML payload and no schema attached to it (at least not to the outside world), but the hierarchy and makeup of the payload did change over time. Keeping up with those changes without schema validation was a challenge.


JSON validation is something that there doesn't seem to be a clear standard or best practice on, so I opted not do anything with it. I'd rather not have some sort of schema than pick one, have it not be what is used by most people and have to either change it or deal with whatever missing features or components it has.
Reply Quote
Web & Mobile Team
05/28/2011 12:52 AMPosted by Blabbermouth
In keeping with the design of the current API’s, the above mentioned URL’s would return the latest version of a schema.


API versioning is something that has my interest and I know we should do. What will likely happen is we will implement API versioning once we introduce deprecated APIs. I like the idea of having the core/root API urls act as references to whatever is the latest and providing versioned APIs for specific backwards comparability. This is all based on a lot of what-ifs, but when we need to introduce versioning, it'll likely look something like this:

Old APIs:

/api/wow/realm/status

*change that is made to deprecate it*

New APIs

/api/wow/realms
/api/wow/v1/realm/status
Reply Quote
Web & Mobile Team
By the way, great job. I really admire the depth of your writeup. Thanks.
Reply Quote
85 Draenei Mage
12090
05/28/2011 10:12 AMPosted by Straton
The API payload is a JSON format and since there is not yet a JSON schema language approved by a standards body, validating the payload is not a given. The old armory had an XML payload and no schema attached to it (at least not to the outside world), but the hierarchy and makeup of the payload did change over time. Keeping up with those changes without schema validation was a challenge.


JSON validation is something that there doesn't seem to be a clear standard or best practice on, so I opted not do anything with it. I'd rather not have some sort of schema than pick one, have it not be what is used by most people and have to either change it or deal with whatever missing features or components it has.


However wouldn't a standard set by the Internet Engineering Task Force (IETF) which is a closely working organization with the W3C have some sway in how you look at the issue? Currently the IETF does have a working draft (version 3 currently) with several core schema definitions in place.

http://tools.ietf.org/html/draft-zyp-json-schema-03#section-5
Reply Quote
Web & Mobile Team
05/28/2011 11:59 AMPosted by Delritha
However wouldn't a standard set by the Internet Engineering Task Force (IETF) which is a closely working organization with the W3C have some sway in how you look at the issue? Currently the IETF does have a working draft (version 3 currently) with several core schema definitions in place.


Not really. By the time we finish character profile APIs, guild profile APIs, updated feeds, auction house dumps, arena team feeds, pvp data APIs, static data APIs, internal and external application registration we cane take on the numerous "nice to have" features that this falls under.
Reply Quote
10 Human Paladin
0
05/28/2011 10:08 AMPosted by Straton
Why not also have the Realm Status API reflect parameter usage in the same manner and have realms be a single parameter with comma separated values?


You can:

"http://us.battle.net/api/wow/realm/status?realms=Medivh,Lightbringer"


Oh my, I feel stupid for not testing before posting. But glad to see that the API’s indeed are consistent on all fronts.

05/28/2011 10:12 AMPosted by Straton
JSON validation is something that there doesn't seem to be a clear standard or best practice on, so I opted not do anything with it. I'd rather not have some sort of schema than pick one, have it not be what is used by most people and have to either change it or deal with whatever missing features or components it has.


05/28/2011 10:16 AMPosted by Straton
API versioning is something that has my interest and I know we should do. What will likely happen is we will implement API versioning once we introduce deprecated APIs. I like the idea of having the core/root API urls act as references to whatever is the latest and providing versioned APIs for specific backwards comparability.


I can understand your apprehension in implementing non standardised specifications. Introducing versioning to the API’s would render JSON validation, for the purpose of building fault-tolerant systems, obsolete. I will very much welcome versioning if and when it is introduced to the respective API’s.

05/28/2011 10:17 AMPosted by Straton
By the way, great job. I really admire the depth of your writeup. Thanks.


Thank you for the compliment. I always find that you stand a better change to get your point across when it isn’t left open for interpretation. That and the name isn’t chosen randomly :)
Reply Quote
85 Troll Shaman
7780
Why not also have the Realm Status API reflect parameter usage in the same manner and have realms be a single parameter with comma separated values?


You can:

"http://us.battle.net/api/wow/realm/status?realms=Medivh,Lightbringer"


This falls down when specifying realm names containing spaces.

The following result is the same as not specifying any realms at all:

http://us.battle.net/api/wow/realm/status?realms=The%20Forgotten%20Coast,The%20Scryers

However, if realms are specified by slug rather than name (undocumented), the result is as expected:

http://us.battle.net/api/wow/realm/status?realms=the-forgotten-coast,the-scryers

Edit: Forgot to mention that using multiple realm parameters with space-containing realm names works as expected:

http://us.battle.net/api/wow/realm/status?realm=The%20Scryers&realm=The%20Forgotten%20Coast
Edited by Guster on 5/30/2011 8:57 AM PDT
Reply Quote
Web & Mobile Team
05/30/2011 08:46 AMPosted by Guster
This falls down when specifying realm names containing spaces.


Technically speaking, the token "Medivh" is different than the token "Medivh " and the one with the space doesn't match a realm name. For the realm slugs, we do normalize spaces out of it, for the raw names we don't. It may seem a bit strange, but this is working as intended. I'll make note to state in the documentation that the onus is on the developer to trim strings appropriately.
Reply Quote
85 Troll Shaman
7780
05/31/2011 08:31 AMPosted by Straton
This falls down when specifying realm names containing spaces.


Technically speaking, the token "Medivh" is different than the token "Medivh " and the one with the space doesn't match a realm name. For the realm slugs, we do normalize spaces out of it, for the raw names we don't. It may seem a bit strange, but this is working as intended. I'll make note to state in the documentation that the onus is on the developer to trim strings appropriately.


In this case by "names containing spaces" I mean "Medivh" vs. "The Scryers". They are treated differently when using a single realms parameter.

As a simpler example, view the result of this request which works as expected:

http://us.battle.net/api/wow/realm/status?realms=Medivh

Now this one, which incorrectly returns the results for all realms:

http://us.battle.net/api/wow/realm/status?realms=The%20Scryers
Reply Quote
Web & Mobile Team
In this case by "names containing spaces" I mean "Medivh" vs. "The Scryers". They are treated differently when using a single realms parameter.


Yeah, totally a bug. We plan on removing the support for multiple 'realm' parameters in the near future. We'll also probably trim the comma separated values just for usability purposes too.
Reply Quote
85 Goblin Warrior
710
05/31/2011 03:49 PMPosted by Straton
In this case by "names containing spaces" I mean "Medivh" vs. "The Scryers". They are treated differently when using a single realms parameter.


Yeah, totally a bug. We plan on removing the support for multiple 'realm' parameters in the near future. We'll also probably trim the comma separated values just for usability purposes too.


The "realms" argument appears to split realms based on space as well as comma:

</a>


http://eu.battle.net/api/wow/realm/status/?realms=Darksorrow%20Darkspear

{
"realms":[
{ "type":"pve", "queue":false, "status":true, "population":"medium", "name":"Darkspear", "slug":"darkspear" },
{ "type":"pvp", "queue":false, "status":true, "population":"high", "name":"Darksorrow", "slug":"darksorrow" }
]
}


Other than that, both forms take a realm name or slug, or any sort of hybrid in between, which is what I'd consider good behaviour :-)

For API authors, it should be sufficient to replace all spaces with hyphens, and do no further processing, but it's handy to know the slugification algorithm so one can reliably compare input requested names to returned slugs to know which realms matched and which did not.
Edited by Cheevos on 5/31/2011 4:59 PM PDT
Reply Quote
85 Troll Shaman
7780
05/31/2011 04:58 PMPosted by Cheevos
For API authors, it should be sufficient to replace all spaces with hyphens, and do no further processing, but it's handy to know the slugification algorithm so one can reliably compare input requested names to returned slugs to know which realms matched and which did not.


I agree publishing the slugging algorithm would be very handy for authors, and could help avoid the "whitespace" issues altogether. Either way, acceptable formats should be well-documented, and invalid formats should fail in predictable ways.
Reply Quote
10 Human Paladin
0
For API authors, it should be sufficient to replace all spaces with hyphens, and do no further processing, but it's handy to know the slugification algorithm so one can reliably compare input requested names to returned slugs to know which realms matched and which did not.


I agree publishing the slugging algorithm would be very handy for authors, and could help avoid the "whitespace" issues altogether. Either way, acceptable formats should be well-documented, and invalid formats should fail in predictable ways.


I have to agree. Getting the slug algorithm would very much be appreciated.

For the Latin based languages you can guesstimate the slug algorithm (make all letter characters fall between the 0x61 and 0x7A range so you lose capitalization and accents, replace spaces with hyphens and lose the apostrophe’s). With non Latin based languages it might not be so apparent. Take a gander at these snippets:

"name":"Король-лич",
"slug":"корольлич"


Here the normal realm name contains a hyphen (0x2D). And the hyphen gets filtered from the slug. The hyphen 0x2D is the exact same character that gets put in by the slug algorithm when it encounters a space. So I’m guessing that, when encountering a hyphen it is OK to filter it out, because the words are linked? And where space separated two words, a hyphen is tolerable. Does this have to do with the ambiguity the interpretation of the hyphen character would fall to if it were tolerated in both situations?

"name":"Разувий",
"slug":"разувии"


Now this situation. The й character has a UTF-8 hex representation of 0x439. This falls smack dab in the middle of the lowercase Cyrillic character range (at least that is what I am guessing when looking at the character ranges*). I think, and please correct me if I’m wrong, the lowercase Cyrillic character ranges from 0x430 to 0x44F. Which, like I said places the й character right in the middle of it. You can tell that it is not a URL friendly character, but it is left open for interpretation as to what to substitute the characters with. And I don’t even want to try and guesstimate how to slug the realms once http://tw.battle.net/api/wow/realm/status goes up.

My solution for now is to have a singleton with a HashMap for each region which I store the realm names with their slug counterpart as key value pairs on bootstrap. I already had this in place for validation purposes, but it can also be used to slugify the normal realm names. But I would love to replace it with an algorithm to remove the overhead of searching through the HashMap.

*I might be European, but I have no clue about the Cyrillic characters.

EDIT: changed comma's into apostrophe’s
Edited by Blabbermouth on 6/2/2011 9:10 AM PDT
Reply Quote
85 Goblin Warrior
710
Blabbermouth, 0x439 is a unicode code point representing the character 'й'. You can also represent that character as the sequence 0x0438 0x0306, which is и followed by a combining character to add the accent. There's a defined normalisation process to reach that decomposed state; it's the NFKD process (normalised form, canonical, decomposed). If you apply it, then remove all the combining characters, you wind up with something that seems to match the slug test, at least for European realms.

In Python:

>>> s = u'\u0439'
>>> nfkd_form = unicodedata.normalize('NFKD', unicode(s))
>>> s = u"".join([c for c in nfkd_form if not unicodedata.combining(c)])
>>> s
u'\u0438'


Not all languages have a unicode normalisation library, though. In practice, the entire EU realm list is slugified by (unicode) lowercasing it, stripping hyphens and single quotes, then replacing é with e, ü with u, й with и, and space with hyphen.
Reply Quote
10 Human Paladin
0
Now this piece of information is golden. I didn’t know of the existence of any normalizer API’s. Java does come equipped with one luckily. Thank you for educating me, Cheevos :D
Reply Quote
3 Night Elf Druid
0
Posting some bugs/missing fields in the item API that I stumbles across recently:


  • Glyph type seems to be missing (minor, major, ...)

  • Unique-Equipped flag is missing

  • Heroic flag is missing

  • Bound-to-Battle.net-Account flag is missing

  • Conjured item flag is missing

  • Item-starts-a-quest flag is missing

  • itemSpells.x.spell is missing an trigger type id (equip, use, chance on hit)


That's all I can remember right now. :-)
Reply Quote
3 Night Elf Druid
0
Character API:

It's impossible to discern if a talentspec is simply reset (untalented) or if the character has not obtained the dual spec feature yet.

PS: I know, I could possibly check for the achievement ID of the "Dual Spec Achievement", but that's very unwieldy... ;-)
Reply Quote

Please report any Code of Conduct violations, including:

Threats of violence. We take these seriously and will alert the proper authorities.

Posts containing personal information about other players. This includes physical addresses, e-mail addresses, phone numbers, and inappropriate photos and/or videos.

Harassing or discriminatory language. This will not be tolerated.

Forums Code of Conduct

Report Post # written by

Reason
Explain (256 characters max)
Submit Cancel

Reported!

[Close]