On RESTful API Standards – Just Be Cool: 11 Rules for Practical API Development (part 1 of 2)

Posted on May 1, 2012 by Larry
25 Comments

There are many articles floating around the internet right now about defining RESTful API standards. I’m guessing this is because Ruby on Rails makes it so easy to build an API that everyone and their mom has started building one. These articles talk about the RFC on HTTP and how REST should fit in. They outline how current implementations are “broken” and want to come up with certain “standards” for governing how a RESTful API should work. These articles make all sorts of claims about how the return type of the data should only be set in the request headers, how the obscure “PATCH” HTTP method should be used, and how doing XYZ does not follow HATEOAS. The problem with these discussions is they propose solutions that are more academic than useful.

I have a response to all of this - ENHANCE YOUR CALM!

Let’s take a step back and think: What is the real purpose of creating an API? The “real purpose” is to increase your user base by making it easy for developers to plug into your system. How does an API accomplish this? By making it simple for developers to integrate with your systems and build awesome applications on top of your systems through all of the functionality you provide. If you make people jump through too many hoops to use your API, they might just not use it – and you get zero new users from that. As a developer who has been building and consuming APIs for several years now, I have found the following rules to be practically (not just theoretically) useful for both the development and consumption of an API:

1. Use GET, POST, PUT, and DELETE to separate out actions. GET should be used to retrieve objects. It should never be used to update an object. There are many reasons for this, but my favorite is you don’t want someone accidentally clicking a hyperlink and screwing up their data set. POST should be for creating objects. Be it parent or child objects – if you are creating an object of sorts, use POST. PUT should be for updating. PUT is when you have an exact URL that you could also use for GETting that object. I know some argue that you can use PUT to create. For example, you could PUT a new item to /pants/12345 which would create a new pants object with id 12345. But honestly, why would your users be creating an object with an id? DELETE deletes an object. I shouldn’t have to explain why.

2. PUT can be used to do partial updates. I know the REST police might say that you have to pass in the entire object each time you do an update, but that can be expensive. Sometimes you have many child objects, while other times an object might just be beefy and sending the whole thing through would be expensive. Why make a slower experience just to adhere so directly to the rules?

3. Add meta data to your responses. This probably breaks HATEOAS or some REST principle in that you expect the object and only the object in your response, but come on. Don’t you want your users to have easy access to errors, error codes, error messages, etc.? Sure you can pass that information in the header somehow, but I’ve seen clients who cannot figure out how to use the header responses correctly. It’s so easy to parse a JSON response. Why make it harder for your users?

4. Return a status and the object on POST and PUT calls. This might violate the tenants of REST, but users need to know when their calls work. Returning the object saves them from having to send a GET call on their end and you from having to handle their separate GET call. This is especially helpful in mobile app development. Who wants a slow mobile app?

5. Return the object type on a GET. By this I mean have your JSON for the banana service look something like ‘….”banana” : {“field1” : 13….}’ Sometimes users like to write generic parsers and it is easier if they can tell which object they are getting back from the response itself.

I’ll leave it here for now. Check in later this week for principles 6 through 11.

This entry was posted in Developer Tools, Development Process. Bookmark the permalink.

25 Comments
  • SiC

    “Return the object type on a GET” – Any reason not to simply include this in the content-type header, e.g. “application/namespace.type+json”? It seems a more appropriate place imho…

    • Larry

      You really could do that. My argument is just that if is easier for consumers if it is all within the JSON object. Some stock API wrappers make it harder to access the headers but it is still super easy to parse and iterate through the JSON

  • SiC
  • Me

    #4 seems to conflict with #2. Why would it make sense to return the whole object in #4 when doing the same is considered wrong in #2?

    • Larry

      You make a valid point. My thought process is that a client’s download bandwidth is a lot cheaper than the upload bandwidth

  • http://sintaxi.com sintaxi

    PATCH is for updating partial objects. Not PUT.

  • Tarun Ramakrishna

    PUT can be used for create

    The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the “origin server CAN CREATE the resource with that URI”. If a new resource is created, the origin server MUST inform the user agent via the 201 (Created) response. If an existing resource is modified, either the 200 (OK) or 204 (No Content) response codes SHOULD be sent to indicate successful completion of the request. If the resource could not be created or modified with the Request-URI, an appropriate error response SHOULD be given that reflects the nature of the problem. The recipient of the entity MUST NOT ignore any Content-* (e.g. Content-Range) headers that it does not understand or implement and MUST return a 501 (Not Implemented) response in such cases.

  • CamShaft

    http://www.amazon.com/gp/aw/d/1449306578 This book goes into great detail of practical RESTful services. This is a must read for API design.

  • Guest

    I don’t get the excitement about REST. Sure, don’t use GET for actions that have side-effects. We’ve known that for a long time. But beyond that? I say just write an API that makes sense for your application, rather than trying to use a crowbar to force it into a REST-style interface.

    Remember the early 2000′s? Everyone got irrationally excited about XML. That’s REST today. Give it a couple of years – it’ll blow over :)

    • Scott

      The reason REST is better than XML is for a couple of reasons:

      1) REST runs on the HTTP protocol, which is how the internet works to begin with.

      2) Adding XML to the body or using SOAP envelopes simply requires additional technologies (dtd, SAX, wsdl) to work, as well as custom XML parsers, when most of the time REST/HTTP is sufficient.

      3) Using JSON on top of REST for responses reduces the need to write a parser at all in most modern web frameworks, and interoperating interfaces / format types reduces the complexity and overhead of systems integration.

      4) No one needs help writing REST APIs that much anymore. Rails, sinatra, Python, all of these frameworks and technologies make it easy to provide a web service using REST, simply because the technology uses the correct abstraction. Most enterprise IDE’s will “auto-generate” your XML web services Java cruft which will be broken more than half the time. But since you didn’t write it you don’t know how to fix it.

      XML was irrational, but so was pretty much all of things that happened in 2000. REST is a return to sanity, a return to the roots of the web, and applies lessons learned from the last tech boom.

      • tomvons

        I think the biggest win of REST is the reminder that the ‘P’ in HTTP stands for ‘Protocol’.

      • Guest

        Oh, I was only comparing REST to XML in terms of the irrational enthusiasm surrounding both items – not functionally. Apples and Oranges. XML is a data format (and a poor one at that – it bears little resemblance to the run-time structures that we’re all familiar with).

        It’s unfortunate that, as you say, the Internet runs on HTTP. HTTP is a truly awful protocol. As a simple document-retrieval mechanism, it’s adequate – albeit overly complex and often poorly implemented. As the foundation for the next generation of web applications, it is *abysmal*.

        The next generation of applications will require persistent connections, dynamic updates, and be based around local/remote object synchronization. Look at the hacks we’ve resorted to over the past few years in order to try and support this: Comet? JSONP?

        We’ve had to invent an entirely new protocol (WebSockets) to get ourselves out of this mess.

        In my (admittedly unpopular) opinion, we should be moving toward decoupling application communication from such an awful transport layer. Instead, REST embraces the more obscure aspects of HTTP (PUT/PATCH?) and has the net result of tying us even more to this truly awful protocol.

      • Jared

        SOAP has its place, just as much as REST does. It’s really about the interface that you’re trying to convey to your client and the guarantees that interface provides. Sometimes RPC with SOAP makes more sense for what you’re doing than making everything RESTful. Especially if you write code for systems that deal with more than comments, likes, and photos (the real world).

        And just because you’re working with crappy tools, doesn’t mean that you shouldn’t use SOAP/REST. You should look at getting better tools, first.

        Btw, XML is just as “RESTful” as JSON; the really idiotic misnomer about REST is that its all JSON. REST can be any media type, it doesn’t matter. It can even be a file.

        I think the unfortunate thing about REST, and this article clearly points it out, is that instead of trying to understand the ideas behind it, what implications it really has for the architecture, and how it can be used in conjunction with other messaging services … our industry just wants to reinvent the wheel. This goes along the same lines as explosion of JavaScript frameworks permeating HackerNews right now. Sure you have to be practical … implementing every HTTP verb would be pointless, but standards and convergence do matter. I think we’ll see that when we start having to replace these system in 5 years.

  • Guest

    I have an awesome idea. Let’s all argue over PUT and PATCH as if it matters.

    • tomvons

      If you care about PUT instead of POST then why wouldn’t you care about PATCH instead of PUT? There are only 9 HTTP methods, using the right thing in the right place is not a big deal.

      • Christian Romney

        “edit: Actually, PATCH isn’t in HTTP 1.1″

        technically correct, but it’s a bit more complicated than that…
        it was in RFC 2068, then removed in RFC 2616, and now it’s back in RFC 5789
        nobody said this stuff was easy ;)

    • http://sintaxi.com sintaxi

      It matters on an article about REST API design rules. Besides, not arguing, PATCH is the appropriate HTTP method for partial payload updates. http://tools.ietf.org/html/rfc5789

  • Joe

    #1 – Duh. I don’t see the hardship in PUT for create, but meh.

    #2 – Is it so hard to use PATCH? It’s two characters more dammit!

    #3 – “Probably breaks HATEOAS or some REST principle…”, umm, no it doesn’t.

    #4 – “This might violate the tenants of REST…”, again, no it doesn’t.

    #5 – Yep this is useful.

    I don’t know why you think that these tips are somehow challenging REST dogma or “bringing the impractical, stuffy world of REST down to earth”. This is mostly solid advice, but your vague references implying that these tips are all somehow “far too practical to be RESTful” seem entirely misguided.

    You seem to be railing against some mysterious REST boogeyman.

  • Christian Romney

    Let me propose some alternatives.

    In point 2. above you say, “entire object each time you do an update, but that can be expensive. Sometimes you have many child objects, while other times an object might just be beefy and sending the whole thing through would be expensive”

    First, I challenge the “expensive” argument. Your average web page is probably an order of magnitude beefier than most of these sorts of API calls. Of course specifics matter, but we’re dealing in generalities and hypotheticals here.

    Second, if you have child objects that need updating, perhaps they should have…their *own* URIs. This sounds like a modeling and composition problem, not a problem with REST. Say you have a grocery list with 3 items: apples, bananas, corn. Do you need to send the whole list just to add milk? Sure, that is one way of modeling the problem, but you could just as easily POST a new item to the existing list. We’re limited only by imagination here…

    In point 3 you talk about metadata, specifically codes and messages. The HTTP response code should be all the “code” you need in most cases. There are a ton of them covering even some relatively obscure stuff. If you have an error message, feel free to return it in the entity body. Sounds like you want a 409 – Conflict response. From RFC2616:

    “The response body SHOULD include enough
    information for the user to recognize the source of the conflict.
    Ideally, the response entity would include enough information for the
    user or user agent to fix the problem; however, that might not be
    possible and is not required.”

    In other words, this is perfectly REST-ful.

    In point 4 you say one should return a status and the object on POST and PUT.

    The RFC says you’re free to return 200 OK or 204 No Content on POST or PUT – so knock yourself out. There’s nothing wrong with returning 200 OK and spitting a representation of the resource back out to the client. The difference between POST and PUT is often misunderstood but the spec is pretty clear. The URI in a PUT identifies the resource itself – the thing being PUT on the server. The URI in POST is a parent resource and the thing being POSTed is a subordinate resource. So I would PUT the entire grocery list, but I would POST a new item to the list.

    In point 5 you say return the object type on a GET.

    Here we just disagree completely. Fielding talks about this specifically here: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven

    The problem is one of inappropriate coupling. In your example, you talk about a “banana” in your JSON snippet, but what does the server-side implementation’s type have to do with the client side? It really shouldn’t have anything to do with it. If you’re mapping to a “logical” entity (this is a banana) as opposed to a physical one (this is an instance of class Banana), the URI should tell you exactly what you’re requesting and the media type should have some documentation that tells you how it is represented across the wire and how you ought to process such entities. Perhaps what you need is not to return application/json, but application/vnd.groceries+json…

    Finally, as for PATCH versus PUT, PATCH was defined (in RFC2068) to address the specific argument you raised in point 2 about large representations. The newer RFC 2616 doesn’t say much about PATCH other than to refer to the older RFC since it is infrequently used. I see nothing wrong with a framework like Rails implementing this part of the spec, though I confess I’m skeptical of the performance gains that *might* be realized…

    • JohnR

      So, if I edit my grocery list, removing banana from the list and adding milk, how do you accomplish that? Do I have to do a DELETE operation for every item I remove from the list, and a POST for every new item? Or is it better just to PUT the whole list each time? Or does it depend?

      The problem I have with using PUT for partial updates is how do you know when something is intended to be removed from the object vs. you excluded it because there are no updates to it? Using the grocery list, if I PUT the list of items, there is no real way for this to be a partial update, is there? You can’t tell if I am telling the server to add the items in the list, or replace the list. You then start inventing things in the JSON like a field for “addToList” and “deleteFromList”, etc.

  • Rathole

    The salient point is in the title…”just be cool.” Sadly, no one does that, it seems.

  • Jessica

    Catch pt 2/2 of Larry’s post: On RESTful API Standards – Just Be Cool: 11 Rules for Practical API Development http://bit.ly/KwW6nM

  • http://pulse.yahoo.com/_EJO72JN2UKUFKNPAMOAFZKVAKY Bryan

    #1 POST and PUT are both for creates and updates. The difference is two fold: 1) with PUT the client supplies the resource’s URI vs POST the server determines it 2) PUT is idempotent, POST is not. This is not REST, it’s HTTP. REST calls for a uniform interface and HTTP supplies one. You can choose to restrict it the way you said and still have a RESTful API, but there’s no real benefit other than dealing with devs who can’t understand be bothered to understand how HTTP really works.

    #2 I’ve argued your position with Roy Fielding. I don’t care what they think it should be, RFC 2616 as-is cannot be interpreted to forbid partial updates via PUT. The http-bis working group is trying to change it to do this by adding a GET what you PUT restriction, but their new proposal doesn’t make sense for resources that support multiple mime types. Refinements work quite well too: /car/123 has tires that can be updated via car/123/tires, but the overlap degrades “visibility”.

    #3 Huh!?!? It doesn’t break HATEOAS. Media types define data formats, and most are extensible, so of course this is fine.

    #4 There’s nothing wrong with it, but making a rule that you always have to do it is silly. You just ruled out the standard pattern for long-running asynchronous operations, for example.

    #5 This is a JSON practice, not a REST practice. It’s orthogonal. But I suspect that you aren’t understanding the value of media types and content negotiation.

    • Sam IT

      Regarding your #5, how would you specify the type of one or more of the value(s) using a single Content-Type header?