@inadarei: Creating Client-Optimized Resource Representations in APIs Page 1 of 5

Creating Client-Optimized Resource

Representations in APIs
1 1

Posted on: June 25, 2015

One of the most important design principles for an API program is to embrace that APIs are products. It
follows that successful APIs, much like products, must provide user-optimized experiences. Case in point: it is
clearly inappropriate to send a mobile application large response payloads. Your API may get away with it
when responding to a request from an enterprise system communicating over a high-speed fiber-optic
connection. Some clients may even want to get a rich response, but in general assuming that network is
reliable and/or bandwidth is infinite is a fallacy, on the web.

A Brute-Force Approach: Field Filtering

The fact that mobile clients are not fans of large response payloads is hardly a shocking news.

API teams have historically allowed customization of API responses using a somewhat brute-force approach
known as "field filtering" allowing an API client to indicate a list of returned fields in the request. It can look
something along the lines of:

GET https://api.example.com/users/123?fields=firstname,lastname,age

We have also seen solutions where all default fields are allowed, but a client has ability to "ban" potentially
expensive fields from the response:

GET https://api.example.com/users/123?exclude=biography,resume

The extreme case of this approach is: GraphQL, which is basically "SQL" over HTTP, entirely concentrated on
querying data-storage on the server and thus tightly coupling clients with the data-models of the server.

There Is a Better Way

One of the main problems with Field Filtering is that it encourages tight coupling of a client with an API server.
By allowing such solution, we expect clients to maintain too granular of understanding for your API
implementation. This can lead to unnecessary breaking changes and API versioning, which as Mark
Nottingham would have you know is evil.

Instead, what we can do is: we can create tiered representations of responses. Each tier of a resource
representation can have size-oriented or some other logical name, satisfying the need of tailoring output to a
client group, without leading to extraneous coupling. As luck would have it: IETF already has an RFC that suits
this purpose: RFC 7240 - Prefer Header for HTTP

This is how you could create "minimal" (vs "standard" or "full") outputs:

http://www.freshblurbs.com/blog/2015/06/25/api-representations-prefer.html 08-Jun-17
@inadarei: Creating Client-Optimized Resource Representations in APIs Page 2 of 5

Get /users/123 HTTP/1.1

Host: api.example.org
Content-Type: application/json
Prefer: return=minimal
Vary: Prefer,Accept,Accept-Encoding

HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Date: Sat, 27 Jun 2015 11:03:32 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Vary: Prefer,Accept,Accept-Encoding
Preference-Applied: return=minimal
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Content-Type

Please note that including "prefer" in the Vary header of the response is required per RFC 7240. It makes
sure that some caching infrastructure won't cache a wrong representation of the resource, since multiple
representations will have the same URL, only differentated by the Prefer header.

You can also see that the response acknowledged the preference by including Preference-Applied:
return=minimal in the response.

The name of the preference batch doesn't always have to be size-oriented. For instance, in case of news
media and blogosphere, a logical preferance that might make sense, could be:

Get /blog/1223 HTTP/1.1

Host: api.example.org
Content-Type: application/json
Prefer: return=teaser
Vary: Prefer,Accept,Accept-Encoding

We've also seen elegant uses of the prefer header when clients ask the API server to transclude some of
the data that would normally be just linked from the API response. This can be especially useful if otherwise
the client would need to make too many (N+1?) additional requests to retrieve the same data. For instance:

Prefer: transclude=availibility_calendar
Vary: Prefer,Accept,Accept-Encoding

Word of caution: we need to make sure that prefer headers, in our designs, refer to logical batches and not:
individual fields. Just moving "Field Filtering" from URL to the HTTP Headers is not a solution, obviously.

Use Business Terms for Prefer

I couldn't recommend enough the approach of using domain-driven/business terms for batching in the HTTP
Prefer approach. The reason being: it leads to batch names that survive changes in your system. As Greg
Young (@gregyoung), of CQRS and Event Sourcing fame, says:

Structure changes more often than behavior.

http://www.freshblurbs.com/blog/2015/06/25/api-representations-prefer.html 08-Jun-17
@inadarei: Creating Client-Optimized Resource Representations in APIs Page 3 of 5

Your use-cases of a system tend to be reasonably stable over long periods of time.
How you interpret your internal data: the structure that you use has a tendency of
changing a lot. (Source: Code on The Beach 2014)

This has also been my experience in building complex systems, therefore: if you choose business terms for
batch names in Prefer , you will be able to defer their change for longer, and as we know: versioning is an
evil we avoid, in APIs.

TL;DR - In Conclusion
Do not use "field filtering" to provide more sensibly-sized response payloads to mobile applications or other
clients. This may create too tight of coupling between the client and the server. Instead, use HTTP Prefer
Header for HTTP - RFC 7240.

Acknowledgements
I first learned this technique (much like most things I know about APIs) from my dear friend and colleague
Mike Amundsen. It builds upon thinking and work by amazing IETF contributors: James Snell and Mark
Nottingham, as well as probably others in the web community. I apologize if I accidentally missed anybody, but
I am very grateful to all who contribute to the future of the better web.

http://www.freshblurbs.com/blog/2015/06/25/api-representations-prefer.html 08-Jun-17
@inadarei: Creating Client-Optimized Resource Representations in APIs Page 4 of 5

7 Comments Fresh Blurbs

1 Login

Recommend 1 Share Sort by Newest

Join the discussion

Willi Schnborn 9 months ago

The grammar for the "return" preference doesn't allow for custom values, unfortunately:

> return = "return" BWS "=" BWS ("representation" / "minimal")

Reply Share

inadarei Mod > Willi Schnborn 9 months ago

Willi, do you mean that something like `return=customsomething` is not allowed? I don't see such limitation in the RFC and
James didn't mention it when we discussed using Prefer header for tuning payloads, before.
Reply Share

Willi Schnborn > inadarei 9 months ago

Exactly. If you check https://tools.ietf.org/html... you will find the grammar that I posted before. It only supports two
possible values for "return", one being "representation" and the other one "minimal".
Reply Share

inadarei Mod > Willi Schnborn 9 months ago

Ha, that is very interesting. Definitely didn't notice it, before. Thanks for pointing out!
Reply Share

petejohanson 2 years ago

The 'transclude' Prefer header makes perfect sense, and would pair nicely with something like HAL's embedded resources. The natural
extension would be for the transclude to specify the link relations that should be included....

Good food for thought!

Reply Share

inadarei Mod > petejohanson 9 months ago

So this came out of this year's RESTfest hackathon: http://spec.transclude.io
Reply Share

inadarei Mod > petejohanson 2 years ago

Thanks, Pete.

That idea sounds awesome to me. I wonder if ALPS can give some special love and make it very easy. Gonna ping Mike about
it. May be too specific of a case for ALPS, but definitely interesting.
Reply Share

FRESH BLURBS

Why Have Open Data Initiatives Not Been More Successful? How to Hack 'Read More' in Drupal

Thanks for your response, Ruben. I respect and Worth looking at https://drupal.org/node/49428
appreciate your viewpoint and while we don't have to agree, I do
strongly object to the notion that the alternative to

Using Multiple SSH Keys with Github Jailbreak Your APIs Presented at Computers, Freedom
and Privacy Conference
Github uses the SSH key to identify a user. If you
have two users with the same key, Github cannot determine the Video of this: http://new.livestream.com/i...
user ;)

Subscribe d Add Disqus to your siteAdd DisqusAdd Privacy

Copyright Irakli Nadareishvili. All rights reserved. 2004- 2017.

http://www.freshblurbs.com/blog/2015/06/25/api-representations-prefer.html 08-Jun-17
@inadarei: Creating Client-Optimized Resource Representations in APIs Page 5 of 5

Irakli Nadareishvili
"There are only two things you need to
be a great programmer: curiosity and
kindness. Everything else you can learn
over time. Everything." ~ @inadarei

Press & Talks

Linked APIs

Node.js Goodness

GoF Design Patterns

OS-X Georgian Keyboard

http://www.freshblurbs.com/blog/2015/06/25/api-representations-prefer.html 08-Jun-17