Netflix Public API - Rest in peace

Netflix decision to  retire its Public API may be based on its own business and IT strategy, however, since it is the front-runner  in Web API trend, this decision needs to be assessed in a broader sense.  Specifically, what this means to the enterprises API program and their vision to increase API exposure.

Web API is a rapidly growing trend, where Enterprises offer programmatic access of their data, services and resources to the developers: internal teams, external partners, or public third-party developers. Web APIs allow access to data and functionality, that is typically available via enterprises webapps (website),  to other consumers such as internal webapps, portals, mobile and B2B partners.

Having realized the value of APIs, through reuse, where a single resource endpoint can service various consumers, the Enterprises are now looking at ways to expand their API program to a wider consumer base. I think this expansion needs a careful thought, and can greatly learn from the Netflix API program, that had to shut-off one of its API consumers (the third-party developers). Netflix may get away from its decision by  upsetting a handful of developers, however, the mainstream enterprises cannot do that without negatively affecting their business relationship and bottom lines.

Every API consumer brings in some cost and complexity that impacts API design and manageability. This sounds counterintuitive, wherein, API design needs to be consumer-agnostic, and a well-designed API should serve any consumer. Again, looking at Netflix and based on my experience with API design over the  last few years, this expectation cannot be met easily.

In my experience, API design, inadvertently gets influenced by the consumers that it is initially developed for; a browser-based webapp, mobile, external partners, etc. The optimization needed for different consumers has to be handled at the API design level. On one hand, API need to handle strict security and policy controls for external users, while on the other hand it needs to  handle course-grained/auto-discovery for an internal webapp. To serve all types of consumers, API needs to stay fine-grained, but that may make the consuming webapp too chatty and result in performance issues. In reality, designing a single API that handles all optimizations for all its consumers gets cumbersome.

Netflix VP of Edge Engineering, Daniel Jacobson, in his blog  Why rest keeps me up at night, points out similar complexity when trying to design one-size-fits-all APIs. Here are a few extracts from his blog:

“Our REST API, while very capable of handling the requests from our devices in a generic way, is optimized for none of them.

That means that each device potentially has to work a little harder (or sometimes a lot harder) to get the data needed to create great user experiences because devices are different from each other.

Because of the differences in these devices, Netflix UI teams would often have to do a range of things to get around our REST API to better serve the users of the device. Sometimes, the API team would be required to extend the base service to handle special cases, often resulting in spaghetti code or undocumented features. "

The design solution to handle consumer-specific complexities are expensive, either consumer has to do extra-work, or services has consumer-specific rules, or an extra proxy (intermediary) is required to handle consumer-specific features.

At the end, adding API consumer has cost implications that need to be assessed against the business value of API expansion.

While Daniel Jacobson may have solved his problem by shutting-down Public APIs and is having a good night's sleep, some of us still need to find a better way to rest.