While It’s Compiling is a continuing series of interviews with experts across a range of bleeding-edge technologies and practices, exclusive to Skills Matter. Be sure to subscribe to this blog for future interviews, or follow us on Twitter.
Find out who we’ll be interviewing next, and get a chance to put your questions forward with the hashtag #whileitscompiling.
Brian Sletten is a software engineer with a focus on forward-leaning technologies. His experience has spanned many industries including retail, banking, online games, defense, finance, hospitality and health care. He focuses on web architecture, resource-oriented computing, social networking, the Semantic Web, data science, 3D graphics, visualization, scalable systems, security consulting and other technologies of the late 20th and early 21st Centuries.
We caught up with Brain ahead of his REST and Resource-Oriented Architecture Bootcamp, which will take place at Skills Matter on the 29th September in London.
This is a common oversight and I think it largely stems from the fact that people think of REST in terms of services because it was always couched in opposition to SOAP. In reality, the resource abstraction is a superset that allows us to uniquely identify documents, data, services and concepts. In this sense, ‘doing’ REST is really just extending the patterns and concepts from the Web of Documents into a Web of Data via a familiar set of architectural constraints.
Incidentally, the exact same ideas will extend to the Web of Things as well. It’s all just The Web. The confusion about the Semantic Web is generally that people think it is something other than an enriched form of the Web we already have. They think it is something we are waiting for. If you re-read Tim Berners-Lee’s original proposal, you will see that the notion of conceptual things (authors, topics, etc.) were very much part of the plan for solving information management and exchange within a single organization. We just zeroed in on the presentation part of the plan.
A major benefit of the Web is letting people request information in the way that they want to. The existence of RDF doesn’t have to mean we stop using JSON, XML or HTML. It comes down to what kind of developer experience you want for your API, where you want to spend your integration effort (or force your clients to) and how you plan for a future that you may not control. These aren’t incompatible ideas and we understand how to adopt them incrementally and in defensible ways. There is absolutely nothing Ivory Tower here; these technologies are grounded in a very pragmatic reality.
So often with software change, it comes down to how well people adapt. How can engineering teams begin to apply Resource Oriented Architecture (ROA)?
It’s aggravating when I see organizations try to mandate top-down standardization of service infrastructure, even if it involves a move toward REST. They will never get an effort like that right no matter how earnest or heavy-handed they are. The Web did not grow in a top-down fashion, nor could it. It was an organic process that grew around standards. The trick to adopting these ideas is just to adopt them incrementally. There is obviously more to it than that, but once you get a clear picture of how to think about resource abstractions it is easy to identify a single piece of information to expose in this way.
In subsequent iterations or releases, you can pick more. As the organization starts to internalize successful resource-abstractions and APIs, it can begin to parallelize the effort and to identify useful tools and approaches to share between teams. If done properly, you will accumulate a series of reusable resources that have lifetimes well beyond what you originally conceived.
Different teams can pick different implementation technologies, but that is a feature, not a bug. You don’t have to go all-in on a single technology stack. You can allow a competition of ideas to occur behind-the-scenes without significant impact to the organization. That same freedom allows you to adopt new technologies without impacting existing systems. Letting things grow organically doesn’t mean initiating chaos.
Having resources work consistently is part of the plan because that is what the Web is. Your browser does not need to read API documentation to interact with a new site it has never seen. The goal is to produce resources with a Uniform Interface, no matter what they are. Don’t overthink the adoption process, just don’t underthink it either.
If there was a simple way to get to the crux of ROA and it’s capabilities, what would it be?
Fundamentally, one of the main problems with large scale, distributed systems is that consensus is pretty much impossible. It’s great when you can achieve it, but that tends to be in smaller communities of interest, teams and co-located business functions. Beyond that, it can be a Fool’s Errand. This is true in terms of world view, technology selection and service and system lifecycles. Any technology or initiative that assumes or requires consensus is unlikely to succeed.
The fallacy of large initiatives like Enterprise Data Warehouses is that there is a single view of information within an organization. Different people have different needs of information in different contexts. This is why so many people ETL (extract, transform and load) data. Absent agreement, they simply copy the data and frame it how they want for their needs.
This of course yields new problems of data consistency. Even powerful technical solutions like Big Data don’t ultimately solve the problem of consensus. What we need is an infrastructure that allows us to agree to disagree. REST alone does not solve this problem, but it goes a long way toward allowing interoperable systems that don’t leak implementation details and choices. It gets organizations to think in terms of resources, allows for content negotiation, independent system life-cycles and technical evolution. In this context, applying the standards associated with the Semantic Web allows us to exchange information with people we have never spoken with. It affords us the ability to achieve collaboration without coordination. This is a ludicrous concept to people who have attempted large-scale data integration in the past, but it is possible because of a few simple building blocks that solve fundamental problems.
People are among technology’s biggest problems, but we can also apply technology to solve the problem of people.
At the heart of semantic web is the aim to standardise data into a digestible form, or rather a ‘web of data’. Could you give an applied enterprise example of how this has worked?
To some extent, these ideas are as widely applicable as the Web has proven itself to be (if not more so!). I have worked with all manner of organizations (big, small, commercial, non-profit) in adopting these technologies. Closed systems that have the freedom to mandate structure can certainly be built without these approaches, but they will never be able to exchange information with the rest of the world as freely and easily as they might need to.
As I have said above, achieving consensus remains the biggest obstacle in most data integration initiatives. Even if you are able to extend your closed system to adopt a single new content source, that approach is not one that scales efficiently or cost-effectively. I have had success at applying these ideas by allowing data integration to occur that would have been too expensive to even consider under normal technical frameworks.
One customer was an organization that sold hundreds of thousands of products. They had their own way of thinking about and describing their products but they needed updates from vendors as new offerings become available. They didn’t have a large internal database of product ratings and reviews so they subscribed to this information from third parties. There were additional data sources that tracked things like parental guidance on product suitability, related accessories, suitable product replacements for out-of-date inventory, etc.
Imagine trying to manage all of that in a world that changes constantly including new category-busting products like the iPad and you’ll get a sense of the issues involved. Trying to standardize on a single world view encoded in a single schema or object model would simply not work. But, a relatively small team was able to adopt resource-oriented thinking and Semantic Web-oriented standards to allow it all to connect with minimal effort.
It suddenly became possible to query the customer’s data, sorted by rating and targeting certain kinds of customers. Information could be accepted from vendors and partners in the forms they currently used with only a minimal mapping effort to make it accessible to the standards we were using. Re-categorizing products took a matter of minutes or hours, not days, weeks or months. In addition to lowering the cost of integration, this allowed us to explore long-tail products in different contexts in ways that impacted the bottom line.
How can adopting resource-oriented technologies minimise business<>technical tensions?
Reducing this tension is one of the main consequences of using these technologies. Business should be in charge, not technology. The people who understand markets and customers and opportunities and threats should be able to imagine initiatives and run with them, not be limited by what the technology currently allows. Relational database systems are fantastic tools for processing information that we understand, but they are terrible for modeling the unexpected.
XML documents are great for having a validating structure of agreed upon syntax, but are less useful for content we’ve never come across before. JSON is a suitable tool for developer-friendly exchanges of data, but integrating across data sources requires custom development. Big Data systems are great for dealing with volumes of data that have previously been untenable, but at some point we cannot put everything into a single container.
Software deployment has gotten much easier, cheaper and safer with the adoption of testing, continuous delivery, dev ops, etc. but sometimes that still isn’t the right solution for a problem of perspective. How can we look at things differently? What happens if we allow this new information into our system?
The resource-oriented abstraction allows us to think about our worlds independent of how we choose to build systems to store information about them. Technical change is business cost. If adopting a new world view or allowing new information into the system has a high cost attached to it, businesses will simply not be able to justify the cost of implementation. We are stuck in the world of what technology makes easy or even possible.
Resource-oriented technologies do not eliminate these existing systems or the value they provide. They simply allow us to operate at a level above them and provide integration opportunities that shield clients from unnecessary details and drive down the costs of integration. This rebalances the relationship between business and technology by allowing cheaper experiments and longer life-cycles for what we have already produced. This isn’t some kind of techno-utopian fantasy. It is possible to adopt these ideas today, incrementally and driven by business value and priorities.
Brian Sletten’s REST and Resource-Oriented Architecture Bootcamp
Brian will be running a three day course at Skills Matter from Monday 29th September, which will provide a broad, example-driven and compelling vision of computing’s future.
While learning how the technical and business value of Web semantics is available and useful today behind firewalls and on the public Web, the course will also help you understand how tensions between the business and technical sides of the house can be mediated by adopting resource-oriented technologies. These provide information-focused, business-friendly solutions that grow with the organisation and its changing business needs.