article

RESTful APIs are required technology for mobile / integration apps, but their additional cost may not be warranted for web apps.  The problem is, mobile / integration requirements may emerge later, and the system is not architected to respond.

But, can we imagine a technology that makes it faster and simpler to create APIs than building monolithic apps?  Such a technology would be a game-changer for the economics of architecture.

Business Driver: the Need for Speed

Industry Leaders like CA have pioneered the vision of  a Software Driven Business.  They argue persuasively that strategic business advantage lies in Time to Market and Time to Decision:

?reveal the need for speed in the application economy.  As companies transform into software-driven enterprises, bringing high-quality applications to market faster becomes one of the most critical differentiators.?

We are all painfully aware of Business User frustration about how long it takes to create and revise systems.  They see problems that look nearly as simple as a spreadsheet take weeks? to months.

How can it months for IT to build a system that takes days on a spreadsheet?

Technical Reality: APIs are right? but expensive

Against this backdrop, consider architecting a web project.  It?s well-known that a RESTful API provides robust, network-oriented database access.  Such an architecture puts the organization in great shape for future requirements such as mobile or integration.

But a REST API is far more than basic data access.  A ?SQL Pass-through? ? simply restifying SQL data ? does not meet  Enterprise-class requirements to scale, integrate and enforce:

• Scale ? APIs require Pagination to address large result sets, Nested Documents to reduce latency, Optimistic Locking to ensure concurrency.  These are not provided in a simple SQL Pass-through ? you must program them, by hand.
• Integrate ? a wizard can produce an API from schema objects, but it cannot address multiple databases, or integrate non-SQL data sources such as ERP, other RESTful services, or NoSQL.
• Enforce ? an API needs to enforce our security (down to the row level), and the integrity of the data.  These are significant tasks, which are sadly often placed in client buttons where they cannot be shared.

Providing these Enterprise class services takes significant time, expertise and expense.  So we have a classic collision between ?right? and time to market:

We know REST is a superior approach.

But time pressures drive us to conventional approaches.

Fat Client: Logic (and cost) In the Buttons

Time pressures don?t stop at the API, they drive right into app design.  It?s more cumbersome to centralize integrity and security logic into services, so we simply place in the the ?buttons? (UI controllers).  This is sometimes called Fat Client (not to be confused with rich client), an anti-pattern:

• Fat Client replicates logic, leading to errors.   For example, placing a new order might check credit, but the button for ?choose different product? might not.
• Fat Client can reduce performance with additional database calls that should be issued closer to the database.
• Fat Client logic cannot be easily extracted into a future service for access by integration or mobile apps; a rewrite is typically required.
• Fat Client logic is not shared between heterogeneous web client technologies.

Institutionalizing the problem

Nonetheless, most organizations are forced to respond to the time pressures, accepting that business requirements override technical correctness.  It?s hard to argue this is wrong.  It?s probably the right business decision, given the current technology landscape.

Some organizations, accepting the inevitability of time vs. quality, go so far as to institutionalize the problem.  The define 2 architectural standards:

• Departmental: for small, tactical projects, use familiar RAD tools, and get the job done fast.
• Enterprise: for large, strategic projects, use a different approach (e.g., J2EE) to drive a Service Oriented Architecture

This is quite dreadful.  We accept that time and quality are inevitably in conflict, and so continue creating sub-optimal systems that face mobilization / integration / scaling problems, supporting multiple technologies (learning curves, staff specialization), etc.

But what if doing it right were just as fast?

In fact, what if doing it right actually saved money?

What?s needed is technology with the power to eliminate the time vs. quality tradeoff.  This has the potential to change the economics driving software architecture.

Making ?right? easy: Instant Enterprise REST

Such technology exists.  We call it Instant Enterprise REST.  

To enable the Software Driven business, we need to raise the level of abstraction from coding to business.  We need Business Driven Software.

It consists of 3 core technologies:

1. Enterprise Pattern Automation ? creates APIs that with Enterprise-class scalability built-in (pagination, nested documents, optimistic locking, etc)
2. Declarative ? specify your API, integration and enforcement policies with spreadsheet-like rules in a simple point-and-click UI
3. Extensibility ? enables the RESTful APIs to invoke your existing logic, inside or outside the JVM, via standard server-side JavaScript.

The combination of these 3 technologies enables you to create RESTful APIs for database backends ? half your system ? 10 times faster.  Let?s briefly examine them below.

Technology 1: Enterprise Pattern Automation

There are well known patterns in the data domain, describing data structure and access via SQL.  There are also well-known patterns for managing SQL data in the context of RESTful services.

Well known patterns can be automated.  Let?s imagine a service (say, a server accessed via a browser) that automates these patterns, as described below, just by connecting the service to a database:

• Schema Discovery ? tables, views, stored procedures: The system creates a complete (default) API for each schema object.  Note this includes Stored Procedures, which often represent a significant investment.
• Enterprise Pattern Automation: the resultant API provides well-known services for Filter, Sort, Pagination, Optimistic Locking, handling Generated Keys and so forth.

So, the service has provided a default Enterprise-class API, instantly.  So, literally seconds into your project, you can test your running API:

Not enough, not done, but a great start.

Technology 2: Declarative

Declarative is the key (?what, not how?).  It has had striking impacts on domains where there are well-understood underlying patterns.  Max Tardiveau has put it well:

Whatever can be declarative, will be declarative.

For example, spreadsheets are declarative ? and they gave birth to the PC industry.  And SQL is declarative ? itself an industry.  Two game-changers.

So, the challenge is to apply the spirit of declarative to REST integration and enforcement.  The stakes are high ? success can deliver breathtaking agility.

Declarative Integration: Multi-Database Custom API, Point and Click

Enterprise Pattern Automation provides a good start, but the API is not rich.  It is a flat, single-table API, really just ?restified? SQL.  What we really need is

• Nested Documents ? returning multiple types (e.g., an Order, a list of Items, and a list of contact names) in a single call can reduce latency (vs. a separate call for each type).  REST is perfect for this.
• Multi-database APIs ? a RESTful server provides the opportunity to integrate multiple databases in single call, shielding clients from underlying complexity.

Nested Documents are easy: define them by simply selecting tables (via a User Interface or Command Line).  Foreign Keys are used to default the joins.  Add the ability to choose / alias columns, and we?re on the way to a pretty good API.

But what about databases that have no Foreign Keys?  Or multi-database APIs?

Leveraging the schema does not mean we are limited to it.  All we need to do is:

• Provide a means to define ?Virtual? Foreign Keys for the service (i.e., stored outside the schema)
• Extend this to Foreign Keys between databases

We now have a rich, multi-database API.  Defined declaratively as shown below, no code required, running in minutes, ready for client development:

 

Declarative Enforcement: Integrity Logic, with spreadsheet-like rules

So now consider enforcement, specifically database integrity.  A very significant portion of any project is the multi-table validations and computations that define how the data is processed.

?Your code goes here? means, well, a lot of code.  We need a more powerful, more declarative, paradigm.

In a spreadsheet, you assign expressions to cells.  Whenever the referenced data is changed, the cell is updated.  Since the cells references can chain, a series of simple expressions can solve remarkably complex problems.

What if we did the same for database data?  We could assign derivation expressions to columns, and validation expressions to tables.  Then, the API could ?watch? for requests that change the referenced column, and recompute (efficiently) the calculated column.  Just as in a spreadsheet, support for chaining and proper ordering is required and implicit.

To address multi-table logic, such expressions would need to address references to related tables.  It?s only at this point that the logic becomes seriously powerful.

Let?s take an example.  To check credit in a Customer  / Purchaseorder / Lineitem application, we could define spreadsheet-like expressions such as:

There is actually a sub-branch of declarative that addresses this: Reactive Programming.  Here it?s declarative, since you don?t need to code a Observer handler.

The result is that the logic above can be fully executable.   No need to code Change Detection / Change Dependency ? it?s invoked and enforced automatically by the API in reaction to RESTful updates.  SQL handling is also implicit, including underlying optimizations (caching, pruning etc).

The impact is massive ? the 5 expressions above express the same logic as hundreds of lines of code.  That?s a massive 40X more concise.  Game changer.

And quality goes up, since the rules are applied automatically.

Declarative Enforcement: Security, filter expressions for role/table

We can provide an analogous approach to security: define filter expressions for roles (like SalesRep), so that when a table is accessed by the role, the API adds the filter.  That way, a user with that role sees only the rows for which they are authorized.

 

Technology 3: Standards-based Extensibility

Declarative is great, but you?re probably thinking ?ok, but you can?t solve every problem declaratively?.  And you?re dead right.

Business Value requires that we integrate a declarative approach with a procedural one that is familiar, standards-based, and enables us to integrate existing software.

Automatic JavaScript Object Model

The first phase of many projects is to build an ORM for natural programmatic access to data: JPA, Hibernate, Entity Framework.  It?s not a small project, and cumbersome to maintain as changes occur.

In fact, the Object Model can be created directly from the schema.  So, you?d have an object type for Purchaseorder, for Lineitem, and so forth.  The model provides access to attributes and related data, and persistence services.  You could then use it as shown below.

JavaScript seems like the best language choice: reasonable across technology bases (everybody uses JavaScript), and its dynamic nature eliminates code generation hassles.

JavaScript Events

In addition to accessors and persistence, the JavaScript objects are Logic Aware.  That is, the save operation above executes any rules associated with OrderAudit (e.g., updated-by), and JavaScript Events.

Here is a sample event for the PurchaseOrder object, where you access the JavaScript Object Model via the system-supplied row variable:

 

Extensible Logic

Auditing is a common pattern.  It should be possible to solve this once in a generic manner, then re-use it (e.g, to audit employees, orders and so forth).

So, Instant Enterprise REST should enable you to provide Extensible Logic ? load your own JavaScript code, and invoke it.  So, the code above could become:

MyLibrary.auditFromTo(orderRow, "OrderAudit");

where auditFromTo creates an instance of OrderAudit, sets the foreign key, sets like-named attributes, and saves it.

Pluggable Authentication

Most organizations have existing data stores that identify users and their roles, such as Active Directory, LDAP, OAuth, etc.  Security should integrate with such systems as a function of enforcing row/column access.

Standard deployment

Finally, the system should deploy in a familiar manner: available on the cloud, or an on-premise virtual appliance or war file.  Standards also enable integration with related critical infrastructure, such as API Management, ERP Systems, etc.

Simple Client Connectivity

REST is a Web Service.  It is simply an HTTP call, which can be issued by any language.  The serialized response (the results) are typically JSON, which can be easily parsed in any language.

There are even standards for documenting REST, such as Swagger.  This has resulted in an ecosystem of value, which includes SDK generation for client artifacts, as a Java POJOs or C# POCOs.  This makes it easy to consume RESTful APIs in web apps.

Instant Enterprise REST: faster/simpler than ?fat client?

Instant Enterprise REST changes the economics of architecture, by creating important portions of our software in largely business terms, rather than technical terms.?