Are Mobile Apps becoming Fat Clients?

Are Enterprise Mobile Apps becoming Fat Clients?

Improper Application Partitioning was an anti-pattern identified in the client/server days, but it?s repeating itself in today?s mobile world not just for enterprise but even for consumer apps. And that makes it even worse. Here?s how to fix it.

Why do we keep doing this to ourselves?

The term ?Fat Client? captures the negative implications of writing code on the client that should be on a shared server. The term was coined long ago in the days of client / server, so you might think this issue would be a thing of the past. Sadly, it?s not.

As I speak to mobile client framework vendors, a common complaint of customers is performance issues stemming directly from improper partitioning. Frustrating ? client frameworks can be great, but they cannot overcome poor architectural choices.

This is an even bigger issue for Mobile apps than Web apps, due to higher latency and lower bandwidth. So more than ever, we need to pay close attention to partitioning to reduce the number of calls and data transfer amounts.

So why is this happening? The forces pushing us in the wrong direction are real, and formidable:

Time ? every project faces time pressure, whether it?s looking to adhere to an Agile Process to produce ?working software?, or focused on a Minimum Viable Product (MVP)
Complexity ? in many architectures, proper partitioning means we need to create servers, learn new languages, and client / server communication approaches. There?s not just a lot to do, there?s a lot to learn.

So, we focus on just getting the app to work ? we can always address performance later.

But we really can?t ? if we coded it wrong, it?s a lot of code, spread over a lot of apps.

And it?s not just performance. Improper partitioning can easily affect integrity and security, make our logic unavailable to other systems for integration, and introduce redundant integrity logic that is difficult to maintain.

In this article, we?ll look into

How to spot improper partitioning
Key objectives for partitioning
REST as a key technology, but key API design objectives must still be designed
Retrieval considerations including Pagination, Row Level Security, Join Strategies, Multi-database considerations
Update considerations including Optimistic Locking, Batched Updates, Generated Keys, Transaction Business Logic, and Refresh for client displays
Final thoughts on strategies for driving proper partitioning across your organization

How to spot improper partitioning

The most immediate manifestation of improper partitioning is performance. It might show up on retrieval:

App is fast on small databases, but slows down on large result sets
App is fast on laptop, but slow when running in production staging

Performance issues might also show up for update:

Updates are fast for laptop development, but reveal excessive network traffic in production staging (this might also show up as excessive costs for usage-based servers). Ideally, a ?save? operation should be 1 request, not many.

More subtlely, look for

long QA cycles ? lots of bugs detected in corner cases,
integration issues ? inability to respond to needs for other systems, or partners, to retrieve or update data due to lack of SOA services

None of these problems can be addressed by improved client code. Instead, business logic that should be in the server, but was implemented in the client, needs to be redone.

Partitioning is the Solution ? Key Objectives

The solution to these problems is proper partitioning of logic between the client and the server. The server has to do its fair share?

The general objective is to minimize latency by reducing the size, and, in particular, the number of data access calls. We do this by:

Introducing a REST based App server
Place it close to the database to minimize latency, and
Architect an API that reduces the number and size of network calls from the client to the server

Note this also enables us to separate concerns, so the work can be split into 2 teams. These teams can work in parallel, coordinated by the API. The server team creates the API and the business logic, the client team consumes it.

So where do we look for this business logic that needs to be shared? Depending on your architecture, we are talking about:

JavaScript code in your mobile/web app
Native iOS / Android / Windows Phone code
View Model code in a .NET app
Controller code in a Java Web app

REST is good, but not enough

But ?proper partitioning? is just a vague guideline? let?s explore it in greater detail.

A good start is a RESTful server. This can address our first 2 objectives: introduce a REST server and place it close to the database.

But REST alone is not enough. In the degenerate ?SQL pass-through? case, you can simply replace ODBC/JDBC calls with REST calls, with the logic still on the client. We also need to address the 3rd objective: an API that minimizes the number and size of calls. And that?s the hard part.

Let?s explore some key aspects of proper partitioning, both for retrieval, and for update.

Retrieval Considerations

Update is often more complex than retrieval, but most systems do far more retrieval than update. So it?s important to optimize retrieval. We will look at the following issues as it relates to retrieval:

Pagination
Security
Joins
Multi-type Requests
Multi-database Support

Pagination

The first thing to look for is provisions for large result sets. In the worst case, they are not provided (again, often due to time pressures). Such issues often show up only when you scale out testing to large databases.

If provisions for large result sets are provided, you will probably observe meaningful amounts of code. You might be able to factor this out into the shared server. This might be a way of saving significant amounts of client code replicated in multiple apps.

For example, imagine your server is RESTful, and a data retrieval response provides 2 services:

automatically deliver only (say) 20 rows per request, and
provide a URL that, should the client scroll, can be sent to the server for the next 20

Security

Most databases contain data that?s not appropriate for everyone to view, or update. Watch out for client logic that provides security filtering to limit the rows/columns shown on the form. This results in returning too-large result sets that could have been filtered on the server. This is both inefficient, and a massive security breach.

Such logic should be moved to the server, which can both reduce traffic, and be properly hidden from clients.

Joins

Consider an app that wants to deliver a Customer Account ?business object? to a client: a Customer, their Orders, and the OrderItems in the Order including their Product information. How should the client pose the query?

Beware of client code that issues a retrieval for each OrderItems to obtain Product information. This is an order of magnitude slower than a single request that enables the join to be processed in the database.

Note this could be tool-induced: if your API is strictly SQL base tables, with no way to express the join, you should probably re-evaluate your technology choices.

If you are used to a relational database, you might join the three objects together. That works, but it places some burden on the client to re-compose the objects that were joined together.

Most queries require data from more than one table or collection. In many cases, they might even span servers.

Joining data on the client is not only more client work, it requires multiple trips to the databases. Again, a server can join the required data in a single request, making intelligent choices about join strategy.

Multi-Type Requests

Performance is well addressed by the join strategy, but it?s not apt to make you any friends in the client dev team. It?s rather a bore to break apart the joined data into the separate objects.

But it?s worse. The join approach breaks down if we introduce what appears to be a modest requirement: let?s also show the Customer?s Payments, along with their Orders/OrderItems.

Relational joins make rather a mess of this: joining a customer to 5 Orders and 10 Payments results in 50 rows. It?s quite unreasonable for the client to process this.

It?s an inherent limitation of restricting results to ?flat? rows. You cannot address this using a RESTful server that is simply a SQL pass-through.

You?ll need a middle-tier server, e.g. RESTful one, that delivers ?nested document?responses, like this Customer JSON:

{

?ID?: ?ALFKI?,

?Balance?: 0,

?Orders?: [

?OrderID?: 10643,

?Total?: 814.5,

?OrderItems?: [

"ProductID": 9004,

"Quantity": 2,

etc?

]

?Payments?: [...]

}

Which is why so many mobile developers love NoSQL databases. Most NoSQL products return data as nested JSON documents, a natural way to express this problem.

If your data is in SQL, look for frameworks / products that not only produce RESTful responses, but also include nested-document support. Avoid SQL pass-through approaches. You?ll probably want to do this a lot, so ideally it?s simple.

Multi-Database Support

Separating logic to a server also offers the opportunity to integrate data from multiple databases within a single API. This simplifies client development.

Such integration requires careful consideration of performance. For example, imagine we retrieve an Order and its OrderItems, joining Product information from another database.

A poor approach would be to invoke the Product database once for each OrderItems. Better is to combine this with pagination: if we retrieve 20 OrderItems, we can send 1 query to the Product Database for the 20 Products. That?s over an order of magnitude fewer network calls

Update Considerations

There are host of issues to verify for proper partitioning of update logic. Here are a few of them:

Optimistic Locking
Batched Update
Generated Key Propagation
Transaction Logic
Refresh

Optimistic Locking

It is a long-established anti-pattern that locking data on retrieval is a terrible idea. Somebody walks away, and others can?t see the data that was left locked. This will manifest as delays, even though the database and network traffic is low.

Optimistic locking strategy is therefore common, wherein the data is not locked on retrieval, so part of update logic is to ensure it has not been altered since retrieval. But how?

The simplest strategy is to provide a time-stamp on each row. This works, but results in a rather messy database design. An alternative is to compute a hash of the retrieved data, and ensure the hash of the current disk contents is the same on update.

In either case, this is a fair bit of work, required on every update button. It?s a great idea to make this part of your server, ideally as services built-in to your framework so you are certain they?re used across all apps.

Batched Update

A good user interface enables users to make many changes, and submit them all with one save button. This should result in one server request.

And the size should be dependent on the data actually changed. It should not be necessary to send the retrieved data that was not changed.

Generated Key Propagation

Batched updates are not as simple as they might sound. Consider adding an Order and a set of OrderItems, and that the Order?s key is generated by the database. It needs to be put into each OrderItem as the foreign key.

It?s a common solution to start a transaction, insert the order, get the key, and then send the OrderItems with the generated Order Number. But this doubles the number of trips we make.

It?s often not so easy, but the server should be performing this work, so the client can issue one call for the Orders and the OrderItems.

Transaction Logic: slow, error prone, redundant

Let?s consider the logic to insert a new OrderItem:

Get the Price from the Part
Adjust the Order total
Adjust the Customer balance

Putting all that logic in the client results in 3 retrievals. That?s going to run slowly. It should be only a single call to the server.

Putting the logic in the client also reduces system quality. For example, the Order Insert button might remember to compute the tax, but that logic might not be in the Order update button.

Update logic in client code is a clear violation of responsibility. It should be encapsulated into the database (server) where it can be debugged once, and changed once.

And it paves the way for system integration. Logic in buttons is unavailable to peer computers seeking to integrate corporate systems. Such integration requires an SOA architecture, with logic shared / enforced in the server.

Refresh

Often overlooked is what happens after an update. In many cases, the server has updated data visible on the screen. Users need to see this so they know the update ?worked?. For example, I reduced the number of items I am buying ? I should see my order total and balance reduced.

The usual technique is to requery the data. But that?s exactly what we want to avoid ? another trip to the server.

Ideally, the update response includes a list of all the triggered updates, so the client can refresh the display without an additional tri

The Way Out: Make Right Easy

Old hands will recognize these as well-known architectural patterns for database applications. These have been documented countless times, across thousands of projects and organizations.

And still they are not followed.

That?s because they take time, and teams are invariably under time pressure to deliver. And we?ve identified a lot of work to build using a framework, and missing in ?Sql Pass-through? products:

Pagination
Server-enforced Row Level Security
Database Joins
Nested-document results
Multi-database optimizations
Optimistic Locking
Batched Updates
Generated Key Propagation
Server-side Transaction Logic
Update Refresh

Since time pressures usually win, we cut corners. It?s an uncomfortable reality that we all have to live with.

The only real, effective solution: make the right way, the easiest way. Imagine a RESTful server that provides point and click definition of:

Foundation Services that are simpler than ?add your code here? frameworks, more powerful that ?SQL Pass-through?, to address all of the items discussed above. That enables clients to depend on their presence. A good approach would be point-and-click creation of RESTful, Nested, Multi-database APIs.
Business Logic alternatives that are dramatically faster and easier than writing network intensive client code. A good approach would be server-side events. A much more promising approach is for declarative rules to provide most of the logic, with events as required.

We take these objectives as our mission here at Espresso Logic. We?d love to hear your views.

Val Huber, CTO, Espresso Logic, Inc.

article