This is an introduction for an idea of mine I’m calling Lift Delegation, a feature that I’m hoping to implement in the Lift Framework that would allow Lift applications to be implemented in a master/delegate model with a small amount of state and content from the master made available to the delegate application. So, what does that mean?
Loosely speaking, this feature has a few principal goals:
So, that’s a lot of jargon all at once isn’t it? If you’re already on the “this is awesome” train skip down to the specification section. If not, let me take a minute to explain the cases where I think this would be useful.
I’ve noticed a lot of cases where there are web applications that are in actuality application systems or a system of multiple user-facing applications that are functionally distinct but strongly related by a usage context. The classic example of these are personal health insurance portals like those provided to customers by companies like United Healthcare and Blue Cross Blue Shield. If you visit such a portal you’ll first land on a login page that has its own url. After logging in you’ll be redirected through a series of redirects that land you at your portal homepage which oftentimes has a distinct domain from the login system. Then, if you click the links to get pricing information on perscriptions you’ll typically be bounced to yet another domain that is run by the prescription insurance provider (often not the same company as your Health Insurance proper).
These are several different applications that the insurance companies rightly determined are closely enough related by their usage context that transitioning between them should be as seamless as possible. They are, as such, an application system.
I typically see these application systems take one of four forms:
There are, of course, problems with all of the above. Both items (1) and (3) above often tend to provide a substandard user experience. Visual style differences are often jarring. Flows that require the user to jump from one application to another are often not optimized for quick access, sometimes resulting in going through ten pages or more to get to a receipt for a recenty prescription refill, for example. There are some cases where these options may not be avoidable, such as when a single company doesn’t have control over all the applications in the application system, but this arrangement is far from optimal.
Most engineering teams know that bouncing from one application to another is a pain, so it’s not that uncommon for them to opt for taking road (4) and building a single, larger application that just does everything their user needs to do. When companies take this route they’re not entirely misguided. Using a single application certainly makes sense if your application system is still evolving and the individual modules are not yet well specified enough or complex enough to warrant the configuration overhead involved in a different arrangement. It’s also much easier to ensure implementation, security, and design uniformity across a single application. However, if your application continues to grow in complexity, you may find that you have issues with changes in one module having unintentional side effects in other modules, that it’s an issue to have all the various modules tied to the same release cycle, or any number of other problems that come up with a large, monolithic application.
Finally, (3) is perhaps the closest to my ideal solution that I’ve seen in the wild to date. In this one, the actual backends are separate and there’s a single application that is unifying the backends and appearing to be one application to the user. The problem here is that you’ve completely decoupled your implementation of the display of your application in a user’s browser from your implementation of the application’s guts at a repository level. I don’t think that this decision is inherently bad and makes sense in many cases, but also comes with its own set of challenges. For example, at times the loose coupling between the backend and the UI may not be a good thing because they are not, in fact loosely coupled. We oftentimes think of coupling as purely a code organization help, but if the truth of your application is that it’s likely by changing a backend component you’ll need to change your UI you’ve still got tight coupling. Your code and project organization just no longer reflects that and you have to deal with that truth as you roll out new functionality. Those things have to stay synchronized and that can involve extra human awareness on an operations level.
So having taken stock of some existing examples of this, what, exactly, am I proposing as an alternative?
HTTP servers have, for a long time, had the concept of a reverse proxy, which retrieves resources on behalf of a client from one or more backend servers. My suggestion is essentally to move that concept to the web application layer itself. However, a pure reverse proxy wouldn’t exactly suffice here for a few reasons.
First, Lift is stateful. With the way we use that state security is a bit easier for us. We can give the client much less information than we might otherwise be required to in order to maintain information about who the client is and what they’re doing. It stands to reason, then, that if you have a larger system of applications that all integrate seamlessly that there might be some state that you want shared between the master application, the one that’s at a public url that the user can interact with, and the delegate application, the backend that the master interacts with on the user’s behalf. Comets, AJAX handlers, and everything else that depends on state to function correctly should be usable from the delegate application as if it was a standalone Lift application interacting with a user.
Second, it’s highly likely for a complex application that there will be a large set of common resources that should be shared amongst all the delegates. I’m thinking common JavaScript libraries, the common page templates, authentication state, and other things. You could, in theory, separate all these things out into some common library that the delegate itself depends on, but then you’re kind of breaking the single-concern principle we’re striving for.
For the reasons above, I think that this type of reverse proxy application system is best implemented by allowing two-way communication between the master and the delegate application. All of these abstracted over the concepts that already exist in Lift so that, ideally, the only places where delegation matters to the user code in the delegate application is when they’re authoring their Boot.scala file that sets up the LiftRules for their application.
In the next section, I’m going to go into exactly how I see this working out on an implementation level.
I think that such a delegation architecture, as mentioned above, could be implemented with a few specific changes to the internal workings of Lift. Specifically, I think we’d be looking at the following API changes to the framework:
LiftRules.delegates of type ()=>Box[Map[String, String]] that produces a map with the short names of delegates as the key and the hostname and port that those delegates can be accessed from as the value. If Empty, the default value, delegation is disabled. This will be defined in the master application.LiftRules.masterApplication of type ()=>Box[String] where the Box[String] contains the hostname/port of the master application. This will be defined in the delegate application.LiftRules.delegateName of type ()=>Box[String] where the Box[String] contains the name this delegate will be referenced from according to the master. This will be defined in the delegate application.Delegated param for SiteMap’s menu entries. Semantically, this would tell the sitemap that everything under a particular URL is delegated to a different delegate Lift application and provide the identifier so that the actual URL can be pulled from LiftRules.delegates.Loc to provide a delegatedResponse method. During a normal request the processRequest method would invoke this method before invoking earlyResponse. If the Loc in question has Delegated attached to it, it would result in the Framework issuing an identical request to the delegate application. It would return the response from the delegate to the user once the delegate finishes doing its thing.comet_request and ajax_request url generation code such that if the Comet or AJAX request was created from within a delegate application, its URL will take the format (comet/ajax)_request/[delegatename]/[identifier]. The master application would continue to use (comet/ajax)_request/[identifier] for its comets and ajax elements.MasterSessionVar that behaves identically to a normal SessionVar excepting that its value can be retrieved by a delegate using a DelegateSessionVar of the same name. Whatever is stored in this Var must be JSON serializable. You may be required to provide a Formats object lift-json can use to write it out to the wire. The DelegateSessionVar will probably exhibit the same lifetime as a RequestVar so that it doesn’t constantly have to go fetch the value.lift:embed snippet provided with Lift to allow it to look in the master’s webapp directory for templates.Most of this is fairly easy to do with simple HTTP request proxying. I think that (7) could be tricky with regard to long polls not occupying threads. With regard to (8) and (9) I think these will best be serviced by an internal REST API that the master and delegate use to talk to each other. The endpoints will be exposed on the master application and probably look something like this:
DelegateSessionVar should have.I expect there will be some good and bad things about this architecture. It will certainly increase application complexity a bit. But I think that for applications that need it, this architecture will be able to provide a superior user experience to the existing alternatives.
There are a few things that I still haven’t quite figured out. Including:
So this is my official request for comments on the above. I’m interested in hearing what the Lift community (or the tech community in general) thinks of the proposal I’ve made. I fully acknowledge that it’s far from done, but I hope that it’s something that people are interested in seeing done.
If you’re interested in contributing feedback, please contribute on the Lift mailing list thread. That’s where we do all our discussion and, as such, comments on this post are disabled.
Thanks for reading and let me know what you think!