Monday, March 24, 2014

CompletableFuture for OSGi Remote Services

As some of you may know, ECF's implementation of OSGi Remote Services has support for asynchronous proxies.   This allows consumers of remote services to easily use asynchronous/non-blocking invocation to access a remote service.  For example...here's a simple service interface that's used in our remote service tutorial:

public interface ITimeService {
    public Long getCurrentTime();
}
As with any OSGi Remote Service, any consumer that discovers this service will potentially block when they call getCurrentTime(). This is just the nature of call/return semantics applied to remoting...and so will be true for any implementation of OSGi remote services.

ECF's impl of OSGi Remote Service offers asynchronous proxies, meaning that iff a related service interface is defined...e.g.

public interface ITimeServiceAsync {
    public Future getCurrentTimeAsync();
}

then consumers will be able to use ITimeServiceAsync to access the service...e.g:

ITimeServiceAsync tsa = ... get ITimeServiceAsync reference

Future timeFuture = tsa.getCurrentTimeAsync();
...do work...
Long time = timeFuture.get();

With ECF 3.8.0 we further added [3]...i.e. the ability to have ITimeServiceAsync be registered as a service automatically on the consumer...so that (e.g.) the ITimeServiceAsync reference can be injected via declarative services...e.g.

...ds component impl...

void bindTimeServiceAsync(ITimeServiceAsync tsa) {
     ...use or store tsa...
}

This [3] is all available in ECF 3.8.0.   Note that the service interfaces have absolutely no reference to ECF classes, nor to OSGi classes.   None are needed now.

As many of you certainly know...java8 just came out, and a big part of java8 is improved support for concurrency, functional programming, and lambdas.    This support for concurrency in java8 is potentially very useful for users of OSGi Remote Services.

Consider CompletableFuture, which as the name implies is a type of java.util.concurrent.Future.    It has some very nice properties for API/service designers...the main one being that it's not at all necessary to call Future.get directly...but rather you can write nice, succinct and *guaranteed to be non-blocking but asynchronous* usage such as:

CompletableFuture cf = ...get CompletableFuture....

cf.thenAccept((time) -> System.out.println("time is: " + time));

This is nice...it's completely non-blocking...and very succinct.  Also you can do very interesting things with asynchronous/event-driven chaining/filtering, etc., etc. All guaranteed to be non-blocking...which is a key guarantee for remoting.

Yesterday I realized that with Java8, our asynchronous proxies could be easily generalized to allow this:

public interface ITimeServiceAsync {
    public CompletableFuture getCurrentTimeAsync();
}

I made some minor additions to the ECF remote service implementation of asynchronous proxies and now this is working. What I mean by this is that consumers of an arbitrary remote service can now do this

...service component impl...
void bindTimeServiceAsync(ITimeServiceAsync tsa) {
       // Get the CompletableFuture...no blocking here
       CompletableFuture cf = tsa.getCurrentTimeAsync();
       // print out time when done...no blocking anywhere!
       cf.thenAccept((time) -> System.out.println("Remote time is: " + time));
}

Note a few things:
  1. There is no blocking anywhere.  This is true even though the actual time value is retrieved via a remote OSGi service
  2. The remote service host doesn't have to provide any implementation of ITimeServiceAsync.  It's constructed by ECF's RS impl automatically
  3. It's very easy to handle failure (e.g. network/io failure) via CompletableFuture.handle.  This is obviously a big deal for remote services...which are much more inclined to fail because of the network.
  4. No reference to either OSGi classes or ECF classes anywhere in host or consumer code
This is nice for remote service consumers, because it gives them strong no blocking guarantees, and can be very efficiently implemented (no additional threads) by remote service providers using asynchronous messaging (e.g. JMS, etc).   It also allows them to use all of CompletableFuture's APIs (see CompletableFuture javadoc).

This will obviously be part of ECF Luna...and I've created some test code (like above) that I intend to use to create another tutorial over the next month. Watch the ECF wiki for that tutorial.

The only drawback is that this does, of course, depend upon Java8...and so requires that both the remote service host and consumer use Java8, and that the distribution provider be enhanced slightly to use CompletableFuture.   Fortunately it's not technically challenging to make these enhancements, and we will make support classes available for those that wish to Java8 enhance existing or their own RS provider.

[1] https://wiki.eclipse.org/ECF/Asynchronous_Remote_Services
[2] https://wiki.eclipse.org/Tutorial:_Building_your_first_OSGi_Remote_Service
[3] https://bugs.eclipse.org/bugs/show_bug.cgi?id=420785

Labels: , , ,

Sunday, March 09, 2014

ECF 3.8.0 Released

ECF has just released 3.8.0.

New and Noteworthy












Labels: , ,

Tuesday, December 17, 2013

New tutorial: Creating a RESTful Remote Service Provider

ECF's implementation of OSGi Remote Services supports the creation of custom distribution providers.  Why would anyone wish to do this, when they could simply reuse one of the existing providers?   Here are some good reasons:

Service Backward Compatibility:   There are cases where it's not a new service being developed, but rather a new facade for an existing (e.g. web) service.  With ECF, one can easily create a custom distribution provider that reuses the existing service to expose it as an OSGi service.  This allows existing services to continue to be exposed as they were originally written, but also be exposed as an OSGi Remote Service.

Custom Transport Requirements:  Remote services also frequently have specific transport requirements...for example using MQTT rather than HTTP, or using JSON rather than XML.  With ECF, distribution providers can completely control these transport-level decisions.

Standardization and Interoperability:  ECF fully implements both the OSGi Remote Services (RS) and Remote Service Admin (RSA) standards.   One thing this means is that all ECF Remote Service providers (both those created by us as well as others) are automatically compliant with the RS/RSA specifications.   Also, Remote Service providers can be proprietary/closed source or open source as desired.

We've created this tutorial to show how a custom RESTful (HTTP+JSON) remote service provider can be easily built:  Creating a RESTFul Remote Service Provider




Labels: , , , ,

Monday, December 09, 2013

New tutorial: OSGi Remote Services

ECF team has created a new introductory tutorial on creating standard OSGi Remote Services

Building your first OSGi Remote Service

Our plan is to create a number of such tutorials on OSGi Remote Services and focus on additional topics such as using/configuring alternative discovery or distribution providers, creating REST-based providers, asynchronous remote services and others.  They will generally first appear here.

Labels: , , , ,

Sunday, October 27, 2013

ECF 3.7.1 Released

Wednesday, October 16, 2013

ECF 3.7

ECF 3.7 was just released.

Highlights/What's New

  • Servlet API -  for Creating OSGi Remote Services with HttpService and Servlets
  • OSGi Remote Service Examples
  • Testing Against OSGi R5 Compatibility Test Suite -  For Remote Services (chapter 100) and Remote Service Admin (chapter 122) in OSGi Enterprise Specification
  • Zookeeper Discovery Server

Thanks and Congratulations are due to ECF committers, contributors, and community


Labels: , , ,

Tuesday, October 01, 2013

Evolution of Cooperation

For some time in the Eclipse community, several people have repeatedly discussed the Tragedy of the Commons problem, and it's applicability to Eclipse.  In short, as 'platforms', Eclipse and OSGi can be seen as a kind of 'commons' that we all use and benefit from...for creating developer tooling/IDEs, for creating web servers, for creating mobile applications, etc.

The problem being that this 'commons'...if not sufficiently supported and maintained, will degrade over time...and all of us that depend upon, use, and in some cases profit from this commons will suffer from that degradation.   Some of us in this community (committers) feel that such degradation has been occurring...and continues to occur.  Even though many of us (including me) expend a lot of personal time/effort/expense to continue to support the community.

The question seems to be:  how do you get everyone to cooperate in order to maintain the commons?   Where 'cooperation' means real cost, and real effort on everyone's part.   It seems to me that a big part of the difficulty of doing this is that you have to get not just individuals (committers/people technically capable and knowledgeable enough about Eclipse to actually maintain things), but also both small and large corporations to recognize the need for cooperation and respond to it with more than what I would call lip service ('sure we'll pay for the EF, but we won't pay 8 full-time committers to work on maintaining the platform').   I've pointed out before that small and large groups have different ways of thinking about cooperation...aka collective action...and I further would assert that individuals have a totally different calculation about whether cooperation to maintain a commons even makes sense for them.

In any event, in a previous life I did research work in the psychology of judgment and decision making, and one of my areas of interest was in game theory and the prisoner's dilemma.  As part of this work I read a fascinating book called the Evolution of Cooperation by Robert Axelrod.  The main message of this work (by my interpretation) is that cooperative behavior..coming from self-interest...can be learned.  In my view this work opens up possibilities for solutions to the commons problem...but the hard nut (IMHO) is that group/org learning is an order of magnitude more difficult than individual learning.    And of course...individual learning of the self-interested benefits of cooperation is hard/slow enough.


Labels: ,