ECF Remote Services for Accessing Existing REST Services

The ECF community has asked about the use of Remote Services for accessing existing web/REST-based services.   To address some of these questions, I've created a tutorial showing how ECF's open APIs can be used to easily expose an existing REST+JSON service as an OSGi Remote Service.

For this tutorial, the public Geonames Timezone service was used, but the techniques shown can be used for exposing any web-based service as an OSGi Remote Service.

ECF 3.9.1 Released

ECF 3.9.1 is now available.   See the New and Noteworthy.

Internet of Things and ECF Remote Services

Last week at EclipseCon Europe ECF committer Wim Jongman gave a talk where he cooked an egg using a hotplate, a Raspberry Pi, ECF Remote Services, and Eclipse Nebula.   Here's a screenshot of the RCP app he created to remotely control the Raspberry Pi and the connected hotplate:

Although I wasn't able to attend, I understand that after cooking, Wim consumed the egg onstage...and it went down pretty easy.  I'm looking forward to watching the video of the talk.  Congrats Wim.

Also, the Eclipse Newsletter has articles focused on Internet of Things, and we have an article about using ECF Remote Services with Raspberry Pi GPIO, which was also used to implement some of the behavior underneath Wim's egg cooker.

For other tutorials on using Remote Services with the Raspberry Pi as well as in other runtime contexts, see the ECF wiki documentation page.

ECF Remote Services Architecture

With the many updates, improvements in ECF's Remote Services releases, along with the tutorials and documentation added recently, it was time to update the Remote Services Architecture diagram.

Shellshock bug, open source maintenance and quality

Interesting NY Times article about shellshock bug, and it's relationship to open source maintenance and quality

ECF 3.9 Released

ECF 3.9.0 is released.   The release provides an update to our implementation of the OSGi Remote Services and Remote Service Admin specifications.   For RS/RSA the latest (R6) version of the specification included additions and changes, and this release supports them.

The implementation has been tested against and passed the OSGi RS RS/RSA compatibility test suite (CT).

To download and install go here.  

Note also that we now have a several tutorials focused on developing your own OSGi Remote Services.  A couple of these tutorials show how to use Remote Services with the Raspberry Pi.

Raspberry Pi GPIO using OSGi Services

I've created an API that abstracts individual GPIO Pins as OSGi services, and made it available via the ECF github repository.  This allows applications to easily use the Raspberry Pi's GPIO to send output to or receive input from peripherals.

I've also created a short tutorial on how to use these services to control a single LED.

This tutorial now has a short demonstration of how to use OSGi Remote Services to do remote control of a single GPIO Pin with Eclipse as the user interface.

ECF 3.8.1 Released

ECF's 3.8.1/Luna has been released.   Some of the highlights

• Fully compliant implementation of OSGi Remote Services and Remote Service Admin standards
• Support for Java8's CompleteableFuture for Asynchronous Remote Services
• Tutorials and documentation for OSGi Remote Services
• Support for creating and running standard remote services on MQTT protocol

A more complete list is provided by the New and Noteworthy

Congratulations to the ECF community

OSGi Remote Services on Raspberry Pi

ECF has created an example and tutorial showing the use of Java 8, OSGi, and OSGi Remote Services on the Raspberry Pi.   Any comments, suggestions, or other contributions are always welcome.

The costs of open source maintenance

Another very interesting story about Heartbleed/OpenSSL...wrt maintenance and support

• Who Should Pay to Keep the Internet's Locks Secure

Open Source In News

Two interesting articles about Heartbleed bug/OpenSSL and the Linux Foundation and OpenSSL Foundation in today's NYTimes:

• Heartbleed Highlights a Contradiction in the Web 
• OpenSSL and Linux: A Tale of Two Open-Source Projects

New tutorial on using Java8 and Asynchronous Remote Services

ECF has a new tutorial explaining how OSGi service designers may now use Java8 CompletableFuture for non-blocking remote services in ECF 3.8.1/Luna. Also present in the ECF wiki is page with additional examples and details on Asynchronous Remote Services.

CompletableFuture for OSGi Remote Services

As some of you may know, ECF's implementation of OSGi Remote Services has support for asynchronous proxies.   This allows consumers of remote services to easily use asynchronous/non-blocking invocation to access a remote service.  For example...here's a simple service interface that's used in our remote service tutorial:

public interface ITimeService {
    public Long getCurrentTime();
}

As with any OSGi Remote Service, any consumer that discovers this service will potentially block when they call getCurrentTime(). This is just the nature of call/return semantics applied to remoting...and so will be true for any implementation of OSGi remote services.

ECF's impl of OSGi Remote Service offers asynchronous proxies, meaning that iff a related service interface is defined...e.g.

public interface ITimeServiceAsync {
    public Future getCurrentTimeAsync();
}

then consumers will be able to use ITimeServiceAsync to access the service...e.g:

ITimeServiceAsync tsa = ... get ITimeServiceAsync reference

Future timeFuture = tsa.getCurrentTimeAsync();
...do work...
Long time = timeFuture.get();

With ECF 3.8.0 we further added [3]...i.e. the ability to have ITimeServiceAsync be registered as a service automatically on the consumer...so that (e.g.) the ITimeServiceAsync reference can be injected via declarative services...e.g.

...ds component impl...

void bindTimeServiceAsync(ITimeServiceAsync tsa) {
     ...use or store tsa...
}

This [3] is all available in ECF 3.8.0.   Note that the service interfaces have absolutely no reference to ECF classes, nor to OSGi classes.   None are needed now.

As many of you certainly know...java8 just came out, and a big part of java8 is improved support for concurrency, functional programming, and lambdas.    This support for concurrency in java8 is potentially very useful for users of OSGi Remote Services.

Consider CompletableFuture, which as the name implies is a type of java.util.concurrent.Future.    It has some very nice properties for API/service designers...the main one being that it's not at all necessary to call Future.get directly...but rather you can write nice, succinct and *guaranteed to be non-blocking but asynchronous* usage such as:

CompletableFuture cf = ...get CompletableFuture....

cf.thenAccept((time) -> System.out.println("time is: " + time));

This is nice...it's completely non-blocking...and very succinct.  Also you can do very interesting things with asynchronous/event-driven chaining/filtering, etc., etc. All guaranteed to be non-blocking...which is a key guarantee for remoting.

Yesterday I realized that with Java8, our asynchronous proxies could be easily generalized to allow this:

public interface ITimeServiceAsync {
    public CompletableFuture getCurrentTimeAsync();
}

I made some minor additions to the ECF remote service implementation of asynchronous proxies and now this is working. What I mean by this is that consumers of an arbitrary remote service can now do this

...service component impl...

void bindTimeServiceAsync(ITimeServiceAsync tsa) {
       // Get the CompletableFuture...no blocking here
       CompletableFuture cf = tsa.getCurrentTimeAsync();
       // print out time when done...no blocking anywhere!
       cf.thenAccept((time) -> System.out.println("Remote time is: " + time));
}

Note a few things:

1. There is no blocking anywhere.  This is true even though the actual time value is retrieved via a remote OSGi service
2. The remote service host doesn't have to provide any implementation of ITimeServiceAsync.  It's constructed by ECF's RS impl automatically
3. It's very easy to handle failure (e.g. network/io failure) via CompletableFuture.handle.  This is obviously a big deal for remote services...which are much more inclined to fail because of the network.
4. No reference to either OSGi classes or ECF classes anywhere in host or consumer code

This is nice for remote service consumers, because it gives them strong no blocking guarantees, and can be very efficiently implemented (no additional threads) by remote service providers using asynchronous messaging (e.g. JMS, etc).   It also allows them to use all of CompletableFuture's APIs (see CompletableFuture javadoc).

This will obviously be part of ECF Luna...and I've created some test code (like above) that I intend to use to create another tutorial over the next month. Watch the ECF wiki for that tutorial.

The only drawback is that this does, of course, depend upon Java8...and so requires that both the remote service host and consumer use Java8, and that the distribution provider be enhanced slightly to use CompletableFuture.   Fortunately it's not technically challenging to make these enhancements, and we will make support classes available for those that wish to Java8 enhance existing or their own RS provider.

[1] https://wiki.eclipse.org/ECF/Asynchronous_Remote_Services
[2] https://wiki.eclipse.org/Tutorial:_Building_your_first_OSGi_Remote_Service
[3] https://bugs.eclipse.org/bugs/show_bug.cgi?id=420785

ECF 3.8.0 Released

ECF has just released 3.8.0.

New and Noteworthy

• Easily use ECF Remote Services in Apache Karaf
• New Tutorials, Examples, and Documentation
• MQTT-based Remote Services Provider
• Feature Refactoring...for Use of Remote Services Outside of Eclipse
• New Support for BndTools

New tutorial: Creating a RESTful Remote Service Provider

ECF's implementation of OSGi Remote Services supports the creation of custom distribution providers.  Why would anyone wish to do this, when they could simply reuse one of the existing providers?   Here are some good reasons:

Service Backward Compatibility:   There are cases where it's not a new service being developed, but rather a new facade for an existing (e.g. web) service.  With ECF, one can easily create a custom distribution provider that reuses the existing service to expose it as an OSGi service.  This allows existing services to continue to be exposed as they were originally written, but also be exposed as an OSGi Remote Service.

Custom Transport Requirements:  Remote services also frequently have specific transport requirements...for example using MQTT rather than HTTP, or using JSON rather than XML.  With ECF, distribution providers can completely control these transport-level decisions.

Standardization and Interoperability:  ECF fully implements both the OSGi Remote Services (RS) and Remote Service Admin (RSA) standards.   One thing this means is that all ECF Remote Service providers (both those created by us as well as others) are automatically compliant with the RS/RSA specifications.   Also, Remote Service providers can be proprietary/closed source or open source as desired.

We've created this tutorial to show how a custom RESTful (HTTP+JSON) remote service provider can be easily built:  Creating a RESTFul Remote Service Provider

New tutorial: OSGi Remote Services

ECF team has created a new introductory tutorial on creating standard OSGi Remote Services

Building your first OSGi Remote Service

Our plan is to create a number of such tutorials on OSGi Remote Services and focus on additional topics such as using/configuring alternative discovery or distribution providers, creating REST-based providers, asynchronous remote services and others.  They will generally first appear here.

ECF 3.7.1 Released

Just in time for EclipseCon Europe 2013, ECF has had our 3.7.1 maintenance release.  

ECF 3.7

ECF 3.7 was just released.

Highlights/What's New

• Servlet API -  for Creating OSGi Remote Services with HttpService and Servlets
• OSGi Remote Service Examples
• Testing Against OSGi R5 Compatibility Test Suite -  For Remote Services (chapter 100) and Remote Service Admin (chapter 122) in OSGi Enterprise Specification
• Zookeeper Discovery Server

Thanks and Congratulations are due to ECF committers, contributors, and community

Evolution of Cooperation

For some time in the Eclipse community, several people have repeatedly discussed the Tragedy of the Commons problem, and it's applicability to Eclipse.  In short, as 'platforms', Eclipse and OSGi can be seen as a kind of 'commons' that we all use and benefit from...for creating developer tooling/IDEs, for creating web servers, for creating mobile applications, etc.

The problem being that this 'commons'...if not sufficiently supported and maintained, will degrade over time...and all of us that depend upon, use, and in some cases profit from this commons will suffer from that degradation.   Some of us in this community (committers) feel that such degradation has been occurring...and continues to occur.  Even though many of us (including me) expend a lot of personal time/effort/expense to continue to support the community.

The question seems to be:  how do you get everyone to cooperate in order to maintain the commons?   Where 'cooperation' means real cost, and real effort on everyone's part.   It seems to me that a big part of the difficulty of doing this is that you have to get not just individuals (committers/people technically capable and knowledgeable enough about Eclipse to actually maintain things), but also both small and large corporations to recognize the need for cooperation and respond to it with more than what I would call lip service ('sure we'll pay for the EF, but we won't pay 8 full-time committers to work on maintaining the platform').   I've pointed out before that small and large groups have different ways of thinking about cooperation...aka collective action...and I further would assert that individuals have a totally different calculation about whether cooperation to maintain a commons even makes sense for them.

In any event, in a previous life I did research work in the psychology of judgment and decision making, and one of my areas of interest was in game theory and the prisoner's dilemma.  As part of this work I read a fascinating book called the Evolution of Cooperation by Robert Axelrod.  The main message of this work (by my interpretation) is that cooperative behavior..coming from self-interest...can be learned.  In my view this work opens up possibilities for solutions to the commons problem...but the hard nut (IMHO) is that group/org learning is an order of magnitude more difficult than individual learning.    And of course...individual learning of the self-interested benefits of cooperation is hard/slow enough.

Article on OSGi Remote Services in Newsletter

ECF has an article in the Eclipse Newsletter about our support of Remote Services/Remote Service Admin Standards.