Eclipse Communication Framework

ECF 3.5.4 and Restlet-based remote services

2011-12-30T12:22:00.000-08:00

ECF has just released version 3.5.4. See here for download.

Through our github repo, we now have new OSGi remote services provider based upon Restlet. This allows Restlet to be used as the underlying implementation for OSGi remote services.

The small size and simple implementation of the Restlet-based provider (as well as all the providers) is made possible by ECF's provider architecture. This architecture also allows any transport (rest-based or not) to be easily used to implement any OSGi remote service.

It's probably unnecessary to say, but the use of OSGi services (and remote services), brings many systemic advantages...including built-in support for dynamism, security, version management, modularity, among other things.

ECF 3.5.3 and Restlet Remote Services

2011-11-15T10:31:00.000-08:00

ECF has produced a maintenance/bug fix release...version 3.5.3. Go here to download. As per maintenance releases, this release does not have new features or API, but does have bug fixes. Go here for 3.5 New and Noteworthy.

Additionally...some ECF committers have been working with the Restlet team to create an OSGi remote services provider based upon Restlet.

This shows the flexibility of the ECF implementation of OSGi remote services/RSA specifications...as any communications protocol (rest-based or not), can be easily used to create a standards-compliant remote services provider.

Simulation Using OSGi, ECF remote services

2011-10-06T14:19:00.000-07:00

There's a new paper about using OSGi (Equinox) and ECF remote services to create a transport-independent, service-oriented, simulation framework. Their paper is here.

I have thought for some time that the combination of OSGi, with standardized, open, remote/distributed services (as is provided by OSGI remote services and ECF's implementation of that spec)...would be a strong simulation environment, and now Martin Petzold, Oliver Ullrich, and Ewald Speckenmeyer have shown that thought to have some merit.

As well, the authors have made their own framework available as open source.

Thanks to Martin, Oliver, and Ewald for doing and reporting some terrific work...and to the ECF community for providing support.

ECF 3.5.2

2011-08-29T07:50:00.000-07:00

I'm pleased to announce the immediate availability of ECF 3.5.2. This is a maintenance release, with bug fixes only. Much of the emphasis for this maintenance release was on OSGi remote services and Remote Service Admin (RSA) support.

Congratulations are due to the ECF community.

Restlet for OSGi Remote Services

2011-07-18T14:08:00.000-07:00

The ECF project released a new version of it's implementation of the OSGi 4.2 Remote Services Admin (RSA) standard.

ECF's provider architecture allows new distribution modules (known as providers) to easily be created and inserted underneath the ECF RSA implementation. The remote service consumer can now use the OSGi services model for accessing remote services...without regard to the underlying transport. If desired, one can create a service using one transport (e.g. r-osgi), test it using another (e.g. ecf generic) and deploy it using yet a third (e.g. your custom protocol)...even changing the distribution protocol for a remote service at runtime. The application requires no code changes to change providers. This is the beauty of standardization (no lockin) for distribution systems.

What does this have to do with Restlet?

Now that Restlet has been well integrated with OSGi it's now easy to use Restlet as a distribution provider module...and that's what I've just finished implementing. So now, one can use standard remote services API (i.e. OSGi remote services spec)...along with standardized enterprise remote services management (i.e. OSGi RSA spec)...and use Restlet/http+rest as the underlying distribution mechanism for exposing and accessing the remote service.

Here's a simple Restlet example


public class HelloResource extends ServerResource {
  @Get("txt")
  public String sayHello() {
    return "Hello RESTful World";
  }
}

Note the @Get("txt") annotation...this is Restlet annotation that defines that http access to this method.

To turn this into an OSGi remote service, all that's necessary is to expose the desired service as a service interface


public interface IHello {
  @Get("txt")
  public String sayHello();
}

and then add '...implements IHello' to the HelloResource class...e.g.


public class HelloResource extends ServerResource implements IHello
...

And that's it. Now (with the Restlet provider and ECF 3.5.1 remote service admin) when the HelloResource is exposed via Restlet, a IHello service is exported...and published for remote discovery (via Zookeeper, Zeroconf, DNSSD, SLP, file-based discovery, or some custom discovery). Then, as per the OSGi RSA specification, remote service consumers will discover the remote service and import the remote service as a IHello proxy (with RSA's support for versioning, etc). For the client/service consumer, all of the mechanics of import is handled by RSA...the programmer does not have to be concerned with it if they don't wish to be.

As an example, here's the code for a java-based client (assuming DS injection/binding):


void bindHelloService(IHello hello) {
    // Now that we have discovered the service
    // We'll use it.  The implementation of sayHello 
    // remoting is provided by Restlet
    String response = hello.sayHello();
    System.out.println("Response to our hello was: '"+response+"'");
}

When run with the HelloResource server, the response is:


Response to our hello was: 'Hello RESTful World'

Of course, other clients (e.g. browser/javascript-based, php-based, etc) can also be used to access the same RESTful service.

Another nice aspect of this use of the ECF provider architecture is that other REST frameworks...e.g. JAX-RS, etc...can be used similarly...and even run concurrently in the same server, if desired.

ECF 3.5.1/Indigo - Supporting Standards

2011-05-29T13:32:00.000-07:00

With ECF 3.5, we released an implementation of the OSGi 4.2 enterprise standard known as Remote Services Admin (RSA).

Using ECF's modular provider architecture, it's now possible to get the benefits of being completely standards compliant, while...if you wish...still using your favorite remote services distribution API (e.g. REST-based, SOAP-based, JMS, proprietary, open...your choice). Service-Oriented Architecture and Modularity working together via open implementations of open standards...hmmm :).

Standardized APIs make creating and managing remote services much easier...without sacrificing necessary flexibility. Standardization also allows easy integration with other frameworks, such as Declarative Services, Spring, and/or others.

In March we got access to the OSGi Test Compatibility Kit for Remote Services, and since then have fixed bugs in the implementation to guarantee full spec compliance, as well as addressed bugs reported by the community.

We've also significantly increased our documentation and examples...for remote services as well as other parts of ECF...and created a ECF documentation project to more easily incorporate community contributions...in the docs areas identified as most important by our community.

ECF 3.5.1 is available now here, and is part of Indigo simultaneous release.

See also a recent EclipseZone article about ECF for other exciting things that are part of this release.

Restlet and OSGI remote services - Part 2

2011-04-04T15:31:00.000-07:00

In a previous posting, I described some of the advantages of integrating the Restlet framework with ECF's implementation of OSGi remote services admin (RSA).

In this posting, I'll describe a couple of the advantages of doing this for the service host side of things (the server that exports and implements the remote service).

Advantages for Remote Service Host

Modular re-use of Restlet Framework. The Restlet framework can be used to easily define remote services that are exposed via http access methods (e.g. GET, POST, PUT, DELETE). Restlet has become popular as a way to create and expose remote services, and all existing uses of Restlet can immediately and modularly be reused.

Use of standardized meta-data format. One of the most valuable things about the Remote Service Admin specification, I believe, is the standardization of the meta-data for a remote service. This is accomplished by standardizing the EndpointDescription format for remote services. Among other advantages, standardization of this meta-data allows the easy creation of tooling for reading/parsing, as well as writing these meta-data.

Modular re-use of network discovery. Since EndpointDescriptions are standardized, they can be easily published and discovered via various discovery protocols. ECF's discovery API is a transport-independent API for advertising and discovering services, and this API is used by the ECF RSA implementation. This allows discovery providers to be substituted...without any required changes in the export or import of a remote service. So not only can any of the existing ECF discovery providers be used interchangeably for EndpointDescription discovery (Apache Zookeeper, Zeroconf/Bonjour, Service Locator Protocol, DNSSD, xml-file-based discovery), it's also easy to create your own discovery provider, using proprietary or open protocols for remote service discovery...to meet enterprise requirements for security, integration, and customization.

The conclusion, I believe, is that standardization provided by the OSGi RS/RSA specs, along with ECF's modular, provider-based implementation (enabled by OSGi modularity and OSGi services) makes it easy to develop, deploy, manage, and maintain standardized remote services, without giving up flexibility...to determine how those remote services are discovered, accessed and managed in SOA-based systems.

Summary: Modularity and Standardization are complimentary for reuse, flexibility, and interoperability of remote services.

Restlet and OSGI remote services - Part 1

2011-03-29T09:43:00.001-07:00

ECF recently released a standards-compliant implementation of OSGi 4.2 remote services admin (RSA).

RSA promises the easy integration with existing SOA frameworks in a standardized OSGi remote services context. To prove the utility of this to myself I decided to integrate the popular Restlet API with ECF's RSA impl expose REST-based web services as standards-compliant OSGi remote services.

I was very happy to find that with the Restlet API, the Restlet-OSGi-integration work, ECF's RSA impl, and ECF's REST API, that doing this was about two-days' work. In addition to being simple to do, there are several advantages of doing this...both for service consumers and service hosts.

Advantages for Service Consumers

Many clients can/are immediately supported (e.g. browser, new clients, servers that access the service, etc)

No client-side development at all. ECF's RSA impl creates a proxy (as well as an asynchronous proxy), and makes that proxy available within the local OSGi service registry...with no development at all. This makes it easy to also use OSGi declarative services, Spring/Virgo, or other frameworks to access remote services

OSGi classloading subtleties are fully dealt-with, as ECF's RSA impl handles the proxy creation in a standardized, secure, service-independent way

Service interface versioning is automatically supported...by the RSA spec

RSA's discovery can be used to publish and discover a remote service. With ECF's impl of RSA, this allows the modular use of a variety of network discovery protocols, including Apache Zookeeper, DNS-SD, Service Locator Protocol, Zeroconf/Bonjour, xml-file-based...and also enables using one's own discovery mechanism (proprietary or not)

In another posting, I'll describe some of the advantages on the service host side (i.e. the OSGi server that publishes/exposes the Restlet service).

ECF enables Thermonuclear War at EclipseCon 2011

2011-03-18T09:32:00.000-07:00

How's that for a title? :). ECF committers Mustafa Isik and Sebastian Schmidt are giving this talk at EclipseCon on Monday:

INTERSTELLAR THERMONUCLEAR WAR ... with ECF

Mustafa previously did much of the initiating work on real-time shared editing in ECF as part of his Google Summer of Code project. For Google Summer of Code 2010, Sebastian implemented a Google Wave provider for ECF.

Now they are at it again :). Mustafa and Sebastian are using/integrating several great technologies to do innovative and fun things with the Wave protocol for concurrency control in multiplayer games, OSGi servers, remote services, Android clients, ECF's multi-provider APIs, and other exciting technologies.

I know from working with Sebastian and Mustafa, as well as working on some of these technologies myself, that it will be a great talk. Please enjoy.

ECF 3.5 - Remote Services Admin

2011-03-14T12:44:00.000-07:00

ECF 3.5 was just released. One of the New and Noteworthy for is a complete implementation of the OSGi enterprise standard known as Remote Services Admin (chapter 122 in the enterprise spec).

First: What are OSGi Remote Services?

OSGi remote services defines a simple, standard API...using normal OSGi services...for exposing services for remote access. ECF has supported the OSGi Remote Services specification for more than a year...and it's been hardened through community usage, bug reporting, and feedback.

What is RSA?

RSA is an enterprise management agent for OSGi Remote Services. As of ECF 3.5), we fully support the RSA specification, which allows very fine-grained control, management, and security for enterprise remote services. Specifically, it's possible for the both the remote service discovery and distribution to be customized or extended as dictated by the (enterprise) use case...without resorting to non-standard API.

Why ECF's Implementation?

ECF's implementation has a number of unique technical attributes, including transport independence through multi-provider architecture, support for asynchronous remote services, support for Felix and other OSGi frameworks, small code size, and open, community-based development process.

ECF 3.5

2011-03-13T15:50:00.000-07:00

ECF 3.5 has just been released.

New and Noteworthy

Full implementation of OSGi Remote Services Admin (RSA). Chapter 122 from the OSGi enterprise spec

XML-RPC remote services

ECF on Felix

Documentation Project for community-contributed documentation

To be fair and balanced, give up centralized control

2011-03-08T10:26:00.000-08:00

This posting is in response to Ed Merks' recent meandering To Be Fair and Balanced, That is the Question.

My suggestion is that to be fair and balanced, one has to give up control...and in this case turn the decision of project-level resource allocation away from any centralized body (like the EF Board of Directors...or the committer reps, or the EMO, or the strategic members, etc). In short, give that decision making power to the people that matter...the communities that the projects serve. That is the purpose of this new FOE disbursement bug.

The point is this: it's seems unlikely to me that any fair and balanced decision can/could be made by me, the committer reps, the EMO or the Board about project resource allocation across many very different projects...because there is probably permanent disagreement about what is fair.

And to Ed: let's get past the snarkiness and personal discrediting/attacking, shall we?

ECF 3.5 supports OSGi 4.2 Remote Services Admin (RSA)

2011-01-26T14:47:00.000-08:00

OSGi 4.2 remote services support was the major theme for ECF 3.3 and 3.4.

For ECF 3.5 (late Feb 2011), we will release a full implementation of the OSGi Remote Service Admin (RSA) specification from the enterprise experts group. The RSA spec (chap 122) extends the remote service spec, and provides standard ways to monitor, control, secure, and extend the use of OSGi remote services.

ECF's impl of this spec is now complete, and we are engaged in testing (with the OSGI TCK), integrating with examples, adding new examples, and adding documentation.

One exciting thing about this implementation is that with ECF's open provider architecture, it's possible for other discovery and/or distribution systems to be easily introduced by anyone (us or others)...and all providers will automatically be standard compliant. This vastly simplifies the job of taking an existing protocols and transports (for example a REST-based protocol) and exposing them as OSGi remote services.

Further, ECF's impl already supports asynchronous remote services, and this support is exposed in a standards-compliant way.

ECF 3.4 Remote Services

2010-11-02T09:54:00.000-07:00

ECF 3.4 was recently released. This release (along with Helios/3.3 and upcoming releases) heavily emphasized the implementation of OSGi 4.2's Remote Services specification. Our community is pushing us to continue this emphasis, and so we will.

Here are some reasons to use ECF's OSGi 4.2 Remote Services implementation:

Standards Compliant: It is fully compliant with the Remote Services standard. No lock-in...now and forever

Asynchronous Remote Services: Unlike other implementations of this standard, right now it provides support for Asynchronous Remote Services

Multi-Transport: Right now it supports multiple network discovery protocols (e.g. Zookeeper, DNS-SD, SLP, Zeroconf, static xml-file), and multiple distribution transports (e.g. r-OSGi, ECF generic, XMPP, JMS, Http/REST-based protocols, JavaGroups)

Extensibility through Modularity: The open discovery and remote services APIs allow new discovery and distribution implementations to be substituted at will...proprietary or open...without requiring any additional work to support the OSGi standard

Enterprise support: We are completing (for ECF 3.5) our implementation of the Remote Services Admin specification. The progress on this can be easily and publicly tracked...contributions, test/testing, and early uses are welcomed and encouraged.

Open Community: ECF is not just open source, but also has a completely open, diverse, growing, active...and most importantly...a contributing community

Open Process: We've moved to GIT, to make community support and contributions easier

Multi-Framework: ECF remote services now runs on Felix (and probably other OSGi frameworks as well)

Robustness through Community Usage

Low-license fee: $0 :)

Innovation and Openness

2010-11-01T19:03:00.000-07:00

There is a Sunday NY Times article about what promises to be an interesting book:

Innovation: It Isn't a Matter of Left or Right

Johnson apparently makes the claim that 'collaborative, non-proprietary, open networks' are of high importance for technology innovation. This strikes me as true, and explains my intuition that open source projects like Eclipse and ECF are well-positioned to create value through innovation.

ECF 3.4 Released

2010-10-31T11:49:00.001-07:00

ECF 3.4 is now available. There have been many community-driven, and community-contributed improvements...here are some of the highlights:

DNS-SD Remote Service Discovery for WAN

Atom/RSS REST Enhancements

Distributed EventAdmin Improvements

OSGi 4.2 Remote Services enhancements

Bugs Fixed

Moved to GIT

Remote services runs on other OSGi frameworks

Congratulations are due to the ECF committers and community. Remember to see Fun with Remote Services talk at ESE for a taste of the things provided by ECF's open implementation of this OSGi standard.

Asynchronous Remote Services - choices, choices

2010-09-02T07:17:00.000-07:00

In ECF's Helios release, we released an implementation of the OSGi remote services standard specification (chapter 13 in compendium).

In addition to the full spec implementation...which is based upon synchronous remote service proxies...we added support for asynchronous remote services. This provides non-blocking access to remote OSGi services. This gives remote service consumers choices...allowing them to invoke remote services synchronously (i.e. by making a blocking method call on the proxy), and/or asynchronously (with a guarantee that the calling thread will not block).

I think that one nice thing about this approach is that the service host implementer has to do exactly nothing to make these consumer choices available. The implementation of the service host is exactly the same.

There are two styles of asynchronous access supported: an asynchronous callback (like GWT), and a future result, from the Actor model of computation. These two styles of of asynchronous access...along with the specified synchronous proxy...provides remote services consumers with some useful choices for creating reliable distributed systems and applications.

Making Sense of Complexity

2010-05-02T10:38:00.000-07:00

The NY Times Sunday Opinion section has an article today: Making Sense of Complexity.

The ideas are (mostly) presented in reference to complexity in social systems...but as someone interested in (reducing) complexity in software systems, as well as the psychology of complex system design and development...I also found the thoughts interesting from a software architecture and design point of view.

Asynchronous Remote Services - The future or the callback

2010-04-27T10:54:00.000-07:00

In previous postings I described how ECF is now making it very easy for OSGi service developers to expose asynchronous/non-blocking remote method calls to clients.

In short, all that's now required is to create an asynchronous version of the service's OSGi service interface. See this documentation for example and source. Just declaring this asynchronous interface is all that's needed. At proxy discovery time, ECF's implementation of OSGi remote services will provide the implementation of this asynchronous interface.

Future or Callback

There are various approaches to doing asynchronous remote method invocation, and two common ones are callbacks and futures. For example, GWT uses callbacks, while Amazon EC2 uses futures for exposing asynchronous access to their APIs (like SNS, SQS, etc). ECF's asynchronous remote services supports both of these approaches (futures and callbacks). The asynchronous service interface declaration can, for a given synchronous method declaration, use either a callback, or a future, or both.

For example, let's say we have the following synchronous service interface method:

String foo(String bar);

The async declaration for this method using a callback would look like this:

void fooAsync(String bar, IAsyncCallback);

The async declaration for thie method using a future would look like this:

IFuture fooAsync(String bar);

And that's it. The remote service client can then use either/both of these fooAsync methods (if they are declared, of course), simply by casting the proxy to the async service interface type and calling the appropriate fooAsync method with the necessary params.

In this way, the remote service designer can determine what asynchronous style the client will have available...by declaring fooAsync using callback, future, both, or neither.

Asynchronous Remote Services - part 2

2010-04-16T07:33:00.001-07:00

In a previous posting, I described how ECF has introduced a simplified approach for allowing OSGi remote services to be accessed asynchronously.

In contrast to my recent postings, that have been getting rather long, I'll just redirect you to a wiki page describing how to use asynchronous services...and leave it at that. Happy Friday.

OSGi Remote Services and ECF - Asynchronous services

2010-04-14T17:39:00.000-07:00

In a previous posting, I discussed/presented some of the support for asynchronous access to OSGi remote services that currently exists in ECF's implementation.

In a blog posting earlier this week, Peter Kriens discussed some of the efforts going on in the EEG on adding asynchronous support for remote (and even local) services. One of his comments in that blog posting was that ECF's asynchronous support could be considered awkward, because of the complexity/unfamiliarity of using the API.

I've been intending to add easier/more natural mechanisms for asynchronous remote access than what we already have, and what's going on in the EEG and Peter's blog was great incentive to complete some more of that work. The existing mechanisms are somewhat awkward, but they also make a very strong/flexible foundation...and so it's possible to build new mechanisms on the existing mechanisms.

Normal/Synchronous Proxies

In our 'hello' remote services example, we have this service interface:


public interface IHello {
    public void hello(String from); 
}

Consumers of this remote service receive a proxy that implements the IHello interface, and then clients can synchronously invoke the hello method to make a remote call:


proxy.hello{"slewis");

Since in java method calls are blocking, the thread that calls the hello method will block if (e.g.) the network is slow, the service host is slow (or blocks). It would be nice if we had a way (on the consumer/client) to call the hello method and guarantee that it will not block...while still somehow getting the result (if any)...when the remote call is successful...or getting information about the failure if things fail/go wrong (e.g. because of network failure).

Asynchronous Proxies

We've just added support for asynchronous proxies in ECF're remote services implementation. What this means is that if an interface is declared like this (and in the same package as the IHello interface):


public interface IHelloAsync extends IAsyncRemoteServiceProxy {

    public void helloAsync(String from, IAsyncCallback callback);

    public IFuture helloAsync(String from);

}

the ECF remote service distribution system will automatically create a proxy that implements the IHelloAsync interface on the consumer/client.

If the helloAsync(String,IAsyncCallback) method is called by the consumer:


proxy.helloAsync("slewis",new IAsyncCallback() {
    void onSuccess(Object result) {
        System.out.println("we got result="+result);
    }
    void onFailure(Throwable exception) {
        System.out.println("oh no!");
        exception.printStackTrace();
    }
});

the consumer thread will not block, and success/result or failure will be asynchronously communicated to the caller via proxy calling the appropriate method on IAsyncCallback.

In addition to using the callback, futures (IFuture) are also supported. All that must be done to allow the consumer to use a future result is to declare a helloAsync method that returns an IFuture:


    public IFuture helloAsync(String from);

The only thing required to get this to happen on the consumer/client is to declare the *Async interface (IHelloAsync). Then, at proxy creation time on the remote service consumer, if this *Async interface exists, it will be implemented by the proxy, and usable by the client.

Note that the *Async interface declaration is the only thing that's needed to get this to work with any service interface. The service host implementation doesn't need to actually implement the *Async interface, and the ECF remote services distribution will create a proxy that implements the *Async interface automatically. Further, like other things ECF, this is all done in a transport-independent way, so all the existing providers (JMS, XMPP, ECF generic, JavaGroups, Skype, REST, SOAP, etc., etc.) support this addition immediately with no further work.

Google Web Toolkit uses a very similar approach to support asynchronous remote procedure call. In addition to callbacks, however, ECF's asynchronous proxy also has support for futures. This allows the consumer/client to choose the desired invocation style: synchronous, asynchronous-callback, or asynchronous-futures.

OSGi Remote Services from ECF - Distribution

2010-04-10T13:26:00.000-07:00

In a previous posting, I discussed the use of the ECF discovery API as part of our implementation of the OSGi 4.2 remote services specification.

The second major part of ECF's implementation of OSGi 4.2 remote services is distribution.

What is Distribution?

Distribution is what happens to actually invoke a remote service and optionally return some result. Here's a brief summary of the essential functions of distribution:

Remote Service Consumer

[Prior to caller actually using service]
1. Create a proxy for the remote service
[When caller actually uses remote service]
2. Marshal/Serial any arguments for the remote call
3. Put call request (method and serialized parameter) on the wire using some protocol

Remote Service Host

1. Take request off the wire (using same protocol)
2. Un-marshal method and arguments
3. Lookup corresponding service/method
4. Invoke appropriate service with given arguments
5. Marshal return value
6. Put result on the wire using some protocol

Remote Service Consumer

4. Take response off the wire (using same protocol)
5. Un-marshal result
6. Return result to caller

One way to think of it is that distribution is responsible for making what looks like a local method call to a local OSGi service actually be a remote call.

Two of the critical functions of distribution...for both Consumer and Host are

1) Marshaling/Serialization...of arguments and return values
2) Use some protocol to communicate request/response over network

As with discovery, the ECF project has created an abstract API for distribution, which is called the ECF remote services API. Like other ECF APIs, this is a transport-independent API, which exposes a programmatic way to accomplish the functions of distribution (as described above), but does not imply/require any particular implementation of marshaling/serialization, nor imply/require any particular network protocol.

ECF has providers that define specific implementations of marshaling and network protocol. For example, we have a REST-API, that supports the creation of specific REST providers. This REST API includes JSON and/or xml-based serialization, and uses HTTP as the protocol. We also have a similar SOAP API for SOAP-based services.

We also have a number of other providers that are complete and available...e.g. ones based upon XMPP, JMS, ECF generic, Skype's app protocol, JavaGroups/multicast. Further, since all of these providers are open source, if desired they can be extended or copied to implement custom providers based upon whatever serialization and wire protocol (e.g. an existing system) is desired...with our without the ECF team's involvement.

Note the ECF implementation of the OSGi 4.2 remote services specification is guaranteed to work with any of these providers...no matter who writes it. This because our implementation of the OSGi 4.2 remote services spec simply uses any all implementations of the ECF remote service API (no matter what the serialization and/or networking protocol).

The flexibility here is extremely useful when selecting serialization formats and/or network protocols, because there are/will always be so many serialization formats and/or network protocols to choose from...their appropriateness will always depend upon the use case...as well as the need for integration with existing systems. For example...e.g. json over http, custom xml over http, object serialization over tcp, xml over jms, soap over http, etc, etc...which makes sense depends upon the use case and things like networking/interoperability requirements.

Since this distribution function is separated out into a distinct, abstract, module (i.e. the ECF remote services API), it makes it possible to mix and match existing protocols and new protocols...both closed and open...with existing serialization formats or new serialization formats...crossed with whatever discovery protocol is appropriate and/or desired.

OSGi Remote Services from ECF - Discovery

2010-04-06T10:15:00.001-07:00

Released in Feb, ECF 3.2 has full support for the OSGi 4.2 remote services specification.

As with any general technology, there are potentially many use cases for remoting OSGi services, and any given implementation won't support all those use cases. It's therefore very important that any technology be extensible to support use cases that were not envisioned originally.

OSGi Remote Services: A tale of discovery and distribution

In providing access to a remote service there are at least two network-created issues that must be addressed for any remoting technology to work. In this post I'll discuss discovery, and in subsequent posts talk about distribution.

Modularity for Network Discovery

When a new service is made available...via a server, or a peer, or a device, etc...any consumers/clients of that service must somehow be made aware of that service, and given sufficient information to be able to access that service. A very common example of being made 'aware' of a web service is receiving (via email, or a web page, or twitter, or whatever) the URL for that service...e.g. the twitter user status service URL is http://twitter.com/statuses/user_timeline.json.

With OSGi remote services the notion of a URL is generalized to an endpoint. As with all OSGi services, service properties provide metadata about the remote service (including but not limited to the endpoint)...and this metadata is sufficient for a consumer to actually access/use the service.

There are potentially many ways to discover a remote service. There are network discovery protocols (e.g. zeroconf/bonjour, Service Location Protocol (SLP), Apache Zookeeper), as well as static xml or other formatted files, custom http-based service registries, etc., etc.

To deal with the required flexibility, ECF has an abstract discovery API (org.eclipse.ecf.discovery). This is a network-protocol-independent API for discovering things over the network. I use 'things' because the discovery API isn't only for discovering remote OSGi services, and it can also be used to discover devices, other applications (an example of this is that since Apple's iTunes uses zeroconf to publish itself, it's possible to interoperate with iTunes and/or other iMac and iPhone applications from within an OSGi runtime).

The providers/protocols that we ship with ECF now are zeroconf/bonjour, SLP, and we have a pending contribution for Apache Zookeeper. We also currently have support for static xml-file-based discovery of remote services and are working on support for use of DNS-SD for wide-area dns-based discovery.

The ECF discovery API effectively separates network discovery into a distinct module, and allows the reuse of existing network protocol implementations, OR substitution of one's own approach to discovery to meet custom use cases (such as discovering remote services only behind a firewall, etc).

This modularization enables reuse, since all the other parts of ECF's remote services implementation (e.g. the distribution...i.e. remote method marshalling/unmarshalling, etc) can be reused without modification. This is so because ECF's OSGi remote services implementation simply uses any/all discovery API providers at runtime to publish the remote service. This makes any new discovery API provider automatically and immediately compliant with the OSGi remote services specification.

The reuse and extensibility is a positive side effect of the modularity provided inherently by OSGi, along with the separation of concerns built into ECF's implementation of OSGi remote services. In a future posting(s) I'll discuss the distribution module of ECF's remote services implementation...referred to as the ECF remote services API.

Reference:

OSGi 4.2 Remote Services
ecf-dev mailing list

OSGi Enterprise and ECF Remote Services

2010-03-23T11:04:00.000-07:00

Earlier today the OSGi 4.2 Enterprise spec was announced by David Bosschaert.

As part this announcement David listed some implementations of the OSGi 4.2 Remote Services specification, but for some strange reason he neglected to include the EclipseRT implementation from from ECF project.

So, just to be clear, EclipseRT/ECF 3.2 also already has support for OSGi 4.2 Remote Services. See here for details, links to docs, examples, and public support forums...as well as descriptions of other features in this implementation.

OSGi/EclipseRT in Amazon Cloud - p2

2010-03-23T07:00:00.000-07:00

In a previous post, I announced the availability of a public Amazon Image (AMI) for the Amazon EC2 service that includes several EclipseRT 3.6 technologies...including Jetty 7.0.1 and Equinox 3.6M5.

p2 is also included in the image, and this allows install/update in a running OSGi web application server. Also included in the AMI is Jetty + Equinox 3.6M5 that does not include p2.

OSGi/EclipseRT in Amazon Cloud

2010-03-16T20:23:00.001-07:00

I've created a public Amazon Image (AMI) from a recent build (3.6 stream) of the EclipseRT. The parts of EclipseRT included in this image were Jetty, Equinox, p2 provisioning, and a very simple Hello World servlet application (with source).

Here is documentation about how to get the image, start an EC2 instance, and run your own server.

Here is an instance of this Hello World servlet application running on my instance.

ECF 3.2 Now Available

2010-02-19T09:44:00.000-08:00

ECF 3.2 is now available here. The emphasis/theme for this release is support for Service-Oriented Architecture (SOA), specifically through support of the OSGi 4.2 Remote Services standard.

Highlights

Implementation of OSGi 4.2 Remote Services standard

Support for REST providers

Support for SOAP providers

Remote Services Examples and Docs and recent blogging about the tech for this release

Congratulations and thanks are due to the ECF community

OSGi Remote Services and Sync vs. Async

2010-02-16T11:06:00.000-08:00

ECF 3.2 contains an implementation of the new OSGi 4.2 remote services standard. This release is coming out later this week (Feb 19).

One thing that developers may discover when building and testing distributed applications is that synchronous remote procedure call (RPC) can have surprising behaviors. In my view, this is because we reflexively understand that normal/local/in memory method call is synchronous and fast...i.e. that the calling thread blocks until the method is complete (and optionally a result is returned), OR the method fails/throws an exception in languages that have structured exception handling.

So at best the remote call's I/O behavior will lead to large performance variability (i.e. the remote call will be orders of magnitude slower...and variable based upon network performance), and at worst the caller thread could hang/block indefinitely. This violates our expectations about method invocation.

To address this problem, frequently asynchronous remote method call and/or non-blocking messaging is used...so that the caller can be guaranteed that the calling thread will not block. Note that depending upon the application requirements and expectations, it may be fine that synchronous/blocking RPC is used. OTOH, it may be very important that remote services not block...for user experience, and or overall system performance expectations. It depends upon the use case...and I don't believe there is any one, 'right' answer for all situations.

ECF's implementation of the OSGi 4.2 remote services spec has support for asynchronous remote method call. This support is exposed via our IRemoteService contract, which is made available to remote service consumers (for example code, see tutorial here). This contract exposes two mechanisms for making asynchronous remote method calls

Asynchronous callback via IRemoteService.callAsync/2

Futures via IRemoteService.callAsync/1

The IRemoteService reference associated with a remote service proxy is accessible via any/all ECF remote services. If not needed, however, it's invisible and so doesn't impose any complexity burden.

Currently, the OSGi 4.2 remote services spec does not articulate any methods for asynchronously accessing a remote service, but my understanding is that this is an area for future standardization.

Goodness through OSGi Standards

2010-02-04T10:24:00.000-08:00

ECF recently announced full support for OSGi 4.2's remote services standard with our upcoming 3.2 release.

Today, I learned that a community member has successfully used Spring dm, along with ECF's remote services implementation to do declaratively-specified remote services. They have agreed to contribute the example to ECF, and so expect to see it as part of ECF soon.

People have also used ECF remote services with OSGi declarative services.

And, of course, one can use remote services programmatically as well.

Among other things, this allows a wide variety of existing tooling to be used to construct, use, and debug remote services...all made possible by having an open standard for distributing an OSGi service.

Motivation 3.0

2010-01-12T13:45:00.000-08:00

A question that I ask myself periodically is this: Why work on open source projects? In my case, why work on ECF? What is my motivation to do so?

There's an interesting book by Daniel Pink that examines human motivation called Drive: The Surprising Truth About What Motivates Us. There's also a nice YouTube video by the author from last year's TED conference.

In reading the first part of the book, I've resonated with his assertion that intrinsic motivation is sometimes stronger than extrinsic motivation...particularly when creativity (aka innovation) is involved. And I suspect I am not alone in this...at least among people who are passionate about innovation, community, software technology, and open systems.

I believe understanding motivations is important...because lots of assumptions about how innovation comes about...i.e. who does it and why...are actually dependent upon underlying assumptions about motivation.

SOAP, REST, and ECF remote services

2010-01-10T11:15:00.000-08:00

In addition to supporting the OSGi 4.2 remote services specification, we on the ECF team have also been working on support for accessing REST-style services, as well as those that use the Simple Object Access Protocol (SOAP).

The ability to support all these styles of service-oriented architecture is fundamentally enabled by ECF's provider architecture, but since mentioned that in a recent posting, in this post I'm going to touch on something useful that's enabled by this provider architecture.

Most remote services (whether web services, OSGi remote services, REST-based services, etc) have two basic roles

The service host...aka the 'server'

The service consumer...aka the 'client'

Typically, the service host is first 'registered'...to make it available for remote access, and after that the service consumer then accesses/uses the remote service (e.g. makes remote method calls on a proxy, issues http requests for resources, sends/receives messages, or has some other way of actually accessing/calling the remote service).

Note that the service host need not be exposed by a 'server' (although it frequently is)...in some systems clients can register/expose services as well as servers. And it's probably obvious that service consumers don't have to be 'clients' either...i.e. they can be servers that are communicating with other servers. This is why I use the role names 'host' and 'consumer' rather than 'server' and 'client' when referring to remote services.

OSGi remote services use the OSGi service registry...for both the host's registration of services, and the consumer's lookup and access to a remote service. Whatever the transport used to implement the distribution, typically both the host's registration and the consumer's lookup are done via the OSGi service registry. This typical use case implies, however, that both the host framework and the consumer framework have access to the OSGi service registry (since both registration and lookup are via the service registry).

But what if you would like to consume a service that doesn't use OSGi (and therefore doesn't have a service registry)? With ECF's remote services API, along with our recently added REST and SOAP support, we've enabled this use case (non-OSGi service host) while still providing the benefits of using OSGi remote services for the consumer.

How, you say? First of all, ECF's transport independent architecture allows clients to talk whatever protocol they require to communicate with a remote process...so, for example, the XMPP provider is able to communicate with any remote system that uses the (standardized) XMPP protocol. This does *not* have to be an OSGi-based system.

Second, ECF now has a very small, remote service client API...specifically to allow consumers to interact with non-OSGi services, while still using the OSGi service registry (if they wish) on the client. Since both the ECF remote services API, as well as this new remote service client API are also transport independent, and have both explicit support for REST-style and SOAP-based transports, providers for specific REST-based protocols and/or SOAP-based protocols are easy to create.

As an example, I recently created a SOAP/Web services client (for an existing web service), using Apache Axis (to convert wsdl to java), the WTP tooling (for automating the generation of the java code from wsdl), and this new remote services client API. To the non-OSGi service host, this client looks/behaves like all other clients. The same idea...for an existing REST-based API (Twitter in this case), is shown by this demo.

Unlike other clients, however, these clients can use OSGi to maximum value: i.e. to structure the client in a modular way, to handle the dynamic requirements of a remote/networked/unreliable service, or even (re) expose the proxy as a another remote service...that other consumers can access. This can be used for building load balancing of web services, or to aggregate sets of services...as well as other purposes.

Note there is no additional tooling required to build such a client...since your favorite tools for creating the SOAP-based and/or REST-based clients may be used, alongside APIs and tooling for interacting with the OSGi service registry...e.g. Eclipse PDE, OSGi declarative services and/or others.

OSGi Remote Services from ECF

2010-01-03T12:42:00.000-08:00

The ECF project has just finished our initial implementation of the OSGi 4.2 Remote Service specification (chapter 13 in compendium section).

I want to highlight a few distinctive features of ECF's implementation. I'll be doing other/more blog posts to go into details about some of these.

Transport Independence

ECF's implementation currently works with the following distribution transports: JMS (ActiveMQ), r-OSGi, JavaGroups, XMPP, Skype, ECF generic, and with the following discovery protocols: SLP, zeroconf, static xml file-based discovery. The ECF implementation is immediately usable with any...or all...of these providers...even within a single application if desired. Further, ECF's remote services and discovery APIs allow other distribution and/or discovery systems to be plugged in underneath our implementation...meaning that other distribution systems (e.g. open source distribution systems such as Riena, Apache CXF, and/or proprietary distribution systems) can reuse/leverage our implementation...and avoid the work otherwise necessary to implement the OSGi specification themselves. ECF's implementation can also run alongside other remote services implementations without conflict.

Asynchronous and Synchronous

Distributed applications and services frequently need to use asynchronous/non-blocking remote invocation patterns...in addition to synchronous, proxy-based remote method call. Right now, ECF has built-in support for both asynchronous invocation (e.g. asynchronous listeners, futures, one-ways), as well as proxy/synchronous invocation...giving the services and application programmer a transport-independent choice of which to use. Currently, the OSGi 4.2 spec does not yet specify asynchronous invocation patterns for remote services, but ECF's implementation does have it.

Lightweight

The entire ECF implementation of the spec...along with ECF's remote services API...is < 150k of code. Further, the OSGi execution environment requirements are minimal (CDC 1.1/Foundation 1.1). Combined with a small provider (like r-OSGi), and this allows even small devices to both expose/host and consume standardized remote services.

Standard, Open Source, Open Team, Open Process

The ECF implementation is fully compliant with the OSGi 4.2 remote services specification...for any and all current and future ECF remote services providers. Further, as mentioned above, other providers can now get this compliance for free. Of course, ECF's implementation is open source, but it is also produced by a vendor-neutral project team, and fully open community-driven process. In addition, we have support for a distributed version of the OSGi EventAdmin service, for doing publish-and-subscribe-based applications using a standard event bus API/service.

Extensible

ECF's implementation is deeply extensible, allowing control or customization of every aspect of the distribution of an OSGi service (e.g. custom discovery, custom marshalling/serialization, custom transport/wire protocol, etc) for those that need it. This is enabled by the open APIs that ECF exposes, and allows a wide variety of deployment requirements and application-level use cases to be easily supported.

Summary

It's All About Modules. ECF's modular structure is indeed enabled by OSGi's modularity, but we have also applied modular design to every level of APIs and implementations. For example, lightweightness, extensibility, and transport independence all are enabled by our modular designs...to separate API from implementation, and to separate distinct subsystems (e.g. discovery and remote services APIs).

We are preparing a release that includes this code...so to immediately download/use ECF's implementation, or engage the community...for requesting enhancements, reporting bugs, contributing (e.g. examples, providers, customizations), helping with testing, or getting support please consider joining the ecf dev mailing list or see the ecf dev resources page.

Some new browser-based smartphone tooling

2009-12-18T13:45:00.001-08:00

Project Ares is a new browser-based development tool for Palm/WebOS

Cloud + OSGi + GWT + ECF Rest + Twitter API = Modular web services

2009-12-09T13:22:00.000-08:00

Using several technologies, I've recently created a Twitter user status service...i.e. a web service that retrieves the latest user status for a given user.

Click here to use/try it

You will need a Twitter username and password to get that user's status.

Here's what was used

Web Server: EclipseRT, p2, Equinox servletbridge
Ajax Web UI: Google Web Toolkit (GWT)
REST API: ECF REST/remote services API, Twitter REST API/service
Cloud provider: Amazon Cloud/Web Services (AWS)

The use of OSGi modularity makes this not only possible, but lightweight...as this server/service consists only of the bundles necessary to actually provide this service...and each of those used are small.

why we cooperate

2009-12-01T11:15:00.000-08:00

It's a little hard for me to believe these days, but cooperation seems to be built in http://www.nytimes.com/2009/12/01/science/01human.html

Perhaps it's time to behave more like children.

ECF provides some additional REST

2009-11-19T08:51:00.001-08:00

ECF was very fortunate to have 4 Google Summer of Code projects coming out of the summer of 2009. All of these innovative codebases will make it into ECF itself...and more importantly we have gained several excellent new committers.

Coming out of the GSOC project, we've just finished a major refactoring of the REST API, to make it simpler, smaller, as well as better integrated with ECF remote services.

The integration with ECF remote services is particularly interesting, because it provides both proxy-based/synchronous invocation, as well as full support for asynchronous invocation patterns (i.e. asynchronous-with-listener-callback and futures). With the integration with ECF remote services, the choice of synchronous/asynchronous invocation comes for free for all REST-based remote services. See here to get the REST projects and use the API.

We are working on example code, and invite contributions from community members to this effort...so that we can continuously improve/simplify the API through community feedback, as well as get a wide range of example code and documentation from existing and new REST-based services. See the ecf-dev mailing list for feedback/discussion/coordination.

The current test code uses the popular Twitter REST API...as an example of using the API to interoperate with non-OSGi, non-Eclipse, perhaps non-Java web service (I don't know/care how the Twitter service is actually implemented...which is what protocol-level interoperability is about).

Load balancing remote services

2009-10-19T16:52:00.000-07:00

Hi Folks,

The ECF wiki now has example code for doing dynamic load balancing of remote services.

This implementation uses a JMS Queue (via ActiveMQ implementation of JMS) to do the dynamic load balancing of ECF remote service requests/method invocations.

The thing that I like most about this is that it's extremely simple, small...and doesn't require any special/custom code to use. Clients/consumers just lookup/use a remote service, and upon usage the service implementation load balances the requests among an arbitrary set of target servers.

ECF 3.1 released

2009-10-12T19:31:00.000-07:00

ECF 3.1 is now released. See here for download, and here for New and Noteworthy. Lots of very cool stuff being contributed (and that's without mentioning the upcoming Google Wave provider :).

Congrats to the ECF contributors, committers, and community.

ECF 3.1

2009-10-10T12:28:00.000-07:00

ECF 3.1 is being released on Monday next (10/12/2009), and there are some exciting additions:

REST API - An extension of the ECF remote services API to allow interoperation between OSGi remote services and REST-based services
Distributed EventAdmin - a Distributed implementation of the OSGi EventAdmin service, that uses JMS/ActiveMQ for transport (and/or other providers)
Lots of small improvements on ECF remote services, as well as examples and additional documentation
File-based Discovery - a contribution from Siemens for doing remote service discovery using a static endpoint-URI exposed via an xml format

Here are the release review slides.

Innovation

2008-11-02T12:40:00.000-08:00

There's an article about technology innovation in bad economic conditions in today's NY Times:

It's No Time to Forget About Innovation

I believe this is something to remember...particularly for Eclipse committers...since in my view they are the innovators in the Eclipse community.

ECF and Coffee in the Classroom

2008-10-14T11:24:00.000-07:00

There was a very cool announcement on the ECF newsgroup recently about a collaboration framework that uses ECF called Coffee. See here for the announcement. The Coffee end user site is here, and the development site is here. It's exciting for me to see ECF being used by projects both inside and outside the Eclipse Foundation...and as as you can see from the announcement they are keen to both collaborate and contribute back to the open source community.

Enjoy the beverage!

Planning for ECF 3.0

2008-10-03T12:00:00.000-07:00

ECF is nearly finished moving from Technology to the new Runtime project, and we are doing planning for ECF 3.0/Galileo. See the new plan here.

Please make enhancement requests and/or start discussion about desired features in the dev mailing list...and let us know what features, changes, or bug fixes you want for ECF 3.0.

Some Disturbing Things from Microsoft

2008-08-07T15:33:00.000-07:00

Open Source: A 'Growing Challenge' to Microsoft

There's an MS quote that is particularly disturbing to me

"The availability of protocol licenses may enable competitors to develop software products that better mimic the functionality of Microsoft's own products which could result in a reduction in sales of our products."

This is disturbing because it's pits protocol/service interoperability (a user need) against MS's sales (a MS shareholder need). Let's hope this doesn't result in MS pulling back from protocol openness.

Learning to Think Small?

2008-08-03T14:54:00.000-07:00

There is an interesting NY Times article today:

Even the Giants Can Learn to Think Small

ECF 2.0.0 New Features video

2008-07-28T16:14:00.000-07:00

It can't match Ed's pictures, but below is a pleasant scene, where most of the beauty is hidden beneath what is visible

But to the point: ECF has a new video showing some of the new things in ECF 2.0:

Screen shot sharing
URL sharing
Real-Time shared editing
Discovery API, Service Discovery, Remote OSGI Services
Async File Download

Enjoy

Freed from email

2008-07-03T10:21:00.000-07:00

There's an interesting article about use/abuse of email in the NY Times today:

I Freed Myself From E-Mail’s Grip

FWIW, I think ECF is doing it's part on this problem, by providing multiple ways for Eclipse users to easily, cheaply, and openly communicate and collaborate.

Also, for those interested, there is a longstanding proposal to add an XMPP server at Eclipse Foundation in order to help teams build and maintain community.

ECF 2.0.0

2008-06-25T10:01:00.000-07:00

With Ganymede the ECF project has it's 2.0.0 release. As for what's new....there were many little things, as well as many big but invisible things (i.e. APIs), but the things that really stand out for me are

Real-time shared editing (with a quickly-becoming popular video)
Discovery API: Simplified API and added jSLP provider
Remote OSGi Services API: Added provider based upon r-OSGi
P2: ECF's asynchronous file transfer API is used for P2's multi-protocol download, and so is part of the SDK.
Community Growth: We added some great new committers to the project: Markus, Jan, Ted, Mustafa, Marcelo...as well as many more contributors.
New and Noteworthy for details and more

This is ECF's last release as a technology project...as we will soon be moving to the new Runtime top-level project for 2.1 and beyond.

I like to say: 'the community is the team' :)...so congratulations and thanks to the ECF community.

P.S. You can find/install ECF's features in the Ganymede Communications category.

Adding Shared Editing To Your Favorite Text Editor

2008-06-21T16:06:00.000-07:00

With ECF 2.0/Ganymede, ECF introduces Real-Time Shared Editing. By default, the ECF real-time shared editing capability has been added to the JDT Java Source Code editor and Eclipse's Default Text Editor.

I've begun documenting how those with other types of Eclipse-based text editors (e.g. C/C++, php, javascript, xml editors, etc) can add real-time shared editing to their editors...simply by adding a little bit of markup to plugin.xml. See here for explanation and example.

Articles on ECF

2008-06-17T11:48:00.000-07:00

There are now a couple of articles on Eclipse Communication Framework for version 2.0.0/Ganymede release:

InfoQ: http://www.infoq.com/news/2008/06/eclipse-ganymede-ecf
DZone Interview: http://java.dzone.com/news/the-ecf-project-an-interview-w

Eclipse and Volunteerism

2008-06-05T09:21:00.000-07:00

In response to a recent news thread, Bjorn Freeman-Benson says

As Dan Ariely explains in "Predictably Irrational", there is a huge difference between real volunteer and paid volunteer (Eclipse is 'paid volunteer').

I want to examine briefly Bjorn's assertion that Eclipse is 'paid volunteer'.

In looking at the Dash commit count sorted by # of commits, the thing that jumps out at me is that for 2008 individuals, are right after IBM in terms of # of active committers (23.56%), commits (159,279), and lines of code (4,982,101).

There are/should be many qualifiers for these data: I'm sure many of the independents are paid to work on Eclipse...but conversely it's quite possible (and likely, I think) that some corporate-supported committers work more on Eclipse-related work...by their own choice...than their employers explicitly sanction. But in any event, I think the overwhelming message from the data is that the contribution from independents (many of whom are unpaid volunteers I would suppose) is large...and growing rapidly year-over-year.

All I'm saying is that 'Eclipse' is not necessarily 'paid volunteer'. But it does have a strong aspect of unpaid volunteerism, which as Bjorn (and Eric Rizzo) point out, should be encouraged and supported:

The place that Eclipse overall has not done a good job is making our projects attractive to unpaid, part-time volunteers. They're different. Their motivations are different. The Rizzo Ceiling is real for them.

For some pointers to interesting thoughts about motivations, please see

When Self Interest Isn't Everything

Seems that while I was typing this out, Boris Bokowski has also made some similar points.

projectDiversity.equals(productivity)

2008-02-26T15:38:00.000-08:00

I have started reading the book described in this article

In Professor's Model, Diversity = Productivity

After reading some of his book, it seems that cognitive diversity (defined as group differences in perspective, heuristics, interpretation, and prediction) generally lead to improvement in overall productivity. There are other kinds of group differences, of course...e.g. what Page calls identity differences (e.g. racial, gender, and social group), and other sorts of differences, but Page identifies cognitive differences as key.

Increased productivity is dependent upon the problem/task at hand, of course. Group tasks/problems that hinge on questions of representation and interpretation get greater value from diverse perspectives, since such diversity allows more alternatives to be generated and considered.

My thought on reading this was that many sw design tasks are like this (i.e. hinge upon how the problem is represented) and so should be particularly good candidates to benefit from teams with high cognitive diversity...and the many perspectives that come with such groups.

Interestingly, Page also identifies a type of diversity that seems to result in lower group productivity...I'll talk about that after I've read a little more.

When Self-Interest Isn't Everything

2008-02-10T12:16:00.000-08:00

An interesting editorial by an economist in today's NY Times

When Self-Interest Isn't Everything

Is Hirschman's analysis (described in editorial) relevant for the open source 'movement'? How much of open source is/will be about self-interest?

Discovery and Access for Remote OSGi Services

2008-02-07T17:07:00.000-08:00

See here for some work combining the ECF discovery API with the ECF remote services API.

Interesting things about this:

ECF APIs have multiple providers...i.e. zeroconf/bonjour and SLP for discovery, r-OSGi, ECF generic, xmpp, javagroups, and multiple flavors of JMS for remote services. This allows middleware and UI written to these ECF APIs to run unmodified on any/all of these providers.
The ECF remote services API can be used in either 'transparent' or 'non-transparent' manner...allowing programmers to decide whether network transparency for remote procedure call (RPC) is appropriate for their application...or not.

In time for EclipseCon, I intend to expose some of the Equinox p2 APIs as remote services, so that OSGi containers can be remotely provisioned and managed. If you would like to participate in this, please contact us and offer to contribute.

Project Diversity = Productivity?

2008-01-08T12:10:00.000-08:00

In today's New York Times is this intriguing interview:

In Professor's Model, Diversity = Productivity

Given the past year's frequent postings regarding project diversity (e.g Bjorn, Doug, Committer Reps, and many of the always-beautiful postings by Ed Merks), this research seems relevant.

I haven't had a chance to read the original research yet, but this caught my eye in the interview:

What the model showed was that diverse groups of problem solvers outperformed the groups of the best individuals at solving problems. The reason: the diverse groups got stuck less often than the smart individuals, who tended to think similarly.
The other thing we did was to show in mathematical terms how when making predictions, a group’s errors depend in equal parts on the ability of its members to predict and their diversity. This second theorem can be expressed as an equation: collective accuracy = average accuracy + diversity.

Real-Time Shared Editing over XMPP

2007-11-20T22:17:00.000-08:00

The ECF project has been adding some real-time collaboration functions for dev team support...for example, sharing selected text in Eclipse editors, as well as real-time collaborative editing...all over the ECF datashare API...which means you can use your favorite public IM accounts like google talk, or skype rather than be forced to have yet another account. We've also got some new sharing of Mylyn tasks as per bug 195737.

ECF 1.2

2007-10-22T17:15:00.000-07:00

ECF 1.2 has just been released. Highlights: lots of UI work (not all of it with zx's picture...is that a UI improvement? ;-), cool work on remote services, discovery API improvments, and a fair amount more. Lots of great community contributions via BugDay.

Remote OSGi Services: Questions of Transparency

2007-10-17T17:33:00.000-07:00

Recently, I've been having discussions with various folks about whether network transparency for remote OSGi services is a 'good thing'.

I think a reasonable summary of the discussion is that under the right conditions, it can be a good thing, because it makes it possible to easily create distributed applications using consistent APIs...without learning network specific service APIs.

OTOH, under the wrong conditions and assumptions, too much network transparency can make it very hard or impossible to build reliable distributed components or applications (for details see the classic Note on Distributed Computing Paper).

With our own Remote Services API in ECF, we've tried to create an API so that it's easy to use either approach...and attempt to steer clear of the religious wars. So, for example, in ECF here's how you register a remote service (for the 'server' of that service) to be accessed ...either transparently or not-transparently:

IConcatService serviceImpl = new ServiceImpl();
Dictionary props = new Hashtable();
props.put(Constants.LOCAL_SERVICE_REGISTRATION, "true");
// Register serviceImpl
context.registerRemoteService(new String[] {IConcatService.class.getName()}, serviceImpl, props);

And then on the service client, you can access it like any other OSGi service...and underneath a proxy is created (this looks exactly the same whether the service is local or remote):

ServiceReference ref = bc.getServiceReference(IConcatService.class.getName());
IConcatService concatService = (IConcatService) bc.getService(ref);
concatService.call();
// call concatService...calls proxy and sends the call to remote serviceImpl

Alternatively, clients can also access a IRemoteService instance via a service property, and do things like send an asynchronous message to invoke the service...for example:

ServiceReference ref = bc.getServiceReference(IConcatService.class.getName());
IRemoteService remoteService = (IRemoteService) ref.getProperty(Constants.REMOTE_SERVICE);
// Call it with listener for return/result. This returns immediately and then later
// calls the remoteCallListener
remoteService.callAsynch(remoteCall, remoteCallListener);

Of course you could claim that is chickening out on the transparency battles, but I tend to think of it as simply recognizing that there are lots of distributed applications out there that can (and should and will) be implemented on OSGi runtimes, and they all differ in their needs...e.g. for synch vs. asynch, or handling of partial failure, or in their timing/performance requirements.

Messaging for Servers

2007-09-10T13:44:00.000-07:00

Among some of the New and Noteworthy for ECF 1.1 are some new providers

ActiveMQ/Websphere CE (JMS)
BEA Weblogic (JMS)
JGroups

The first two depend upon/use commercial implementations of the Java Messaging Service (JMS) associated with Websphere Community Edition and BEA Weblogic, respectively. ActiveMQ is also an Apache open source project (activemq.apache.org). The third is an open source reliable multicast implementation used in JBoss called JavaGroups.

Included with each of these providers is an Eclipse IApplication that allows them to be run as a stand-alone OSGi/ECF-based server.

This also expands the set of providers that support the ECF remote services API: an API for invoking OSGi services remotely...via synchronous method call and/or asynchronous invocation. Each of these providers implements the ECF remote services API, showing that a single API for remoting really can (and does) run on multiple wire transports and multiple network topologies...an elusive goal for much of the web services world.

Remote OSGI over XMPP

2007-07-23T16:35:00.000-07:00

In a previous post, I showed how real-time collaboration could be added to Eclipse/RCP apps using ECF datashare API running on the ECF XMPP provider.

One can also use ECF to make remote OSGi calls using the ECF remoteservices API.

For example, on an ECF XMPP peer that is to be the 'server' for a given service (a trivial IConcatService in this case):


  // Get ECF adapter for remote services
  IRemoteServiceContainerAdapter adapter =  ... (e.g. from contacts list)
  // Get targetID[] buddies who are to receive service registration
  ID [] targetIDs = ... (e.g. from contacts list)
  Dictionary props = new Hashtable();
  props.put(Constants.SERVICE_REGISTRATION_TARGETS,targetIDs);
  // Create service instance...this does the actual concatenate
  Object concatService = new IConcatService() {
          public String concat(String string1, String string2) {
              return string1.concat(string2);
          }
      };
  // Register service instance
  adapter.registerRemoteService(new String[] { "IConcatService" }, concatService, props);
  // Service now registered

And for clients to lookup and use the service:


   // Get adapter for accessing remote services
  IRemoteServiceContainerAdapter adapter =  ...
  // Get remote service reference
  IRemoteServiceReference[] refs = adapter.getRemoteServiceReferences(null, "IConcatService", null);
  // Get service for remote reference
  IRemoteService service = adapter.getRemoteService(refs[0]);
  // Create remote call
  IRemoteCall remoteCall =  new IRemoteCall() {
          public String getMethod() {
              return "concat";
          }
          public Object[] getParameters() {
              return new Object[] { "Eclipse", " is cool" };
          }

          public long getTimeout() {
              return 3000;
          }};
  // Call synchronously
  String result = (String) service.callSynch(remoteCall);
  System.out.println("concat result is: "+result);
  // prints out:  concat result is: Eclipse is cool

Further, note that the client can use any of the 'style's of remote invocation desired:


IRemoteService.callSynch(...)
IRemoteService.callAsynch(...)
IRemoteService.fireAsynch(...)
IRemoteService.getProxy()

With 'getProxy()' this returns an instance that implements IConcatService, and allows direct method invocation:


IConcatService concat = (IConcatService) remoteService.getProxy();
String result = concat.concat("OSGi"," is cool");
System.out.println("concat result is: "+result);
// prints out:  concat result is: OSGi is cool

There is more test and example code available here. Note that with the IRemoteService.callAsynch, callSynch, and fireAsynch methods it is not even necessary that the client have the service interface locally. All that is needed is the string identifying the desired class name.

The support for remote services on the ECF generic provider is already in ECF 1.0.0, and the support for doing remote services on XMPP will be available in ECF 1.0.2.

Real-time Collab Over plain 'ol XMPP

2007-05-08T17:56:00.000-07:00

Something cool: real-time collaboration in Eclipse via XMPP. The point here is that with the ECF Datashare API, it's possible to send arbitrary messages to folks on your XMPP (or Google Talk, or soon Skype) buddy list, and with the new Eclipse 3.3 menuContribution API, it's very easy to create your own plugin, that uses ECF's MultiRosterView to add on your own actions (like making a Skype call, or sending an URL to team mate, or playing Sudoku with a buddy, etc). No need to understand XMPP, to use any of this, just send/receive data (byte []s).

Now available via dev/integration build. See here for links to latest download.

Here's a class which is responsible for sending/receiving the URL (most of it is UI code. See sendURL and handleMessage/showURL).

Skype Provider for ECF

2007-04-21T14:21:00.000-07:00

ECF is soon going to make available a dev build for a Skype Provider. If you have some appropriate sounds, please consider donating them to us to use as a ring tone within Eclipse. See bug #182840.

ECF 1.0.0 milestone 6 released

2007-04-02T09:54:00.000-07:00

ECF 1.0.0 milestone 6 released. Some cool new stuff...e.g.

1) A new 'bot' API/plugin (org.eclipse.ecf.presence.bot) to allow people to create IM and chat room robots that can respond to messages.
2) Committer Remy Suen has created a very cool Bot Framework Tutorial.
3) A history API for IM/chat room history.
4) Peer-to-peer file transfer over XMPP
5) Work on file transfer API to support the Equinox provisioning incubator effort.

Here's the download page: http://www.eclipse.org/ecf/downloads.php
New and Noteworthy: http://www.eclipse.org/ecf/NewAndNoteworthy.html

And we've done lots of work on making the ECF core and API plugins able to run on any/all Equinox runtimes (e.g. servers, RCP, Eclipse, etc). See this bug here for details: bug #176322.

Next Up: Implementation of ECF call API with Jingle (Google Talk), and Skype as well as support for Europa M7, further work on supporting Equinox provisioning as needed, a continuous build server for daily integration builds, and bug fixes (of course).

ECF 0.9.6 stable released

2007-01-14T08:30:00.000-08:00

ECF is releasing 0.9.6 stable today (Jan 13, 2007). It has quite a few positive changes in the simplification of the presence API, the easier creation and customization of roster user interfaces via the new plugin: org.eclipse.ecf.presence.ui, use of the new connect and configuration wizard extension points, and lots of bug fixes. See here for the download, here for the New and Noteworthy, and here for the planned and completed fixes and enhancements.

ECF 0.9.4 Stable Released

2006-12-04T13:16:00.000-08:00

ECF 0.9.4 stable released. Download here. New and Noteworthy here. Also created project milestone plan (tentative) and put here.

ECF 0.9.4 on 12/2/2006

2006-11-29T14:08:00.000-08:00

ECF 0.9.4 stable will be available on 12/2/2006. The new things will be two UI extension points, to allow ECF providers to create custom configuration wizards and connect wizards. The extension points are org.eclipse.ecf.ui.configurationWizards and org.eclipse.ecf.ui.connectWizards. Both are in the org.eclipse.ecf.ui plugin.

Also, the UI/Example application refactoring has begun, with initial changes/refactoring of the RosterView. For the next few releases (0.9.4+) there will be many more changes to the UI and example application code as the focus has shifted from UI to application refactoring.

ECF 0.9.3 stable released

2006-11-29T14:02:00.000-08:00

ECF 0.9.3 (the 'API refactoring' release) is available here. The new and noteworthy describes the changes made to the APIs. NOTE: This is a backward-compatibility breaking release, as some of the code moved out of existing packages and into new plugins and packages. The API refactoring was driven by requests and requirements described here.

OSGI components on server and ECF

2006-01-22T14:47:00.000-08:00

There's an excellent article about the recent Equinox incubator
work.

One point that's not mentioned in the article: in addition to web servers, collaboration and messaging servers (e.g. IM, file sharing, voip/conferencing, Asterisk, etc., etc) can also
benefit tremendously from being able to run the Eclipse/OSGI component model on the server. For example, for ECF's 'generic server' we will be able to introduce
plugins dynamically on both clients and servers, using the same basic component model and the same
plugin development tools (Eclipse). This is a big deal for ECF.

Here is the Equinox incubator project

ECF 0.6.0 and 0.6.1

2006-01-21T13:37:00.000-08:00

ECF 0.6.0 is available here
Here is the 0.6.0 New and Noteworthy.

We will be doing a 0.6.1 build during week of 1/23/06. The main new addition
for 0.6.1 will be a new API called 'datashare' that allows the easy
creation of various asynchronous channels that have different msg delivery
ordering semantics.

ECF 0.6.0

2006-01-10T21:33:00.000-08:00

The ETA for ECF 0.6.0 is 1/15/06. This release will be mostly API changes and bug fixes
rather than new features. Specifically:

SharedObjectFactory extension point (new) so that plugins can register themselves as implementers of shared object types

Fixes for JmDNS hang issues (zeroconf/bonjour provider implementation)

Revised API signature for ID and IContainer instance construction

Jingle

2005-12-30T16:55:00.000-08:00

Some very exciting things coming out of Google Talk. Jingle for example.

Greetings

2005-12-30T16:45:00.000-08:00

I (Scott Lewis) lead the Eclipse Communication Framework project