Saturday, April 10, 2010

OSGi Remote Services from ECF - Distribution

In a previous posting, I discussed the use of the ECF discovery API as part of our implementation of the OSGi 4.2 remote services specification.

The second major part of ECF's implementation of OSGi 4.2 remote services is distribution.

What is Distribution?

Distribution is what happens to actually invoke a remote service and optionally return some result. Here's a brief summary of the essential functions of distribution:

Remote Service Consumer

[Prior to caller actually using service]
1. Create a proxy for the remote service
[When caller actually uses remote service]
2. Marshal/Serial any arguments for the remote call
3. Put call request (method and serialized parameter) on the wire using some protocol

Remote Service Host

1. Take request off the wire (using same protocol)
2. Un-marshal method and arguments
3. Lookup corresponding service/method
4. Invoke appropriate service with given arguments
5. Marshal return value
6. Put result on the wire using some protocol

Remote Service Consumer

4. Take response off the wire (using same protocol)
5. Un-marshal result
6. Return result to caller

One way to think of it is that distribution is responsible for making what looks like a local method call to a local OSGi service actually be a remote call.

Two of the critical functions of distribution...for both Consumer and Host are

1) Marshaling/Serialization...of arguments and return values
2) Use some protocol to communicate request/response over network

As with discovery, the ECF project has created an abstract API for distribution, which is called the ECF remote services API. Like other ECF APIs, this is a transport-independent API, which exposes a programmatic way to accomplish the functions of distribution (as described above), but does not imply/require any particular implementation of marshaling/serialization, nor imply/require any particular network protocol.

ECF has providers that define specific implementations of marshaling and network protocol. For example, we have a REST-API, that supports the creation of specific REST providers. This REST API includes JSON and/or xml-based serialization, and uses HTTP as the protocol. We also have a similar SOAP API for SOAP-based services.

We also have a number of other providers that are complete and available...e.g. ones based upon XMPP, JMS, ECF generic, Skype's app protocol, JavaGroups/multicast. Further, since all of these providers are open source, if desired they can be extended or copied to implement custom providers based upon whatever serialization and wire protocol (e.g. an existing system) is desired...with our without the ECF team's involvement.

Note the ECF implementation of the OSGi 4.2 remote services specification is guaranteed to work with any of these providers...no matter who writes it. This because our implementation of the OSGi 4.2 remote services spec simply uses any all implementations of the ECF remote service API (no matter what the serialization and/or networking protocol).

The flexibility here is extremely useful when selecting serialization formats and/or network protocols, because there are/will always be so many serialization formats and/or network protocols to choose from...their appropriateness will always depend upon the use case...as well as the need for integration with existing systems. For example...e.g. json over http, custom xml over http, object serialization over tcp, xml over jms, soap over http, etc, etc...which makes sense depends upon the use case and things like networking/interoperability requirements.

Since this distribution function is separated out into a distinct, abstract, module (i.e. the ECF remote services API), it makes it possible to mix and match existing protocols and new protocols...both closed and open...with existing serialization formats or new serialization formats...crossed with whatever discovery protocol is appropriate and/or desired.

7 comments:

Anonymous said...

Hi Scott,

Thank you for the great blog! It is really worth reading it! After your posts about OSGI Discovery and Distribution several questions came to my mind. Please correct me if I am wrong. In ECF you have several implementations for Service Discovery - slp, zeroconf, zookeeper and Service Distribution - ecf generic, jgroups, r-osgi, etc. So I can pick any of the discovery implementations and it should work with any of the distribution implementations, is this correct?
Is it possible to use jgroups for discovery and distribution?
What is the benefit of using r-osgi instead of ecf generic?
Thank you again for the great work!

Regards,
John

Scott Lewis said...

@John,

>So I can pick any of the discovery >implementations and it should work >with any of the distribution >implementations, is this correct?

Yes, that's correct.

>Is it possible to use jgroups for >discovery and distribution?

Yes it's possible to use jgroups for both. We currently have a jgroups-based *distribution* provider, but not a jgroups-based discovery provider (the distribution provider it at http://ecf1.osuosl.org). It would be possible to create a jgroups-based discovery provider, and if you decide to do this please work with us and consider contributing it back to the community.

>What is the benefit of using >r-osgi instead of ecf generic?

r-OSGi is an http-based protocol and so is able to more easily work through firewalls if that's a concern for you. Also, r-OSGi existed before ECF's remote services work, so it's possible to interoperate with existing services based upon r-osgi alone (i.e. without ECF).

They also differ in their implementation details.

Thanks for the nice words. Please make ECF known to others if you like it, use it, or want to enhance it.

Unknown said...

Hi scott
I am making a service call where my method argument is a bean which is imported in both consumer and provider bundle.But method call is failing saying Class not found sharedObjectID=StringID[org.eclipse.ecf.remoteservice.IRemoteServiceContainerAdapter] containerID=StringID[YXlTL5AXRQQsSboUM2GvlHG0zZI=] .Why is it refering to shared object.It should be simple serialize and deserialize

Unknown said...

Hi scott
I am making a service call where my method argument is a bean which is imported in both consumer and provider bundle.But method call is failing saying Class not found sharedObjectID=StringID[org.eclipse.ecf.remoteservice.IRemoteServiceContainerAdapter] containerID=StringID[YXlTL5AXRQQsSboUM2GvlHG0zZI=] .Why is it refering to shared object.It should be simple serialize and deserialize

Scott Lewis said...

@anonymous: For some of the providers, the serialization/deserialization of parameters is done by a shared object of class RegistrySharedObject. I would check to make sure that your parameter class (and all referenced classes) is declared as Serializable *and* the package it is in is exported.

Anonymous said...

Does ECF support inout argument usage, or is the return value of the called service the only marshalled return data.

Scott Lewis said...

Anonymous said:

>Does ECF support inout argument >usage, or is the return value of >the called service the only >marshalled return data.

With the existing providers, the return value of the called service is the only returned data.

However, ECF supports the notion of creating one's own remote service provider (specifically, creating an implementation of the org.eclipse.ecf.remoteservice API)...and/or extending one of the existing ones...and one's own implementation is free to do whatever is desired with method arguments, the wire protocol, and return values.