A question that I ask myself periodically is this: Why work on open source projects? In my case, why work on ECF? What is my motivation to do so?
There's an interesting book by Daniel Pink that examines human motivation called Drive: The Surprising Truth About What Motivates Us. There's also a nice YouTube video by the author from last year's TED conference.
In reading the first part of the book, I've resonated with his assertion that intrinsic motivation is sometimes stronger than extrinsic motivation...particularly when creativity (aka innovation) is involved. And I suspect I am not alone in this...at least among people who are passionate about innovation, community, software technology, and open systems.
I believe understanding motivations is important...because lots of assumptions about how innovation comes about...i.e. who does it and why...are actually dependent upon underlying assumptions about motivation.
Tuesday, January 12, 2010
Sunday, January 10, 2010
SOAP, REST, and ECF remote services
In addition to supporting the OSGi 4.2 remote services specification, we on the ECF team have also been working on support for accessing REST-style services, as well as those that use the Simple Object Access Protocol (SOAP).
The ability to support all these styles of service-oriented architecture is fundamentally enabled by ECF's provider architecture, but since mentioned that in a recent posting, in this post I'm going to touch on something useful that's enabled by this provider architecture.
Most remote services (whether web services, OSGi remote services, REST-based services, etc) have two basic roles
Typically, the service host is first 'registered'...to make it available for remote access, and after that the service consumer then accesses/uses the remote service (e.g. makes remote method calls on a proxy, issues http requests for resources, sends/receives messages, or has some other way of actually accessing/calling the remote service).
Note that the service host need not be exposed by a 'server' (although it frequently is)...in some systems clients can register/expose services as well as servers. And it's probably obvious that service consumers don't have to be 'clients' either...i.e. they can be servers that are communicating with other servers. This is why I use the role names 'host' and 'consumer' rather than 'server' and 'client' when referring to remote services.
OSGi remote services use the OSGi service registry...for both the host's registration of services, and the consumer's lookup and access to a remote service. Whatever the transport used to implement the distribution, typically both the host's registration and the consumer's lookup are done via the OSGi service registry. This typical use case implies, however, that both the host framework and the consumer framework have access to the OSGi service registry (since both registration and lookup are via the service registry).
But what if you would like to consume a service that doesn't use OSGi (and therefore doesn't have a service registry)? With ECF's remote services API, along with our recently added REST and SOAP support, we've enabled this use case (non-OSGi service host) while still providing the benefits of using OSGi remote services for the consumer.
How, you say? First of all, ECF's transport independent architecture allows clients to talk whatever protocol they require to communicate with a remote process...so, for example, the XMPP provider is able to communicate with any remote system that uses the (standardized) XMPP protocol. This does *not* have to be an OSGi-based system.
Second, ECF now has a very small, remote service client API...specifically to allow consumers to interact with non-OSGi services, while still using the OSGi service registry (if they wish) on the client. Since both the ECF remote services API, as well as this new remote service client API are also transport independent, and have both explicit support for REST-style and SOAP-based transports, providers for specific REST-based protocols and/or SOAP-based protocols are easy to create.
As an example, I recently created a SOAP/Web services client (for an existing web service), using Apache Axis (to convert wsdl to java), the WTP tooling (for automating the generation of the java code from wsdl), and this new remote services client API. To the non-OSGi service host, this client looks/behaves like all other clients. The same idea...for an existing REST-based API (Twitter in this case), is shown by this demo.
Unlike other clients, however, these clients can use OSGi to maximum value: i.e. to structure the client in a modular way, to handle the dynamic requirements of a remote/networked/unreliable service, or even (re) expose the proxy as a another remote service...that other consumers can access. This can be used for building load balancing of web services, or to aggregate sets of services...as well as other purposes.
Note there is no additional tooling required to build such a client...since your favorite tools for creating the SOAP-based and/or REST-based clients may be used, alongside APIs and tooling for interacting with the OSGi service registry...e.g. Eclipse PDE, OSGi declarative services and/or others.
The ability to support all these styles of service-oriented architecture is fundamentally enabled by ECF's provider architecture, but since mentioned that in a recent posting, in this post I'm going to touch on something useful that's enabled by this provider architecture.
Most remote services (whether web services, OSGi remote services, REST-based services, etc) have two basic roles
- The service host...aka the 'server'
- The service consumer...aka the 'client'
Typically, the service host is first 'registered'...to make it available for remote access, and after that the service consumer then accesses/uses the remote service (e.g. makes remote method calls on a proxy, issues http requests for resources, sends/receives messages, or has some other way of actually accessing/calling the remote service).
Note that the service host need not be exposed by a 'server' (although it frequently is)...in some systems clients can register/expose services as well as servers. And it's probably obvious that service consumers don't have to be 'clients' either...i.e. they can be servers that are communicating with other servers. This is why I use the role names 'host' and 'consumer' rather than 'server' and 'client' when referring to remote services.
OSGi remote services use the OSGi service registry...for both the host's registration of services, and the consumer's lookup and access to a remote service. Whatever the transport used to implement the distribution, typically both the host's registration and the consumer's lookup are done via the OSGi service registry. This typical use case implies, however, that both the host framework and the consumer framework have access to the OSGi service registry (since both registration and lookup are via the service registry).
But what if you would like to consume a service that doesn't use OSGi (and therefore doesn't have a service registry)? With ECF's remote services API, along with our recently added REST and SOAP support, we've enabled this use case (non-OSGi service host) while still providing the benefits of using OSGi remote services for the consumer.
How, you say? First of all, ECF's transport independent architecture allows clients to talk whatever protocol they require to communicate with a remote process...so, for example, the XMPP provider is able to communicate with any remote system that uses the (standardized) XMPP protocol. This does *not* have to be an OSGi-based system.
Second, ECF now has a very small, remote service client API...specifically to allow consumers to interact with non-OSGi services, while still using the OSGi service registry (if they wish) on the client. Since both the ECF remote services API, as well as this new remote service client API are also transport independent, and have both explicit support for REST-style and SOAP-based transports, providers for specific REST-based protocols and/or SOAP-based protocols are easy to create.
As an example, I recently created a SOAP/Web services client (for an existing web service), using Apache Axis (to convert wsdl to java), the WTP tooling (for automating the generation of the java code from wsdl), and this new remote services client API. To the non-OSGi service host, this client looks/behaves like all other clients. The same idea...for an existing REST-based API (Twitter in this case), is shown by this demo.
Unlike other clients, however, these clients can use OSGi to maximum value: i.e. to structure the client in a modular way, to handle the dynamic requirements of a remote/networked/unreliable service, or even (re) expose the proxy as a another remote service...that other consumers can access. This can be used for building load balancing of web services, or to aggregate sets of services...as well as other purposes.
Note there is no additional tooling required to build such a client...since your favorite tools for creating the SOAP-based and/or REST-based clients may be used, alongside APIs and tooling for interacting with the OSGi service registry...e.g. Eclipse PDE, OSGi declarative services and/or others.
Sunday, January 03, 2010
OSGi Remote Services from ECF
The ECF project has just finished our initial implementation of the OSGi 4.2 Remote Service specification (chapter 13 in compendium section).
I want to highlight a few distinctive features of ECF's implementation. I'll be doing other/more blog posts to go into details about some of these.
Transport Independence
ECF's implementation currently works with the following distribution transports: JMS (ActiveMQ), r-OSGi, JavaGroups, XMPP, Skype, ECF generic, and with the following discovery protocols: SLP, zeroconf, static xml file-based discovery. The ECF implementation is immediately usable with any...or all...of these providers...even within a single application if desired. Further, ECF's remote services and discovery APIs allow other distribution and/or discovery systems to be plugged in underneath our implementation...meaning that other distribution systems (e.g. open source distribution systems such as Riena, Apache CXF, and/or proprietary distribution systems) can reuse/leverage our implementation...and avoid the work otherwise necessary to implement the OSGi specification themselves. ECF's implementation can also run alongside other remote services implementations without conflict.
Asynchronous and Synchronous
Distributed applications and services frequently need to use asynchronous/non-blocking remote invocation patterns...in addition to synchronous, proxy-based remote method call. Right now, ECF has built-in support for both asynchronous invocation (e.g. asynchronous listeners, futures, one-ways), as well as proxy/synchronous invocation...giving the services and application programmer a transport-independent choice of which to use. Currently, the OSGi 4.2 spec does not yet specify asynchronous invocation patterns for remote services, but ECF's implementation does have it.
Lightweight
The entire ECF implementation of the spec...along with ECF's remote services API...is < 150k of code. Further, the OSGi execution environment requirements are minimal (CDC 1.1/Foundation 1.1). Combined with a small provider (like r-OSGi), and this allows even small devices to both expose/host and consume standardized remote services.
Standard, Open Source, Open Team, Open Process
The ECF implementation is fully compliant with the OSGi 4.2 remote services specification...for any and all current and future ECF remote services providers. Further, as mentioned above, other providers can now get this compliance for free. Of course, ECF's implementation is open source, but it is also produced by a vendor-neutral project team, and fully open community-driven process. In addition, we have support for a distributed version of the OSGi EventAdmin service, for doing publish-and-subscribe-based applications using a standard event bus API/service.
Extensible
ECF's implementation is deeply extensible, allowing control or customization of every aspect of the distribution of an OSGi service (e.g. custom discovery, custom marshalling/serialization, custom transport/wire protocol, etc) for those that need it. This is enabled by the open APIs that ECF exposes, and allows a wide variety of deployment requirements and application-level use cases to be easily supported.
Summary
It's All About Modules. ECF's modular structure is indeed enabled by OSGi's modularity, but we have also applied modular design to every level of APIs and implementations. For example, lightweightness, extensibility, and transport independence all are enabled by our modular designs...to separate API from implementation, and to separate distinct subsystems (e.g. discovery and remote services APIs).
We are preparing a release that includes this code...so to immediately download/use ECF's implementation, or engage the community...for requesting enhancements, reporting bugs, contributing (e.g. examples, providers, customizations), helping with testing, or getting support please consider joining the ecf dev mailing list or see the ecf dev resources page.
I want to highlight a few distinctive features of ECF's implementation. I'll be doing other/more blog posts to go into details about some of these.
Transport Independence
ECF's implementation currently works with the following distribution transports: JMS (ActiveMQ), r-OSGi, JavaGroups, XMPP, Skype, ECF generic, and with the following discovery protocols: SLP, zeroconf, static xml file-based discovery. The ECF implementation is immediately usable with any...or all...of these providers...even within a single application if desired. Further, ECF's remote services and discovery APIs allow other distribution and/or discovery systems to be plugged in underneath our implementation...meaning that other distribution systems (e.g. open source distribution systems such as Riena, Apache CXF, and/or proprietary distribution systems) can reuse/leverage our implementation...and avoid the work otherwise necessary to implement the OSGi specification themselves. ECF's implementation can also run alongside other remote services implementations without conflict.
Asynchronous and Synchronous
Distributed applications and services frequently need to use asynchronous/non-blocking remote invocation patterns...in addition to synchronous, proxy-based remote method call. Right now, ECF has built-in support for both asynchronous invocation (e.g. asynchronous listeners, futures, one-ways), as well as proxy/synchronous invocation...giving the services and application programmer a transport-independent choice of which to use. Currently, the OSGi 4.2 spec does not yet specify asynchronous invocation patterns for remote services, but ECF's implementation does have it.
Lightweight
The entire ECF implementation of the spec...along with ECF's remote services API...is < 150k of code. Further, the OSGi execution environment requirements are minimal (CDC 1.1/Foundation 1.1). Combined with a small provider (like r-OSGi), and this allows even small devices to both expose/host and consume standardized remote services.
Standard, Open Source, Open Team, Open Process
The ECF implementation is fully compliant with the OSGi 4.2 remote services specification...for any and all current and future ECF remote services providers. Further, as mentioned above, other providers can now get this compliance for free. Of course, ECF's implementation is open source, but it is also produced by a vendor-neutral project team, and fully open community-driven process. In addition, we have support for a distributed version of the OSGi EventAdmin service, for doing publish-and-subscribe-based applications using a standard event bus API/service.
Extensible
ECF's implementation is deeply extensible, allowing control or customization of every aspect of the distribution of an OSGi service (e.g. custom discovery, custom marshalling/serialization, custom transport/wire protocol, etc) for those that need it. This is enabled by the open APIs that ECF exposes, and allows a wide variety of deployment requirements and application-level use cases to be easily supported.
Summary
It's All About Modules. ECF's modular structure is indeed enabled by OSGi's modularity, but we have also applied modular design to every level of APIs and implementations. For example, lightweightness, extensibility, and transport independence all are enabled by our modular designs...to separate API from implementation, and to separate distinct subsystems (e.g. discovery and remote services APIs).
We are preparing a release that includes this code...so to immediately download/use ECF's implementation, or engage the community...for requesting enhancements, reporting bugs, contributing (e.g. examples, providers, customizations), helping with testing, or getting support please consider joining the ecf dev mailing list or see the ecf dev resources page.
Subscribe to:
Posts (Atom)