Saturday, February 06, 2021

gRPC Code Generation using Bndtools and ECF Remote Services

There's a new video tutorial that demonstrates using ECF Remote Services, bndtools, and Eclipse to create an OSGi Remote Service.   

Bndtools has recently added the ability to run code generators as part of a bnd-based project, and with ECF's bndtools workspace template, a single proto3 file added to a project will automatically generate an entire Java remote service API and update/regenerate the API as changes are made to the proto3 file.   No command-line execution of protoc needed.

Further with ECF's project templates, the generated API can be easily implemented and exported as an OSGi Remote Service.

Please watch the video here

Thursday, January 07, 2021

ECF 3.14.19 released - simplify remote service discovery via properties

 ECF 3.14.19 has been released.

Along with the usual bug fixes, this release includes new documentation on the use of properties for discovering and importing remote services.   The docs describe the use of properties files for simplifying the import of remote services.   

This capability is especially useful for Eclipse RCP clients accessing Jax-RS/REST remote services.

Patrick Paulin describes a production usage his blog posting here.

Tuesday, December 08, 2020

Using properties to simplify discovery of OSGi Remote Services

OSGi Remote Services are discovered by ECF's Remote Services implementation in two ways:  

1. Via a network discovery protocol provider such as:  Zeroconf, jSLP, etcd, Zookeeper, or some custom protocol

2. Via an xml format known as an Endpoint Description Extender Format (EDEF)

 The EDEF format is specified by the OSGi Remote Service Admin specification.   

When importing an EDEF-defined remote service, it's typically necessary to construct the entire EDEF file 'by hand' rather than having he EDEF generated automatically.  This can be quite complicated to construct by hand as some properties are required, others are optional and it's not obvious what all of the values must be for successful import.

A new capability has been added to ECF's Remote Service Admin implementation that allows EDEF Properties to be used with the EDEF, thus simplifying the creation of remote service consumers that use EDEF for import.

This capability was added to support the usage of JaxRS Remote Services in an Eclipse RCP client.  See a description of this use case here.



Monday, September 21, 2020

Using gRPC-java code generation to create OSGi Services

OSGi Services are usually first created by declaring a java service interface class.  As an OSGi service, this interface class serves as both the name for the service in the service registry, and defines the service contract (i.e. the interface method signatures...i.e. the method name, argument types, and return types) for that version of the service.

gRPC (Google RPC) is a popular and high-performance rpc approach that allows developers to define networked services based upon protocol buffers (proto3).

By extending bndtools recently-added code generation capability, it's now possible to generate an OSGi (remote) service API from just a proto3 service declaration.  All the classes necessary for an OSGi Remote Service API (service interface, arg and return types) can be generated by bndtools within Eclipse from a single proto3 file, immediately and completely.

Ready to implement-and-consume OSGi Services can be generated by Eclipse+bndtools + a proto3 service declaration.

Further, the proto3 service declaration can be modified, and the tooling will immediately generate new service API classes, compile, and package them into a bundle, all from within Eclipse+bndtools.

To get this bndtools-grpc generation with an example see here.




Tuesday, July 14, 2020

ECF 3.14.12 released - Now with gRPC for OSGi Remote Services

ECF 3.14.12 was just released

Highlights of this Release

New OSGi Remote Services Distribution provider based upon gRPC/Protocol Buffers. Along with a grpc-osgi-generator project...which allows the generation of a service API from a proto3 service declaration...this provider allows gRPC-based services to be exported and imported as OSGi Remote Services.  This now includes support for unary, server-streaming, and client-streaming gRPC calls.

Enhanced Support for Bndtools-based development of OSGi Remote Services.   The ECF Bndtools Workspace now includes the latest version of ECF Remote Services, along with the gRPC distribution provider, a Hazelcast-based discovery and distribution provider, and project and bndrun templates for creating, running, testing, and debugging OSGi Remote Services in Eclipse+Bndtools 5.


Tuesday, May 05, 2020

Using Google's grpc-java for OSGi Remote Services


A cool thing about Google's grpc is that a service creator can declare a service via a protocol buffers file (.proto file), and then the protoc compiler (along with grpc-java compiler plugin) generates many of the Java classes for both implementing and using that service.

OSGi Remote Services require a service interface to represent the service contract, and this service interface is usually created directly by the programmer.   Through a additional plugin, protoc can now generate a OSGi service interface along with all grpc classes... from the .proto file service declaration.   

For example, consider the following protocol buffers input file:
syntax = "proto3";
package grpc.health.v1;
option java_multiple_files = true;
option java_outer_classname = "HealthProto";
option java_package = "io.grpc.health.v1";
message HealthCheckRequest {
  string message = 1;
}
message HealthCheckResponse {
  enum ServingStatus {
    UNKNOWN = 0;
    SERVING = 1;
    NOT_SERVING = 2;
    SERVICE_UNKNOWN = 3;  // Used only by the Watch method.
  }
  ServingStatus status = 1;
}
service HealthCheck {
  // Unary method
  rpc Check(HealthCheckRequest) returns (HealthCheckResponse);
  // Streaming method
  rpc Watch(HealthCheckRequest) returns (stream HealthCheckResponse);
}
Running protoc+grpc-java+grpc-osgi-generator on this file results in generation of a HealthCheckService class along with all message classes (e.g. HealthCheckRequest, HealthCheckResponse, HealthProto, etc).   All of the Java classes in this example directory were created simply by running protoc+grpc-java+grpc-osgi-generator on the above proto file.

The generated Java classes can then be used to implement an OSGi Remote Service, with HealthCheckService as the service interface.   Tt runtime, the HealthCheckServiceImpl can be exported (via the Grpc Provider) which uses grpc to provide the comm and json serialization for the HealthCheckService method calls.

The net effect is that remote service programmers can easily and quickly go from abstract service declaration (in proto file) to a running/functioning OSGi remote service:
  1. Declare a service in proto file -- example proto file
  2. Run protoc+grpc-java+grpc-osgi-generator to generate the Java code for the declared service - example Java generated code
  3. Implement the service API - example service implementation
  4. Use Declarative Services to export using ECF Remote Services + Grpc Distribution Provider - example (see @Component annotation for OSGi Remote Services-required service properties to trigger export)
The remote service programmer writes no communication nor serialization code (both are provided by the Grpc distribution provider).   See here for the complete generated healthcheck api plugin, here for the impl plugin, and here for a simple remote service consumer.

Tuesday, March 03, 2020

ECF 3.14.7 released

ECF 3.14.7 has been released and may be downloaded here.

In concert with this bug fix release have been a number of additions to ECF's github projects for Remote Services Development.

Distribution and Discovery Providers

Enhanced:  Hazelcast-based Distribution Provider v1.7.0.  Upgraded to use Hazelcast 4
Enhanced:  System and Service-Properties docs for Distribution Providers and Discovery Providers

Bndtools Development

Enhanced:  Bndtools Workspace template with new Bndrun templates for Remote Services Development
Enhanced:  Tutorial for using Bndtools for Remote Services Development

Saturday, August 31, 2019

OSGi Remote Services with Apache Dubbo

ECF's implementation of OSGi R7 Remote Services allows for replacing the underlying distribution system (repsonsible for the object serialization, transport, and other things). 

This makes it relatively easy to replace one kind of distribution (e.g. Jersey, ActiveMQ) with other/new distribution systems. 

Apache Dubbo has recently been contributed to Apache, and we've created a distribution provider based upon Apache Dubbo.

Here's a list of open ECF Remote Service distribution providers.   If you would like Remote Services support for a particular transport, or you've created your own distribution (or discovery) based upon some other transport and wish to make it available to others please let us know.


Wednesday, April 03, 2019

New Release: Python<->Java Remote Services

There is a new release (2.9.0) of the ECF distribution provider for OSGi R7 Remote Services between Java and Python.

This release has:

An upgraded version of Py4j
An upgraded version of Google Protocol Buffers
Enhancements to the distribution provider based upon the improved Py4j and Protobuf libs

In this previous blog posting there are links to tutorials and examples showing how to use remote services between Python<->Java.

Python<->Java remote services can be consumed or implemented in either Java or Python.

Monday, February 25, 2019

RESTful OSGi R7 Remote Services with Jersey 2.28 or Apache CXF 3.3

For some time, ECF has had remote service distribution providers that use the Jersey or the CXF implementation of standard Java API for RESTful Web Services (JaxRS). 

These distribution providers allow OSGi R7 Remote Services to be defined via JaxRS annotations and implemented by either Jersey 2.28 or CXF 3.3

OSGi R7 Remote Services provides support for renite service discovery, dynamics, versioning, configuration and extension of the distribution providers, and asynchronous remote calls as well as other features of the OSGi R7 Remote Services and Remote Service Admin specs.

This tutorial shows the use of OSGi Remote Services with these JaxRS distribution providers on Apache Karaf.

There is also a new version of the ECF Bndtools workspace template with example Bndtools projects showing the use of these distribution providers to define, configure, run and deploy RESTful OSGi R7 Remote Services with Bndtools 4.2+.

Sunday, November 04, 2018

ECF 3.14.4 released

ECF 3.14.4 was recently released.  This was a bug-fix release.  There are notes on some of the recent additions here.

Wednesday, August 22, 2018

OSGi Remote Services Between Python and Java

In a previous post, I described the support for OSGi Remote Services and Remote Service Admin in iPOPO 0.8.0 release.   The previous post refers to a tutorial showing a Python service impl and Python consumer.

Python<->Java Distribution Provider

Included with iPOPO 0.8.0 is a distribution provider that allows remote services between Java and Python frameworks.   For example, this is a tutorial, that uses Karaf on the Java side as the remote service implementation, and has a Python consumer that calls the Java-implemented remote service.

Python Service Implementation with Java Consumers

This distribution provider also supports Python-implemented remote services, with Java/OSGi consumers.   With iPOPO for dynamic service injection in Python, and Declarative Services for Java/OSGi, this allows very easy Python<->Java service-level interaction, with support for all dynamics, RSA management agent, extensible/customizable topology management, management of complicated service dependencies, pluggable local and network discovery, and other RS/RSA features handled consistently in both Java and Python.  As well, the use of OSGi-specified EndpointDescription service metadata allows service-level interoperability across languages.

Python<->Java with Protocol Buffers Serialization

Also included with this distribution provider is serialization using Google's protocol buffers.  This allows open, extensible, and efficient rpc between Python and Java.


Tuesday, August 21, 2018

Python for OSGi Remote Services

The iPOPO project is a Python implementation of key parts of a standard OSGi framework...e.g. bundles, the service registry and servicereference api, and a dynamic service injection framework similar to the Apache iPOJO project...thus the name iPOPO.

With the 0.8.0 release of iPOPO, there is now a Python implementation of the OSGi Remote Services and Remote Service Admin (RSA) specifications.   To distinguish from the previously-provided remote services in iPOPO, this is known as RSA Remote Services.

iPOPO's RSA Remote Services has many of the same advantages as Java-based Remote Services/RSA.  Some of these advantages:

Decoupling - name/service contract is decoupled from the implementation (and distribution)

Dynamics - The service registry dynamics behavior, along with all notifications, etc is available in Python

Injection and Service Dependency Management - iPOPO provides service injection and dependency management, built upon the Python API

Standard RemoteServiceAdmin Management Agent, RSA Console Commands

API for Distribution and Discovery Providers - There are documented APIs for creating new distribution and discovery providers, making it easy to support other transports and implementations for distribution (e.g. REST/JaxRS, Jsonrpc, MQTT, Zeroconf Discovery, etc).

Other advantages of Java-based OSGi services are described here.   All of these advantages apply to Python/iPOPO-based services, but Python can be used to implement and/or consume services.

Currently, there are two distribution providers included with iPOPO 0.8.0:  XmlRpc, Python-Java and one discovery provider: etcd.   See here for tutorials showing their usage with included sample remote services. 

Other distribution and discovery providers are being considered or worked on.  If you are interested in seeing a particular transport supported for distribution or discovery please open an issue on the iPOPO project.

Remote Services between Python and Java

The Python-Java distribution provider makes it possible to use OSGi Remote Services between Python and Java...on both sides.    This allows Remote Services to be exported from Python, and discovered/imported and consumed in Java, or exported from Java and discovered/imported/consumed from Python.  See here for a sample and tutorial.   These capabilities and the underlying distribution provider will be described in more detail in a subsequent posting.


Tuesday, July 03, 2018

Rest Remote Services with CXF or Jersey

ECF's Photon Release now includes an example of using Karaf with the JaxRS distribution provider.

This distribution provider now fully supports OSGi R7 remote services, including async remote services, using either the Jersey or CXF JaxRS implementations.

This allows service developers to easily use only JAX-RS annotations to define and implement OSGi R7 remote services.

Wednesday, June 20, 2018

ECF Photon supports OSGi R7 Async Services - part 2

In a previous post, I described a usage of OSGi R7's Async Remote Services. This specification makes it easy to define, implement and use non-blocking remote services. ECF's implementation allows the use of pluggable transports...known as distribution providers.

Here's a partial list of distribution providers:

R-OSGi
ECF generic
JMS/ActiveMQ
XML-RPC
Hazelcast
MQTT
Jax-RS Jersey
Jax-RS CXF
JavaGroups
Python.Java (Supports async remote services between Java and Python with protocol buffers serialization)

 It's also straightforward to creation your own distribution provider, using private or legacy transport and/or serialization. This can be done by extending one of the distribution providers above or creating a new one.

Most of these distribution providers have updated examples and/or tutorials, and many of them now have templates included in the Bndtools (4.0+) Support added for Photon.

Separating the remote service contract from the underlying distribution provider via OSGi remote services allows implementers and consumers to create, debug, and test remote services without being bound to a single transport, while still allowing consistent (specified) runtime behavior.

For more info and links, please see the New and Noteworthy.

Tuesday, May 01, 2018

ECF Photon supports OSGi Async Remote Services

In a previous post, I indicated that ECF Photon/3.14.0 will support the recently-approved OSGi R7 specification.   What does this support provide for  developers?

Support osgi.async remote service intent

The OSGi R7 Remote Services specification has been enhanced with remote service intents.  Remote Service Intents allow service authors to specify requirements on the underlying distribution system in a standardized way.   Standardization of service behavior guarantees the same runtime behavior across distribution providers and implementations.

The osgi.async intent allows the service interface to use return types such as Java8's CompletableFuture or OSGi's Promise.   With a supporting distribution provider, the proxy will automatically implement the asynchronous/non-blocking behavior for the service consumer.

For example, consider a service interface:
public interface Hello {
    CompletableFuture<String> hello(String greetingMessage);
}
When an implementation of this service is registered and exported as a remote service with the osgi.async intent:
@Component(property = { "service.exported.interfaces=*", "service.intents=osgi.async" })
public class HelloImpl implements Hello {
    public CompletableFuture<String> hello(String greetingMessage) {
          CompletableFuture<String> future = new CompletableFuture<String>();
          future.complete("Hi.  This a response to the greeting: "+greetingMessage);
          return future;
    }
}
Then when a Hello service consumer (on same or other process) discovers, imports and then remote service is injected by DS:
@Component(immediate=true)
public class HelloConsumer {

    @Reference
    private Hello helloService;

    @Activate
    void activate() throws Exception {
        // Call helloService.hello remote service without blocking
        helloService.hello("hi there").whenComplete((result,exception) -> {
            if (exception != null)
                exception.printStackTrace(exception);
            else
                System.out.println("hello service responds: " + result);
        });
    }
}
The injected helloService instance (a distribution-provider-constructed proxy) will automatically implement the asynchronous remote call.   Since the proxy is constructed by the distribution provider, there is no need for the consumer to implement anything other than calling the 'hello' method and handling the response via the Java8-provided whenComplete method.   Java8's CompletionStage, Future, and OSGi's Promise are also supported return types.  (Only the return type is used to identify asynchronous remote methods, any method name can be used).  For example: the following signature is also supported as an async remote service:
public interface Hello {
    org.osgi.util.promise.Promise<String> hello(String greetingMessage);
}

Further, OSGi R7 Remote Services supports a timeout property:
@Component(property = { "service.exported.interfaces=*", "service.intents=osgi.async", "osgi.basic.timeout=20000" })
public class HelloImpl implements Hello {
    public CompletableFuture<String> hello(String greetingMessage) {
          CompletableFuture<String> future = new CompletableFuture<String>();
          future.complete("Hi.  This a response to the greeting: "+greetingMessage);
          return future;
    }
}
With ECF's RSA implementation and distribution providers, this timeout will be honored by the underlying distribution system. That is, if the remote implementation does not return within 20000ms, then the returned CompletableFuture will complete with a TimeoutException.

Async Remote Services make it very easy for service developers to define, implement, and consume loosely-coupled and dynamic asynchronous remote services.   It also makes asynchronous remote service contracts transport independent, allowing the swapping of distribution providers or creating/using custom providers without changes to the service contract.

For the documented example code, see here

Monday, April 23, 2018

ECF Photon adds Gogo Commands

A third major enhancement for ECF's implementation of OSGi Remote Services is the addition of Apache Gogo console commands for examining the existing state of remote services, and the ability to export a service and import an endpoint from the OSGi console.

See this wiki page describing the new commands and their usage.


Thursday, April 19, 2018

ECF Photon supports Bndtools

A second major enhancement for ECF Photon is adding support for using Bndtools to develop and test OSGi Remote Services.   Bndtools is increasingly popular for developing OSGi-based applications and frameworks, and we've added support for Bndtools Workspace, Project, and Run Descriptor templates for developing and testing remote services.

Initial documentation is available at Bndtools Support for Remote Services Development.

Note that these templates and the RSA impl may change slightly before ECF Photon, and new/additional templates will be added to (e.g.) support other distribution and discovery providers.