Building MCP Servers: Tool Descriptions + Service Contracts = Dynamic Tool Groups

The Model Context Protocol (MCP) can easily be used to expose APIs and services in the form of MCP tools...i.e. functions/methods that can take input, perform some actions based upon that input, and produce output, without specifying a particular language or runtime.

OSGi Services (and Remote Services) provide a dynamic, flexible, secure environment for microservices, with clear well-established mechanisms for separating service contracts from service implementations.

One way to think of a service contract for large language models (LLMs) is that the service contract can be enhanced to provide LLM-processable metadata for each tool/method/function.  Any service contract can still be used by human developers (API consumers), but with tool-specific meta-data/descriptions added, larger service contracts can be also used by any model.

Since service contracts in most languages are sets of functions/methods, the service contract can also be used to represent groupings of MCP tools, or Dynamic MCP ToolGroups.   The example on the MCPToolGroups page and on the Bndtools project templates, is a simple example of grouping a set of related functions/methods into a service contract and including MCP tool meta-data (tool and tool param text descriptions).

Building MCP Servers: Alternative Transports

Many developers are creating their own MCP Servers, to integrate their applications and APIs with AI models in a consistent way across multiple models.

The MCP SDKs currently provide two transports:  stdio (standard io), and http sse.  

I've created a new open source repo for alternative mcp transports.  

The intention is to provide alternative transports with other hardware, security, and trust properties.  

The first instance is based upon Unix Domain Sockets, which restricts the inter-process communication to processes running on the same operating system.

Building MCP Servers - part 3: Security

There have been recent reports of critical security vulnerabilities on the mcp-remote project, and the mcp inspector project.

I do not know all the technical details of the exploits, but it appears to me that in both cases it has to do vulnerabilities introduced by the MCP Server implementation. and use of the stdio MCP transport.

I want to emphasize that example described in these two posts

Integration via Remote Tools

Example Using Remote Services

is using mechanisms that are...though heavy usage by commercial server technologies over the past 10 years...not subject to the same sorts of remote vulnerabilities seen by the mcp-remote and mcp-inspector projects.   

Also, the flexibility in discovery and distribution provided by the RSA Specification and the RSA implementation used, allows for addressing MCP Server remote tools, or protocol weaknesses, quickly and easily, without having to update the MCP Server or tooling implementation code.  

 

 

 

Building MCP Servers - part 2: Example Using Remote Services and Bndtools

In a previous post, I described how using dynamic remote tools could make building MCP Servers more flexible, more secure, and more scalable.  In this post, I show an example MCP Server that uses remote services and Bndtools to build.

To use/try this example yourself see Installing Bndtools with Remote Services Support below.

Create a Bndtools Workspace via File->New->Bndtools Workspace, and choose the ECF/bndtools.workspace template from this dialog

Choose Next->Finish

Choose File->New->Bnd OSGi Project to show this dialog

There are 4 project templates.  They should each be created in your workspace in turn:

1. MCP ArithmeticTools API Project  (example name: org.test.api)

2. MCP ArithmeticTools Impl Server Project (ArimethicTools RS Server) (example name: org.test.impl)

3. MCP ArithmeticTools Consumer Project (MCP Server Impl/ArithmeticTools Consumer) (example name: org.test.mcpserver)

4. MCP ArithmeticTools Test Client Project (MCP Client) (example name: org.test.mcpclient)

Note:  For the Impl/Consumer/Test Client project creations you will be prompted to provide the name of the project that you specified for the API Project (1)

Click Finish for each of the first 3 (server) projects.

If you wish to use your own MCP Client, and start the MCP Server (MCP ArithmeticTools Consumer Project) from your own MCP Client (stdio transport), see the Readme.md in the MCP ArithmeticTools Consumer project.

MCP ArithmeticTools Example Test Client

There is also a MCP Example Test Client Project template, implemented via the MCP Java SDK that makes a couple of test calls to the MCP Server/Arithmetic Tools Consumer server.

Note:  When creating the test client, the project template will as for you to specify the names of the API project (1) and the MCP Server/ArithmeticTools Consumer project (3).   For example:

Click Finish

You should have these four projects in the Bndtools Explorer

You can open source for any/all the projects, set breakpoints, etc.

Launching the ArithmeticTools Remote Service Impl server

The ArithmeticTools Impl server (2) must be launched first

To launch the ArithmeticTools Impl (2), see the Readme.md in that project

Launching MCP Client and MCP Server (with stdio transport)

The MCP Client (4) may then be launched, and it will launch the MCP Server (3) and use the MCP stdio transport.   The MCP Client (4) Readme.md more info on launch and expected output of the MCP Client.

You may examine or set breakpoints in the ArithmeticTools Impl Server (2) or the MCP Client (4) to examine the communication sequence for calling the ArithmeticTools.add or multiple tools.

Installing Bndtools with Remote Services Support

1. Install Bndtools 7.1 

2. Add ECF Latest Update site to Eclipse install with URL: https://download.eclipse.org/rt/ecf/latest/site.p2

3. Install Feature:  SDK for Bndtools 7.1+

Building MCP Servers: Integration via Remote Tools

It has become popular to build Model Context Protocol Servers.  This makes a lot of sense from the developer-as-integrator point of view, since the MCP specification and multi-language SDKs make it possible to easily integrate resources, prompts, and tools into multiple LLMs without having to use the model-and-language-specific model APIs directly.

MCP tools spec provides a general way for LLMs to use tool meta-data (e.g. text descriptions) for the tool's required input data, behavior, and output data.  These text descriptions can then be used by the LLM...in combination with interaction with the user...to decide when and how to use the tool...i.e. to call the function and provide some output to the LLM and/or the user.

Building an MCP Server

When creating a new MCP Server, it's easiest to create the tool metadata and implement the tool functionality as part of a new MCP server implementation.   But this approach requires that every new tool (or integration with existing API/servers) results in a new MCP server or an update/new version of an existing MCP server.

Remote Tools

It's frequently better architecture to decouple the meta-data declaration and implementation of a given tool from the MCP Server itself, and allow the MCP Server to dynamically add/remove tools at runtime, as tools can then be discovered, added, meta-data made available to the model(s), called, evaluated, and potentially removed or updated without the creation of an entirely new MCP Server, but rather dynamically discovering, securing, importing, using, evaluating, updating, and removing tools from an MCP Server.  

This approach is potentially more secure (as it allows tool-specific authentication and access control), more flexible, and more scalable, since remote tools can be distributed on multiple hosts over a network.  And it allows easy integration with existing APIs.

In the next post I describe a working example that uses remote tools.

Visual Studio Adding Support for MCP Servers

 

https://code.visualstudio.com/docs/copilot/chat/mcp-servers 

OSGi Services with gRPC - Let's be reactive

ECF has just introduced an upgrade to the grpc distribution provider.   Previously, this distribution provider used ReaxtiveX java version 2 only.  With this release, ReactiveX java version 3 is also supported.

As many know, gRPC allows services (both traditional call/response [aka unary] and streaming services) to be defined by a 'proto3' file.  For example, here is a simple service with four methods, one unary (check) and 3 streaming (server streaming, client streaming, and bi-directional streaming)

syntax = "proto3";

package grpc.health.v1;

option java_multiple_files = true;
option java_outer_classname = "HealthProto";
option java_package = "io.grpc.health.v1.rx3";

message HealthCheckRequest {
  string message = 1;
}

message HealthCheckResponse {
  enum ServingStatus {
    UNKNOWN = 0;
    SERVING = 1;
    NOT_SERVING = 2;
    SERVICE_UNKNOWN = 3;  // Used only by the Watch method.
  }
  ServingStatus status = 1;
}

service HealthCheck {
  // Unary method
  rpc Check(HealthCheckRequest) returns (HealthCheckResponse);
  // Server streaming method
  rpc WatchServer(HealthCheckRequest) returns (stream HealthCheckResponse);
  // Client streaming method
  rpc WatchClient(stream HealthCheckRequest) returns (HealthCheckResponse);
  // bidi streaming method
  rpc WatchBidi(stream HealthCheckRequest) returns (stream HealthCheckResponse);
}

The gRPC project provides a plugin so that when protoc is run, java code (or other language code) is generated that can then be used on the server and/or clients.

With some additional plugins, the classes generated by protoc can use the ReactiveX API for generating code.   So, for example, here is the java code generated by running protoc, grpc, reactive-grpc, and the osgi-generator plugins on the above HealthCheck service definition.  

Note in particular the HealthCheckService interface generated by the osgi-generator protoc plugin:

package io.grpc.health.v1.rx3;

import io.reactivex.rxjava3.core.Single;
import io.reactivex.rxjava3.core.Flowable;

@javax.annotation.Generated(
value = "by grpc-osgi-generator (REACTIVEX) - A protoc plugin for ECF's grpc remote services distribution provider at https://github.com/ECF/grpc-RemoteServiceSProvider ",
comments = "Source: health.proto.  ")
public interface HealthCheckService {
    /**
     * <pre>
     *  Unary method
     * </pre>
     */
    default Single<io.grpc.health.v1.rx3.HealthCheckResponse> check(Single<io.grpc.health.v1.rx3.HealthCheckRequest> requests)  {
        return null;
    }
    /**
     * <pre>
     *  Server streaming method
     * </pre>
     */
    default Flowable<io.grpc.health.v1.rx3.HealthCheckResponse> watchServer(Single<io.grpc.health.v1.rx3.HealthCheckRequest> requests)  {
        return null;
    }
    /**
     * <pre>
     *  Client streaming method
     * </pre>
     */
    default Single<io.grpc.health.v1.rx3.HealthCheckResponse> watchClient(Flowable<io.grpc.health.v1.rx3.HealthCheckRequest> requests)  {
        return null;

    }

    /**
     * <pre>
     *  bidi streaming method
     * </pre>
     */
    default Flowable<io.grpc.health.v1.rx3.HealthCheckResponse> watchBidi(Flowable<io.grpc.health.v1.rx3.HealthCheckRequest> requests)  {
        return null;
    }
}

Note that it uses the two ReactiveX 3 classes: io.reactivex.rxjava3.core.Single, and io.reactivex.rxjava3.core.Flowable. These two classes provide api for event-driven/reactive sending and receiving of unary (Single) and streaming (Flowable) arguments and return values.

The ReactiveX API...particularly Flowable...makes it very easy to implement both consumers and implementers of the streaming API, while maintaining ordered delivery and non-blocking communication.

For example, this is a simple implementation of the HealthCheckService. Note how the Single and flowable methods are able to express the implementation logic through methods such as Flowable.map.
Here is a simple implementation of a consumer of the HealthCheckService.

The use of the ReactiveX API simplifies both the implementation and the consumer use of both unary and streaming services. As an added bonus: the reactive-grpc library used in the ECF Distribution provider provides *flow-control* using backpressure.

In next article I'll describe how OSGi Remote Services can be easily used to export, publish, discover, and import remote services with full support for service versioning, security, and dynamics. I'll also describe one can use tools like maven or bndtools+eclipse to generate source code (as above) from a proto3 file and easily run a generated service as an OSGi Remote Service.

gRPC Remote Services Development with Bndtools - video tutorials

Here are four new videos that show how to define, implement and run/debug gRPC-based remote services using bndtools, eclipse, and ECF remote services.

Part 1 - API Generation - The generation of a OSGi remote service API using bndtools code generation and the protoc/gRPC compiler. The example service API has both unary and streaming gRPC method types supported by the reactivex API.

Part 2 - Implementation and Part 3 - Consumer - bndtools-project-template-based creation of remote service impl and consumer projects

Part 4 - Debugging - Eclipse/bndtools-based running/debugging of the remote service creating in parts 1-3.

gRPC and OSGi Remote Services

 gRPC is a popular framework for creating high-performance remote procedure call-based microservices.  

OSGi Remote Services is a transport-agnostic specification for creating dynamic, versionable, modular, remote services.

The ECF project provides an open implementation of the OSGi Remote Services spec, and has a provider implementation based-upon gRPC.   What this means is that gRPC can be used to create and run as an OSGi remote service, with all the support for service dynamics (particularly important for network-based services), versioning, and other features provided by OSGi remote services.

The architectural fit between gRPC and OSGi Remote Services is very good, since gRPC is concerned with transport-level efficiency (i.e. http/2, binary serialization format), and OSGi Remote Services are completely transport-agnostic, and focuses instead upon service-level concerns (e.g. dynamics, versioning, and service discovery).

gRPC offers support for server and client-based streaming.   In ECF's implementation, streaming rpcs are mapped to the reactivex api.  This means that consumers and implementers of a streaming rpc can simply call methods and provide callbacks (using Flowable), and non-blocking streaming calls will be made.  In addition, the use of reactivex and backpressure will result in transport-level flow control for these streaming APIs!

Another advantage of gRPC for OSGi remote services is it's polyglot nature.   This means that if (for example) a gRPC remote service is run as an OSGi/Java server, clients can be easily implemented in any of the languages supported by gRPC.  As well, servers written in some other language can easily created and accessed from OSGi consumers.  An example of this is the ECF etcd3 discovery provider, which communicates with an etcd server (written in Go) to publish and discover OSGi remote services.

Finally, with bndtools (an Eclipse plugin for OSGi bundle development), ECF Remote Service workspace template, and it's support for generating code as part of Eclipse's incremental build, gRPC code generation can be seemlessly integrated into the Eclipse development environment so that gRPC code generation, compile, and bundle packaging can happen immediately and continuously as part of gRPC remote service development.  For a video tutorial demonstrating this, please see here.

gRPC Code Generation using Bndtools and ECF Remote Services

There's a new video tutorial that demonstrates using ECF Remote Services, bndtools, and Eclipse to create an OSGi Remote Service.   

Bndtools has recently added the ability to run code generators as part of a bnd-based project, and with ECF's bndtools workspace template, a single proto3 file added to a project will automatically generate an entire Java remote service API and update/regenerate the API as changes are made to the proto3 file.   No command-line execution of protoc needed.

Further with ECF's project templates, the generated API can be easily implemented and exported as an OSGi Remote Service.

Please watch the video here

Using gRPC-java code generation to create OSGi Services

OSGi Services are usually first created by declaring a java service interface class.  As an OSGi service, this interface class serves as both the name for the service in the service registry, and defines the service contract (i.e. the interface method signatures...i.e. the method name, argument types, and return types) for that version of the service.

gRPC (Google RPC) is a popular and high-performance rpc approach that allows developers to define networked services based upon protocol buffers (proto3).

By extending bndtools recently-added code generation capability, it's now possible to generate an OSGi (remote) service API from just a proto3 service declaration.  All the classes necessary for an OSGi Remote Service API (service interface, arg and return types) can be generated by bndtools within Eclipse from a single proto3 file, immediately and completely.

Ready to implement-and-consume OSGi Services can be generated by Eclipse+bndtools + a proto3 service declaration.

Further, the proto3 service declaration can be modified, and the tooling will immediately generate new service API classes, compile, and package them into a bundle, all from within Eclipse+bndtools.

To get this bndtools-grpc generation with an example see here.

ECF 3.14.12 released - Now with gRPC for OSGi Remote Services

ECF 3.14.12 was just released

Highlights of this Release

New OSGi Remote Services Distribution provider based upon gRPC/Protocol Buffers. Along with a grpc-osgi-generator project...which allows the generation of a service API from a proto3 service declaration...this provider allows gRPC-based services to be exported and imported as OSGi Remote Services.  This now includes support for unary, server-streaming, and client-streaming gRPC calls.

Enhanced Support for Bndtools-based development of OSGi Remote Services.   The ECF Bndtools Workspace now includes the latest version of ECF Remote Services, along with the gRPC distribution provider, a Hazelcast-based discovery and distribution provider, and project and bndrun templates for creating, running, testing, and debugging OSGi Remote Services in Eclipse+Bndtools 5.

ECF 3.14.6 Released

ECF 3.14.6 has been released and may be downloaded here.

In concert with this bug fix release have been a number of additions to ECF's github projects for Remote Services Development.

Distribution and Discovery Providers

New:  Apache Dubbo-based Distribution Provider
Enhanced:  Hazelcast-based Distribution Provider and New: Discovery Provider
Enhanced:  System and Service-Properties docs for Distribution Providers and Discovery Providers

Bndtools Development

Enhanced:  Bndtools Workspace, Project, and Bndrun templates for Remote Services Development
New:  Bndtools Workspace with Jersey JaxRS Distribution Provider and Tasklist Example
Enhanced:  Tutorial for using Bndtools for Remote Services Development

 

RESTful OSGi R7 Remote Services with Jersey 2.28 or Apache CXF 3.3

For some time, ECF has had remote service distribution providers that use the Jersey or the CXF implementation of standard Java API for RESTful Web Services (JaxRS). 

These distribution providers allow OSGi R7 Remote Services to be defined via JaxRS annotations and implemented by either Jersey 2.28 or CXF 3.3. 

OSGi R7 Remote Services provides support for renite service discovery, dynamics, versioning, configuration and extension of the distribution providers, and asynchronous remote calls as well as other features of the OSGi R7 Remote Services and Remote Service Admin specs.

This tutorial shows the use of OSGi Remote Services with these JaxRS distribution providers on Apache Karaf.

There is also a new version of the ECF Bndtools workspace template with example Bndtools projects showing the use of these distribution providers to define, configure, run and deploy RESTful OSGi R7 Remote Services with Bndtools 4.2+.

ECF 3.14.3 released

ECF 3.14.3 has been released.  This is a bug fix release, but in addition to the fixes a number of usage examples have been created:

Remote Services Between Java and Python and Python for OSGi Remote Services

RESTful Remote Services Using either CXF or Jersey

Using OSGi R7 Async Remote Services

Using Gogo Commands and RSA for Controlling Remote Services

Using BndTools 4 for Developing OSGi Remote Services

ECF Photon supports Bndtools

A second major enhancement for ECF Photon is adding support for using Bndtools to develop and test OSGi Remote Services.   Bndtools is increasingly popular for developing OSGi-based applications and frameworks, and we've added support for Bndtools Workspace, Project, and Run Descriptor templates for developing and testing remote services.

Initial documentation is available at Bndtools Support for Remote Services Development.

Note that these templates and the RSA impl may change slightly before ECF Photon, and new/additional templates will be added to (e.g.) support other distribution and discovery providers.