kedro grpc server

Kedro gRPC Server is a Kedro plugin that creates a gRPC server for triggering and monitoring pipeline runs using a general-purpose RPC framework gRPC

mmchougule
15
1
Python

Kedro gRPC Server

Coverage
Current Version
Python Version
Code style: black

This is a Kedro plugin that creates a gRPC server for your kedro pipelines. Exposed RPC calls can be triggered using any of the programming languages that support gRPC.

Demo:

gRPC server
Go Kedro Client

Kedro gRPC clients can be in many programming languages.

Some kedro client examples below that call RPCs on a gRPC server running in any kedro project:

More on grpc.io

  • Code generation support in all gRPC frameworks makes it super easy to use for clients who may have a different tech stack and language preference. Simply sharing a kedro.proto file, clients can use their preferred programming language to generate strongly typed kedro clients in any of the languages mentioned above.
  • Allowing engineers to natively integrate running of kedro pipelines with their preferred programming language.
  • Getting status of kedro pipeline run as a streaming response through HTTP/2. gRPC provides first-class support for this.
  • Getting all the benefits of gRPC
  • And, enabling business users to interact with analytics from a front-end application and trigger actions or models (e.g. scoring model) on demand.

How do I install Kedro gRPC Server?

Prerequisites

Kedro gRPC Server requires Python 3.6+ and Git to be setup.

Installation

You can install Kedro gRPC Server directly from GitHub with:

pip install kedro-grpc-server

How do I use Kedro gRPC Server?

To start the server, simply run the following command inside your Kedro project:

kedro server grpc-start

You can specify the host through the flag like so:

kedro server grpc-start --host

Similarly, you can set the port number using --port.

Run

gRPC API

Exposing 3 RPC calls:

ListPipelines -> Returns current list of pipelines

Run -> Runs a pipeline with or without arguments

Status -> Provides run status of a pipeline with run_id.
The response for this rpc call is a Server Streaming response of all logged events.

Contributing

Please read CONTRIBUTING.md for:

  • The contribution code of conduct
  • The process for submitting pull requests

Versioning

We use SemVer for versioning. The best way to safely upgrade is to check our release notes for any notable breaking changes.