distributed transactions kafka

Distributed Transactions in Microservices with Kafka Streams and Spring Boot

ashutoshsahoo
8
8
Java

Distributed Transactions in Microservices with Kafka Streams and Spring Boot

Description

There are three microservices:
order-service - it sends Order events to the Kafka topic and orchestrates the process of a distributed transaction
payment-service - it performs local transaction on the customer account basing on the Order price
stock-service - it performs local transaction on the store basing on number of products in the Order

Here’s the diagram with our architecture:

image

(1) order-service send a new Order -> status == NEW
(2) payment-service and stock-service receive Order and handle it by performing a local transaction on the data
(3) payment-service and stock-service send a reponse Order -> status == ACCEPT or status == REJECT
(4) order-service process incoming stream of orders from payment-service and stock-service, join them by Order id and sends Order with a new status -> status == CONFIRMATION or status == ROLLBACK or status == REJECTED
(5) payment-service and stock-service receive Order with a final status and “commit” or “rollback” a local transaction make before

Run application

  • Use docker to deploy kafka with registry.

Set docker host ip

export DOCKER_HOST_IP=127.0.0.1

or on windows powershell

$env:DOCKER_HOST_IP = "127.0.0.1"

docker compose -f zk-single-kafka-multiple-schema-registry-ui.yml up -d

Open UI for Apache Kafka at Kafka Control Center .

  • Run gradle build to generate avro source files
cd common-lib
gradle clean build

  • Run application

    • order-service
    cd order-service
gradle clean bootRun

    Open swagger-ui.

    • payment-service
    cd payment-service
gradle clean bootRun
    • stock-service
    cd stock-service
gradle clean bootRun

  • Use following request to generate orders

curl --location --request POST 'http://localhost:8080/orders/generate'
  • Use following request to check orders status
curl --location --request GET 'http://localhost:8080/orders'

Stop application

  • Use ctrl+c to cancel running applications
  • Use following command to stop and remove kafka containers
docker compose -f zk-single-kafka-multiple-schema-registry-ui.yml down

Kubernetes Deployment

  • Start kafka
helm repo add confluentinc https://confluentinc.github.io/cp-helm-charts/
helm repo update
helm install my-kafka -f ./k8s/kafka/values.yaml confluentinc/cp-helm-charts
  • Application Deployment
gradle clean dockerTag
kubectl apply -f ./k8s/zipkin
kubectl apply -f ./k8s/app
  • Use following request to generate orders
curl --location --request POST 'http://localhost:30001/orders/generate'
  • Use following request to check orders status
curl --location --request GET 'http://localhost:30001/orders'
  • Clean up
kubectl delete -f ./k8s/app
kubectl delete -f ./k8s/zipkin
helm delete my-kafka
  • Expose Confluent Control Center via NodePort
kubectl patch svc my-kafka-cp-control-center --type='json' -p '[{"op":"replace","path":"/spec/type","value":"NodePort"}]'

Reference