Blog

  • Kafka Configuration with SpringBoot

    Posted by Dipen Chawla | Last Updated: 29-Sep-17

    Kafka is a distributed streaming platform.

     

     

    1. It keep running as a group on at least one cluster.

     

    2. The Kafka group stores surges of records in classes called points.

     

    3. Each record comprises of a key, an esteem, and a timestamp.

     

    4. Kafka is quick.

     

    It performs 2 million transactions per second.This makes it simple to exchange information from page reserve to arrange attachment. Kafka is high throughput frameworks. Kafka works extremely well as a swap for some more conventional message specialist like RabbitMQ, ActiveMQ and so forth.

     

     

    There are different benefits of Kafka:

     

    1.Reliable

    2.Scalable

    3.Durable

    4.Performance

     

     

    Installation steps:

     

    Step 1 — Install Java

    sudo apt-get update

    sudo apt-get install default-jre

     

     

    Step 2 — Install ZooKeeper
    sudo apt-get install zookeeperd

    telnet localhost 2181

     

     

    Step 3 — Download and Extract Kafka Binaries

    mkdir -p ~/Downloads

    wget "http://mirror.cc.columbia.edu/pub/software/apache/kafka/0.8.2.1/kafka_2.11-0.8.2.1.tgz" -O ~/Downloads/kafka.tgz

     

     

    Step 4 — Configure Kafka Server

    Open server.properties using vi:

    vi ~/kafka/config/server.properties

    delete.topic.enable = true

     

     

    Step 5 — Start the Kafka Server

    nohup ~/kafka/bin/kafka-server-start.sh ~/kafka/config/server.properties > ~/kafka/kafka.log 2>&1 &

    Configure Kafka with SpringBoot:

     

    Step1 : Maven Dependency

     

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
      <modelVersion>4.0.0</modelVersion>
    
      <groupId>com.codenotfound</groupId>
      <artifactId>spring-kafka-helloworld</artifactId>
      <version>0.0.1-SNAPSHOT</version>
    
      <name>spring-kafka-helloworld</name>
      <description>Spring Kafka - Consumer Producer Example</description>
      <url>https://www.codenotfound.com/spring-kafka-consumer-producer-example.html</url>
    
      <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.4.RELEASE</version>
      </parent>
    
      <properties>
        <java.version>1.8</java.version>
    
        <spring-kafka.version>1.2.2.RELEASE</spring-kafka.version>
      </properties>
    
      <dependencies>
        <!-- spring-boot -->
        <dependency>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-starter-test</artifactId>
          <scope>test</scope>
        </dependency>
        <!-- spring-kafka -->
        <dependency>
          <groupId>org.springframework.kafka</groupId>
          <artifactId>spring-kafka</artifactId>
          <version>${spring-kafka.version}</version>
        </dependency>
        <dependency>
          <groupId>org.springframework.kafka</groupId>
          <artifactId>spring-kafka-test</artifactId>
          <version>${spring-kafka.version}</version>
          <scope>test</scope>
        </dependency>
      </dependencies>
    
      <build>
        <plugins>
          <!-- spring-boot-maven-plugin -->
          <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
          </plugin>
        </plugins>
      </build>
    </project>

     

     

    Step 2: kafka Sender

     

    public class Sender {
    
      private static final Logger LOGGER = LoggerFactory.getLogger(Sender.class);
    
      @Autowired
      private KafkaTemplate<String, String> kafkaTemplate;
    
      public void send(String topic, String payload) {
        LOGGER.info("sending payload='{}' to topic='{}'", payload, topic);
        kafkaTemplate.send(topic, payload);
      }
    }

     

     

    Step 3: kafka consumer

     

    public class Receiver {
    
      private static final Logger LOGGER = LoggerFactory.getLogger(Receiver.class);
    
      private CountDownLatch latch = new CountDownLatch(1);
    
      public CountDownLatch getLatch() {
        return latch;
      }
    
      @KafkaListener(topics = "${kafka.topic.helloworld}")
      public void receive(String payload) {
        LOGGER.info("received payload='{}'", payload);
        latch.countDown();
      }
    }

     

Tags: