How to integrate Elastic Search with MongoDB

Posted By : Yash Vishwakarma | 05-Jan-2021

mongodb to elasticsearch

 

Introduction of MongoDB:-

 

MongoDB is a NoSQL database that stores the data in form of key-value pair. You can achieve high performance and scalability in addition to data modeling and data management of vast sets of data by using it as an Open-source document database in an enterprise application.


Introduction of Elastic search:-

 

Elasticsearch is a search engine based on Lucene's library. You get a distributed multitenant capable full-text search engine with an HTTP web interface & schema-free JSON document with Elasticsearch. Also, it is developed in Java.


Configuring MongoDB and Installing MongoDB:-


steps to install MongoDB-

 

# wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -

# sudo apt-get install gnupg

# wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -

# /etc/apt/sources.list.d/mongodb-org-4.4.list
  
  echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb- org-4.4.list

# sudo apt-get install -y mongodb-org
                           
# sudo systemctl start mongod


 NOTE:-

 

Unless you want to export the data & then import them into Elastic Search, the only way to connects MongoDB and ES & using the replication system  MongoDB.

By configuring MongoDB as a replica set we enable the automatic transfers of data between the two database systems.
To do this is very simple basically just open the configurations file mongo “mongod.conf” & set the replica set by the line
replset = RS0
then restarts mongo and enable replication using the command
rs.initiate ()

If all went well your cursor will change in mongo:-

RS0: PRIMARY>

 

How To Install Elastic Search:-

 

Before proceeding has to do much attention cone versions of the various components. We have avviente via a plugin {MongoDB River} which is responsible for transfers of data during the integration of MongoDB and Elastic Search.
 
If that doesn't work you will find your index in elastic search empty.
To make sure you have to the correct version used in this table :-

River ES MongoDB
master — 1.0.0 — 2.4.9
2.0.0 — 1.0.0 — 2.4.9
1.7.4 — 0.90.10 — 2.4.8
1.7.3 — 0.90.7 — 2.4.8
1.7.2 — 0.90.5 — 2.4.8
1.7.1 — 0.90.5 — 2.4.6
1.7.0 — 0.90.3 — 2.4.5
1.6.11 — 0.90.2 — 2.4.5
1.6.9 — 0.90.1 — 2.4.4
1.6.8 — 0.90.0 — 2.4.3
1.6.7 — 0.90.0 — 2.4.3
1.6.6 — 0.90.0 — 2.4.3

 

To install MongoDB 2.4.8, I used the following versions:

 

 -> 0.90.10 ElasticSearch

 -> Plugin MongoDB

 -> Plugin 1.7.4 River: MapperAttachments 1.90

 -> Plugin: Elasticsearch-head

 

Also Read: The Advantages of MongoDB Over Relational Database

 

Installation and Elastic Search plugin:

 

# sudo apt search openjdk

# sudo apt update

# sudo apt install apt-transport-https openjdk-8-jre-headless

# wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

# echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-6.x.list

# sudo apt-get update && sudo apt-get install elasticsearch

# sudo /bin/systemctl daemon-reload

# sudo /bin/systemctl enable elasticsearch.service

# sudo systemctl start elasticsearch.service

# sudo ls -la /var/log/elasticsearch/

# sudo cat /var/log/elasticsearch/elasticsearch.log

# curl -XGET 'localhost:9200/?pretty'


  {
  "name" : "c29HHBX",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "LweifnvTSKeP12nRNu0DAg",
  "version" : {
    "number" : "6.1.10",
    "build_hash" : "b727a60",
    "build_date" : "2018-06-06T15:48:34.860Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.1"
  },
  "tagline": "You Know, for Search"
}

Somewhere you have to create a file named install-elasticsearch.sh and then proceed as follows:

1. chmod +x install-elasticsearch.sh
2. ./install-elasticsearch.sh

We have to install the necessary plugins, position the folder where you installed Elastic Search.

 -> bin / plugin -install elasticsearch / elasticsearch-mapper-attachments / 1.9.0
 -> bin / plugin —install com.github.richardwilly98.elasticsearch / elasticsearch-river-mongodb / 1.7.4
 -> bin / plugin -install mobz / elasticsearch-head
 -> elasticsearch es.config f D = / usr / local / opt / elasticsearch / config / elasticsearch.yml
 -> localhost: 9200
 -> http: // localhost: 9200 / _plugin / head

 

Also Read: An Introduction To Studio 3T Tool

 

Configuring Indexes River:-

 

In this point, we need to configure Active ES for you to connect with MongoDB & the organization of data for future queries.
This Will be done through the creation of the index River.
We eliminate that what we have been created by default by the plugin,
open a terminal window & run:

# -XDELETE curl localhost: 9200 / _river

curl -XPUT “localhost: 9200 / _river / index_name / _meta ‘d’
{
 “Type”: “mongodb”
 “Mongodb”: {
 “Db”: “NOME_DEL_VOSTRO_DB”
 “Collection”: “NOME_DELLA_VOSTRA_COLLECTION”
 },
 “Index”: {
 “Name”: “index_name”
 “Type”: “type_name”
 }

 

If everything works fine you will get this message:-

 

{“Ok”: true, “_ index”: “_ river”, “_ type”: “type_name”, “_ id”: “meta _”, “_ version”: 1}

 

 

Why Choose Oodles For Software Development?

 

We are a 360-degree software development company that provides complete web and mobile app development solutions for varied project requirements. Our end-to-end SaaS app development services address your mission-critical project requirements through scalable, responsive, and feature-rich software applications that are easy to scale. We carefully analyze your project requirements and formulate effective strategies to build enterprise-grade web and mobile applications for multiple platforms. For more info, contact us at [email protected]

About Author

Author Image
Yash Vishwakarma

He is DevOps engineer and enthusiastic to work in Linux, Cloud technology like AWS (Amazon web services).AWS, docker, gitlab in DevOps envirnment and willing to learn new technologies and like to play table tennis and snooker.

Request for Proposal

Name is required

Comment is required

Sending message..