How to integrate Elastic Search with MongoDB
Posted By : Yash Vishwakarma | 05-Jan-2021
Introduction of MongoDB:-
MongoDB is a NoSQL database that stores the data in form of key-value pair. You can achieve high performance and scalability in addition to data modeling and data management of vast sets of data by using it as an Open-source document database in an enterprise application.
Introduction of Elastic search:-
Elasticsearch is a search engine based on Lucene's library. You get a distributed multitenant capable full-text search engine with an HTTP web interface & schema-free JSON document with Elasticsearch. Also, it is developed in Java.
Configuring MongoDB and Installing MongoDB:-
steps to install MongoDB-
# wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -
# sudo apt-get install gnupg
# wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -
# /etc/apt/sources.list.d/mongodb-org-4.4.list
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb- org-4.4.list
# sudo apt-get install -y mongodb-org
# sudo systemctl start mongod
NOTE:-
Unless you want to export the data & then import them into Elastic Search, the only way to connects MongoDB and ES & using the replication system MongoDB.
By configuring MongoDB as a replica set we enable the automatic transfers of data between the two database systems.
To do this is very simple basically just open the configurations file mongo “mongod.conf” & set the replica set by the line
replset = RS0
then restarts mongo and enable replication using the command
rs.initiate ()
If all went well your cursor will change in mongo:-
RS0: PRIMARY>
How To Install Elastic Search:-
Before proceeding has to do much attention cone versions of the various components. We have avviente via a plugin {MongoDB River} which is responsible for transfers of data during the integration of MongoDB and Elastic Search.
If that doesn't work you will find your index in elastic search empty.
To make sure you have to the correct version used in this table :-
River ES MongoDB
master — 1.0.0 — 2.4.9
2.0.0 — 1.0.0 — 2.4.9
1.7.4 — 0.90.10 — 2.4.8
1.7.3 — 0.90.7 — 2.4.8
1.7.2 — 0.90.5 — 2.4.8
1.7.1 — 0.90.5 — 2.4.6
1.7.0 — 0.90.3 — 2.4.5
1.6.11 — 0.90.2 — 2.4.5
1.6.9 — 0.90.1 — 2.4.4
1.6.8 — 0.90.0 — 2.4.3
1.6.7 — 0.90.0 — 2.4.3
1.6.6 — 0.90.0 — 2.4.3
To install MongoDB 2.4.8, I used the following versions:
-> 0.90.10 ElasticSearch
-> Plugin MongoDB
-> Plugin 1.7.4 River: MapperAttachments 1.90
-> Plugin: Elasticsearch-head
Also Read: The Advantages of MongoDB Over Relational Database
Installation and Elastic Search plugin:
# sudo apt search openjdk
# sudo apt update
# sudo apt install apt-transport-https openjdk-8-jre-headless
# wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
# echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-6.x.list
# sudo apt-get update && sudo apt-get install elasticsearch
# sudo /bin/systemctl daemon-reload
# sudo /bin/systemctl enable elasticsearch.service
# sudo systemctl start elasticsearch.service
# sudo ls -la /var/log/elasticsearch/
# sudo cat /var/log/elasticsearch/elasticsearch.log
# curl -XGET 'localhost:9200/?pretty'
{
"name" : "c29HHBX",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "LweifnvTSKeP12nRNu0DAg",
"version" : {
"number" : "6.1.10",
"build_hash" : "b727a60",
"build_date" : "2018-06-06T15:48:34.860Z",
"build_snapshot" : false,
"lucene_version" : "6.6.1"
},
"tagline": "You Know, for Search"
}
Somewhere you have to create a file named install-elasticsearch.sh and then proceed as follows:
1. chmod +x install-elasticsearch.sh
2. ./install-elasticsearch.sh
We have to install the necessary plugins, position the folder where you installed Elastic Search.
-> bin / plugin -install elasticsearch / elasticsearch-mapper-attachments / 1.9.0
-> bin / plugin —install com.github.richardwilly98.elasticsearch / elasticsearch-river-mongodb / 1.7.4
-> bin / plugin -install mobz / elasticsearch-head
-> elasticsearch es.config f D = / usr / local / opt / elasticsearch / config / elasticsearch.yml
-> localhost: 9200
-> http: // localhost: 9200 / _plugin / head
Also Read: An Introduction To Studio 3T Tool
Configuring Indexes River:-
In this point, we need to configure Active ES for you to connect with MongoDB & the organization of data for future queries.
This Will be done through the creation of the index River.
We eliminate that what we have been created by default by the plugin,
open a terminal window & run:
# -XDELETE curl localhost: 9200 / _river
curl -XPUT “localhost: 9200 / _river / index_name / _meta ‘d’
{
“Type”: “mongodb”
“Mongodb”: {
“Db”: “NOME_DEL_VOSTRO_DB”
“Collection”: “NOME_DELLA_VOSTRA_COLLECTION”
},
“Index”: {
“Name”: “index_name”
“Type”: “type_name”
}
}
If everything works fine you will get this message:-
{“Ok”: true, “_ index”: “_ river”, “_ type”: “type_name”, “_ id”: “meta _”, “_ version”: 1}
Why Choose Oodles For Software Development?
We are a 360-degree software development company that provides complete web and mobile app development solutions for varied project requirements. Our end-to-end SaaS app development services address your mission-critical project requirements through scalable, responsive, and feature-rich software applications that are easy to scale. We carefully analyze your project requirements and formulate effective strategies to build enterprise-grade web and mobile applications for multiple platforms. For more info, contact us at [email protected].
Cookies are important to the proper functioning of a site. To improve your experience, we use cookies to remember log-in details and provide secure log-in, collect statistics to optimize site functionality, and deliver content tailored to your interests. Click Agree and Proceed to accept cookies and go directly to the site or click on View Cookie Settings to see detailed descriptions of the types of cookies and choose whether to accept certain cookies while on the site.
About Author
Yash Vishwakarma
He is DevOps engineer and enthusiastic to work in Linux, Cloud technology like AWS (Amazon web services).AWS, docker, gitlab in DevOps envirnment and willing to learn new technologies and like to play table tennis and snooker.