🚀 DevOps & SRE Certification Program 📅 Starting: 1st of Every Month 🤝 +91 8409492687 🔍 Contact@DevOpsSchool.com

Upgrade & Secure Your Future with DevOps, SRE, DevSecOps, MLOps!

We spend hours on Instagram and YouTube and waste money on coffee and fast food, but won’t spend 30 minutes a day learning skills to boost our careers.
Master in DevOps, SRE, DevSecOps & MLOps!

Learn from Guru Rajesh Kumar and double your salary in just one year.


Get Started Now!

Uncategorised

Elasticsearch Lab#1 – CRUD

September 24, 2021 comments off
=======================================================================================
Elasticsearch VERIFICATION
=======================================================================================
$ curl http://localhost:9200
{
"name" : "0qn0pX_",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "rLRval0BQu6haI7JKw-tIQ",
"version" : {
"number" : "6.4.0",
"build_flavor" : "oss",
"build_type" : "zip",
"build_hash" : "595516e",
"build_date" : "2018-08-17T23:18:47.308994Z",
"build_snapshot" : false,
"lucene_version" : "7.4.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
=======================================================================================
To check the health of Elasticsearch
=======================================================================================
The cat API - The cat API print the info in JSON format, compact and aligned text which is easy to read by Human eyes.
All the cat commands accept a query string parameter "help" to see all the headers and info they provide, and the /_cat command alone lists all the available commands.
$ curl -X GET "localhost:9200/_cat"
=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates
/_cat/ml/anomaly_detectors
/_cat/ml/anomaly_detectors/{job_id}
/_cat/ml/trained_models
/_cat/ml/trained_models/{model_id}
/_cat/ml/datafeeds
/_cat/ml/datafeeds/{datafeed_id}
/_cat/ml/data_frame/analytics
/_cat/ml/data_frame/analytics/{id}
/_cat/transforms
/_cat/transforms/{transform_id}
=======================================================================================
Verbose
=======================================================================================
Each of the commands accepts a query string parameter v to turn on verbose output.\
$ curl -X GET "localhost:9200/_cat/master?v"
=======================================================================================
Help
=======================================================================================
Each of the commands accepts a query string parameter help which will output its available columns.
$ curl -X GET "localhost:9200/_cat/master?help"
=======================================================================================
Headers
=======================================================================================
Each of the commands accepts a query string parameter h which forces only those columns to appear.
$ curl -X GET 'localhost:9200/_cat/nodes?h=ip,port,heapPercent,name'
=======================================================================================
Few CAT Commands Output
=======================================================================================
# Check node’s health status:
curl -XGET 'localhost:9200/_cluster/health?pretty'
curl -XGET 'localhost:9200/_cat/health?v'
curl localhost:9200/_cat/health?help
curl 'localhost:9200/_cat/health?format=json&pretty'
curl -XGET 'localhost:9200/_cat/nodes?pretty'
curl -XGET 'localhost:9200/_cat/nodes?pretty'
curl localhost:9200/_cat/nodes?v
curl localhost:9200/_cat/nodes?help
curl 'localhost:9200/_cat/nodes?format=json&pretty'
curl -XGET 'localhost:9200/_cat/shards?pretty'
curl -XGET 'localhost:9200/_cat/shards?pretty'
curl localhost:9200/_cat/shards?v
curl localhost:9200/_cat/shards?help
curl 'localhost:9200/_cat/shards?format=json&pretty'
curl -XGET 'localhost:9200/_cat/master?pretty'
curl localhost:9200/_cat/master?v
curl localhost:9200/_cat/master?help
curl 'localhost:9200/_cat/master?format=json&pretty'
#List all indices
curl -XGET 'localhost:9200/_cat/indices?pretty'
curl localhost:9200/_cat/indices?v
curl localhost:9200/_cat/indices?help
curl 'localhost:9200/_cat/indices?format=json&pretty'
=======================================================================================
Understanding Elasticsearch Keywords and Terminology
=======================================================================================
https://www.devopsschool.com/blog/understanding-elasticsearch-keywords-and-terminology/
https://www.devopsschool.com/blog/understanding-elasticsearch-index-mapping-schema/
=======================================================================================
Elasticsearch Workflow - CRUD Operations
=======================================================================================
https://www.devopsschool.com/blog/understanding-rest-http-method-get-post-put-head-delete/
Create
Read
Update
Delete
FORMAT
curl -X<VERB> '<PROTOCOL>://<HOST>:<PORT>/<PATH>?<QUERY_STRING>' -d '<BODY>'
VERB - GET, POST, PUT, HEAD, or DELETE.
GET (Retrieve information)
POST (Create + Update)
PUT (Create + Update)
DELETE (Remove)
PROTOCOL - http or https
=======================================================================================
CURL TO HTTP
=======================================================================================
Status codes indicate the result of the HTTP request.
1XX – informational
2XX – success
3XX – redirection
4XX – client error
5XX – server error
=======================================================================================
Demo 1 - curl and listing all indices
curl -XGET 'localhost:9200/_cat/indices?v&pretty'
# To create products index
curl -XPUT 'localhost:9200/products?&pretty'
# To create products-temp index
curl -XPUT 'localhost:9200/products-temp?&pretty'
curl -XGET 'localhost:9200/_cat/indices?v&pretty'
# To create customers index
curl -XPUT 'localhost:9200/customers?&pretty'
# To create orders index
curl -XPUT 'localhost:9200/orders?&pretty'
curl -XGET 'localhost:9200/_cat/indices?v&pretty'
=======================================================================================
Index and Query a document
=======================================================================================
products -> index
mobiles - Category OR TYPE
1 - index UUID
-d - Content of the document in JSON BODY
curl -XPUT 'localhost:9200/products/mobiles/1?pretty' -H 'Content-Type: application/json' -d'
{
"name": "iPhone 7",
"camera": "12MP",
"storage": "256GB",
"display": "4.7inch",
"battery": "1,960mAh",
"reviews": ["Incredibly happy after having used it for one week", "Best iPhone so far", "Very expensive, stick to Android"]
}
'
---OUTPUT------
{
"_index" : "products",
"_type" : "mobiles",
"_id" : "1",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
{
"name": "iPhone 7",
"camera": "12MP",
"storage": "256GB",
"display": "4.7inch",
"battery": "1,960mAh",
"reviews": ["Incredibly happy after having used it for one week", "Best iPhone so far", "Very expensive, stick to Android"]
}
'
---OUTPUT------
curl -XPUT 'localhost:9200/products/mobiles/2?pretty' -H 'Content-Type: application/json' -d'
{
"name": "Samsung Galaxy",
"camera": "8MP",
"storage": "128GB",
"display": "5.2inch",
"battery": "1,500mAh",
"reviews": ["Best Android phone", "I love it!"]
}
'
curl -XPUT 'localhost:9200/products/mobiles/3?pretty' -H 'Content-Type: application/json' -d'
{
"name": "Xiaomi Note",
"camera": "10MP",
"storage": "128GB",
"display": "5.5inch",
"battery": "1,500mAh",
"reviews": ["Really love Xiaomi products", "Too large to use easily"]
}
'
ERROR
--------
Multiple mapping types are not supported in indices created in 6.0
------------------------------
curl -XPUT 'localhost:9200/products/laptops/1?pretty' -H 'Content-Type: application/json' -d'
{
"name": "Macbook Pro",
"storage": "500GB",
"RAM": "8GB",
"display": "13inch",
"os": "El Capitan",
"reviews": ["A little bulky but a great Mac laptop", "The larger storage capacity is great"]
}
'
ERROR
--------
Multiple mapping types are not supported in indices created in 6.0
------------------------------
curl -XPUT 'localhost:9200/products-temp/laptops/1?pretty' -H 'Content-Type: application/json' -d'
{
"name": "Macbook Pro",
"storage": "500GB",
"RAM": "8GB",
"display": "13inch",
"os": "El Capitan",
"reviews": ["A little bulky but a great Mac laptop", "The larger storage capacity is great"]
}
'
curl -XPUT 'localhost:9200/products-temp/laptops/2?pretty' -H 'Content-Type: application/json' -d'
{
"name": "Dell",
"storage": "1TB",
"RAM": "8GB",
"display": "14inch",
"os": "Windows 10",
"reviews": ["Great work laptop", "Not too heavy"]
}
'
curl -XPOST 'localhost:9200/products/mobiles?pretty' -H 'Content-Type: application/json' -d'
{
"name": "iPhone 6s",
"camera": "12MP",
"storage": "128GB",
"display": "4.7inch",
"battery": "1,960mAh",
"reviews": ["Incredibly happy after having used it for one week", "Best iPhone so far", "Very expensive, stick to Android"]
}
'
curl -XPOST 'localhost:9200/products/mobiles/4?pretty' -H 'Content-Type: application/json' -d'
{
"name": "iPhone 6s",
"camera": "12MP",
"storage": "128GB",
"display": "4.7inch",
"battery": "1,960mAh",
"reviews": ["Incredibly happy after having used it for one week", "Best iPhone so far", "Very expensive, stick to Android"]
}
'
========================
PUT VS POST DEMO PENDING
================================
POST - CREATE + UPDATE
PUT - CREATE + UPDATE (IDEOMPOTENT)
The only difference - POST can be used to achieve auto-generation of ids whereas a PUT is used when you want to specify an id.
It's all about REST semantics. POST basically that you are posting a request which is going to modify the server state. POST index/type { "foo": "bar" } will generate an _id server side and will index the document with this _id. PUT is used to send a resource to the server. PUT index/type/id { "foo": "bar" } will put or update a document named index/type/id in the server.
Example are given here.
https://www.devopsschool.com/blog/understanding-rest-http-method-get-post-put-head-delete/
=======================================================================================
DO YOU WANT TO UNDERSTAND THE INDEX MAPPING
=======================================================================================
https://www.devopsschool.com/blog/understanding-elasticsearch-index-mapping-schema/
=======================================================================================
Fetching whole documents
=======================================================================================
curl -XGET 'localhost:9200/_cat/indices?pretty'
curl -XGET 'localhost:9200/products/mobiles/1'
curl -XGET 'localhost:9200/products/mobiles/1?pretty'
curl -XGET 'localhost:9200/products/mobiles/2?pretty'
curl -XGET 'localhost:9200/products/mobiles/3?pretty'
curl -XGET 'localhost:9200/products/mobiles/4?pretty'
curl -XGET 'localhost:9200/products-temp/laptops/1?pretty'
curl -XGET 'localhost:9200/products-temp/laptops/10?pretty'
How to count number of documents in index
count provides quick access to the document count of the entire cluster, or individual indices.
$ curl localhost:9200/_cat/count?v
Or for a single index:
-------------------------
$ curl 'localhost:9200/_cat/count/products?v'
----------------------------
=======================================================================================
Fetching partial documents
=======================================================================================
curl -XGET 'localhost:9200/products/mobiles/1?pretty'
curl -XGET 'localhost:9200/products/mobiles/1?pretty&_source=false'
curl -XGET 'localhost:9200/products/mobiles/1?pretty&_source=name,reviews'
curl -XGET 'localhost:9200/products/mobiles/1?pretty&_source=name,reviews,storage'
curl -XGET 'localhost:9200/products-temp/laptops/1?pretty&_source=name,RAM,storage'
=======================================================================================
Updating whole and partial documents
=======================================================================================
curl -XGET 'localhost:9200/products/mobiles/3?pretty'
curl -XPUT 'localhost:9200/products/mobiles/13?pretty' -H 'Content-Type: application/json' -d'
{
"name": "Xiaomi Note",
"camera": "12MP",
"storage": "256GB",
"display": "5.5inch",
"battery": "1,800mAh",
"reviews": ["Really love Xiaomi products", "Too large to use easily"]
}
'
=======================================================================================
Partial Updates using the _update API with "doc"
=======================================================================================
https://www.devopsschool.com/blog/understanding-the-update-api-in-elasticsearch/
curl -XGET 'localhost:9200/products/mobiles/2?pretty'
curl -XPOST 'localhost:9200/products/mobiles/2/_update?pretty' -H 'Content-Type: application/json' -d'
{
"doc": {
"color": "black"
}
}
'
curl -XGET 'localhost:9200/products/mobiles/2?pretty'
curl -XPOST 'localhost:9200/products/mobiles/2/_update?pretty' -H 'Content-Type: application/json' -d'
{
"doc": {
"reviews": ["Best Android phone", "I love it!", "Samsung is my favorite"],
"texture": "smooth"
}
}
'
=======================================================================================
Updates using the _update API with "script"
=======================================================================================
curl -XGET 'localhost:9200/products'
curl -XGET 'localhost:9200/products?pretty'
curl -XGET 'localhost:9200/products/_search?pretty=true&q=*:*'
curl -XPUT 'localhost:9200/products-shoe?&pretty'
curl -XPUT 'localhost:9200/products-shoe/shoes/1?pretty' -H 'Content-Type: application/json' -d'
{
"name": "Nike",
"size": 8,
"color": "white"
}
'
curl -XGET 'localhost:9200/products-shoe/shoes/1?pretty'
curl -XPOST 'localhost:9200/products-shoe/shoes/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
"script": "ctx._source.size += 4"
}
'
curl -XGET 'localhost:9200/products-shoe/shoes/1?pretty'
curl -XGET 'localhost:9200/products-shoe/shoes/2?pretty'
curl -XPOST 'localhost:9200/products-shoe/shoes/2/_update?pretty' -H 'Content-Type: application/json' -d'
{
"script": "ctx._source.size -= 4"
}
'
curl -XGET 'localhost:9200/products-shoe/shoes/2?pretty'
=======================================================================================
Deleting individual Documents
=======================================================================================
curl -XGET 'localhost:9200/products/mobiles/2?pretty'
curl -XDELETE 'localhost:9200/products/mobiles/2?pretty'
curl -XGET 'localhost:9200/products/mobiles/2?pretty'
-------------------
curl -i -XHEAD 'localhost:9200/products/mobiles/2?pretty'
curl -i -XHEAD 'localhost:9200/products/mobiles/1?pretty'
--------------------
=======================================================================================
Deleting an index
=======================================================================================
curl -XGET 'localhost:9200/_cat/indices?v&pretty'
curl -XDELETE 'localhost:9200/orders?pretty'
curl -XDELETE 'localhost:9200/products?pretty'
curl -XDELETE 'localhost:9200/products-shoe?pretty'
curl -XDELETE 'localhost:9200/products-temp?pretty'
curl -XDELETE 'localhost:9200/customers?pretty'
curl -XGET 'localhost:9200/_cat/indices?v&pretty'
view raw Elasticsearch-Lab1-CRUD.txt hosted with ❤ by GitHub

Reference

  • Sample Data & Query – https://github.com/devopsschool-demo-labs-projects/elasticsearch

Certification Courses

DevOpsSchool has introduced a series of professional certification courses designed to enhance your skills and expertise in cutting-edge technologies and methodologies. Whether you are aiming to excel in development, security, or operations, these certifications provide a comprehensive learning experience. Explore the following programs:

DevOps Certification, SRE Certification, and DevSecOps Certification by DevOpsSchool

Explore our DevOps Certification, SRE Certification, and DevSecOps Certification programs at DevOpsSchool. Gain the expertise needed to excel in your career with hands-on training and globally recognized certifications.