Requirements
- Access to a computer with Mac/Windows/Linux operating system installed. Hadoop setup instructions are provided in course for all Mac/Windows users.
- Internet connection (to watch course videos and download necessary tools for the course)
- Passion about Big Data technologies.
Description
Master Integrating ElasticSearch in Hadoop Ecosystem with hands on examples of building real world Data Pipelines using Apache Hive, PIG, MapReduce and LogStash.
This is the only course on Internet covering integration of ElasticSearch with Hadoop and creating various real world applications.
Use coupon code - DATASHARKACADEMY_ES for upto 95% discount
This comprehensive course focuses on building real world like data pipelines to move data from one system to another. A common practice for any data engineer. No other course can cover so much ground as we will do in this one.
In this course you will learn:
Section 1 – Ingestion Flows (Hadoop to ElasticSearch)
In this section of the course, you will learn to move data from various Hadoop applications (such as Hive, Pig, MR) & LogStash & load it into an index under ElasticSearch cluster. This is ideal use case for generating business analytics from your data. Here are four major topics that will be covered in this section:
- Learn how to install Apache Hive on your computer and integrate it with ElasticSearch
- Learn how to install Apache PIG on your computer and index data into ElasticSearch using Apache PIG.
- Learn how to load an index into ElasticSearch using Hadoop MapReduce (Java Program)
- Learn how to make LogStash work with ElasticSearch to move data into an index.
Section 2 – Egression Flows (ElasticSearch to Hadoop)
In this section of the course, you will learn to use indexed data from an ElasticSearch cluster and load it back into Hadoop cluster. After data is loaded back into Hadoop, you will learn how to directly import it into Hive, Pig, M/R or LogStash. Here are four major topics that we will cover under this section:
- Learn how to import an ElasticSearch index directly into Apache Hive table
- Learn how to import an ElasticSearch indexed data into Hadoop using Apache PIG scripts
- Learn how to import an ElasticSearch indexed data into Hadoop using Java MapReduce program
- Learn how to import an ElasticSearch indexed data using LogStash application
Section 3 – Data Visualization (Business Intelligence)
In part of the course, you will learn how to use indexed data from an ElasticSearch cluster and create dynamic dashboards using Kibana.
This will be a very important lesson for Data Analysts and Data Scientists.
Section 4 – Production Cluster Monitor tool (Administration)
No knowledge is complete without learning how to maintain an application in production. In this section of the course, you will learn how to monitor your ElasticSearch cluster using Marvel plugins. Here are few things that you will learn:
- Cluster Health monitoring at Index, Shard, Node levels
- Parsing ElasticSearch Cluster statistics using Linux utilities
- Setting up wait-for-trigger mechanism and much more
You will also learn about awesome search capabilities offered by ElasticSearch and how to query vast index of data in real time. This will be really fun!!!
We will cover lots of basics to build foundation required to understand ElasticSearch. You will also learn about behind the scenes on how a search engine and specifically ElasticSearch works in a single or multiple node cluster.
You will also get step by step instructions for installing all required tools and components on your machine in order to run all examples provided in this course. Each video will explain entire process in detail and easy to understand manner.
You will get access to working code for you to play with it and expand on it. All code examples are working and will be demonstrated in video lessons.
Windows users will need to install virtual machine on their device to setup single node hadoop cluster. Instructions are available inside the course.
Who is the target audience?
- This will be an excellent course for anyone who wants to learn about Big Data technologies and how to use them together in order to create amazing Big Data applications.
- Big Data Developers, Architects, Data Scientists, Data Analysts and Students (with zero experience)
