Project Description

Treinamento oficial Hortonworks onde somos a primeira empresa no Brasil a ministrar esse curso. Curso para desenvolvedores e analistas em sistemas voltados para Big Data. Serão abordados os módulos Pig e Hive. Material em inglês. Curso ministrado em português.

This course is designed for developers who need to create applications to analyze Big Data stored in Apache Hadoop using Pig and Hive. Topics include: Hadoop, YARN, HDFS, MapReduce, data ingestion, workflow definition, using Pig and Hive to perform data analytics on Big Data and an introduction to Spark Core and Spark SQL.

Público-Alvo

Somente treinamentos para empresas (In-Company). Desenvolvedores de software que precisam entender e desenvolver aplicativos para o Hadoop.

Requisitos

Somente treinamentos para empresas (In-Company). É importante conhecer o ambiente e o ecossistema Hadoop / Curso Apache Hadoop Essentials.

Conteúdo Programático

Este é o conteúdo que será abordado durante o curso. [inglês]

1. AN INTRODUCTION TO THE HADOOP DISTRIBUTED FILE SYSTEM

OBJECTIVES

  • Understanding Hadoop
  • O Sistema de Arquivos Distribuídos Hadoop
  • Ingesting Data into HDFS
  • The MapReduce Framework

LABS

  • Starting an HDP Cluster
  • Demonstration: Understanding Block Storage
  • Using HDFS Commands
  • Importing RDBMS Data into HDFS
  • Exporting HDFS Data to an RDBMS
  • Importing Log Data into HDFS Using Flume
  • Demonstration: Understanding MapReduce
  • Running a MapReduce Job

2. AN INTRODUCTION TO APACHE PIG

OBJECTIVES

  • Introduction to Apache Pig
  • Advanced Apache Pig Programming

LABS

  • Demonstration: Understanding Apache Pig
  • Getting Starting with Apache Pig
  • Exploring Data with Apache Pig
  • Splitting a Dataset
  • Joining Datasets with Apache Pig
  • Preparing Data for Apache Hive
  • Demonstration: Computing Page Rank
  • Analyzing Clickstream Data
  • Analyzing Stock Market Data Using Quantiles

3. AN INTRODUCTION TO APACHE HIVE

OBJECTIVES

  • Apache Hive Programming
  • Using HCatalog
  • Advanced Apache Hive Programming

LABS

  • Understanding Hive Tables
  • Understanding Partition and Skew
  • Analyzing Big Data with Apache Hive
  • Demonstration: Computing NGrams
  • Joining Datasets in Apache Hive
  • Computing NGrams of Emails in Avro Format
  • Using HCatalog with Apache Pig

4. WORKING WITH SPARK CORE, SPARK SQL AND OOZIE

OBJECTIVES

  • Advanced Apache Hive Programming (Continued)
  • Hadoop 2 and YARN
  • Introduction to Spark Core and Spark SQL
  • Defining Workflow with Oozie

LABS

  • Advanced Apache Hive Programming
  • Running a YARN Application
  • Getting Started with Apache Spark
  • Exploring Apache Spark SQL
  • Defining an Apache Oozie Workflow