Big Data Development with Apache Spark E-Learning

Getting Started with Apache Spark

57 Minutes

Course Introduction
Overview of Apache Spark
Downloading and Installing Apache Spark
Downloading and Installing Apache Spark on Mac OS
Building Spark
Working with Spark Shell
Linking to Spark
Spark Configuration
Initializing Apache Spark
Running Spark on Clusters

Practice: Using Apache Spark Shell

5 Minutes

Exercise: Use Apache Spark Shell
Apache Spark SQL

Apache Spark SQL Introduction

1 Hour, 7 Minutes

Course Introduction
Apache Spark SQL Overview
SparkSession
DataFrames
Aggregations
SQL Queries
Temporary View
Datasets
JSON Datasets
Load/Save Functions
Specifying a Data Source
Querying with SQL
SaveMode
Parquet Files
Persistent Tables
Partitioning

Practice: Using Spark SQL

3 Minutes

Exercise: Use Spark SQL
Structured Streaming
Introduction to Spark Streaming
1 Hour, 8 Minutes

Course Introduction
Structured Streaming Overview
Stream Input
Stream Output
Windowing
Continuous Applications
Deduplication
File Sinks
Streaming Query
Streaming Query Manager
Checkpointing
Word Count

Practice: Spark Streaming Basics

5 Minutes

Exercise: Identify Spark Streaming Basics
Spark Monitoring and Tuning

Monitoring Spark Applications

17 Minutes

Course Introduction
Web UI
Environment Configuration
REST API
Memory Allocation

Tuning Spark Applications

38 Minutes

Speculation
Serialization
Memory Tuning
Executor Memory
Garbage Collection Tuning
Parallelism
Broadcast Functionality
Explain Query Execution
Data Compression

Practice: Monitoring Spark Applications

4 Minutes

Exercise: Monitor Spark Applications
Spark Security
Course Introduction
Spark UI
Secure Event Logs
SSL Settings
Shared Secret
YARN Deployments
SASL Encryption
Network Security
Exercise: Configure Spark Security