Data Infrastructure with Apache Kafka E-Learning Training 

De Data Infrastructure with Apache Kafka LearningKit biedt een diepgaande kennismaking met Apache Kafka en de integratie met moderne data-infrastructuren en tools zoals Python, Spark, Cassandra en Confluent.

Onderwerpen:

• Introductie en Architectuur

Overzicht van de Kafka-architectuur voor event streaming
Het opzetten van topics en brokers
Berichtafhandeling en configuratie van brokers

• Werken met Kafka

Produceren en consumeren van berichten
Werken met consumentengroepen
Integratie van Kafka met Python

• Integraties en Geavanceerde Toepassingen

Integratie met Apache Spark voor gestructureerde streaming
Bouwen van Spark-applicaties met DataFrames voor verwerking van Kafka-data
Integratie van Kafka met Cassandra voor NoSQL-data
Gebruik van Confluent om Kafka-implementaties te optimaliseren

• Prestatieoptimalisatie

Tuning en optimalisatie van Kafka-prestaties
Best practices voor schaalbaarheid en betrouwbaarheid

Deze training geeft deelnemers praktische kennis om Apache Kafka effectief te gebruiken voor event streaming en data-integratie in complexe infrastructuren.

Deze LearningKit met meer dan 7 leeruren is verdeeld in drie sporen:

Demo Data Infrastructure with Apache Kafka Training

Cursusinhoud

Track 1: Intro to Data Infrastructure

In this track, the focus will be on data infrastructure in an organization, data mesh architecture, data tools, messaging platforms, and data stores.
Courses (½ hour +):

Setting up the Data Infrastructure in an Organization

Course: 46 Minutes

• Course Overview
• Data Infrastructure in an Organization
• Data Mesh Architecture
• Tools for Data Management
• Messaging Platforms
• Data Stores
• Course Summary

Track 2: Apache Kafka

In this track, the focus will be on Apache Kafka and Apache Spark. Apache Kafka is a popular event streaming platform used by Fortune 100 companies for both real-time and batch data processing. Apache Spark is a powerful distributed data processing engine that can handle petabytes of data by chunking that data and dividing across a cluster of resources.
Courses (7 hours +)

Processing Data: Getting Started with Apache Kafka

Course: 1 Hour, 32 Minutes

• Course Overview
• Work with Real-time Data
• Stream Events with Kafka
• Kafka Topics
• Downloading and Installing Kafka
• Creating a Topic, Producer, and Consumer with Kafka
• Working with Multiple Kafka Topics
• Configuring a Multi-node Kafka Cluster
• Monitoring a Kafka Cluster
• Using Partitions and Replicas with Kafka
• Course Summary

Processing Data: Integrating Kafka with Python & Using Consumer Groups

Course: 1 Hour, 24 Minutes

• Course Overview
• Developing Kafka Producers and Consumers in Python
• Processing Messages at the Consumer
• Tweaking Kafka Broker Configurations
• Defining Automatic Topic Creation in Kafka
• Generating Fake Data for Kafka Consumption
• Setting a Destination Partition in Kafka
• Kafka Consumer Groups
• Creating and Using Consumer Groups in Kafka
• Working with Consumer Groups and Partitions in Kafka
• Kafka Configuration
• Course Summary

Processing Data: Introducing Apache Spark

Course: 1 Hour, 44 Minutes

• Course Overview
• Apache Spark
• Apache Spark Architecture
• Structured Streaming in Apache Spark
• Downloading and Installing Spark
• Deploying a Spark Cluster
• Launching a Spark Job
• Monitoring Spark Apps with the Web UI
• Configuring a Spark Cluster
• Building a Spark Streaming App
• Running Apps on a Standalone Cluster
• Running Apps on Spark Local
• Course Summary

Processing Data: Integrating Kafka with Apache Spark

Course: 1 Hour, 46 Minutes

• Course Overview
• Integrating Spark with Kafka
• Transforming Kafka Messages with PySpark
• Reading from Multiple Kafka Topics
• Setting up a Producer and Consumer with Kafka
• Publishing to Kafka from PySpark
• Transforming Data with Spark SQL
• Aggregations on Streaming Data
• Exploring Grouping and Ordering
• Defining Window Operations
• Creating Tumbling and Sliding Windows
• Course Summary

Processing Data: Using Kafka with Cassandra & Confluent

Course: 42 Minutes

• Course Overview
• Installing and Setting up Apache Cassandra
• Integrating Spark with Kafka and Cassandra
• Confluent and Kafka
• Setting up the Confluent Platform
• Working with Kafka Using Confluent
• Course Summary

Assessment:

• Final Exam: Apache Kafka