Data Engineering Foundations Part 2: Building Data Pipelines with Kafka and Nifi

Description

Data Engineering Foundations Part 2: Building Data Pipelines with Kafka and NiFi provides over four hours of video introducing you to creating data pipelines at scale with Kafka and NiFi. You learn to work with the Kafka message broker and discover how to establish NiFi dataflow. You also learn about data movement and storage. All software used in videos is open source and freely available for your use and experimentation on the included virtual machine.

About the Instructor

Doug Eadline, PhD, began his career as a practitioner and a chronicler of the Linux Cluster HPC revolution and now documents big data analytics. Starting with the first Beowulf How To document, Dr. Eadline has written hundreds of articles, white papers, and instructional documents covering virtually all aspects of HPC computing. Prior to starting and editing the popular ClusterMonkey.net website in 2005, he served as editor-in-chief for ClusterWorld Magazine and was Senior HPC Editor for Linux Magazine. Currently, he is a consultant to the HPC industry and writes a monthly column in HPC Admin Magazine. He has practical hands-on experience in many aspects of HPC, including hardware and software design, benchmarking, storage, GPU, cloud, and parallel computing. He is the co-author of the Apache Hadoop YARN book and author of Hadoop Fundamentals LiveLessons and Apache Hadoop YARN LiveLessons.

Skill Level:

• Beginner
• Intermediate

Learn How To:

• Understand Kafka topics, brokers, and partitions
• Implement basic Kafka usage modes
• Use Kafka producers and consumers with Python
• Utilize the KafkaEsque graphical user interface
• Understand the core concepts of NiFi
• Understand NiFi flow and web UI components
• Understand direct data movement with HDFS
• Use HBase with Python Happybase
• Use Sqoop for database movement

Who Should Take This Course:

• Users, developers, and administrators interested in learning the fundamental aspects and operations of date engineering and scalable systems

Course Requirements:

• Basic understanding of programming and development
• A working knowledge of Linux systems and tools
• Familiarity with Python

Released 12/2022

Download Links

Direct Download

Data Engineering Foundations Part 2: Building Data Pipelines with Kafka and Nifi.zip (967.2 MB) | Mirror