Data Engineering Essentials Hands-on SQL Python and Spark

Data Engineering Essentials Hands-on SQL Python and Spark Free Download. Learn key Data Engineering Skills such as SQL, Python and PySpark with tons of Hands-on tasks and exercises using labs.

Data Engineering Essentials Hands-on SQL Python and Spark Description.

Data Engineering is nothing but processing the data depending upon our downstream needs. We need to build different pipelines such as Batch Pipelines, Streaming Pipelines, etc as part of Data Engineering. All roles related to Data Processing are consolidated under Data Engineering. Conventionally, they are known as ETL Development, Data Warehouse Development, etc. As part of this course, you will learn all the Data Engineering Essentials related to building Data Pipelines using SQL, Python as well as Spark.

Requirements

Laptop with decent configuration (Minimum 4 GB RAM and Dual Core)
Sign up for GCP with the available credit or AWS Access
Setup self support lab on cloud platforms (you might have to pay the applicable cloud fee unless you have credit)
CS or IT degree or prior IT experience is highly desired

What you’ll learn

Setup Development Environment to learn building Data Engineering Applications on GCP
Database Essentials for Data Engineering using Postgres such as creating tables, indexes, running SQL Queries, using important pre-defined functions, etc.
Data Engineering Programming Essentials using Python such as basic programming constructs, collections, Pandas, Database Programming, etc.
Data Engineering using Spark Dataframe APIs (PySpark). Learn all important Spark Data Frame APIs such as select, filter, groupBy, orderBy, etc.
Data Engineering using Spark SQL (PySpark and Spark SQL). Learn how to write high quality Spark SQL queries using SELECT, WHERE, GROUP BY, ORDER BY, ETC.
Relevance of Spark Metastore and integration of Dataframes and Spark SQL
Ability to build Data Engineering Pipelines using Spark leveraging Python as Programming Language
Use of different file formats such as Parquet, JSON, CSV etc in building Data Engineering Pipelines
Setup self support single node Hadoop and Spark Cluster to get enough practice on HDFS and YARN
Understanding Complete Spark Application Development Life Cycle to build Spark Applications using Pyspark. Review the applications using Spark UI.