arrow_back
Introduction
Welcome to the Course Video
The Fundamentals
Data VS Information
Data Storage and Processing
Data Sources
Big Data Introduction
Fundamentals Assessment
The Foundations of Big Data
2.1 Emergence of Big Data
Emergence of Big Data
Basic Terminologies
Foundations Assessment 1
2.2 Central Theme of Big Data
Central Theme of Big Data
Requirements of Programming Model
Understand Distributed Processing through a Story
Foundations Assessment 2
Environment and Installations
OracleVMInstallation
1Oracle_VM_Installation_1
Google Cloud Platform Setup
Google Cloud Platform Setup
How to install Ubuntu operating system on Virtual box
How to install PySpark on Ubuntu with Java and Python_3
How to configure Pyspark with Pycharm_with_Installation
Hadoop Ecosystem
3.1 Introduction to Hadoop Ecosystem
Introduction to Hadoop Ecosystem
Hadoop Ecosystem Assessment 1
Python for PySpark
Introduction to Programming
Introduction to Programming
Python Programming
Introduction to Python
Environment for Python
Executing Python Code
Python Assessment 1
Syntax, Indentation and Comments
Syntax, Indentation and Comments - Practical
Variables
Variable Practical's
Python Datatypes
Python Datatypes Practicals
Python Assessment 2
Python Operator Concepts
Python Operator Praticals
Control Flows in Python
Control Flows - IF ELSE Concepts
If Else Practical
Loops Theory
Loops Practical
Python Assessment 3
Python Function Concepts
Python Function Hands-on
Apache Spark
Introduction to Spark
Why Spark?
Advantages of Spark
What is Spark?
Components of Spark
History of Spark
Introduction to Spark Assessment1
LiveClassMay82023ArchitecutreofSpark
LiveclassMay52023WhatisSpark
LiveClassMay42023WhySparkandAdvantagesofSpark
Overview of the Spark
Architecture of Spark
Spark Session
Spark Sessionin Terminal & Jupyter notebook
Spark Sessionin Terminal & Jupyter notebook
Spark Language API
Overview of the Spark Assessment1
Dataframes and Partitions
How to Create Dataframe in Terminal and in Jupyter Notebook?
Spark Transformations
Spark Actions
Overview of the Spark Assessment2
Structured API Overview
Structured APIs - Dataframes and Datasets
Schema Definition
Spark Types
Structured API Execution
Structured API Overview Assessment1
Operations on Dataframes
Dataframe Columns
Columns as Expression
Dataframe Rows
Operations on Dataframe Assessment1
Ways of Creating Dataframe
Methods to Manipulate Columns
DataFrame Transformations
Operations on Dataframe Assessment2
Dataframe Transformation - Columns
Dataframe Transformations - Rows Part1
Dataframe Transformation - Rows Part2
Operations on Dataframe Assessment3
Working with Different Types of Data
Introduction to working with Different Types of Data
Working with Booleans
Working with Numbers
Working with Strings
Working with Strings Practical1
Working with String Practical2
Introduction to working with Different Types of Data Assessment 1
Working with Date and Time Stamps
Working with Null Concepts
Working with Nulls Practicals
Working with Complex Types
User Defined Functions - Concepts
User Defined Functions - Practicals
Working with Complex types practical
Introduction to working with Different Types of Data Assessment 2
Creating Dataframes from different sources
Data Sources Introduction
Read-API- Data Sources
Read-API-Practical
Write-API-Data Sources
Write-API-Practical
Creating Dataframes from different sources Assessment 1
Reading from CSV Files
Writing into CSV Files
Reading from JSON Files and Writing into JSON
Reading from Parquet and writing into Parquet
Reading from ORC and writing into ORC
Unstructured Data - Text File - Reading and Writing
Introduction to reading data from structured sources
Reading data from structured sources - Database - Concepts
Reading data from structured sources - Database - Practicals
Query Pushdown Concepts
Query Pushdown Praticals
Writing into structured sources - Database - Concepts
Writing into structured sources - Database - Practicals
Creating Dataframes from different sources Assessment 2
Aggregations
Introduction to Aggregations
Aggregataion Concepts - Count
Aggregation_Practical-1-Count
Aggregation1
Aggregation Concepts - First, Sum and Average
Aggregation - Practical - 2FirstLastAverage
Aggregation concepts - Statistical Functions
Aggregation-Practical-3-StatisticalFunctions
Aggregation Concepts - Grouping
Aggregation-Practical-4-GroupBy
Aggregation Concepts - Window Functions
Aggregation-Practical-5-WindowFunctions
Aggregation Concepts - RollUp and Cube
Aggregation-Practical-6-RollupandCube
Spark Joins
Spark Joins Theory-1-Introduction
Spark Joins Theory-2-How Joins Work
Spark Joins-Theory-3-Inner Joins
Spark Joins -Practical -1-Innerjoins
Saprk Joins - Theory-4 - Outer Joins
Spark Joins -Practical - OuterJoins
Spark Joins -Theory - 5-Left Semi & Anti Joins
Spark Joins - Practical - LeftSemiAntiJoins
Spark Joins -Theory -6-CrossJoin
Spark Joins - Practical- CrossJoins
Spark Joins -Theory -7-ChallengesInJoins
Spark Joins-5-Practical-TacklingtheChallengesinJoins
Spark Joins -Theory -8-CommunicationStrategies
Resilient Distributed Datasets-RDDs
What is an RDD ?
Introduction to Low Level APIs
Properties Of RDD
When to use RDDs
Creating RDDs
RDD Practical-1-Creating RDDs
RDD Lineage
RDD Transformations
RDD - Transformations Practical
RDD Actions
RDD Actions - Practical
RDDT Saving To File
RDD Saving to a File - Practical
Distributed Variables
Distributed Variables - Introduction
Broadcast Variables
Broadcast Variables - Practical
Accumulators
Accumulators - Practical
How Spark runs on a Cluster
Introduction
How Spark runs on a Cluster - ClusterManager
How Spark runs on a Cluster - ExecutionModes
Life Cycle a Spark Application - Outside Spark
Life Cycle of a Spark Application - Inside Spark
LiveclassMay92023PySparkSparkSession
LiveclassMay112023PysparkTransformations
LiveclassMay122023PySparkActions
LiveclassMay152023SparkStructuredAPIDatatypes
LiveclassMay162023PySparkLogicalPhysicalCatalystOptimizer
LiveclassMay182023PySparkColumnsandRows
LiveclassMay192023PysparkCreatingDataframes
LiveclassMay222023PySparkColumnManipulation
LiveClassMay242023PySparkRowTransformationsSort
LiveclassMay252023PySparkBooleans
LiveclassMay262023PySparkNumbers&Spaces
LiveclassMay292023PySparkStringManipulationDate
LiveclassMay30PySparkNullpracticals&ComplexDataTypes
LiveclassJune12023PySparkCompleTypesPracticalUDFTheory
LiveclassJune22023PySparkUDFPracticals
LiveclassJune52023PySparkDataSources1
LiveclassJune072023PySparkWritemode
LiveclassJune082023PySparkCSVJSONParquet
LiveclassJune092023DataSourceTextFileandplans
LiveclassJune132023PySparkReadingfromDatabase
LiveClassJune142023PySparkWritingtoDBandAggregationINtro
LiveclassJune152023PySparkAggregations
LiveClassJune162023PySparkAggregationsGroupBY
LiveclassJune192023PySparkWindowRollUPCube
LiveclassJune202023PySparkJOins1
LiveClassJune212023PysparkJoins2
LiveClassJune272023PySparkCommunicationStrategies
LiveClassJune282023PySparkJoinStrategiesHandson
LiveclassJuly032023PySparkJoinStrategyHints
LiveClassJuly42023PySparkRDD1
LiveClassJuly52023PySparkRDD2Transformation
LiveClassJuly062023PySparkRDDActions
LiveClassJuly72023PySparkDistributedVariables