Understanding Spark

• Understanding Spark through Map Reduce
• Spark Input Formats And Parallel factor
• Spark Code Analysis
• Understanding Spark Serialization

 

Spark with Examples

• Word Count in Spark
• Read Write Parquet Files using Spark
• Word Count using CombineByKey in Spark
• Write a CSV text file from Spark

 

InputFormats

• CombineParquetFileInputFormat

 

Oozie schedular

• How to submit spark job through oozie

 

Spark Transformations

• Spark Map
• Spark mapPartitions
• Spark Map Vs mapPartitions

 

Joins

• DataFrame  Join in Spark
• Map Side Joins in Spark
• DataFrame Broadcast Join

 

ReduceByKey And CombineByKey

• Word Count in Spark reduceByKey
• Word Count using CombineByKey in Spark
• reduceByKey vs CombineByKey

 

GroupByKey And CombineByKey

 

• GroupByKey
• Collect all value for a given key using CombineByKey
• CombineByKey to find Max
• GroupByKey Vs ReduceBykey/CombineByKey

 

Aerospike I/O

• Write to Aerospike via Map
• Write to Aerospike via MapPartition
• Read from Aerospike via Map
• Read From Aerospike via MapPartition

 

Utils(Avro)

• Spark Avro Util
• Convert Avro Object To Row
• Define Avro Schema and create Avro Class
• Avro SerDe with Confluent
• Avro schema to class interconversion
• Avro schema evolution compatibility check