Load hive table into spark using Scala
Requirement Assume you have the hive table named as reports. It is required to process this dataset in spark. OnceRead more.
Calculate percentage in spark using scala
Requirement You have marks of all the students of a class with roll number in CSV file, It is neededRead more.
How to create spark application in IntelliJ
Requirement In spark-shell, it creates an instance of spark context as sc. Also, we don’t require to resolve dependency whileRead more.
How to get partition record in Spark Using Scala
Requirement Suppose we are having a text format data file which contains employees basic details. When we load this fileRead more.
Find max value in Spark RDD using Scala
Requirement Suppose we are having a source file, which contains basic information about Employees like employee number, employee name, designation,Read more.
Read CSV file in Spark Scala
Requirement Suppose we have a dataset which is in CSV format. We want to read the file in spark usingRead more.
How to calculate Rank in dataframe using scala with example
Requirement : You have marks of all the students of class and you want to find ranks of students usingRead more.
Join in spark using scala with example
Requirement You have two table named as A and B. and you want to perform all types of join inRead more.
How to execute Scala script in Spark without creating Jar
Requirement The spark-shell is an environment where we can run the spark scala code and see the output on theRead more.
How to read JSON file in Spark
Requirement Let’s say we have a set of data which is in JSON format. The file may contain data eitherRead more.
How to add new column in Spark Dataframe
Requirement When we ingest data from source to Hadoop data lake, we used to add some additional columns with theRead more.
Load spark dataframe into non existing hive table
Requirement: You have a dataframe which you want to save into hive table for future use. But you do notRead more.
Create a spark dataframe from sample data
Requirement: You have sample data of some students and you want to create a dataframe to perform some operations. Given:Read more.