Requirement In this post, we are going to extract or get column value from Data Frame as List in Spark. Let’s take an example, you have a data frame with some schema and would like to get a list of values of a column for any further process. Solution Let’sRead More →
Requirement In this post, we are going to create an RDD and then will read the content in Spark. Solution //Create RDD: val dummyRDD = sc.parallelize(Seq( (“1001”, “Ename1”, “Designation1”, “Manager1”) ,(“1003”, “Ename2”, “Designation2”, “Manager2”) , (“1001”, “Ename3”, “Designation3”, “Manager3”) )) //Read RDD dummyRDD.collect().foreach(println(_)) //Read specific Column: dummyRDD.collect().foreach(data => println(data._1,Read More →
Requirement You have marks of all the students of a class with roll number in CSV file. It is needed to calculate the percentage of marks of students in Spark using Scala. Given: Download the sample CSV file which have 7 columns, 1st column is Roll no and other 6Read More →
