convert rdd to dataframe

Tagged: , , , , , ,

Requirement In this post, we will convert RDD to Dataframe in Spark with Scala. Solution Approach 1: Using Schema Struct Type //Create RDD: val dummyRDD = sc.parallelize(Seq(                             (“1001”, “Ename1”, “Designation1”, “Manager1”)                            ,(“1003”, “Ename2”, “Designation2”, “Manager2”)                            ,(“1001”, “Ename3”, “Designation3”, “Manager3”)                             )) val schema = StructType( StructField(“empno”, StringType, true) ::Read More →

Tagged: , ,

Requirement In this post, we will convert RDD to Dataframe in Pyspark. Solution Let’s create dummy data and load it into an RDD. After that, we will convert RDD to Dataframe with a defined schema. # Create RDD empData = [(7389, “SMITH”, “CLEARK”, 9902, “2010-12-17”, 8000.00, 20),             (7499, “ALLEN”, “SALESMAN”,Read More →