createDataFrame Archives - BIG DATA PROGRAMMERS

Get column value from Data Frame as list in Spark

Tagged: collect, createDataFrame, map, RDD, row, Spark, spark dataframe

Requirement In this post, we are going to extract or get column value from Data Frame as List in Spark. Let’s take an example, you have a data frame with some schema and would like to get a list of values of a column for any further process. Solution Let’sRead More →

Create an Empty DataFrame in Spark

Tagged: createDataFrame, dataframe, empty dataframe, schema, Spark

Requirement In this post, we are going to learn how to create an empty dataframe in Spark with and without schema. Prerequisite Spark 2.x or above Solution We will see create an empty DataFrame with different approaches: PART I: Empty DataFrame with Schema Approach 1:Using createDataFrame Function import org.apache.spark.sql.types.{StructType, StructField,Read More →