IntelliJ

Read CSV file in Spark Scala

Tagged: case-class, csv, dataframe, IntelliJ, sbt, scala, Spark

Requirement Suppose we have a dataset which is in CSV format. We want to read the file in spark using Scala. So the requirement is to create a spark application which read CSV file in spark data frame using Scala. Components Involved Following components are involved: Spark RDD/Data Frame ScalaRead More →

Read CSV file in Spark Using Scala

Tagged: csv, dataframe, IntelliJ, sbt, scala, Spark

Find max value in Spark RDD using Scala

Tagged: IntelliJ, maxvalue, RDD, sbt, scala, Spark, sparkapplication

How to create spark application in IntelliJ

Tagged: IntelliJ, RDD, sbt, scala, Spark, sparkjob

Requirement In spark-shell, it creates an instance of spark context as sc. Also, we don’t require to resolve dependency while working on spark shell. But it all requires if you move from spark shell to IDE. So how to create spark application in IntelliJ? In this post, we are goingRead More →