Spark-Scala Quiz-1

Spark-Scala Quiz-1

Welcome to your Spark-Scala Quiz -1

1. There is table in hive named as “products”. What is the correct syntax to load this table into spark dataframe using Scala?
2. Let's Say you have dataframe "mydf" with all columns as String datatype .It have few null values.It is needed to replace all null values with "NA". What is the correct syntax to replace null values with "NA" ?

3. While doing Coding ,it is needed to see the datatype of all columns of dataframe. How would you get this information?
4. "mydf " is a dataframe having thousands of records. You need to look only 10 records .How would you get it done?
5. How to get count of distinct records of a dataframe?
6. Use of Cache will improve Processing ,Once the data is cached ?
7. How would you get the number of partitions of a dataframe "mydf" ?
8. How would you convert "mydf" dataframe to rdd?

0

Join in hive with example

Requirement You have two table named as A and B. and you want to perform all types of join in ...
Read More

Join in pyspark with example

Requirement You have two table named as A and B. and you want to perform all types of join in ...
Read More

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.