Requirement

In this post, we are going to learn about Temporary View in Databricks or can say in Spark. We will also see when to create temporary view and how to access it.

Solution

For this exercise, we are going to use the below sample dataset:

Step 1: Load Data into DataFrame

First, we have to read the data from the CSV file. Here is the code for the same:

 %scala
val file_location = "/FileStore/tables/emp_data1-3.csv"

val df = spark.read.format("csv")
  .option("inferSchema", "true")
  .option("header", "true")
  .option("sep", ",")
  .load(file_location)

display(df)

Here, we have loaded the data into the dataframe. Now, we can create a temporary view to refer to this data.

Step 2: Create Temporary View in Databricks

The temporary view or temp view will be created and accessible within the session. Once the session expires or end, the view will not be available to access. It can be used as a cache.

 df.createOrReplaceTempView("df_tempview")

Here, we have created a temp view named df_tempview on dataframe df. You can keep any name for the temp view. It will be accessible using the created name.

Now, if you query on the temp view, you will be able to see the data:

This temp view will be available for a session and within the same notebook. If you try to access it in any other notebook, it will throw an error saying Table or view not found as shown below.

Wrapping Up

The temporary view is useful if you want to access the same data multiple times within the notebook. Also, it will not copy the actual data at any place. The data will be available for the same session and once the session expires or end, it will drop automatically.

Sharing is caring!

Subscribe to our newsletter

Name*

Email*