Requirement In the last post, we have imported the CSV file and created a table using the UI interface in Databricks. In this post, we are going to create a delta table from a CSV file using Spark in databricks. Solution Let’s use the same sample data: empno ename designationRead More →
Requirement Let’s say there is a system which used to extract the data from any source (can be Databases, Rest API, etc.) and dumping into Azure Data Lake Storage aka. ADLS Gen2 storage. Now, we want to access and read these files in Spark for further processing for our businessRead More →
Tutorial – 1 To Learn Python, You don’t need to have python installed on your system, Rather I would suggest you to make use of colab. Colab is the abbreviation of Colaboratry by Google. It allows you to write and execute Python in Your Browser. Step 1: Google Account: Read More →
Requirement : You have marks of all the students of class and you want to find ranks of students using python. Given : A pipe separated file which contains roll number and marks of students : below are the sample values :- R_no marks 101 389 102 412 103 435Read More →
Requirement In the last post, we have demonstrated how to load JSON data in Hive non-partitioned table. This time we are having the same sample JSON data. The requirement is to load JSON Data into Hive Partitioned table using Spark. The hive table will be partitioned by some column(s). TheRead More →
