In this post, we are going to learn about the dbutils and its’ command available DBFS Databricks File System.
Overview
The dbutils contain file-related commands. It used to contain all these utilities in dbutils.fs. It makes it easy to work with files available at databricks file system.
dbutils.fs Commands
Below are the listed command:
| COMMAND | DESCRIPTION | USE |
| ls | Used to list files under a directory | %python or %scala dbutils.fs.ls(“/directory_name”) OR %fs |
| mkdirs | Used to create directory | %python or %scala dbutils.fs.mkdirs(“/directory_name”) OR %fs mkdirs /directory_name |
| put | Used to overwrite file with provided value | %python or %scala dbutils.fs.put(“file_name.txt”, “Input String”) OR %fs |
| head | Used to read the top content of the file | %python or %scala dbutils.fs.head(“file_name.txt”) OR %fs |
| rm | Remove file | %python or %scala dbutils.fs.rm(“file_name.txt”) OR %fs |
You can prefix with dbfs:/ (eg. dbfs:/file_name.txt) with the path to access the file/directory available at the databricks file system.
For deleting the files of a folder recursively, use the below command:
%fs rm -f <folder_name>
Wrapping
We can use these dbutils commands on the databricks notebook to access the files available in the databricks file system to perform any action listed above.