Hadoop: Java: Delete file

FileSystem class provides delete method to delete a file (or) directory.

boolean delete(Path f, boolean recursive) throws IOException

Delete a file. To delete a directory which has contents, set recursive to true.

Following application deletes a directory “/user/harikrishna_gurram/dir1”, which has contents.

$ hadoop fs -ls /user/harikrishna_gurram/dir1

Found 2 items

-rw-rw-r--   5 Phalgun            group1             20 2015-06-18 12:45 /user/harikrishna_gurram/dir1/append.txt

-rw-r--r--   5 harikrishna_gurram supergroup        468 2015-06-18 12:45 /user/harikrishna_gurram/dir1/sample.txt

Step 1: Set JAVA_HOME (If it is not set already)

Step 2: Set HADOOP_CLASSPATH like following

export HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar

Step 3: Following is the java application that deletes directory from HDFS.

import java.io.IOException;
import java.net.URI;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;

public class DeleteDir {
 private static final String uri = "hdfs://localhost/user/harikrishna_gurram";
 private static final Configuration config = new Configuration();

 public static void printFileInfo() throws IOException {
  /* Get FileSystem object for given uri */
  FileSystem fs = FileSystem.get(URI.create(uri), config);
  String deleteDir = uri + "/dir1";

  boolean flag = fs.delete(new Path(deleteDir), true);
  System.out.println("Is directory deleted " + flag);
 }

 public static void main(String args[]) throws IOException {
  printFileInfo();
 }
}

String uri = "hdfs://localhost/user/harikrishna_gurram";

“uri” is used to locate file location in HDFS. Host details for above uri are configured in “hadoop-2.6.0/etc/hadoop/core-site.xml” file.

<configuration>

        <property>

                <name>fs.defaultFS</name>

                <value>hdfs://localhost/</value>

                <description>NameNode URI</description>

        </property>

</configuration>

Please refer to the setup for hadoop here.

Step 4: Compile above java file.

$ hadoop com.sun.tools.javac.Main DeleteDir.java

Step 5: Create jar file.

$ jar cf delete.jar DeleteDir*class

Step 6: Run jar file.    

$ hadoop jar delete.jar DeleteDir

$ hadoop jar delete.jar DeleteDir

Is directory deleted true

$

$ hadoop fs -ls /user/harikrishna_gurram/dir1

ls: `/user/harikrishna_gurram/dir1': No such file or directory

Previous                                                 Next                                                 Home