Hadoop: HDFS: java program to create directory

In this post, I am going to explain how to create a directory in HDFS using Java API. org.apache.hadoop.fs.FileSystem class is used to access and manage files/directories in HDFS. "FileSystem” class provide mkdirs method to create a directory. This method create parent directories, if they are not exist.

public boolean mkdirs(Path f) throws IOException

Method returns true, if directory created successfully, else false.

 

Following is the step-by-step procedure to create directory in HDFS.

Step 1: Set JAVA_HOME (If it is not set already)

Step 2: Set HADOOP_CLASSPATH like following

export HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar

Step 3: Following is the java application that creates a directory.

import java.io.File;
import java.io.IOException;
import java.net.URI;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;

public class CreateDirectory {
 private static final String uri = "hdfs://localhost/user/harikrishna_gurram/";
 private static final String directory = uri + "directory1" + File.separator
   + "directory2";

 private static final Configuration config = new Configuration();

 public static void createDirectory() throws IOException {

  /* Get FileSystem object for given uri */
  FileSystem fs = FileSystem.get(URI.create(uri), config);
  boolean isCreated = fs.mkdirs(new Path(directory));

  if (isCreated) {
   System.out.println("Directory created");
  } else {
   System.out.println("Directory creation failed");
  }
 }

 public static void main(String args[]) throws IOException {
  createDirectory();
 }

}

Host details for above uri is configured in “hadoop-2.6.0/etc/hadoop/core-site.xml” file.

<configuration>

        <property>

                <name>fs.defaultFS</name>

                <value>hdfs://localhost/</value>

                <description>NameNode URI</description>

        </property>

</configuration>

Please refer to the setup for hadoop here.

Step 4: Compile above java file.

$ hadoop com.sun.tools.javac.Main  CreateDirectory.java

Step 5: Create jar file.

$ jar cf dir.jar CreateDirectory*class

Step 6: Run jar file.    

$ hadoop jar dir.jar CreateDirectory

$ hadoop jar dir.jar CreateDirectory

Directory created

$

$ hadoop fs -ls /user/harikrishna_gurram/directory1

Found 1 items

drwxr-xr-x   - harikrishna_gurram supergroup          0 2015-06-22 10:44 /user/harikrishna_gurram/directory1/directory2

$

Previous                                                 Next                                                 Home