Hadoop : getmerge: merge files conent into a single file

'getmerge' command is used to get all the files in the directories that match the source file pattern and merge and sort them to only one file on local file system.

 

Merge all the files in a directory

Takes a source directory and a destination file as input and concatenates files in source directory into the destination local file.

Usage:

hadoop fs [generic options] -getmerge [-nl] <src> <localdst>

localdst: is the destination file, presented in local file system

src: Is the directory in HDFS.

$ hadoop fs -getmerge /user/cloudera/dir1/  /Users/krishna/new.txt

Above statement copies all files in HDFS directory “/user/cloudera/dir1/” to local system file “/Users/krishna/new.txt”.

Previous                                                 Next                                                 Home