Saturday, 2 January 2016

Hadoop: HDFS: Extended attributes

Extended attributes are used to add additional metadata to a file (or) directory. This additional metadata is not interpreted by system; this is used by client applications. For example, you can specify character encoding of the file, encryption used etc.,

Hadoop provides following command to work with extended attributes.

setfattr: Sets an extended attribute name and value for a file or directory.
Usage:
hadoop fs [generic options] -setfattr {-n name [-v value] | -x name} <path>
        
Options
-n name: The extended attribute name.
-v value: The extended attribute value. There are three different encoding methods for the value. If the argument is enclosed in double quotes, then the value is the string inside the quotes. If the argument is prefixed with 0x or 0X, then it is taken as a hexadecimal number. If the argument begins with 0s or 0S, then it is taken as a base64 encoding.
-x name: Remove the extended attribute.

getfattr: Displays the extended attribute names and values (if any) for a file or directory.
        
Usage:
hadoop fs [generic options] -getfattr [-R] {-n name | -d} [-e en] <path>
        
Options:
-R: Recursively list the attributes for all files and directories.
-n name: Dump the named extended attribute value.
-d: Dump all extended attribute values associated with pathname.

-e encoding: Encode values after retrieving them. Valid encodings are “text”, “hex”, and “base64”.

$ hadoop fs -setfattr -n user.encoding -v "UTF-8" /user/harikrishna_gurram/dir1/append.txt
$ hadoop fs -setfattr -n user.encryption -v "SHA-1" /user/harikrishna_gurram/dir1/append.txt
$ 
$ 
$ hadoop fs -getfattr -n user.encoding  /user/harikrishna_gurram/dir1/append.txt
# file: /user/harikrishna_gurram/dir1/append.txt
user.encoding="UTF-8"
$
$ hadoop fs -getfattr -n user.encryption /user/harikrishna_gurram/dir1/append.txt
# file: /user/harikrishna_gurram/dir1/append.txt
user.encryption="SHA-1"




Previous                                                 Next                                                 Home

No comments:

Post a Comment