Write FlowFile data to Hadoop Distributed File System (HDFS)
Tags: hadoop, HDFS, put, copy, filesystem
Properties:
In the list below, the names of required properties appear in bold. Anyother properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language (or simply EL), and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.
| Name | Description | Default Value | Valid Values | EL | Sensitive |
|---|---|---|---|---|---|
| Hadoop Configuration Resources | A file or comma separated list of files which contains the Hadoop file system configuration. Without this, Hadoop will search the classpath for a 'core-site.xml' and 'hdfs-site.xml' file or will revert to a default configuration. | No | No | ||
| Directory | The parent HDFS directory to which files should be written | Yes | No | ||
| Conflict Resolution Strategy | Indicates what should happen when a file with the same name already exists in the output directory | fail |
| No | No |
| Block Size | Size of each block as written to HDFS. This overrides the Hadoop Configuration | No | No | ||
| IO Buffer Size | Amount of memory to use to buffer file contents during IO. This overrides the Hadoop Configuration | No | No | ||
| Replication | Number of times that HDFS will replicate each file. This overrides the Hadoop Configuration | No | No | ||
| Permissions umask | A umask represented as an octal number which determines the permissions of files written to HDFS. This overrides the Hadoop Configuration dfs.umaskmode | No | No | ||
| Remote Owner | Changes the owner of the HDFS file to this value after it is written. This only works if NiFi is running as a user that has HDFS super user privilege to change owner | No | No | ||
| Remote Group | Changes the group of the HDFS file to this value after it is written. This only works if NiFi is running as a user that has HDFS super user privilege to change group | No | No |
Relationships:
| Name | Description |
|---|---|
| failure | Files that could not be written to HDFS for some reason are transferred to this relationship |
| success | Files that have been successfully written to HDFS are transferred to this relationship |