Tuesday, September 13, 2016

Write files to Hadoop which are having space in the file name

Assume we have a file called RCA_10th Sep_Double Demand.docx in our local system and we want to move this to HDFS.

Commonly we write the HDFS statement like below:

hadoop fs -put /home/bandaru.nagesh_gmail/RCA_10th Sep_Double Demand.docx /user/bandaru.nagesh_gmail/RCA_10th Sep_Double Demand.docx

You will be getting below error:

put: `Demand.docx': No such file or directory


So, Let's try differently and provide an Quote marks for the path:

hadoop fs -put "/home/bandaru.nagesh_gmail/RCA_10th Sep_Double Demand.docx" "/user/bandaru.nagesh_gmail/RCA_10th Sep_Double Demand.docx"

Opps! You will be getting below error:

put: unexpected URISyntaxException

Solution:

Here is the solution for the issue, we need to replace the space with \%20
So that Hadoop recognize this as space and allow to run command.
so, lets see how this work now. run below command

hadoop fs -put /<Local Path>/RCA_10th\%20Sep_Double\%20Demand.docx /<HDFS Path>/RCA_10th_Sep_Double_Demand.docx
Wow! It works

Results: You can find the file moved to HDFS

Note: Destination file should be provided with out any space, but the source file can be provided with \%20 with replace of space





Let us try differently and try to put the space in the HDFS also and see what will happen:

hadoop fs -put /home/bandaru.nagesh_gmail/RCA_10th\%20Sep_Double\%20Demand.docx /user/bandaru.nagesh_gmail/RCA_10th_Sep\%20Double\%20Demand.docx

See below Results after above two statements:
We can see that file has been copied with \%20, so that is the reason I have explained to remove space in the HDFS file.



No comments:

Post a Comment