What is the path to a local variable from hadoop fs, bash?

debugcn 投稿 Dev

Finn Maunsell

Windows user here. Will be a really simple answer, it must have been asked but I mustn't know the terminology to find the question here in Stack.

I'm trying to reference any file outside of the hadoop fs (hadoop filesystem?) on my Linux VM. An example is this:

hadoop fs -put home/udacity_training/data/access_logs.txt access_logs.txt

In the above call I'm trying to copy the file from the 'data' folder to my hadoop filesystem. However for every path I try reference I get 'no such file or directory' error. Even when trying to use 'copyFromLocal' like so:

hadoop fs -copyFromLocal access_logs.txt access_logs.txt

It doesn't work!

Any help will be massively appreciated.

Ramesh Maharjan

With the following command

hadoop fs -put home/udacity_training/data/access_logs.txt access_logs.txt

You are telling hadoop to copy access_logs.txt from local file system to access_logs.txt directory in hdfs. What hadoop is doing is trying to find that directory inside home directory of the hadoop user

So you can create home directory of hadoop user with which you are running hadoop commands as

hadoop dfs -mkdir -f /user/Finn

I am assuming Finn is the username

Then you apply

hadoop fs -put home/udacity_training/data/access_logs.txt /user/Finn/

You will see that access_logs.txt should be copied

この記事はインターネットから収集されたものであり、転載の際にはソースを示してください。

侵害の場合は、連絡してください[email protected]

編集2021-06-1

コメントを追加

サインイン

分類Dev

Related 関連記事

記事

What is the path to a local variable from hadoop fs, bash?

What is the path to a local variable from hadoop fs, bash?

What is the benefit of using a local variable?

Multiple path environment variable setup lines with bash

Iterate Over Files in Variable Path (Bash)

Bash: Combine creating a file and storing the Path in a variable

Run cd with path variable in bash script not working

javascript: modify global variable from local variable

What is the effect of a path separator at the beginning of an environment variable?

Get the current UNC path from a local path in powershell

hadoop fs、bashからローカル変数へのパスは何ですか？

Use data from variable in bash

Hadoop accessing 3rd party libraries from local file system of a Hadoop node

In BASH is there a way to read Variable names from a variable?

undefined local variable or method `createaddress_hotel_users_path'

read a file from Server path - bash

Problems with node path from bash scripts

Handling from json reaponse function to local variable

using local variable from one function to another

delete items from path environment variable

hadoop fs -ls内存不足错误

What is Shadowing outer local variable in rubocop and how do I fix this?

What is the right way to verify if local Storage variable exists? - XAMARIN

Trying to load images from local path fails after restart in iOS

What does asterisk "*" mean in a unix PATH environment variable?

Bash get last line from a variable

Pass a variable from bash into chef template

Getting variable values from variable names listed in array in Bash

What is sequence file in hadoop?

How to assign local variable from observable Angular2

Variable from API not getting passed into local storage in Angular