I am extremely new to Hadoop, know almost nothing about this concept.
I just started a class and attempted to install a Hadoop. I simply followed instructions in the notes
And finally executed the following command
$ ./yarn jar /usr/local/hadoop/hadoop2.9.1/share/hadoop/mapreduce/hadoop-mapreduce-examples2.9.1.jar wordcount /in /out
The output was multiple 'retrying to connect' statements with 'java.net.ConnectException: your endpoint configuration is wrong' at the ends
For some reason I can't find the 'jps' command to check if services are running
What can I check?
Thank you for your help!
First of all, my Java wasn't fully installed; thus, I had to finalized the installation
yum list java*devel*
sudo yum install java-1.7.0-openjdk-devel.x86_64
That let me use the jps command
Then checked hdfs-site.xml for a property dfs.datanode.data.dir
<property>
<name>dfs.datanode.data.dir</name>
<value>file:///data/1/dfs/dn</value>
</property>
Since the NameNode is formatted you don’t have an option other than formatting (deleting) the contents in DataNode as well. So go to the directory location specified in dfs.datanode.data.dir and issue a delete on that directory across all DataNodes.
Once the data directories are formatted in all DataNodes, I restarted the DataNode process on all DataNodes
Then checked for failed cluster ID's in Datanode's logs
Copied to clipboard datanode clusterID for example, CID-8bf63244-0510-4db6-a949-8f74b50f2be9
and run following command under HADOOP_HOME/bin directory
./hdfs namenode -format -clusterId CID-8bf63244-0510-4db6-a949-8f74b50f2be9
Everything worked !
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments