hadoop

hadoop environment setup

. set use key login linux command

1
2
3

ssh-keygen -t rsa 
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 
chmod 0600 ~/.ssh/authorized_keys

. check key login is true
use ssh localhost can login linux that is not input password

you must config JAVA_HOME environment

sudo nano ~/.bashrc

1	export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64

source ~/.bashrc

echo $JAVA_HOME
1
/usr/lib/jvm/java-8-openjdk-amd64

1 2	YangTianT4900c-00:~/work/service/hadoop-3.2.1$ ./sbin/start-yarn.sh YangTianT4900c-00:~/work/service/hadoop-3.2.1$ ./sbin/stop-yarn.sh

-put

1	YangTianT4900c-00:~/work/service/hadoop-3.2.1$ ./bin/hadoop fs -put LICENSE.txt input

-ls

1	YangTianT4900c-00:~/work/service/hadoop-3.2.1$ ./bin/hadoop fs -ls hdfsData

. wordcount can count word numbers

1	YangTianT4900c-00:~/work/service/hadoop-3.2.1$ ./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.1.jar wordcount hdfsData/LICENSE.txt output

. show result

1	YangTianT4900c-00:~/work/service/hadoop-3.2.1$ cat output/part-r-00000

最后更新： 2019年11月07日 19:47

赏