- Install File
- 설치 파일 준비
Hadoop 다운로드 페이지에서 releases > Apache release archive site에 가면 모든 버전을 다운 받을 수 있다.
cd 다운로드
tar xvfz jdk-8u161-linux-x64.tar.gz
cp -r jdk1.8.0_161/ /usr/local/jdk1.8
cd /usr/bin
ln -s /usr/local/jdk1.8/bin/java java
tar xvfz hadoop-2.9.0.tar.gz
cp -r hadoop-2.9.0 /usr/local/
tar xvfz apache-hive-2.2.0-bin.tar.gz
cp -r apache-hive-2.2.0-bin /usr/local/hive
Generate SSH Key
SSH Key : 서버간의 통신을 위한 프로토콜
ssh-keygen is a standard component of the Secure Shell (SSH) protocol suite found on Unix and Unix-like computer systems used to establish secure shell sessions between remote computers over insecure networks, through the use of various cryptographic techniques.
# ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
# ssh-copy-id -i /root/.ssh/id_dsa.pub root@slave1
SSH Key
JAVA_HOME=/usr/local/jdk1.8
HADOOP_HOME=/usr/local/hadoop-2.9.0
HIVE_HOME=/usr/local/hive
CLASSPATH=/usr/local/jdk1.8/lib
export JAVA_HOME HADOOP_HOME HIVE_HOME CLASSPATH
PATH=$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin:$PATH
- Hadoop environment setting
export HADOOP_HOME=/usr/local/hadoop-2.9.0
export HADOOP_PID_DIR=/usr/local/hadoop-2.9.0/pids
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop
# The java implementation to use.
export JAVA_HOME=/usr/local/jdk1.8
masters
slaves
core-site.xml
hdfs-site.xml
mapred-site.xml
yarn-env.sh
yarn-site.xml