jdk1.8/Hadoop2.9.0/Hive2.2/

Install File

설치 파일 준비

Hadoop 다운로드 페이지에서 releases > Apache release archive site에 가면 모든 버전을 다운 받을 수 있다.

하둡다운로드

하이브다운로드

jdk다운로드

↓

 cd 다운로드
 tar xvfz jdk-8u161-linux-x64.tar.gz 
 cp -r jdk1.8.0_161/ /usr/local/jdk1.8
 
 cd /usr/bin
 ln -s /usr/local/jdk1.8/bin/java java

 tar xvfz hadoop-2.9.0.tar.gz 
 cp -r hadoop-2.9.0 /usr/local/
 
 tar xvfz apache-hive-2.2.0-bin.tar.gz 
 cp -r apache-hive-2.2.0-bin /usr/local/hive

Generate SSH Key

SSH Key : 서버간의 통신을 위한 프로토콜
ssh-keygen is a standard component of the Secure Shell (SSH) protocol suite found on Unix and Unix-like computer systems used to establish secure shell sessions between remote computers over insecure networks, through the use of various cryptographic techniques.

# ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
# ssh-copy-id -i /root/.ssh/id_dsa.pub root@slave1

SSH Key

profile setting

JAVA_HOME=/usr/local/jdk1.8
HADOOP_HOME=/usr/local/hadoop-2.9.0
HIVE_HOME=/usr/local/hive
CLASSPATH=/usr/local/jdk1.8/lib
export JAVA_HOME HADOOP_HOME HIVE_HOME CLASSPATH

PATH=$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin:$PATH

Hadoop environment setting

hadoop-env.sh

export HADOOP_HOME=/usr/local/hadoop-2.9.0
export HADOOP_PID_DIR=/usr/local/hadoop-2.9.0/pids
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop

# The java implementation to use.
export JAVA_HOME=/usr/local/jdk1.8