Installing Hadoop 1.0.x-stable On 10+ Nodes With HBase, ZooKeeper, Thrift, HappyBase – Part 2!

We left off setting up the NameNode for HDFS. Let’s configure/install the JobTracker (www4) and rest of the slaves www5-www15.

• Set up the HDFS unix user on each node with the same gid/uid; In our case it is id 509 groupadd -g 509 hdfs ; useradd -u 509 -g 509 hdfs
• Install java as before as root yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel
• wget http://apache.petsads.us/hadoop/common/stable/hadoop-1.0.4.tar.gz
• tar -C /usr/local -zxvf hadoop-1.0.4.tar.gz
• ln -s /usr/local/hadoop-1.0.4 /usr/local/hadoop
• mkdir /usr/local/hadoop/namenode ; mkdir /usr/local/hadoop/datanode
• mkdir -p /var/hadoop/temp ; chown hdfs /var/hadoop/temp
• chown -R hdfs /usr/local/hadoop-1.0.4
• Copy the config files modified in part 1 over… from the master node: scp /usr/local/hadoop/conf/* hdfs@www4:/usr/local/hadoop/conf/
• Put in your ~/.bashrc / rc file your JAVA_HOME … export JAVA_HOME=/usr/lib/jvm/java

Download HBase and get started with its installation:

• wget http://apache.mesi.com.ar/hbase/stable/hbase-0.94.4.tar.gz

• tar -C /usr/local -zxvf hbase-0.94.4.tar.gz

• ln -s /usr/local/hbase-0.94.4 /usr/local/hbase

• edit /usr/local/hbase/conf/hbase-site.xml:

  <configuration>
    <property>
      <name>hbase.rootdir</name>
      <value>hdfs://www3:8020/hbase</value>
      <description>The directory shared by RegionServers.
      </description>
    </property>
    <property>
      <name>hbase.cluster.distributed</name>
      <value>true</value>
    </property>
    <property>
      <name>hbase.zookeeper.quorum</name>
      <value>www3,www5,www7,www9,www11</value>
    </property>
    <property>
      <name>hbase.zookeeper.property.dataDir</name>
      <value>/usr/local/hbase/zookeeper</value>
    </property>
  </configuration>
  

• start-hbase.sh