How-to Quick-Start with Hadoop on Ubuntu 15.10 Wily 32-64bit Step-by-Step Easy Guide

October 1st, 2015 by thelinuxevangelist

Hadoop 2.X Quick-Start for Ubuntu 15.10 Wily Linux




The Linux Tutorial Shows You Step-by-Step How-to Install and Getting-Started with Apache Hadoop/Map-Reduce vanilla in Pseudo-Distributed mode on Ubuntu 15.10 Wily Werewolf i386/amd64 GNU/Linux Desktop.

Hadoop is a distributed master-slave that consists of the Hadoop Distributed File System (HDFS) for storage and Map-Reduce for computational capabilities.

The Guide Describe a System-Wide Installation with Root Privileges but You Can Easily Convert the Procedure to a Local One.

The Apache Hadoop for Ubuntu 15.10 Wily Require an Oracle JDK 7+Installation on System.

Install Hadoop for Ubuntu 15.10 Wily - Featured
  1. Download Latest Apache Hadoop Stable Release:

    Apache Hadoop Binary tar.gz

  2. Double-Click on Archive and Extract Into /tmp Directory
    Ubuntu Apache Hadoop Stable Quick-Start - Unity Extract tar.gz Archive
  3. Open a Shell Terminal emulator window
    Ctrl+Alt+t
    (Press “Enter” to Execute Commands)

    Install Hadoop for Ubuntu 15.10 Wily - Open Terminal
  4. Relocate Apache Hadoop Directory
    sudo su

    If Got “User is Not in Sudoers file” then Look: Solution

    mv /tmp/hadoop* /usr/local/
    ln -s /usr/local/hadoop* /usr/local/hadoop
    mkdir /usr/local/hadoop/tmp
    chown -R root:root /usr/local/hadoop*
  5. How-to Install Required Java JDK 7+ on Ubuntu:

    Install Oracle JDK 7+ for Ubuntu
  6. Set JAVA_HOME in Hadoop Env File

    sudo su

    If Got “User is Not in Sudoers file” then Look: Solution

    mkdir /usr/local/hadoop/conf
    nano /usr/local/hadoop/conf/hadoop-env.sh

    Append:

    export JAVA_HOME=/usr/lib/jvm/<oracleJdkVersion>

    Ctrl+x to Save & Exit from nano Editor :)

  7. Hadoop Configuration for Pseudo-Distributed mode
    nano /usr/local/hadoop/conf/core-site.xml

    Append:

     <?xml version="1.0"?>
     <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
     <configuration>
     <property>
     <name>hadoop.tmp.dir</name>
     <value>/usr/local/hadoop/tmp</value>
     </property>
     <property>
     <name>fs.default.name</name>
     <value>hdfs://localhost:8020</value>
     </property>
     </configuration>
    

    Next:

    nano /usr/local/hadoop/conf/hdfs-site.xml

    Append:

     <?xml version="1.0"?>
     <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
     <configuration>
     <property>
     <name>dfs.replication</name>
     <value>1</value>
     </property>
     <property>
     <!-- specify this so that running 'hadoop namenode -format'
     formats the right dir -->
     <name>dfs.name.dir</name>
     <value>/usr/local/hadoop/cache/hadoop/dfs/name</value>
     </property>
     </configuration>
    

    Last:

    nano /usr/local/hadoop/conf/mapred-site.xml

    Append:

     <?xml version="1.0"?>
     <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
     <configuration>
     <property>
     <name>mapred.job.tracker</name>
     <value>localhost:8021</value>
     </property>
     </configuration>
    
  8. SetUp Local Path & Environment

    exit
    cd $HOME
    nano .bashrc

    Inserts:

    HADOOP_HOME=/usr/local/hadoop
    export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
    export JAVA_HOME=/usr/lib/jvm/<oracleJdkVersion>

    Then Load the New Setup:

    source $HOME/.bashrc
  9. SetUp Needed Local SSH Connection
    sudo su -c "openssh-server"

    Generate SSH Keys to Access:

    ssh-keygen -b 2048 -t rsa
    cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys

    Testing Connection:

    ssh 127.0.0.1
  10. Formatting HDFS
    hadoop namenode -format

    Install Hadoop for Ubuntu 15.10 Wily - Terminal Apache Hadoop HDFS Formatting Succcess

  11. Starting Up Hadoop Database
    start-all.sh
  12. Apache Hadoop Database Quick-Start Guide:

    Hadoop MapReduce Quick-Start
(Visited 7 times, 1 visits today)
Share on Tumblr

Tags: , , , , , , , , , , , , , ,


Comments are disabled