How-to Install Hadoop on Fedora 26 Easy Guide

July 12th, 2017 by thelinuxevangelist

Apache Hadoop Quick-Start for Fedora 26




Hello Fedora User! This Tutorial Shows You Step-by-Step How-to Install Hadoop on Fedora 26 32/64bit GNU/Linux Desktop.

Especially relevant: this guide shows an Apache Hadoop Vanilla Installation in Pseudo-Distributed mode on Fedora 26.

First of all Hadoop is a distributed master-slave that consists of the Hadoop Distributed File System (HDFS) for storage and Map-Reduce for computational capabilities.

And Hadoop Distributed File System (HDFS) is a distributed file system that spreads data blocks across the storage defined for the Hadoop cluster.

Furthermore the foundation of Hadoop is the two core frameworks YARN and HDFS. These two frameworks deal with Processing and Storage.

The Guide Describe a System-Wide Installation with Root Privileges but You Can Easily Convert the Procedure to a Local One.

Finally the Tutorial’s Content and Details are Expressly Reduced to Give Focus to the Essentials Instructions and Commands.

How-to Install Hadoop on Fedora 26 - Featured
  1. First Open a Shell Terminal emulator window
    (Press “Enter” to Execute Commands)

    How-to Install Hadoop on Fedora 26 -  Kde5 Open  Terminal
  2. Download Latest Apache Hadoop Stable Release:

    Apache Hadoop Binary tar.gz

  3. Then Extract Into /tmp Directory
    Double-Click on Archive:

    How-to Install Hadoop on Fedora 26 - Unity Extract tar.gz Archive

    Or from Shell:

    tar xvzf *hadoop*tar.gz -C /tmp
  4. And Relocate Apache Hadoop Directory
    Get SuperUser Privileges (This simply to make shorter the command’s series):

    sudo su

    If Got “User is Not in Sudoers file” then Look: Solution
    Then Switch the contents with:

    mv /tmp/hadoop* /usr/local/

    Furthermore make an hadoop symlink directory:

    ln -s /usr/local/hadoop* /usr/local/hadoop
  5. Make Hadoop Temporary Directory:

    mkdir /usr/local/hadoop/tmp

    And Set the root as Owner:

    chown -R root:root /usr/local/hadoop*
  6. How-to Install Required Java JDK 7+ on Fedora:

    Install Oracle JDK 7+ for Fedora
  7. Set JAVA_HOME in Hadoop Env File
    First Make the Conf directory:

    mkdir /usr/local/hadoop/conf

    And then Make an Env file:

    nano /usr/local/hadoop/conf/hadoop-env.sh

    So now Append:

    export JAVA_HOME=/usr/lib/jvm/[oracleJdkVersion]

    Change [oracleJdkVersion] with the current Version:
    Ctrl+x to Save & Exit from nano Editor :)

  8. Eclipse Hadoop 2.X Integration with Free Plugin:

    Hadoop 2.X Eclipse Plugin SetUp
  9. Setup Hadoop Configuration for Pseudo-Distributed mode
    nano /usr/local/hadoop/conf/core-site.xml

    And Append:

     <?xml version="1.0"?>
     <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
     <configuration>
     <property>
     <name>hadoop.tmp.dir</name>
     <value>/usr/local/hadoop/tmp</value>
     </property>
     <property>
     <name>fs.default.name</name>
     <value>hdfs://localhost:8020</value>
     </property>
     </configuration>
    

    Next:

    nano /usr/local/hadoop/conf/hdfs-site.xml

    And Append:

     <?xml version="1.0"?>
     <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
     <configuration>
     <property>
     <name>dfs.replication</name>
     <value>1</value>
     </property>
     <property>
     <!-- specify this so that running 'hadoop namenode -format'
     formats the right dir -->
     <name>dfs.name.dir</name>
     <value>/usr/local/hadoop/cache/hadoop/dfs/name</value>
     </property>
     </configuration>
    

    Last:

    nano /usr/local/hadoop/conf/mapred-site.xml

    And Append:

     <?xml version="1.0"?>
     <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
     <configuration>
     <property>
     <name>mapred.job.tracker</name>
     <value>localhost:8021</value>
     </property>
     </configuration>
    
  10. Again SetUp Local Path & Environment
    Exit from SuperUser to the normal User:

    exit

    Then Change to the Home directory:

    cd $HOME

    And Edit the bash Config file:

    nano .bashrc

    So Inserts:

    HADOOP_HOME=/usr/local/hadoop
    export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
    export JAVA_HOME=/usr/lib/jvm/<oracleJdkVersion>

    Finally to Load the New Setup simply:

    bash
  11. Follow to SetUp Needed Local SSH Connection

    sudo su -c "openssh-server"

    Generate SSH Keys to Access:

    ssh-keygen -b 2048 -t rsa
    cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys

    Testing Connection:

    ssh 127.0.0.1
  12. Furthermore to Format the HDFS

    hadoop namenode -format

    How-to Install Hadoop on Fedora 26 - Terminal Apache Hadoop HDFS Formatting Succcess

  13. Finally to Start Up Hadoop Database

    start-all.sh
  14. Apache Hadoop Database Quick-Start Guide:

    Hadoop MapReduce Quick-Start
(Visited 114 times, 1 visits today)
Share on Tumblr

Tags: , , , , , , , , , , , , , , , , ,


Comments are disabled