Run Map Reduce application from Eclipse

Run a Map/Reduce project from on the configured Eclipse in previous blog.

1, create project
"File" > "New Project" > "Map/Reduce Project"
Assume project name as "ttt".
You can see your project structure like following.

2, create package "com.hadoop.test" under "src" 

3, copy <PiEstimator.java> under package "com.hadoop.test" in the project

4, modify package declaration of of <PiEstimator.java> from
"package org.apache.hadoop.examples;"
to
"package com.hadoop.test;"


5, modify "Run Configuration" of this project
Since the Pi estimator application requires 2 arguments, add 2 arguments "10 10000" in the tab "Arguments"

6, Run the application

Get following print out in "Console" tab.

Number of Maps  = 10
Samples per Map = 10000
12/04/19 15:24:10 INFO security.UserGroupInformation: JAAS Configuration already set up for Hadoop, not re-installing.
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
...
...
Job Finished in 7.448 seconds
Estimated value of Pi is 3.14120000000000000000

7, Extra:
a) invoke HDFS API to show content of a file in HDFS

String uri = "hdfs://10.1.2.124/user/td.txt";
 Configuration conf = new Configuration();
 FileSystem fs = FileSystem.get(URI.create(uri), conf);
 InputStream in = null;
 try{
 in = fs.open(new Path(uri));
 IOUtils.copyBytes(in, System.out, 4096, false);
 }finally{
 IOUtils.closeStream(in);
 }


b) invoke HBase API:
> Download HBase package, same as the runtime version.
e.g.http://archive.cloudera.com/cdh/3/hbase-0.90.4-cdh3u3.tar.gz
> add some libraries. In "project properties" > "Libraries" > "Add External JARs...", add at least HBASE_PKG/<hbase-0.90.4-cdh3u3.jar> and HBASE_PKG/lib/<zookeeper-3.3.4-cdh3u3.jar>.
Code slice, show properties of HBase table "test".


Configuration config = HBaseConfiguration.create();
    config.set("hbase.zookeeper.quorum", "10.1.2.124");
    
    String tablename = "test";
    HTable table = new HTable(config, tablename);
    Scan scan = new Scan();
    ResultScanner scanner = table.getScanner(scan);
    try{
    for(Result scannerResult: scanner)
    System.out.println("scan:" + scannerResult);
    }finally{
    scanner.close();
    }