Java.io.IOException: Job failed! when running a sample app on the osx with hadoop-0.19.1

hadoopioexceptionjava-6macos

bash-3.2$ echo $JAVA_HOME
/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
bash-3.2$ bin/hadoop dfs -copyFromLocal conf /user/yokkom/input2
bash-3.2$ bin/hadoop jar hadoop-*-examples.jar grep input2 output 'dfs[a-z.]+'
09/04/17 10:09:32 INFO mapred.FileInputFormat: Total input paths to process : 10
09/04/17 10:09:33 INFO mapred.JobClient: Running job: job_200904171309_0001
java.io.IOException: Job failed!
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)
        at org.apache.hadoop.examples.Grep.run(Grep.java:69)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.Grep.main(Grep.java:93)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:141)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:61)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
        at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)

Do anyone have any idea why this happens? The same job runs perfectly well on linux machines. And, after "Job failed" happens, the whole Hadoop cluster stops responding.

My MacOS version is 10.5.6.

EDIT The same result for hadoop-0.20.1

Best Answer

I googled for it and the same error (at the exact same line) has happened before. Read about it here and here.

The second link suggests the following:

  • Delete conf/log4j.properties to see a detailed report
  • Try to increase heap memory (using -Xms and -Xmx switches)
Related Topic