I am getting the below error: when I try to execute mapreduce job written in Python.. I am not able to locate the streaming*.jar... Please suggest how this issue can be solved..
Can you please also guide me in writing bash file for the below commands..
hduser#hadoop-PC:~/hadoop/contrib$ hadoop jar contrib/streaming/hadoop-*streaming*.jar -file /home/hduser/mapper.py -mapper /home/hduser/mapper.py -file /home/hduser/reducer.py -reducer /home/hduser/reducer.py -input /user/hduser/gutenberg/* -output /user/hduser/gutenberg-output
Warning: $HADOOP_HOME is deprecated.
Exception in thread "main" java.io.IOException: Error opening job jar: contrib/streaming/hadoop-*streaming*.jar
at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
Caused by: java.io.FileNotFoundException: contrib/streaming/hadoop-*streaming*.jar (No such file or directory)
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.<init>(ZipFile.java:215)
at java.util.zip.ZipFile.<init>(ZipFile.java:145)
at java.util.jar.JarFile.<init>(JarFile.java:153)
at java.util.jar.JarFile.<init>(JarFile.java:90)
at org.apache.hadoop.util.RunJar.main(RunJar.java:88)
Well, I think, not hadoop-streaming.jar, it should be hadoop-streaming-1.1.2.jar, so try this:
hduser#hadoop-PC:~/hadoop$./bin/hadoop jar contrib/streaming/hadoop-streaming-1.1.2.jar** -file /home/hduser/mapper.py -mapper /home/hduser/mapper.py -file /home/hduser/reducer.py -reducer /home/hduser/reducer.py -input /user/hduser/gutenberg/* -output /user/hduser/gutenberg-output
Um, dumb question but did you verify that A) the file exists and B) you have permissions to access it? Also, try accessing the file like ./contrib/streaming/hadoop-streaming.jar or even the fully qualified path to it.
I just noticed that you were running your code from ~/hadoop/contrib. You need to take off the leading contrib from you paths
Related
I'm trying to run an application using the command prompt. It's built in Eclipse, using JavaFx. It's a digital diary program that I've been writing for a while and would like to send it to some friends. It is exported with packaged libraries as a runnable jar file.
I'm using the line
java -jar -Djavafx.verbose=true main.jar --module-path /users/home/javafx-sdk-11.0.2/lib --add-modules javafx.controls,javafx.fxml
to run it.
I've traced the errors (through use of -Djavafx.verbose=true) to java.lang.UnsatisfiedLinkError.
One of them is as follows:
WARNING: java.lang.UnsatisfiedLinkError: Invalid URL for class: jar:rsrc:javafx.graphics.jar!/com/sun/glass/utils/NativeLibLoader.class
What's causing this error?
EDIT after moving the --add-modules before main.jar, I get some new errors.
Exception in thread "main" java.lang.ExceptionInInitializerError
...
Caused by: java.lang.IllegalStateException: This operation is permitted on the event thread only; currentThread = main
...
When specifying arguments for the java virtual machine being run with a -jar command, these arguments need to go before the jar file name in the command line.
Do not use this
It will treat the module arguments as program arguments rather than VM arguments:
java -jar -Djavafx.verbose=true main.jar --module-path /users/home/javafx-sdk-11.0.2/lib --add-modules javafx.controls,javafx.fxml
Instead, use this
java --module-path /users/home/javafx-sdk-11.0.2/lib --add-modules javafx.controls,javafx.fxml -Djavafx.verbose=true -jar main.jar
I tried to install sbt on high sierra,
I have installed the build tool using : "brew install sbt" , after that i tried to run the sbt on "./.brew/Cellar/sbt/1.1.0/bin/sbt
i got that error.
java.io.IOException: Operation not supported
at sun.nio.ch.FileDispatcherImpl.lock0(Native Method)
at sun.nio.ch.FileDispatcherImpl.lock(FileDispatcherImpl.java:90)
at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1115)
at java.nio.channels.FileChannel.tryLock(FileChannel.java:1155)
at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:88)
at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:80)
at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:99)
at xsbt.boot.Using$.withResource(Using.scala:10)
at xsbt.boot.Using$.apply(Using.scala:9)
at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:60)
at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
at xsbt.boot.Locks$.apply0(Locks.scala:31)
at xsbt.boot.Locks$.apply(Locks.scala:28)
at xsbt.boot.Launch.locked(Launch.scala:238)
at xsbt.boot.Launch.app(Launch.scala:147)
at xsbt.boot.Launch.app(Launch.scala:145)
at xsbt.boot.Launch$.run(Launch.scala:102)
at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
at xsbt.boot.Launch$.launch(Launch.scala:117)
at xsbt.boot.Launch$.apply(Launch.scala:18)
at xsbt.boot.Boot$.runImpl(Boot.scala:41)
at xsbt.boot.Boot$.main(Boot.scala:17)
at xsbt.boot.Boot.main(Boot.scala)
Error during sbt execution: java.io.IOException: Operation not supported
You actually don't need homebrew to install SBT. Download it directly at https://www.scala-sbt.org/download.html, then extract and drop it into your home directory.
With Terminal, update permissions:
chmod -R 700 sbt/bin/sb
Open the Bash Profile.
nano .bash_profile
Add the following line to the end of the file:
export PATH=/Users/username/sbt/bin:$PATH
Username is your user name. For example, export PATH=/Users/john/sbt/bin:$PATH
To test, close and re-open Terminal. Run:
sbt --version
I wrote step-by-step instructions on my blog earlier.
openmeetings-2.1.1 in my system,i have install ghostscript,ImageMagick-6.8.9 and swftools, when i try to upload document file like .ppt,.doc file, it upload successfully but when i drag to white board it give me error like below.
key: processSWF
process: generateSwf
command; [pdf2swf, -s, insertstop, -s, poly2bitmap, -i, -j, 85, -s, zoom=72,
/home/PROJECTNAME/web/PROJECTNAME/FOLDER/apache-
openmeetings/webapps/openmeetings/upload/files/
be44f8c982e2ca52610b691c3dbda38e/be44f8c982e2c
a52610b691c3dbda38e.pdf,
/home/PROJECTNAME/web/FOLDER/public_html/apache-
openmeetings/webapps/openmeetings/upload/files/
be44f8c982e2ca52610b691c3dbda38e/be44f8c982e2
ca52610b691c3dbda38e.swf]
exception: java.io.IOException: Cannot run program "pdf2swf": error=2, No such file or
directory
error: Cannot run program "pdf2swf": error=2, No such file or directory
exitValue: java.io.IOException: Cannot run program "pdf2swf": error=2, No such file or
directory
i got this error in dialog box.i have Google a lot, kindly help me to figure out this problem.
thank you.
You Have to edit the "swftools_path" in configuration!
If you don't have swftools you should install version 2012-10-15-1307, the newest version doesn't work!
I have OutOfMemoryError when I compile my project using sbt. So I need to enlarge the java compiler heap space. I use java 6. And I have set JAVA_HOME to java 6 in my bash env.
I tried to add this line to build.sbt
javacOptions ++= Seq(<other arguments>,"-J-Xmx1028m")
When I compile, I have error like this:
[error] javac: invalid flag: -J-Xmx1028m
[error] Usage: javac <options> <source files>
[error] use -help for a list of possible options
The funny thing is: this error disappears when I specify the javaHome in build.sbt, which I have already done it in my bash env:
javaHome := Some(file("path to java 6"))
Does it mean that I have to specify JAVA_HOME twice? once in bash env, once in build.sbt? I don't want to specify a absolute path in build.sbt, which makes it may not work properly on other computers.
I have built a Camel project in Eclipse with Maven dependencies.
It ran successfully and also Built the Jar file and ran it from the command prompt
which is running as required. But when I moved the JAR file onto to our Linux machine
which is like a Job Manager server and when I try to run the JAR file as below
I am getting the below error message.
When I try to run with the below command
$ java –jar mycamelproject
I am getting the below error, but I do have the below mentioned dependency in the Dependency-Jars folder.
Exception in thread "main" java.lang.NoClassDefFoundError: org/springframework/c
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
at java.lang.Class.getMethod0(Class.java:2774)
at java.lang.Class.getMethod(Class.java:1663)
at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: org.springframework.context.support
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 6 more
Then I tried running with the below command.
$ mvn -X exec:java -Dexec.mainClass=mycamelpackage.mycamelmainclass
I am getting a series of the below errors such as below
[DEBUG] Could not find metadata org.codehaus.mojo/maven-metadata.xml in local (/home/ec2-user/.m2/repository)
[DEBUG] Skipped remote update check for org.codehaus.mojo/maven-metadata.xml, already updated during this session.
[WARNING] Failure to transfer org.codehaus.mojo/maven-metadata.xml from
http://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced. Original error: Could not transfer metadata org.codehaus.mojo/maven-metadata.xml from/to central (http://repo.maven.apache.org/maven2): proxy.host.net
[ERROR] No plugin found for prefix 'exec' in the current project
and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo]
available from the repositories
How should java –jar mycamelproject work if no classpath is set? And mycamelproject should be a JAR file anyway (and if it is, you should add the *.jar extension).
Beside that, I guess your local Maven repository may be corrupt. That's usually the reason why you see the resolution will not be reattempted error message. The easiest way to fix this is to remove the corrupt directories and start Maven again.