the SparkContext in SparkR (v1.5.1) is a
Java ref type org.apache.spark.api.java.JavaSparkContext
however when creating my class:
.jnew("com.example.MyClass","sc")
for my scala class: class TableReader(sc: JavaSparkContext), I'm getting a: java.lang.NoSuchMethodError:
What is this "Java ref type" and how can I get the actual context from it to send through rJava?
SparkR seems to have its own Java interoperability implemented in backend.R. Calls are made in the form SparkR:::newJObject(className, args), though I can't find any specific documentation, other than in tests in the same project.
sqlContext needs to be initialized and relevant jars loaded during startup using --jars {csv jars} or --packages as noted in the documentation.
Related
I have a Groovy script which specifies dependencies using the Grape #Grab annotation. Included with that are a dependency on spring-web to use RestTemplate, and a dependency on slf4j-nop to avoid the Failed to load class "org.slf4j.impl.StaticLoggerBinder" warning.
#!/usr/bin/env groovy
#Grab('org.springframework:spring-web:5.3.18')
#Grab('org.slf4j:slf4j-nop:1.7.36')
import org.springframework.web.client.RestTemplate
new RestTemplate().getForObject('http://www.example.com', String)
However, despite this, I am still getting the SLF4J warning:
$ ./restTemplateLoggingTest.groovy
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Given that it's a script, it's important that the output not contain extraneous noise, as it may be used and manipulated programmatically.
What can I do to prevent this logging warning from being output when I run my script?
Experimentation has shown that attaching the dependencies to the system classloader using #GrabConfig(systemClassLoader=true) causes the logs to no longer be emitted:
#!/usr/bin/env groovy
#GrabConfig(systemClassLoader=true)
#Grab('org.springframework:spring-web:5.3.18')
#Grab('org.slf4j:slf4j-nop:1.7.36')
import org.springframework.web.client.RestTemplate
new RestTemplate().getForObject('http://www.example.com', String)
I don't know for sure why this is the cause, though I have some vague guesses.
Note that despite addressing the issue, this isn't a use that's described by the Javadocs for GrabConfig#systemClassLoader:
Set to true if you want to use the system classloader when loading the grape. This is normally only required when a core Java class needs to reference the grabbed classes, e.g. for a database driver accessed using DriverManager.
I am using java1.6,grails 2.0. I want to process Docx documents.For this i have added jars poi-ooxml-3.5-FINAL.jar,ooxml-schemas-1.1.jar.
But in code line this.document = new XWPFDocument(buffInputStream);
i got below error
Class java.lang.ClassNotFoundException
Message org.apache.xmlbeans.XmlOptions
I am trying to add jars either xmlbeans-2.5.0.jar or xbean-2.1.0.jar.
I am getting below error
| Error Fatal error during compilation
org.apache.tools.ant.BuildException: java .lang.LinkageError: loader
constraint violation: loader (instance of bootloader)
previously initiated loading for a different type with name "org/w3c/dom/Type Info" (Use --stacktrace to see the full trace)
If i verify my Jre System Library, rt.jar contains this TypeInfo class. how
to solve this problem? Please advice. Thank you.
Hi,
I have tried removing library and used jarscan to search this class. But i didn't find any where. Please help out to figure out how else i can find where is this class and how to resolve this
This happened to me with the JMX MBeanServer class. Groovy was loading one of them with its classloader and so was the JVM.
In your case it sounds like the TypeInfo class exists in rt.jar just fine, but another class loader loads it from that jar (or another jar) again.
Running the JRE with "-verbose:class" showed that double load behavior. Then I moved the offending jar out of my classpath and it worked just fine with the one loaded by Groovy.
I would recommend moving rt.jar out of your classpath, or if you need other classes inside of rt.jar try unjaring and rejaring it without the Type class. That way the classloaders will be forced to use the ones in xmlbeans-2.5.0.jar or xbean-2.1.0.jar.
I have install Groovy 2.1.1 in my unix box. While executing groovysh getting the below error.
Exception in thread "main" java.lang.NoClassDefFoundError: error:
Caused by: java.lang.ClassNotFoundException: error:
at java.net.URLClassLoader.findClass(URLClassLoader.java:423)
at java.lang.ClassLoader.loadClass(ClassLoader.java:660)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:346)
at java.lang.ClassLoader.loadClass(ClassLoader.java:626)
Could not find the main class: error:. Program will exit.
What could be the possible root cause for this error...???
Somewhere, you try to use the class error: (and yes, Groovy thinks that the colon is part of the class name), either in the script you execute (i.e. there must be new error: somewhere) or you wrote something like groovysh error: or you import error: (maybe indirectly)
Since you are using IBM J9, according to the Grails FAQ, the J9 need an argument to work well with Groovy, otherwise you may get a NoClassDefFound error:
Add -Xverify:none to JVM arguments
Download Groovy Binary From http://groovy.codehaus.org/Download
Download zip: Binary Release
Extract Local Disk say D;\GROOVY\ groovy-2.3.9
It contains the Folder Structure
D:.
├───bin
├───conf
├───embeddable
├───indy
├───lib
└───META-INF
Go to Control Panel\User Accounts\User Accounts Change My Environment Variables
Set/new GROOVY_HOME = D:\GROOVY\groovy-2.3.9 (don’t put : semicolon)
Set PATH = C:\Program Files\Java\jdk1.8.0_25\bin;%GROOVY_HOME%\bin;
Add groovy-all.jar to CLASSPATH
D:\ GROOVY\ \groovy-2.3.9\embeddable\groovy-all-2.3.9.jar;.
Close and Open Command and Say –groovy
For console -groovyConsole
I used to be able to run my standalone java app that calls EJB and MQ using JNDI on RAD 7.5.5. I upgraded to RAD 8.5 and when I tried to run it, I received the error Failed to create InitialContext using factory specified in hashtable. I'm also using the same WAS 7.0.0.9 for both RAD version so I'm not sure why it's not working now.
Is there anything special that I need to do in RAD 8.5?
This is my bootstrap classpath entries: WAS v7.0 JRE, %WAS_RUNTIME%\com.ibm.ws.ejb.thinclient_7.0.0.jar, WAS v7.0 lib.
Below are the lines where I set up the environment (the second line seems to be needed for RAD 8.5):
System.setProperty(Context.PROVIDER_URL, "iiop://localhost:2809");
System.setProperty("com.ibm.ws.naming.wsn.factory.initial", "com.ibm.websphere.naming.WsnInitialContextFactory");
Below is the stack trace from my code:
2012-11-30 16:50:14,945 main FATAL ram.hs.edi.EDIDaemon(line 183) - Unable to get running status: javax.naming.NoInitialContextException: Failed to create InitialContext using factory specified in hashtable. Root exception is java.lang.NullPointerException
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:243)
at javax.naming.InitialContext.initializeDefaultInitCtx(InitialContext.java:327)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:357)
at javax.naming.InitialContext.internalInit(InitialContext.java:295)
at javax.naming.InitialContext.<init>(InitialContext.java:212)
at ram.hs.edi.utils.MQReader.open(MQReader.java:85)
at ram.hs.edi.EDIDaemon.isRunning(EDIDaemon.java:406)
at ram.hs.edi.EDIDaemon.run(EDIDaemon.java:168)
at ram.hs.edi.EDIDaemon.main(EDIDaemon.java:898)
Caused by: java.lang.NullPointerException
at com.ibm.ws.naming.util.CommonHelpers$2.run(CommonHelpers.java:268)
at com.ibm.ws.naming.util.CommonHelpers$2.run(CommonHelpers.java:264)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:63)
at com.ibm.ws.naming.util.CommonHelpers.mergeJndiProviderProperties(CommonHelpers.java:263)
at com.ibm.ws.naming.util.CommonHelpers.mergeWsnJndiProperties(CommonHelpers.java:159)
at com.ibm.websphere.naming.WsnInitialContextFactory.getInitialContext(WsnInitialContextFactory.java:186)
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:235)
... 8 more
Thanks
This may be little late answering it but please try to do the following:
1) Add the jar "com.ibm.ws.orb_7.0.0.jar" or equivalent jar on the Bootstrap path of your standalone Java Class.
2) Add the jar "com.ibm.ws.webservices.thinclient_7.0.0" or equivalent jar on the JVM Class path.
After doing the above I was able to get the InitalContext successfully. Hope this helps.
Java -version (Sco Openserver 5)
Java(TM) 2 Runtime Environment, Standard Edition (build SCO-UNIX-J2SE-1.3.1_22:*
FCS*:20080305)
Classic VM (build SCO-UNIX-J2SE-1.3.1_22:*FCS*:20080305, green threads, sunwjit)
Problem :
OracleDataSource ods = OracleDatasource();
It occurs error :
The type javax.sql.DataSource cannot be resolved. It is indirectly referenced from required .
class files
How to solve this problem?
You are using a wrong driver. Your driver requires java 1.4.
The javax.sql.DataSource has been added with java 1.4.
http://docs.oracle.com/javase/7/docs/api/javax/sql/DataSource.html
Make sure that you are using a JDBC driver that supports java 1.3.
http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html the classes12.jar should work for you.
This means that your program is missing a library on the class path.
You must make sure that, for example, when you export a compiled jar file, that you export the libraries along with it ...
Of course.... I'm assuming you were able to compile your program correctly.