teradata export from hive to teradata class not found - teradata

Question

I am trying to export from hive table to teradata using TDCH connector, I am getting below errror:-
15/05/07 08:01:03 INFO tool.ConnectorExportTool: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/MetaException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at com.teradata.connector.common.tool.ConnectorJobRunner.runJob(ConnectorJobRunner.java:81)
at com.teradata.connector.common.tool.ConnectorExportTool.run(ConnectorExportTool.java:61)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at com.teradata.hadoop.tool.TeradataExportTool.main(TeradataExportTool.java:24)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.metastore.api.MetaException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 13 more
I understand from error that hive-metastore jar is missing. But it is already there in hive/lib folder
hive-metastore.jar -> hive-metastore-0.9.0.jar
Is already present in the path: /usr/hdp/2.2.4.2-2/hive/lib

Answer 1

You have to define environment variables LIB_JARS and HADOOP_CLASSPATH.
You can use the environment variable LIB_JARS with the parameter -libjars
You can find an example under https://developer.teradata.com/sites/all/files/Teradata%20Connector%20for%20Hadoop%20Tutorial%20v1%200%20final.pdf.
According to the README, following jars are necessairy:
Hive Job(version 0.11.0 as example):
a) hive-metastore-0.11.0.jar
b) hive-exec-0.11.0.jar
c) hive-cli-0.11.0.jar
d) libthrift-0.9.0.jar
e) libfb303-0.9.0.jar
f) jdo2-api-2.3-ec.jar
g) slf4j-api-1.6.1.jar
h) datanucleus-core-3.0.9.jar
i) datanucleus-rdbms-3.0.8.jar
j) commons-dbcp-1.4.jar
k) commons-pool-1.5.4.jar
l) antlr-runtime-3.4.jar
m) datanucleus-api-jdo-3.0.7.jar
HCatalog Job:
a) above Hive required jar files
b) hcatalog-core-0.11.0.jar

Answer 2

The problem is that org/apache/hadoop/hive/metastore/api/MetaException class is not available to java runtime. can you ensure that the Jar is under valid runtime classpath.
few try outs can be try passing the classpath explicitly to jvm via -cp , alternatively you can try putting the jar in launch directory to ensure its present.

Answer 3

Sometimes you may have all the jar files in your "lib" folder, but for some reason Oozie may not be reading it simply because you have a typo in your job.properties or coordinator.properties. Double check your properties file.
The "job.properties" file should read like the following -
oozie.use.system.libpath=true
oozie.wf.application.path=${nameNode}/apps/myapp/workflow/
The "coordinator.properties" file should read like the following -
oozie.use.system.libpath=true
oozie.coord.application.path=${nameNode}/apps/myapp/workflow/
and your "lib" folder with jars should be at -
/apps/myapp/workflow/lib