Issue when flink upload a job with stream sql query - sbt

Question

I used the latest flink version(1.10.0) and sbt(1.3.7). I have this exception when upload a job with streaming sql query:
Caused by: java.lang.ClassCastException:
org.codehaus.janino.CompilerFactory cannot be cast to
org.codehaus.commons.compiler.ICompilerFactory
at org.codehaus.commons.compiler.CompilerFactoryFactory.getCompilerFactory(CompilerFactoryFactory.java:129)
at org.codehaus.commons.compiler.CompilerFactoryFactory.getDefaultCompilerFactory(CompilerFactoryFactory.java:79)
at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:432)
When I running main class with sbt run it works perfectly. I made jar with the sbt assembly command and I have conflicts between libraries. For this reason add this in the build.sbt:
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs # _*) => MergeStrategy.discard
case x => MergeStrategy.first
}
I read a similar case with hive connector https://issues.apache.org/jira/browse/FLINK-14849 and this is the answer:
After https://issues.apache.org/jira/browse/FLINK-13749 , flink-client will use default child-first resolve-order.
If user jar has some conflict dependents, there will be some problem.
My question is: How to resolve these conflicts? Any assembly merge strategy suggest for this case?
Help would be appreciated.

Answer 1

I faced the same issue and the following solution helped me get through :
assembly / assemblyMergeStrategy := {
case PathList("org", "codehaus", "janino", "CompilerFactory.class") => MergeStrategy.discard // discard
case x => MergeStrategy.first // first
}
Also, took help from the this link.
The above addition needs to be done in sbt.build file and make sure you're building using assembly command.
In case one is building using package-bin in case of legacy applications, follow this link.
Another solution includes checking the scope of the flink libraries. If by any chance you have accidentally pushed libraries that were earlier set to 'provided' with something like 'compile', do revert the changes and test.
Hope this helps.

Answer 2

I have same problem.
Maybe you have package your code with flink-table-planner_${scala.binary.version}, so you need to change your maven config with those settings:
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-planner_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-planner-blink_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-planner_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>

Answer 3

I had the same issue. My problem was, that I added the dependency
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-planner_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
without the provided scope. Therefore, maven packaged it in my jar, which lead to the ClassCastException when trying to submit the job to my local flink cluster.