Flink 远程提交任务 java.lang.ClassCastException LinkedMap

问题来源

Flink本地提交任务运行正常，然后打包成jar在远程的Flink上运行失败。
注意：
打包的jar不能通过java -jar 运行，即使你的jar和Flink在一台机器上。

例如
Flink安装在E:\software\flink-1.7.2-bin-scala_2.12\flink-1.7.2，
jar放在E:\software\flink-1.7.2-bin-scala_2.12\flink-1.7.2\examples\streaming

需要先进入Windows cmd
E:\software\flink-1.7.2-bin-scala_2.12\flink-1.7.2\bin>
然后执行命令
flink.bat run …\examples\streaming\FlinkRemoteDemoJar-1.0-SNAPSHOT-jar-with-dependencies.jar --bootstrap.servers a.b.c.d:9092 --limitEnabled 2 --timeLimit 2 --group.id grp01 --nodeId a1b2c34

症状

无论是通过命令行还是web界面都报如下错误

Caused by: java.lang.ClassCastException: cannot assign instance of org.apache.commons.collections.map.LinkedMap to field org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.pendingOffsetsToCommit of type org.apache.commons.collections.map.LinkedMap in instance of org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer

也就是
“LinkedMap cannot be cast to LinkedMap exceptions”.

截图

解决办法

在c/onf/flink-conf.yaml 添加如下内容并重启 flink.

classloader.resolve-order: parent-first

本质原因

LinkedMap class is being loaded from two different packages, and those are being assigned to each other.

官方文档

https://ci.apache.org/projects/flink/flink-docs-release-1.8/monitoring/debugging_classloading.html