问题背景
解决完FileNotExsit的问题后,重新Build Cube,发现在fact table distinct阶段还是报错,错误如下:
错误
java.lang.NoSuchMethodError: org.apache.hadoop.yarn.proto.YarnProtos$LocalResourceProto.hashLong(J)I
at org.apache.hadoop.yarn.proto.YarnProtos$LocalResourceProto.hashCode(YarnProtos.java:11864)
at org.apache.hadoop.yarn.api.records.impl.pb.LocalResourcePBImpl.hashCode(LocalResourcePBImpl.java:62)
at java.util.HashMap.hash(HashMap.java:338)
at java.util.HashMap.put(HashMap.java:611)
at org.apache.hadoop.mapred.LocalDistributedCacheManager.setup(LocalDistributedCacheManager.java:133)
at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:163)
at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:731)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:240)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.kylin.engine.mr.common.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:149)
at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:108)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
问题定位
从异常来看,应该是类的方法找不到,查看源码,找到LocalResourceProto,发现方法hashLong(long):int是来自其子类AbstractMessage中
/**
* Helper method for implementing {@link Message#hashCode()}.
* @see Boolean#hashCode()
*/
protected static int hashLong(long n) {
return (int) (n ^ (n >>> 32));
}
奇怪了,明显是存在,为什么还会出下NoSuchMethodError?�想了一下,可能原因是版本冲突,修改kylin的启动参数,在启动参数中新增-versbose:class
,查看类是从哪个jar包中加载的。修改${KYLIN_HOME}/bin下面的setenv.sh脚本。
export KYLIN_EXTRA_START_OPTS="-verbose:class"
重启程序后查看类加载日志,截取的日志如下
kylin.out:[Loaded org.apache.logging.log4j.message.AbstractMessageFactory from file:/usr/local/apache-hive-2.1.1-bin/lib/log4j-api-2.4.1.jar]
kylin.out:[Loaded com.google.protobuf.AbstractMessageLite from file:/usr/local/apache-kylin-1.6.0-hbase1.x-bin/lib/kylin-jdbc-1.6.0.jar]
kylin.out:[Loaded com.google.protobuf.AbstractMessage from file:/usr/local/apache-kylin-1.6.0-hbase1.x-bin/lib/kylin-jdbc-1.6.0.jar]
kylin.out:[Loaded com.google.protobuf.AbstractMessageLite$Builder$LimitedInputStream from file:/usr/local/apache-kylin-1.6.0-hbase1.x-bin/lib/kylin-jdbc-1.6.0.jar]
kylin.out:[Loaded com.google.protobuf.AbstractMessageLite$Builder from file:/usr/local/apache-kylin-1.6.0-hbase1.x-bin/lib/kylin-jdbc-1.6.0.jar]
kylin.out:[Loaded com.google.protobuf.AbstractMessage$Builder from file:/usr/local/apache-kylin-1.6.0-hbase1.x-bin/lib/kylin-jdbc-1.6.0.jar]
发现类AbstractMessage是从包/usr/local/apache-kylin-1.6.0-hbase1.x-bin/lib/kylin-jdbc-1.6.0.jar加载的,而不是加载的hadoop classpath下的protobuf-java-2.5.0,由于查看了下kylin-jdbc工程的pom依赖
[INFO] --- maven-dependency-plugin:2.10:tree (default-cli) @ kylin-jdbc ---
[INFO] org.apache.kylin:kylin-jdbc:jar:1.6.0
[INFO] +- org.apache.calcite.avatica:avatica:jar:1.8.0:compile
[INFO] | +- org.apache.calcite.avatica:avatica-metrics:jar:1.8.0:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.6.3:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.6.3:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.2.4:compile
[INFO] | +- com.google.protobuf:protobuf-java:jar:3.0.0-beta-1:compile
[INFO] | \- org.apache.httpcomponents:httpcore:jar:4.4.4:compile
[INFO] +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile
[INFO] | +- commons-logging:commons-logging:jar:1.1.1:compile
[INFO] | \- commons-codec:commons-codec:jar:1.6:compile
[INFO] +- junit:junit:jar:4.12:test
[INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO] +- log4j:log4j:jar:1.2.17:provided
[INFO] +- org.slf4j:slf4j-log4j12:jar:1.7.21:provided
[INFO] +- org.slf4j:jcl-over-slf4j:jar:1.7.21:compile
[INFO] \- org.slf4j:slf4j-api:jar:1.7.21:compile
其依赖的是protobuf的3.0.0-beta-1版本,和hadoop中proto编译时的版本冲突,于是一种解决方式时先暂时删除kylin-jdbc-1.6.0.jar中protobuf的相关包
jar xvf kylin-jdbc-1.6.0.jar
rm -rf com/google/protobuf/*
rm -rf kylin-jdbc-1.6.0.jar
jar cvf kylin-jdbc-1.6.0.jar *
遗留问题
kylin依赖了Apache Calcite包,而它依赖protobuf-3.0.0-beta-1,如果按照上述的方式修改,如果calcite依赖了3.0.0的新特性,则也会出现相关的错误;一种解决方式是找一个兼容双方的版本;但这种方式很难,比如Hadoop版本可能是公司级别,很难统一处理;另外一种方式是通过maven-shade-plugin来进行相关处理,为不同版本的包加一个前缀。