我在hadoop+zookeeper+hbase配置中出现的错误

最新推荐文章于 2024-12-20 16:25:21 发布

原创最新推荐文章于 2024-12-20 16:25:21 发布 · 3.1k 阅读

0 ·

本内容遵循CC 4.0 BY-SA版权协议

收录于

hadoop平台搭建

在配置Hadoop、Zookeeper和HBase的过程中遇到了错误。启动HBase时，由于Zookeeper已启动，导致Address already in use错误。解决方法是修改hbase-env.sh文件，将`export HBASE_MANAGES_ZK=true`更改为`export HBASE_MANAGES_ZK=false`或直接注释掉。在执行YCSB加载HBase时，出现了NoClassDefFoundError，原因是缺少hbase/lib目录下的jar文件。解决办法是将hbase/lib下的所有jar文件复制到ycsb/hbase-binding/lib目录。

1、启动hbase时候报错

master: java.net.BindException: Address already in use

master:         at sun.nio.ch.Net.bind(Native Method)
master:         at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:124)
master:         at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)
master:         at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:52)
master:         at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:111)
master:         at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:130)
master:         at org.apache.hadoop.hbase.zookeeper.HQuorumPeer.runZKServer(HQuorumPeer.java:73)

master: at org.apache.hadoop.hbase.zookeeper.HQuorumPeer.main(HQuorumPeer.java:63)

报错的原因是 zookeeper已经启动了。

纠正：修改将hbase-env.sh 文件中的

export HBASE_MANAGES_ZK=true

改为

export HBASE_MANAGES_ZK=false

或者直接注销掉

2、执行

./bin/ycsb load hbase -P workloads/workloada -p threads=500 -pcolumnfamily=f1 -p recordcount=10000000 -p hosts=$ip -s>load_hbaseA10G.dat

时报错

Exception in thread "Thread-2" java.lang.NoClassDefFoundError: org/apache/hadoop/util/PlatformName
   at org.apache.hadoop.security.UserGroupInformation.getOSLoginModuleName(UserGroupInformation.java:303)
   at org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:348)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.hbase.util.Methods.call(Methods.java:39)
   at org.apache.hadoop.hbase.security.User.call(User.java:414)
   at org.apache.hadoop.hbase.security.User.callStatic(User.java:404)
   at org.apache.hadoop.hbase.security.User.access$200(User.java:48)
   at org.apache.hadoop.hbase.security.User$SecureHadoopUser.<init>(User.java:221)
   at org.apache.hadoop.hbase.security.User$SecureHadoopUser.<init>(User.java:216)
   at org.apache.hadoop.hbase.security.User.getCurrent(User.java:139)
   at org.apache.hadoop.hbase.client.HConnectionKey.<init>(HConnectionKey.java:67)
   at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:240)
   at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:187)
   at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:149)
   at com.yahoo.ycsb.db.HBaseClient.getHTable(HBaseClient.java:118)
   at com.yahoo.ycsb.db.HBaseClient.update(HBaseClient.java:302)
   at com.yahoo.ycsb.db.HBaseClient.insert(HBaseClient.java:357)
   at com.yahoo.ycsb.DBWrapper.insert(DBWrapper.java:145)
   at com.yahoo.ycsb.workloads.CoreWorkload.doInsert(CoreWorkload.java:461)
   at com.yahoo.ycsb.ClientThread.run(Client.java:269)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.PlatformName