HiveServer2 HA模型部署测试

2023年10月20日 529次阅读来源: 分裂四人组

配置zookeeper，启动多个HiveServer

配置HA Zookeeper信息如下：

    <property>
      <name>hive.server2.transport.mode</name>
      <value>binary</value>
    </property>
    
    <property>
      <name>hive.server2.zookeeper.namespace</name>
      <value>hiveserver2-lsm</value>
    </property>

    <property>
      <name>hive.vectorized.execution.enabled</name>
      <value>true</value>
    </property>

    <property>
      <name>hive.zookeeper.quorum</name>
      <value>hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181</value>
    </property>

首先，在host1上启动MetaStore和HiveServer进程；
然后，在另外一台机器上启动HiveServer进程，组成简单的HA集群；

2017-06-27T10:31:18,263  INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/home/hzlishuming/env/apache-hive-2.1.1-bin
2017-06-27T10:31:18,264  INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181 sessionTimeout=1200000 watcher=org.apache.curator.ConnectionState@3f9b7fe1
2017-06-27T10:31:18,264  INFO [Thread-11] zookeeper.ZooKeeper: Initiating client connection, connectString=hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@47b17830
2017-06-27T10:31:18,295  INFO [main-SendThread(hadoop711.lt.163.org:2181)] zookeeper.Login: successfully logged in.
2017-06-27T10:31:18,296  INFO [Thread-12] zookeeper.Login: TGT refresh thread started.
2017-06-27T10:31:18,297  INFO [Thread-11-SendThread(hadoop710.lt.163.org:2181)] zookeeper.Login: successfully logged in.
2017-06-27T10:31:18,298  INFO [Thread-13] zookeeper.Login: TGT refresh thread started.
2017-06-27T10:31:18,301  INFO [main-SendThread(hadoop711.lt.163.org:2181)] client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism.
2017-06-27T10:31:18,301  INFO [Thread-11-SendThread(hadoop710.lt.163.org:2181)] client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism.
2017-06-27T10:31:18,304  INFO [Thread-11-SendThread(hadoop710.lt.163.org:2181)] zookeeper.ClientCnxn: Opening socket connection to server hadoop710.lt.163.org/10.120.219.54:2181. Will attempt to SASL-authenticate using Login Context section 'HiveZooKeeperClient'
2017-06-27T10:31:18,304  INFO [main-SendThread(hadoop711.lt.163.org:2181)] zookeeper.ClientCnxn: Opening socket connection to server hadoop711.lt.163.org/10.120.219.55:2181. Will attempt to SASL-authenticate using Login Context section 'HiveZooKeeperClient'
2017-06-27T10:31:18,304  INFO [Thread-11-SendThread(hadoop710.lt.163.org:2181)] zookeeper.ClientCnxn: Socket connection established to hadoop710.lt.163.org/10.120.219.54:2181, initiating session
2017-06-27T10:31:18,304  INFO [main-SendThread(hadoop711.lt.163.org:2181)] zookeeper.ClientCnxn: Socket connection established to hadoop711.lt.163.org/10.120.219.55:2181, initiating session

Beeline连接

./bin/beeline !connect jdbc:hive2://hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-lsm hadoop “”

beeline> !connect jdbc:hive2://hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-lsm hadoop ""
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hzlishuming/env/apache-hive-2.1.1-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/ndp/0.1.0/yarn_client/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-lsm
17/06/27 14:04:09 [main]: INFO jdbc.HiveConnection: Connected to hadoop692.lt.163.org:10000
Connected to: Apache Hive (version 2.1.1)
Driver: Hive JDBC (version 2.1.1)
17/06/27 14:04:09 [main]: WARN jdbc.HiveConnection: Request to set autoCommit to false; Hive does not support autoCommit=false.
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://hadoop710.lt.163.org:2181,had> show databases;
+--------------------+--+
|   database_name    |
+--------------------+--+
| default            |
| hive_examples      |
| tpcds_data_test_2  |
+--------------------+--+
3 rows selected (2.017 seconds)

Zookeeper状态

目前有两台机器组成的集群，查看Zookeeper中的状态信息：

[zk: hadoop712.lt.163.org(CONNECTED) 1] ls /hiveserver2-lsm

[serverUri=hadoop691.lt.163.org:10000;version=2.1.1;sequence=0000000003, serverUri=hadoop692.lt.163.org:10000;version=2.1.1;sequence=0000000002]

[zk: hadoop712.lt.163.org(CONNECTED) 2] get /hiveserver2-lsm/serverUri=hadoop691.lt.163.org:10000;version=2.1.1;sequence=0000000003

hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hadoop691.lt.163.org;hive.server2.thrift.port=10000;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_HOST@TEST.AMBARI.NETEASE.COM
cZxid = 0x70001d53d
ctime = Tue Jun 27 11:02:34 CST 2017
mZxid = 0x70001d53d
mtime = Tue Jun 27 11:02:34 CST 2017
pZxid = 0x70001d53d
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x25cc3695ce40f40
dataLength = 296
numChildren = 0

高可用性模拟

当前Session正连接与hadoop692.lt.163.org机器，现将该机器上的hiveserver进程kill掉，观察当前Session以及后续创建的Session会有如何影响？

当前Session处理会出现异常，如下：

0: jdbc:hive2://hadoop710.lt.163.org:2181,had> show databases;Unexpected end of file when reading from HS2 server. The root cause might be too many concurrent connections. Please ask the administrator to check the number of active connections, and adjust hive.server2.thrift.max.worker.threads if applicable.
Error: org.apache.thrift.transport.TTransportException (state=08S01,code=0)

再次创建Session，连接另外一台Host成功，符合预期：

hzlishuming@hadoop691:~/env/hive$ ./bin/beeline
Beeline version 2.1.1 by Apache Hive
beeline> !connect jdbc:hive2://hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-lsm hadoop ""
17/06/28 10:06:30 [main]: INFO jdbc.HiveConnection: Connected to hadoop691.lt.163.org:10000
Connected to: Apache Hive (version 2.1.1)
Driver: Hive JDBC (version 2.1.1)
17/06/28 10:06:30 [main]: WARN jdbc.HiveConnection: Request to set autoCommit to false; Hive does not support autoCommit=false.
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://hadoop710.lt.163.org:2181,had> show databases;

总结

HiveServer2 HA模型配置、实现较为简单，是基于负载均衡的方式实现路由高可用的，Zookeeper层保存各节点的Host+Port信息，在创建Session的时候随机从中选择一台Host进行连接；
当有机器挂掉时，会在Zookeeper中删除该节点，实现高可用性能；

参考：

原理：http://blog.csdn.net/wulantian/article/details/42418231
实现：http://blog.csdn.net/wulantian/article/details/42173095
HiveServer2 HA模式部署：https://toutiao.io/posts/8rllx9/preview

    原文作者：分裂四人组
    原文地址: https://www.jianshu.com/p/4e1345a659b8
    本文转自网络文章，转载此文章仅为分享知识，如有侵权，请联系博主进行删除。