Hive用户认证配置流程

  1. 首先确保Hadoop环境和Hive环境兼容且配置好,我这里Hive的元数据库使用的MySQL。

  2. 编辑$HIVE_HOME/conf/hive-site.xml,对server2适当配置并启动server。
    使用server2是因为HiveServer2支持多客户端的并发和认证,为开放API客户端如JDBC、ODBC提供更好的支持。server/server2区别以及配置
    当然也可以不配置server2相关内容,保留默认配置即可,需注意默认tcp port is 10000(default http port is 10001)
    启动server2:

     hive --service hiveserver2 &
    

    使用beeline连接,在没有配置用户的情况下任何人都可以连接,密码为空:

     $ beeline
     Beeline version 1.2.1.spark2 by Apache Hive
     beeline> !connect jdbc:hive2://localhost:10000 hadoop
     Connecting to jdbc:hive2://localhost:10000
     Enter password for jdbc:hive2://localhost:10000: 
     SLF4J: Class path contains multiple SLF4J bindings.
     SLF4J: Found binding in [jar:file:/usr/local/lib/spark-2.2.0/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
     SLF4J: Found binding in [jar:file:/usr/local/lib/hadoop-2.8.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
     SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
     SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
     Connected to: Apache Hive (version 2.3.0)
     Driver: Hive JDBC (version 1.2.1.spark2)
     Transaction isolation: TRANSACTION_REPEATABLE_READ
    
  3. 编码完成CustomHiveServer2Auth类,以及Authentication file
    authentication file是保存明文用户名和MD5加密密码的文本文件,如下所示。该文件需要放置在server中,我这里放在$HIVE_HOME/conf下并命名为hive.server2.users.conf了。

     opticalix,88b6ed7f53340e0d45ff60f277e4c420
     other_username,88b6ed7f53340e0d45ff60f277e4c421
    

    CustomHiveServer2Auth类实现如下,参考博文见此

     package com.opticalix;
     
     import com.opticalix.utils.MD5Utils;
     import org.apache.hadoop.conf.Configuration;
     import org.apache.hadoop.hive.conf.HiveConf;
     import org.apache.hive.service.auth.PasswdAuthenticationProvider;
     
     import javax.security.sasl.AuthenticationException;
     import java.io.BufferedReader;
     import java.io.File;
     import java.io.FileReader;
     import java.io.IOException;
     
     /**
      * 根据用户名密码文件,遍历看是否能匹配上一个用户。如果不能则抛出异常
      */
     public class CustomHiveServer2Auth implements PasswdAuthenticationProvider {
         public static final String HIVE_SERVER2_CUSTOM_AUTHENTICATION_FILE = "hive.server2.custom.authentication.file";
     
         @Override
         public void Authenticate(String username, String password) throws AuthenticationException {
             boolean ok = false;
             String passMd5 = new MD5Utils().md5(password);
             //load hive conf, get auth file location
             HiveConf hiveConf = new HiveConf();
             Configuration conf = new Configuration(hiveConf);
             String filePath = conf.get(HIVE_SERVER2_CUSTOM_AUTHENTICATION_FILE);
             System.out.println("hive.server2.custom.authentication.file [" + filePath + "] ..");
             //try match
             File file = new File(filePath);
             BufferedReader reader = null;
             try {
                 reader = new BufferedReader(new FileReader(file));
                 String tempString = null;
                 while ((tempString = reader.readLine()) != null) {
                     System.out.println("read line=" + tempString);
                     String[] split = tempString.split(",", -1);
                     if (split.length != 2) continue;
                     //ok
                     if (split[0].equals(username) && split[1].equals(passMd5)) {
                         ok = true;
                         break;
                     }
                 }
                 reader.close();
             } catch (Exception e) {
                 e.printStackTrace();
             } finally {
                 if (reader != null) {
                     try {
                         reader.close();
                     } catch (IOException ignored) {
                     }
                 }
             }
     
             //throw exception if not matched
             if (ok) {
                 System.out.println("user [" + username + "] auth check ok .. ");
             } else {
                 System.out.println("user [" + username + "] auth check fail .. ");
                 throw new AuthenticationException("user [" + username + "] auth check fail .. ");
             }
         }
     }
    

    编码时需要添加必要的dependency,maven配置如下,版本号需要根据自己的情况而定:

     <dependency>
         <groupId>org.apache.hive</groupId>
         <artifactId>hive-jdbc</artifactId>
         <version>2.3.0</version>
     </dependency>
     <dependency>
         <groupId>org.apache.hive</groupId>
         <artifactId>hive-metastore</artifactId>
         <version>2.3.0</version>
     </dependency>
     <dependency>
         <groupId>org.apache.hive</groupId>
         <artifactId>hive-exec</artifactId>
         <version>2.3.0</version>
     </dependency>
     <dependency>
         <groupId>org.apache.hive</groupId>
         <artifactId>hive-service</artifactId>
         <version>2.3.0</version>
     </dependency>
     <dependency>
         <groupId>org.apache.hadoop</groupId>
         <artifactId>hadoop-common</artifactId>
         <version>2.8.1</version>
     </dependency>
    
  4. 编辑$HIVE_HOME/conf/hive-site.xml,增加或修改以下property。
    其中authentication file和authentication class要根据自己的情况修改。

     <property>
       <name>hive.server2.authentication</name>
       <value>CUSTOM</value>
       <description>
         Expects one of [nosasl, none, ldap, kerberos, pam, custom].
         Client authentication types.
           NONE: no authentication check
           LDAP: LDAP/AD based authentication
           KERBEROS: Kerberos/GSSAPI authentication
           CUSTOM: Custom authentication provider
                   (Use with property hive.server2.custom.authentication.class)
           PAM: Pluggable authentication module
           NOSASL:  Raw transport
       </description>
     </property>
     <property>
       <name>hive.server2.custom.authentication.class</name>
       <value>com.opticalix.CustomHiveServer2Auth</value>
       <description>
         Custom authentication class. Used when property
         'hive.server2.authentication' is set to 'CUSTOM'. Provided class
         must be a proper implementation of the interface
         org.apache.hive.service.auth.PasswdAuthenticationProvider. HiveServer2
         will call its Authenticate(user, passed) method to authenticate requests.
         The implementation may optionally implement Hadoop's
         org.apache.hadoop.conf.Configurable class to grab Hive's Configuration object.
       </description>
     </property>
     <property>
       <name>hive.server2.custom.authentication.file</name>
       <value>/usr/local/lib/hive-2.3.0/conf/hive.server2.users.conf</value>
     </property>
    
  5. 打jar包。不需要dependencies一同打进jar。这里利用maven plugin: maven-assembly-plugin(Jar打包博文见此)。添加该plugin后执行命令mvn assembly:assembly即可在target中得到jar。然后
    利用scp命令将jar放在集群的$HIVE_HOME/lib,这样在hive启动时会自动加载lib中的jars。

  6. 重新启动hive-server2(如果刚才的hive server进程未结束,可以用kill -9 $pid来结束进程),并使用beeline连接,此时用刚才设置的用户名/密码才能连接成功:

     beeline> !connect jdbc:hive2://localhost:10000
     Connecting to jdbc:hive2://localhost:10000
     Enter username for jdbc:hive2://localhost:10000: opticalix
     Enter password for jdbc:hive2://localhost:10000: **************
     ...
     Connected to: Apache Hive (version 2.3.0)
     Driver: Hive JDBC (version 1.2.1.spark2)
     Transaction isolation: TRANSACTION_REPEATABLE_READ
    

    Server控制台也会输出log:

     hive.server2.custom.authentication.file [/usr/local/lib/hive-2.3.0/conf/hive.server2.users.conf] ..
     read line=opticalix,88b6ed7f53340e0d45ff60f277e4c420
     user [opticalix] auth check ok ..
    
    原文作者:Opticalix
    原文地址: https://www.jianshu.com/p/8fdf13e60edd
    本文转自网络文章,转载此文章仅为分享知识,如有侵权,请联系博主进行删除。
点赞