Hive常用设置

2023年12月30日 270次阅读 来源: 小甜瓜Melon
  1. 切换计算引擎(Hive CLI内设置)
    设置MapReduce为计算引擎 set hive.execution.engine=mr;
    设置Spark为计算引擎 set hive.execution.engine=spark;
    设置Spark的master set spark.master=spark://master:7077
    设置Spark的master为YARN set spark.master=yarn
  1. hive中查看HDFS的位置
0: jdbc:hive2://secondnamenode-c1:10000/> show create table testdb.tttest;

INFO  : Compiling command(queryId=hive_20170721163434_87f6cb07-5fc2-4de7-9792-8ad8782f1fe4): show create table testdb.tttest
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:createtab_stmt, type:string, comment:from deserializer)], properties:null)
INFO  : Completed compiling command(queryId=hive_20170721163434_87f6cb07-5fc2-4de7-9792-8ad8782f1fe4); Time taken: 0.009 seconds
INFO  : Executing command(queryId=hive_20170721163434_87f6cb07-5fc2-4de7-9792-8ad8782f1fe4): show create table testdb.tttest
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20170721163434_87f6cb07-5fc2-4de7-9792-8ad8782f1fe4); Time taken: 0.008 seconds
INFO  : OK
+-----------------------------------------------------------------+--+
|                         createtab_stmt                          |
+-----------------------------------------------------------------+--+
| CREATE TABLE `testdb.tttest`(                                   |
|   `username` string,                                            |
|   `sex` string)                                                 |
| COMMENT 'Imported by sqoop on 2017/04/17 10:11:26'              |
| ROW FORMAT SERDE                                                |
|   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'          |
| WITH SERDEPROPERTIES (                                          |
|   'field.delim'='\t',                                           |
|   'line.delim'='\n',                                            |
|   'serialization.format'='\t')                                  |
| STORED AS INPUTFORMAT                                           |
|   'org.apache.hadoop.mapred.TextInputFormat'                    |
| OUTPUTFORMAT                                                    |
|   'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'  |
| LOCATION                                                        |
|   'hdfs://nameservice1/user/hive/warehouse/testdb.db/tttest'    |
| TBLPROPERTIES (                                                 |
|   'COLUMN_STATS_ACCURATE'='true',                               |
|   'numFiles'='1',                                               |
|   'numRows'='0',                                                |
|   'rawDataSize'='0',                                            |
|   'totalSize'='66',                                             |
|   'transient_lastDdlTime'='1492395090')                         |
+-----------------------------------------------------------------+--+
23 rows selected (0.039 seconds)

在hive命令行用dfs查看文件存储位置

0: jdbc:hive2://secondnamenode-c1:10000/> dfs -ls hdfs://nameservice1/user/hive/warehouse/testdb.db/tttest;

+-----------------------------------------------------------------------------------------------------------------------------+--+
|                                                         DFS Output                                                          |
+-----------------------------------------------------------------------------------------------------------------------------+--+
| Found 1 items                                                                                                               |
| -rwxrwxrwt   2 hdfs hive         66 2017-04-17 10:11 hdfs://nameservice1/user/hive/warehouse/testdb.db/tttest/part-m-00000  |
+-----------------------------------------------------------------------------------------------------------------------------+--+
2 rows selected (0.008 seconds)
0: jdbc:hive2://secondnamenode-c1:10000/>
    原文作者：小甜瓜Melon
    原文地址: https://www.jianshu.com/p/a5be537426d7
    本文转自网络文章，转载此文章仅为分享知识，如有侵权，请联系博主进行删除。
点赞