alluxio1.8.0版本集成hive metastor时存在连接耗时较长的问题，更换1.7.1版本后正常，安装步骤相同

官方安装文档：https://www.alluxio.org/docs/1.8/cn/Getting-Started.html

1、下载上传解压

地址：http://downloads.alluxio.org/downloads/files/1.8.0/alluxio-1.8.0-hadoop-2.7.tar.gz

tar -zxf alluxio-1.8.0-hadoop-2.7.tar.gz
cd alluxio-1.8.0-hadoop-2.7

2、配置alluxio-site.properties

cp conf/alluxio-site.properties.template conf/alluxio-site.propertiescat > conf/alluxio-site.properties << EOF

##关联hdfs配置文件
alluxio.underfs.hdfs.configuration=/etc/hadoop/conf/core-site.xml:/etc/hadoop/conf/hdfs-site.xml

##下面是高可用的配置
alluxio.master.hostname=xg-sensorsdata1.xcar.com.cn
alluxio.zookeeper.enabled=true
alluxio.zookeeper.address=xg-sensorsdata1.xcar.com.cn:2181,xg-sensorsdata2.xcar.com.cn:2181,xg-sensorsdata3.xcar.com.cn:2181

#配置日志文件目录
alluxio.master.journal.folder=hdfs://standyNamenode/alluxio/journal

##配置用户模拟，允许yarn用户模拟任意用户
alluxio.master.security.impersonation.yarn.users=*

3、配置集群woker节点

cat > conf /workers << EOF
xg-sensorsdata2.xcar.com.cn
xg-sensorsdata3.xcar.com.cn
xg-sensorsdata4.xcar.com.cn
EOF

4、修改日志路径

sed -i "s#\${alluxio.logs.dir}#\/var\/log\/alluxio#g" conf/log4j.properties`

5、分发安装包到woker节点

echo 'xg-sensorsdata2.xcar.com.cn
xg-sensorsdata3.xcar.com.cn
xg-sensorsdata4.xcar.com.cn' | xargs -I {} scp -r alluxio-1.8.0-hadoop-2.7 {}:/root/

集成hdfs

1、修改hdfs组件的core-site.xml，增加如下配置项

alluxio.zookeeper.address=xg-sensorsdata1.xcar.com.cn:2181,xg-sensorsdata2.xcar.com.cn:2181,xg-sensorsdata3.xcar.com.cn:2181
alluxio.zookeeper.enabled=true
fs.alluxio.impl=alluxio.hadoop.FileSystem 

2、分发客户端依赖包

cd /root/alluxio-1.8.0-hadoop-2.7/client
cp alluxio-1.8.0-client.jar /usr/hdp/current/hadoop-client/lib
echo 'xg-sensorsdata2.xcar.com.cn
xg-sensorsdata3.xcar.com.cn
xg-sensorsdata4.xcar.com.cn' | xargs -I {} scp alluxio-1.8.0-client.jar {}:/usr/hdp/current/hadoop-client/lib

3、重启影响的组件

集成tez（llap模式下跑不通）

cd /root/alluxio-1.8.0-hadoop-2.7/client
cp alluxio-1.8.0-client.jar /usr/hdp/current/tez-client/lib
echo 'xg-sensorsdata2.xcar.com.cn
xg-sensorsdata3.xcar.com.cn
xg-sensorsdata4.xcar.com.cn' | xargs -I {} scp alluxio-1.8.0-client.jar {}:/usr/hdp/current/tez-client/lib

修改tez-site.xml

tez.use.cluster.hadoop-libs=true ##此配置需要集群的运行hive tez任务的节点都安装了tez-client

集成hive

1、分发客户端依赖包

cd /root/alluxio-1.8.0-hadoop-2.7/client
cp alluxio-1.8.0-client.jar /usr/hdp/current/hive-server2-hive2/lib
echo 'xg-sensorsdata2.xcar.com.cn
xg-sensorsdata3.xcar.com.cn
xg-sensorsdata4.xcar.com.cn' | xargs -I {} scp alluxio-1.8.0-client.jar {}:/usr/hdp/current/hive-server2-hive2/lib

2、重启hive

集成spark

1、分发alluxio-client包

cd /root/alluxio-1.8.0-hadoop-2.7/client
cp alluxio-1.8.0-client.jar /usr/hdp/current/spark2-client/jars/
echo 'xg-sensorsdata2.xcar.com.cn
xg-sensorsdata3.xcar.com.cn
xg-sensorsdata4.xcar.com.cn' | xargs -I {} scp alluxio-1.8.0-client.jar {}:/usr/hdp/current/spark2-client/jars/

问题集：

1、ramfs is smaller than the configured size: ramfs size: 1024.00MB, configured size: 4096.00MB

alluxio无法动态修改ramfs的大小

Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalStateException: ramfs is smaller than the configured size: ramfs size: 1024.00MB, configured size: 4096.00MB
at alluxio.worker.AlluxioWorkerProcess.<init>(AlluxioWorkerProcess.java:176)
at alluxio.worker.WorkerProcess$Factory.create(WorkerProcess.java:45)
at alluxio.worker.WorkerProcess$Factory.create(WorkerProcess.java:37)
at alluxio.worker.AlluxioWorker.main(AlluxioWorker.java:55)
Caused by: java.lang.IllegalStateException: ramfs is smaller than the configured size: ramfs size: 1024.00MB, configured size: 4096.00MB
at alluxio.worker.block.meta.StorageTier.checkEnoughMemSpace(StorageTier.java:146)
at alluxio.worker.block.meta.StorageTier.initStorageTier(StorageTier.java:100)
at alluxio.worker.block.meta.StorageTier.newStorageTier(StorageTier.java:169)
at alluxio.worker.block.BlockMetadataManager.<init>(BlockMetadataManager.java:67)
at alluxio.worker.block.BlockMetadataManager.createBlockMetadataManager(BlockMetadataManager.java:82)
at alluxio.worker.block.TieredBlockStore.<init>(TieredBlockStore.java:128)
at alluxio.worker.block.DefaultBlockWorker.<init>(DefaultBlockWorker.java:129)
at alluxio.worker.block.BlockWorkerFactory.create(BlockWorkerFactory.java:43)
at alluxio.worker.block.BlockWorkerFactory.create(BlockWorkerFactory.java:26)
at alluxio.worker.AlluxioWorkerProcess$1.call(AlluxioWorkerProcess.java:129)
at alluxio.worker.AlluxioWorkerProcess$1.call(AlluxioWorkerProcess.java:125)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)