CentOS 6.9 下GreenPlum源码编译安装及部署教程

2019年6月8日 279次阅读 来源: GanTa

《CentOS 6.9 下GreenPlum源码编译安装及部署教程》 image

GreenPlum介绍

PostgreSql

PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions. This distribution also contains C language bindings.

GreenPlum

The Greenplum Database (GPDB) is an advanced, fully featured, open source data warehouse. It provides powerful and rapid analytics on petabyte scale data volumes. Uniquely geared toward big data analytics, Greenplum Database is powered by the world’s most advanced cost-based query optimizer delivering high analytical query performance on large data volumes.
Greenplum 是基于 PostgreSQL 开发的，语法与 PostgreSQL 几乎一样。
本质上，Greenplum 是一个关系型数据库集群，由多个独立的数据库服务组 合成的逻辑数据库，对于用户来说 Greenplum 与 Oracle、PostgreSQL 并无不同， 都可以通过标准 SQL 语句对数据进行访问。

GreenPlum源码编译安装

以CentOS 6.9为例，下载地址为阿里云镜像。

CentOS-6.9-x86_64-bin-DVD1.iso
CentOS-6.9-x86_64-bin-DVD2.iso

强烈建议以gpadmin为用户名。

准备工作

添加用户root权限

vim 打开 /etc/sudoers 文件 （以root权限打开）
找到以下两行

## Allow root to run any commands anywhere
root    ALL=(ALL)       ALL

并在下一行添加

gpadmin ALL=(ALL)       ALL #注意对齐

保存并返回即可

安装gcc-c++

sudo yum install gcc-c++

安装git

sudo yum install git

安装cmake

cd /home/gpadmin
wget https://cmake.org/files/v3.5/cmake-3.5.2.tar.gz
tar -zxvf cmake-3.5.2.tar.gz
cd cmake-3.5.2
./configure --prefix=/usr/local
make
sudo make install

安装pip

sudo yum -y install epel-release
sudo yum -y install python-pip
pip install --upgrade #pip版本至少为 7.x.x

确保已安装必需的python模块

  • psutil
  • lockfile (>= 0.9.1)
  • paramiko
  • setuptools

确保动态链接库共享

添加 /usr/local/lib 和 /usr/local/lib64 到 /etc/ld.so.conf 中
并运行命令

sudo ldconfig

支持 c++11

sudo yum install -y centos-release-scl
sudo yum install -y devtoolset-6-toolchain
echo 'source scl_source enable devtoolset-6' >> ~/.bashrc
source ~/.bashrc

安装依赖

sudo yum -y install gcc git apr bison flex  readline gcc-c++ curl-devel bzip2-devel python-devel readline-devel apr-devel libevent-devel openssl-devel perl-ExtUtils-Embed libxml2-devel openldap-devel pam pam-devel

安装GPORCA

clone GPORCA

cd /home/gpadmin
git clone https://github.com/greenplum-db/gporca.git

安装 gp-xerces

git clone https://github.com/greenplum-db/gp-xerces.git
cd gp-xerces
mkdir build
cd build
../configure --prefix=/usr/local
make
sudo make install

安装ninja

cd /home/gpadmin
git clone git://github.com/ninja-build/ninja.git && cd ninja
./configure.py --bootstrap
sudo cp ninja /usr/bin

编译安装GPORCA

cd /home/gpadmin/gporca
cmake -GNinja -H. -Bbuild
sudo ninja install -C build

编译GreenPlum

cd /home/gpadmin
git clone https://github.com/greenplum-db/gpdb.git
cd gpdb
./configure --with-perl --with-python --with-libxml --with-gssapi --prefix=/usr/local/gpdb

# Compile and install
make -j8
sudo make -j8 install

# Bring in greenplum environment into your running shell
source /usr/local/gpdb/greenplum_path.sh

若出现

Greenplum Database installation complete.

则说明安装成功

部署

以在同一台主机上的一个master节点和一个segment节点为例

配置host文件

查看ip

ip addr show eth0
# 选择inet后的ip

修改hosts文件

sudo vim /etc/hosts

添加两行

<ip> mdw
<ip> sdw1
#此处ip相同

修改network文件

sudo vim /etc/sysconfig/network
#修改   HOSTNAME=mdw

创建主机列表文件和segment文件

cd /home/gpadmin
mkdir conf
cd conf
sudo vim hostlist

添加以下内容

mdw
sdw1

sudo vim seg_hosts

添加以下内容

sdw1

共享内存与网络参数配置

sudo vim /etc/sysctl.conf

对照替换成以下内容

net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 1
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.sem = 250 64000 100 512
kernel.shmmax = 500000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 64000 100 512
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.core.netdev_max_backlog = 10000
vm.overcommit_memory = 2
net.ipv4.conf.all.arp_filter = 1

sudo sysctl -p

加入限制参数

sudo vim /etc/security/limits.conf

添加以下内容

* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072
#  星号不能去掉

节点部署

环境变量设置

source /usr/local/gpdb/greenplum_path.sh

配置免密码登陆

sudo pip install psutil
gpssh-exkeys -f /home/gpadmin/conf/hostlist

创建数据目录

cd /home/gpadmin
mkdir gpdata
cd gpdata
mkdir segmentdata1 masterdata

环境变量配置

vim ~/.bash_profile

添加以下内容

source /usr/local/gpdb/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/home/gpadmin/gpdata/masterdata/gpseg-1
export PGPORT=2345
export PGDATABASE=testDB

source ~/.bash_profile

编写数据库启动参数文件

cd /home/gpadmin/conf
mkdir gpconfigs
cd gpconfigs
vim gpinitsystem_config

添加以下内容

#启动参数文件内容
ARRAY_NAME="Greenplum"
SEG_PREFIX=gpseg
PORT_BASE=40000
declare -a DATA_DIRECTORY=(/home/gpadmin/gpdata/segmentdata1 /home/gpadmin/gpdata/segmentdata1)
MASTER_HOSTNAME=mdw
MASTER_DIRECTORY=/home/gpadmin/gpdata/masterdata
##### Port number for the master instance.
MASTER_PORT=2345
# #### Shell utility used to connect to remote hosts.
TRUSTED_SHELL=/usr/bin/ssh
##### Maximum log file segments between automatic WAL checkpoints.
CHECK_POINT_SEGMENTS=8
ENCODING=UNICODE
#MIRROR_PORT_BASE=50000
REPLICATION_PORT_BASE=41000
#MIRROR_REPLICATION_PORT_BASE=51000
#declare -a MIRROR_DATA_DIRECTORY=(/home/gpadmin/gpdata/segmentmirror1 #/home/gpadmin/gpdata/segmentmirror)
MACHINE_LIST_FILE=/home/gpadmin/conf/seg_hosts

初始化数据库

gpinitsystem -c /home/gpadmin/conf/gpconfigs/gpinitsystem_config –a

数据库使用

gpstart #开启数据库服务
gpstop #关闭数据库服务
gprecoverseg #恢复启动失败的节点
gpstate -m #查看节点状态
createdb testDB #创建数据库
psql #连接数据库 psql –help查看更多选项

TPC-H测试

运行Greenplum TPC-H测试
参考https://yq.aliyun.com/articles/93?commentId=29

如有兴趣，请访问个人博客YuanZi’s Blog

    原文作者：GanTa
    原文地址: https://www.jianshu.com/p/724353932655
    本文转自网络文章，转载此文章仅为分享知识，如有侵权，请联系博主进行删除。
点赞