bucardo是PostgreSQL数据库中实现双向同步的软件,可以实现PostgreSQL数据库的双master的方案,不过bucardo中的同步都是异步的,它是通过触发器记录变化,程序是perl写的.bucardo可以实现postgresql的多主复制、主从同步,甚至可以以postgresql为源库,可以和oracle、mysql、mongodb等很多数据库进行数据异步同步。 而pg原生的stream replication;虽可以同步;只能单向同步。只能是只读操作。而bucardo不支持DDL的同步; 所以说 相当于对stream replication的关系;相当于同oracle中 ogg对dg。
一 , 安装软件包 1. 首先安装依赖包: yum install perl-boolean wget
http://bucardo.org/downloads/dbix_safe.tar.gz wget
http://www.cpan.org/authors/id/T/TU/TURNSTEP/DBD-Pg-3.5.0.tar.gz wget
http://search.cpan.org/CPAN/authors/id/T/TI/TIMB/DBI-1.630.tar.gz 2. 安装Bucardo wget
http://bucardo.org/downloads/Bucardo-5.4.1.tar.gz 对上面perl安装包;安装过程
perl Makefile.PL
make
make install
3. 上面安装好了软件;下面实例化Bucardo;在postgres用户下面新建.bucardorc 文件
log_conflict_file = /home/postgres/log/bucardo_conflict.log piddir = /home/postgres/run reason_file = /home/postgres/log/bucardo.restart.reason.log warning_file = /home/postgres/log/bucardo.warning.log
4. 初始化bucardo
[postgres@sdserver40_222 ~]$ bucardo install This will install the bucardo database into an existing Postgres cluster. Postgres must have been compiled with Perl support, and you must connect as a superuser Current connection settings: 1. Host: <none> 2. Port: 5432 3. User: postgres 4. Database: bucardo 5. PID directory: /home/postgres/run Enter a number to change it, P to proceed, or Q to quit: 1 Change the host to: 127.0.0.1 Changed host to: 127.0.0.1 Current connection settings: 1. Host: 127.0.0.1 2. Port: 5432 3. User: postgres 4. Database: bucardo 5. PID directory: /home/postgres/run Enter a number to change it, P to proceed, or Q to quit: 4 Change the database name to: mydb Changed database name to: mydb Current connection settings: 1. Host: 127.0.0.1 2. Port: 5432 3. User: postgres 4. Database: mydb 5. PID directory: /home/postgres/run Enter a number to change it, P to proceed, or Q to quit: P Postgres version is: 9.5 Attempting to create and populate the bucardo database and schema Database creation is complete Updated configuration setting "piddir" Installation is now complete. If you see errors or need help, please email bucardo-general@bucardo.org You may want to check over the configuration variables next, by running: bucardo show all Change any setting by using: bucardo set foo=bar
这步可能会出现问题;大多数是依赖包没装的问题;现后面FAQ
二, 配置bucardo环境 下面的配置的环境: 主机1: 192.168.1.201 主机2: 192.168.1.202 1 添加源数据库 在192.168.1.201 执行
[postgres@sdserver40_222 ~]$ bucardo add db db1 dbname=mydb host=127.0.0.1 port=5432 user=postgres Added database "db1" [postgres@sdserver40_222 ~]$ bucardo add db db2 dbname=mydb host=192.168.1.202 port=5432 user=postgres Added database "db2"
这里需要注意的是;psql -h 192.168.1.202是否可以连通;看是不是防火墙;在主机192.168.1.202上pg_hba.conf是否配置。 2. 添加表: [
postgres@sdserver40_222 ~]$ bucardo add all tables New tables added: 0 Already added: 13 [
postgres@sdserver40_222 ~]$ bucardo add all sequences Sorry, no sequences were found New sequences added: 0 如果要添加某一个表,则可以bucardo add table tablename 3. 添加复制群组(即复制一组表): [
postgres@sdserver40_222 ~]$ bucardo add relgroup relgroup1 lottu03 lottu01; Created relgroup “relgroup1” The following tables or sequences are now part of the relgroup “relgroup1”: public.lottu01 public.lottu03 [
postgres@sdserver40_222 ~]$ bucardo add dbgroup dbgroup1 db1:source db2:target Created dbgroup “dbgroup1” Added database “db1” to dbgroup “dbgroup1” as source Added database “db2” to dbgroup “dbgroup1” as target
1)若添加若有表则用 all tables;
2) 只能同步存在主键的table;像这里的lottu01我是没建主键的;下面添加同步步骤会出现错误。
4. 添加同步
[postgres@sdserver40_222 ~]$ bucardo add sync sync1 relgroup=relgroup1 dbs=dbgroup1
Added sync "sync1"
1)这里注意的是;同步到表需要主键约束;不然同步是添加不了的。 会报如下错误;这个是跟触发器有关系;bucardo是建立在触发器之上的。
Failed to add sync: DBD::Pg::st execute failed: ERROR: Table "public.lottu01" must specify a primary key! at line 117. at line 30. CONTEXT: PL/Perl function "validate_sync" at /usr/local/bin/bucardo line 4612
2)删除同步 :bucardo add sync syncname 3 ) bucardo 添加同步;参数详情 如下;
add sync: bucardo add sync name relgroup=xxx dbs=xxx Adds a sync, which is a named replication event containing information about what to replicate from where to where. The supported parameters are: "dbs" The name of a dbgroup or comma-delimited list of databases. All of the specified databases will be synchronized. Required. "dbgroup" The name of a dbgroup. All of the databases within this group will be part of the sync. If the dbgroup does not exists and a separate list of databases is given, the group will be created and populated. "relgroup" The name of a relgroup to synchronize. All of the tables and/or sequences in the relgroup will be synchronized. Required unless "tables" is specified. "tables" List of tables to add to the sync. This implicitly creates a relgroup with the same name as the sync. Required unless "relgroup" is specified. "status" Indicates whether or not the sync is active. Must be either "active" or "inactive". Defaults to "active". "rebuild_index" Boolean indicating whether or not to rebuild indexes after every sync. Defaults to off. "lifetime" Number of seconds a KID can live before being reaped. No limit by default. "maxkicks" Number of times a KID may be kicked before being reaped. No limit by default. "conflict_strategy" The conflict resolution strategy to use in the sync. Supported values: "bucardo_source" The rows on the "source" database always "win". In other words, in a conflict, Bucardo copies rows from source to target. "bucardo_target" The rows on the "target" database always win. "bucardo_skip" Any conflicting rows are simply not replicated. Not recommended for most cases. "bucardo_random" Each database has an equal chance of winning each time. This is the default. "bucardo_latest" The row that was most recently changed wins. "bucardo_abort" The sync is aborted on a conflict. "onetimecopy" Determines whether or not a sync should switch to a full copy mode for a single run. Supported values are: 0: off 1: always full copy 2: only copy tables that are empty on the target "stayalive" Boolean indicating whether or not the sync processes (CTL) should be persistent. Defaults to false. "kidsalive" Boolean indicating whether or not the sync child processes (KID) should be persistent. Defaults to false. "autokick" Boolean indicating whether or not tables in the sync should automatically send kick messages when they're modified. May be overridden by the "autokick" parameter of individual tables. "checktime" An interval specifying the maximum time a sync should go before being kicked. Useful for busy systems where you don't want the overhead of notify triggers. "priority" An integer indicating the priority of the sync. Lower numbers are higher priority. Currently used only for display purposes. "analyze_after_copy" Boolean indicating whether or not to analyze tables after every sync. Off by default. Optional. "overdue" An interval specifying the amount of time after which the sync has not run that it should be considered overdue. "check_bucardo_sync" issues a warning when a sync has not been run in this amount of time. "expired" An interval specifying the amount of time after which the sync has not run that it should be considered expired. "check_bucardo_sync" issues a critical message when a sync has not been run in this amount of time. "track_rates" Boolean indicating whether or not to track synchronization rates. "rebuild_index" Boolean indicating whether or not to rebuild indexes after every sync. Off by default. Optional. "strict_check" Boolean indicating whether or not to be strict when comparing tables in the sync. If the columns have different names or data types, the validation will fail. But perhaps the columns are allowed to have different names or data types. If so, disable "strict_check" and column differences will result in warnings rather than failing the validation. Defaults to true.
5. 启动bucardo
bucardo start
bucardo stop
bucardo restart
三,验证操作 在原库操作
[postgres@sdserver40_222 ~]$ psql mydb psql (9.5.0) Type "help" for help. mydb=# delete from lottu03; DELETE 998 mydb=# insert into lottu03 values (1001,'bucardo'); INSERT 0 1
在目标库查看
[postgres@sdserver40_210 ~]$ psql mydb psql (9.5.0) Type "help" for help. mydb=# select * from lottu03; id | name ------+--------- 1001 | bucardo (1 row)
四, FAQ Q1. 在执行bucardo install 操作出现
Creating superuser 'bucardo' Attempting to create and populate the bucardo database and schema INSTALLATION FAILED! (psql:/usr/local/share/bucardo/bucardo.schema:46: ERROR: language "plperlu" does not exist HINT: Use CREATE LANGUAGE to load the language into the database.)
解决方法;看提示知道需要“create language plperlu;”但是在我的环境之下
mydb=# create language plperlu; ERROR: could not access file "$libdir/plperl": No such file or directory
出现这种问题不奇怪;因在我的博客
http://www.cnblogs.com/lottu/p/5149191.html 安装postgres中初始化时未加–
with-perl;所以安装后lib目录下缺少plperl.so文件; 解决方法:再来一边初始化postgrs; 把plperl.so复制到对应的lib目录下即可。
postgres=# CREATE LANGUAGE plperlu; CREATE LANGUAG
Q2: 解决Q1;再一次执行bucardo install 操作出现
install_driver(Pg) failed: DBI version 1.614 required--this is only version 1.609 at /usr/share/perl5/Exporter/Heavy.pm line 240, <> line 5. Compilation failed in require at (eval 9) line 3, <> line 5.
解决方法: 这个是我开始安装依赖包时;偷懒未安装DBI 包; 提示是需要更新
DBI 包;目前最新的是1.633版本;这里我们安装16.30就行 下载并安装: wget
http://search.cpan.org/CPAN/authors/id/T/TI/TIMB/DBI-1.630.tar.gz Q3: 安装bucardo install 之后错误提示
perl: symbol lookup error: /usr/local/lib64/perl5/auto/DBD/Pg/Pg.so: undefined symbol: DBIc_AIADESTROY
解决方法: 这个缺少symbol: DBIc_AIADESTROY; 这个是Q2问题遗留下来的; 之前安装DBD_pg时是之前的DBI;现DBI更新了;需要重新安装DBD_pg Q4:在启动bucardo start出现错误提示
Checking for existing processes Can't locate boolean.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /usr/local/share/perl5/Bucardo.pm line 34. BEGIN failed--compilation aborted at /usr/local/share/perl5/Bucardo.pm line 34. Compilation failed in require at /usr/local/bin/bucardo line 833.
解决方法: yum install perl-boolean