前言
Liquibase是一个用于跟踪、管理和应用数据库变化的开源的数据库重构工具。它将所有数据库的变化(包括结构和数据)都保存在XML文件中,便于版本控制。
那么在spring boot 中如何集成Liquibase,如何实现自动装配,如何通过actuator的方式对其进行监控,本文从以下3点来进行讲解:
- spring boot与Liquibase 的集成
- spring boot中Liquibase的自动装配源码分析
- spring boot actuator 中LiquibaseEndpoint源码分析
spring boot与Liquibase的集成
在pom.xml 文件中加入如下依赖:
<dependency> <groupId>org.liquibase</groupId> <artifactId>liquibase-core</artifactId> </dependency>
由于我使用的spring boot 版本为1.5.9.RELEASE,其默认依赖liquibase-core的版本为3.5.3
当然,在项目中需要加入数据库的驱动,如下:
<dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <scope>runtime</scope> </dependency>
在src/main/resources 下 创建 /db/changelog 的文件夹,如图:
在 src/main/resources/db/changelog 目录下新建master.xml,其内容如下:
<?xml version="1.0" encoding="utf-8"?> <databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.4.xsd"> <include file="classpath:/db/changelog/2017-01-15-init-schema.xml" relativeToChangelogFile="false"/> <include file="classpath:/db/changelog/2017-01-15-init-data.xml" relativeToChangelogFile="false"/> </databaseChangeLog>
其中relativeToChangelogFile = false,说明配置的file路径为绝对路径,不需要通过相对路径去查找文件
2017-01-15-init-schema.xml 文件内容如下:
<?xml version="1.0" encoding="utf-8"?> <databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.4.xsd"> <property name="autoIncrement" value="true" dbms="mysql" /> <changeSet id="2017-01-15" author="Harry"> <comment>init schema</comment> <createTable tableName="user"> <column name="id" type="bigint" autoIncrement="${autoIncrement}"> <constraints primaryKey="true" nullable="false" /> </column> <column name="nick_name" type="varchar(255)"> <constraints nullable="false" /> </column> <column name="email" type="varchar(255)"> <constraints nullable="false" /> </column> <column name="register_time" type="timestamp" defaultValueComputed="CURRENT_TIMESTAMP"> <constraints nullable="false" /> </column> </createTable> </changeSet> </databaseChangeLog>
其中changeSet 中的id 说明了本次变更的id,与author一起用于进行版本的跟踪
2017-01-15-init-data.xml 内容如下:
<?xml version="1.0" encoding="utf-8"?> <databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.4.xsd"> <!-- 添加数据 --> <changeSet id="2017-01-15-test-insert" author="Harry"> <insert tableName="user"> <column name="nick_name" value="harry"/> <column name="email" value="xxx@xxx.com"/> <column name="register_time" value="2017-01-15 22:00:02"/> </insert> </changeSet> <!-- 添加数据 --> <changeSet id="2017-01-15-test-insert-2" author="Harry"> <insert tableName="user"> <column name="nick_name" value="harry2"/> <column name="email" value="xxx@xxx.com"/> <column name="register_time" value="2017-01-15 22:00:02"/> </insert> </changeSet> <!-- 修改字段 --> <changeSet id="2017-01-15-test-chage" author="Harry"> <renameColumn tableName="user" oldColumnName="email" newColumnName="email_new" columnDataType="varchar(255)"/> </changeSet> </databaseChangeLog>
在application.properties中加入如下配置:
\# liquibase 主配置文件的路径 liquibase.change-log=classpath:/db/changelog/master.xml liquibase.user=xxx liquibase.password=xxx liquibase.url=你的数据库连接 \# 如果配置为true,则会每次执行时都会把对应的数据库drop掉,默认为false liquibase.drop-first=false
直接启动吧,启动完毕后就会发现在配置的数据库(liquibase.url)中有如下3张表:
databasechangelog –> liquibase 自动创建,用于保存每次变更的记录,创建语句如下:
CREATE TABLE `databasechangelog` ( `ID` varchar(255) NOT NULL, `AUTHOR` varchar(255) NOT NULL, `FILENAME` varchar(255) NOT NULL, `DATEEXECUTED` datetime NOT NULL, `ORDEREXECUTED` int(11) NOT NULL, `EXECTYPE` varchar(10) NOT NULL, `MD5SUM` varchar(35) DEFAULT NULL, `DESCRIPTION` varchar(255) DEFAULT NULL, `COMMENTS` varchar(255) DEFAULT NULL, `TAG` varchar(255) DEFAULT NULL, `LIQUIBASE` varchar(20) DEFAULT NULL, `CONTEXTS` varchar(255) DEFAULT NULL, `LABELS` varchar(255) DEFAULT NULL, `DEPLOYMENT_ID` varchar(10) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
databasechangeloglock–> liquibase 自动创建,创建语句如下:
CREATE TABLE `databasechangeloglock` ( `ID` int(11) NOT NULL, `LOCKED` bit(1) NOT NULL, `LOCKGRANTED` datetime DEFAULT NULL, `LOCKEDBY` varchar(255) DEFAULT NULL, PRIMARY KEY (`ID`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
- user –> 我们配置的表
同时发现在user表中的记录如下:
同时可以发现user表中的email改为了email_new
至此,liquibase和spring boot 的集成就介绍到这里,更多的知识可以百度..
Liquibase自动装配
Liquibase 自动装配是在org.springframework.boot.autoconfigure.liquibase 包下,如图:
LiquibaseProperties–> 个性化配置SpringLiquibase的配置类.该类声明了如下字段:
@ConfigurationProperties(prefix = "liquibase", ignoreUnknownFields = false) public class LiquibaseProperties { // 配置文件的路径 private String changeLog = "classpath:/db/changelog/db.changelog-master.yaml"; // 是否检查文件是否存在,默认为true private boolean checkChangeLogLocation = true; // 逗号分隔的运行上下文,在区分环境时有用 private String contexts; // 默认的数据库库名 private String defaultSchema; // 是否执行前先drop数据库,默认为false private boolean dropFirst; // 是否开启liquibase的支持,默认为true private boolean enabled = true; // 用来迁移数据的数据库用户名 private String user; // 用来迁移数据的数据库账户密码 private String password; // jdbc的链接 private String url; // 逗号分隔的运行时使用的label private String labels; // 参数 private Map<String, String> parameters; // 当执行更新时回滚sql所在的文件 private File rollbackFile;
由于该类声明了@ConfigurationProperties(prefix = “liquibase”, ignoreUnknownFields = false)注解,因此可以通过liquibase.xxx的方式进行配置,同时,如果配置的属性在LiquibaseProperties没有对应值,会抛出异常.
@LiquibaseDataSource –>指定要注入到Liquibase的数据源.如果该注解用于第二个数据源,则另一个(主)数据源通常需要被标记为@Primary注解.代码如下:
@Target({ ElementType.FIELD, ElementType.METHOD, ElementType.PARAMETER, ElementType.TYPE, ElementType.ANNOTATION_TYPE }) @Retention(RetentionPolicy.RUNTIME) @Documented @Qualifier public @interface LiquibaseDataSource { }
LiquibaseAutoConfiguration–> Liquibase的自动化配置类
LiquibaseAutoConfiguration 该类声明了如下注解:
@Configuration @ConditionalOnClass(SpringLiquibase.class) @ConditionalOnBean(DataSource.class) @ConditionalOnProperty(prefix = "liquibase", name = "enabled", matchIfMissing = true) @AutoConfigureAfter({ DataSourceAutoConfiguration.class, HibernateJpaAutoConfiguration.class })
- @Configuration–> 配置类
- @ConditionalOnClass(SpringLiquibase.class)–> 在类路径下存在SpringLiquibase.class时生效
- @ConditionalOnBean(DataSource.class)–> 在BeanFactory中存在DataSource类型的bean时生效
- @ConditionalOnProperty(prefix = “liquibase”, name = “enabled”, matchIfMissing = true) –> 如果配置有liquibase.enabled=true则该配置生效,如果没有配置的话,默认生效
由于此时我们引入了liquibase-core,因此,该配置是默认生效的.
老套路了,由于LiquibaseAutoConfiguration有2个配置内部类,因此,在解析加载的时候会首先处理内部类.
LiquibaseConfiguration
该类有如下注解:
@Configuration @ConditionalOnMissingBean(SpringLiquibase.class) @EnableConfigurationProperties(LiquibaseProperties.class) @Import(LiquibaseJpaDependencyConfiguration.class)
- @Configuration–> 配置类
- @ConditionalOnMissingBean(SpringLiquibase.class) –> 当BeanFactory中缺少SpringLiquibase类型的bean时生效
- @EnableConfigurationProperties(LiquibaseProperties.class) –> 引入LiquibaseProperties配置类
- @Import(LiquibaseJpaDependencyConfiguration.class) –> 导入LiquibaseJpaDependencyConfiguration 配置类
由于该类声明了@EnableConfigurationProperties(LiquibaseProperties.class) 和@Import(LiquibaseJpaDependencyConfiguration.class)注解,因此在ConfigurationClassParser#doProcessConfigurationClass中会首先调用processImports进行处理,此时获取的是EnableConfigurationPropertiesImportSelector, LiquibaseJpaDependencyConfiguration
EnableConfigurationPropertiesImportSelector:由于是ImportSelector的类型,因此会调用其selectImports方法,该类返回的是ConfigurationPropertiesBeanRegistrar,ConfigurationPropertiesBindingPostProcessorRegistrar.
接下来接着调用processImports 处理其返回值;
- ConfigurationPropertiesBeanRegistrar –> 由于是ImportBeanDefinitionRegistrar的实例,因此会加入到LiquibaseConfiguration对应的ConfigurationClass的中的importBeanDefinitionRegistrars
- ConfigurationPropertiesBindingPostProcessorRegistrar–>同样,由于是ImportBeanDefinitionRegistrar的实例,加入到LiquibaseConfiguration对应的ConfigurationClass的中的importBeanDefinitionRegistrars
LiquibaseJpaDependencyConfiguration–> 由于不是ImportSelector,ImportBeanDefinitionRegistrar的实例,因此会调用processConfigurationClass方法当做1个配置类来处理.
该类声明了如下注解:
@Configuration @ConditionalOnClass(LocalContainerEntityManagerFactoryBean.class) @ConditionalOnBean(AbstractEntityManagerFactoryBean.class)
- @Configuration –> 配置类
- @ConditionalOnClass(LocalContainerEntityManagerFactoryBean.class)–> 在当前的类路径下存在LocalContainerEntityManagerFactoryBean.class时生效
- @ConditionalOnBean(AbstractEntityManagerFactoryBean.class)–> 当beanFactory中存在AbstractEntityManagerFactoryBean类型的bean时生效
由于此时,我们没有加入JPA相关的依赖,因此,该配置不会生效.
该类继承了EntityManagerFactoryDependsOnPostProcessor,目的是–>使EntityManagerFactory 依赖 liquibase bean.其类图如下:
LiquibaseJpaDependencyConfiguration类的构造器如下:
public LiquibaseJpaDependencyConfiguration() { super("liquibase"); }
调用EntityManagerFactoryDependsOnPostProcessor的构造器,代码如下:
public EntityManagerFactoryDependsOnPostProcessor(String... dependsOn) { super(EntityManagerFactory.class, AbstractEntityManagerFactoryBean.class, dependsOn); }
调用AbstractDependsOnBeanFactoryPostProcessor的构造器,代码如下:
protected AbstractDependsOnBeanFactoryPostProcessor(Class<?> beanClass, Class<? extends FactoryBean<?>> factoryBeanClass, String... dependsOn) { this.beanClass = beanClass; this.factoryBeanClass = factoryBeanClass; this.dependsOn = dependsOn; }
注意,此时该类对应的字段值分别如下:
- beanClass = EntityManagerFactory.class
- factoryBeanClass=AbstractEntityManagerFactoryBean.class
- dependsOn= liquibase
由于该类实现了BeanFactoryPostProcessor接口,因此会调用其postProcessBeanFactory方法,代码如下:
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) { // 1. 获得beanClass,factoryBeanClass类型的beanid for (String beanName : getBeanNames(beanFactory)) { // 2. 获得对应的BeanDefinition BeanDefinition definition = getBeanDefinition(beanName, beanFactory); // 3. 添加设置的dependsOn到原先的DependsOn中 String[] dependencies = definition.getDependsOn(); for (String bean : this.dependsOn) { dependencies = StringUtils.addStringToArray(dependencies, bean); } definition.setDependsOn(dependencies); } }
获得beanClass,factoryBeanClass类型的bean id,遍历处理之.代码如下:
private Iterable<String> getBeanNames(ListableBeanFactory beanFactory) { Set<String> names = new HashSet<String>(); // 1. 获得beanClass类型的bean的id names.addAll(Arrays.asList(BeanFactoryUtils.beanNamesForTypeIncludingAncestors( beanFactory, this.beanClass, true, false))); // 2. 获得factoryBeanClass类型的工厂,然后将其转换成bean的id后加入到names中 for (String factoryBeanName : BeanFactoryUtils.beanNamesForTypeIncludingAncestors( beanFactory, this.factoryBeanClass, true, false)) { names.add(BeanFactoryUtils.transformedBeanName(factoryBeanName)); } return names; }
- 获得beanClass类型的bean的id,对于当前就是EntityManagerFactory类型的bean的id
- 获得factoryBeanClass类型的工厂,然后将其转换成bean的id后加入到names中,对应当前,就是AbstractEntityManagerFactoryBean类型的bean的id
- 根据beanId获得对应的BeanDefinition
- 添加设置的dependsOn到原先的DependsOn中,对应当前,就是添加liquibase到依赖中.
该类只声明了1个@Bean方法,如下:
@Bean public SpringLiquibase liquibase() { // 1. 创建SpringLiquibase SpringLiquibase liquibase = createSpringLiquibase(); // 2. 设置属性 liquibase.setChangeLog(this.properties.getChangeLog()); liquibase.setContexts(this.properties.getContexts()); liquibase.setDefaultSchema(this.properties.getDefaultSchema()); liquibase.setDropFirst(this.properties.isDropFirst()); liquibase.setShouldRun(this.properties.isEnabled()); liquibase.setLabels(this.properties.getLabels()); liquibase.setChangeLogParameters(this.properties.getParameters()); liquibase.setRollbackFile(this.properties.getRollbackFile()); return liquibase; }
- @Bean –> 注册1个id为 liquibasee,类型为SpringLiquibase的bean
该方法的逻辑如下:
- 创建SpringLiquibase
- 设置属性
其中, createSpringLiquibase代码如下:
private SpringLiquibase createSpringLiquibase() { // 1. 获得数据源,如果获取到,则直接实例化SpringLiquibase,并对其设置DataSource后直接返回即可 DataSource liquibaseDataSource = getDataSource(); if (liquibaseDataSource != null) { SpringLiquibase liquibase = new SpringLiquibase(); liquibase.setDataSource(liquibaseDataSource); return liquibase; } // 2. 否则,创建DataSourceClosingSpringLiquibase,通过createNewDataSource 创建DataSource,一般都会执行到这1步 SpringLiquibase liquibase = new DataSourceClosingSpringLiquibase(); liquibase.setDataSource(createNewDataSource()); return liquibase; }
获得数据源,如果获取到,则直接实例化SpringLiquibase,并对其设置DataSource后直接返回即可.代码如下:
private DataSource getDataSource() { // 1. 如果注入的liquibaseDataSource 不等null,则返回liquibaseDataSource,一般都不会注入的 if (this.liquibaseDataSource != null) { return this.liquibaseDataSource; } // 2. 如果没有配置的liquibase.url,则返dataSource(id为dataSource),此时意味着是对id为dataSource的数据源进行数据库版本迁移 if (this.properties.getUrl() == null) { return this.dataSource; } // 3. 其他情况(liquibase.url配置了,但是liquibaseDataSource没有配置),返回null return null; }
- 如果注入的liquibaseDataSource 不等null,则返回liquibaseDataSource,一般都不会注入的
- 如果没有配置的liquibase.url,则返dataSource(id为dataSource),此时意味着是对id为dataSource的数据源进行数据库版本迁移
- 其他情况(liquibase.url配置了,但是liquibaseDataSource没有配置),返回null
对应我们前面给出的示例,这里返回的null.
否则,创建DataSourceClosingSpringLiquibase,通过createNewDataSource 创建DataSource,一般都会执行到这步.
DataSourceClosingSpringLiquibase –> 继承SpringLiquibase来实现一旦实现变更同步就关闭数据源.实现变更同步是在afterPropertiesSet中完成的。代码如下:
private static final class DataSourceClosingSpringLiquibase extends SpringLiquibase { @Override public void afterPropertiesSet() throws LiquibaseException { super.afterPropertiesSet(); closeDataSource(); } private void closeDataSource() { // 1. 获得数据源所对应的class Class<?> dataSourceClass = getDataSource().getClass(); // 2. 尝试获得其声明的close方法,如果有的话,通过反射的方式进行调用 Method closeMethod = ReflectionUtils.findMethod(dataSourceClass, "close"); if (closeMethod != null) { ReflectionUtils.invokeMethod(closeMethod, getDataSource()); } } }
注意,迁移工作是在SpringLiquibase中的afterPropertiesSet完成的,这点只需看源码就知道了.
视线回到LiquibaseAutoConfiguration中的第2个内部类–> LiquibaseJpaDependencyConfiguration,该类已经在LiquibaseConfiguration中解析过了,这里就不再分析了. 注意1点的是,该配置类默认不会生效的.
由于LiquibaseAutoConfiguration没有定义@Bean方法,因此在ConfigurationClassParser#processConfigurationClass的解析就结束了
接下来,会调用ConfigurationClassBeanDefinitionReader#loadBeanDefinitions 进行处理:
LiquibaseConfiguration:
由于该类是被LiquibaseAutoConfiguration导入的,因此,会调用ConfigurationClassBeanDefinitionReader#registerBeanDefinitionForImportedConfigurationClass 进行注册.
由于该类有@Bean方法,因此会调用 ConfigurationClassBeanDefinitionReader#loadBeanDefinitionsForBeanMethod 依次进行注册
同时,由于该类存在importBeanDefinitionRegistrar,因此调用ConfigurationClassBeanDefinitionReader#loadBeanDefinitionsFromRegistrars进行处理.依次调用其registerBeanDefinitions方法.此时获得的是ConfigurationPropertiesBeanRegistrar, ConfigurationPropertiesBindingPostProcessorRegistrar
ConfigurationPropertiesBeanRegistrar–> 如果beanFactory中不存在id为liquibase-org.springframework.boot.autoconfigure.liquibase.LiquibaseProperties的bean的话,则进行注册,类型为LiquibaseProperties
ConfigurationPropertiesBindingPostProcessorRegistrar–> 如果beanFactory中不存在id为org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor的bean话,则注册类型ConfigurationPropertiesBindingPostProcessor的bean,id分别为 org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor,org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor.store
LiquibaseAutoConfiguration:
- 由于该类是被LiquibaseAutoConfiguration导入的,因此,会调用ConfigurationClassBeanDefinitionReader#registerBeanDefinitionForImportedConfigurationClass 进行注册.
在获取LiquibaseConfiguration bean的时候,由于该类有被@PostConstruct 注解的方法,因此会在该bean 初始化,执行如下方法:
@PostConstruct public void checkChangelogExists() { if (this.properties.isCheckChangeLogLocation()) { Resource resource = this.resourceLoader .getResource(this.properties.getChangeLog()); Assert.state(resource.exists(), "Cannot find changelog location: " + resource + " (please add changelog or check your Liquibase " + "configuration)"); } }
如果配置了liquibase.check-change-log-location = true(默认为true),则会通过ResourceLoader来对配置的liquibase.change-log 进行加载,如果不存在,则会抛出断言异常.
LiquibaseEndpoint解析
LiquibaseEndpoint在org.springframework.boot.actuate.endpoint包中,继承自AbstractEndpoint.
字段,构造器如下:
// key--> bean id,value --> SpringLiquibase的实例 private final Map<String, SpringLiquibase> liquibases; public LiquibaseEndpoint(Map<String, SpringLiquibase> liquibases) { super("liquibase"); Assert.notEmpty(liquibases, "Liquibases must be specified"); this.liquibases = liquibases; }
invoke 实现:
public List<LiquibaseReport> invoke() { List<LiquibaseReport> reports = new ArrayList<LiquibaseReport>(); // 1. 实例化DatabaseFactory和StandardChangeLogHistoryService DatabaseFactory factory = DatabaseFactory.getInstance(); StandardChangeLogHistoryService service = new StandardChangeLogHistoryService(); // 2. 遍历liquibases for (Map.Entry<String, SpringLiquibase> entry : this.liquibases.entrySet()) { try { // 2.1 根据配置信息获取到DataSource,创建JdbcConnection DataSource dataSource = entry.getValue().getDataSource(); JdbcConnection connection = new JdbcConnection( dataSource.getConnection()); try { // 2.2 根据JdbcConnection获得Database Database database = factory .findCorrectDatabaseImplementation(connection); // 2.3 如果配置有默认数据库,则对Database 进行赋值 String defaultSchema = entry.getValue().getDefaultSchema(); if (StringUtils.hasText(defaultSchema)) { database.setDefaultSchemaName(defaultSchema); } // 2.4 实例化LiquibaseReport 添加到reports中 reports.add(new LiquibaseReport(entry.getKey(), // 进行查询,sql 语句 为: select * from databasechangelog order by DATEEXECUTED ASC,ORDEREXECUTED ASC service.queryDatabaseChangeLogTable(database))); } finally { // 2.5 关闭资源 connection.close(); } } catch (Exception ex) { throw new IllegalStateException("Unable to get Liquibase changelog", ex); } } return reports; }
- 实例化DatabaseFactory和StandardChangeLogHistoryService
遍历liquibases
- 根据配置信息获取到DataSource,创建JdbcConnection
- 根据JdbcConnection获得Database
- 如果配置有默认数据库,则对Database 进行赋值
调用StandardChangeLogHistoryService#queryDatabaseChangeLogTable进行查询,sql 语句为: select * from databasechangelog order by DATEEXECUTED ASC,ORDEREXECUTED ASC,代码如下:
public List<Map<String, ?>> queryDatabaseChangeLogTable(Database database) throws DatabaseException { SelectFromDatabaseChangeLogStatement select = new SelectFromDatabaseChangeLogStatement(new ColumnConfig().setName("*").setComputed(true)).setOrderBy("DATEEXECUTED ASC", "ORDEREXECUTED ASC"); return ExecutorService.getInstance().getExecutor(database).queryForList(select); }
实例化LiquibaseReport 添加到reports中,代码如下:
public static class LiquibaseReport { // SpringLiquibase bean的id private final String name; // key--> databasechangelog 表的字段名,value--> 字段值 private final List<Map<String, ?>> changeLogs; public LiquibaseReport(String name, List<Map<String, ?>> changeLogs) { this.name = name; this.changeLogs = changeLogs; } public String getName() { return this.name; } public List<Map<String, ?>> getChangeLogs() { return this.changeLogs; } }
- 关闭资源