lua-resty-sync – synchronizing data based on version changes
Table of Contents
Synopsis
http {
lua_shared_dict sync 5m;
lua_shared_dict locks 1m;
init_worker_by_lua_block {
local sync = require "resty.sync"
local syncer, err = sync.new(5, "sync")
if not syncer then
ngx.log(ngx.WARN, "failed to create sync object: ", err)
return
end
local callback = function(mode)
if mode == sync.ACTION_DATA then
-- GET DATA
return "data " .. math.random(100) -- just some fake data
else
-- GET VERSION
return "version " .. math.random(100)
end
end
-- register some tasks
syncer:register(callback, "ex1")
-- start to run
syncer:start()
-- save it
SYNCER = syncer
}
server {
server_name _;
listen *:9080;
location = /t {
content_by_lua_block {
local sync = require "resty.sync"
local syncer = SYNCER
local version, err = syncer:get_version("ex1")
if not version then
ngx.log(ngx.WARN, "failed to fetch version: ", err)
return
end
local data, err = syncer:get_data("ex1")
if not data then
ngx.log(ngx.WARN, "failed to fetch data: ", err)
return
end
ngx.say("task ex1, data: ", data, " and version: ", version)
ngx.sleep(5)
-- after 5s
local version2, err = syncer:get_version("ex1")
if not version2 then
ngx.log(ngx.WARN, "failed to fetch version: ", err)
return
end
local data2, err = syncer:get_data("ex1")
if not data2 then
ngx.log(ngx.WARN, "failed to fetch data: ", err)
return
end
ngx.say("after 5s, task ex1, data: ", data2, " and version: ", version2)
}
}
}
}Description
This lua-resty library help you to synchronize data(from redis, mysql, memcached and so on) based on the version changes.
It will check the freshness by comparing the version cached by itself(stored in shared memory) and the one from your external suits, data will be updated when the cached one is stale or for the first time. See the Synopsis and Methods for learning how to use this library.
Note this lua module relies on lua-resty-lock.
Status
Probably production ready in most cases, though not yet proven in the wild. Please check the issues list and let me know if you have any problems / questions.
Methods
new
syntax: local syncer, err = sync.new(interval, shm)
phase: init_worker
Create and return an instance of the sync.
The first argument, interval, indicates the interval of two successive operations(in seconds), which shall be greater than 0.
The second argument shm, holds a Lua string, represents a shared memory.
In the case of failure, nil and a Lua string described the corresponding error will be given.
register
syntax: local ok, err = syncer:register(callback, tag)
phase: init_worker
Register a task to the instance syncer which created by new.
The first argument callback, can be any Lua function which will be invoked later in a background “light thread”.
The callback function not only used for capturing data, but also used for fetching version.
Only one argument mode can be passed to this function and the value always is:
- sync.ACTION_DATA – capturing data this time.
- sync.ACTION_VERSION – fetching version this time.
The second argument tag is a Lua string which is used for distinguishing different tasks,
so it can’t be duplicate with one task registered previously.
In the case of failure, nil and a Lua string described the corresponding error will be given.
start
syntax: local ok, err = syncer:start()
phase: init_worker
Let the instance syncer starts to work. Note there will be only one timer created among all workers.
The uniqueness is kept throughout your service’s lifetime even the timer owner worker is crash or nginx reload happens.
Callback in this instance will be run orderly(accroding the order of register).
In the case of failure, nil and a Lua string described the corresponding error will be given.
get_version
syntax: local version, err = syncer:get_version(tag)
phase: set_by_lua, rewrite_by_lua, access_by_lua, content_by_lua, header_filter_by_lua, body_filter_by_lua, log_by_lua,
ngx.timer.*, balancer_by_lua, ssl_certificate_by_lua, ssl_session_fetch_by_lua, ssl_session_store_by_lua
Get the current version of one task(specified by tag).
In the case of failure, nil and a Lua string described the corresponding error will be given.
In particually, nil and "no data" will be given when there is no data.
get_data
syntax: local data, err = syncer:get_data(tag)
phase: set_by_lua, rewrite_by_lua, access_by_lua, content_by_lua, header_filter_by_lua, body_filter_by_lua, log_by_lua,
ngx.timer.*, balancer_by_lua, ssl_certificate_by_lua, ssl_session_fetch_by_lua, ssl_session_store_by_lua
Get the current data of one task(specified by tag).
In the case of failure, nil and a Lua string described the corresponding error will be given.
In particually, nil and "no data" will be given when there is no data.
get_last_modified_time
syntax: local timestamp, err = syncer:get_last_modified_time(tag)
phase: set_by_lua, rewrite_by_lua, access_by_lua, content_by_lua, header_filter_by_lua, body_filter_by_lua, log_by_lua,
ngx.timer.*, balancer_by_lua, ssl_certificate_by_lua, ssl_session_fetch_by_lua, ssl_session_store_by_lua
Get the last update time(unix timestamp) of one task(specified by tag).
In the case of failure, nil and a Lua string described the corresponding error will be given.
In particually, nil and "no data" will be given when there is no data.
TODO
- Do the updates cocurrently in one sync instance.
Author
Alex Zhang(张超) zchao1995@gmail.com, UPYUN Inc.
Copyright and License
The bundle itself is licensed under the 2-clause BSD license.
Copyright (c) 2017, UPYUN(又拍云) Inc.
This module is licensed under the terms of the BSD license.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
- Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
