Low level consumer: At least once consuming
https://arnaud-lb.github.io/php-rdkafka/phpdoc/book.rdkafka.html
问题:
$topic->consumeStart($partition, $offset);
从一个指定的offset开始消费,消费完也执行了
$topic->offsetStore($msg->partition, $msg->offset);
消费进程重启之后,RD_KAFKA_OFFSET_STORED 不是上次提交的offset呢?edenhill/librdkafka$topic->offsetStore($msg->partition, $msg->offset);
消费进程重启之后,RD_KAFKA_OFFSET_STORED 不是上次提交的offset呢?
这个问题:
Find last stored commited offset
Find last stored commited offset. · Issue #78 · arnaud-lb/php-rdkafka
目前php-rdkafka还未实现这个方法,可以考虑用c librdkafka,实现一下
getting some stats · Issue #148 · arnaud-lb/php-rdkafka
librdkafka Consumer offset management
https://github.com/edenhill/librdkafka/wiki/Consumer-offset-management
总结:学习新技术,还是要先系统看文档,不能病急乱投医
zookeeper:
ZooKeeper Getting Started Guide
zookeeper kafka data structure
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper