Redis 的 client-output-buffer-limit 可以用來強制斷開無法足夠快從 redis 伺服器端讀取資料的使用者端。
保護機制規則如下:
該引數一般用在以下幾類使用者端中:
該引數的設定語法:
client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>
設定範例:
# 普通client buffer限制
client-output-buffer-limit normal 0 0 0
# slave client buffer限制
client-output-buffer-limit slave 256mb 64mb 60
# pubsub client buffer限制
client-output-buffer-limit pubsub 32mb 8mb 60
將 hard limit 和 soft limit 同時設定為 0,則表示關閉該限制。
該設定表示,對於 slave 使用者端來說,如果 output-buffer 佔用記憶體達到 256M 或者超過 64M 的時間達到 60s,則關閉使用者端連線。
該設定表示,對於 Pub/Sub 使用者端來說,若 output-buffer 佔用記憶體達到 32M 或者超過 8M 的時間達到 60s,則關閉使用者端連線。
概括說明:
一般情況下,對於普通使用者端,client-output-buffer 是不設限制的,因為 server 只會在 client 請求資料的時候才會傳送,不會產生積壓。
而在 server 主動傳送,client 來處理的場景下,這種一般都是非同步處理的,會劃出一個緩衝區來「暫存」未處理的資料,若 server 傳送資料比 client 處理資料快時,就會發生緩衝區積壓。對於用作 Pub/Sub 和 slave 的使用者端,server 會主動把資料推播給他們,故需要設定 client-output-buffer 的限制。
下面我們以主從同步時的 slave 使用者端,來具體分析下。
在 redis 在主從同步時,master 會為 slave 建立一個輸出緩衝區。在 master 儲存 rdb,將 rdb 檔案傳輸給 slave,slave 載入 rdb 完成之前,master 會將接收到的所有寫命令,寫入到記憶體中的這個輸出緩衝區去。
若 rdb 的儲存,傳輸,載入耗時過長,或者在此期間的寫命令過多,則可能會造成超過緩衝區限制,造成 master 和 slave 的連線斷開。此時則需要適當調整下 client-output-buffer-limit slave
設定。
基於 redis5.0 版本原始碼
redis server 通過 addReply 將資料傳送給使用者端,以下原始碼見 https://github.com/redis/redis/blob/5.0/src/networking.c
/* Add the object 'obj' string representation to the client output buffer. */
void addReply(client *c, robj *obj) {
if (prepareClientToWrite(c) != C_OK) return;
if (sdsEncodedObject(obj)) {
if (_addReplyToBuffer(c,obj->ptr,sdslen(obj->ptr)) != C_OK)
_addReplyStringToList(c,obj->ptr,sdslen(obj->ptr));
} else if (obj->encoding == OBJ_ENCODING_INT) {
char buf[32];
size_t len = ll2string(buf,sizeof(buf),(long)obj->ptr);
if (_addReplyToBuffer(c,buf,len) != C_OK)
_addReplyStringToList(c,buf,len);
} else {
serverPanic("Wrong obj->encoding in addReply()");
}
}
在函數的開頭,會通過prepareClientToWrite(c)
判斷是否需要將資料寫入使用者端的 output buffer 中。我們看下什麼條件下資料會被寫入使用者端的 output buffer 中,即返回 C_OK
。
/* This function is called every time we are going to transmit new data
* to the client. The behavior is the following:
*
* If the client should receive new data (normal clients will) the function
* returns C_OK, and make sure to install the write handler in our event
* loop so that when the socket is writable new data gets written.
*
* If the client should not receive new data, because it is a fake client
* (used to load AOF in memory), a master or because the setup of the write
* handler failed, the function returns C_ERR.
*
* The function may return C_OK without actually installing the write
* event handler in the following cases:
*
* 1) The event handler should already be installed since the output buffer
* already contains something.
* 2) The client is a slave but not yet online, so we want to just accumulate
* writes in the buffer but not actually sending them yet.
*
* Typically gets called every time a reply is built, before adding more
* data to the clients output buffers. If the function returns C_ERR no
* data should be appended to the output buffers. */
int prepareClientToWrite(client *c) {
/* If it's the Lua client we always return ok without installing any
* handler since there is no socket at all. */
if (c->flags & (CLIENT_LUA|CLIENT_MODULE)) return C_OK;
/* CLIENT REPLY OFF / SKIP handling: don't send replies. */
if (c->flags & (CLIENT_REPLY_OFF|CLIENT_REPLY_SKIP)) return C_ERR;
/* Masters don't receive replies, unless CLIENT_MASTER_FORCE_REPLY flag
* is set. */
if ((c->flags & CLIENT_MASTER) &&
!(c->flags & CLIENT_MASTER_FORCE_REPLY)) return C_ERR;
if (c->fd <= 0) return C_ERR; /* Fake client for AOF loading. */
/* Schedule the client to write the output buffers to the socket, unless
* it should already be setup to do so (it has already pending data). */
if (!clientHasPendingReplies(c)) clientInstallWriteHandler(c);
/* Authorize the caller to queue in the output buffer of this client. */
return C_OK;
}
/* Return true if the specified client has pending reply buffers to write to
* the socket. */
int clientHasPendingReplies(client *c) {
return c->bufpos || listLength(c->reply);
}
void clientInstallWriteHandler(client *c) {
/* Schedule the client to write the output buffers to the socket only
* if not already done and, for slaves, if the slave can actually receive
* writes at this stage. */
if (!(c->flags & CLIENT_PENDING_WRITE) &&
(c->replstate == REPL_STATE_NONE ||
(c->replstate == SLAVE_STATE_ONLINE && !c->repl_put_online_on_ack)))
{
c->flags |= CLIENT_PENDING_WRITE;
listAddNodeHead(server.clients_pending_write,c);
}
}
由於函數預設返回C_OK
,我們只需要看哪幾類情況返回的不是C_OK
,即C_ERR
,資料就不會被寫入到使用者端的 output buffer 中。
返回C_ERR
的情況:
If the client should not receive new data, because it is a fake client (used to load AOF in memory), a master or because the setup of the write handler failed, the function returns C_ERR.
在 master 儲存和傳送 rdb 檔案時,slave 的狀態是以下幾種,所以在這期間的寫命令都會儲存在 slave 的 output buffer。由於沒有設定回撥函數,資料並不會傳送到 slave 上,僅儲存在 master 為 slave 建立的 output buffer 內。
#define SLAVE_STATE_WAIT_BGSAVE_START 6 /* We need to produce a new RDB file. */
#define SLAVE_STATE_WAIT_BGSAVE_END 7 /* Waiting RDB file creation to finish. */
#define SLAVE_STATE_SEND_BULK 8 /* Sending RDB file to slave. */
那麼何時才會從 output buffer 中「刷入」slave 呢?直到 master 將 rdb 檔案完全傳送給 slave 後,master 會在 sendBulkToSlave
函數中進行相關操作。以下原始碼見:https://github.com/redis/redis/blob/5.0/src/replication.c
void sendBulkToSlave(aeEventLoop *el, int fd, void *privdata, int mask) {
// 此處省略部分原始碼
// rdb 檔案已完全傳送給 slave
if (slave->repldboff == slave->repldbsize) {
close(slave->repldbfd);
slave->repldbfd = -1;
aeDeleteFileEvent(server.el,slave->fd,AE_WRITABLE);
putSlaveOnline(slave);
}
}
void putSlaveOnline(client *slave) {
slave->replstate = SLAVE_STATE_ONLINE;
slave->repl_put_online_on_ack = 0;
slave->repl_ack_time = server.unixtime; /* Prevent false timeout. */
if (aeCreateFileEvent(server.el, slave->fd, AE_WRITABLE,
sendReplyToClient, slave) == AE_ERR) {
serverLog(LL_WARNING,"Unable to register writable event for replica bulk transfer: %s", strerror(errno));
freeClient(slave);
return;
}
refreshGoodSlavesCount();
serverLog(LL_NOTICE,"Synchronization with replica %s succeeded",
replicationGetSlaveName(slave));
}
此處會將 slave 狀態改為 SLAVE_STATE_ONLINE
,並將repl_put_online_on_ack
置為0,(有沒有很熟悉,對了,就是上面clientInstallWriteHandler
中判斷的內容)。同時也會設定回撥函數sendReplyToClient
,將此前 master 為 slave 建立的 output buffer 中的寫操作全部傳送到 slave 上。同時 slave 狀態的變更,會使得後續 master 上的寫操作可以正常的 push 到 slave 上了(直接,無需走 output buffer)。
本次我們通過 client-output-buffer-limit
引數,瞭解了其使用場景,並重點就主從同步時 output buffer 寫入情況進行了原始碼的簡單分析。今天的學習就到這裡,我們改天接著肝。