opensourcemirror
/
redis
mirror of https://github.com/antirez/redis.git
1
0
Fork 0
Code Issues Releases Wiki Activity

Merge pull request #10829 from oranagra/release-7.0.1 

Release 7.0.1
This commit is contained in:
Oran Agra 2022-06-08 12:56:24 +03:00 committed by GitHub
parent d375595d5e 595e725db4
commit 2667c41235
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
110 changed files with 2649 additions and 816 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
.github/ISSUE_TEMPLATE/crash_report.md vendored
View File

@ -1,12 +1,17 @@
---
name: Crash report
about: Submit a crash report
title: '[CRASH]'
title: '[CRASH] <short description>'
labels: ''
assignees: ''
---
Notice!
- If a Redis module was involved, please open an issue in the module's repo instead!
- If you're using docker on Apple M1, please make sure the image you're using was compiled for ARM!
**Crash report**
Paste the complete crash log between the quotes below. Please include a few lines from the log preceding the crash report to provide some context.

7
.github/workflows/codeql-analysis.yml vendored
View File

@ -1,7 +1,6 @@
name: "CodeQL"
on:
push:
pull_request:
schedule:
# run weekly new vulnerability was added to the database
@ -23,12 +22,12 @@ jobs:
uses: actions/checkout@v3
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
- name: Autobuild
uses: github/codeql-action/autobuild@v1
uses: github/codeql-action/autobuild@v2
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
uses: github/codeql-action/analyze@v2

83
00-RELEASENOTES
View File

@ -11,6 +11,89 @@ CRITICAL: There is a critical bug affecting MOST USERS. Upgrade ASAP.
SECURITY: There are security fixes in the release.
--------------------------------------------------------------------------------
================================================================================
Redis 7.0.1 Released Wed Jun 8 12:00:00 IST 2022
================================================================================
Upgrade urgency: MODERATE, specifically if you're using a previous release of
Redis 7.0, contains some behavior changes for new 7.0 features and important
fixes for bugs in previous 7.0 releases.
Improvements
============
* Add warning for suspected slow system clocksource setting
Add --check-system command line option. (#10636)
* Allow read-only scripts (*_RO commands, and ones with `no-writes` flag)
during CLIENT PAUSE WRITE (#10744)
* Add `readonly` flag in COMMAND command for EVAL_RO, EVALSHA_RO and FCALL_RO (#10728)
* redis-server command line arguments now accept one string with spaces
for multi-arg configs (#10660)
Potentially Breaking Changes
============================
* Omitting a config option value in command line argument no longer works (#10660)
* Hide the `may_replicate` flag from the COMMAND command response (#10744)
Potentially Breaking Changes for new Redis 7.0 features
-------------------------------------------------------
* Protocol: Sharded pubsub publish emits `smessage` instead of `message` (#10792)
* CLUSTER SHARDS returns slots as RESP integers, not strings (#10683)
* Block PFCOUNT and PUBLISH in read-only scripts (*_RO commands, and no-writes) (#10744)
* Scripts that declare the `no-writes` flag are implicitly `allow-oom` too (#10699)
Changes in CLI tools
====================
* redis-cli --bigkeys, --memkeys, --hotkeys, --scan. Finish nicely after Ctrl+C (#10736)
Platform / toolchain support related improvements
=================================================
* Support tcp-keepalive config interval on MacOs (#10667)
* Support RSS metrics on Haiku OS (#10687)
INFO fields and introspection changes
=====================================
* Add isolated network metrics for replication. (#10062, #10810)
Module API changes
==================
* Add two more new checks to RM_Call script mode (#10786)
* Add new RM_Call flag to let Redis automatically refuse `deny-oom` commands (#10786)
* Add module API RM_MallocUsableSize (#10795)
* Add missing REDISMODULE_NOTIFY_NEW (#10688)
* Fix cursor type in RedisModuleScanCursor to handle more than 2^31 elements (#10698)
* Fix RM_Yield bugs and RM_Call("EVAL") OOM check bug (#10786)
* Fix bugs in enum configs with overlapping bit flags (#10661)
Bug Fixes
=========
* FLUSHALL correctly resets rdb_changes_since_last_save INFO field (#10691)
* FLUSHDB is now propagated to replicas / AOF, even if the db is empty (#10691)
* Replica fail and retry the PSYNC if the master is unresponsive (#10726)
* Fix ZRANGESTORE crash when zset_max_listpack_entries is 0 (#10767)
Fixes for issues in previous release candidates of Redis 7.0
------------------------------------------------------------
* CONFIG REWRITE could cause a config change to be dropped for aliased configs (#10811)
* CONFIG REWRITE would omit rename-command and include lines (#10761)
NOTE: Affected users who used Redis 7.0.0 to rewrite their configuration file
should review and fix the file.
* Fix broken protocol after MISCONF (persistence) error (#10786)
* Fix --save command line regression (#10690)
* Fix possible regression around TLS config changes. re-load files even if the
file name didn't change. (#10713)
* Re-add SENTINEL SLAVES command, missing in redis 7.0 (#10723)
* BZMPOP gets unblocked by non-key args and returns them (#10764)
* Fix possible memory leak in XADD and XTRIM (#10753)
================================================================================
Redis 7.0.0 GA Released Wed Apr 27 12:00:00 IST 2022
================================================================================

4
SECURITY.md
View File

@ -11,10 +11,10 @@ unless this is not possible or feasible with a reasonable effort.
| Version | Supported |
| ------- | ------------------ |
| 7.0.x | :white_check_mark: |
| 6.2.x | :white_check_mark: |
| 6.0.x | :white_check_mark: |
| 5.0.x | :white_check_mark: |
| < 5.0 | :x: |
| < 6.0 | :x: |
## Reporting a Vulnerability

4
deps/README.md vendored
View File

@ -100,8 +100,8 @@ Hdr_Histogram
---
Updated source can be found here: https://github.com/HdrHistogram/HdrHistogram_c
We use a customized version 0.11.5
1. Compare all changes under /hdr_histogram directory to version 0.11.5
We use a customized version based on master branch commit e4448cf6d1cd08fff519812d3b1e58bd5a94ac42.
1. Compare all changes under /hdr_histogram directory to upstream master commit e4448cf6d1cd08fff519812d3b1e58bd5a94ac42
2. Copy updated files from newer version onto files in /hdr_histogram.
3. Apply the changes from 1 above to the updated files.

78
deps/hdr_histogram/README.md vendored
View File

@ -1,10 +1,78 @@
HdrHistogram_c v0.11.5
HdrHistogram_c: 'C' port of High Dynamic Range (HDR) Histogram
HdrHistogram
----------------------------------------------
This port contains a subset of the 'C' version of High Dynamic Range (HDR) Histogram available at [github.com/HdrHistogram/HdrHistogram_c](https://github.com/HdrHistogram/HdrHistogram_c).
[![Gitter chat](https://badges.gitter.im/HdrHistogram/HdrHistogram.png)](https://gitter.im/HdrHistogram/HdrHistogram)
This port contains a subset of the functionality supported by the Java
implementation. The current supported features are:
The code present on `hdr_histogram.c`, `hdr_histogram.h`, and `hdr_atomic.c` was Written by Gil Tene, Michael Barker,
and Matt Warren, and released to the public domain, as explained at
http://creativecommons.org/publicdomain/zero/1.0/.
* Standard histogram with 64 bit counts (32/16 bit counts not supported)
* All iterator types (all values, recorded, percentiles, linear, logarithmic)
* Histogram serialisation (encoding version 1.2, decoding 1.0-1.2)
* Reader/writer phaser and interval recorder
Features not supported, but planned
* Auto-resizing of histograms
Features unlikely to be implemented
* Double histograms
* Atomic/Concurrent histograms
* 16/32 bit histograms
# Simple Tutorial
## Recording values
```C
#include <hdr_histogram.h>
struct hdr_histogram* histogram;
// Initialise the histogram
hdr_init(
1, // Minimum value
INT64_C(3600000000), // Maximum value
3, // Number of significant figures
&histogram) // Pointer to initialise
// Record value
hdr_record_value(
histogram, // Histogram to record to
value) // Value to record
// Record value n times
hdr_record_values(
histogram, // Histogram to record to
value, // Value to record
10) // Record value 10 times
// Record value with correction for co-ordinated omission.
hdr_record_corrected_value(
histogram, // Histogram to record to
value, // Value to record
1000) // Record with expected interval of 1000.
// Print out the values of the histogram
hdr_percentiles_print(
histogram,
stdout, // File to write to
5, // Granularity of printed values
1.0, // Multiplier for results
CLASSIC); // Format CLASSIC/CSV supported.
```
## More examples
For more detailed examples of recording and logging results look at the
[hdr_decoder](examples/hdr_decoder.c)
and [hiccup](examples/hiccup.c)
examples. You can run hiccup and decoder
and pipe the results of one into the other.
```
$ ./examples/hiccup | ./examples/hdr_decoder
```

38
deps/hdr_histogram/hdr_histogram.c vendored
View File

@ -165,16 +165,6 @@ static int32_t count_leading_zeros_64(int64_t value)
#endif
}
static int64_t get_count_at_index_given_bucket_base_idx(const struct hdr_histogram* h, int32_t bucket_base_idx, int32_t sub_bucket_idx)
{
return h->counts[(bucket_base_idx + sub_bucket_idx) - h->sub_bucket_half_count];
}
static int32_t get_bucket_base_index(const struct hdr_histogram* h, int32_t bucket_index)
{
return (bucket_index + 1) << h->sub_bucket_half_count_magnitude;
}
static int32_t get_bucket_index(const struct hdr_histogram* h, int64_t value)
{
int32_t pow2ceiling = 64 - count_leading_zeros_64(value | h->sub_bucket_mask); /* smallest power of 2 containing value */
@ -679,33 +669,21 @@ int64_t hdr_min(const struct hdr_histogram* h)
static int64_t get_value_from_idx_up_to_count(const struct hdr_histogram* h, int64_t count_at_percentile)
{
int64_t count_to_idx = 0;
int64_t value_from_idx = 0;
int32_t sub_bucket_idx = -1;
int32_t bucket_idx = 0;
int32_t bucket_base_idx = get_bucket_base_index(h, bucket_idx);
// Overflow check
if (count_at_percentile > h->total_count)
count_at_percentile = 0 < count_at_percentile ? count_at_percentile : 1;
for (int32_t idx = 0; idx < h->counts_len; idx++)
{
count_at_percentile = h->total_count;
}
while (count_to_idx < count_at_percentile)
{
// increment bucket
sub_bucket_idx++;
if (sub_bucket_idx >= h->sub_bucket_count)
count_to_idx += h->counts[idx];
if (count_to_idx >= count_at_percentile)
{
sub_bucket_idx = h->sub_bucket_half_count;
bucket_idx++;
bucket_base_idx = get_bucket_base_index(h, bucket_idx);
return hdr_value_at_index(h, idx);
}
count_to_idx += get_count_at_index_given_bucket_base_idx(h, bucket_base_idx, sub_bucket_idx);
value_from_idx = ((int64_t)(sub_bucket_idx)) << (((int64_t)(bucket_idx)) + h->unit_magnitude);
}
return value_from_idx;
return 0;
}
int64_t hdr_value_at_percentile(const struct hdr_histogram* h, double percentile)
{
double requested_percentile = percentile < 100.0 ? percentile : 100.0;

3
deps/lua/src/lua_bit.c vendored
View File

@ -98,7 +98,8 @@ static int bit_bnot(lua_State *L) { BRET(~barg(L, 1)) }
#define BIT_OP(func, opr) \
static int func(lua_State *L) { int i; UBits b = barg(L, 1); \
for (i = lua_gettop(L); i > 1; i--) b opr barg(L, i); BRET(b) }
for (i = lua_gettop(L); i > 1; i--) b opr barg(L, i); \
BRET(b) }
BIT_OP(bit_band, &=)
BIT_OP(bit_bor, |=)
BIT_OP(bit_bxor, ^=)

5
src/Makefile
View File

@ -316,13 +316,14 @@ endif
REDIS_SERVER_NAME=redis-server$(PROG_SUFFIX)
REDIS_SENTINEL_NAME=redis-sentinel$(PROG_SUFFIX)
REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o eval.o bio.o rio.o rand.o memtest.o crcspeed.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o tracking.o connection.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o mt19937-64.o resp_parser.o call_reply.o script_lua.o script.o functions.o function_lua.o commands.o
REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o eval.o bio.o rio.o rand.o memtest.o syscheck.o crcspeed.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o tracking.o connection.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o mt19937-64.o resp_parser.o call_reply.o script_lua.o script.o functions.o function_lua.o commands.o
REDIS_CLI_NAME=redis-cli$(PROG_SUFFIX)
REDIS_CLI_OBJ=anet.o adlist.o dict.o redis-cli.o zmalloc.o release.o ae.o redisassert.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o mt19937-64.o
REDIS_BENCHMARK_NAME=redis-benchmark$(PROG_SUFFIX)
REDIS_BENCHMARK_OBJ=ae.o anet.o redis-benchmark.o adlist.o dict.o zmalloc.o redisassert.o release.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o mt19937-64.o
REDIS_CHECK_RDB_NAME=redis-check-rdb$(PROG_SUFFIX)
REDIS_CHECK_AOF_NAME=redis-check-aof$(PROG_SUFFIX)
ALL_SOURCES=$(sort $(patsubst %.o,%.c,$(REDIS_SERVER_OBJ) $(REDIS_CLI_OBJ) $(REDIS_BENCHMARK_OBJ)))
all: $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME)
@echo ""
@ -330,7 +331,7 @@ all: $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCH
@echo ""
Makefile.dep:
-$(REDIS_CC) -MM *.c > Makefile.dep 2> /dev/null || true
-$(REDIS_CC) -MM $(ALL_SOURCES) > Makefile.dep 2> /dev/null || true
ifeq (0, $(words $(findstring $(MAKECMDGOALS), $(NODEPS))))
-include Makefile.dep

41
src/acl.c
View File

@ -144,7 +144,7 @@ void ACLFreeLogEntry(void *le);
int ACLSetSelector(aclSelector *selector, const char *op, size_t oplen);
/* The length of the string representation of a hashed password. */
#define HASH_PASSWORD_LEN SHA256_BLOCK_SIZE*2
#define HASH_PASSWORD_LEN (SHA256_BLOCK_SIZE*2)
/* =============================================================================
* Helper functions for the rest of the ACL implementation
@ -153,42 +153,13 @@ int ACLSetSelector(aclSelector *selector, const char *op, size_t oplen);
/* Return zero if strings are the same, non-zero if they are not.
* The comparison is performed in a way that prevents an attacker to obtain
* information about the nature of the strings just monitoring the execution
* time of the function.
*
* Note that limiting the comparison length to strings up to 512 bytes we
* can avoid leaking any information about the password length and any
* possible branch misprediction related leak.
* time of the function. Note: The two strings must be the same length.
*/
int time_independent_strcmp(char *a, char *b) {
char bufa[CONFIG_AUTHPASS_MAX_LEN], bufb[CONFIG_AUTHPASS_MAX_LEN];
/* The above two strlen perform len(a) + len(b) operations where either
* a or b are fixed (our password) length, and the difference is only
* relative to the length of the user provided string, so no information
* leak is possible in the following two lines of code. */
unsigned int alen = strlen(a);
unsigned int blen = strlen(b);
unsigned int j;
int time_independent_strcmp(char *a, char *b, int len) {
int diff = 0;
/* We can't compare strings longer than our static buffers.
* Note that this will never pass the first test in practical circumstances
* so there is no info leak. */
if (alen > sizeof(bufa) || blen > sizeof(bufb)) return 1;
memset(bufa,0,sizeof(bufa)); /* Constant time. */
memset(bufb,0,sizeof(bufb)); /* Constant time. */
/* Again the time of the following two copies is proportional to
* len(a) + len(b) so no info is leaked. */
memcpy(bufa,a,alen);
memcpy(bufb,b,blen);
/* Always compare all the chars in the two buffers without
* conditional expressions. */
for (j = 0; j < sizeof(bufa); j++) {
diff |= (bufa[j] ^ bufb[j]);
for (int j = 0; j < len; j++) {
diff |= (a[j] ^ b[j]);
}
/* Length must be equal as well. */
diff |= alen ^ blen;
return diff; /* If zero strings are the same. */
}
@ -1414,7 +1385,7 @@ int ACLCheckUserCredentials(robj *username, robj *password) {
sds hashed = ACLHashPassword(password->ptr,sdslen(password->ptr));
while((ln = listNext(&li))) {
sds thispass = listNodeValue(ln);
if (!time_independent_strcmp(hashed, thispass)) {
if (!time_independent_strcmp(hashed, thispass, HASH_PASSWORD_LEN)) {
sdsfree(hashed);
return C_OK;
}

1
src/ae_select.c
View File

@ -50,6 +50,7 @@ static int aeApiCreate(aeEventLoop *eventLoop) {
}
static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
AE_NOTUSED(eventLoop);
/* Just ensure we have enough room in the fd_set type. */
if (setsize >= FD_SETSIZE) return -1;
return 0;

7
src/anet.c
View File

@ -162,6 +162,13 @@ int anetKeepAlive(char *err, int fd, int interval)
anetSetError(err, "setsockopt TCP_KEEPCNT: %s\n", strerror(errno));
return ANET_ERR;
}
#elif defined(__APPLE__)
/* Set idle time with interval */
val = interval;
if (setsockopt(fd, IPPROTO_TCP, TCP_KEEPALIVE, &val, sizeof(val)) < 0) {
anetSetError(err, "setsockopt TCP_KEEPALIVE: %s\n", strerror(errno));
return ANET_ERR;
}
#else
((void) interval); /* Avoid unused var warning for non Linux systems. */
#endif

47
src/aof.c
View File

@ -231,14 +231,14 @@ sds getAofManifestAsString(aofManifest *am) {
void aofLoadManifestFromDisk(void) {
server.aof_manifest = aofManifestCreate();
if (!dirExists(server.aof_dirname)) {
serverLog(LL_NOTICE, "The AOF directory %s doesn't exist", server.aof_dirname);
serverLog(LL_DEBUG, "The AOF directory %s doesn't exist", server.aof_dirname);
return;
}
sds am_name = getAofManifestFileName();
sds am_filepath = makePath(server.aof_dirname, am_name);
if (!fileExist(am_filepath)) {
serverLog(LL_NOTICE, "The AOF manifest file %s doesn't exist", am_name);
serverLog(LL_DEBUG, "The AOF manifest file %s doesn't exist", am_name);
sdsfree(am_name);
sdsfree(am_filepath);
return;
@ -714,15 +714,16 @@ void aofOpenIfNeededOnServerStart(void) {
}
/* If we start with an empty dataset, we will force create a BASE file. */
if (!server.aof_manifest->base_aof_info &&
!listLength(server.aof_manifest->incr_aof_list))
{
size_t incr_aof_len = listLength(server.aof_manifest->incr_aof_list);
if (!server.aof_manifest->base_aof_info && !incr_aof_len) {
sds base_name = getNewBaseFileNameAndMarkPreAsHistory(server.aof_manifest);
sds base_filepath = makePath(server.aof_dirname, base_name);
if (rewriteAppendOnlyFile(base_filepath) != C_OK) {
exit(1);
}
sdsfree(base_filepath);
serverLog(LL_NOTICE, "Creating AOF base file %s on server start",
base_name);
}
/* Because we will 'exit(1)' if open AOF or persistent manifest fails, so
@ -746,6 +747,12 @@ void aofOpenIfNeededOnServerStart(void) {
}
server.aof_last_incr_size = getAppendOnlyFileSize(aof_name, NULL);
if (incr_aof_len) {
serverLog(LL_NOTICE, "Opening AOF incr file %s on server start", aof_name);
} else {
serverLog(LL_NOTICE, "Creating AOF incr file %s on server start", aof_name);
}
}
int aofFileExist(char *filename) {
@ -801,6 +808,9 @@ int openNewIncrAofForAppend(void) {
goto cleanup;
}
}
serverLog(LL_NOTICE, "Creating AOF incr file %s on background rewrite",
new_aof_name);
sdsfree(new_aof_name);
/* If reaches here, we can safely modify the `server.aof_manifest`
@ -1343,7 +1353,7 @@ int loadSingleAppendOnlyFile(char *filename) {
off_t valid_up_to = 0; /* Offset of latest well-formed command loaded. */
off_t valid_before_multi = 0; /* Offset before MULTI command loaded. */
off_t last_progress_report_size = 0;
int ret = C_OK;
int ret = AOF_OK;
sds aof_filepath = makePath(server.aof_dirname, filename);
FILE *fp = fopen(aof_filepath, "r");
@ -1492,7 +1502,10 @@ int loadSingleAppendOnlyFile(char *filename) {
if (fakeClient->flags & CLIENT_MULTI &&
fakeClient->cmd->proc != execCommand)
{
queueMultiCommand(fakeClient);
/* Note: we don't have to attempt calling evalGetCommandFlags,
* since this is AOF, the checks in processCommand are not made
* anyway.*/
queueMultiCommand(fakeClient, cmd->flags);
} else {
cmd->proc(fakeClient);
}
@ -1523,7 +1536,7 @@ int loadSingleAppendOnlyFile(char *filename) {
goto uxeof;
}
loaded_ok: /* DB loaded, cleanup and return C_OK to the caller. */
loaded_ok: /* DB loaded, cleanup and return success (AOF_OK or AOF_TRUNCATED). */
loadingIncrProgress(ftello(fp) - last_progress_report_size);
server.aof_state = old_aof_state;
goto cleanup;
@ -1584,7 +1597,7 @@ cleanup:
/* Load the AOF files according the aofManifest pointed by am. */
int loadAppendOnlyFiles(aofManifest *am) {
serverAssert(am != NULL);
int status, ret = C_OK;
int status, ret = AOF_OK;
long long start;
off_t total_size = 0, base_size = 0;
sds aof_name;
@ -2258,7 +2271,7 @@ int rewriteAppendOnlyFileRio(rio *aof) {
}
/* In fork child process, we can try to release memory back to the
* OS and possibly avoid or decrease COW. We guve the dismiss
* OS and possibly avoid or decrease COW. We give the dismiss
* mechanism a hint about an estimated size of the object we stored. */
size_t dump_size = aof->processed_bytes - aof_bytes_before_key;
if (server.in_fork_child) dismissObject(o, dump_size);
@ -2348,7 +2361,6 @@ int rewriteAppendOnlyFile(char *filename) {
stopSaving(0);
return C_ERR;
}
serverLog(LL_NOTICE,"SYNC append only file rewrite performed");
stopSaving(1);
return C_OK;
@ -2403,6 +2415,8 @@ int rewriteAppendOnlyFileBackground(void) {
redisSetCpuAffinity(server.aof_rewrite_cpulist);
snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", (int) getpid());
if (rewriteAppendOnlyFile(tmpfile) == C_OK) {
serverLog(LL_NOTICE,
"Successfully created the temporary AOF base file %s", tmpfile);
sendChildCowInfo(CHILD_INFO_TYPE_AOF_COW_SIZE, "AOF rewrite");
exitFromChild(0);
} else {
@ -2543,7 +2557,7 @@ void backgroundRewriteDoneHandler(int exitcode, int bysignal) {
latencyStartMonitor(latency);
if (rename(tmpfile, new_base_filepath) == -1) {
serverLog(LL_WARNING,
"Error trying to rename the temporary AOF file %s into %s: %s",
"Error trying to rename the temporary AOF base file %s into %s: %s",
tmpfile,
new_base_filepath,
strerror(errno));
@ -2553,20 +2567,21 @@ void backgroundRewriteDoneHandler(int exitcode, int bysignal) {
}
latencyEndMonitor(latency);
latencyAddSampleIfNeeded("aof-rename", latency);
serverLog(LL_NOTICE,
"Successfully renamed the temporary AOF base file %s into %s", tmpfile, new_base_filename);
/* Rename the temporary incr aof file to 'new_incr_filename'. */
if (server.aof_state == AOF_WAIT_REWRITE) {
/* Get temporary incr aof name. */
sds temp_incr_aof_name = getTempIncrAofName();
sds temp_incr_filepath = makePath(server.aof_dirname, temp_incr_aof_name);
sdsfree(temp_incr_aof_name);
/* Get next new incr aof name. */
sds new_incr_filename = getNewIncrAofName(temp_am);
new_incr_filepath = makePath(server.aof_dirname, new_incr_filename);
latencyStartMonitor(latency);
if (rename(temp_incr_filepath, new_incr_filepath) == -1) {
serverLog(LL_WARNING,
"Error trying to rename the temporary incr AOF file %s into %s: %s",
"Error trying to rename the temporary AOF incr file %s into %s: %s",
temp_incr_filepath,
new_incr_filepath,
strerror(errno));
@ -2575,11 +2590,15 @@ void backgroundRewriteDoneHandler(int exitcode, int bysignal) {
aofManifestFree(temp_am);
sdsfree(temp_incr_filepath);
sdsfree(new_incr_filepath);
sdsfree(temp_incr_aof_name);
goto cleanup;
}
latencyEndMonitor(latency);
latencyAddSampleIfNeeded("aof-rename", latency);
serverLog(LL_NOTICE,
"Successfully renamed the temporary AOF incr file %s into %s", temp_incr_aof_name, new_incr_filename);
sdsfree(temp_incr_filepath);
sdsfree(temp_incr_aof_name);
}
/* Change the AOF file type in 'incr_aof_list' from AOF_FILE_TYPE_INCR

5
src/call_reply.c
View File

@ -528,7 +528,10 @@ CallReply *callReplyCreate(sds reply, list *deferred_error_list, void *private_d
/* Create a new CallReply struct from the reply blob representing an error message.
* Automatically creating deferred_error_list and set a copy of the reply in it.
* Refer to callReplyCreate for detailed explanation. */
* Refer to callReplyCreate for detailed explanation.
* Reply string can come in one of two forms:
* 1. A protocol reply starting with "-CODE" and ending with "\r\n"
* 2. A plain string, in which case this function adds the protocol header and footer. */
CallReply *callReplyCreateError(sds reply, void *private_data) {
sds err_buff = reply;
if (err_buff[0] != '-') {

4
src/cluster.c
View File

@ -4077,7 +4077,7 @@ void clusterCron(void) {
orphaned_masters++;
}
if (okslaves > max_slaves) max_slaves = okslaves;
if (nodeIsSlave(myself) && myself->slaveof == node)
if (myself->slaveof == node)
this_slaves = okslaves;
}
@ -5071,7 +5071,7 @@ void addShardReplyForClusterShards(client *c, clusterNode *node, uint16_t *slot_
serverAssert((slot_pairs_count % 2) == 0);
addReplyArrayLen(c, slot_pairs_count);
for (int i = 0; i < slot_pairs_count; i++)
addReplyBulkLongLong(c, (unsigned long)slot_info_pairs[i]);
addReplyLongLong(c, (unsigned long)slot_info_pairs[i]);
} else {
/* If no slot info pair is provided, the node owns no slots */
addReplyArrayLen(c, 0);

78
src/commands.c
View File

@ -124,7 +124,7 @@ struct redisCommandArg BITFIELD_RO_encoding_offset_Subargs[] = {
/* BITFIELD_RO argument table */
struct redisCommandArg BITFIELD_RO_Args[] = {
{"key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE},
{"encoding_offset",ARG_TYPE_BLOCK,-1,"GET",NULL,NULL,CMD_ARG_MULTIPLE,.subargs=BITFIELD_RO_encoding_offset_Subargs},
{"encoding_offset",ARG_TYPE_BLOCK,-1,"GET",NULL,NULL,CMD_ARG_MULTIPLE|CMD_ARG_MULTIPLE_TOKEN,.subargs=BITFIELD_RO_encoding_offset_Subargs},
{0}
};
@ -1595,7 +1595,7 @@ NULL
/* RENAME argument table */
struct redisCommandArg RENAME_Args[] = {
{"key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE},
{"newkey",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE},
{"newkey",ARG_TYPE_KEY,1,NULL,NULL,NULL,CMD_ARG_NONE},
{0}
};
@ -1613,7 +1613,7 @@ commandHistory RENAMENX_History[] = {
/* RENAMENX argument table */
struct redisCommandArg RENAMENX_Args[] = {
{"key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE},
{"newkey",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE},
{"newkey",ARG_TYPE_KEY,1,NULL,NULL,NULL,CMD_ARG_NONE},
{0}
};
@ -2602,6 +2602,13 @@ struct redisCommandArg PFCOUNT_Args[] = {
/* PFDEBUG tips */
#define PFDEBUG_tips NULL
/* PFDEBUG argument table */
struct redisCommandArg PFDEBUG_Args[] = {
{"subcommand",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_NONE},
{"key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE},
{0}
};
/********** PFMERGE ********************/
/* PFMERGE history */
@ -3153,7 +3160,7 @@ struct redisCommandArg PUBSUB_SHARDCHANNELS_Args[] = {
/* PUBSUB SHARDNUMSUB argument table */
struct redisCommandArg PUBSUB_SHARDNUMSUB_Args[] = {
{"channel",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_OPTIONAL|CMD_ARG_MULTIPLE},
{"shardchannel",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_OPTIONAL|CMD_ARG_MULTIPLE},
{0}
};
@ -3163,8 +3170,8 @@ struct redisCommand PUBSUB_Subcommands[] = {
{"help","Show helpful text about the different subcommands","O(1)","6.2.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,PUBSUB_HELP_History,PUBSUB_HELP_tips,pubsubCommand,2,CMD_LOADING|CMD_STALE,0},
{"numpat","Get the count of unique patterns pattern subscriptions","O(1)","2.8.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,PUBSUB_NUMPAT_History,PUBSUB_NUMPAT_tips,pubsubCommand,2,CMD_PUBSUB|CMD_LOADING|CMD_STALE,0},
{"numsub","Get the count of subscribers for channels","O(N) for the NUMSUB subcommand, where N is the number of requested channels","2.8.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,PUBSUB_NUMSUB_History,PUBSUB_NUMSUB_tips,pubsubCommand,-2,CMD_PUBSUB|CMD_LOADING|CMD_STALE,0,.args=PUBSUB_NUMSUB_Args},
{"shardchannels","List active shard channels","O(N) where N is the number of active shard channels, and assuming constant time pattern matching (relatively short channels).","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,PUBSUB_SHARDCHANNELS_History,PUBSUB_SHARDCHANNELS_tips,pubsubCommand,-2,CMD_PUBSUB|CMD_LOADING|CMD_STALE,0,.args=PUBSUB_SHARDCHANNELS_Args},
{"shardnumsub","Get the count of subscribers for shard channels","O(N) for the SHARDNUMSUB subcommand, where N is the number of requested channels","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,PUBSUB_SHARDNUMSUB_History,PUBSUB_SHARDNUMSUB_tips,pubsubCommand,-2,CMD_PUBSUB|CMD_LOADING|CMD_STALE,0,.args=PUBSUB_SHARDNUMSUB_Args},
{"shardchannels","List active shard channels","O(N) where N is the number of active shard channels, and assuming constant time pattern matching (relatively short shard channels).","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,PUBSUB_SHARDCHANNELS_History,PUBSUB_SHARDCHANNELS_tips,pubsubCommand,-2,CMD_PUBSUB|CMD_LOADING|CMD_STALE,0,.args=PUBSUB_SHARDCHANNELS_Args},
{"shardnumsub","Get the count of subscribers for shard channels","O(N) for the SHARDNUMSUB subcommand, where N is the number of requested shard channels","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,PUBSUB_SHARDNUMSUB_History,PUBSUB_SHARDNUMSUB_tips,pubsubCommand,-2,CMD_PUBSUB|CMD_LOADING|CMD_STALE,0,.args=PUBSUB_SHARDNUMSUB_Args},
{0}
};
@ -3200,7 +3207,7 @@ struct redisCommandArg PUNSUBSCRIBE_Args[] = {
/* SPUBLISH argument table */
struct redisCommandArg SPUBLISH_Args[] = {
{"channel",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_NONE},
{"shardchannel",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_NONE},
{"message",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_NONE},
{0}
};
@ -3215,7 +3222,7 @@ struct redisCommandArg SPUBLISH_Args[] = {
/* SSUBSCRIBE argument table */
struct redisCommandArg SSUBSCRIBE_Args[] = {
{"channel",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE},
{"shardchannel",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_MULTIPLE},
{0}
};
@ -3243,7 +3250,7 @@ struct redisCommandArg SUBSCRIBE_Args[] = {
/* SUNSUBSCRIBE argument table */
struct redisCommandArg SUNSUBSCRIBE_Args[] = {
{"channel",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_OPTIONAL|CMD_ARG_MULTIPLE},
{"shardchannel",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_OPTIONAL|CMD_ARG_MULTIPLE},
{0}
};
@ -3951,6 +3958,20 @@ struct redisCommandArg SENTINEL_SIMULATE_FAILURE_Args[] = {
{0}
};
/********** SENTINEL SLAVES ********************/
/* SENTINEL SLAVES history */
#define SENTINEL_SLAVES_History NULL
/* SENTINEL SLAVES tips */
#define SENTINEL_SLAVES_tips NULL
/* SENTINEL SLAVES argument table */
struct redisCommandArg SENTINEL_SLAVES_Args[] = {
{"master-name",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_NONE},
{0}
};
/* SENTINEL command table */
struct redisCommand SENTINEL_Subcommands[] = {
{"ckquorum","Check for a Sentinel quorum",NULL,"2.8.4",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SENTINEL,SENTINEL_CKQUORUM_History,SENTINEL_CKQUORUM_tips,sentinelCommand,3,CMD_ADMIN|CMD_SENTINEL|CMD_ONLY_SENTINEL,0,.args=SENTINEL_CKQUORUM_Args},
@ -3973,6 +3994,7 @@ struct redisCommand SENTINEL_Subcommands[] = {
{"sentinels","List the Sentinel instances","O(N) where N is the number of Sentinels","2.8.4",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SENTINEL,SENTINEL_SENTINELS_History,SENTINEL_SENTINELS_tips,sentinelCommand,3,CMD_ADMIN|CMD_SENTINEL|CMD_ONLY_SENTINEL,0,.args=SENTINEL_SENTINELS_Args},
{"set","Change the configuration of a monitored master","O(1)","2.8.4",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SENTINEL,SENTINEL_SET_History,SENTINEL_SET_tips,sentinelCommand,-5,CMD_ADMIN|CMD_SENTINEL|CMD_ONLY_SENTINEL,0,.args=SENTINEL_SET_Args},
{"simulate-failure","Simulate failover scenarios",NULL,"3.2.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SENTINEL,SENTINEL_SIMULATE_FAILURE_History,SENTINEL_SIMULATE_FAILURE_tips,sentinelCommand,-3,CMD_ADMIN|CMD_SENTINEL|CMD_ONLY_SENTINEL,0,.args=SENTINEL_SIMULATE_FAILURE_Args},
{"slaves","List the monitored slaves","O(N) where N is the number of slaves","2.8.0",CMD_DOC_DEPRECATED,"`SENTINEL REPLICAS`","5.0.0",COMMAND_GROUP_SENTINEL,SENTINEL_SLAVES_History,SENTINEL_SLAVES_tips,sentinelCommand,3,CMD_ADMIN|CMD_SENTINEL|CMD_ONLY_SENTINEL,0,.args=SENTINEL_SLAVES_Args},
{0}
};
@ -4924,11 +4946,28 @@ struct redisCommandArg REPLICAOF_Args[] = {
/********** RESTORE_ASKING ********************/
/* RESTORE_ASKING history */
#define RESTORE_ASKING_History NULL
commandHistory RESTORE_ASKING_History[] = {
{"3.0.0","Added the `REPLACE` modifier."},
{"5.0.0","Added the `ABSTTL` modifier."},
{"5.0.0","Added the `IDLETIME` and `FREQ` options."},
{0}
};
/* RESTORE_ASKING tips */
#define RESTORE_ASKING_tips NULL
/* RESTORE_ASKING argument table */
struct redisCommandArg RESTORE_ASKING_Args[] = {
{"key",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE},
{"ttl",ARG_TYPE_INTEGER,-1,NULL,NULL,NULL,CMD_ARG_NONE},
{"serialized-value",ARG_TYPE_STRING,-1,NULL,NULL,NULL,CMD_ARG_NONE},
{"replace",ARG_TYPE_PURE_TOKEN,-1,"REPLACE",NULL,"3.0.0",CMD_ARG_OPTIONAL},
{"absttl",ARG_TYPE_PURE_TOKEN,-1,"ABSTTL",NULL,"5.0.0",CMD_ARG_OPTIONAL},
{"seconds",ARG_TYPE_INTEGER,-1,"IDLETIME",NULL,"5.0.0",CMD_ARG_OPTIONAL},
{"frequency",ARG_TYPE_INTEGER,-1,"FREQ",NULL,"5.0.0",CMD_ARG_OPTIONAL},
{0}
};
/********** ROLE ********************/
/* ROLE history */
@ -6437,6 +6476,7 @@ struct redisCommandArg XINFO_GROUPS_Args[] = {
/* XINFO STREAM history */
commandHistory XINFO_STREAM_History[] = {
{"6.0.0","Added the `FULL` modifier."},
{"7.0.0","Added the `max-deleted-entry-id`, `entries-added`, `recorded-first-entry-id`, `entries-read` and `lag` fields"},
{0}
};
@ -6862,7 +6902,7 @@ struct redisCommandArg INCRBYFLOAT_Args[] = {
/* LCS argument table */
struct redisCommandArg LCS_Args[] = {
{"key1",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE},
{"key2",ARG_TYPE_KEY,1,NULL,NULL,NULL,CMD_ARG_NONE},
{"key2",ARG_TYPE_KEY,0,NULL,NULL,NULL,CMD_ARG_NONE},
{"len",ARG_TYPE_PURE_TOKEN,-1,"LEN",NULL,NULL,CMD_ARG_OPTIONAL},
{"idx",ARG_TYPE_PURE_TOKEN,-1,"IDX",NULL,NULL,CMD_ARG_OPTIONAL},
{"len",ARG_TYPE_INTEGER,-1,"MINMATCHLEN",NULL,NULL,CMD_ARG_OPTIONAL},
@ -7122,7 +7162,7 @@ struct redisCommand redisCommandTable[] = {
/* bitmap */
{"bitcount","Count set bits in a string","O(N)","2.6.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_BITMAP,BITCOUNT_History,BITCOUNT_tips,bitcountCommand,-2,CMD_READONLY,ACL_CATEGORY_BITMAP,{{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}},.args=BITCOUNT_Args},
{"bitfield","Perform arbitrary bitfield integer operations on strings","O(1) for each subcommand specified","3.2.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_BITMAP,BITFIELD_History,BITFIELD_tips,bitfieldCommand,-2,CMD_WRITE|CMD_DENYOOM,ACL_CATEGORY_BITMAP,{{"This command allows both access and modification of the key",CMD_KEY_RW|CMD_KEY_UPDATE|CMD_KEY_ACCESS|CMD_KEY_VARIABLE_FLAGS,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}},bitfieldGetKeys,.args=BITFIELD_Args},
{"bitfield_ro","Perform arbitrary bitfield integer operations on strings. Read-only variant of BITFIELD","O(1) for each subcommand specified","6.2.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_BITMAP,BITFIELD_RO_History,BITFIELD_RO_tips,bitfieldroCommand,-2,CMD_READONLY|CMD_FAST,ACL_CATEGORY_BITMAP,{{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}},.args=BITFIELD_RO_Args},
{"bitfield_ro","Perform arbitrary bitfield integer operations on strings. Read-only variant of BITFIELD","O(1) for each subcommand specified","6.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_BITMAP,BITFIELD_RO_History,BITFIELD_RO_tips,bitfieldroCommand,-2,CMD_READONLY|CMD_FAST,ACL_CATEGORY_BITMAP,{{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}},.args=BITFIELD_RO_Args},
{"bitop","Perform bitwise operations between strings","O(N)","2.6.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_BITMAP,BITOP_History,BITOP_tips,bitopCommand,-4,CMD_WRITE|CMD_DENYOOM,ACL_CATEGORY_BITMAP,{{NULL,CMD_KEY_OW|CMD_KEY_UPDATE,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_RANGE,.fk.range={0,1,0}},{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={3},KSPEC_FK_RANGE,.fk.range={-1,1,0}}},.args=BITOP_Args},
{"bitpos","Find first bit set or clear in a string","O(N)","2.8.7",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_BITMAP,BITPOS_History,BITPOS_tips,bitposCommand,-3,CMD_READONLY,ACL_CATEGORY_BITMAP,{{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}},.args=BITPOS_Args},
{"getbit","Returns the bit value at offset in the string value stored at key","O(1)","2.2.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_BITMAP,GETBIT_History,GETBIT_tips,getbitCommand,3,CMD_READONLY|CMD_FAST,ACL_CATEGORY_BITMAP,{{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}},.args=GETBIT_Args},
@ -7201,7 +7241,7 @@ struct redisCommand redisCommandTable[] = {
/* hyperloglog */
{"pfadd","Adds the specified elements to the specified HyperLogLog.","O(1) to add every element.","2.8.9",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_HYPERLOGLOG,PFADD_History,PFADD_tips,pfaddCommand,-2,CMD_WRITE|CMD_DENYOOM|CMD_FAST,ACL_CATEGORY_HYPERLOGLOG,{{NULL,CMD_KEY_RW|CMD_KEY_INSERT,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}},.args=PFADD_Args},
{"pfcount","Return the approximated cardinality of the set(s) observed by the HyperLogLog at key(s).","O(1) with a very small average constant time when called with a single key. O(N) with N being the number of keys, and much bigger constant times, when called with multiple keys.","2.8.9",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_HYPERLOGLOG,PFCOUNT_History,PFCOUNT_tips,pfcountCommand,-2,CMD_READONLY|CMD_MAY_REPLICATE,ACL_CATEGORY_HYPERLOGLOG,{{"RW because it may change the internal representation of the key, and propagate to replicas",CMD_KEY_RW|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={-1,1,0}}},.args=PFCOUNT_Args},
{"pfdebug","Internal commands for debugging HyperLogLog values","N/A","2.8.9",CMD_DOC_SYSCMD,NULL,NULL,COMMAND_GROUP_HYPERLOGLOG,PFDEBUG_History,PFDEBUG_tips,pfdebugCommand,-3,CMD_WRITE|CMD_DENYOOM|CMD_ADMIN,ACL_CATEGORY_HYPERLOGLOG,{{NULL,CMD_KEY_RW|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_RANGE,.fk.range={0,1,0}}}},
{"pfdebug","Internal commands for debugging HyperLogLog values","N/A","2.8.9",CMD_DOC_SYSCMD,NULL,NULL,COMMAND_GROUP_HYPERLOGLOG,PFDEBUG_History,PFDEBUG_tips,pfdebugCommand,3,CMD_WRITE|CMD_DENYOOM|CMD_ADMIN,ACL_CATEGORY_HYPERLOGLOG,{{NULL,CMD_KEY_RW|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_RANGE,.fk.range={0,1,0}}},.args=PFDEBUG_Args},
{"pfmerge","Merge N different HyperLogLogs into a single one.","O(N) to merge N HyperLogLogs, but with high constant times.","2.8.9",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_HYPERLOGLOG,PFMERGE_History,PFMERGE_tips,pfmergeCommand,-2,CMD_WRITE|CMD_DENYOOM,ACL_CATEGORY_HYPERLOGLOG,{{NULL,CMD_KEY_RW|CMD_KEY_ACCESS|CMD_KEY_INSERT,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}},{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_RANGE,.fk.range={-1,1,0}}},.args=PFMERGE_Args},
{"pfselftest","An internal command for testing HyperLogLog values","N/A","2.8.9",CMD_DOC_SYSCMD,NULL,NULL,COMMAND_GROUP_HYPERLOGLOG,PFSELFTEST_History,PFSELFTEST_tips,pfselftestCommand,1,CMD_ADMIN,ACL_CATEGORY_HYPERLOGLOG},
/* list */
@ -7235,15 +7275,15 @@ struct redisCommand redisCommandTable[] = {
{"spublish","Post a message to a shard channel","O(N) where N is the number of clients subscribed to the receiving shard channel.","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,SPUBLISH_History,SPUBLISH_tips,spublishCommand,3,CMD_PUBSUB|CMD_LOADING|CMD_STALE|CMD_FAST|CMD_MAY_REPLICATE,0,{{NULL,CMD_KEY_NOT_KEY,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}},.args=SPUBLISH_Args},
{"ssubscribe","Listen for messages published to the given shard channels","O(N) where N is the number of shard channels to subscribe to.","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,SSUBSCRIBE_History,SSUBSCRIBE_tips,ssubscribeCommand,-2,CMD_PUBSUB|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE,0,{{NULL,CMD_KEY_NOT_KEY,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={-1,1,0}}},.args=SSUBSCRIBE_Args},
{"subscribe","Listen for messages published to the given channels","O(N) where N is the number of channels to subscribe to.","2.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,SUBSCRIBE_History,SUBSCRIBE_tips,subscribeCommand,-2,CMD_PUBSUB|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,.args=SUBSCRIBE_Args},
{"sunsubscribe","Stop listening for messages posted to the given shard channels","O(N) where N is the number of clients already subscribed to a channel.","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,SUNSUBSCRIBE_History,SUNSUBSCRIBE_tips,sunsubscribeCommand,-1,CMD_PUBSUB|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE,0,{{NULL,CMD_KEY_NOT_KEY,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={-1,1,0}}},.args=SUNSUBSCRIBE_Args},
{"sunsubscribe","Stop listening for messages posted to the given shard channels","O(N) where N is the number of clients already subscribed to a shard channel.","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,SUNSUBSCRIBE_History,SUNSUBSCRIBE_tips,sunsubscribeCommand,-1,CMD_PUBSUB|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE,0,{{NULL,CMD_KEY_NOT_KEY,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={-1,1,0}}},.args=SUNSUBSCRIBE_Args},
{"unsubscribe","Stop listening for messages posted to the given channels","O(N) where N is the number of clients already subscribed to a channel.","2.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_PUBSUB,UNSUBSCRIBE_History,UNSUBSCRIBE_tips,unsubscribeCommand,-1,CMD_PUBSUB|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SENTINEL,0,.args=UNSUBSCRIBE_Args},
/* scripting */
{"eval","Execute a Lua script server side","Depends on the script that is executed.","2.6.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SCRIPTING,EVAL_History,EVAL_tips,evalCommand,-3,CMD_NOSCRIPT|CMD_SKIP_MONITOR|CMD_MAY_REPLICATE|CMD_NO_MANDATORY_KEYS|CMD_STALE,ACL_CATEGORY_SCRIPTING,{{"We cannot tell how the keys will be used so we assume the worst, RW and UPDATE",CMD_KEY_RW|CMD_KEY_ACCESS|CMD_KEY_UPDATE,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},evalGetKeys,.args=EVAL_Args},
{"evalsha","Execute a Lua script server side","Depends on the script that is executed.","2.6.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SCRIPTING,EVALSHA_History,EVALSHA_tips,evalShaCommand,-3,CMD_NOSCRIPT|CMD_SKIP_MONITOR|CMD_MAY_REPLICATE|CMD_NO_MANDATORY_KEYS|CMD_STALE,ACL_CATEGORY_SCRIPTING,{{NULL,CMD_KEY_RW|CMD_KEY_ACCESS|CMD_KEY_UPDATE,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},evalGetKeys,.args=EVALSHA_Args},
{"evalsha_ro","Execute a read-only Lua script server side","Depends on the script that is executed.","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SCRIPTING,EVALSHA_RO_History,EVALSHA_RO_tips,evalShaRoCommand,-3,CMD_NOSCRIPT|CMD_SKIP_MONITOR|CMD_NO_MANDATORY_KEYS|CMD_STALE,ACL_CATEGORY_SCRIPTING,{{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},evalGetKeys,.args=EVALSHA_RO_Args},
{"eval_ro","Execute a read-only Lua script server side","Depends on the script that is executed.","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SCRIPTING,EVAL_RO_History,EVAL_RO_tips,evalRoCommand,-3,CMD_NOSCRIPT|CMD_SKIP_MONITOR|CMD_NO_MANDATORY_KEYS|CMD_STALE,ACL_CATEGORY_SCRIPTING,{{"We cannot tell how the keys will be used so we assume the worst, RO and ACCESS",CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},evalGetKeys,.args=EVAL_RO_Args},
{"evalsha_ro","Execute a read-only Lua script server side","Depends on the script that is executed.","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SCRIPTING,EVALSHA_RO_History,EVALSHA_RO_tips,evalShaRoCommand,-3,CMD_NOSCRIPT|CMD_SKIP_MONITOR|CMD_NO_MANDATORY_KEYS|CMD_STALE|CMD_READONLY,ACL_CATEGORY_SCRIPTING,{{NULL,CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},evalGetKeys,.args=EVALSHA_RO_Args},
{"eval_ro","Execute a read-only Lua script server side","Depends on the script that is executed.","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SCRIPTING,EVAL_RO_History,EVAL_RO_tips,evalRoCommand,-3,CMD_NOSCRIPT|CMD_SKIP_MONITOR|CMD_NO_MANDATORY_KEYS|CMD_STALE|CMD_READONLY,ACL_CATEGORY_SCRIPTING,{{"We cannot tell how the keys will be used so we assume the worst, RO and ACCESS",CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},evalGetKeys,.args=EVAL_RO_Args},
{"fcall","Invoke a function","Depends on the function that is executed.","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SCRIPTING,FCALL_History,FCALL_tips,fcallCommand,-3,CMD_NOSCRIPT|CMD_SKIP_MONITOR|CMD_MAY_REPLICATE|CMD_NO_MANDATORY_KEYS|CMD_STALE,ACL_CATEGORY_SCRIPTING,{{"We cannot tell how the keys will be used so we assume the worst, RW and UPDATE",CMD_KEY_RW|CMD_KEY_ACCESS|CMD_KEY_UPDATE,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},functionGetKeys,.args=FCALL_Args},
{"fcall_ro","Invoke a read-only function","Depends on the function that is executed.","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SCRIPTING,FCALL_RO_History,FCALL_RO_tips,fcallroCommand,-3,CMD_NOSCRIPT|CMD_SKIP_MONITOR|CMD_NO_MANDATORY_KEYS|CMD_STALE,ACL_CATEGORY_SCRIPTING,{{"We cannot tell how the keys will be used so we assume the worst, RO and ACCESS",CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},functionGetKeys,.args=FCALL_RO_Args},
{"fcall_ro","Invoke a read-only function","Depends on the function that is executed.","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SCRIPTING,FCALL_RO_History,FCALL_RO_tips,fcallroCommand,-3,CMD_NOSCRIPT|CMD_SKIP_MONITOR|CMD_NO_MANDATORY_KEYS|CMD_STALE|CMD_READONLY,ACL_CATEGORY_SCRIPTING,{{"We cannot tell how the keys will be used so we assume the worst, RO and ACCESS",CMD_KEY_RO|CMD_KEY_ACCESS,KSPEC_BS_INDEX,.bs.index={2},KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},functionGetKeys,.args=FCALL_RO_Args},
{"function","A container for function commands","Depends on subcommand.","7.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SCRIPTING,FUNCTION_History,FUNCTION_tips,NULL,-2,0,0,.subcommands=FUNCTION_Subcommands},
{"script","A container for Lua scripts management commands","Depends on subcommand.","2.6.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SCRIPTING,SCRIPT_History,SCRIPT_tips,NULL,-2,0,0,.subcommands=SCRIPT_Subcommands},
/* sentinel */
@ -7269,7 +7309,7 @@ struct redisCommand redisCommandTable[] = {
{"psync","Internal command used for replication",NULL,"2.8.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SERVER,PSYNC_History,PSYNC_tips,syncCommand,-3,CMD_NO_ASYNC_LOADING|CMD_ADMIN|CMD_NO_MULTI|CMD_NOSCRIPT,0,.args=PSYNC_Args},
{"replconf","An internal command for configuring the replication stream","O(1)","3.0.0",CMD_DOC_SYSCMD,NULL,NULL,COMMAND_GROUP_SERVER,REPLCONF_History,REPLCONF_tips,replconfCommand,-1,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_ALLOW_BUSY,0},
{"replicaof","Make the server a replica of another instance, or promote it as master.","O(1)","5.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SERVER,REPLICAOF_History,REPLICAOF_tips,replicaofCommand,3,CMD_NO_ASYNC_LOADING|CMD_ADMIN|CMD_NOSCRIPT|CMD_STALE,0,.args=REPLICAOF_Args},
{"restore-asking","An internal command for migrating keys in a cluster","O(1) to create the new key and additional O(N*M) to reconstruct the serialized value, where N is the number of Redis objects composing the value and M their average size. For small string values the time complexity is thus O(1)+O(1*M) where M is small, so simply O(1). However for sorted set values the complexity is O(N*M*log(N)) because inserting values into sorted sets is O(log(N)).","3.0.0",CMD_DOC_SYSCMD,NULL,NULL,COMMAND_GROUP_SERVER,RESTORE_ASKING_History,RESTORE_ASKING_tips,restoreCommand,-4,CMD_WRITE|CMD_DENYOOM|CMD_ASKING,ACL_CATEGORY_KEYSPACE|ACL_CATEGORY_DANGEROUS,{{NULL,CMD_KEY_OW|CMD_KEY_UPDATE,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}}},
{"restore-asking","An internal command for migrating keys in a cluster","O(1) to create the new key and additional O(N*M) to reconstruct the serialized value, where N is the number of Redis objects composing the value and M their average size. For small string values the time complexity is thus O(1)+O(1*M) where M is small, so simply O(1). However for sorted set values the complexity is O(N*M*log(N)) because inserting values into sorted sets is O(log(N)).","3.0.0",CMD_DOC_SYSCMD,NULL,NULL,COMMAND_GROUP_SERVER,RESTORE_ASKING_History,RESTORE_ASKING_tips,restoreCommand,-4,CMD_WRITE|CMD_DENYOOM|CMD_ASKING,ACL_CATEGORY_KEYSPACE|ACL_CATEGORY_DANGEROUS,{{NULL,CMD_KEY_OW|CMD_KEY_UPDATE,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={0,1,0}}},.args=RESTORE_ASKING_Args},
{"role","Return the role of the instance in the context of replication","O(1)","2.8.12",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SERVER,ROLE_History,ROLE_tips,roleCommand,1,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_FAST|CMD_SENTINEL,ACL_CATEGORY_ADMIN|ACL_CATEGORY_DANGEROUS},
{"save","Synchronously save the dataset to disk","O(N) where N is the total number of keys in all databases","1.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SERVER,SAVE_History,SAVE_tips,saveCommand,1,CMD_NO_ASYNC_LOADING|CMD_ADMIN|CMD_NOSCRIPT|CMD_NO_MULTI,0},
{"shutdown","Synchronously save the dataset to disk and then shut down the server","O(N) when saving, where N is the total number of keys in all databases when saving data, otherwise O(1)","1.0.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_SERVER,SHUTDOWN_History,SHUTDOWN_tips,shutdownCommand,-1,CMD_ADMIN|CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_NO_MULTI|CMD_SENTINEL|CMD_ALLOW_BUSY,0,.args=SHUTDOWN_Args},
@ -7376,6 +7416,6 @@ struct redisCommand redisCommandTable[] = {
{"exec","Execute all commands issued after MULTI","Depends on commands in the transaction","1.2.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_TRANSACTIONS,EXEC_History,EXEC_tips,execCommand,1,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_SKIP_SLOWLOG,ACL_CATEGORY_TRANSACTION},
{"multi","Mark the start of a transaction block","O(1)","1.2.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_TRANSACTIONS,MULTI_History,MULTI_tips,multiCommand,1,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_FAST|CMD_ALLOW_BUSY,ACL_CATEGORY_TRANSACTION},
{"unwatch","Forget about all watched keys","O(1)","2.2.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_TRANSACTIONS,UNWATCH_History,UNWATCH_tips,unwatchCommand,1,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_FAST|CMD_ALLOW_BUSY,ACL_CATEGORY_TRANSACTION},
{"watch","Watch the given keys to determine execution of the MULTI/EXEC block","O(1) for every key.","2.2.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_TRANSACTIONS,WATCH_History,WATCH_tips,watchCommand,-2,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_FAST|CMD_ALLOW_BUSY,ACL_CATEGORY_TRANSACTION,{{NULL,0,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={-1,1,0}}},.args=WATCH_Args},
{"watch","Watch the given keys to determine execution of the MULTI/EXEC block","O(1) for every key.","2.2.0",CMD_DOC_NONE,NULL,NULL,COMMAND_GROUP_TRANSACTIONS,WATCH_History,WATCH_tips,watchCommand,-2,CMD_NOSCRIPT|CMD_LOADING|CMD_STALE|CMD_FAST|CMD_ALLOW_BUSY,ACL_CATEGORY_TRANSACTION,{{NULL,CMD_KEY_RO,KSPEC_BS_INDEX,.bs.index={1},KSPEC_FK_RANGE,.fk.range={-1,1,0}}},.args=WATCH_Args},
{0}
};

2
src/commands/bitfield.json
View File

@ -46,7 +46,7 @@
{
"name": "operation",
"type": "oneof",
"multiple": "true",
"multiple": true,
"arguments": [
{
"token": "GET",

5
src/commands/bitfield_ro.json
View File

@ -3,7 +3,7 @@
"summary": "Perform arbitrary bitfield integer operations on strings. Read-only variant of BITFIELD",
"complexity": "O(1) for each subcommand specified",
"group": "bitmap",
"since": "6.2.0",
"since": "6.0.0",
"arity": -2,
"function": "bitfieldroCommand",
"command_flags": [
@ -43,7 +43,8 @@
"token": "GET",
"name": "encoding_offset",
"type": "block",
"multiple": "true",
"multiple": true,
"multiple_token": true,
"arguments": [
{
"name": "encoding",

3
src/commands/eval_ro.json
View File

@ -11,7 +11,8 @@
"NOSCRIPT",
"SKIP_MONITOR",
"NO_MANDATORY_KEYS",
"STALE"
"STALE",
"READONLY"
],
"acl_categories": [
"SCRIPTING"

3
src/commands/evalsha_ro.json
View File

@ -11,7 +11,8 @@
"NOSCRIPT",
"SKIP_MONITOR",
"NO_MANDATORY_KEYS",
"STALE"
"STALE",
"READONLY"
],
"acl_categories": [
"SCRIPTING"

3
src/commands/fcall_ro.json
View File

@ -11,7 +11,8 @@
"NOSCRIPT",
"SKIP_MONITOR",
"NO_MANDATORY_KEYS",
"STALE"
"STALE",
"READONLY"
],
"acl_categories": [
"SCRIPTING"

2
src/commands/lcs.json
View File

@ -41,7 +41,7 @@
{
"name": "key2",
"type": "key",
"key_spec_index": 1
"key_spec_index": 0
},
{
"name": "len",

13
src/commands/pfdebug.json
View File

@ -4,7 +4,7 @@
"complexity": "N/A",
"group": "hyperloglog",
"since": "2.8.9",
"arity": -3,
"arity": 3,
"function": "pfdebugCommand",
"doc_flags": [
"SYSCMD"
@ -36,6 +36,17 @@
}
}
}
],
"arguments": [
{
"name": "subcommand",
"type": "string"
},
{
"name": "key",
"type": "key",
"key_spec_index": 0
}
]
}
}

2
src/commands/pubsub-shardchannels.json
View File

@ -1,7 +1,7 @@
{
"SHARDCHANNELS": {
"summary": "List active shard channels",
"complexity": "O(N) where N is the number of active shard channels, and assuming constant time pattern matching (relatively short channels).",
"complexity": "O(N) where N is the number of active shard channels, and assuming constant time pattern matching (relatively short shard channels).",
"group": "pubsub",
"since": "7.0.0",
"arity": -2,

4
src/commands/pubsub-shardnumsub.json
View File

@ -1,7 +1,7 @@
{
"SHARDNUMSUB": {
"summary": "Get the count of subscribers for shard channels",
"complexity": "O(N) for the SHARDNUMSUB subcommand, where N is the number of requested channels",
"complexity": "O(N) for the SHARDNUMSUB subcommand, where N is the number of requested shard channels",
"group": "pubsub",
"since": "7.0.0",
"arity": -2,
@ -14,7 +14,7 @@
],
"arguments": [
{
"name": "channel",
"name": "shardchannel",
"type": "string",
"optional": true,
"multiple": true

2
src/commands/rename.json
View File

@ -62,7 +62,7 @@
{
"name": "newkey",
"type": "key",
"key_spec_index": 0
"key_spec_index": 1
}
]
}

2
src/commands/renamenx.json
View File

@ -67,7 +67,7 @@
{
"name": "newkey",
"type": "key",
"key_spec_index": 0
"key_spec_index": 1
}
]
}

57
src/commands/restore-asking.json
View File

@ -6,6 +6,20 @@
"since": "3.0.0",
"arity": -4,
"function": "restoreCommand",
"history": [
[
"3.0.0",
"Added the `REPLACE` modifier."
],
[
"5.0.0",
"Added the `ABSTTL` modifier."
],
[
"5.0.0",
"Added the `IDLETIME` and `FREQ` options."
]
],
"doc_flags": [
"SYSCMD"
],
@ -37,6 +51,49 @@
}
}
}
],
"arguments": [
{
"name": "key",
"type": "key",
"key_spec_index": 0
},
{
"name": "ttl",
"type": "integer"
},
{
"name": "serialized-value",
"type": "string"
},
{
"name": "replace",
"token": "REPLACE",
"type": "pure-token",
"optional": true,
"since": "3.0.0"
},
{
"name": "absttl",
"token": "ABSTTL",
"type": "pure-token",
"optional": true,
"since": "5.0.0"
},
{
"token": "IDLETIME",
"name": "seconds",
"type": "integer",
"optional": true,
"since": "5.0.0"
},
{
"token": "FREQ",
"name": "frequency",
"type": "integer",
"optional": true,
"since": "5.0.0"
}
]
}
}

27
src/commands/sentinel-slaves.json Normal file
View File

@ -0,0 +1,27 @@
{
"SLAVES": {
"summary": "List the monitored slaves",
"complexity": "O(N) where N is the number of slaves",
"group": "sentinel",
"since": "2.8.0",
"arity": 3,
"container": "SENTINEL",
"function": "sentinelCommand",
"deprecated_since": "5.0.0",
"replaced_by": "`SENTINEL REPLICAS`",
"doc_flags": [
"DEPRECATED"
],
"command_flags": [
"ADMIN",
"SENTINEL",
"ONLY_SENTINEL"
],
"arguments": [
{
"name": "master-name",
"type": "string"
}
]
}
}

2
src/commands/spublish.json
View File

@ -15,7 +15,7 @@
],
"arguments": [
{
"name": "channel",
"name": "shardchannel",
"type": "string"
},
{

2
src/commands/ssubscribe.json
View File

@ -14,7 +14,7 @@
],
"arguments": [
{
"name": "channel",
"name": "shardchannel",
"type": "string",
"multiple": true
}

4
src/commands/sunsubscribe.json
View File

@ -1,7 +1,7 @@
{
"SUNSUBSCRIBE": {
"summary": "Stop listening for messages posted to the given shard channels",
"complexity": "O(N) where N is the number of clients already subscribed to a channel.",
"complexity": "O(N) where N is the number of clients already subscribed to a shard channel.",
"group": "pubsub",
"since": "7.0.0",
"arity": -1,
@ -14,7 +14,7 @@
],
"arguments": [
{
"name": "channel",
"name": "shardchannel",
"type": "string",
"optional": true,
"multiple": true

3
src/commands/watch.json
View File

@ -18,6 +18,9 @@
],
"key_specs": [
{
"flags": [
"RO"
],
"begin_search": {
"index": {
"pos": 1

4
src/commands/xinfo-stream.json
View File

@ -7,6 +7,10 @@
"arity": -3,
"container": "XINFO",
"history": [
[
"6.0.0",
"Added the `FULL` modifier."
],
[
"7.0.0",
"Added the `max-deleted-entry-id`, `entries-added`, `recorded-first-entry-id`, `entries-read` and `lag` fields"

93
src/config.c
View File

@ -311,18 +311,21 @@ int configEnumGetValue(configEnum *ce, sds *argv, int argc, int bitflags) {
/* Get enum name/s from value. If no matches are found "unknown" is returned. */
static sds configEnumGetName(configEnum *ce, int values, int bitflags) {
sds names = NULL;
int matches = 0;
int unmatched = values;
for( ; ce->name != NULL; ce++) {
if (values == ce->val) { /* Short path for perfect match */
sdsfree(names);
return sdsnew(ce->name);
}
if (bitflags && (values & ce->val)) {
/* Note: for bitflags, we want them sorted from high to low, so that if there are several / partially
* overlapping entries, we'll prefer the ones matching more bits. */
if (bitflags && ce->val && ce->val == (unmatched & ce->val)) {
names = names ? sdscatfmt(names, " %s", ce->name) : sdsnew(ce->name);
matches |= ce->val;
unmatched &= ~ce->val;
}
}
if (!names || values != matches) {
if (!names || unmatched) {
sdsfree(names);
return sdsnew("unknown");
}
@ -484,9 +487,23 @@ void loadServerConfigFromString(char *config) {
err = "wrong number of arguments";
goto loaderr;
}
/* Set config using all arguments that follows */
if (!config->interface.set(config, &argv[1], argc-1, &err)) {
goto loaderr;
if ((config->flags & MULTI_ARG_CONFIG) && argc == 2 && sdslen(argv[1])) {
/* For MULTI_ARG_CONFIGs, if we only have one argument, try to split it by spaces.
* Only if the argument is not empty, otherwise something like --save "" will fail.
* So that we can support something like --config "arg1 arg2 arg3". */
sds *new_argv;
int new_argc;
new_argv = sdssplitargs(argv[1], &new_argc);
if (!config->interface.set(config, new_argv, new_argc, &err)) {
goto loaderr;
}
sdsfreesplitres(new_argv, new_argc);
} else {
/* Set config using all arguments that follows */
if (!config->interface.set(config, &argv[1], argc-1, &err)) {
goto loaderr;
}
}
sdsfreesplitres(argv,argc);
@ -1125,10 +1142,21 @@ struct rewriteConfigState *rewriteConfigReadOldFile(char *path) {
/* Not a comment, split into arguments. */
argv = sdssplitargs(line,&argc);
if (argv == NULL || (!server.sentinel_mode && !lookupConfig(argv[0]))) {
/* Apparently the line is unparsable for some reason, for
* instance it may have unbalanced quotes, or may contain a
* config that doesn't exist anymore. Load it as a comment. */
if (argv == NULL ||
(!lookupConfig(argv[0]) &&
/* The following is a list of config features that are only supported in
* config file parsing and are not recognized by lookupConfig */
strcasecmp(argv[0],"include") &&
strcasecmp(argv[0],"rename-command") &&
strcasecmp(argv[0],"user") &&
strcasecmp(argv[0],"loadmodule") &&
strcasecmp(argv[0],"sentinel")))
{
/* The line is either unparsable for some reason, for
* instance it may have unbalanced quotes, may contain a
* config that doesn't exist anymore, for instance a module that got
* unloaded. Load it as a comment. */
sds aux = sdsnew("# ??? ");
aux = sdscatsds(aux,line);
if (argv) sdsfreesplitres(argv, argc);
@ -1143,18 +1171,13 @@ struct rewriteConfigState *rewriteConfigReadOldFile(char *path) {
* Append the line and populate the option -> line numbers map. */
rewriteConfigAppendLine(state,line);
/* Translate options using the word "slave" to the corresponding name
* "replica", before adding such option to the config name -> lines
* mapping. */
char *p = strstr(argv[0],"slave");
if (p) {
sds alt = sdsempty();
alt = sdscatlen(alt,argv[0],p-argv[0]);
alt = sdscatlen(alt,"replica",7);
alt = sdscatlen(alt,p+5,strlen(p+5));
/* If this is a alias config, replace it with the original name. */
standardConfig *s_conf = lookupConfig(argv[0]);
if (s_conf && s_conf->flags & ALIAS_CONFIG) {
sdsfree(argv[0]);
argv[0] = alt;
argv[0] = sdsnew(s_conf->alias);
}
/* If this is sentinel config, we use sentinel "sentinel <config>" as option
to avoid messing up the sequence. */
if (server.sentinel_mode && argc > 1 && !strcasecmp(argv[0],"sentinel")) {
@ -1796,7 +1819,7 @@ static int boolConfigSet(standardConfig *config, sds *argv, int argc, const char
*(config->data.yesno.config) = yn;
return 1;
}
return 2;
return (config->flags & VOLATILE_CONFIG) ? 1 : 2;
}
static sds boolConfigGet(standardConfig *config) {
@ -1838,7 +1861,7 @@ static int stringConfigSet(standardConfig *config, sds *argv, int argc, const ch
zfree(prev);
return 1;
}
return 2;
return (config->flags & VOLATILE_CONFIG) ? 1 : 2;
}
static sds stringConfigGet(standardConfig *config) {
@ -1875,7 +1898,7 @@ static int sdsConfigSet(standardConfig *config, sds *argv, int argc, const char
return 1;
}
if (config->flags & MODULE_CONFIG && prev) sdsfree(prev);
return 2;
return (config->flags & VOLATILE_CONFIG) ? 1 : 2;
}
static sds sdsConfigGet(standardConfig *config) {
@ -1959,7 +1982,7 @@ static int enumConfigSet(standardConfig *config, sds *argv, int argc, const char
*(config->data.enumd.config) = enumval;
return 1;
}
return 2;
return (config->flags & VOLATILE_CONFIG) ? 1 : 2;
}
static sds enumConfigGet(standardConfig *config) {
@ -2155,7 +2178,7 @@ static int numericConfigSet(standardConfig *config, sds *argv, int argc, const c
return setNumericType(config, ll, err);
}
return 2;
return (config->flags & VOLATILE_CONFIG) ? 1 : 2;
}
static sds numericConfigGet(standardConfig *config) {
@ -2592,8 +2615,10 @@ static int setConfigSaveOption(standardConfig *config, sds *argv, int argc, cons
int j;
/* Special case: treat single arg "" as zero args indicating empty save configuration */
if (argc == 1 && !strcasecmp(argv[0],""))
if (argc == 1 && !strcasecmp(argv[0],"")) {
resetServerSaveParams();
argc = 0;
}
/* Perform sanity check before setting the new config:
* - Even number of args
@ -3078,15 +3103,15 @@ standardConfig static_configs[] = {
createEnumConfig("tls-auth-clients", NULL, MODIFIABLE_CONFIG, tls_auth_clients_enum, server.tls_auth_clients, TLS_CLIENT_AUTH_YES, NULL, NULL),
createBoolConfig("tls-prefer-server-ciphers", NULL, MODIFIABLE_CONFIG, server.tls_ctx_config.prefer_server_ciphers, 0, NULL, applyTlsCfg),
createBoolConfig("tls-session-caching", NULL, MODIFIABLE_CONFIG, server.tls_ctx_config.session_caching, 1, NULL, applyTlsCfg),
createStringConfig("tls-cert-file", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.cert_file, NULL, NULL, applyTlsCfg),
createStringConfig("tls-key-file", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.key_file, NULL, NULL, applyTlsCfg),
createStringConfig("tls-cert-file", NULL, VOLATILE_CONFIG | MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.cert_file, NULL, NULL, applyTlsCfg),
createStringConfig("tls-key-file", NULL, VOLATILE_CONFIG | MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.key_file, NULL, NULL, applyTlsCfg),
createStringConfig("tls-key-file-pass", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.key_file_pass, NULL, NULL, applyTlsCfg),
createStringConfig("tls-client-cert-file", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.client_cert_file, NULL, NULL, applyTlsCfg),
createStringConfig("tls-client-key-file", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.client_key_file, NULL, NULL, applyTlsCfg),
createStringConfig("tls-client-cert-file", NULL, VOLATILE_CONFIG | MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.client_cert_file, NULL, NULL, applyTlsCfg),
createStringConfig("tls-client-key-file", NULL, VOLATILE_CONFIG | MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.client_key_file, NULL, NULL, applyTlsCfg),
createStringConfig("tls-client-key-file-pass", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.client_key_file_pass, NULL, NULL, applyTlsCfg),
createStringConfig("tls-dh-params-file", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.dh_params_file, NULL, NULL, applyTlsCfg),
createStringConfig("tls-ca-cert-file", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.ca_cert_file, NULL, NULL, applyTlsCfg),
createStringConfig("tls-ca-cert-dir", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.ca_cert_dir, NULL, NULL, applyTlsCfg),
createStringConfig("tls-dh-params-file", NULL, VOLATILE_CONFIG | MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.dh_params_file, NULL, NULL, applyTlsCfg),
createStringConfig("tls-ca-cert-file", NULL, VOLATILE_CONFIG | MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.ca_cert_file, NULL, NULL, applyTlsCfg),
createStringConfig("tls-ca-cert-dir", NULL, VOLATILE_CONFIG | MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.ca_cert_dir, NULL, NULL, applyTlsCfg),
createStringConfig("tls-protocols", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.protocols, NULL, NULL, applyTlsCfg),
createStringConfig("tls-ciphers", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.ciphers, NULL, NULL, applyTlsCfg),
createStringConfig("tls-ciphersuites", NULL, MODIFIABLE_CONFIG, EMPTY_STRING_IS_NULL, server.tls_ctx_config.ciphersuites, NULL, NULL, applyTlsCfg),

27
src/db.c
View File

@ -602,18 +602,11 @@ void flushAllDataAndResetRDB(int flags) {
server.dirty += emptyData(-1,flags,NULL);
if (server.child_type == CHILD_TYPE_RDB) killRDBChild();
if (server.saveparamslen > 0) {
/* Normally rdbSave() will reset dirty, but we don't want this here
* as otherwise FLUSHALL will not be replicated nor put into the AOF. */
int saved_dirty = server.dirty;
rdbSaveInfo rsi, *rsiptr;
rsiptr = rdbPopulateSaveInfo(&rsi);
rdbSave(SLAVE_REQ_NONE,server.rdb_filename,rsiptr);
server.dirty = saved_dirty;
}
/* Without that extra dirty++, when db was already empty, FLUSHALL will
* not be replicated nor put into the AOF. */
server.dirty++;
#if defined(USE_JEMALLOC)
/* jemalloc 5 doesn't release pages back to the OS when there's no traffic.
* for large databases, flushdb blocks for long anyway, so a bit more won't
@ -632,7 +625,13 @@ void flushdbCommand(client *c) {
if (getFlushCommandFlags(c,&flags) == C_ERR) return;
/* flushdb should not flush the functions */
server.dirty += emptyData(c->db->id,flags | EMPTYDB_NOFUNCTIONS,NULL);
/* Without the forceCommandPropagation, when DB was already empty,
* FLUSHDB will not be replicated nor put into the AOF. */
forceCommandPropagation(c, PROPAGATE_REPL | PROPAGATE_AOF);
addReply(c,shared.ok);
#if defined(USE_JEMALLOC)
/* jemalloc 5 doesn't release pages back to the OS when there's no traffic.
* for large databases, flushdb blocks for long anyway, so a bit more won't
@ -650,6 +649,11 @@ void flushallCommand(client *c) {
if (getFlushCommandFlags(c,&flags) == C_ERR) return;
/* flushall should not flush the functions */
flushAllDataAndResetRDB(flags | EMPTYDB_NOFUNCTIONS);
/* Without the forceCommandPropagation, when DBs were already empty,
* FLUSHALL will not be replicated nor put into the AOF. */
forceCommandPropagation(c, PROPAGATE_REPL | PROPAGATE_AOF);
addReply(c,shared.ok);
}
@ -1080,7 +1084,7 @@ void shutdownCommand(client *c) {
return;
}
if (!(flags & SHUTDOWN_NOSAVE) && scriptIsTimedout()) {
if (!(flags & SHUTDOWN_NOSAVE) && isInsideYieldingLongCommand()) {
/* Script timed out. Shutdown allowed only with the NOSAVE flag. See
* also processCommand where these errors are returned. */
if (server.busy_module_yield_flags && server.busy_module_yield_reply) {
@ -1878,7 +1882,6 @@ int getKeysFromCommandWithSpecs(struct redisCommand *cmd, robj **argv, int argc,
/* Flags indicating that we have a getkeys callback */
int has_module_getkeys = cmd->flags & CMD_MODULE_GETKEYS;
int has_native_getkeys = !(cmd->flags & CMD_MODULE) && cmd->getkeys_proc;
/* The key-spec that's auto generated by RM_CreateCommand sets VARIABLE_FLAGS since no flags are given.
* If the module provides getkeys callback, we'll prefer it, but if it didn't, we'll use key-spec anyway. */
@ -1900,14 +1903,14 @@ int getKeysFromCommandWithSpecs(struct redisCommand *cmd, robj **argv, int argc,
/* We use native getkeys as a last resort, since not all these native getkeys provide
* flags properly (only the ones that correspond to INVALID, INCOMPLETE or VARIABLE_FLAGS do.*/
if (has_native_getkeys)
if (cmd->getkeys_proc)
return cmd->getkeys_proc(cmd,argv,argc,result);
return 0;
}
/* This function returns a sanity check if the command may have keys. */
int doesCommandHaveKeys(struct redisCommand *cmd) {
return (!(cmd->flags & CMD_MODULE) && cmd->getkeys_proc) || /* has getkeys_proc (non modules) */
return cmd->getkeys_proc || /* has getkeys_proc (non modules) */
(cmd->flags & CMD_MODULE_GETKEYS) || /* module with GETKEYS */
(getAllKeySpecsFlags(cmd, 1) & CMD_KEY_NOT_KEY); /* has at least one key-spec not marked as NOT_KEY */
}
@ -2056,7 +2059,7 @@ int getKeysUsingLegacyRangeSpec(struct redisCommand *cmd, robj **argv, int argc,
int getKeysFromCommand(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result) {
if (cmd->flags & CMD_MODULE_GETKEYS) {
return moduleGetCommandKeysViaAPI(cmd,argv,argc,result);
} else if (!(cmd->flags & CMD_MODULE) && cmd->getkeys_proc) {
} else if (cmd->getkeys_proc) {
return cmd->getkeys_proc(cmd,argv,argc,result);
} else {
return getKeysUsingLegacyRangeSpec(cmd,argv,argc,result);

2
src/debug.c
View File

@ -2014,7 +2014,9 @@ void bugReportEnd(int killViaSignal, int sig) {
"\n=== REDIS BUG REPORT END. Make sure to include from START to END. ===\n\n"
" Please report the crash by opening an issue on github:\n\n"
" http://github.com/redis/redis/issues\n\n"
" If a Redis module was involved, please open in the module's repo instead.\n\n"
" Suspect RAM error? Use redis-server --test-memory to verify it.\n\n"
" Some other issues could be detected by redis-server --check-system\n"
);
/* free(messages); Don't call free() with possibly corrupted memory. */

195
src/eval.c
View File

@ -285,6 +285,124 @@ void scriptingReset(int async) {
* EVAL and SCRIPT commands implementation
* ------------------------------------------------------------------------- */
static void evalCalcFunctionName(int evalsha, sds script, char *out_funcname) {
/* We obtain the script SHA1, then check if this function is already
* defined into the Lua state */
out_funcname[0] = 'f';
out_funcname[1] = '_';
if (!evalsha) {
/* Hash the code if this is an EVAL call */
sha1hex(out_funcname+2,script,sdslen(script));
} else {
/* We already have the SHA if it is an EVALSHA */
int j;
char *sha = script;
/* Convert to lowercase. We don't use tolower since the function
* managed to always show up in the profiler output consuming
* a non trivial amount of time. */
for (j = 0; j < 40; j++)
out_funcname[j+2] = (sha[j] >= 'A' && sha[j] <= 'Z') ?
sha[j]+('a'-'A') : sha[j];
out_funcname[42] = '\0';
}
}
/* Helper function to try and extract shebang flags from the script body.
* If no shebang is found, return with success and COMPAT mode flag.
* The err arg is optional, can be used to get a detailed error string.
* The out_shebang_len arg is optional, can be used to trim the shebang from the script.
* Returns C_OK on success, and C_ERR on error. */
int evalExtractShebangFlags(sds body, uint64_t *out_flags, ssize_t *out_shebang_len, sds *err) {
ssize_t shebang_len = 0;
uint64_t script_flags = SCRIPT_FLAG_EVAL_COMPAT_MODE;
if (!strncmp(body, "#!", 2)) {
int numparts,j;
char *shebang_end = strchr(body, '\n');
if (shebang_end == NULL) {
if (err)
*err = sdsnew("Invalid script shebang");
return C_ERR;
}
shebang_len = shebang_end - body;
sds shebang = sdsnewlen(body, shebang_len);
sds *parts = sdssplitargs(shebang, &numparts);
sdsfree(shebang);
if (!parts || numparts == 0) {
if (err)
*err = sdsnew("Invalid engine in script shebang");
sdsfreesplitres(parts, numparts);
return C_ERR;
}
/* Verify lua interpreter was specified */
if (strcmp(parts[0], "#!lua")) {
if (err)
*err = sdscatfmt(sdsempty(), "Unexpected engine in script shebang: %s", parts[0]);
sdsfreesplitres(parts, numparts);
return C_ERR;
}
script_flags &= ~SCRIPT_FLAG_EVAL_COMPAT_MODE;
for (j = 1; j < numparts; j++) {
if (!strncmp(parts[j], "flags=", 6)) {
sdsrange(parts[j], 6, -1);
int numflags, jj;
sds *flags = sdssplitlen(parts[j], sdslen(parts[j]), ",", 1, &numflags);
for (jj = 0; jj < numflags; jj++) {
scriptFlag *sf;
for (sf = scripts_flags_def; sf->flag; sf++) {
if (!strcmp(flags[jj], sf->str)) break;
}
if (!sf->flag) {
if (err)
*err = sdscatfmt(sdsempty(), "Unexpected flag in script shebang: %s", flags[jj]);
sdsfreesplitres(flags, numflags);
sdsfreesplitres(parts, numparts);
return C_ERR;
}
script_flags |= sf->flag;
}
sdsfreesplitres(flags, numflags);
} else {
/* We only support function flags options for lua scripts */
if (err)
*err = sdscatfmt(sdsempty(), "Unknown lua shebang option: %s", parts[j]);
sdsfreesplitres(parts, numparts);
return C_ERR;
}
}
sdsfreesplitres(parts, numparts);
}
if (out_shebang_len)
*out_shebang_len = shebang_len;
*out_flags = script_flags;
return C_OK;
}
/* Try to extract command flags if we can, returns the modified flags.
* Note that it does not guarantee the command arguments are right. */
uint64_t evalGetCommandFlags(client *c, uint64_t cmd_flags) {
char funcname[43];
int evalsha = c->cmd->proc == evalShaCommand || c->cmd->proc == evalShaRoCommand;
if (evalsha && sdslen(c->argv[1]->ptr) != 40)
return cmd_flags;
evalCalcFunctionName(evalsha, c->argv[1]->ptr, funcname);
char *lua_cur_script = funcname + 2;
dictEntry *de = dictFind(lctx.lua_scripts, lua_cur_script);
uint64_t script_flags;
if (!de) {
if (evalsha)
return cmd_flags;
if (evalExtractShebangFlags(c->argv[1]->ptr, &script_flags, NULL, NULL) == C_ERR)
return cmd_flags;
} else {
luaScript *l = dictGetVal(de);
script_flags = l->flags;
}
if (script_flags & SCRIPT_FLAG_EVAL_COMPAT_MODE)
return cmd_flags;
return scriptFlagsToCmdFlags(cmd_flags, script_flags);
}
/* Define a Lua function with the specified body.
* The function name will be generated in the following form:
*
@ -305,7 +423,7 @@ void scriptingReset(int async) {
sds luaCreateFunction(client *c, robj *body) {
char funcname[43];
dictEntry *de;
uint64_t script_flags = SCRIPT_FLAG_EVAL_COMPAT_MODE;
uint64_t script_flags;
funcname[0] = 'f';
funcname[1] = '_';
@ -317,56 +435,10 @@ sds luaCreateFunction(client *c, robj *body) {
/* Handle shebang header in script code */
ssize_t shebang_len = 0;
if (!strncmp(body->ptr, "#!", 2)) {
int numparts,j;
char *shebang_end = strchr(body->ptr, '\n');
if (shebang_end == NULL) {
addReplyError(c,"Invalid script shebang");
return NULL;
}
shebang_len = shebang_end - (char*)body->ptr;
sds shebang = sdsnewlen(body->ptr, shebang_len);
sds *parts = sdssplitargs(shebang, &numparts);
sdsfree(shebang);
if (!parts || numparts == 0) {
addReplyError(c,"Invalid engine in script shebang");
sdsfreesplitres(parts, numparts);
return NULL;
}
/* Verify lua interpreter was specified */
if (strcmp(parts[0], "#!lua")) {
addReplyErrorFormat(c,"Unexpected engine in script shebang: %s", parts[0]);
sdsfreesplitres(parts, numparts);
return NULL;
}
script_flags &= ~SCRIPT_FLAG_EVAL_COMPAT_MODE;
for (j = 1; j < numparts; j++) {
if (!strncmp(parts[j], "flags=", 6)) {
sdsrange(parts[j], 6, -1);
int numflags, jj;
sds *flags = sdssplitlen(parts[j], sdslen(parts[j]), ",", 1, &numflags);
for (jj = 0; jj < numflags; jj++) {
scriptFlag *sf;
for (sf = scripts_flags_def; sf->flag; sf++) {
if (!strcmp(flags[jj], sf->str)) break;
}
if (!sf->flag) {
addReplyErrorFormat(c,"Unexpected flag in script shebang: %s", flags[jj]);
sdsfreesplitres(flags, numflags);
sdsfreesplitres(parts, numparts);
return NULL;
}
script_flags |= sf->flag;
}
sdsfreesplitres(flags, numflags);
} else {
/* We only support function flags options for lua scripts */
addReplyErrorFormat(c,"Unknown lua shebang option: %s", parts[j]);
sdsfreesplitres(parts, numparts);
return NULL;
}
}
sdsfreesplitres(parts, numparts);
sds err = NULL;
if (evalExtractShebangFlags(body->ptr, &script_flags, &shebang_len, &err) == C_ERR) {
addReplyErrorSds(c, err);
return NULL;
}
/* Note that in case of a shebang line we skip it but keep the line feed to conserve the user's line numbers */
@ -430,26 +502,7 @@ void evalGenericCommand(client *c, int evalsha) {
return;
}
/* We obtain the script SHA1, then check if this function is already
* defined into the Lua state */
funcname[0] = 'f';
funcname[1] = '_';
if (!evalsha) {
/* Hash the code if this is an EVAL call */
sha1hex(funcname+2,c->argv[1]->ptr,sdslen(c->argv[1]->ptr));
} else {
/* We already have the SHA if it is an EVALSHA */
int j;
char *sha = c->argv[1]->ptr;
/* Convert to lowercase. We don't use tolower since the function
* managed to always show up in the profiler output consuming
* a non trivial amount of time. */
for (j = 0; j < 40; j++)
funcname[j+2] = (sha[j] >= 'A' && sha[j] <= 'Z') ?
sha[j]+('a'-'A') : sha[j];
funcname[42] = '\0';
}
evalCalcFunctionName(evalsha, c->argv[1]->ptr, funcname);
/* Push the pcall error handler function on the stack. */
lua_getglobal(lua, "__redis__err__handler");

2
src/evict.c
View File

@ -481,7 +481,7 @@ void startEvictionTimeProc(void) {
static int isSafeToPerformEvictions(void) {
/* - There must be no script in timeout condition.
* - Nor we are loading data right now. */
if (scriptIsTimedout() || server.loading) return 0;
if (isInsideYieldingLongCommand() || server.loading) return 0;
/* By default replicas should ignore maxmemory
* and just be masters exact copies. */

8
src/expire.c
View File

@ -662,22 +662,22 @@ void expireGenericCommand(client *c, long long basetime, int unit) {
}
}
/* EXPIRE key seconds */
/* EXPIRE key seconds [ NX | XX | GT | LT] */
void expireCommand(client *c) {
expireGenericCommand(c,mstime(),UNIT_SECONDS);
}
/* EXPIREAT key time */
/* EXPIREAT key unix-time-seconds [ NX | XX | GT | LT] */
void expireatCommand(client *c) {
expireGenericCommand(c,0,UNIT_SECONDS);
}
/* PEXPIRE key milliseconds */
/* PEXPIRE key milliseconds [ NX | XX | GT | LT] */
void pexpireCommand(client *c) {
expireGenericCommand(c,mstime(),UNIT_MILLISECONDS);
}
/* PEXPIREAT key ms_time */
/* PEXPIREAT key unix-time-milliseconds [ NX | XX | GT | LT] */
void pexpireatCommand(client *c) {
expireGenericCommand(c,0,UNIT_MILLISECONDS);
}

19
src/functions.c
View File

@ -293,7 +293,7 @@ static void libraryUnlink(functionsLibCtx *lib_ctx, functionLibInfo* li) {
entry = dictUnlink(lib_ctx->libraries, li->name);
dictSetVal(lib_ctx->libraries, entry, NULL);
dictFreeUnlinkedEntry(lib_ctx->libraries, entry);
lib_ctx->cache_memory += libraryMallocSize(li);
lib_ctx->cache_memory -= libraryMallocSize(li);
/* update stats */
functionsLibEngineStats *stats = dictFetchValue(lib_ctx->engines_stats, li->ei->name);
@ -325,7 +325,7 @@ static void libraryLink(functionsLibCtx *lib_ctx, functionLibInfo* li) {
/* Takes all libraries from lib_ctx_src and add to lib_ctx_dst.
* On collision, if 'replace' argument is true, replace the existing library with the new one.
* Otherwise abort and leave 'lib_ctx_dst' and 'lib_ctx_src' untouched.
* Return C_OK on success and C_ERR if aborted. If C_ERR is retunred, set a relevant
* Return C_OK on success and C_ERR if aborted. If C_ERR is returned, set a relevant
* error message on the 'err' out parameter.
* */
static int libraryJoin(functionsLibCtx *functions_lib_ctx_dst, functionsLibCtx *functions_lib_ctx_src, int replace, sds *err) {
@ -599,10 +599,22 @@ void functionDeleteCommand(client *c) {
addReply(c, shared.ok);
}
/* FUNCTION KILL */
void functionKillCommand(client *c) {
scriptKill(c, 0);
}
/* Try to extract command flags if we can, returns the modified flags.
* Note that it does not guarantee the command arguments are right. */
uint64_t fcallGetCommandFlags(client *c, uint64_t cmd_flags) {
robj *function_name = c->argv[1];
functionInfo *fi = dictFetchValue(curr_functions_lib_ctx->functions, function_name->ptr);
if (!fi)
return cmd_flags;
uint64_t script_flags = fi->f_flags;
return scriptFlagsToCmdFlags(cmd_flags, script_flags);
}
static void fcallCommandGeneric(client *c, int ro) {
robj *function_name = c->argv[1];
functionInfo *fi = dictFetchValue(curr_functions_lib_ctx->functions, function_name->ptr);
@ -778,6 +790,7 @@ load_error:
}
}
/* FUNCTION FLUSH [ASYNC | SYNC] */
void functionFlushCommand(client *c) {
if (c->argc > 3) {
addReplySubcommandSyntaxError(c);
@ -803,6 +816,7 @@ void functionFlushCommand(client *c) {
addReply(c,shared.ok);
}
/* FUNCTION HELP */
void functionHelpCommand(client *c) {
const char *help[] = {
"LOAD <ENGINE NAME> <LIBRARY NAME> [REPLACE] [DESCRIPTION <LIBRARY DESCRIPTION>] <LIBRARY CODE>",
@ -960,6 +974,7 @@ sds functionsCreateWithLibraryCtx(sds code, int replace, sds* err, functionsLibC
old_li = dictFetchValue(lib_ctx->libraries, md.name);
if (old_li && !replace) {
old_li = NULL;
*err = sdscatfmt(sdsempty(), "Library '%S' already exists", md.name);
goto error;
}

24
src/help.h
View File

@ -1,4 +1,4 @@
/* Automatically generated by ./generate-command-help.rb, do not edit. */
/* Automatically generated by ./utils/generate-command-help.rb, do not edit. */
#ifndef __REDIS_HELP_H
#define __REDIS_HELP_H
@ -135,7 +135,7 @@ struct commandHelp {
15,
"3.2.0" },
{ "BITFIELD_RO",
"key GET encoding offset [encoding offset ...]",
"key GET encoding offset [GET encoding offset ...]",
"Perform arbitrary bitfield integer operations on strings. Read-only variant of BITFIELD",
15,
"6.2.0" },
@ -255,7 +255,7 @@ struct commandHelp {
8,
"2.6.9" },
{ "CLIENT TRACKING",
"ON|OFF [REDIRECT client-id] [PREFIX prefix [prefix ...]] [BCAST] [OPTIN] [OPTOUT] [NOLOOP]",
"ON|OFF [REDIRECT client-id] [PREFIX prefix [PREFIX prefix ...]] [BCAST] [OPTIN] [OPTOUT] [NOLOOP]",
"Enable or disable server assisted client side caching support",
8,
"6.0.0" },
@ -1025,7 +1025,7 @@ struct commandHelp {
9,
"4.0.0" },
{ "MODULE LOADEX",
"path [CONFIG name value [name value ...]] [ARGS arg [arg ...]]",
"path [CONFIG name value [CONFIG name value ...]] [ARGS arg [arg ...]]",
"Load a module with extended parameters",
9,
"7.0.0" },
@ -1120,7 +1120,7 @@ struct commandHelp {
11,
"2.8.9" },
{ "PFDEBUG",
"",
"subcommand key",
"Internal commands for debugging HyperLogLog values",
11,
"2.8.9" },
@ -1195,7 +1195,7 @@ struct commandHelp {
6,
"7.0.0" },
{ "PUBSUB SHARDNUMSUB",
"[channel [channel ...]]",
"[shardchannel [shardchannel ...]]",
"Get the count of subscribers for shard channels",
6,
"7.0.0" },
@ -1255,7 +1255,7 @@ struct commandHelp {
0,
"2.6.0" },
{ "RESTORE-ASKING",
"",
"key ttl serialized-value [REPLACE] [ABSTTL] [IDLETIME seconds] [FREQ frequency]",
"An internal command for migrating keys in a cluster",
9,
"3.0.0" },
@ -1450,12 +1450,12 @@ struct commandHelp {
3,
"1.0.0" },
{ "SORT",
"key [BY pattern] [LIMIT offset count] [GET pattern [pattern ...]] [ASC|DESC] [ALPHA] [STORE destination]",
"key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA] [STORE destination]",
"Sort the elements in a list, set or sorted set",
0,
"1.0.0" },
{ "SORT_RO",
"key [BY pattern] [LIMIT offset count] [GET pattern [pattern ...]] [ASC|DESC] [ALPHA]",
"key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA]",
"Sort the elements in a list, set or sorted set. Read-only variant of SORT.",
0,
"7.0.0" },
@ -1465,7 +1465,7 @@ struct commandHelp {
3,
"1.0.0" },
{ "SPUBLISH",
"channel message",
"shardchannel message",
"Post a message to a shard channel",
6,
"7.0.0" },
@ -1485,7 +1485,7 @@ struct commandHelp {
3,
"2.8.0" },
{ "SSUBSCRIBE",
"channel [channel ...]",
"shardchannel [shardchannel ...]",
"Listen for messages published to the given shard channels",
6,
"7.0.0" },
@ -1515,7 +1515,7 @@ struct commandHelp {
3,
"1.0.0" },
{ "SUNSUBSCRIBE",
"[channel [channel ...]]",
"[shardchannel [shardchannel ...]]",
"Stop listening for messages posted to the given shard channels",
6,
"7.0.0" },

9
src/hyperloglog.c
View File

@ -1495,8 +1495,13 @@ cleanup:
if (o) decrRefCount(o);
}
/* PFDEBUG <subcommand> <key> ... args ...
* Different debugging related operations about the HLL implementation. */
/* Different debugging related operations about the HLL implementation.
*
* PFDEBUG GETREG <key>
* PFDEBUG DECODE <key>
* PFDEBUG ENCODING <key>
* PFDEBUG TODENSE <key>
*/
void pfdebugCommand(client *c) {
char *cmd = c->argv[1]->ptr;
struct hllhdr *hdr;

33
src/latency.c
View File

@ -59,39 +59,6 @@ dictType latencyTimeSeriesDictType = {
/* ------------------------- Utility functions ------------------------------ */
#ifdef __linux__
#include <sys/prctl.h>
/* Returns 1 if Transparent Huge Pages support is enabled in the kernel.
* Otherwise (or if we are unable to check) 0 is returned. */
int THPIsEnabled(void) {
char buf[1024];
FILE *fp = fopen("/sys/kernel/mm/transparent_hugepage/enabled","r");
if (!fp) return 0;
if (fgets(buf,sizeof(buf),fp) == NULL) {
fclose(fp);
return 0;
}
fclose(fp);
return (strstr(buf,"[always]") != NULL) ? 1 : 0;
}
/* since linux-3.5, kernel supports to set the state of the "THP disable" flag
* for the calling thread. PR_SET_THP_DISABLE is defined in linux/prctl.h */
int THPDisable(void) {
int ret = -EINVAL;
if (!server.disable_thp)
return ret;
#ifdef PR_SET_THP_DISABLE
ret = prctl(PR_SET_THP_DISABLE, 1, 0, 0, 0);
#endif
return ret;
}
#endif
/* Report the amount of AnonHugePages in smap, in bytes. If the return
* value of the function is non-zero, the process is being targeted by
* THP support, and is likely to have memory usage / latency issues. */

2
src/latency.h
View File

@ -63,8 +63,6 @@ struct latencyStats {
void latencyMonitorInit(void);
void latencyAddSample(const char *event, mstime_t latency);
int THPIsEnabled(void);
int THPDisable(void);
/* Latency monitoring macros. */

163
src/module.c
View File

@ -56,6 +56,7 @@
#include "slowlog.h"
#include "rdb.h"
#include "monotonic.h"
#include "script.h"
#include "call_reply.h"
#include <dlfcn.h>
#include <sys/stat.h>
@ -355,6 +356,7 @@ typedef struct RedisModuleServerInfoData {
#define REDISMODULE_ARGV_SCRIPT_MODE (1<<6)
#define REDISMODULE_ARGV_NO_WRITES (1<<7)
#define REDISMODULE_ARGV_CALL_REPLIES_AS_ERRORS (1<<8)
#define REDISMODULE_ARGV_RESPECT_DENY_OOM (1<<9)
/* Determine whether Redis should signalModifiedKey implicitly.
* In case 'ctx' has no 'module' member (and therefore no module->options),
@ -781,7 +783,7 @@ void moduleCreateContext(RedisModuleCtx *out_ctx, RedisModule *module, int ctx_f
/* This Redis command binds the normal Redis command invocation with commands
* exported by modules. */
void RedisModuleCommandDispatcher(client *c) {
RedisModuleCommand *cp = (void*)(unsigned long)c->cmd->getkeys_proc;
RedisModuleCommand *cp = c->cmd->module_cmd;
RedisModuleCtx ctx;
moduleCreateContext(&ctx, cp->module, REDISMODULE_CTX_NONE);
@ -816,7 +818,7 @@ void RedisModuleCommandDispatcher(client *c) {
* the context in a way that the command can recognize this is a special
* "get keys" call by calling RedisModule_IsKeysPositionRequest(ctx). */
int moduleGetCommandKeysViaAPI(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result) {
RedisModuleCommand *cp = (void*)(unsigned long)cmd->getkeys_proc;
RedisModuleCommand *cp = cmd->module_cmd;
RedisModuleCtx ctx;
moduleCreateContext(&ctx, cp->module, REDISMODULE_CTX_KEYS_POS_REQUEST);
@ -836,7 +838,7 @@ int moduleGetCommandKeysViaAPI(struct redisCommand *cmd, robj **argv, int argc,
* moduleGetCommandKeysViaAPI, for modules that declare "getchannels-api"
* during registration. Unlike keys, this is the only way to declare channels. */
int moduleGetCommandChannelsViaAPI(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result) {
RedisModuleCommand *cp = (void*)(unsigned long)cmd->getkeys_proc;
RedisModuleCommand *cp = cmd->module_cmd;
RedisModuleCtx ctx;
moduleCreateContext(&ctx, cp->module, REDISMODULE_CTX_CHANNELS_POS_REQUEST);
@ -1130,10 +1132,7 @@ RedisModuleCommand *moduleCreateCommandProxy(struct RedisModule *module, sds dec
/* Create a command "proxy", which is a structure that is referenced
* in the command table, so that the generic command that works as
* binding between modules and Redis, can know what function to call
* and what the module is.
*
* Note that we use the Redis command table 'getkeys_proc' in order to
* pass a reference to the command proxy structure. */
* and what the module is. */
cp = zcalloc(sizeof(*cp));
cp->module = module;
cp->func = cmdfunc;
@ -1143,7 +1142,7 @@ RedisModuleCommand *moduleCreateCommandProxy(struct RedisModule *module, sds dec
cp->rediscmd->group = COMMAND_GROUP_MODULE;
cp->rediscmd->proc = RedisModuleCommandDispatcher;
cp->rediscmd->flags = flags | CMD_MODULE;
cp->rediscmd->getkeys_proc = (redisGetKeysProc*)(unsigned long)cp;
cp->rediscmd->module_cmd = cp;
cp->rediscmd->key_specs_max = STATIC_KEY_SPECS_NUM;
cp->rediscmd->key_specs = cp->rediscmd->key_specs_static;
if (firstkey != 0) {
@ -1186,7 +1185,7 @@ RedisModuleCommand *RM_GetCommand(RedisModuleCtx *ctx, const char *name) {
if (!cmd || !(cmd->flags & CMD_MODULE))
return NULL;
RedisModuleCommand *cp = (void*)(unsigned long)cmd->getkeys_proc;
RedisModuleCommand *cp = cmd->module_cmd;
if (cp->module != ctx->module)
return NULL;
@ -1230,7 +1229,7 @@ int RM_CreateSubcommand(RedisModuleCommand *parent, const char *name, RedisModul
if (parent_cmd->parent)
return REDISMODULE_ERR; /* We don't allow more than one level of subcommands */
RedisModuleCommand *parent_cp = (void*)(unsigned long)parent_cmd->getkeys_proc;
RedisModuleCommand *parent_cp = parent_cmd->module_cmd;
if (parent_cp->func)
return REDISMODULE_ERR; /* A parent command should be a pure container of subcommands */
@ -1444,7 +1443,7 @@ moduleCmdArgAt(const RedisModuleCommandInfoVersion *version,
* begin search step. (Usually it's just after `keynumidx`, in
* which case it should be set to `keynumidx + 1`.)
*
* * `keystep`: How many argumentss should we skip after finding a
* * `keystep`: How many arguments should we skip after finding a
* key, in order to find the next one?
*
* Key-spec flags:
@ -1769,7 +1768,7 @@ static int moduleValidateCommandInfo(const RedisModuleCommandInfo *info) {
if (!isPowerOfTwo(spec->flags & key_flags)) {
serverLog(LL_WARNING,
"Invalid command info: key_specs[%zd].flags: "
"Exactly one of the flags RO, RW, OW, RM reqired", j);
"Exactly one of the flags RO, RW, OW, RM required", j);
return 0;
}
if ((spec->flags & write_flags) != 0 &&
@ -1977,7 +1976,7 @@ int moduleIsModuleCommand(void *module_handle, struct redisCommand *cmd) {
return 0;
if (module_handle == NULL)
return 0;
RedisModuleCommand *cp = (void*)(unsigned long)cmd->getkeys_proc;
RedisModuleCommand *cp = cmd->module_cmd;
return (cp->module == module_handle);
}
@ -2304,7 +2303,7 @@ RedisModuleString *RM_CreateStringPrintf(RedisModuleCtx *ctx, const char *fmt, .
}
/* Like RedisModule_CreatString(), but creates a string starting from a long long
/* Like RedisModule_CreatString(), but creates a string starting from a `long long`
* integer instead of taking a buffer and its length.
*
* The returned string must be released with RedisModule_FreeString() or by
@ -2517,9 +2516,9 @@ const char *RM_StringPtrLen(const RedisModuleString *str, size_t *len) {
* Higher level string operations
* ------------------------------------------------------------------------- */
/* Convert the string into a long long integer, storing it at `*ll`.
/* Convert the string into a `long long` integer, storing it at `*ll`.
* Returns REDISMODULE_OK on success. If the string can't be parsed
* as a valid, strict long long (no spaces before/after), REDISMODULE_ERR
* as a valid, strict `long long` (no spaces before/after), REDISMODULE_ERR
* is returned. */
int RM_StringToLongLong(const RedisModuleString *str, long long *ll) {
return string2ll(str->ptr,sdslen(str->ptr),ll) ? REDISMODULE_OK :
@ -2692,7 +2691,7 @@ client *moduleGetReplyClient(RedisModuleCtx *ctx) {
}
}
/* Send an integer reply to the client, with the specified long long value.
/* Send an integer reply to the client, with the specified `long long` value.
* The function always returns REDISMODULE_OK. */
int RM_ReplyWithLongLong(RedisModuleCtx *ctx, long long ll) {
client *c = moduleGetReplyClient(ctx);
@ -2838,8 +2837,8 @@ int RM_ReplyWithSet(RedisModuleCtx *ctx, long len) {
/* Add attributes (metadata) to the reply. Should be done before adding the
* actual reply. see https://github.com/antirez/RESP3/blob/master/spec.md#attribute-type
*
* After starting an attributes reply, the module must make `len*2` calls to other
* `ReplyWith*` style functions in order to emit the elements of the attribtute map.
* After starting an attribute's reply, the module must make `len*2` calls to other
* `ReplyWith*` style functions in order to emit the elements of the attribute map.
* See Reply APIs section for more details.
*
* Use RM_ReplySetAttributeLength() to set deferred length.
@ -3488,7 +3487,7 @@ int RM_GetContextFlags(RedisModuleCtx *ctx) {
}
}
if (server.in_script)
if (scriptIsRunning())
flags |= REDISMODULE_CTX_FLAGS_LUA;
if (server.in_exec)
@ -3622,7 +3621,7 @@ static void moduleInitKeyTypeSpecific(RedisModuleKey *key) {
}
}
/* Return an handle representing a Redis key, so that it is possible
/* Return a handle representing a Redis key, so that it is possible
* to call other APIs with the key handle as argument to perform
* operations on the key.
*
@ -3986,7 +3985,7 @@ int RM_StringTruncate(RedisModuleKey *key, size_t newlen) {
* RM_ListInsert, the internal iterator is invalidated so the next operation
* will require a linear seek.
*
* Modifying a list in any another way, for examle using RM_Call(), while a key
* Modifying a list in any another way, for example using RM_Call(), while a key
* is open will confuse the internal iterator and may cause trouble if the key
* is used after such modifications. The key must be reopened in this case.
*
@ -4840,7 +4839,7 @@ int RM_HashSet(RedisModuleKey *key, int flags, ...) {
return count;
}
/* Get fields from an hash value. This function is called using a variable
/* Get fields from a hash value. This function is called using a variable
* number of arguments, alternating a field name (as a RedisModuleString
* pointer) with a pointer to a RedisModuleString pointer, that is set to the
* value of the field if the field exists, or NULL if the field does not exist.
@ -4874,7 +4873,7 @@ int RM_HashSet(RedisModuleKey *key, int flags, ...) {
* RedisModule_HashGet(mykey,REDISMODULE_HASH_EXISTS,argv[1],&exists,NULL);
*
* The function returns REDISMODULE_OK on success and REDISMODULE_ERR if
* the key is not an hash value.
* the key is not a hash value.
*
* Memory management:
*
@ -5451,7 +5450,7 @@ RedisModuleCallReply *RM_CallReplyArrayElement(RedisModuleCallReply *reply, size
return callReplyGetArrayElement(reply, idx);
}
/* Return the long long of an integer reply. */
/* Return the `long long` of an integer reply. */
long long RM_CallReplyInteger(RedisModuleCallReply *reply) {
return callReplyGetLongLong(reply);
}
@ -5466,7 +5465,7 @@ const char *RM_CallReplyBigNumber(RedisModuleCallReply *reply, size_t *len) {
return callReplyGetBigNumber(reply, len);
}
/* Return the value of an verbatim string reply,
/* Return the value of a verbatim string reply,
* An optional output argument can be given to get verbatim reply format. */
const char *RM_CallReplyVerbatim(RedisModuleCallReply *reply, size_t *len, const char **format) {
return callReplyGetVerbatim(reply, len, format);
@ -5503,7 +5502,7 @@ RedisModuleCallReply *RM_CallReplyAttribute(RedisModuleCallReply *reply) {
return callReplyGetAttribute(reply);
}
/* Retrieve the 'idx'-th key and value of a attribute reply.
/* Retrieve the 'idx'-th key and value of an attribute reply.
*
* Returns:
* - REDISMODULE_OK on success.
@ -5551,7 +5550,7 @@ RedisModuleString *RM_CreateStringFromCallReply(RedisModuleCallReply *reply) {
* items and *argvlenp with the length of the allocated argv.
*
* The integer pointed by 'flags' is populated with flags according
* to special modifiers in "fmt". For now only one exists:
* to special modifiers in "fmt".
*
* "!" -> REDISMODULE_ARGV_REPLICATE
* "A" -> REDISMODULE_ARGV_NO_AOF
@ -5627,6 +5626,8 @@ robj **moduleCreateArgvFromUserFormat(const char *cmdname, const char *fmt, int
if (flags) (*flags) |= REDISMODULE_ARGV_SCRIPT_MODE;
} else if (*p == 'W') {
if (flags) (*flags) |= REDISMODULE_ARGV_NO_WRITES;
} else if (*p == 'M') {
if (flags) (*flags) |= REDISMODULE_ARGV_RESPECT_DENY_OOM;
} else if (*p == 'E') {
if (flags) (*flags) |= REDISMODULE_ARGV_CALL_REPLIES_AS_ERRORS;
} else {
@ -5656,7 +5657,7 @@ fmterr:
* * `b` -- The argument is a buffer and is immediately followed by another
* argument that is the buffer's length.
* * `c` -- The argument is a pointer to a plain C string (null-terminated).
* * `l` -- The argument is long long integer.
* * `l` -- The argument is a `long long` integer.
* * `s` -- The argument is a RedisModuleString.
* * `v` -- The argument(s) is a vector of RedisModuleString.
* * `!` -- Sends the Redis command and its arguments to replicas and AOF.
@ -5668,14 +5669,15 @@ fmterr:
* same as the client attached to the given RedisModuleCtx. This will
* probably used when you want to pass the reply directly to the client.
* * `C` -- Check if command can be executed according to ACL rules.
* * 'S' -- Run the command in a script mode, this means that it will raise
* * `S` -- Run the command in a script mode, this means that it will raise
* an error if a command which are not allowed inside a script
* (flagged with the `deny-script` flag) is invoked (like SHUTDOWN).
* In addition, on script mode, write commands are not allowed if there are
* not enough good replicas (as configured with `min-replicas-to-write`)
* or when the server is unable to persist to the disk.
* * 'W' -- Do not allow to run any write command (flagged with the `write` flag).
* * 'E' -- Return error as RedisModuleCallReply. If there is an error before
* * `W` -- Do not allow to run any write command (flagged with the `write` flag).
* * `M` -- Do not allow `deny-oom` flagged commands when over the memory limit.
* * `E` -- Return error as RedisModuleCallReply. If there is an error before
* invoking the command, the error is returned using errno mechanism.
* This flag allows to get the error also as an error CallReply with
* relevant error message.
@ -5693,7 +5695,7 @@ fmterr:
* * ENETDOWN: operation in Cluster instance when cluster is down.
* * ENOTSUP: No ACL user for the specified module context
* * EACCES: Command cannot be executed, according to ACL rules
* * ENOSPC: Write command is not allowed
* * ENOSPC: Write or deny-oom command is not allowed
* * ESPIPE: Command not allowed on script mode
*
* Example code fragment:
@ -5785,8 +5787,21 @@ RedisModuleCallReply *RM_Call(RedisModuleCtx *ctx, const char *cmdname, const ch
}
}
if (cmd->flags & CMD_WRITE) {
if (flags & REDISMODULE_ARGV_NO_WRITES) {
if (flags & REDISMODULE_ARGV_RESPECT_DENY_OOM) {
if (cmd->flags & CMD_DENYOOM) {
if (server.pre_command_oom_state) {
errno = ENOSPC;
if (error_as_call_replies) {
sds msg = sdsdup(shared.oomerr->ptr);
reply = callReplyCreateError(msg, ctx);
}
goto cleanup;
}
}
}
if (flags & REDISMODULE_ARGV_NO_WRITES) {
if (cmd->flags & CMD_WRITE) {
errno = ENOSPC;
if (error_as_call_replies) {
sds msg = sdscatfmt(sdsempty(), "Write command '%S' was "
@ -5795,14 +5810,17 @@ RedisModuleCallReply *RM_Call(RedisModuleCtx *ctx, const char *cmdname, const ch
}
goto cleanup;
}
}
if (flags & REDISMODULE_ARGV_SCRIPT_MODE) {
/* Script mode tests */
if (flags & REDISMODULE_ARGV_SCRIPT_MODE) {
if (cmd->flags & CMD_WRITE) {
/* on script mode, if a command is a write command,
* We will not run it if we encounter disk error
* or we do not have enough replicas */
if (!checkGoodReplicasStatus()) {
errno = ENOSPC;
errno = ESPIPE;
if (error_as_call_replies) {
sds msg = sdsdup(shared.noreplicaserr->ptr);
reply = callReplyCreateError(msg, ctx);
@ -5814,7 +5832,7 @@ RedisModuleCallReply *RM_Call(RedisModuleCtx *ctx, const char *cmdname, const ch
int obey_client = mustObeyClient(server.current_client);
if (deny_write_type != DISK_ERROR_TYPE_NONE && !obey_client) {
errno = ENOSPC;
errno = ESPIPE;
if (error_as_call_replies) {
sds msg = writeCommandsGetDiskErrorMessage(deny_write_type);
reply = callReplyCreateError(msg, ctx);
@ -5822,6 +5840,24 @@ RedisModuleCallReply *RM_Call(RedisModuleCtx *ctx, const char *cmdname, const ch
goto cleanup;
}
if (server.masterhost && server.repl_slave_ro && !obey_client) {
errno = ESPIPE;
if (error_as_call_replies) {
sds msg = sdsdup(shared.roslaveerr->ptr);
reply = callReplyCreateError(msg, ctx);
}
goto cleanup;
}
}
if (server.masterhost && server.repl_state != REPL_STATE_CONNECTED &&
server.repl_serve_stale_data == 0 && !(cmd->flags & CMD_STALE)) {
errno = ESPIPE;
if (error_as_call_replies) {
sds msg = sdsdup(shared.masterdownerr->ptr);
reply = callReplyCreateError(msg, ctx);
}
goto cleanup;
}
}
@ -6088,7 +6124,7 @@ const char *moduleTypeModuleName(moduleType *mt) {
const char *moduleNameFromCommand(struct redisCommand *cmd) {
serverAssert(cmd->proc == RedisModuleCommandDispatcher);
RedisModuleCommand *cp = (void*)(unsigned long)cmd->getkeys_proc;
RedisModuleCommand *cp = cmd->module_cmd;
return cp->module->name;
}
@ -6531,7 +6567,7 @@ RedisModuleString *RM_LoadString(RedisModuleIO *io) {
return moduleLoadString(io,0,NULL);
}
/* Like RedisModule_LoadString() but returns an heap allocated string that
/* Like RedisModule_LoadString() but returns a heap allocated string that
* was allocated with RedisModule_Alloc(), and can be resized or freed with
* RedisModule_Realloc() or RedisModule_Free().
*
@ -6693,7 +6729,7 @@ ssize_t rdbSaveModulesAux(rio *rdb, int when) {
*
* Because Sets are not ordered, so every element added has a position that
* does not depend from the other. However if instead our elements are
* ordered in pairs, like field-value pairs of an Hash, then one should
* ordered in pairs, like field-value pairs of a Hash, then one should
* use:
*
* foreach key,value {
@ -6717,7 +6753,7 @@ void RM_DigestAddStringBuffer(RedisModuleDigest *md, const char *ele, size_t len
mixDigest(md->o,ele,len);
}
/* Like `RedisModule_DigestAddStringBuffer()` but takes a long long as input
/* Like `RedisModule_DigestAddStringBuffer()` but takes a `long long` as input
* that gets converted into a string before adding it to the digest. */
void RM_DigestAddLongLong(RedisModuleDigest *md, long long ll) {
char buf[LONG_STR_SIZE];
@ -7061,7 +7097,7 @@ void unblockClientFromModule(client *c) {
*/
RedisModuleBlockedClient *moduleBlockClient(RedisModuleCtx *ctx, RedisModuleCmdFunc reply_callback, RedisModuleCmdFunc timeout_callback, void (*free_privdata)(RedisModuleCtx*,void*), long long timeout_ms, RedisModuleString **keys, int numkeys, void *privdata) {
client *c = ctx->client;
int islua = server.in_script;
int islua = scriptIsRunning();
int ismulti = server.in_exec;
c->bpop.module_blocked_handle = zmalloc(sizeof(RedisModuleBlockedClient));
@ -7134,7 +7170,7 @@ int moduleTryServeClientBlockedOnKey(client *c, robj *key) {
return served;
}
/* Block a client in the context of a blocking command, returning an handle
/* Block a client in the context of a blocking command, returning a handle
* which will be used, later, in order to unblock the client with a call to
* RedisModule_UnblockClient(). The arguments specify callback functions
* and a timeout after which the client is unblocked.
@ -8056,7 +8092,7 @@ void RM_SetClusterFlags(RedisModuleCtx *ctx, uint64_t flags) {
/* --------------------------------------------------------------------------
* ## Modules Timers API
*
* Module timers are an high precision "green timers" abstraction where
* Module timers are a high precision "green timers" abstraction where
* every module can register even millions of timers without problems, even if
* the actual event loop will just have a single timer that is used to awake the
* module timers subsystem in order to process the next event.
@ -8663,7 +8699,7 @@ int RM_ACLCheckKeyPermissions(RedisModuleUser *user, RedisModuleString *key, int
* access flags. See RM_ChannelAtPosWithFlags for more information about the
* possible flags that can be passed in.
*
* If the user is able to acecss the pubsub channel then REDISMODULE_OK is returned, otherwise
* If the user is able to access the pubsub channel then REDISMODULE_OK is returned, otherwise
* REDISMODULE_ERR is returned and errno is set to one of the following values:
*
* * EINVAL: The provided flags are invalid.
@ -8783,7 +8819,7 @@ int RM_AuthenticateClientWithACLUser(RedisModuleCtx *ctx, const char *name, size
}
/* Deauthenticate and close the client. The client resources will not be
* be immediately freed, but will be cleaned up in a background job. This is
* immediately freed, but will be cleaned up in a background job. This is
* the recommended way to deauthenticate a client since most clients can't
* handle users becoming deauthenticated. Returns REDISMODULE_ERR when the
* client doesn't exist and REDISMODULE_OK when the operation was successful.
@ -9000,7 +9036,7 @@ int RM_DictIteratorReseekC(RedisModuleDictIter *di, const char *op, void *key, s
return raxSeek(&di->ri,op,key,keylen);
}
/* Like RedisModule_DictIteratorReseekC() but takes the key as as a
/* Like RedisModule_DictIteratorReseekC() but takes the key as a
* RedisModuleString. */
int RM_DictIteratorReseek(RedisModuleDictIter *di, const char *op, RedisModuleString *key) {
return RM_DictIteratorReseekC(di,op,key->ptr,sdslen(key->ptr));
@ -9584,7 +9620,7 @@ int moduleUnregisterFilters(RedisModule *module) {
*
* 1. Invocation by a client.
* 2. Invocation through `RedisModule_Call()` by any module.
* 3. Invocation through Lua 'redis.call()`.
* 3. Invocation through Lua `redis.call()`.
* 4. Replication of a command from a master.
*
* The filter executes in a special filter context, which is different and more
@ -9764,6 +9800,12 @@ size_t RM_MallocSize(void* ptr) {
return zmalloc_size(ptr);
}
/* Similar to RM_MallocSize, the difference is that RM_MallocUsableSize
* returns the usable size of memory by the module. */
size_t RM_MallocUsableSize(void *ptr) {
return zmalloc_usable_size(ptr);
}
/* Same as RM_MallocSize, except it works on RedisModuleString pointers.
*/
size_t RM_MallocSizeString(RedisModuleString* str) {
@ -9809,7 +9851,7 @@ typedef struct {
} ScanCBData;
typedef struct RedisModuleScanCursor{
int cursor;
unsigned long cursor;
int done;
}RedisModuleScanCursor;
@ -9868,14 +9910,14 @@ void RM_ScanCursorDestroy(RedisModuleScanCursor *cursor) {
*
* The way it should be used:
*
* RedisModuleCursor *c = RedisModule_ScanCursorCreate();
* RedisModuleScanCursor *c = RedisModule_ScanCursorCreate();
* while(RedisModule_Scan(ctx, c, callback, privateData));
* RedisModule_ScanCursorDestroy(c);
*
* It is also possible to use this API from another thread while the lock
* is acquired during the actual call to RM_Scan:
*
* RedisModuleCursor *c = RedisModule_ScanCursorCreate();
* RedisModuleScanCursor *c = RedisModule_ScanCursorCreate();
* RedisModule_ThreadSafeContextLock(ctx);
* while(RedisModule_Scan(ctx, c, callback, privateData)){
* RedisModule_ThreadSafeContextUnlock(ctx);
@ -9963,7 +10005,7 @@ static void moduleScanKeyCallback(void *privdata, const dictEntry *de) {
*
* The way it should be used:
*
* RedisModuleCursor *c = RedisModule_ScanCursorCreate();
* RedisModuleScanCursor *c = RedisModule_ScanCursorCreate();
* RedisModuleKey *key = RedisModule_OpenKey(...)
* while(RedisModule_ScanKey(key, c, callback, privateData));
* RedisModule_CloseKey(key);
@ -9972,7 +10014,7 @@ static void moduleScanKeyCallback(void *privdata, const dictEntry *de) {
* It is also possible to use this API from another thread while the lock is acquired during
* the actual call to RM_ScanKey, and re-opening the key each time:
*
* RedisModuleCursor *c = RedisModule_ScanCursorCreate();
* RedisModuleScanCursor *c = RedisModule_ScanCursorCreate();
* RedisModule_ThreadSafeContextLock(ctx);
* RedisModuleKey *key = RedisModule_OpenKey(...)
* while(RedisModule_ScanKey(ctx, c, callback, privateData)){
@ -10936,7 +10978,7 @@ int moduleFreeCommand(struct RedisModule *module, struct redisCommand *cmd) {
if (cmd->proc != RedisModuleCommandDispatcher)
return C_ERR;
RedisModuleCommand *cp = (void*)(unsigned long)cmd->getkeys_proc;
RedisModuleCommand *cp = cmd->module_cmd;
if (cp->module != module)
return C_ERR;
@ -11679,7 +11721,11 @@ int RM_RegisterBoolConfig(RedisModuleCtx *ctx, const char *name, int default_val
* }
* ...
* RedisModule_RegisterEnumConfig(ctx, "enum", 0, REDISMODULE_CONFIG_DEFAULT, enum_vals, int_vals, 3, getEnumConfigCommand, setEnumConfigCommand, NULL, NULL);
*
*
* Note that you can use REDISMODULE_CONFIG_BITFLAGS so that multiple enum string
* can be combined into one integer as bit flags, in which case you may want to
* sort your enums so that the preferred combinations are present first.
*
* See RedisModule_RegisterStringConfig for detailed general information about configs. */
int RM_RegisterEnumConfig(RedisModuleCtx *ctx, const char *name, int default_val, unsigned int flags, const char **enum_values, const int *int_values, int num_enum_vals, RedisModuleConfigGetEnumFunc getfn, RedisModuleConfigSetEnumFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata) {
RedisModule *module = ctx->module;
@ -12016,7 +12062,7 @@ int *RM_GetCommandKeysWithFlags(RedisModuleCtx *ctx, RedisModuleString **argv, i
}
/* Bail out if command has no keys */
if (cmd->getkeys_proc == NULL && cmd->key_specs_num == 0) {
if (!doesCommandHaveKeys(cmd)) {
errno = 0;
return NULL;
}
@ -12050,7 +12096,7 @@ int *RM_GetCommandKeysWithFlags(RedisModuleCtx *ctx, RedisModuleString **argv, i
return res;
}
/* Identinal to RM_GetCommandKeysWithFlags when flags are not needed. */
/* Identical to RM_GetCommandKeysWithFlags when flags are not needed. */
int *RM_GetCommandKeys(RedisModuleCtx *ctx, RedisModuleString **argv, int argc, int *num_keys) {
return RM_GetCommandKeysWithFlags(ctx, argv, argc, num_keys, NULL);
}
@ -12564,6 +12610,7 @@ void moduleRegisterCoreAPI(void) {
REGISTER_API(GetBlockedClientReadyKey);
REGISTER_API(GetUsedMemoryRatio);
REGISTER_API(MallocSize);
REGISTER_API(MallocUsableSize);
REGISTER_API(MallocSizeString);
REGISTER_API(MallocSizeDict);
REGISTER_API(ScanCursorCreate);

6
src/multi.c
View File

@ -56,7 +56,7 @@ void freeClientMultiState(client *c) {
}
/* Add a new command into the MULTI commands queue */
void queueMultiCommand(client *c) {
void queueMultiCommand(client *c, uint64_t cmd_flags) {
multiCmd *mc;
/* No sense to waste memory if the transaction is already aborted.
@ -75,8 +75,8 @@ void queueMultiCommand(client *c) {
mc->argv_len = c->argv_len;
c->mstate.count++;
c->mstate.cmd_flags |= c->cmd->flags;
c->mstate.cmd_inv_flags |= ~c->cmd->flags;
c->mstate.cmd_flags |= cmd_flags;
c->mstate.cmd_inv_flags |= ~cmd_flags;
c->mstate.argv_len_sums += c->argv_len_sum + sizeof(robj*)*c->argc;
/* Reset the client's args since we copied them into the mstate and shouldn't

20
src/networking.c
View File

@ -1953,7 +1953,13 @@ int writeToClient(client *c, int handler_installed) {
zmalloc_used_memory() < server.maxmemory) &&
!(c->flags & CLIENT_SLAVE)) break;
}
atomicIncr(server.stat_net_output_bytes, totwritten);
if (getClientType(c) == CLIENT_TYPE_SLAVE) {
atomicIncr(server.stat_net_repl_output_bytes, totwritten);
} else {
atomicIncr(server.stat_net_output_bytes, totwritten);
}
if (nwritten == -1) {
if (connGetState(c->conn) != CONN_STATE_CONNECTED) {
serverLog(LL_VERBOSE,
@ -2499,7 +2505,7 @@ int processInputBuffer(client *c) {
* condition on the slave. We want just to accumulate the replication
* stream (instead of replying -BUSY like we do with other clients) and
* later resume the processing. */
if (scriptIsTimedout() && c->flags & CLIENT_MASTER) break;
if (isInsideYieldingLongCommand() && c->flags & CLIENT_MASTER) break;
/* CLIENT_CLOSE_AFTER_REPLY closes the connection once the reply is
* written to the client. Make sure to not let the reply grow after
@ -2655,8 +2661,13 @@ void readQueryFromClient(connection *conn) {
if (c->querybuf_peak < qblen) c->querybuf_peak = qblen;
c->lastinteraction = server.unixtime;
if (c->flags & CLIENT_MASTER) c->read_reploff += nread;
atomicIncr(server.stat_net_input_bytes, nread);
if (c->flags & CLIENT_MASTER) {
c->read_reploff += nread;
atomicIncr(server.stat_net_repl_input_bytes, nread);
} else {
atomicIncr(server.stat_net_input_bytes, nread);
}
if (!(c->flags & CLIENT_MASTER) && sdslen(c->querybuf) > server.client_max_querybuf_len) {
sds ci = catClientInfoString(sdsempty(),c), bytes = sdsempty();
@ -3557,7 +3568,6 @@ void replaceClientCommandVector(client *c, int argc, robj **argv) {
int j;
retainOriginalCommandVector(c);
freeClientArgv(c);
zfree(c->argv);
c->argv = argv;
c->argc = argc;
c->argv_len_sum = 0;

3
src/object.c
View File

@ -930,7 +930,6 @@ char *strEncoding(int encoding) {
case OBJ_ENCODING_INT: return "int";
case OBJ_ENCODING_HT: return "hashtable";
case OBJ_ENCODING_QUICKLIST: return "quicklist";
case OBJ_ENCODING_ZIPLIST: return "ziplist";
case OBJ_ENCODING_LISTPACK: return "listpack";
case OBJ_ENCODING_INTSET: return "intset";
case OBJ_ENCODING_SKIPLIST: return "skiplist";
@ -998,8 +997,6 @@ size_t objectComputeSize(robj *key, robj *o, size_t sample_size, int dbid) {
samples++;
} while ((node = node->next) && samples < sample_size);
asize += (double)elesize/samples*ql->len;
} else if (o->encoding == OBJ_ENCODING_ZIPLIST) {
asize = sizeof(*o)+zmalloc_size(o->ptr);
} else {
serverPanic("Unknown list encoding");
}

23
src/pubsub.c
View File

@ -39,6 +39,7 @@ typedef struct pubsubtype {
dict **serverPubSubChannels;
robj **subscribeMsg;
robj **unsubscribeMsg;
robj **messageBulk;
}pubsubtype;
/*
@ -78,6 +79,7 @@ pubsubtype pubSubType = {
.serverPubSubChannels = &server.pubsub_channels,
.subscribeMsg = &shared.subscribebulk,
.unsubscribeMsg = &shared.unsubscribebulk,
.messageBulk = &shared.messagebulk,
};
/*
@ -89,7 +91,8 @@ pubsubtype pubSubShardType = {
.subscriptionCount = clientShardSubscriptionsCount,
.serverPubSubChannels = &server.pubsubshard_channels,
.subscribeMsg = &shared.ssubscribebulk,
.unsubscribeMsg = &shared.sunsubscribebulk
.unsubscribeMsg = &shared.sunsubscribebulk,
.messageBulk = &shared.smessagebulk,
};
/*-----------------------------------------------------------------------------
@ -101,12 +104,12 @@ pubsubtype pubSubShardType = {
* message. However if the caller sets 'msg' as NULL, it will be able
* to send a special message (for instance an Array type) by using the
* addReply*() API family. */
void addReplyPubsubMessage(client *c, robj *channel, robj *msg) {
void addReplyPubsubMessage(client *c, robj *channel, robj *msg, robj *message_bulk) {
if (c->resp == 2)
addReply(c,shared.mbulkhdr[3]);
else
addReplyPushLen(c,3);
addReply(c,shared.messagebulk);
addReply(c,message_bulk);
addReplyBulk(c,channel);
if (msg) addReplyBulk(c,msg);
}
@ -461,7 +464,7 @@ int pubsubPublishMessageInternal(robj *channel, robj *message, pubsubtype type)
listRewind(list,&li);
while ((ln = listNext(&li)) != NULL) {
client *c = ln->value;
addReplyPubsubMessage(c,channel,message);
addReplyPubsubMessage(c,channel,message,*type.messageBulk);
updateClientMemUsage(c);
receivers++;
}
@ -604,10 +607,10 @@ void pubsubCommand(client *c) {
" Return number of subscriptions to patterns.",
"NUMSUB [<channel> ...]",
" Return the number of subscribers for the specified channels, excluding",
" pattern subscriptions(default: no channels)."
" pattern subscriptions(default: no channels).",
"SHARDCHANNELS [<pattern>]",
" Return the currently active shard level channels matching a <pattern> (default: '*').",
"SHARDNUMSUB [<channel> ...]",
"SHARDNUMSUB [<shardchannel> ...]",
" Return the number of subscribers for the specified shard level channel(s)",
NULL
};
@ -639,7 +642,7 @@ NULL
sds pat = (c->argc == 2) ? NULL : c->argv[2]->ptr;
channelList(c,pat,server.pubsubshard_channels);
} else if (!strcasecmp(c->argv[1]->ptr,"shardnumsub") && c->argc >= 2) {
/* PUBSUB SHARDNUMSUB [Channel_1 ... Channel_N] */
/* PUBSUB SHARDNUMSUB [ShardChannel_1 ... ShardChannel_N] */
int j;
addReplyArrayLen(c, (c->argc-2)*2);
@ -676,7 +679,7 @@ void channelList(client *c, sds pat, dict *pubsub_channels) {
setDeferredArrayLen(c,replylen,mblen);
}
/* SPUBLISH <channel> <message> */
/* SPUBLISH <shardchannel> <message> */
void spublishCommand(client *c) {
int receivers = pubsubPublishMessageAndPropagateToCluster(c->argv[1],c->argv[2],1);
if (!server.cluster_enabled)
@ -684,7 +687,7 @@ void spublishCommand(client *c) {
addReplyLongLong(c,receivers);
}
/* SSUBSCRIBE channel [channel ...] */
/* SSUBSCRIBE shardchannel [shardchannel ...] */
void ssubscribeCommand(client *c) {
if (c->flags & CLIENT_DENY_BLOCKING) {
/* A client that has CLIENT_DENY_BLOCKING flag on
@ -708,7 +711,7 @@ void ssubscribeCommand(client *c) {
}
/* SUNSUBSCRIBE [channel ...] */
/* SUNSUBSCRIBE [shardchannel [shardchannel ...]] */
void sunsubscribeCommand(client *c) {
if (c->argc == 1) {
pubsubUnsubscribeShardAllChannels(c, 1);

2
src/quicklist.c
View File

@ -43,7 +43,7 @@
#endif
/* Optimization levels for size-based filling.
* Note that the largest possible limit is 16k, so even if each record takes
* Note that the largest possible limit is 64k, so even if each record takes
* just one byte, it still won't overflow the 16 bit count field. */
static const size_t optimization_level[] = {4096, 8192, 16384, 32768, 65536};

2
src/quicklist.h
View File

@ -39,7 +39,7 @@
* We use bit fields keep the quicklistNode at 32 bytes.
* count: 16 bits, max 65536 (max lp bytes is 65k, so max count actually < 32k).
* encoding: 2 bits, RAW=1, LZF=2.
* container: 2 bits, PLAIN=1, PACKED=2.
* container: 2 bits, PLAIN=1 (a single item as char array), PACKED=2 (listpack with multiple items).
* recompress: 1 bit, bool, true if node is temporary decompressed for usage.
* attempted_compress: 1 bit, boolean, used for verifying during testing.
* extra: 10 bits, free for future use; pads out the remainder of 32 bits */

8
src/rdb.c
View File

@ -2746,7 +2746,7 @@ void stopLoading(int success) {
}
void startSaving(int rdbflags) {
/* Fire the persistence modules end event. */
/* Fire the persistence modules start event. */
int subevent;
if (rdbflags & RDBFLAGS_AOF_PREAMBLE && getpid() != server.pid)
subevent = REDISMODULE_SUBEVENT_PERSISTENCE_AOF_START;
@ -2782,6 +2782,9 @@ void rdbLoadProgressCallback(rio *r, const void *buf, size_t len) {
processEventsWhileBlocked();
processModuleLoadingProgressEvent(0);
}
if (server.repl_state == REPL_STATE_TRANSFER && rioCheckType(r) == RIO_TYPE_CONN) {
atomicIncr(server.stat_net_repl_input_bytes, len);
}
}
/* Save the given functions_ctx to the rdb.
@ -3476,6 +3479,9 @@ void saveCommand(client *c) {
addReplyError(c,"Background save already in progress");
return;
}
server.stat_rdb_saves++;
rdbSaveInfo rsi, *rsiptr;
rsiptr = rdbPopulateSaveInfo(&rsi);
if (rdbSave(SLAVE_REQ_NONE,server.rdb_filename,rsiptr) == C_OK) {

26
src/redis-cli.c
View File

@ -8220,6 +8220,11 @@ static void getKeySizes(redisReply *keys, typeinfo **types,
}
}
static void longStatLoopModeStop(int s) {
UNUSED(s);
force_cancel_loop = 1;
}
static void findBigKeys(int memkeys, unsigned memkeys_samples) {
unsigned long long sampled = 0, total_keys, totlen=0, *sizes=NULL, it=0, scan_loops = 0;
redisReply *reply, *keys;
@ -8237,6 +8242,7 @@ static void findBigKeys(int memkeys, unsigned memkeys_samples) {
typeinfo_add(types_dict, "zset", &type_zset);
typeinfo_add(types_dict, "stream", &type_stream);
signal(SIGINT, longStatLoopModeStop);
/* Total keys pre scanning */
total_keys = getDbSize();
@ -8315,14 +8321,14 @@ static void findBigKeys(int memkeys, unsigned memkeys_samples) {
}
freeReplyObject(reply);
} while(it != 0);
} while(force_cancel_loop == 0 && it != 0);
if(types) zfree(types);
if(sizes) zfree(sizes);
/* We're done */
printf("\n-------- summary -------\n\n");
if (force_cancel_loop) printf("[%05.2f%%] ", pct);
printf("Sampled %llu keys in the keyspace!\n", sampled);
printf("Total key length in bytes is %llu (avg len %.2f)\n\n",
totlen, totlen ? (double)totlen/sampled : 0);
@ -8401,6 +8407,7 @@ static void findHotKeys(void) {
unsigned int arrsize = 0, i, k;
double pct;
signal(SIGINT, longStatLoopModeStop);
/* Total keys pre scanning */
total_keys = getDbSize();
@ -8466,13 +8473,13 @@ static void findHotKeys(void) {
}
freeReplyObject(reply);
} while(it != 0);
} while(force_cancel_loop ==0 && it != 0);
if (freqs) zfree(freqs);
/* We're done */
printf("\n-------- summary -------\n\n");
if(force_cancel_loop)printf("[%05.2f%%] ",pct);
printf("Sampled %llu keys in the keyspace!\n", sampled);
for (i=1; i<= HOTKEYS_SAMPLE; i++) {
@ -8643,7 +8650,7 @@ static void statMode(void) {
static void scanMode(void) {
redisReply *reply;
unsigned long long cur = 0;
signal(SIGINT, longStatLoopModeStop);
do {
reply = sendScan(&cur);
for (unsigned int j = 0; j < reply->element[1]->elements; j++) {
@ -8658,7 +8665,7 @@ static void scanMode(void) {
}
freeReplyObject(reply);
if (config.interval) usleep(config.interval);
} while(cur != 0);
} while(force_cancel_loop == 0 && cur != 0);
exit(0);
}
@ -8785,11 +8792,6 @@ unsigned long compute_something_fast(void) {
return output;
}
static void intrinsicLatencyModeStop(int s) {
UNUSED(s);
force_cancel_loop = 1;
}
static void sigIntHandler(int s) {
UNUSED(s);
@ -8807,7 +8809,7 @@ static void intrinsicLatencyMode(void) {
run_time = (long long)config.intrinsic_latency_duration * 1000000;
test_end = ustime() + run_time;
signal(SIGINT, intrinsicLatencyModeStop);
signal(SIGINT, longStatLoopModeStop);
while(1) {
long long start, end, latency;

11
src/redismodule.h
View File

@ -185,11 +185,12 @@ This flag should not be used directly by the module.
#define REDISMODULE_NOTIFY_KEY_MISS (1<<11) /* m (Note: This one is excluded from REDISMODULE_NOTIFY_ALL on purpose) */
#define REDISMODULE_NOTIFY_LOADED (1<<12) /* module only key space notification, indicate a key loaded from rdb */
#define REDISMODULE_NOTIFY_MODULE (1<<13) /* d, module key space notification */
#define REDISMODULE_NOTIFY_NEW (1<<14) /* n, new key notification */
/* Next notification flag, must be updated when adding new flags above!
This flag should not be used directly by the module.
* Use RedisModule_GetKeyspaceNotificationFlagsAll instead. */
#define _REDISMODULE_NOTIFY_NEXT (1<<14)
#define _REDISMODULE_NOTIFY_NEXT (1<<15)
#define REDISMODULE_NOTIFY_ALL (REDISMODULE_NOTIFY_GENERIC | REDISMODULE_NOTIFY_STRING | REDISMODULE_NOTIFY_LIST | REDISMODULE_NOTIFY_SET | REDISMODULE_NOTIFY_HASH | REDISMODULE_NOTIFY_ZSET | REDISMODULE_NOTIFY_EXPIRED | REDISMODULE_NOTIFY_EVICTED | REDISMODULE_NOTIFY_STREAM | REDISMODULE_NOTIFY_MODULE) /* A */
@ -248,7 +249,7 @@ typedef uint64_t RedisModuleTimerID;
/* When set, Redis will not call RedisModule_SignalModifiedKey(), implicitly in
* RedisModule_CloseKey, and the module needs to do that when manually when keys
* are modified from the user's sperspective, to invalidate WATCH. */
* are modified from the user's perspective, to invalidate WATCH. */
#define REDISMODULE_OPTION_NO_IMPLICIT_SIGNAL_MODIFIED (1<<1)
/* Declare that the module can handle diskless async replication with RedisModule_SetModuleOptions. */
@ -346,7 +347,7 @@ typedef struct {
const char *keyword;
/* An index in argv from which to start searching.
* Can be negative, which means start search from the end, in reverse
* (Example: -2 means to start in reverse from the panultimate arg) */
* (Example: -2 means to start in reverse from the penultimate arg) */
int startfrom;
} keyword;
} bs;
@ -1160,6 +1161,7 @@ REDISMODULE_API int (*RedisModule_ExitFromChild)(int retcode) REDISMODULE_ATTR;
REDISMODULE_API int (*RedisModule_KillForkChild)(int child_pid) REDISMODULE_ATTR;
REDISMODULE_API float (*RedisModule_GetUsedMemoryRatio)() REDISMODULE_ATTR;
REDISMODULE_API size_t (*RedisModule_MallocSize)(void* ptr) REDISMODULE_ATTR;
REDISMODULE_API size_t (*RedisModule_MallocUsableSize)(void *ptr) REDISMODULE_ATTR;
REDISMODULE_API size_t (*RedisModule_MallocSizeString)(RedisModuleString* str) REDISMODULE_ATTR;
REDISMODULE_API size_t (*RedisModule_MallocSizeDict)(RedisModuleDict* dict) REDISMODULE_ATTR;
REDISMODULE_API RedisModuleUser * (*RedisModule_CreateModuleUser)(const char *name) REDISMODULE_ATTR;
@ -1190,7 +1192,7 @@ REDISMODULE_API const RedisModuleString * (*RedisModule_GetKeyNameFromDefragCtx)
REDISMODULE_API int (*RedisModule_EventLoopAdd)(int fd, int mask, RedisModuleEventLoopFunc func, void *user_data) REDISMODULE_ATTR;
REDISMODULE_API int (*RedisModule_EventLoopDel)(int fd, int mask) REDISMODULE_ATTR;
REDISMODULE_API int (*RedisModule_EventLoopAddOneShot)(RedisModuleEventLoopOneShotFunc func, void *user_data) REDISMODULE_ATTR;
REDISMODULE_API int (*RedisModule_RegisterBoolConfig)(RedisModuleCtx *ctx, char *name, int default_val, unsigned int flags, RedisModuleConfigGetBoolFunc getfn, RedisModuleConfigSetBoolFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata) REDISMODULE_ATTR;
REDISMODULE_API int (*RedisModule_RegisterBoolConfig)(RedisModuleCtx *ctx, const char *name, int default_val, unsigned int flags, RedisModuleConfigGetBoolFunc getfn, RedisModuleConfigSetBoolFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata) REDISMODULE_ATTR;
REDISMODULE_API int (*RedisModule_RegisterNumericConfig)(RedisModuleCtx *ctx, const char *name, long long default_val, unsigned int flags, long long min, long long max, RedisModuleConfigGetNumericFunc getfn, RedisModuleConfigSetNumericFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata) REDISMODULE_ATTR;
REDISMODULE_API int (*RedisModule_RegisterStringConfig)(RedisModuleCtx *ctx, const char *name, const char *default_val, unsigned int flags, RedisModuleConfigGetStringFunc getfn, RedisModuleConfigSetStringFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata) REDISMODULE_ATTR;
REDISMODULE_API int (*RedisModule_RegisterEnumConfig)(RedisModuleCtx *ctx, const char *name, int default_val, unsigned int flags, const char **enum_values, const int *int_values, int num_enum_vals, RedisModuleConfigGetEnumFunc getfn, RedisModuleConfigSetEnumFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata) REDISMODULE_ATTR;
@ -1493,6 +1495,7 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(KillForkChild);
REDISMODULE_GET_API(GetUsedMemoryRatio);
REDISMODULE_GET_API(MallocSize);
REDISMODULE_GET_API(MallocUsableSize);
REDISMODULE_GET_API(MallocSizeString);
REDISMODULE_GET_API(MallocSizeDict);
REDISMODULE_GET_API(CreateModuleUser);

43
src/replication.c
View File

@ -1358,7 +1358,7 @@ void sendBulkToSlave(connection *conn) {
freeClient(slave);
return;
}
atomicIncr(server.stat_net_output_bytes, nwritten);
atomicIncr(server.stat_net_repl_output_bytes, nwritten);
sdsrange(slave->replpreamble,nwritten,-1);
if (sdslen(slave->replpreamble) == 0) {
sdsfree(slave->replpreamble);
@ -1387,7 +1387,7 @@ void sendBulkToSlave(connection *conn) {
return;
}
slave->repldboff += nwritten;
atomicIncr(server.stat_net_output_bytes, nwritten);
atomicIncr(server.stat_net_repl_output_bytes, nwritten);
if (slave->repldboff == slave->repldbsize) {
close(slave->repldbfd);
slave->repldbfd = -1;
@ -1419,7 +1419,7 @@ void rdbPipeWriteHandlerConnRemoved(struct connection *conn) {
void rdbPipeWriteHandler(struct connection *conn) {
serverAssert(server.rdb_pipe_bufflen>0);
client *slave = connGetPrivateData(conn);
int nwritten;
ssize_t nwritten;
if ((nwritten = connWrite(conn, server.rdb_pipe_buff + slave->repldboff,
server.rdb_pipe_bufflen - slave->repldboff)) == -1)
{
@ -1431,7 +1431,7 @@ void rdbPipeWriteHandler(struct connection *conn) {
return;
} else {
slave->repldboff += nwritten;
atomicIncr(server.stat_net_output_bytes, nwritten);
atomicIncr(server.stat_net_repl_output_bytes, nwritten);
if (slave->repldboff < server.rdb_pipe_bufflen) {
slave->repl_last_partial_write = server.unixtime;
return; /* more data to write.. */
@ -1491,7 +1491,7 @@ void rdbPipeReadHandler(struct aeEventLoop *eventLoop, int fd, void *clientData,
int stillAlive = 0;
for (i=0; i < server.rdb_pipe_numconns; i++)
{
int nwritten;
ssize_t nwritten;
connection *conn = server.rdb_pipe_conns[i];
if (!conn)
continue;
@ -1511,7 +1511,7 @@ void rdbPipeReadHandler(struct aeEventLoop *eventLoop, int fd, void *clientData,
/* Note: when use diskless replication, 'repldboff' is the offset
* of 'rdb_pipe_buff' sent rather than the offset of entire RDB. */
slave->repldboff = nwritten;
atomicIncr(server.stat_net_output_bytes, nwritten);
atomicIncr(server.stat_net_repl_output_bytes, nwritten);
}
/* If we were unable to write all the data to one of the replicas,
* setup write handler (and disable pipe read handler, below) */
@ -1817,11 +1817,16 @@ void readSyncBulkPayload(connection *conn) {
/* If repl_transfer_size == -1 we still have to read the bulk length
* from the master reply. */
if (server.repl_transfer_size == -1) {
if (connSyncReadLine(conn,buf,1024,server.repl_syncio_timeout*1000) == -1) {
nread = connSyncReadLine(conn,buf,1024,server.repl_syncio_timeout*1000);
if (nread == -1) {
serverLog(LL_WARNING,
"I/O error reading bulk count from MASTER: %s",
strerror(errno));
goto error;
} else {
/* nread here is returned by connSyncReadLine(), which calls syncReadLine() and
* convert "\r\n" to '\0' so 1 byte is lost. */
atomicIncr(server.stat_net_repl_input_bytes, nread+1);
}
if (buf[0] == '-') {
@ -1892,7 +1897,7 @@ void readSyncBulkPayload(connection *conn) {
cancelReplicationHandshake(1);
return;
}
atomicIncr(server.stat_net_input_bytes, nread);
atomicIncr(server.stat_net_repl_input_bytes, nread);
/* When a mark is used, we want to detect EOF asap in order to avoid
* writing the EOF mark into the file... */
@ -2223,8 +2228,8 @@ char *receiveSynchronousResponse(connection *conn) {
/* Read the reply from the server. */
if (connSyncReadLine(conn,buf,sizeof(buf),server.repl_syncio_timeout*1000) == -1)
{
return sdscatprintf(sdsempty(),"-Reading from master: %s",
strerror(errno));
serverLog(LL_WARNING, "Failed to read response from the server: %s", strerror(errno));
return NULL;
}
server.repl_transfer_lastio = server.unixtime;
return sdsnew(buf);
@ -2407,6 +2412,13 @@ int slaveTryPartialResynchronization(connection *conn, int read_reply) {
/* Reading half */
reply = receiveSynchronousResponse(conn);
/* Master did not reply to PSYNC */
if (reply == NULL) {
connSetReadHandler(conn, NULL);
serverLog(LL_WARNING, "Master did not reply to PSYNC, will try later");
return PSYNC_TRY_LATER;
}
if (sdslen(reply) == 0) {
/* The master may send empty newlines after it receives PSYNC
* and before to reply, just to keep the connection alive. */
@ -2566,6 +2578,9 @@ void syncWithMaster(connection *conn) {
if (server.repl_state == REPL_STATE_RECEIVE_PING_REPLY) {
err = receiveSynchronousResponse(conn);
/* The master did not reply */
if (err == NULL) goto no_response_error;
/* We accept only two replies as valid, a positive +PONG reply
* (we just check for "+") or an authentication error.
* Note that older versions of Redis replied with "operation not
@ -2652,6 +2667,7 @@ void syncWithMaster(connection *conn) {
/* Receive AUTH reply. */
if (server.repl_state == REPL_STATE_RECEIVE_AUTH_REPLY) {
err = receiveSynchronousResponse(conn);
if (err == NULL) goto no_response_error;
if (err[0] == '-') {
serverLog(LL_WARNING,"Unable to AUTH to MASTER: %s",err);
sdsfree(err);
@ -2666,6 +2682,7 @@ void syncWithMaster(connection *conn) {
/* Receive REPLCONF listening-port reply. */
if (server.repl_state == REPL_STATE_RECEIVE_PORT_REPLY) {
err = receiveSynchronousResponse(conn);
if (err == NULL) goto no_response_error;
/* Ignore the error if any, not all the Redis versions support
* REPLCONF listening-port. */
if (err[0] == '-') {
@ -2683,6 +2700,7 @@ void syncWithMaster(connection *conn) {
/* Receive REPLCONF ip-address reply. */
if (server.repl_state == REPL_STATE_RECEIVE_IP_REPLY) {
err = receiveSynchronousResponse(conn);
if (err == NULL) goto no_response_error;
/* Ignore the error if any, not all the Redis versions support
* REPLCONF ip-address. */
if (err[0] == '-') {
@ -2697,6 +2715,7 @@ void syncWithMaster(connection *conn) {
/* Receive CAPA reply. */
if (server.repl_state == REPL_STATE_RECEIVE_CAPA_REPLY) {
err = receiveSynchronousResponse(conn);
if (err == NULL) goto no_response_error;
/* Ignore the error if any, not all the Redis versions support
* REPLCONF capa. */
if (err[0] == '-') {
@ -2810,6 +2829,10 @@ void syncWithMaster(connection *conn) {
server.repl_transfer_lastio = server.unixtime;
return;
no_response_error: /* Handle receiveSynchronousResponse() error when master has no reply */
serverLog(LL_WARNING, "Master did not respond to command during SYNC handshake");
/* Fall through to regular error handling */
error:
if (dfd != -1) close(dfd);
connClose(conn);

14
src/rio.c
View File

@ -438,6 +438,20 @@ void rioSetAutoSync(rio *r, off_t bytes) {
r->io.file.autosync = bytes;
}
/* Check the type of rio. */
uint8_t rioCheckType(rio *r) {
if (r->read == rioFileRead) {
return RIO_TYPE_FILE;
} else if (r->read == rioBufferRead) {
return RIO_TYPE_BUFFER;
} else if (r->read == rioConnRead) {
return RIO_TYPE_CONN;
} else {
/* r->read == rioFdRead */
return RIO_TYPE_FD;
}
}
/* --------------------------- Higher level interface --------------------------
*
* The following higher level functions use lower level rio.c functions to help

7
src/rio.h
View File

@ -40,6 +40,11 @@
#define RIO_FLAG_READ_ERROR (1<<0)
#define RIO_FLAG_WRITE_ERROR (1<<1)
#define RIO_TYPE_FILE (1<<0)
#define RIO_TYPE_BUFFER (1<<1)
#define RIO_TYPE_CONN (1<<2)
#define RIO_TYPE_FD (1<<3)
struct _rio {
/* Backend functions.
* Since this functions do not tolerate short writes or reads the return
@ -174,5 +179,5 @@ int rioWriteBulkObject(rio *r, struct redisObject *obj);
void rioGenericUpdateChecksum(rio *r, const void *buf, size_t len);
void rioSetAutoSync(rio *r, off_t bytes);
uint8_t rioCheckType(rio *r);
#endif

82
src/script.c
View File

@ -92,8 +92,8 @@ int scriptInterrupt(scriptRunCtx *run_ctx) {
serverLog(LL_WARNING,
"Slow script detected: still in execution after %lld milliseconds. "
"You can try killing the script using the %s command.",
elapsed, (run_ctx->flags & SCRIPT_EVAL_MODE) ? "SCRIPT KILL" : "FUNCTION KILL");
"You can try killing the script using the %s command. Script name is: %s.",
elapsed, (run_ctx->flags & SCRIPT_EVAL_MODE) ? "SCRIPT KILL" : "FUNCTION KILL", run_ctx->funcname);
enterScriptTimedoutMode(run_ctx);
/* Once the script timeouts we reenter the event loop to permit others
@ -108,6 +108,25 @@ int scriptInterrupt(scriptRunCtx *run_ctx) {
return (run_ctx->flags & SCRIPT_KILLED) ? SCRIPT_KILL : SCRIPT_CONTINUE;
}
uint64_t scriptFlagsToCmdFlags(uint64_t cmd_flags, uint64_t script_flags) {
/* If the script declared flags, clear the ones from the command and use the ones it declared.*/
cmd_flags &= ~(CMD_STALE | CMD_DENYOOM | CMD_WRITE);
/* NO_WRITES implies ALLOW_OOM */
if (!(script_flags & (SCRIPT_FLAG_ALLOW_OOM | SCRIPT_FLAG_NO_WRITES)))
cmd_flags |= CMD_DENYOOM;
if (!(script_flags & SCRIPT_FLAG_NO_WRITES))
cmd_flags |= CMD_WRITE;
if (script_flags & SCRIPT_FLAG_ALLOW_STALE)
cmd_flags |= CMD_STALE;
/* In addition the MAY_REPLICATE flag is set for these commands, but
* if we have flags we know if it's gonna do any writes or not. */
cmd_flags &= ~CMD_MAY_REPLICATE;
return cmd_flags;
}
/* Prepare the given run ctx for execution */
int scriptPrepareForRun(scriptRunCtx *run_ctx, client *engine_client, client *caller, const char *funcname, uint64_t script_flags, int ro) {
serverAssert(!curr_run_ctx);
@ -123,12 +142,6 @@ int scriptPrepareForRun(scriptRunCtx *run_ctx, client *engine_client, client *ca
return C_ERR;
}
if (!(script_flags & SCRIPT_FLAG_ALLOW_OOM) && server.script_oom && server.maxmemory) {
addReplyError(caller, "-OOM allow-oom flag is not set on the script, "
"can not run it when used memory > 'maxmemory'");
return C_ERR;
}
if (running_stale && !(script_flags & SCRIPT_FLAG_ALLOW_STALE)) {
addReplyError(caller, "-MASTERDOWN Link with MASTER is down, "
"replica-serve-stale-data is set to 'no' "
@ -142,7 +155,7 @@ int scriptPrepareForRun(scriptRunCtx *run_ctx, client *engine_client, client *ca
* 2. no disk error detected
* 3. command is not `fcall_ro`/`eval[sha]_ro` */
if (server.masterhost && server.repl_slave_ro && !obey_client) {
addReplyError(caller, "Can not run script with write flag on readonly replica");
addReplyError(caller, "-READONLY Can not run script with write flag on readonly replica");
return C_ERR;
}
@ -177,6 +190,17 @@ int scriptPrepareForRun(scriptRunCtx *run_ctx, client *engine_client, client *ca
return C_ERR;
}
}
/* Check OOM state. the no-writes flag imply allow-oom. we tested it
* after the no-write error, so no need to mention it in the error reply. */
if (server.pre_command_oom_state && server.maxmemory &&
!(script_flags & (SCRIPT_FLAG_ALLOW_OOM|SCRIPT_FLAG_NO_WRITES)))
{
addReplyError(caller, "-OOM allow-oom flag is not set on the script, "
"can not run it when used memory > 'maxmemory'");
return C_ERR;
}
} else {
/* Special handling for backwards compatibility (no shebang eval[sha]) mode */
if (running_stale) {
@ -202,8 +226,6 @@ int scriptPrepareForRun(scriptRunCtx *run_ctx, client *engine_client, client *ca
script_client->flags |= CLIENT_MULTI;
}
server.in_script = 1;
run_ctx->start_time = getMonotonicUs();
run_ctx->snapshot_time = mstime();
@ -216,6 +238,8 @@ int scriptPrepareForRun(scriptRunCtx *run_ctx, client *engine_client, client *ca
run_ctx->flags |= SCRIPT_READ_ONLY;
}
if (!(script_flags & SCRIPT_FLAG_EVAL_COMPAT_MODE) && (script_flags & SCRIPT_FLAG_ALLOW_OOM)) {
/* Note: we don't need to test the no-writes flag here and set this run_ctx flag,
* since only write commands can are deny-oom. */
run_ctx->flags |= SCRIPT_ALLOW_OOM;
}
@ -236,7 +260,6 @@ void scriptResetRun(scriptRunCtx *run_ctx) {
/* After the script done, remove the MULTI state. */
run_ctx->c->flags &= ~CLIENT_MULTI;
server.in_script = 0;
server.script_caller = NULL;
if (scriptIsTimedout()) {
@ -323,16 +346,23 @@ static int scriptVerifyACL(client *c, sds *err) {
}
static int scriptVerifyWriteCommandAllow(scriptRunCtx *run_ctx, char **err) {
if (!(run_ctx->c->cmd->flags & CMD_WRITE)) {
return C_OK;
}
if (run_ctx->flags & SCRIPT_READ_ONLY) {
/* We know its a write command, on a read only run we do not allow it. */
/* A write command, on an RO command or an RO script is rejected ASAP.
* Note: For scripts, we consider may-replicate commands as write commands.
* This also makes it possible to allow read-only scripts to be run during
* CLIENT PAUSE WRITE. */
if (run_ctx->flags & SCRIPT_READ_ONLY &&
(run_ctx->c->cmd->flags & (CMD_WRITE|CMD_MAY_REPLICATE)))
{
*err = sdsnew("Write commands are not allowed from read-only scripts.");
return C_ERR;
}
/* The other checks below are on the server state and are only relevant for
* write commands, return if this is not a write command. */
if (!(run_ctx->c->cmd->flags & CMD_WRITE))
return C_OK;
/* Write commands are forbidden against read-only slaves, or if a
* command marked as non-deterministic was already called in the context
* of this script. */
@ -362,16 +392,6 @@ static int scriptVerifyWriteCommandAllow(scriptRunCtx *run_ctx, char **err) {
return C_OK;
}
static int scriptVerifyMayReplicate(scriptRunCtx *run_ctx, char **err) {
if (run_ctx->c->cmd->flags & CMD_MAY_REPLICATE &&
server.client_pause_type == CLIENT_PAUSE_WRITE) {
*err = sdsnew("May-replicate commands are not allowed when client pause write.");
return C_ERR;
}
return C_OK;
}
static int scriptVerifyOOM(scriptRunCtx *run_ctx, char **err) {
if (run_ctx->flags & SCRIPT_ALLOW_OOM) {
/* Allow running any command even if OOM reached */
@ -385,8 +405,8 @@ static int scriptVerifyOOM(scriptRunCtx *run_ctx, char **err) {
if (server.maxmemory && /* Maxmemory is actually enabled. */
!mustObeyClient(run_ctx->original_client) && /* Don't care about mem for replicas or AOF. */
!(run_ctx->flags & SCRIPT_WRITE_DIRTY) && /* Script had no side effects so far. */
server.script_oom && /* Detected OOM when script start. */
!(run_ctx->flags & SCRIPT_WRITE_DIRTY) && /* Script had no side effects so far. */
server.pre_command_oom_state && /* Detected OOM when script start. */
(run_ctx->c->cmd->flags & CMD_DENYOOM))
{
*err = sdsdup(shared.oomerr->ptr);
@ -526,10 +546,6 @@ void scriptCall(scriptRunCtx *run_ctx, robj* *argv, int argc, sds *err) {
goto error;
}
if (scriptVerifyMayReplicate(run_ctx, err) != C_OK) {
goto error;
}
if (scriptVerifyOOM(run_ctx, err) != C_OK) {
goto error;
}

1
src/script.h
View File

@ -93,6 +93,7 @@ typedef struct scriptFlag {
extern scriptFlag scripts_flags_def[];
uint64_t scriptFlagsToCmdFlags(uint64_t cmd_flags, uint64_t script_flags);
int scriptPrepareForRun(scriptRunCtx *r_ctx, client *engine_client, client *caller, const char *funcname, uint64_t script_flags, int ro);
void scriptResetRun(scriptRunCtx *r_ctx);
int scriptSetResp(scriptRunCtx *r_ctx, int resp);

7
src/script_lua.c
View File

@ -1651,8 +1651,11 @@ void luaCallFunction(scriptRunCtx* run_ctx, lua_State *lua, robj** keys, size_t
* {err='<error msg>', source='<source file>', line=<line>}
* We can construct the error message from this information */
if (!lua_istable(lua, -1)) {
/* Should not happened, and we should considered assert it */
addReplyErrorFormat(c,"Error running script (call to %s)\n", run_ctx->funcname);
const char *msg = "execution failure";
if (lua_isstring(lua, -1)) {
msg = lua_tostring(lua, -1);
}
addReplyErrorFormat(c,"Error running script %s, %.100s\n", run_ctx->funcname, msg);
} else {
errorInfo err_info = {0};
sds final_msg = sdsempty();

2
src/sentinel.c
View File

@ -4127,7 +4127,7 @@ numargserr:
void addInfoSectionsToDict(dict *section_dict, char **sections);
/* SENTINEL INFO [section] */
/* INFO [<section> [<section> ...]] */
void sentinelInfoCommand(client *c) {
char *sentinel_sections[] = {"server", "clients", "cpu", "stats", "sentinel", NULL};
int sec_all = 0, sec_everything = 0;

315
src/server.c
View File

@ -36,6 +36,7 @@
#include "atomicvar.h"
#include "mt19937-64.h"
#include "functions.h"
#include "syscheck.h"
#include <time.h>
#include <signal.h>
@ -638,6 +639,11 @@ int isMutuallyExclusiveChildType(int type) {
return type == CHILD_TYPE_RDB || type == CHILD_TYPE_AOF || type == CHILD_TYPE_MODULE;
}
/* Returns true when we're inside a long command that yielded to the event loop. */
int isInsideYieldingLongCommand() {
return scriptIsTimedout() || server.busy_module_yield_flags;
}
/* Return true if this instance has persistence completely turned off:
* both RDB and AOF are disabled. */
int allPersistenceDisabled(void) {
@ -1187,14 +1193,21 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
run_with_period(100) {
long long stat_net_input_bytes, stat_net_output_bytes;
long long stat_net_repl_input_bytes, stat_net_repl_output_bytes;
atomicGet(server.stat_net_input_bytes, stat_net_input_bytes);
atomicGet(server.stat_net_output_bytes, stat_net_output_bytes);
atomicGet(server.stat_net_repl_input_bytes, stat_net_repl_input_bytes);
atomicGet(server.stat_net_repl_output_bytes, stat_net_repl_output_bytes);
trackInstantaneousMetric(STATS_METRIC_COMMAND,server.stat_numcommands);
trackInstantaneousMetric(STATS_METRIC_NET_INPUT,
stat_net_input_bytes);
stat_net_input_bytes + stat_net_repl_input_bytes);
trackInstantaneousMetric(STATS_METRIC_NET_OUTPUT,
stat_net_output_bytes);
stat_net_output_bytes + stat_net_repl_output_bytes);
trackInstantaneousMetric(STATS_METRIC_NET_INPUT_REPLICATION,
stat_net_repl_input_bytes);
trackInstantaneousMetric(STATS_METRIC_NET_OUTPUT_REPLICATION,
stat_net_repl_output_bytes);
}
/* We have just LRU_BITS bits per object for LRU information.
@ -1754,6 +1767,7 @@ void createSharedObjects(void) {
shared.unsubscribebulk = createStringObject("$11\r\nunsubscribe\r\n",18);
shared.ssubscribebulk = createStringObject("$10\r\nssubscribe\r\n", 17);
shared.sunsubscribebulk = createStringObject("$12\r\nsunsubscribe\r\n", 19);
shared.smessagebulk = createStringObject("$8\r\nsmessage\r\n", 14);
shared.psubscribebulk = createStringObject("$10\r\npsubscribe\r\n",17);
shared.punsubscribebulk = createStringObject("$12\r\npunsubscribe\r\n",19);
@ -2354,6 +2368,8 @@ void resetServerStats(void) {
server.stat_aofrw_consecutive_failures = 0;
atomicSet(server.stat_net_input_bytes, 0);
atomicSet(server.stat_net_output_bytes, 0);
atomicSet(server.stat_net_repl_input_bytes, 0);
atomicSet(server.stat_net_repl_output_bytes, 0);
server.stat_unexpected_error_replies = 0;
server.stat_total_error_replies = 0;
server.stat_dump_payload_sanitizations = 0;
@ -2500,7 +2516,6 @@ void initServer(void) {
server.pubsub_patterns = dictCreate(&keylistDictType);
server.pubsubshard_channels = dictCreate(&keylistDictType);
server.cronloops = 0;
server.in_script = 0;
server.in_exec = 0;
server.busy_module_yield_flags = BUSY_MODULE_YIELD_NONE;
server.busy_module_yield_reply = NULL;
@ -2716,7 +2731,8 @@ void commandAddSubcommand(struct redisCommand *parent, struct redisCommand *subc
void setImplicitACLCategories(struct redisCommand *c) {
if (c->flags & CMD_WRITE)
c->acl_categories |= ACL_CATEGORY_WRITE;
if (c->flags & CMD_READONLY)
/* Exclude scripting commands from the RO category. */
if (c->flags & CMD_READONLY && !(c->acl_categories & ACL_CATEGORY_SCRIPTING))
c->acl_categories |= ACL_CATEGORY_READ;
if (c->flags & CMD_ADMIN)
c->acl_categories |= ACL_CATEGORY_ADMIN|ACL_CATEGORY_DANGEROUS;
@ -3392,8 +3408,12 @@ void call(client *c, int flags) {
(CLIENT_FORCE_AOF|CLIENT_FORCE_REPL|CLIENT_PREVENT_PROP);
/* If the client has keys tracking enabled for client side caching,
* make sure to remember the keys it fetched via this command. */
if (c->cmd->flags & CMD_READONLY) {
* make sure to remember the keys it fetched via this command. Scripting
* works a bit differently, where if the scripts executes any read command, it
* remembers all of the declared keys from the script. */
if ((c->cmd->flags & CMD_READONLY) && (c->cmd->proc != evalRoCommand)
&& (c->cmd->proc != evalShaRoCommand) && (c->cmd->proc != fcallroCommand))
{
client *caller = (c->flags & CLIENT_SCRIPT && server.script_caller) ?
server.script_caller : c;
if (caller->flags & CLIENT_TRACKING &&
@ -3482,11 +3502,8 @@ void afterCommand(client *c) {
* spec doesn't cover all of them. */
void populateCommandMovableKeys(struct redisCommand *cmd) {
int movablekeys = 0;
if (cmd->getkeys_proc && !(cmd->flags & CMD_MODULE)) {
/* Redis command with getkeys proc */
movablekeys = 1;
} else if (cmd->flags & CMD_MODULE_GETKEYS) {
/* Module command with getkeys proc */
if (cmd->getkeys_proc || (cmd->flags & CMD_MODULE_GETKEYS)) {
/* Command with getkeys proc */
movablekeys = 1;
} else {
/* Redis command without getkeys proc, but possibly has
@ -3569,7 +3586,7 @@ int processCommand(client *c) {
* That is unless lua_timedout, in which case client may run
* some commands. */
serverAssert(!server.in_exec);
serverAssert(!server.in_script);
serverAssert(!scriptIsRunning());
}
moduleCallCommandFilters(c);
@ -3618,19 +3635,33 @@ int processCommand(client *c) {
}
}
int is_read_command = (c->cmd->flags & CMD_READONLY) ||
/* If we're executing a script, try to extract a set of command flags from
* it, in case it declared them. Note this is just an attempt, we don't yet
* know the script command is well formed.*/
uint64_t cmd_flags = c->cmd->flags;
if (c->cmd->proc == evalCommand || c->cmd->proc == evalShaCommand ||
c->cmd->proc == evalRoCommand || c->cmd->proc == evalShaRoCommand ||
c->cmd->proc == fcallCommand || c->cmd->proc == fcallroCommand)
{
if (c->cmd->proc == fcallCommand || c->cmd->proc == fcallroCommand)
cmd_flags = fcallGetCommandFlags(c, cmd_flags);
else
cmd_flags = evalGetCommandFlags(c, cmd_flags);
}
int is_read_command = (cmd_flags & CMD_READONLY) ||
(c->cmd->proc == execCommand && (c->mstate.cmd_flags & CMD_READONLY));
int is_write_command = (c->cmd->flags & CMD_WRITE) ||
int is_write_command = (cmd_flags & CMD_WRITE) ||
(c->cmd->proc == execCommand && (c->mstate.cmd_flags & CMD_WRITE));
int is_denyoom_command = (c->cmd->flags & CMD_DENYOOM) ||
int is_denyoom_command = (cmd_flags & CMD_DENYOOM) ||
(c->cmd->proc == execCommand && (c->mstate.cmd_flags & CMD_DENYOOM));
int is_denystale_command = !(c->cmd->flags & CMD_STALE) ||
int is_denystale_command = !(cmd_flags & CMD_STALE) ||
(c->cmd->proc == execCommand && (c->mstate.cmd_inv_flags & CMD_STALE));
int is_denyloading_command = !(c->cmd->flags & CMD_LOADING) ||
int is_denyloading_command = !(cmd_flags & CMD_LOADING) ||
(c->cmd->proc == execCommand && (c->mstate.cmd_inv_flags & CMD_LOADING));
int is_may_replicate_command = (c->cmd->flags & (CMD_WRITE | CMD_MAY_REPLICATE)) ||
int is_may_replicate_command = (cmd_flags & (CMD_WRITE | CMD_MAY_REPLICATE)) ||
(c->cmd->proc == execCommand && (c->mstate.cmd_flags & (CMD_WRITE | CMD_MAY_REPLICATE)));
int is_deny_async_loading_command = (c->cmd->flags & CMD_NO_ASYNC_LOADING) ||
int is_deny_async_loading_command = (cmd_flags & CMD_NO_ASYNC_LOADING) ||
(c->cmd->proc == execCommand && (c->mstate.cmd_flags & CMD_NO_ASYNC_LOADING));
int obey_client = mustObeyClient(c);
@ -3718,7 +3749,7 @@ int processCommand(client *c) {
* the event loop since there is a busy Lua script running in timeout
* condition, to avoid mixing the propagation of scripts with the
* propagation of DELs due to eviction. */
if (server.maxmemory && !scriptIsTimedout()) {
if (server.maxmemory && !isInsideYieldingLongCommand()) {
int out_of_memory = (performEvictions() == EVICT_FAIL);
/* performEvictions may evict keys, so we need flush pending tracking
@ -3750,16 +3781,12 @@ int processCommand(client *c) {
return C_OK;
}
/* Save out_of_memory result at script start, otherwise if we check OOM
* until first write within script, memory used by lua stack and
* arguments might interfere. */
if (c->cmd->proc == evalCommand ||
c->cmd->proc == evalShaCommand ||
c->cmd->proc == fcallCommand ||
c->cmd->proc == fcallroCommand)
{
server.script_oom = out_of_memory;
}
/* Save out_of_memory result at command start, otherwise if we check OOM
* in the first write within script, memory used by lua stack and
* arguments might interfere. We need to save it for EXEC and module
* calls too, since these can call EVAL, but avoid saving it during an
* interrupted / yielding busy script / module. */
server.pre_command_oom_state = out_of_memory;
}
/* Make sure to use a reasonable amount of memory for client side
@ -3787,6 +3814,8 @@ int processCommand(client *c) {
}
} else {
sds err = writeCommandsGetDiskErrorMessage(deny_write_type);
/* remove the newline since rejectCommandSds adds it. */
sdssubstr(err, 0, sdslen(err)-2);
rejectCommandSds(c, err);
return C_OK;
}
@ -3859,7 +3888,7 @@ int processCommand(client *c) {
* the MULTI plus a few initial commands refused, then the timeout
* condition resolves, and the bottom-half of the transaction gets
* executed, see Github PR #7022. */
if ((scriptIsTimedout() || server.busy_module_yield_flags) && !(c->cmd->flags & CMD_ALLOW_BUSY)) {
if (isInsideYieldingLongCommand() && !(c->cmd->flags & CMD_ALLOW_BUSY)) {
if (server.busy_module_yield_flags && server.busy_module_yield_reply) {
rejectCommandFormat(c, "-BUSY %s", server.busy_module_yield_reply);
} else if (server.busy_module_yield_flags) {
@ -3900,7 +3929,7 @@ int processCommand(client *c) {
c->cmd->proc != quitCommand &&
c->cmd->proc != resetCommand)
{
queueMultiCommand(c);
queueMultiCommand(c, cmd_flags);
addReply(c,shared.queued);
} else {
call(c,CMD_CALL_FULL);
@ -4226,7 +4255,7 @@ sds writeCommandsGetDiskErrorMessage(int error_code) {
ret = sdsdup(shared.bgsaveerr->ptr);
} else {
ret = sdscatfmt(sdsempty(),
"-MISCONF Errors writing to the AOF file: %s",
"-MISCONF Errors writing to the AOF file: %s\r\n",
strerror(server.aof_last_write_errno));
}
return ret;
@ -4312,7 +4341,7 @@ void addReplyFlagsForCommand(client *c, struct redisCommand *cmd) {
{CMD_ASKING, "asking"},
{CMD_FAST, "fast"},
{CMD_NO_AUTH, "no_auth"},
{CMD_MAY_REPLICATE, "may_replicate"},
/* {CMD_MAY_REPLICATE, "may_replicate"},, Hidden on purpose */
/* {CMD_SENTINEL, "sentinel"}, Hidden on purpose */
/* {CMD_ONLY_SENTINEL, "only_sentinel"}, Hidden on purpose */
{CMD_NO_MANDATORY_KEYS, "no_mandatory_keys"},
@ -4709,7 +4738,7 @@ void getKeysSubcommandImpl(client *c, int with_flags) {
if (!cmd) {
addReplyError(c,"Invalid command specified");
return;
} else if (cmd->getkeys_proc == NULL && cmd->key_specs_num == 0) {
} else if (!doesCommandHaveKeys(cmd)) {
addReplyError(c,"The command has no key arguments");
return;
} else if ((cmd->arity > 0 && cmd->arity != c->argc-2) ||
@ -4911,7 +4940,7 @@ void commandInfoCommand(client *c) {
}
}
/* COMMAND DOCS [<command-name> ...] */
/* COMMAND DOCS [command-name [command-name ...]] */
void commandDocsCommand(client *c) {
int i;
if (c->argc == 2) {
@ -5571,6 +5600,7 @@ sds genRedisInfoString(dict *section_dict, int all_sections, int everything) {
if (all_sections || (dictFind(section_dict,"stats") != NULL)) {
long long stat_total_reads_processed, stat_total_writes_processed;
long long stat_net_input_bytes, stat_net_output_bytes;
long long stat_net_repl_input_bytes, stat_net_repl_output_bytes;
long long current_eviction_exceeded_time = server.stat_last_eviction_exceeded_time ?
(long long) elapsedUs(server.stat_last_eviction_exceeded_time): 0;
long long current_active_defrag_time = server.stat_last_active_defrag_time ?
@ -5579,6 +5609,8 @@ sds genRedisInfoString(dict *section_dict, int all_sections, int everything) {
atomicGet(server.stat_total_writes_processed, stat_total_writes_processed);
atomicGet(server.stat_net_input_bytes, stat_net_input_bytes);
atomicGet(server.stat_net_output_bytes, stat_net_output_bytes);
atomicGet(server.stat_net_repl_input_bytes, stat_net_repl_input_bytes);
atomicGet(server.stat_net_repl_output_bytes, stat_net_repl_output_bytes);
if (sections++) info = sdscat(info,"\r\n");
info = sdscatprintf(info,
@ -5588,8 +5620,12 @@ sds genRedisInfoString(dict *section_dict, int all_sections, int everything) {
"instantaneous_ops_per_sec:%lld\r\n"
"total_net_input_bytes:%lld\r\n"
"total_net_output_bytes:%lld\r\n"
"total_net_repl_input_bytes:%lld\r\n"
"total_net_repl_output_bytes:%lld\r\n"
"instantaneous_input_kbps:%.2f\r\n"
"instantaneous_output_kbps:%.2f\r\n"
"instantaneous_input_repl_kbps:%.2f\r\n"
"instantaneous_output_repl_kbps:%.2f\r\n"
"rejected_connections:%lld\r\n"
"sync_full:%lld\r\n"
"sync_partial_ok:%lld\r\n"
@ -5631,10 +5667,14 @@ sds genRedisInfoString(dict *section_dict, int all_sections, int everything) {
server.stat_numconnections,
server.stat_numcommands,
getInstantaneousMetric(STATS_METRIC_COMMAND),
stat_net_input_bytes,
stat_net_output_bytes,
stat_net_input_bytes + stat_net_repl_input_bytes,
stat_net_output_bytes + stat_net_repl_output_bytes,
stat_net_repl_input_bytes,
stat_net_repl_output_bytes,
(float)getInstantaneousMetric(STATS_METRIC_NET_INPUT)/1024,
(float)getInstantaneousMetric(STATS_METRIC_NET_OUTPUT)/1024,
(float)getInstantaneousMetric(STATS_METRIC_NET_INPUT_REPLICATION)/1024,
(float)getInstantaneousMetric(STATS_METRIC_NET_OUTPUT_REPLICATION)/1024,
server.stat_rejected_conn,
server.stat_sync_full,
server.stat_sync_partial_ok,
@ -5920,6 +5960,7 @@ sds genRedisInfoString(dict *section_dict, int all_sections, int everything) {
return info;
}
/* INFO [<section> [<section> ...]] */
void infoCommand(client *c) {
if (server.sentinel_mode) {
sentinelInfoCommand(c);
@ -5970,168 +6011,38 @@ int checkIgnoreWarning(const char *warning) {
}
#ifdef __linux__
int linuxOvercommitMemoryValue(void) {
FILE *fp = fopen("/proc/sys/vm/overcommit_memory","r");
char buf[64];
#include <sys/prctl.h>
/* since linux-3.5, kernel supports to set the state of the "THP disable" flag
* for the calling thread. PR_SET_THP_DISABLE is defined in linux/prctl.h */
static int THPDisable(void) {
int ret = -EINVAL;
if (!fp) return -1;
if (fgets(buf,64,fp) == NULL) {
fclose(fp);
return -1;
}
fclose(fp);
if (!server.disable_thp)
return ret;
return atoi(buf);
#ifdef PR_SET_THP_DISABLE
ret = prctl(PR_SET_THP_DISABLE, 1, 0, 0, 0);
#endif
return ret;
}
void linuxMemoryWarnings(void) {
if (linuxOvercommitMemoryValue() == 0) {
serverLog(LL_WARNING,"WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.");
sds err_msg;
if (checkOvercommit(&err_msg) < 0) {
serverLog(LL_WARNING,"WARNING %s", err_msg);
sdsfree(err_msg);
}
if (THPIsEnabled()) {
if (checkTHPEnabled(&err_msg) < 0) {
server.thp_enabled = 1;
if (THPDisable() == 0) {
server.thp_enabled = 0;
return;
} else {
serverLog(LL_WARNING, "WARNING %s", err_msg);
}
serverLog(LL_WARNING,"WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled (set to 'madvise' or 'never').");
sdsfree(err_msg);
}
}
#ifdef __arm64__
/* Get size in kilobytes of the Shared_Dirty pages of the calling process for the
* memory map corresponding to the provided address, or -1 on error. */
static int smapsGetSharedDirty(unsigned long addr) {
int ret, in_mapping = 0, val = -1;
unsigned long from, to;
char buf[64];
FILE *f;
f = fopen("/proc/self/smaps", "r");
if (!f) return -1;
while (1) {
if (!fgets(buf, sizeof(buf), f))
break;
ret = sscanf(buf, "%lx-%lx", &from, &to);
if (ret == 2)
in_mapping = from <= addr && addr < to;
if (in_mapping && !memcmp(buf, "Shared_Dirty:", 13)) {
sscanf(buf, "%*s %d", &val);
/* If parsing fails, we remain with val == -1 */
break;
}
}
fclose(f);
return val;
}
/* Older arm64 Linux kernels have a bug that could lead to data corruption
* during background save in certain scenarios. This function checks if the
* kernel is affected.
* The bug was fixed in commit ff1712f953e27f0b0718762ec17d0adb15c9fd0b
* titled: "arm64: pgtable: Ensure dirty bit is preserved across pte_wrprotect()"
* Return -1 on unexpected test failure, 1 if the kernel seems to be affected,
* and 0 otherwise. */
int linuxMadvFreeForkBugCheck(void) {
int ret, pipefd[2] = { -1, -1 };
pid_t pid;
char *p = NULL, *q;
int bug_found = 0;
long page_size = sysconf(_SC_PAGESIZE);
long map_size = 3 * page_size;
/* Create a memory map that's in our full control (not one used by the allocator). */
p = mmap(NULL, map_size, PROT_READ, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
if (p == MAP_FAILED) {
serverLog(LL_WARNING, "Failed to mmap(): %s", strerror(errno));
return -1;
}
q = p + page_size;
/* Split the memory map in 3 pages by setting their protection as RO|RW|RO to prevent
* Linux from merging this memory map with adjacent VMAs. */
ret = mprotect(q, page_size, PROT_READ | PROT_WRITE);
if (ret < 0) {
serverLog(LL_WARNING, "Failed to mprotect(): %s", strerror(errno));
bug_found = -1;
goto exit;
}
/* Write to the page once to make it resident */
*(volatile char*)q = 0;
/* Tell the kernel that this page is free to be reclaimed. */
#ifndef MADV_FREE
#define MADV_FREE 8
#endif
ret = madvise(q, page_size, MADV_FREE);
if (ret < 0) {
/* MADV_FREE is not available on older kernels that are presumably
* not affected. */
if (errno == EINVAL) goto exit;
serverLog(LL_WARNING, "Failed to madvise(): %s", strerror(errno));
bug_found = -1;
goto exit;
}
/* Write to the page after being marked for freeing, this is supposed to take
* ownership of that page again. */
*(volatile char*)q = 0;
/* Create a pipe for the child to return the info to the parent. */
ret = anetPipe(pipefd, 0, 0);
if (ret < 0) {
serverLog(LL_WARNING, "Failed to create pipe: %s", strerror(errno));
bug_found = -1;
goto exit;
}
/* Fork the process. */
pid = fork();
if (pid < 0) {
serverLog(LL_WARNING, "Failed to fork: %s", strerror(errno));
bug_found = -1;
goto exit;
} else if (!pid) {
/* Child: check if the page is marked as dirty, page_size in kb.
* A value of 0 means the kernel is affected by the bug. */
ret = smapsGetSharedDirty((unsigned long) q);
if (!ret)
bug_found = 1;
else if (ret == -1) /* Failed to read */
bug_found = -1;
if (write(pipefd[1], &bug_found, sizeof(bug_found)) < 0)
serverLog(LL_WARNING, "Failed to write to parent: %s", strerror(errno));
exitFromChild(0);
} else {
/* Read the result from the child. */
ret = read(pipefd[0], &bug_found, sizeof(bug_found));
if (ret < 0) {
serverLog(LL_WARNING, "Failed to read from child: %s", strerror(errno));
bug_found = -1;
}
/* Reap the child pid. */
waitpid(pid, NULL, 0);
}
exit:
/* Cleanup */
if (pipefd[0] != -1) close(pipefd[0]);
if (pipefd[1] != -1) close(pipefd[1]);
if (p != NULL) munmap(p, map_size);
return bug_found;
}
#endif /* __arm64__ */
#endif /* __linux__ */
void createPidFile(void) {
@ -6568,6 +6479,8 @@ void loadDataFromDisk(void) {
int ret = loadAppendOnlyFiles(server.aof_manifest);
if (ret == AOF_FAILED || ret == AOF_OPEN_ERR)
exit(1);
if (ret != AOF_NOT_EXIST)
serverLog(LL_NOTICE, "DB loaded from append only file: %.3f seconds", (float)(ustime()-start)/1000000);
} else {
rdbSaveInfo rsi = RDB_SAVE_INFO_INIT;
errno = 0; /* Prevent a stale value from affecting error checking */
@ -6950,6 +6863,8 @@ int main(int argc, char **argv) {
fprintf(stderr,"Example: ./redis-server --test-memory 4096\n\n");
exit(1);
}
} if (strcmp(argv[1], "--check-system") == 0) {
exit(syscheck() ? 0 : 1);
}
/* Parse command line options
* Precedence wise, File, stdin, explicit options -- last config is the one that matters.
@ -6962,6 +6877,7 @@ int main(int argc, char **argv) {
server.exec_argv[1] = zstrdup(server.configfile);
j = 2; // Skip this arg when parsing options
}
int handled_last_config_arg = 1;
while(j < argc) {
/* Either first or last argument - Should we read config from stdin? */
if (argv[j][0] == '-' && argv[j][1] == '\0' && (j == 1 || j == argc-1)) {
@ -6970,16 +6886,20 @@ int main(int argc, char **argv) {
/* All the other options are parsed and conceptually appended to the
* configuration file. For instance --port 6380 will generate the
* string "port 6380\n" to be parsed after the actual config file
* and stdin input are parsed (if they exist). */
else if (argv[j][0] == '-' && argv[j][1] == '-') {
* and stdin input are parsed (if they exist).
* Only consider that if the last config has at least one argument. */
else if (handled_last_config_arg && argv[j][0] == '-' && argv[j][1] == '-') {
/* Option name */
if (sdslen(options)) options = sdscat(options,"\n");
/* argv[j]+2 for removing the preceding `--` */
options = sdscat(options,argv[j]+2);
options = sdscat(options," ");
handled_last_config_arg = 0;
} else {
/* Option argument */
options = sdscatrepr(options,argv[j],strlen(argv[j]));
options = sdscat(options," ");
handled_last_config_arg = 1;
}
j++;
}
@ -7019,13 +6939,18 @@ int main(int argc, char **argv) {
serverLog(LL_WARNING,"Server initialized");
#ifdef __linux__
linuxMemoryWarnings();
sds err_msg;
if (checkXenClocksource(&err_msg) < 0) {
serverLog(LL_WARNING, "WARNING %s", err_msg);
sdsfree(err_msg);
}
#if defined (__arm64__)
int ret;
if ((ret = linuxMadvFreeForkBugCheck())) {
if (ret == 1)
serverLog(LL_WARNING,"WARNING Your kernel has a bug that could lead to data corruption during background save. "
"Please upgrade to the latest stable kernel.");
else
if ((ret = checkLinuxMadvFreeForkBug(&err_msg)) <= 0) {
if (ret < 0) {
serverLog(LL_WARNING, "WARNING %s", err_msg);
sdsfree(err_msg);
} else
serverLog(LL_WARNING, "Failed to test the kernel for a bug that could lead to data corruption during background save. "
"Your system could be affected, please report this error.");
if (!checkIgnoreWarning("ARM64-COW-BUG")) {

39
src/server.h
View File

@ -70,7 +70,6 @@ typedef long long ustime_t; /* microsecond time type. */
#include "adlist.h" /* Linked lists */
#include "zmalloc.h" /* total memory usage aware version of malloc/free */
#include "anet.h" /* Networking the easy way */
#include "ziplist.h" /* Compact list data structure */
#include "intset.h" /* Compact integer set structure */
#include "version.h" /* Version macro */
#include "util.h" /* Misc functions useful in many places */
@ -86,11 +85,14 @@ typedef long long ustime_t; /* microsecond time type. */
/* Following includes allow test functions to be called from Redis main() */
#include "zipmap.h"
#include "ziplist.h" /* Compact list data structure */
#include "sha1.h"
#include "endianconv.h"
#include "crc64.h"
/* min/max */
#undef min
#undef max
#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))
@ -112,7 +114,6 @@ typedef long long ustime_t; /* microsecond time type. */
#define LOG_MAX_LEN 1024 /* Default maximum length of syslog messages.*/
#define AOF_REWRITE_ITEMS_PER_CMD 64
#define AOF_ANNOTATION_LINE_MAX_LEN 1024
#define CONFIG_AUTHPASS_MAX_LEN 512
#define CONFIG_RUN_ID_SIZE 40
#define RDB_EOF_MARK_SIZE 40
#define CONFIG_REPL_BACKLOG_MIN_SIZE (1024*16) /* 16k */
@ -153,9 +154,11 @@ typedef long long ustime_t; /* microsecond time type. */
/* Instantaneous metrics tracking. */
#define STATS_METRIC_SAMPLES 16 /* Number of samples per metric. */
#define STATS_METRIC_COMMAND 0 /* Number of commands executed. */
#define STATS_METRIC_NET_INPUT 1 /* Bytes read to network .*/
#define STATS_METRIC_NET_INPUT 1 /* Bytes read to network. */
#define STATS_METRIC_NET_OUTPUT 2 /* Bytes written to network. */
#define STATS_METRIC_COUNT 3
#define STATS_METRIC_NET_INPUT_REPLICATION 3 /* Bytes read to network during replication. */
#define STATS_METRIC_NET_OUTPUT_REPLICATION 4 /* Bytes written to network during replication. */
#define STATS_METRIC_COUNT 5
/* Protocol and I/O related defines */
#define PROTO_IOBUF_LEN (1024*16) /* Generic I/O buffer size */
@ -289,7 +292,7 @@ extern int configOOMScoreAdjValuesDefaults[CONFIG_OOM_COUNT];
#define AOF_ON 1 /* AOF is on */
#define AOF_WAIT_REWRITE 2 /* AOF waits rewrite to start appending */
/* AOF return values for loadAppendOnlyFile() */
/* AOF return values for loadAppendOnlyFiles() and loadSingleAppendOnlyFile() */
#define AOF_OK 0
#define AOF_NOT_EXIST 1
#define AOF_EMPTY 2
@ -674,6 +677,7 @@ struct redisObject;
struct RedisModuleDefragCtx;
struct RedisModuleInfoCtx;
struct RedisModuleKeyOptCtx;
struct RedisModuleCommand;
/* Each module type implementation should export a set of methods in order
* to serialize and deserialize the value in the RDB file, rewrite the AOF
@ -825,9 +829,9 @@ typedef struct RedisModuleDigest {
#define OBJ_ENCODING_RAW 0 /* Raw representation */
#define OBJ_ENCODING_INT 1 /* Encoded as integer */
#define OBJ_ENCODING_HT 2 /* Encoded as hash table */
#define OBJ_ENCODING_ZIPMAP 3 /* Encoded as zipmap */
#define OBJ_ENCODING_ZIPMAP 3 /* No longer used: old hash encoding. */
#define OBJ_ENCODING_LINKEDLIST 4 /* No longer used: old list encoding. */
#define OBJ_ENCODING_ZIPLIST 5 /* Encoded as ziplist */
#define OBJ_ENCODING_ZIPLIST 5 /* No longer used: old list/hash/zset encoding. */
#define OBJ_ENCODING_INTSET 6 /* Encoded as intset */
#define OBJ_ENCODING_SKIPLIST 7 /* Encoded as skiplist */
#define OBJ_ENCODING_EMBSTR 8 /* Embedded sds string encoding */
@ -1077,6 +1081,7 @@ typedef struct {
typedef struct client {
uint64_t id; /* Client incremental unique ID. */
uint64_t flags; /* Client flags: CLIENT_* macros. */
connection *conn;
int resp; /* RESP protocol version. Can be 2 or 3. */
redisDb *db; /* Pointer to currently SELECTed DB. */
@ -1110,7 +1115,6 @@ typedef struct client {
int slot; /* The slot the client is executing against. Set to -1 if no slot is being used */
time_t lastinteraction; /* Time of the last interaction, used for timeout */
time_t obuf_soft_limit_reached_time;
uint64_t flags; /* Client flags: CLIENT_* macros. */
int authenticated; /* Needed when the default user requires auth. */
int replstate; /* Replication state if this is a slave. */
int repl_start_cmd_stream_on_ack; /* Install slave write handler on first ACK. */
@ -1225,7 +1229,7 @@ struct sharedObjectsStruct {
*time, *pxat, *absttl, *retrycount, *force, *justid, *entriesread,
*lastid, *ping, *setid, *keepttl, *load, *createconsumer,
*getack, *special_asterick, *special_equals, *default_username, *redacted,
*ssubscribebulk,*sunsubscribebulk,
*ssubscribebulk,*sunsubscribebulk, *smessagebulk,
*select[PROTO_SHARED_SELECT_CMDS],
*integers[OBJ_SHARED_INTEGERS],
*mbulkhdr[OBJ_SHARED_BULKHDR_LEN], /* "*<value>\r\n" */
@ -1473,7 +1477,6 @@ struct redisServer {
int sentinel_mode; /* True if this instance is a Sentinel. */
size_t initial_memory_usage; /* Bytes used after initialization. */
int always_show_logo; /* Show logo even for non-stdout logging. */
int in_script; /* Are we inside EVAL? */
int in_exec; /* Are we inside EXEC? */
int busy_module_yield_flags; /* Are we inside a busy module? (triggered by RM_Yield). see BUSY_MODULE_YIELD_ flags. */
const char *busy_module_yield_reply; /* When non-null, we are inside RM_Yield. */
@ -1584,6 +1587,8 @@ struct redisServer {
struct malloc_stats cron_malloc_stats; /* sampled in serverCron(). */
redisAtomic long long stat_net_input_bytes; /* Bytes read from network. */
redisAtomic long long stat_net_output_bytes; /* Bytes written to network. */
redisAtomic long long stat_net_repl_input_bytes; /* Bytes read during replication, added to stat_net_input_bytes in 'info'. */
redisAtomic long long stat_net_repl_output_bytes; /* Bytes written during replication, added to stat_net_output_bytes in 'info'. */
size_t stat_current_cow_peak; /* Peak size of copy on write bytes. */
size_t stat_current_cow_bytes; /* Copy on write bytes while child is active. */
monotime stat_current_cow_updated; /* Last update time of stat_current_cow_bytes */
@ -1884,7 +1889,7 @@ struct redisServer {
/* Scripting */
client *script_caller; /* The client running script right now, or NULL */
mstime_t busy_reply_threshold; /* Script / module timeout in milliseconds */
int script_oom; /* OOM detected when script start */
int pre_command_oom_state; /* OOM before command (script?) was started */
int script_disable_deny_script; /* Allow running commands marked "no-script" inside a script. */
/* Lazy free */
int lazyfree_lazy_eviction;
@ -2258,6 +2263,7 @@ struct redisCommand {
dict *subcommands_dict; /* A dictionary that holds the subcommands, the key is the subcommand sds name
* (not the fullname), and the value is the redisCommand structure pointer. */
struct redisCommand *parent;
struct RedisModuleCommand *module_cmd; /* A pointer to the module command data (NULL if native command) */
};
struct redisError {
@ -2586,7 +2592,7 @@ void listElementsRemoved(client *c, robj *key, int where, robj *o, long count, i
void unwatchAllKeys(client *c);
void initClientMultiState(client *c);
void freeClientMultiState(client *c);
void queueMultiCommand(client *c);
void queueMultiCommand(client *c, uint64_t cmd_flags);
size_t multiStateMemOverhead(client *c);
void touchWatchedKey(redisDb *db, robj *key);
int isWatchedKeyExpired(client *c);
@ -2988,7 +2994,7 @@ void pubsubUnsubscribeShardChannels(robj **channels, unsigned int count);
int pubsubUnsubscribeAllPatterns(client *c, int notify);
int pubsubPublishMessage(robj *channel, robj *message, int sharded);
int pubsubPublishMessageAndPropagateToCluster(robj *channel, robj *message, int sharded);
void addReplyPubsubMessage(client *c, robj *channel, robj *msg);
void addReplyPubsubMessage(client *c, robj *channel, robj *msg, robj *message_bulk);
int serverPubsubSubscriptionCount();
int serverPubsubShardSubscriptionCount();
@ -3010,6 +3016,10 @@ sds keyspaceEventsFlagsToString(int flags);
#define DENY_LOADING_CONFIG (1ULL<<6) /* This config is forbidden during loading. */
#define ALIAS_CONFIG (1ULL<<7) /* For configs with multiple names, this flag is set on the alias. */
#define MODULE_CONFIG (1ULL<<8) /* This config is a module config */
#define VOLATILE_CONFIG (1ULL<<9) /* The config is a reference to the config data and not the config data itself (ex.
* a file name containing more configuration like a tls key). In this case we want
* to apply the configuration change even if the new config value is the same as
* the old. */
#define INTEGER_CONFIG 0 /* No flags means a simple integer configuration */
#define MEMORY_CONFIG (1<<0) /* Indicates if this value can be loaded as a memory value */
@ -3191,6 +3201,9 @@ void sha1hex(char *digest, char *script, size_t len);
unsigned long evalMemory();
dict* evalScriptsDict();
unsigned long evalScriptsMemory();
uint64_t evalGetCommandFlags(client *c, uint64_t orig_flags);
uint64_t fcallGetCommandFlags(client *c, uint64_t orig_flags);
int isInsideYieldingLongCommand();
typedef struct luaScript {
uint64_t flags;

372
src/syscheck.c Normal file
View File

@ -0,0 +1,372 @@
/*
* Copyright (c) 2022, Redis Ltd.
* Copyright (c) 2016, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "fmacros.h"
#include "config.h"
#include "syscheck.h"
#include "sds.h"
#include "anet.h"
#include <time.h>
#include <sys/resource.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <sys/wait.h>
#ifdef __linux__
#include <sys/mman.h>
#endif
#ifdef __linux__
static sds read_sysfs_line(char *path) {
char buf[256];
FILE *f = fopen(path, "r");
if (!f) return NULL;
if (!fgets(buf, sizeof(buf), f)) {
fclose(f);
return NULL;
}
fclose(f);
sds res = sdsnew(buf);
res = sdstrim(res, " \n");
return res;
}
/* Verify our clokcsource implementation doesn't go through a system call (uses vdso).
* Going through a system call to check the time degrades Redis performance. */
static int checkClocksource(sds *error_msg) {
unsigned long test_time_us, system_hz;
struct timespec ts;
unsigned long long start_us;
struct rusage ru_start, ru_end;
system_hz = sysconf(_SC_CLK_TCK);
if (getrusage(RUSAGE_SELF, &ru_start) != 0)
return 0;
if (clock_gettime(CLOCK_MONOTONIC, &ts) < 0) {
return 0;
}
start_us = (ts.tv_sec * 1000000 + ts.tv_nsec / 1000);
/* clock_gettime() busy loop of 5 times system tick (for a system_hz of 100 this is 50ms)
* Using system_hz is required to ensure accurate measurements from getrusage().
* If our clocksource is configured correctly (vdso) this will result in no system calls.
* If our clocksource is inefficient it'll waste most of the busy loop in the kernel. */
test_time_us = 5 * 1000000 / system_hz;
while (1) {
unsigned long long d;
if (clock_gettime(CLOCK_MONOTONIC, &ts) < 0)
return 0;
d = (ts.tv_sec * 1000000 + ts.tv_nsec / 1000) - start_us;
if (d >= test_time_us) break;
}
if (getrusage(RUSAGE_SELF, &ru_end) != 0)
return 0;
long long stime_us = (ru_end.ru_stime.tv_sec * 1000000 + ru_end.ru_stime.tv_usec) - (ru_start.ru_stime.tv_sec * 1000000 + ru_start.ru_stime.tv_usec);
long long utime_us = (ru_end.ru_utime.tv_sec * 1000000 + ru_end.ru_utime.tv_usec) - (ru_start.ru_utime.tv_sec * 1000000 + ru_start.ru_utime.tv_usec);
/* If more than 10% of the process time was in system calls we probably have an inefficient clocksource, print a warning */
if (stime_us * 10 > stime_us + utime_us) {
sds avail = read_sysfs_line("/sys/devices/system/clocksource/clocksource0/available_clocksource");
sds curr = read_sysfs_line("/sys/devices/system/clocksource/clocksource0/current_clocksource");
*error_msg = sdscatprintf(sdsempty(),
"Slow system clocksource detected. This can result in degraded performance. "
"Consider changing the system's clocksource. "
"Current clocksource: %s. Available clocksources: %s. "
"For example: run the command 'echo tsc > /sys/devices/system/clocksource/clocksource0/current_clocksource' as root. "
"To permanently change the system's clocksource you'll need to set the 'clocksource=' kernel command line parameter.",
curr ? curr : "", avail ? avail : "");
sdsfree(avail);
sdsfree(curr);
return -1;
} else {
return 1;
}
}
/* Verify we're not using the `xen` clocksource. The xen hypervisor's default clocksource is slow and affects
* Redis's performance. This has been measured on ec2 xen based instances. ec2 recommends using the non-default
* tsc clock source for these instances. */
int checkXenClocksource(sds *error_msg) {
sds curr = read_sysfs_line("/sys/devices/system/clocksource/clocksource0/current_clocksource");
int res = 1;
if (curr == NULL) {
res = 0;
} else if (strcmp(curr, "xen") == 0) {
*error_msg = sdsnew(
"Your system is configured to use the 'xen' clocksource which might lead to degraded performance. "
"Check the result of the [slow-clocksource] system check: run 'redis-server --check-system' to check if "
"the system's clocksource isn't degrading performance.");
res = -1;
}
sdsfree(curr);
return res;
}
/* Verify overcommit is enabled.
* When overcommit memory is disabled Linux will kill the forked child of a background save
* if we don't have enough free memory to satisfy double the current memory usage even though
* the forked child uses copy-on-write to reduce its actual memory usage. */
int checkOvercommit(sds *error_msg) {
FILE *fp = fopen("/proc/sys/vm/overcommit_memory","r");
char buf[64];
if (!fp) return -1;
if (fgets(buf,64,fp) == NULL) {
fclose(fp);
return 0;
}
fclose(fp);
if (atoi(buf)) {
*error_msg = sdsnew(
"WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. "
"To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the "
"command 'sysctl vm.overcommit_memory=1' for this to take effect.");
return -1;
} else {
return 1;
}
}
/* Make sure transparent huge pages aren't always enabled. When they are this can cause copy-on-write logic
* to consume much more memory and reduce performance during forks. */
int checkTHPEnabled(sds *error_msg) {
char buf[1024];
FILE *fp = fopen("/sys/kernel/mm/transparent_hugepage/enabled","r");
if (!fp) return 0;
if (fgets(buf,sizeof(buf),fp) == NULL) {
fclose(fp);
return 0;
}
fclose(fp);
if (strstr(buf,"[always]") != NULL) {
*error_msg = sdsnew(
"You have Transparent Huge Pages (THP) support enabled in your kernel. "
"This will create latency and memory usage issues with Redis. "
"To fix this issue run the command 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled' as root, "
"and add it to your /etc/rc.local in order to retain the setting after a reboot. "
"Redis must be restarted after THP is disabled (set to 'madvise' or 'never').");
return -1;
} else {
return 1;
}
}
#ifdef __arm64__
/* Get size in kilobytes of the Shared_Dirty pages of the calling process for the
* memory map corresponding to the provided address, or -1 on error. */
static int smapsGetSharedDirty(unsigned long addr) {
int ret, in_mapping = 0, val = -1;
unsigned long from, to;
char buf[64];
FILE *f;
f = fopen("/proc/self/smaps", "r");
if (!f) return -1;
while (1) {
if (!fgets(buf, sizeof(buf), f))
break;
ret = sscanf(buf, "%lx-%lx", &from, &to);
if (ret == 2)
in_mapping = from <= addr && addr < to;
if (in_mapping && !memcmp(buf, "Shared_Dirty:", 13)) {
sscanf(buf, "%*s %d", &val);
/* If parsing fails, we remain with val == -1 */
break;
}
}
fclose(f);
return val;
}
/* Older arm64 Linux kernels have a bug that could lead to data corruption
* during background save in certain scenarios. This function checks if the
* kernel is affected.
* The bug was fixed in commit ff1712f953e27f0b0718762ec17d0adb15c9fd0b
* titled: "arm64: pgtable: Ensure dirty bit is preserved across pte_wrprotect()"
*/
int checkLinuxMadvFreeForkBug(sds *error_msg) {
int ret, pipefd[2] = { -1, -1 };
pid_t pid;
char *p = NULL, *q;
int res = 1;
long page_size = sysconf(_SC_PAGESIZE);
long map_size = 3 * page_size;
/* Create a memory map that's in our full control (not one used by the allocator). */
p = mmap(NULL, map_size, PROT_READ, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
if (p == MAP_FAILED) {
return 0;
}
q = p + page_size;
/* Split the memory map in 3 pages by setting their protection as RO|RW|RO to prevent
* Linux from merging this memory map with adjacent VMAs. */
ret = mprotect(q, page_size, PROT_READ | PROT_WRITE);
if (ret < 0) {
res = 0;
goto exit;
}
/* Write to the page once to make it resident */
*(volatile char*)q = 0;
/* Tell the kernel that this page is free to be reclaimed. */
#ifndef MADV_FREE
#define MADV_FREE 8
#endif
ret = madvise(q, page_size, MADV_FREE);
if (ret < 0) {
/* MADV_FREE is not available on older kernels that are presumably
* not affected. */
if (errno == EINVAL) goto exit;
res = 0;
goto exit;
}
/* Write to the page after being marked for freeing, this is supposed to take
* ownership of that page again. */
*(volatile char*)q = 0;
/* Create a pipe for the child to return the info to the parent. */
ret = anetPipe(pipefd, 0, 0);
if (ret < 0) {
res = 0;
goto exit;
}
/* Fork the process. */
pid = fork();
if (pid < 0) {
res = 0;
goto exit;
} else if (!pid) {
/* Child: check if the page is marked as dirty, page_size in kb.
* A value of 0 means the kernel is affected by the bug. */
ret = smapsGetSharedDirty((unsigned long) q);
if (!ret)
res = -1;
else if (ret == -1) /* Failed to read */
res = 0;
ret = write(pipefd[1], &res, sizeof(res)); /* Assume success, ignore return value*/
exit(0);
} else {
/* Read the result from the child. */
ret = read(pipefd[0], &res, sizeof(res));
if (ret < 0) {
res = 0;
}
/* Reap the child pid. */
waitpid(pid, NULL, 0);
}
exit:
/* Cleanup */
if (pipefd[0] != -1) close(pipefd[0]);
if (pipefd[1] != -1) close(pipefd[1]);
if (p != NULL) munmap(p, map_size);
if (res == -1)
*error_msg = sdsnew(
"Your kernel has a bug that could lead to data corruption during background save. "
"Please upgrade to the latest stable kernel.");
return res;
}
#endif /* __arm64__ */
#endif /* __linux__ */
/*
* Standard system check interface:
* Each check has a name `name` and a functions pointer `check_fn`.
* `check_fn` should return:
* -1 in case the check fails.
* 1 in case the check passes.
* 0 in case the check could not be completed (usually because of some unexpected failed system call).
* When (and only when) the check fails and -1 is returned and error description is places in a new sds pointer to by
* the single `sds*` argument to `check_fn`. This message should be freed by the caller via `sdsfree()`.
*/
typedef struct {
const char *name;
int (*check_fn)(sds*);
} check;
check checks[] = {
#ifdef __linux__
{.name = "slow-clocksource", .check_fn = checkClocksource},
{.name = "xen-clocksource", .check_fn = checkXenClocksource},
{.name = "overcommit", .check_fn = checkOvercommit},
{.name = "THP", .check_fn = checkTHPEnabled},
#ifdef __arm64__
{.name = "madvise-free-fork-bug", .check_fn = checkLinuxMadvFreeForkBug},
#endif
#endif
{.name = NULL, .check_fn = NULL}
};
/* Performs various system checks, returns 0 if any check fails, 1 otherwise. */
int syscheck(void) {
check *cur_check = checks;
int ret = 1;
sds err_msg;
while (cur_check->check_fn) {
int res = cur_check->check_fn(&err_msg);
printf("[%s]...", cur_check->name);
if (res == 0) {
printf("skipped\n");
} else if (res == 1) {
printf("OK\n");
} else {
printf("WARNING:\n");
printf("%s\n", err_msg);
sdsfree(err_msg);
ret = 0;
}
cur_check++;
}
return ret;
}

46
src/syscheck.h Normal file
View File

@ -0,0 +1,46 @@
/*
* Copyright (c) 2022, Redis Ltd.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __SYSCHECK_H
#define __SYSCHECK_H
#include "sds.h"
#include "config.h"
int syscheck(void);
#ifdef __linux__
int checkXenClocksource(sds *error_msg);
int checkTHPEnabled(sds *error_msg);
int checkOvercommit(sds *error_msg);
#ifdef __arm64__
int checkLinuxMadvFreeForkBug(sds *error_msg);
#endif
#endif
#endif

7
src/t_list.c
View File

@ -679,7 +679,8 @@ void lposCommand(client *c) {
return;
if (rank == 0) {
addReplyError(c,"RANK can't be zero: use 1 to start from "
"the first match, 2 from the second, ...");
"the first match, 2 from the second ... "
"or use negative to start from the end of the list");
return;
}
} else if (!strcasecmp(opt,"COUNT") && moreargs) {
@ -1197,12 +1198,12 @@ void lmpopGenericCommand(client *c, int numkeys_idx, int is_block) {
}
}
/* LMPOP numkeys [<key> ...] LEFT|RIGHT [COUNT count] */
/* LMPOP numkeys <key> [<key> ...] (LEFT|RIGHT) [COUNT count] */
void lmpopCommand(client *c) {
lmpopGenericCommand(c, 1, 0);
}
/* BLMPOP timeout numkeys [<key> ...] LEFT|RIGHT [COUNT count] */
/* BLMPOP timeout numkeys <key> [<key> ...] (LEFT|RIGHT) [COUNT count] */
void blmpopCommand(client *c) {
lmpopGenericCommand(c, 2, 1);
}

14
src/t_set.c
View File

@ -203,7 +203,7 @@ sds setTypeNextObject(setTypeIterator *si) {
* The caller provides both pointers to be populated with the right
* object. The return value of the function is the object->encoding
* field of the object and is used by the caller to check if the
* int64_t pointer or the redis object pointer was populated.
* int64_t pointer or the sds pointer was populated.
*
* Note that both the sdsele and llele pointers should be passed and cannot
* be NULL since the function will try to defensively populate the non
@ -805,7 +805,7 @@ void srandmemberWithCountCommand(client *c) {
}
}
/* SRANDMEMBER [<count>] */
/* SRANDMEMBER <key> [<count>] */
void srandmemberCommand(client *c) {
robj *set;
sds ele;
@ -850,7 +850,7 @@ int qsortCompareSetsByRevCardinality(const void *s1, const void *s2) {
return 0;
}
/* SINTER / SINTERSTORE / SINTERCARD
/* SINTER / SMEMBERS / SINTERSTORE / SINTERCARD
*
* 'cardinality_only' work for SINTERCARD, only return the cardinality
* with minimum processing and memory overheads.
@ -1055,6 +1055,7 @@ void sunionDiffGenericCommand(client *c, robj **setkeys, int setnum,
sds ele;
int j, cardinality = 0;
int diff_algo = 1;
int sameset = 0;
for (j = 0; j < setnum; j++) {
robj *setobj = lookupKeyRead(c->db, setkeys[j]);
@ -1067,6 +1068,9 @@ void sunionDiffGenericCommand(client *c, robj **setkeys, int setnum,
return;
}
sets[j] = setobj;
if (j > 0 && sets[0] == sets[j]) {
sameset = 1;
}
}
/* Select what DIFF algorithm to use.
@ -1078,7 +1082,7 @@ void sunionDiffGenericCommand(client *c, robj **setkeys, int setnum,
* the sets.
*
* We compute what is the best bet with the current input here. */
if (op == SET_OP_DIFF && sets[0]) {
if (op == SET_OP_DIFF && sets[0] && !sameset) {
long long algo_one_work = 0, algo_two_work = 0;
for (j = 0; j < setnum; j++) {
@ -1120,6 +1124,8 @@ void sunionDiffGenericCommand(client *c, robj **setkeys, int setnum,
}
setTypeReleaseIterator(si);
}
} else if (op == SET_OP_DIFF && sameset) {
/* At least one of the sets is the same one (same key) as the first one, result must be empty. */
} else if (op == SET_OP_DIFF && sets[0] && diff_algo == 1) {
/* DIFF Algorithm 1:
*

4
src/t_stream.c
View File

@ -401,7 +401,7 @@ void streamGetEdgeID(stream *s, int first, int skip_tombstones, streamID *edge_i
streamID min_id = {0, 0}, max_id = {UINT64_MAX, UINT64_MAX};
*edge_id = first ? max_id : min_id;
}
streamIteratorStop(&si);
}
/* Adds a new item into the stream 's' having the specified number of
@ -946,7 +946,7 @@ static int streamParseAddOrTrimArgsOrReply(client *c, streamAddTrimArgs *args, i
}
args->approx_trim = 0;
char *next = c->argv[i+1]->ptr;
/* Check for the form MINID ~ <id>|<age>. */
/* Check for the form MINID ~ <id> */
if (moreargs >= 2 && next[0] == '~' && next[1] == '\0') {
args->approx_trim = 1;
i++;

17
src/t_zset.c
View File

@ -927,7 +927,7 @@ int zzlLexValueLteMax(unsigned char *p, zlexrangespec *spec) {
}
/* Returns if there is a part of the zset is in range. Should only be used
* internally by zzlFirstInRange and zzlLastInRange. */
* internally by zzlFirstInLexRange and zzlLastInLexRange. */
int zzlIsInLexRange(unsigned char *zl, zlexrangespec *range) {
unsigned char *p;
@ -1186,9 +1186,10 @@ void zsetConvert(robj *zobj, int encoding) {
zs->zsl = zslCreate();
eptr = lpSeek(zl,0);
serverAssertWithInfo(NULL,zobj,eptr != NULL);
sptr = lpNext(zl,eptr);
serverAssertWithInfo(NULL,zobj,sptr != NULL);
if (eptr != NULL) {
sptr = lpNext(zl,eptr);
serverAssertWithInfo(NULL,zobj,sptr != NULL);
}
while (eptr != NULL) {
score = zzlGetScore(sptr);
@ -4041,7 +4042,7 @@ void blockingGenericZpopCommand(client *c, robj **keys, int numkeys, int where,
/* If the keys do not exist we must block */
struct blockPos pos = {where};
blockForKeys(c,BLOCKED_ZSET,c->argv+1,c->argc-2,count,timeout,NULL,&pos,NULL);
blockForKeys(c,BLOCKED_ZSET,keys,numkeys,count,timeout,NULL,&pos,NULL);
}
// BZPOPMIN key [key ...] timeout
@ -4054,7 +4055,7 @@ void bzpopmaxCommand(client *c) {
blockingGenericZpopCommand(c, c->argv+1, c->argc-2, ZSET_MAX, c->argc-1, -1, 0, 0);
}
static void zarndmemberReplyWithListpack(client *c, unsigned int count, listpackEntry *keys, listpackEntry *vals) {
static void zrandmemberReplyWithListpack(client *c, unsigned int count, listpackEntry *keys, listpackEntry *vals) {
for (unsigned long i = 0; i < count; i++) {
if (vals && c->resp > 2)
addReplyArrayLen(c,2);
@ -4135,7 +4136,7 @@ void zrandmemberWithCountCommand(client *c, long l, int withscores) {
sample_count = count > limit ? limit : count;
count -= sample_count;
lpRandomPairs(zsetobj->ptr, sample_count, keys, vals);
zarndmemberReplyWithListpack(c, sample_count, keys, vals);
zrandmemberReplyWithListpack(c, sample_count, keys, vals);
}
zfree(keys);
zfree(vals);
@ -4234,7 +4235,7 @@ void zrandmemberWithCountCommand(client *c, long l, int withscores) {
if (withscores)
vals = zmalloc(sizeof(listpackEntry)*count);
serverAssert(lpRandomPairsUnique(zsetobj->ptr, count, keys, vals) == count);
zarndmemberReplyWithListpack(c, count, keys, vals);
zrandmemberReplyWithListpack(c, count, keys, vals);
zfree(keys);
zfree(vals);
zuiClearIterator(&src);

2
src/tracking.c
View File

@ -295,7 +295,7 @@ void sendTrackingMessage(client *c, char *keyname, size_t keylen, int proto) {
} else if (using_redirection && c->flags & CLIENT_PUBSUB) {
/* We use a static object to speedup things, however we assume
* that addReplyPubsubMessage() will not take a reference. */
addReplyPubsubMessage(c,TrackingChannelName,NULL);
addReplyPubsubMessage(c,TrackingChannelName,NULL,shared.messagebulk);
} else {
/* If are here, the client is not using RESP3, nor is
* redirecting to another client. We can't send anything to

4
src/util.c
View File

@ -522,7 +522,7 @@ int string2ld(const char *s, size_t slen, long double *dp) {
if (isspace(buf[0]) || eptr[0] != '\0' ||
(size_t)(eptr-buf) != slen ||
(errno == ERANGE &&
(value == HUGE_VAL || value == -HUGE_VAL || value == 0)) ||
(value == HUGE_VAL || value == -HUGE_VAL || fpclassify(value) == FP_ZERO)) ||
errno == EINVAL ||
isnan(value))
return 0;
@ -546,7 +546,7 @@ int string2d(const char *s, size_t slen, double *dp) {
isspace(((const char*)s)[0]) ||
(size_t)(eptr-(char*)s) != slen ||
(errno == ERANGE &&
(*dp == HUGE_VAL || *dp == -HUGE_VAL || *dp == 0)) ||
(*dp == HUGE_VAL || *dp == -HUGE_VAL || fpclassify(*dp) == FP_ZERO)) ||
isnan(*dp))
return 0;
return 1;

4
src/version.h
View File

@ -1,2 +1,2 @@
#define REDIS_VERSION "7.0.0"
#define REDIS_VERSION_NUM 0x00070000
#define REDIS_VERSION "7.0.1"
#define REDIS_VERSION_NUM 0x00070001

2
src/ziplist.c
View File

@ -255,7 +255,7 @@
/* Return the pointer to the last byte of a ziplist, which is, the
* end of ziplist FF entry. */
#define ZIPLIST_ENTRY_END(zl) ((zl)+intrev32ifbe(ZIPLIST_BYTES(zl))-1)
#define ZIPLIST_ENTRY_END(zl) ((zl)+intrev32ifbe(ZIPLIST_BYTES(zl))-ZIPLIST_END_SIZE)
/* Increment the number of items field in the ziplist header. Note that this
* macro should never overflow the unsigned 16 bit integer, since entries are

79
src/zmalloc.c
View File

@ -55,6 +55,8 @@ void zlibc_free(void *ptr) {
#include "zmalloc.h"
#include "atomicvar.h"
#define UNUSED(x) ((void)(x))
#ifdef HAVE_MALLOC_SIZE
#define PREFIX_SIZE (0)
#define ASSERT_NO_SIZE_OVERFLOW(sz)
@ -395,35 +397,58 @@ void zmadvise_dontneed(void *ptr) {
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#endif
size_t zmalloc_get_rss(void) {
int page = sysconf(_SC_PAGESIZE);
size_t rss;
/* Get the i'th field from "/proc/self/stats" note i is 1 based as appears in the 'proc' man page */
int get_proc_stat_ll(int i, long long *res) {
#if defined(HAVE_PROC_STAT)
char buf[4096];
char filename[256];
int fd, count;
int fd, l;
char *p, *x;
snprintf(filename,256,"/proc/%ld/stat",(long) getpid());
if ((fd = open(filename,O_RDONLY)) == -1) return 0;
if (read(fd,buf,4096) <= 0) {
if ((fd = open("/proc/self/stat",O_RDONLY)) == -1) return 0;
if ((l = read(fd,buf,sizeof(buf)-1)) <= 0) {
close(fd);
return 0;
}
close(fd);
buf[l] = '\0';
if (buf[l-1] == '\n') buf[l-1] = '\0';
p = buf;
count = 23; /* RSS is the 24th field in /proc/<pid>/stat */
while(p && count--) {
p = strchr(p,' ');
if (p) p++;
}
/* Skip pid and process name (surrounded with parentheses) */
p = strrchr(buf, ')');
if (!p) return 0;
x = strchr(p,' ');
if (!x) return 0;
*x = '\0';
p++;
while (*p == ' ') p++;
if (*p == '\0') return 0;
i -= 3;
if (i < 0) return 0;
rss = strtoll(p,NULL,10);
while (p && i--) {
p = strchr(p, ' ');
if (p) p++;
else return 0;
}
x = strchr(p,' ');
if (x) *x = '\0';
*res = strtoll(p,&x,10);
if (*x != '\0') return 0;
return 1;
#else
UNUSED(i);
UNUSED(res);
return 0;
#endif
}
#if defined(HAVE_PROC_STAT)
size_t zmalloc_get_rss(void) {
int page = sysconf(_SC_PAGESIZE);
long long rss;
/* RSS is the 24th field in /proc/<pid>/stat */
if (!get_proc_stat_ll(24, &rss)) return 0;
rss *= page;
return rss;
}
@ -492,6 +517,23 @@ size_t zmalloc_get_rss(void) {
return 0L;
}
#elif defined(__HAIKU__)
#include <OS.h>
size_t zmalloc_get_rss(void) {
area_info info;
thread_info th;
size_t rss = 0;
ssize_t cookie = 0;
if (get_thread_info(find_thread(0), &th) != B_OK)
return 0;
while (get_next_area_info(th.team, &cookie, &info) == B_OK)
rss += info.ram_size;
return rss;
}
#elif defined(HAVE_PSINFO)
#include <unistd.h>
#include <sys/procfs.h>
@ -731,7 +773,6 @@ size_t zmalloc_get_memory_size(void) {
}
#ifdef REDIS_TEST
#define UNUSED(x) ((void)(x))
int zmalloc_test(int argc, char **argv, int flags) {
void *ptr;

2
src/zmalloc.h
View File

@ -136,6 +136,8 @@ size_t zmalloc_usable_size(void *ptr);
#define zmalloc_usable_size(p) zmalloc_size(p)
#endif
int get_proc_stat_ll(int i, long long *res);
#ifdef REDIS_TEST
int zmalloc_test(int argc, char **argv, int flags);
#endif

2
tests/README.md
View File

@ -38,7 +38,7 @@ The following compatibility and capability tags are currently used:
| `external:skip` | Not compatible with external servers. |
| `cluster:skip` | Not compatible with `--cluster-mode`. |
| `large-memory` | Test that requires more than 100mb |
| `tls:skip` | Not campatible with `--tls`. |
| `tls:skip` | Not compatible with `--tls`. |
| `needs:repl` | Uses replication and needs to be able to `SYNC` from server. |
| `needs:debug` | Uses the `DEBUG` command or other debugging focused commands (like `OBJECT`). |
| `needs:pfdebug` | Uses the `PFDEBUG` command. |

3
tests/assets/default.conf
View File

@ -13,9 +13,8 @@ databases 16
latency-monitor-threshold 1
repl-diskless-sync-delay 0
# Note the infrastructure in server.tcl uses a dict, we can't provide several save directives
save 900 1
save 300 10
save 60 10000
rdbcompression yes
dbfilename dump.rdb

8
tests/cluster/tests/25-pubsubshard-slot-migration.tcl
View File

@ -29,7 +29,7 @@ test "Migrate a slot, verify client receives sunsubscribe on primary serving the
# Verify subscribe is still valid, able to receive messages.
$nodefrom(link) spublish $channelname hello
assert_equal {message mychannel hello} [$subscribeclient read]
assert_equal {smessage mychannel hello} [$subscribeclient read]
assert_equal {OK} [$nodefrom(link) cluster setslot $slot node $nodeto(id)]
@ -66,7 +66,7 @@ test "Client subscribes to multiple channels, migrate a slot, verify client rece
# Verify subscribe is still valid, able to receive messages.
$nodefrom(link) spublish $channelname hello
assert_equal {message ch3 hello} [$subscribeclient read]
assert_equal {smessage ch3 hello} [$subscribeclient read]
assert_equal {OK} [$nodefrom(link) cluster setslot $slot node $nodeto(id)]
@ -82,7 +82,7 @@ test "Client subscribes to multiple channels, migrate a slot, verify client rece
# Verify the client is still connected and receives message from the other channel.
set msg [$subscribeclient read]
assert {"message" eq [lindex $msg 0]}
assert {"smessage" eq [lindex $msg 0]}
assert {$anotherchannelname eq [lindex $msg 1]}
assert {"hello" eq [lindex $msg 2]}
@ -114,7 +114,7 @@ test "Migrate a slot, verify client receives sunsubscribe on replica serving the
# Verify subscribe is still valid, able to receive messages.
$nodefrom(link) spublish $channelname hello
assert_equal {message mychannel1 hello} [$subscribeclient read]
assert_equal {smessage mychannel1 hello} [$subscribeclient read]
assert_equal {OK} [$nodefrom(link) cluster setslot $slot node $nodeto(id)]
assert_equal {OK} [$nodeto(link) cluster setslot $slot node $nodeto(id)]

4
tests/instances.tcl
View File

@ -441,7 +441,7 @@ proc run_tests {} {
file delete $::leaked_fds_file
}
if {[llength $::run_matching] != 0 && [search_pattern_list $test $::run_matching true] == -1} {
if {[llength $::run_matching] != 0 && ![search_pattern_list $test $::run_matching true]} {
continue
}
if {[file isdirectory $test]} continue
@ -699,4 +699,4 @@ proc redis_client {type id} {
proc redis_client_by_addr {host port} {
set client [redis $host $port 0 $::tls]
return $client
}
}

46
tests/integration/aof.tcl
View File

@ -632,4 +632,50 @@ tags {"aof external:skip"} {
} result
assert_match "*Failed to truncate AOF*to timestamp*because it is not the last file*" $result
}
start_server {overrides {appendonly yes appendfsync always}} {
test {FLUSHDB / FLUSHALL should persist in AOF} {
set aof [get_last_incr_aof_path r]
r set key value
r flushdb
r set key value2
r flushdb
# DB is empty
r flushdb
r flushdb
r flushdb
r set key value
r flushall
r set key value2
r flushall
# DBs are empty.
r flushall
r flushall
r flushall
# Assert that each FLUSHDB command is persisted even the DB is empty.
# Assert that each FLUSHALL command is persisted even the DBs are empty.
assert_aof_content $aof {
{select *}
{set key value}
{flushdb}
{set key value2}
{flushdb}
{flushdb}
{flushdb}
{flushdb}
{set key value}
{flushall}
{set key value2}
{flushall}
{flushall}
{flushall}
{flushall}
}
}
}
}

2
tests/integration/logging.tcl
View File

@ -9,7 +9,7 @@ if {$system_name eq {darwin}} {
} elseif {$system_name eq {linux}} {
# Avoid the test on libmusl, which does not support backtrace
set ldd [exec ldd src/redis-server]
if {![string match {*libc.musl*} $ldd]} {
if {![string match {*libc.*musl*} $ldd]} {
set backtrace_supported 1
}
}

47
tests/integration/rdb.tcl
View File

@ -278,7 +278,7 @@ start_server {overrides {save ""}} {
set current_save_keys_total [s current_save_keys_total]
if {$::verbose} {
puts "Keys before bgsave start: current_save_keys_total"
puts "Keys before bgsave start: $current_save_keys_total"
}
# on each iteration, we will write some key to the server to trigger copy-on-write, and
@ -366,4 +366,49 @@ start_server [list overrides [list "dir" $server_path "dbfilename" "scriptbackup
}
}
start_server {} {
test "failed bgsave prevents writes" {
r config set rdb-key-save-delay 10000000
populate 1000
r set x x
r bgsave
set pid1 [get_child_pid 0]
catch {exec kill -9 $pid1}
waitForBgsave r
# make sure a read command succeeds
assert_equal [r get x] x
# make sure a write command fails
assert_error {MISCONF *} {r set x y}
# repeate with script
assert_error {MISCONF *} {r eval {
return redis.call('set','x',1)
} 1 x
}
assert_equal {x} [r eval {
return redis.call('get','x')
} 1 x
]
# again with script using shebang
assert_error {MISCONF *} {r eval {#!lua
return redis.call('set','x',1)
} 1 x
}
assert_equal {x} [r eval {#!lua flags=no-writes
return redis.call('get','x')
} 1 x
]
r config set rdb-key-save-delay 0
r bgsave
waitForBgsave r
# server is writable again
r set x y
} {OK}
}
} ;# tags

42
tests/integration/replication.tcl
View File

@ -225,11 +225,45 @@ start_server {tags {"repl external:skip"}} {
}
}
test {FLUSHALL should replicate} {
test {FLUSHDB / FLUSHALL should replicate} {
set repl [attach_to_replication_stream]
r -1 set key value
r -1 flushdb
r -1 set key value2
r -1 flushall
if {$::valgrind} {after 2000}
list [r -1 dbsize] [r 0 dbsize]
} {0 0}
wait_for_ofs_sync [srv 0 client] [srv -1 client]
assert_equal [r -1 dbsize] 0
assert_equal [r 0 dbsize] 0
# DB is empty.
r -1 flushdb
r -1 flushdb
r -1 flushdb
# DBs are empty.
r -1 flushall
r -1 flushall
r -1 flushall
# Assert that each FLUSHDB command is replicated even the DB is empty.
# Assert that each FLUSHALL command is replicated even the DBs are empty.
assert_replication_stream $repl {
{set key value}
{flushdb}
{set key value2}
{flushall}
{flushdb}
{flushdb}
{flushdb}
{flushall}
{flushall}
{flushall}
}
close_replication_stream $repl
}
test {ROLE in master reports master with a slave} {
set res [r -1 role]

13
tests/modules/fork.c
View File

@ -23,7 +23,8 @@ void done_handler(int exitcode, int bysignal, void *user_data) {
int fork_create(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
{
long long code_to_exit_with;
if (argc != 2) {
long long usleep_us;
if (argc != 3) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
@ -34,20 +35,22 @@ int fork_create(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
}
RedisModule_StringToLongLong(argv[1], &code_to_exit_with);
RedisModule_StringToLongLong(argv[2], &usleep_us);
exitted_with_code = -1;
child_pid = RedisModule_Fork(done_handler, (void*)0xdeadbeef);
if (child_pid < 0) {
int fork_child_pid = RedisModule_Fork(done_handler, (void*)0xdeadbeef);
if (fork_child_pid < 0) {
RedisModule_ReplyWithError(ctx, "Fork failed");
return REDISMODULE_OK;
} else if (child_pid > 0) {
} else if (fork_child_pid > 0) {
/* parent */
child_pid = fork_child_pid;
RedisModule_ReplyWithLongLong(ctx, child_pid);
return REDISMODULE_OK;
}
/* child */
RedisModule_Log(ctx, "notice", "fork child started");
usleep(500000);
usleep(usleep_us);
RedisModule_Log(ctx, "notice", "fork child exiting");
RedisModule_ExitFromChild(code_to_exit_with);
/* unreachable */

48
tests/modules/misc.c
View File

@ -310,6 +310,50 @@ int test_monotonic_time(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
return REDISMODULE_OK;
}
/* wrapper for RM_Call */
int test_rm_call(RedisModuleCtx *ctx, RedisModuleString **argv, int argc){
if(argc < 2){
return RedisModule_WrongArity(ctx);
}
const char* cmd = RedisModule_StringPtrLen(argv[1], NULL);
RedisModuleCallReply* rep = RedisModule_Call(ctx, cmd, "Ev", argv + 2, argc - 2);
if(!rep){
RedisModule_ReplyWithError(ctx, "NULL reply returned");
}else{
RedisModule_ReplyWithCallReply(ctx, rep);
RedisModule_FreeCallReply(rep);
}
return REDISMODULE_OK;
}
/* wrapper for RM_Call with flags */
int test_rm_call_flags(RedisModuleCtx *ctx, RedisModuleString **argv, int argc){
if(argc < 3){
return RedisModule_WrongArity(ctx);
}
/* Append Ev to the provided flags. */
RedisModuleString *flags = RedisModule_CreateStringFromString(ctx, argv[1]);
RedisModule_StringAppendBuffer(ctx, flags, "Ev", 2);
const char* flg = RedisModule_StringPtrLen(flags, NULL);
const char* cmd = RedisModule_StringPtrLen(argv[2], NULL);
RedisModuleCallReply* rep = RedisModule_Call(ctx, cmd, flg, argv + 3, argc - 3);
if(!rep){
RedisModule_ReplyWithError(ctx, "NULL reply returned");
}else{
RedisModule_ReplyWithCallReply(ctx, rep);
RedisModule_FreeCallReply(rep);
}
RedisModule_FreeString(ctx, flags);
return REDISMODULE_OK;
}
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
REDISMODULE_NOT_USED(argv);
REDISMODULE_NOT_USED(argc);
@ -351,6 +395,10 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"test.monotonic_time", test_monotonic_time,"",0,0,0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx, "test.rm_call", test_rm_call,"allow-stale", 0, 0, 0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx, "test.rm_call_flags", test_rm_call_flags,"allow-stale", 0, 0, 0) == REDISMODULE_ERR)
return REDISMODULE_ERR;
return REDISMODULE_OK;
}

8
tests/modules/moduleconfigs.c
View File

@ -121,13 +121,13 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
}
/* On the stack to make sure we're copying them. */
const char *enum_vals[] = {"none", "one", "two", "three"};
const int int_vals[] = {0, 1, 2, 4};
const char *enum_vals[] = {"none", "five", "one", "two", "four"};
const int int_vals[] = {0, 5, 1, 2, 4};
if (RedisModule_RegisterEnumConfig(ctx, "enum", 1, REDISMODULE_CONFIG_DEFAULT, enum_vals, int_vals, 4, getEnumConfigCommand, setEnumConfigCommand, NULL, NULL) == REDISMODULE_ERR) {
if (RedisModule_RegisterEnumConfig(ctx, "enum", 1, REDISMODULE_CONFIG_DEFAULT, enum_vals, int_vals, 5, getEnumConfigCommand, setEnumConfigCommand, NULL, NULL) == REDISMODULE_ERR) {
return REDISMODULE_ERR;
}
if (RedisModule_RegisterEnumConfig(ctx, "flags", 3, REDISMODULE_CONFIG_DEFAULT | REDISMODULE_CONFIG_BITFLAGS, enum_vals, int_vals, 4, getFlagsConfigCommand, setFlagsConfigCommand, NULL, NULL) == REDISMODULE_ERR) {
if (RedisModule_RegisterEnumConfig(ctx, "flags", 3, REDISMODULE_CONFIG_DEFAULT | REDISMODULE_CONFIG_BITFLAGS, enum_vals, int_vals, 5, getFlagsConfigCommand, setFlagsConfigCommand, NULL, NULL) == REDISMODULE_ERR) {
return REDISMODULE_ERR;
}
/* Memory config here. */

6
tests/sentinel/tests/03-runtime-reconf.tcl
View File

@ -79,8 +79,10 @@ test "Sentinels (re)connection following SENTINEL SET mymaster auth-pass" {
restart_instance sentinel $sent2re
# Verify sentinel that restarted failed to connect master
if {![string match "*disconnected*" [dict get [S $sent2re SENTINEL MASTER mymaster] flags]]} {
fail "Expected to be disconnected from master due to wrong password"
wait_for_condition 100 50 {
[string match "*disconnected*" [dict get [S $sent2re SENTINEL MASTER mymaster] flags]] != 0
} else {
fail "Expected to be disconnected from master due to wrong password"
}
# Update restarted sentinel with master password

22
tests/support/server.tcl
View File

@ -262,16 +262,22 @@ proc create_server_config_file {filename config} {
close $fp
}
proc spawn_server {config_file stdout stderr} {
proc spawn_server {config_file stdout stderr args} {
set cmd [list src/redis-server $config_file]
set args {*}$args
if {[llength $args] > 0} {
lappend cmd {*}$args
}
if {$::valgrind} {
set pid [exec valgrind --track-origins=yes --trace-children=yes --suppressions=[pwd]/src/valgrind.sup --show-reachable=no --show-possibly-lost=no --leak-check=full src/redis-server $config_file >> $stdout 2>> $stderr &]
set pid [exec valgrind --track-origins=yes --trace-children=yes --suppressions=[pwd]/src/valgrind.sup --show-reachable=no --show-possibly-lost=no --leak-check=full {*}$cmd >> $stdout 2>> $stderr &]
} elseif ($::stack_logging) {
set pid [exec /usr/bin/env MallocStackLogging=1 MallocLogFile=/tmp/malloc_log.txt src/redis-server $config_file >> $stdout 2>> $stderr &]
set pid [exec /usr/bin/env MallocStackLogging=1 MallocLogFile=/tmp/malloc_log.txt {*}$cmd >> $stdout 2>> $stderr &]
} else {
# ASAN_OPTIONS environment variable is for address sanitizer. If a test
# tries to allocate huge memory area and expects allocator to return
# NULL, address sanitizer throws an error without this setting.
set pid [exec /usr/bin/env ASAN_OPTIONS=allocator_may_return_null=1 src/redis-server $config_file >> $stdout 2>> $stderr &]
set pid [exec /usr/bin/env ASAN_OPTIONS=allocator_may_return_null=1 {*}$cmd >> $stdout 2>> $stderr &]
}
if {$::wait_server} {
@ -398,6 +404,7 @@ proc start_server {options {code undefined}} {
set overrides {}
set omit {}
set tags {}
set args {}
set keep_persistence false
# parse options
@ -409,6 +416,9 @@ proc start_server {options {code undefined}} {
"overrides" {
set overrides $value
}
"args" {
set args $value
}
"omit" {
set omit $value
}
@ -518,7 +528,7 @@ proc start_server {options {code undefined}} {
send_data_packet $::test_server_fd "server-spawning" "port $port"
set pid [spawn_server $config_file $stdout $stderr]
set pid [spawn_server $config_file $stdout $stderr $args]
# check that the server actually started
set port_busy [wait_server_started $config_file $stdout $pid]
@ -721,7 +731,7 @@ proc restart_server {level wait_ready rotate_logs {reconnect 1} {shutdown sigter
set config_file [dict get $srv "config_file"]
set pid [spawn_server $config_file $stdout $stderr]
set pid [spawn_server $config_file $stdout $stderr {}]
# check that the server actually started
wait_server_started $config_file $stdout $pid

26
tests/support/test.tcl
View File

@ -126,28 +126,36 @@ proc wait_for_condition {maxtries delay e _else_ elsescript} {
}
}
# try to match a value to a list of patterns that is either regex, or plain sub-string
proc search_pattern_list {value pattern_list {substr false}} {
set n 0
# try to match a value to a list of patterns that are either regex (starts with "/") or plain string.
# The caller can specify to use only glob-pattern match
proc search_pattern_list {value pattern_list {glob_pattern false}} {
foreach el $pattern_list {
if {[string length $el] > 0 && ((!$substr && [regexp -- $el $value]) || ($substr && [string match $el $value]))} {
return $n
if {[string length $el] == 0} { continue }
if { $glob_pattern } {
if {[string match $el $value]} {
return 1
}
continue
}
if {[string equal / [string index $el 0]] && [regexp -- [string range $el 1 end] $value]} {
return 1
} elseif {[string equal $el $value]} {
return 1
}
incr n
}
return -1
return 0
}
proc test {name code {okpattern undefined} {tags {}}} {
# abort if test name in skiptests
if {[search_pattern_list $name $::skiptests] >= 0} {
if {[search_pattern_list $name $::skiptests]} {
incr ::num_skipped
send_data_packet $::test_server_fd skip $name
return
}
# abort if only_tests was set but test name is not included
if {[llength $::only_tests] > 0 && [search_pattern_list $name $::only_tests] < 0} {
if {[llength $::only_tests] > 0 && ![search_pattern_list $name $::only_tests]} {
incr ::num_skipped
send_data_packet $::test_server_fd skip $name
return

6
tests/test_helper.tcl
View File

@ -588,15 +588,15 @@ proc print_help_screen {} {
"--single <unit> Just execute the specified unit (see next option). This option can be repeated."
"--verbose Increases verbosity."
"--list-tests List all the available test units."
"--only <test> Just execute tests that match <test> regexp. This option can be repeated."
"--only <test> Just execute the specified test by test name or tests that match <test> regexp (if <test> starts with '/'). This option can be repeated."
"--skip-till <unit> Skip all units until (and including) the specified one."
"--skipunit <unit> Skip one unit."
"--clients <num> Number of test clients (default 16)."
"--timeout <sec> Test timeout in seconds (default 20 min)."
"--force-failure Force the execution of a test that always fails."
"--config <k> <v> Extra config file argument."
"--skipfile <file> Name of a file containing test names or regexp patterns that should be skipped (one per line)."
"--skiptest <test> Test name or regexp pattern to skip. This option can be repeated."
"--skipfile <file> Name of a file containing test names or regexp patterns (if <test> starts with '/') that should be skipped (one per line)."
"--skiptest <test> Test name or regexp pattern (if <test> starts with '/') to skip. This option can be repeated."
"--tags <tags> Run only tests having specified tags or not having '-' prefixed tags."
"--dont-clean Don't delete redis log files after the run."
"--no-latency Skip latency measurements and validation by some tests."

17
tests/unit/acl.tcl
View File

@ -529,6 +529,13 @@ start_server {tags {"acl external:skip"}} {
assert_equal [lsearch [r acl cat stream] "get"] -1
}
test "ACL requires explicit permission for scripting for EVAL_RO, EVALSHA_RO and FCALL_RO" {
r ACL SETUSER scripter on nopass +readonly
assert_equal "This user has no permissions to run the 'eval_ro' command" [r ACL DRYRUN scripter EVAL_RO "" 0]
assert_equal "This user has no permissions to run the 'evalsha_ro' command" [r ACL DRYRUN scripter EVALSHA_RO "" 0]
assert_equal "This user has no permissions to run the 'fcall_ro' command" [r ACL DRYRUN scripter FCALL_RO "" 0]
}
test {ACL #5998 regression: memory leaks adding / removing subcommands} {
r AUTH default ""
r ACL setuser newuser reset -debug +debug|a +debug|b +debug|c
@ -926,3 +933,13 @@ start_server [list overrides [list "dir" $server_path "aclfile" "user.acl"] tags
assert_match {*Duplicate user*} $err
} {} {external:skip}
}
start_server {overrides {user "default on nopass ~* +@all -flushdb"} tags {acl external:skip}} {
test {ACL from config file and config rewrite} {
assert_error {NOPERM *} {r flushdb}
r config rewrite
restart_server 0 true false
assert_error {NOPERM *} {r flushdb}
}
}

32
tests/unit/functions.tcl
View File

@ -424,7 +424,7 @@ start_server {tags {"scripting repl external:skip"}} {
r -1 fcall test 0
} e
set _ $e
} {*Can not run script with write flag on readonly replica*}
} {READONLY You can't write against a read only replica.}
}
}
@ -1052,7 +1052,7 @@ start_server {tags {"scripting"}} {
r config set maxmemory 1
catch {[r fcall f1 1 k]} e
assert_match {*can not run it when used memory > 'maxmemory'*} $e
assert_match {OOM *when used memory > 'maxmemory'*} $e
r config set maxmemory 0
} {OK} {needs:config-maxmemory}
@ -1064,11 +1064,8 @@ start_server {tags {"scripting"}} {
r config set maxmemory 1
catch {r fcall f1 1 k} e
assert_match {*can not run it when used memory > 'maxmemory'*} $e
catch {r fcall_ro f1 1 k} e
assert_match {*can not run it when used memory > 'maxmemory'*} $e
assert_equal [r fcall f1 1 k] hello
assert_equal [r fcall_ro f1 1 k] hello
r config set maxmemory 0
} {OK} {needs:config-maxmemory}
@ -1085,12 +1082,12 @@ start_server {tags {"scripting"}} {
r replicaof 127.0.0.1 1
catch {[r fcall f1 0]} e
assert_match {*'allow-stale' flag is not set on the script*} $e
assert_match {MASTERDOWN *} $e
assert_equal {hello} [r fcall f2 0]
catch {[r fcall f3 0]} e
assert_match {*Can not execute the command on a stale replica*} $e
assert_match {ERR *Can not execute the command on a stale replica*} $e
assert_match {*redis_version*} [r fcall f4 0]
@ -1141,6 +1138,23 @@ start_server {tags {"scripting"}} {
r function stats
} {running_script {} engines {LUA {libraries_count 1 functions_count 1}}}
test {FUNCTION - test function stats on loading failure} {
r FUNCTION FLUSH
r FUNCTION load {#!lua name=test1
redis.register_function('f1', function() return 1 end)
redis.register_function('f2', function() return 1 end)
}
catch {r FUNCTION load {#!lua name=test1
redis.register_function('f3', function() return 1 end)
}} e
assert_match "*Library 'test1' already exists*" $e
r function stats
} {running_script {} engines {LUA {libraries_count 1 functions_count 2}}}
test {FUNCTION - function stats cleaned after flush} {
r function flush
r function stats

116
tests/unit/introspection.tcl
View File

@ -166,11 +166,26 @@ start_server {tags {"introspection"}} {
assert_match [r config get save] {save {3600 1 300 100 60 10000}}
}
# First "save" keyword overrides defaults
# First "save" keyword overrides hard coded defaults
start_server {config "minimal.conf" overrides {save {100 100}}} {
# Defaults
assert_match [r config get save] {save {100 100}}
}
# First "save" keyword in default config file
start_server {config "default.conf"} {
assert_match [r config get save] {save {900 1}}
}
# First "save" keyword appends default from config file
start_server {config "default.conf" args {--save 100 100}} {
assert_match [r config get save] {save {900 1 100 100}}
}
# Empty "save" keyword resets all
start_server {config "default.conf" args {--save {}}} {
assert_match [r config get save] {save {}}
}
} {} {external:skip}
test {CONFIG sanity} {
@ -449,6 +464,70 @@ start_server {tags {"introspection"}} {
assert {[dict exists $res bind]}
}
test {redis-server command line arguments - error cases} {
catch {exec src/redis-server --port} err
assert_match {*'port'*wrong number of arguments*} $err
catch {exec src/redis-server --port 6380 --loglevel} err
assert_match {*'loglevel'*wrong number of arguments*} $err
# Take `6379` and `6380` as the port option value.
catch {exec src/redis-server --port 6379 6380} err
assert_match {*'port "6379" "6380"'*wrong number of arguments*} $err
# Take `--loglevel` and `verbose` as the port option value.
catch {exec src/redis-server --port --loglevel verbose} err
assert_match {*'port "--loglevel" "verbose"'*wrong number of arguments*} $err
# Take `--bla` as the port option value.
catch {exec src/redis-server --port --bla --loglevel verbose} err
assert_match {*'port "--bla"'*argument couldn't be parsed into an integer*} $err
# Take `--bla` as the loglevel option value.
catch {exec src/redis-server --logfile --my--log--file --loglevel --bla} err
assert_match {*'loglevel "--bla"'*argument(s) must be one of the following*} $err
# Using MULTI_ARG's own check, empty option value
catch {exec src/redis-server --shutdown-on-sigint} err
assert_match {*'shutdown-on-sigint'*argument(s) must be one of the following*} $err
catch {exec src/redis-server --shutdown-on-sigint "now force" --shutdown-on-sigterm} err
assert_match {*'shutdown-on-sigterm'*argument(s) must be one of the following*} $err
# Something like `redis-server --some-config --config-value1 --config-value2 --loglevel debug` would break,
# because if you want to pass a value to a config starting with `--`, it can only be a single value.
catch {exec src/redis-server --replicaof 127.0.0.1 abc} err
assert_match {*'replicaof "127.0.0.1" "abc"'*Invalid master port*} $err
catch {exec src/redis-server --replicaof --127.0.0.1 abc} err
assert_match {*'replicaof "--127.0.0.1" "abc"'*Invalid master port*} $err
catch {exec src/redis-server --replicaof --127.0.0.1 --abc} err
assert_match {*'replicaof "--127.0.0.1"'*wrong number of arguments*} $err
} {} {external:skip}
test {redis-server command line arguments - allow option value to use the `--` prefix} {
start_server {config "default.conf" args {--proc-title-template --my--title--template --loglevel verbose}} {
assert_match [r config get proc-title-template] {proc-title-template --my--title--template}
assert_match [r config get loglevel] {loglevel verbose}
}
} {} {external:skip}
test {redis-server command line arguments - save with empty input} {
# Take `--loglevel` as the save option value.
catch {exec src/redis-server --save --loglevel verbose} err
assert_match {*'save "--loglevel" "verbose"'*Invalid save parameters*} $err
start_server {config "default.conf" args {--save {} --loglevel verbose}} {
assert_match [r config get save] {save {}}
assert_match [r config get loglevel] {loglevel verbose}
}
} {} {external:skip}
test {redis-server command line arguments - take one bulk string with spaces for MULTI_ARG configs parsing} {
start_server {config "default.conf" args {--shutdown-on-sigint nosave force now --shutdown-on-sigterm "nosave force"}} {
assert_match [r config get shutdown-on-sigint] {shutdown-on-sigint {nosave now force}}
assert_match [r config get shutdown-on-sigterm] {shutdown-on-sigterm {nosave force}}
}
} {} {external:skip}
# Config file at this point is at a weird state, and includes all
# known keywords. Might be a good idea to avoid adding tests here.
}
@ -503,3 +582,38 @@ test {config during loading} {
exec kill [srv 0 pid]
}
} {} {external:skip}
test {CONFIG REWRITE handles rename-command properly} {
start_server {tags {"introspection"} overrides {rename-command {flushdb badger}}} {
assert_error {ERR unknown command*} {r flushdb}
r config rewrite
restart_server 0 true false
assert_error {ERR unknown command*} {r flushdb}
}
} {} {external:skip}
test {CONFIG REWRITE handles alias config properly} {
start_server {tags {"introspection"} overrides {hash-max-listpack-entries 20 hash-max-ziplist-entries 21}} {
assert_equal [r config get hash-max-listpack-entries] {hash-max-listpack-entries 21}
assert_equal [r config get hash-max-ziplist-entries] {hash-max-ziplist-entries 21}
r config set hash-max-listpack-entries 100
r config rewrite
restart_server 0 true false
assert_equal [r config get hash-max-listpack-entries] {hash-max-listpack-entries 100}
}
# test the order doesn't matter
start_server {tags {"introspection"} overrides {hash-max-ziplist-entries 20 hash-max-listpack-entries 21}} {
assert_equal [r config get hash-max-listpack-entries] {hash-max-listpack-entries 21}
assert_equal [r config get hash-max-ziplist-entries] {hash-max-ziplist-entries 21}
r config set hash-max-listpack-entries 100
r config rewrite
restart_server 0 true false
assert_equal [r config get hash-max-listpack-entries] {hash-max-listpack-entries 100}
}
} {} {external:skip}

26
tests/unit/moduleapi/fork.tcl
View File

@ -13,7 +13,8 @@ start_server {tags {"modules"}} {
test {Module fork} {
# the argument to fork.create is the exitcode on termination
r fork.create 3
# the second argument to fork.create is passed to usleep
r fork.create 3 100000 ;# 100ms
wait_for_condition 20 100 {
[r fork.exitcode] != -1
} else {
@ -23,22 +24,25 @@ start_server {tags {"modules"}} {
} {3}
test {Module fork kill} {
r fork.create 3
after 250
# use a longer time to avoid the child exiting before being killed
r fork.create 3 100000000 ;# 100s
wait_for_condition 20 100 {
[count_log_message "fork child started"] == 2
} else {
fail "fork didn't start"
}
# module fork twice
assert_error {Fork failed} {r fork.create 0 1}
assert {[count_log_message "Can't fork for module: File exists"] eq "1"}
r fork.kill
assert {[count_log_message "fork child started"] eq "2"}
assert {[count_log_message "Received SIGUSR1 in child"] eq "1"}
# check that it wasn't printed again (the print belong to the previous test)
assert {[count_log_message "fork child exiting"] eq "1"}
}
test {Module fork twice} {
r fork.create 0
after 250
catch {r fork.create 0}
assert {[count_log_message "Can't fork for module: File exists"] eq "1"}
}
test "Unload the module - fork" {
assert_equal {OK} [r module unload fork]
}

217
tests/unit/moduleapi/misc.tcl
View File

@ -133,6 +133,223 @@ start_server {tags {"modules"}} {
assert { [r test.monotonic_time] >= $x }
}
test {rm_call OOM} {
r config set maxmemory 1
r config set maxmemory-policy volatile-lru
# sanity test plain call
assert_equal {OK} [
r test.rm_call set x 1
]
# add the M flag
assert_error {OOM *} {
r test.rm_call_flags M set x 1
}
# test a non deny-oom command
assert_equal {1} [
r test.rm_call_flags M get x
]
r config set maxmemory 0
} {OK} {needs:config-maxmemory}
test {rm_call write flag} {
# add the W flag
assert_error {ERR Write command 'set' was called while write is not allowed.} {
r test.rm_call_flags W set x 1
}
# test a non deny-oom command
r test.rm_call_flags W get x
} {1}
test {rm_call EVAL} {
r test.rm_call eval {
redis.call('set','x',1)
return 1
} 1 x
assert_error {ERR Write commands are not allowed from read-only scripts.*} {
r test.rm_call eval {#!lua flags=no-writes
redis.call('set','x',1)
return 1
} 1 x
}
}
test {rm_call EVAL - OOM} {
r config set maxmemory 1
assert_error {OOM command not allowed when used memory > 'maxmemory'. script*} {
r test.rm_call eval {
redis.call('set','x',1)
return 1
} 1 x
}
r test.rm_call eval {#!lua flags=no-writes
redis.call('get','x')
return 2
} 1 x
assert_error {OOM allow-oom flag is not set on the script,*} {
r test.rm_call eval {#!lua
redis.call('get','x')
return 3
} 1 x
}
r test.rm_call eval {
redis.call('get','x')
return 4
} 1 x
r config set maxmemory 0
} {OK} {needs:config-maxmemory}
test "not enough good replicas" {
r set x "some value"
r config set min-replicas-to-write 1
# rm_call in script mode
assert_error {NOREPLICAS *} {r test.rm_call_flags S set x s}
assert_equal [
r test.rm_call eval {#!lua flags=no-writes
return redis.call('get','x')
} 1 x
] "some value"
assert_equal [
r test.rm_call eval {
return redis.call('get','x')
} 1 x
] "some value"
assert_error {NOREPLICAS *} {
r test.rm_call eval {#!lua
return redis.call('get','x')
} 1 x
}
assert_error {NOREPLICAS *} {
r test.rm_call eval {
return redis.call('set','x', 1)
} 1 x
}
r config set min-replicas-to-write 0
}
test {rm_call EVAL - read-only replica} {
r replicaof 127.0.0.1 1
# rm_call in script mode
assert_error {READONLY *} {r test.rm_call_flags S set x 1}
assert_error {READONLY You can't write against a read only replica. script*} {
r test.rm_call eval {
redis.call('set','x',1)
return 1
} 1 x
}
r test.rm_call eval {#!lua flags=no-writes
redis.call('get','x')
return 2
} 1 x
assert_error {READONLY Can not run script with write flag on readonly replica*} {
r test.rm_call eval {#!lua
redis.call('get','x')
return 3
} 1 x
}
r test.rm_call eval {
redis.call('get','x')
return 4
} 1 x
r replicaof no one
} {OK} {needs:config-maxmemory}
test {rm_call EVAL - stale replica} {
r replicaof 127.0.0.1 1
r config set replica-serve-stale-data no
# rm_call in script mode
assert_error {MASTERDOWN *} {
r test.rm_call_flags S get x
}
assert_error {MASTERDOWN *} {
r test.rm_call eval {#!lua flags=no-writes
redis.call('get','x')
return 2
} 1 x
}
assert_error {MASTERDOWN *} {
r test.rm_call eval {
redis.call('get','x')
return 4
} 1 x
}
r replicaof no one
r config set replica-serve-stale-data yes
} {OK} {needs:config-maxmemory}
test "rm_call EVAL - failed bgsave prevents writes" {
r config set rdb-key-save-delay 10000000
populate 1000
r set x x
r bgsave
set pid1 [get_child_pid 0]
catch {exec kill -9 $pid1}
waitForBgsave r
# make sure a read command succeeds
assert_equal [r get x] x
# make sure a write command fails
assert_error {MISCONF *} {r set x y}
# rm_call in script mode
assert_error {MISCONF *} {r test.rm_call_flags S set x 1}
# repeate with script
assert_error {MISCONF *} {r test.rm_call eval {
return redis.call('set','x',1)
} 1 x
}
assert_equal {x} [r test.rm_call eval {
return redis.call('get','x')
} 1 x
]
# again with script using shebang
assert_error {MISCONF *} {r test.rm_call eval {#!lua
return redis.call('set','x',1)
} 1 x
}
assert_equal {x} [r test.rm_call eval {#!lua flags=no-writes
return redis.call('get','x')
} 1 x
]
r config set rdb-key-save-delay 0
r bgsave
waitForBgsave r
# server is writable again
r set x y
} {OK}
test "Unload the module - misc" {
assert_equal {OK} [r module unload misc]
}

29
tests/unit/moduleapi/moduleconfigs.tcl
View File

@ -32,12 +32,27 @@ start_server {tags {"modules"}} {
assert_equal [r config get moduleconfigs.enum] "moduleconfigs.enum two"
r config set moduleconfigs.flags two
assert_equal [r config get moduleconfigs.flags] "moduleconfigs.flags two"
r config set moduleconfigs.flags "two three"
assert_equal [r config get moduleconfigs.flags] "moduleconfigs.flags {two three}"
r config set moduleconfigs.numeric -2
assert_equal [r config get moduleconfigs.numeric] "moduleconfigs.numeric -2"
}
test {Config set commands enum flags} {
r config set moduleconfigs.flags "none"
assert_equal [r config get moduleconfigs.flags] "moduleconfigs.flags none"
r config set moduleconfigs.flags "two four"
assert_equal [r config get moduleconfigs.flags] "moduleconfigs.flags {two four}"
r config set moduleconfigs.flags "five"
assert_equal [r config get moduleconfigs.flags] "moduleconfigs.flags five"
r config set moduleconfigs.flags "one four"
assert_equal [r config get moduleconfigs.flags] "moduleconfigs.flags five"
r config set moduleconfigs.flags "one two four"
assert_equal [r config get moduleconfigs.flags] "moduleconfigs.flags {five two}"
}
test {Immutable flag works properly and rejected strings dont leak} {
# Configs flagged immutable should not allow sets
catch {[r config set moduleconfigs.immutable_bool yes]} e
@ -58,7 +73,7 @@ start_server {tags {"modules"}} {
test {Enums only able to be set to passed in values} {
# Module authors specify what values are valid for enums, check that only those values are ok on a set
catch {[r config set moduleconfigs.enum four]} e
catch {[r config set moduleconfigs.enum asdf]} e
assert_match {*must be one of the following*} $e
}
@ -147,7 +162,7 @@ start_server {tags {"modules"}} {
r config set moduleconfigs.mutable_bool yes
r config set moduleconfigs.memory_numeric 750
r config set moduleconfigs.enum two
r config set moduleconfigs.flags "two three"
r config set moduleconfigs.flags "four two"
r config rewrite
restart_server 0 true false
# Ensure configs we rewrote are present and that the conf file is readable
@ -155,7 +170,7 @@ start_server {tags {"modules"}} {
assert_equal [r config get moduleconfigs.memory_numeric] "moduleconfigs.memory_numeric 750"
assert_equal [r config get moduleconfigs.string] "moduleconfigs.string {super \0secret password}"
assert_equal [r config get moduleconfigs.enum] "moduleconfigs.enum two"
assert_equal [r config get moduleconfigs.flags] "moduleconfigs.flags {two three}"
assert_equal [r config get moduleconfigs.flags] "moduleconfigs.flags {two four}"
assert_equal [r config get moduleconfigs.numeric] "moduleconfigs.numeric -1"
r module unload moduleconfigs
}
@ -230,12 +245,12 @@ start_server {tags {"modules"}} {
set stdout [dict get $noload stdout]
assert_equal [count_message_lines $stdout "Module Configurations were not set, likely a missing LoadConfigs call. Unloading the module."] 1
start_server [list overrides [list loadmodule "$testmodule" moduleconfigs.string "bootedup" moduleconfigs.enum two moduleconfigs.flags "two three"]] {
start_server [list overrides [list loadmodule "$testmodule" moduleconfigs.string "bootedup" moduleconfigs.enum two moduleconfigs.flags "two four"]] {
assert_equal [r config get moduleconfigs.string] "moduleconfigs.string bootedup"
assert_equal [r config get moduleconfigs.mutable_bool] "moduleconfigs.mutable_bool yes"
assert_equal [r config get moduleconfigs.immutable_bool] "moduleconfigs.immutable_bool no"
assert_equal [r config get moduleconfigs.enum] "moduleconfigs.enum two"
assert_equal [r config get moduleconfigs.flags] "moduleconfigs.flags {two three}"
assert_equal [r config get moduleconfigs.flags] "moduleconfigs.flags {two four}"
assert_equal [r config get moduleconfigs.numeric] "moduleconfigs.numeric -1"
assert_equal [r config get moduleconfigs.memory_numeric] "moduleconfigs.memory_numeric 1024"
}

2
tests/unit/moduleapi/publish.tcl
View File

@ -11,7 +11,7 @@ start_server {tags {"modules"}} {
assert_equal {1} [subscribe $rd2 {chan1}]
assert_equal 1 [r publish.shard chan1 hello]
assert_equal 1 [r publish.classic chan1 world]
assert_equal {message chan1 hello} [$rd1 read]
assert_equal {smessage chan1 hello} [$rd1 read]
assert_equal {message chan1 world} [$rd2 read]
}
}

20
tests/unit/other.tcl
View File

@ -38,9 +38,25 @@ start_server {tags {"other"}} {
}
}
start_server {overrides {save ""} tags {external:skip}} {
test {FLUSHALL should not reset the dirty counter if we disable save} {
r set key value
r flushall
assert_morethan [s rdb_changes_since_last_save] 0
}
test {FLUSHALL should reset the dirty counter to 0 if we enable save} {
r config set save "3600 1 300 100 60 10000"
r set key value
r flushall
assert_equal [s rdb_changes_since_last_save] 0
}
}
test {BGSAVE} {
r flushdb
waitForBgsave r
# Use FLUSHALL instead of FLUSHDB, FLUSHALL do a foreground save
# and reset the dirty counter to 0, so we won't trigger an unexpected bgsave.
r flushall
r save
r set x 10
r bgsave

133
tests/unit/pause.tcl
View File

@ -47,18 +47,14 @@ start_server {tags {"pause network"}} {
test "Test read/admin mutli-execs are not blocked by pause RO" {
r SET FOO BAR
r client PAUSE 100000 WRITE
set rd [redis_deferring_client]
$rd MULTI
assert_equal [$rd read] "OK"
$rd PING
assert_equal [$rd read] "QUEUED"
$rd GET FOO
assert_equal [$rd read] "QUEUED"
$rd EXEC
set rr [redis_client]
assert_equal [$rr MULTI] "OK"
assert_equal [$rr PING] "QUEUED"
assert_equal [$rr GET FOO] "QUEUED"
assert_match "PONG BAR" [$rr EXEC]
assert_equal [s 0 blocked_clients] 0
r client unpause
assert_match "PONG BAR" [$rd read]
$rd close
$rr close
}
test "Test write mutli-execs are blocked by pause RO" {
@ -78,20 +74,129 @@ start_server {tags {"pause network"}} {
test "Test scripts are blocked by pause RO" {
r client PAUSE 60000 WRITE
set rd [redis_deferring_client]
set rd2 [redis_deferring_client]
$rd EVAL "return 1" 0
wait_for_blocked_clients_count 1 50 100
# test a script with a shebang and no flags for coverage
$rd2 EVAL {#!lua
return 1
} 0
wait_for_blocked_clients_count 2 50 100
r client unpause
assert_match "1" [$rd read]
assert_match "1" [$rd2 read]
$rd close
$rd2 close
}
test "Test may-replicate commands are rejected in ro script by pause RO" {
test "Test RO scripts are not blocked by pause RO" {
r set x y
# create a function for later
r FUNCTION load replace {#!lua name=f1
redis.register_function{
function_name='f1',
callback=function() return "hello" end,
flags={'no-writes'}
}
}
r client PAUSE 6000000 WRITE
set rr [redis_client]
# test an eval that's for sure not in the script cache
assert_equal [$rr EVAL {#!lua flags=no-writes
return 'unique script'
} 0
] "unique script"
# for sanity, repeat that EVAL on a script that's already cached
assert_equal [$rr EVAL {#!lua flags=no-writes
return 'unique script'
} 0
] "unique script"
# test EVAL_RO on a unique script that's for sure not in the cache
assert_equal [$rr EVAL_RO {
return redis.call('GeT', 'x')..' unique script'
} 1 x
] "y unique script"
# test with evalsha
set sha [$rr script load {#!lua flags=no-writes
return 2
}]
assert_equal [$rr EVALSHA $sha 0] 2
# test with function
assert_equal [$rr fcall f1 0] hello
r client unpause
$rr close
}
test "Test read-only scripts in mutli-exec are not blocked by pause RO" {
r SET FOO BAR
r client PAUSE 100000 WRITE
set rr [redis_client]
assert_equal [$rr MULTI] "OK"
assert_equal [$rr EVAL {#!lua flags=no-writes
return 12
} 0
] QUEUED
assert_equal [$rr EVAL {#!lua flags=no-writes
return 13
} 0
] QUEUED
assert_match "12 13" [$rr EXEC]
assert_equal [s 0 blocked_clients] 0
r client unpause
$rr close
}
test "Test write scripts in mutli-exec are blocked by pause RO" {
set rd [redis_deferring_client]
set rd2 [redis_deferring_client]
# one with a shebang
$rd MULTI
assert_equal [$rd read] "OK"
$rd EVAL {#!lua
return 12
} 0
assert_equal [$rd read] "QUEUED"
# one without a shebang
$rd2 MULTI
assert_equal [$rd2 read] "OK"
$rd2 EVAL {#!lua
return 13
} 0
assert_equal [$rd2 read] "QUEUED"
r client PAUSE 60000 WRITE
assert_error {ERR May-replicate commands are not allowed when client pause write*} {
$rd EXEC
$rd2 EXEC
wait_for_blocked_clients_count 2 50 100
r client unpause
assert_match "12" [$rd read]
assert_match "13" [$rd2 read]
$rd close
$rd2 close
}
test "Test may-replicate commands are rejected in RO scripts" {
# that's specifically important for CLIENT PAUSE WRITE
assert_error {ERR Write commands are not allowed from read-only scripts. script:*} {
r EVAL_RO "return redis.call('publish','ch','msg')" 0
}
r client unpause
assert_error {ERR Write commands are not allowed from read-only scripts. script:*} {
r EVAL {#!lua flags=no-writes
return redis.call('publish','ch','msg')
} 0
}
# make sure that publish isn't blocked from a non-RO script
assert_equal [r EVAL "return redis.call('publish','ch','msg')" 0] 0
}
test "Test multiple clients can be queued up and unblocked" {

Some files were not shown because too many files have changed in this diff Show More