opensourcemirror
/
redis
mirror of https://github.com/antirez/redis.git
1
0
Fork 0
Code Issues Releases Wiki Activity
redis/runtest-moduleapi

44 lines
1.2 KiB
Plaintext
Raw Normal View History

Add runtest-moduleapi with commandfilter coverage.
2019-03-24 11:03:03 +01:00
#!/bin/sh
Support tclsh 8.7 (#9500)
2021-09-15 12:04:31 +02:00
TCL_VERSIONS="8.5 8.6 8.7"
Add runtest-moduleapi with commandfilter coverage.
2019-03-24 11:03:03 +01:00
TCLSH=""
Several (mostly Solaris-related) cleanups (#8171) * Allow runtest-moduleapi use a different 'make', for systems where GNU Make is 'gmake'. * Fix issue with builds on Solaris re-building everything from scratch due to CFLAGS/LDFLAGS not stored. * Fix compile failure on Solaris due to atomicvar and a bunch of warnings. * Fix garbled log timestamps on Solaris.
2020-12-13 16:09:54 +01:00
[ -z "$MAKE" ] && MAKE=make
Add runtest-moduleapi with commandfilter coverage.
2019-03-24 11:03:03 +01:00
for VERSION in $TCL_VERSIONS; do
TCL=`which tclsh$VERSION 2>/dev/null` && TCLSH=$TCL
done
if [ -z $TCLSH ]
then
Correct error message of runtest-cluster and runtest-moduleapi (#7647)
2020-08-11 12:35:20 +02:00
echo "You need tcl 8.5 or newer in order to run the Redis ModuleApi test"
Add runtest-moduleapi with commandfilter coverage.
2019-03-24 11:03:03 +01:00
exit 1
fi
Several (mostly Solaris-related) cleanups (#8171) * Allow runtest-moduleapi use a different 'make', for systems where GNU Make is 'gmake'. * Fix issue with builds on Solaris re-building everything from scratch due to CFLAGS/LDFLAGS not stored. * Fix compile failure on Solaris due to atomicvar and a bunch of warnings. * Fix garbled log timestamps on Solaris.
2020-12-13 16:09:54 +01:00
$MAKE -C tests/modules && \
Module api tests for RM_Call Adding a test for coverage for RM_Call in a new "misc" unit to be used for various short simple tests also solves compilation warnings in redismodule.h and fork.c
2019-10-24 08:38:52 +02:00
$TCLSH tests/test_helper.tcl \
--single unit/moduleapi/commandfilter \
modules: Add newlen == 0 handling to RM_StringTruncate (#3717) (#3718) Previously, passing 0 for newlen would not truncate the string at all. This adds handling of this case, freeing the old string and creating a new empty string. Other changes: - Move `src/modules/testmodule.c` to `tests/modules/basics.c` - Introduce that basic test into the test suite - Add tests to cover StringTruncate - Add `test-modules` build target for the main makefile - Extend `distclean` build target to clean modules too
2021-06-22 11:26:48 +02:00
--single unit/moduleapi/basics \
Module api tests for RM_Call Adding a test for coverage for RM_Call in a new "misc" unit to be used for various short simple tests also solves compilation warnings in redismodule.h and fork.c
2019-10-24 08:38:52 +02:00
--single unit/moduleapi/fork \
--single unit/moduleapi/testrdb \
--single unit/moduleapi/infotest \
add test for modules load/unload and config rewrite
2021-06-01 11:51:30 +02:00
--single unit/moduleapi/infra \
Module api tests for RM_Call Adding a test for coverage for RM_Call in a new "misc" unit to be used for various short simple tests also solves compilation warnings in redismodule.h and fork.c
2019-10-24 08:38:52 +02:00
--single unit/moduleapi/propagate \
--single unit/moduleapi/hooks \
--single unit/moduleapi/misc \
Modules: Test RedisModule_BlockClientOnKeys
2018-03-29 12:46:13 +02:00
--single unit/moduleapi/blockonkeys \
Enabled background and reply time tracking on blocked on keys/blocked on background work clients (#7491) This commit enables tracking time of the background tasks and on replies, opening the door for properly tracking commands that rely on blocking / background work via the slowlog, latency history, and commandstats. Some notes: - The time spent blocked waiting for key changes, or blocked on synchronous replication is not accounted for. - **This commit does not affect latency tracking of commands that are non-blocking or do not have background work.** ( meaning that it all stays the same with exception to `BZPOPMIN`,`BZPOPMAX`,`BRPOP`,`BLPOP`, etc... and module's commands that rely on background threads ). - Specifically for latency history command we've added a new event class named `command-unblocking` that will enable latency monitoring on commands that spawn background threads to do the work. - For blocking commands we're now considering the total time of a command as the time spent on call() + the time spent on replying when unblocked. - For Modules commands that rely on background threads we're now considering the total time of a command as the time spent on call (main thread) + the time spent on the background thread ( if marked within `RedisModule_MeasureTimeStart()` and `RedisModule_MeasureTimeEnd()` ) + the time spent on replying (main thread) To test for this feature we've added a `unit/moduleapi/blockonbackground` test that relies on a module that blocks the client and sleeps on the background for a given time. - check blocked command that uses RedisModule_MeasureTimeStart() is tracking background time - check blocked command that uses RedisModule_MeasureTimeStart() is tracking background time even in timeout - check blocked command with multiple calls RedisModule_MeasureTimeStart() is tracking the total background time - check blocked command without calling RedisModule_MeasureTimeStart() is not reporting background time
2021-01-29 14:38:30 +01:00
--single unit/moduleapi/blockonbackground \
Added scan implementation to module api. The implementation expose the following new functions: 1. RedisModule_CursorCreate - allow to create a new cursor object for keys scanning 2. RedisModule_CursorRestart - restart an existing cursor to restart the scan 3. RedisModule_CursorDestroy - destroy an existing cursor 4. RedisModule_Scan - scan keys The RedisModule_Scan function gets a cursor object, a callback and void* (used as user private data). The callback will be called for each key in the database proving the key name and the value as RedisModuleKey.
2019-10-17 14:37:01 +02:00
--single unit/moduleapi/scan \
Merge branch 'unstable' into scan_module_impl
2019-11-19 11:08:02 +01:00
--single unit/moduleapi/datatype \
Add module APIs for custom authentication
2019-02-26 02:23:11 +01:00
--single unit/moduleapi/auth \
This PR introduces a new loaded keyspace event (#7536) Co-authored-by: Oran Agra <oran@redislabs.com> Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-07-23 11:38:51 +02:00
--single unit/moduleapi/keyspace_events \
RM_ThreadSafeContextTryLock a non-blocking method for acquiring GIL (#7738) Co-authored-by: Yossi Gottlieb <yossigo@gmail.com> Co-authored-by: Oran Agra <oran@redislabs.com>
2020-09-09 15:01:16 +02:00
--single unit/moduleapi/blockedclient \
Modules: add RM_GetCommandKeys(). This is essentially the same as calling COMMAND GETKEYS but provides a more efficient interface that can be used in every context (i.e. not a Redis command).
2020-10-05 16:06:35 +02:00
--single unit/moduleapi/getkeys \
Modules callbacks for lazy free effort, and unlink (#7912) Add two optional callbacks to the RedisModuleTypeMethods structure, which is `free_effort` and `unlink`. the `free_effort` callback indicates the effort required to free a module memory. Currently, if the effort exceeds LAZYFREE_THRESHOLD, the module memory may be released asynchronously. the `unlink` callback indicates the key has been removed from the DB by redis, and may soon be freed by a background thread. Add `lazyfreed_objects` info field, which represents the number of objects that have been lazyfreed since redis was started. Add `RM_GetTypeMethodVersion` API, which return the current redis-server runtime value of `REDISMODULE_TYPE_METHOD_VERSION`. You can use that when calling `RM_CreateDataType` to know which fields of RedisModuleTypeMethods are gonna be supported and which will be ignored.
2020-11-16 09:34:04 +01:00
--single unit/moduleapi/test_lazyfree \
Modules: add defrag API support. (#8149) Add a new set of defrag functions that take a defrag context and allow defragmenting memory blocks and RedisModuleStrings. Modules can register a defrag callback which will be invoked when the defrag process handles globals. Modules with custom data types can also register a datatype-specific defrag callback which is invoked for keys that require defragmentation. The callback and associated functions support both one-step and multi-step options, depending on the complexity of the key as exposed by the free_effort callback.
2020-12-13 08:56:01 +01:00
--single unit/moduleapi/defrag \
A better approach for COMMAND INFO for movablekeys commands (#8324) Fix #7297 The problem: Today, there is no way for a client library or app to know the key name indexes for commands such as ZUNIONSTORE/EVAL and others with "numkeys", since COMMAND INFO returns no useful info for them. For cluster-aware redis clients, this requires to 'patch' the client library code specifically for each of these commands or to resolve each execution of these commands with COMMAND GETKEYS. The solution: Introducing key specs other than the legacy "range" (first,last,step) The 8th element of the command info array, if exists, holds an array of key specs. The array may be empty, which indicates the command doesn't take any key arguments or may contain one or more key-specs, each one may leads to the discovery of 0 or more key arguments. A client library that doesn't support this key-spec feature will keep using the first,last,step and movablekeys flag which will obviously remain unchanged. A client that supports this key-specs feature needs only to look at the key-specs array. If it finds an unrecognized spec, it must resort to using COMMAND GETKEYS if it wishes to get all key name arguments, but if all it needs is one key in order to know which cluster node to use, then maybe another spec (if the command has several) can supply that, and there's no need to use GETKEYS. Each spec is an array of arguments, first one is the spec name, the second is an array of flags, and the third is an array containing details about the spec (specific meaning for each spec type) The initial flags we support are "read" and "write" indicating if the keys that this key-spec finds are used for read or for write. clients should ignore any unfamiliar flags. In order to easily find the positions of keys in a given array of args we introduce keys specs. There are two logical steps of key specs: 1. `start_search`: Given an array of args, indicate where we should start searching for keys 2. `find_keys`: Given the output of start_search and an array of args, indicate all possible indices of keys. ### start_search step specs - `index`: specify an argument index explicitly - `index`: 0 based index (1 means the first command argument) - `keyword`: specify a string to match in `argv`. We should start searching for keys just after the keyword appears. - `keyword`: the string to search for - `start_search`: an index from which to start the keyword search (can be negative, which means to search from the end) Examples: - `SET` has start_search of type `index` with value `1` - `XREAD` has start_search of type `keyword` with value `[“STREAMS”,1]` - `MIGRATE` has start_search of type `keyword` with value `[“KEYS”,-2]` ### find_keys step specs - `range`: specify `[count, step, limit]`. - `lastkey`: index of the last key. relative to the index returned from begin_search. -1 indicating till the last argument, -2 one before the last - `step`: how many args should we skip after finding a key, in order to find the next one - `limit`: if count is -1, we use limit to stop the search by a factor. 0 and 1 mean no limit. 2 means ½ of the remaining args, 3 means ⅓, and so on. - “keynum”: specify `[keynum_index, first_key_index, step]`. - `keynum_index`: is relative to the return of the `start_search` spec. - `first_key_index`: is relative to `keynum_index`. - `step`: how many args should we skip after finding a key, in order to find the next one Examples: - `SET` has `range` of `[0,1,0]` - `MSET` has `range` of `[-1,2,0]` - `XREAD` has `range` of `[-1,1,2]` - `ZUNION` has `start_search` of type `index` with value `1` and `find_keys` of type `keynum` with value `[0,1,1]` - `AI.DAGRUN` has `start_search` of type `keyword` with value `[“LOAD“,1]` and `find_keys` of type `keynum` with value `[0,1,1]` (see https://oss.redislabs.com/redisai/master/commands/#aidagrun) Note: this solution is not perfect as the module writers can come up with anything, but at least we will be able to find the key args of the vast majority of commands. If one of the above specs can’t describe the key positions, the module writer can always fall back to the `getkeys-api` option. Some keys cannot be found easily (`KEYS` in `MIGRATE`: Imagine the argument for `AUTH` is the string “KEYS” - we will start searching in the wrong index). The guarantee is that the specs may be incomplete (`incomplete` will be specified in the spec to denote that) but we never report false information (assuming the command syntax is correct). For `MIGRATE` we start searching from the end - `startfrom=-1` - and if one of the keys is actually called "keys" we will report only a subset of all keys - hence the `incomplete` flag. Some `incomplete` specs can be completely empty (i.e. UNKNOWN begin_search) which should tell the client that COMMAND GETKEYS (or any other way to get the keys) must be used (Example: For `SORT` there is no way to describe the STORE keyword spec, as the word "store" can appear anywhere in the command). We will expose these key specs in the `COMMAND` command so that clients can learn, on startup, where the keys are for all commands instead of holding hardcoded tables or use `COMMAND GETKEYS` in runtime. Comments: 1. Redis doesn't internally use the new specs, they are only used for COMMAND output. 2. In order to support the current COMMAND INFO format (reply array indices 4, 5, 6) we created a synthetic range, called legacy_range, that, if possible, is built according to the new specs. 3. Redis currently uses only getkeys_proc or the legacy_range to get the keys indices (in COMMAND GETKEYS for example). "incomplete" specs: the command we have issues with are MIGRATE, STRALGO, and SORT for MIGRATE, because the token KEYS, if exists, must be the last token, we can search in reverse. it one of the keys is actually the string "keys" will return just a subset of the keys (hence, it's "incomplete") for SORT and STRALGO we can use this heuristic (the keys can be anywhere in the command) and therefore we added a key spec that is both "incomplete" and of "unknown type" if a client encounters an "incomplete" spec it means that it must find a different way (either COMMAND GETKEYS or have its own parser) to retrieve the keys. please note that all commands, apart from the three mentioned above, have "complete" key specs
2021-09-15 10:10:29 +02:00
--single unit/moduleapi/keyspecs \
Modules: In RM_HashSet, add COUNT_ALL flag and set errno (#8446) The added flag affects the return value of RM_HashSet() to include the number of inserted fields, in addition to updated and deleted fields. errno is set on errors, tests are added and documentation updated.
2021-02-15 10:40:05 +01:00
--single unit/moduleapi/hash \
RM_ZsetRem: Delete key if empty (#8453) Without this fix, RM_ZsetRem can leave empty sorted sets which are not allowed to exist. Removing from a sorted set while iterating seems to work (while inserting causes failed assetions). RM_ZsetRangeEndReached is modified to return 1 if the key doesn't exist, to terminate iteration when the last element has been removed.
2021-02-05 18:54:01 +01:00
--single unit/moduleapi/zset \
Modules: Add remaining list API functions (#8439) List functions operating on elements by index: * RM_ListGet * RM_ListSet * RM_ListInsert * RM_ListDelete Iteration is done using a simple for loop over indices. The index based functions use an internal iterator as an optimization. This is explained in the docs: ``` * Many of the list functions access elements by index. Since a list is in * essence a doubly-linked list, accessing elements by index is generally an * O(N) operation. However, if elements are accessed sequentially or with * indices close together, the functions are optimized to seek the index from * the previous index, rather than seeking from the ends of the list. * * This enables iteration to be done efficiently using a simple for loop: * * long n = RM_ValueLength(key); * for (long i = 0; i < n; i++) { * RedisModuleString *elem = RedisModule_ListGet(key, i); * // Do stuff... * } ```
2021-09-14 16:48:06 +02:00
--single unit/moduleapi/list \
Add modules API for streams (#8288) APIs added for these stream operations: add, delete, iterate and trim (by ID or maxlength). The functions are prefixed by RM_Stream. * RM_StreamAdd * RM_StreamDelete * RM_StreamIteratorStart * RM_StreamIteratorStop * RM_StreamIteratorNextID * RM_StreamIteratorNextField * RM_StreamIteratorDelete * RM_StreamTrimByLength * RM_StreamTrimByID The type RedisModuleStreamID is added and functions for converting from and to RedisModuleString. * RM_CreateStringFromStreamID * RM_StringToStreamID Whenever the stream functions return REDISMODULE_ERR, errno is set to provide additional error information. Refactoring: The zset iterator fields in the RedisModuleKey struct are wrapped in a union, to allow the same space to be used for type- specific info for streams and allow future use for other key types.
2021-01-28 15:19:43 +01:00
--single unit/moduleapi/stream \
Enhance mem_usage/free_effort/unlink/copy callbacks and add GetDbFromIO api. (#8999) Create new module type enhanced callbacks: mem_usage2, free_effort2, unlink2, copy2. These will be given a context point from which the module can obtain the key name and database id. In addition the digest and defrag context can now be used to obtain the key name and database id.
2021-06-16 08:45:49 +02:00
--single unit/moduleapi/datatype2 \
Module api tests for RM_Call Adding a test for coverage for RM_Call in a new "misc" unit to be used for various short simple tests also solves compilation warnings in redismodule.h and fork.c
2019-10-24 08:38:52 +02:00
"${@}"