redis/tests/unit
Binbin 804110a487
Allocate Lua VM code with jemalloc instead of libc, and count it used memory (#13133)
## Background
1. Currently Lua memory control does not pass through Redis's zmalloc.c.
Redis maxmemory cannot limit memory problems caused by users abusing lua
since these lua VM memory is not part of used_memory.

2. Since jemalloc is much better (fragmentation and speed), and also we
know it and trust it. we are
going to use jemalloc instead of libc to allocate the Lua VM code and
count it used memory.

## Process:
In this PR, we will use jemalloc in lua. 
1. Create an arena for all lua vm (script and function), which is
shared, in order to avoid blocking defragger.
2. Create a bound tcache for the lua VM, since the lua VM and the main
thread are by default in the same tcache, and if there is no isolated
tcache, lua may request memory from the tcache which has just been freed
by main thread, and vice versa
On the other hand, since lua vm might be release in bio thread, but
tcache is not thread-safe, we need to recreate
    the tcache every time we recreate the lua vm.
3. Remove lua memory statistics from memory fragmentation statistics to
avoid the effects of lua memory fragmentation

## Other
Add the following new fields to `INFO DEBUG` (we may promote them to
INFO MEMORY some day)
1. allocator_allocated_lua: total number of bytes allocated of lua arena
2. allocator_active_lua: total number of bytes in active pages allocated
in lua arena
3. allocator_resident_lua: maximum number of bytes in physically
resident data pages mapped in lua arena
4. allocator_frag_bytes_lua: fragment bytes in lua arena

This is oranagra's idea, and i got some help from sundb.

This solves the third point in #13102.

---------

Co-authored-by: debing.sun <debing.sun@redis.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2024-04-16 12:43:33 +03:00
..
cluster fix scripts access wrong slot if they disagree with pre-declared keys (#12906) 2024-01-15 09:57:12 +08:00
moduleapi Add API RedisModule_ClusterKeySlot and RedisModule_ClusterCanonicalKeyNameInSlot (#13069) 2024-03-12 09:26:12 -07:00
type fix wrong data type conversion in zrangeResultBeginStore (#13148) 2024-03-19 08:52:55 +02:00
acl-v2.tcl fix false success and a memory leak for ACL selector with bad parenthesis combination (#12452) 2023-08-02 10:46:06 +03:00
acl.tcl Exit early if slowlog/acllog max len set to zero (#12965) 2024-01-22 16:01:04 -08:00
aofrw.tcl Attempt to solve MacOS CI issues in GH Actions (#12013) 2023-04-12 09:19:21 +03:00
auth.tcl Fix race condition in tests/unit/auth.tcl (#12444) 2023-08-01 18:03:33 +03:00
bitfield.tcl Add BITFIELD_RO basic tests for non-repl use cases (#12187) 2023-05-18 12:16:46 +03:00
bitops.tcl BITCOUNT and BITPOS with non-existing key and illegal arguments should return error, not 0 (#11734) 2023-08-21 19:48:30 +03:00
client-eviction.tcl Attempt to solve MacOS CI issues in GH Actions (#12013) 2023-04-12 09:19:21 +03:00
dump.tcl Fix SPOP/RESTORE propagation when doing lazy free (#12320) 2023-06-16 08:14:11 -07:00
expire.tcl Fix dictionary use-after-free in active expire and make kvstore iter to respect EMPTY flag (#13135) 2024-03-18 17:41:54 +02:00
functions.tcl Some cleanups around function (#12940) 2024-01-23 14:26:33 +02:00
geo.tcl adding geo command edge cases tests (#12274) 2023-06-20 12:50:03 +03:00
hyperloglog.tcl Hyperloglog avoid allocate more than 'server.hll_sparse_max_bytes' bytes of memory for sparse representation (#11438) 2022-11-28 17:35:31 +02:00
info-command.tcl Make INFO command variadic (#6891) 2022-02-08 13:14:42 +02:00
info.tcl Prevent lua error_reply abuse from causing errorstats to become larger (#13141) 2024-03-19 08:18:22 +02:00
introspection-2.tcl Fix possible crash in command getkeys (#12380) 2023-07-03 12:45:18 +03:00
introspection.tcl Add metrics for WATCH (#12966) 2024-02-18 10:36:41 +02:00
keyspace.tcl Optimize KEYS when pattern includes hashtag and implies a single slot. (#12754) 2023-12-05 16:21:50 +02:00
latency-monitor.tcl Add printing for LATENCY related tests (#12514) 2023-08-27 11:42:55 +03:00
lazyfree.tcl Change FLUSHALL/FLUSHDB SYNC to run as blocking ASYNC (#13167) 2024-04-02 15:09:52 +03:00
limits.tcl Improve test suite to handle external servers better. (#9033) 2021-06-09 15:13:24 +03:00
maxmemory.tcl Fix dict resize allow test (#13016) 2024-01-31 13:11:52 +02:00
memefficiency.tcl Allocate Lua VM code with jemalloc instead of libc, and count it used memory (#13133) 2024-04-16 12:43:33 +03:00
multi.tcl Fix propagation of entries_read by calling streamPropagateGroupID unconditionally (#12898) 2024-02-29 09:48:20 +02:00
networking.tcl Add reply_schema to command json files (internal for now) (#10273) 2023-03-11 10:14:16 +02:00
obuf-limits.tcl Add reply_schema to command json files (internal for now) (#10273) 2023-03-11 10:14:16 +02:00
oom-score-adj.tcl Check user's oom_score_adj write permission for oom-score-adj test (#13111) 2024-03-05 14:42:28 +02:00
other.tcl Add overhead of all DBs and rehashing dict count to info. (#12913) 2024-03-01 13:41:24 +08:00
pause.tcl Bump codespell to 2.2.4, fix typos and outupdated comments (#11911) 2023-03-16 08:50:32 +02:00
printver.tcl Print version info before running the test 2011-05-20 11:44:54 +02:00
protocol.tcl Add reply_schema to command json files (internal for now) (#10273) 2023-03-11 10:14:16 +02:00
pubsub.tcl Fix broken protocol when PUBLISH emits local push inside MULTI (#12326) 2023-06-20 20:41:41 +03:00
pubsubshard.tcl Fix the bug that CLIENT REPLY OFF|SKIP cannot receive push notifications (#11875) 2023-03-12 17:50:44 +02:00
querybuf.tcl Pause cron to prevent premature shrinking in querybuf test (#12126) 2023-05-04 13:02:08 +03:00
quit.tcl flushSlavesOutputBuffers should not write to replicas scheduled to drop (#12242) 2023-06-12 14:05:34 +03:00
replybufsize.tcl Introduce debug command to disable reply buffer resizing (#10360) 2022-03-01 14:40:29 +02:00
scan.tcl Add novalues option to command HSCAN. (#12765) 2024-01-30 20:32:58 +02:00
scripting.tcl Lua eval scripts first in first out LRU eviction (#13108) 2024-03-13 08:27:41 +02:00
shutdown.tcl Tests: Do not save an RDB by default and add a SIGTERM default AOFRW test (#12064) 2023-04-18 16:14:26 +03:00
slowlog.tcl Exit early if slowlog/acllog max len set to zero (#12965) 2024-01-22 16:01:04 -08:00
sort.tcl Fix SORT STORE quicklist with the right options (#13042) 2024-02-08 14:36:11 +02:00
tls.tcl Add support for reading encrypted keyfiles. (#8644) 2021-03-22 13:27:46 +02:00
tracking.tcl Bump codespell from 2.2.4 to 2.2.5 (#12557) 2023-09-08 16:10:17 +03:00
violations.tcl Run large-memory tests as solo. (#10626) 2022-04-24 17:29:35 +03:00
wait.tcl WAITAOF: Update fsynced_reploff_pending even if there's nothing to fsync (#12622) 2023-09-28 17:19:20 +03:00