opensourcemirror
/
redis
mirror of https://github.com/antirez/redis.git
1
0
Fork 0
Code Issues Releases Wiki Activity

Merge branch 'master' into integration

This commit is contained in:
Pieter Noordhuis 2010-05-19 14:53:57 +02:00
parent fdfb02e7ff 30dd89b6b7
commit 46b614edf7
41 changed files with 1858 additions and 50 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -12,3 +12,4 @@ mkrelease.sh
release
myredis.conf
misc/*
release.h

83
Changelog
View File

@ -1,3 +1,86 @@
2010-05-16 Redis version is now 1.3.12 (antirez)
2010-05-16 redis version is now 1.3.11 (antirez)
2010-05-16 random refactoring and speedups (antirez)
2010-05-16 faster INCR with very little efforts... (antirez)
2010-05-15 Merge branch 'redis-cli-fix' of http://github.com/tizoc/redis (antirez)
2010-05-15 added pid info to the check memory leaks test, so that those tests don't appear to be duplicated (antirez)
2010-05-15 Merge branch 'integration' of git://github.com/pietern/redis (antirez)
2010-05-14 more endianess detection fix for SHA1 (antirez)
2010-05-14 fixed a warning seen with some GCC version under Linux (antirez)
2010-05-14 initial rough integration test for replication (Pieter Noordhuis)
2010-05-14 store entire server object on the stack instead of just the client (Pieter Noordhuis)
2010-05-14 proc to retrieve values from INFO properties (Pieter Noordhuis)
2010-05-14 one more fix for endianess detection (antirez)
2010-05-14 Fixed sha1.c compilation on Linux, due to endianess detection lameness (antirez)
2010-05-14 ZUNION,ZINTER -> ZUNIONSTORE,ZINTERSTORE (antirez)
2010-05-14 minor fixes to the new test suite, html doc updated (antirez)
2010-05-14 wait for redis-server to be settled and ready for connections (Pieter Noordhuis)
2010-05-14 fix cleaning up tmp folder (Pieter Noordhuis)
2010-05-14 update makefile to use the new test suite (Pieter Noordhuis)
2010-05-14 check for memory leaks before killing a server (Pieter Noordhuis)
2010-05-14 extract code to kill a server to a separate proc (Pieter Noordhuis)
2010-05-14 start servers on different ports to prevent conflicts (Pieter Noordhuis)
2010-05-14 use DEBUG DIGEST in new test suite (Pieter Noordhuis)
2010-05-14 split test suite into multiple files; runs redis-server in isolation (Pieter Noordhuis)
2010-05-14 use DEBUG DIGEST in the test instead of a function that was doing a similar work, but in a much slower and buggy way (antirez)
2010-05-14 Don't rely on cliReadReply being able to return on shutdown (Bruno Deferrari)
2010-05-14 If command is a shutdown, ignore errors on reply (Bruno Deferrari)
2010-05-14 DEBUG DIGEST implemented, in order to improve the ability to test persistence and replication consistency (antirez)
2010-05-13 makefile deps updated (antirez)
2010-05-13 conflicts resolved (antirez)
2010-05-13 feed SETEX as SET and EXPIREAT to AOF (Pieter Noordhuis)
2010-05-13 very strong speedup in saving time performance when there are many integers in the dataset. Instead of decoding the object before to pass them to the rdbSaveObject layer we check asap if the object is integer encoded and can be written on disk as an integer. (antirez)
2010-05-13 include limits.h otherwise no double precison macros (antirez)
2010-05-13 explicitly checks with ifdefs if our floating point and long long assumptions are verified (antirez)
2010-05-13 Yet another version of the double saving code, with comments explaining what's happening there (antirez)
2010-05-12 added overflow check in the double -> long long conversion trick to avoid integer overflows. I think this was not needed in practical terms, but it is safer (antirez)
2010-05-12 use withscores when performing the dataset digest (antirez)
2010-05-12 If a float can be casted to a long long without rounding loss, we can use the integer conversion function to write the score on disk. This is a seriuous speedup (antirez)
2010-05-12 fixed compilation warnings in the AOF sanity check tool (antirez)
2010-05-12 Merge branch 'vm-speedup' (antirez)
2010-05-11 fix to return error when calling INCR on a non-string type (Pieter Noordhuis)
2010-05-11 load objects encoded from disk directly without useless conversion (antirez)
2010-05-11 fixed a problem leading to crashes, as keys can't be currently specially encoded, so we can't encode integers at object loading time... For now this can be fixed passing a few flags, or later can be fixed allowing encoded keys as well (antirez)
2010-05-11 long long to string conversion speedup applied in other places as well. Still the code has bugs, fixing right now... (antirez)
2010-05-11 hand written code to turn a long long into a string -> very big speed win (antirez)
2010-05-11 added specialized function to compare string objects for perfect match that is optimized for this task (antirez)
2010-05-11 better use of encoding inforamtion in dictEncObjKeyCompare (antirez)
2010-05-10 CONFIG now can change appendfsync policy at run time (antirez)
2010-05-10 CONFIG command now supports hot modification of RDB saving parameters. (antirez)
2010-05-10 while loading the rdb file don't add the key to the dictionary at all if it's already expired, instead of removing it just after the insertion. (antirez)
2010-05-10 Merge branch 'check-aof' of git://github.com/pietern/redis (antirez)
2010-05-08 minor changes to improve code readability (antirez)
2010-05-08 swap objects out directly while loading an RDB file if we detect we can't stay in the vm max memory limits anyway (antirez)
2010-05-07 change command names no longer used to zunion/zinter (Pieter Noordhuis)
2010-05-07 DEBUG POPULATE command for fast creation of test databases (antirez)
2010-05-07 update TODO (Pieter Noordhuis)
2010-05-07 swap arguments in blockClientOnSwappedKeys to be consistent (Pieter Noordhuis)
2010-05-07 added function that preloads all keys needed to execute a MULTI/EXEC block (Pieter Noordhuis)
2010-05-07 add sanity check to zunionInterBlockClientOnSwappedKeys, as the number of keys used is provided as argument to the function (Pieter Noordhuis)
2010-05-07 make prototype of custom function to preload keys from the vm match the prototype of waitForMultipleSwappedKeys (Pieter Noordhuis)
2010-05-07 extract preloading of multiple keys according to the command prototype to a separate function (Pieter Noordhuis)
2010-05-07 make append only filename configurable (Pieter Noordhuis)
2010-05-07 don't load value from VM for EXISTS (Pieter Noordhuis)
2010-05-07 swap file name pid expansion removed. Not suited for mission critical software... (antirez)
2010-05-07 Swap file is now locked (antirez)
2010-05-06 Merge branch 'master' into aof-speedup (antirez)
2010-05-06 log error and quit when the AOF contains an unfinished MULTI (antirez)
2010-05-06 log error and quit when the AOF contains an unfinished MULTI (Pieter Noordhuis)
2010-05-06 Merge branch 'master' into check-aof (Pieter Noordhuis)
2010-05-06 hincrby should report an error when called against a hash key that doesn't contain an integer (Pieter Noordhuis)
2010-05-06 AOF writes are now accumulated into a buffer and flushed into disk just before re-entering the event loop. A lot less writes but still this guarantees that AOF is written before the client gets a positive reply about a write operation, as no reply is trasnmitted before re-entering into the event loop. (antirez)
2010-05-06 clarified a few messages in redis.conf (antirez)
2010-05-05 ask for confirmation before AOF is truncated (Pieter Noordhuis)
2010-05-05 str can be free'd outside readString (Pieter Noordhuis)
2010-05-05 moved argument parsing around (Pieter Noordhuis)
2010-05-05 ignore redis-check-aof binary (Pieter Noordhuis)
2010-05-05 allow AOF to be fixed by truncating to the portion of the file that is valid (Pieter Noordhuis)
2010-05-05 tool to check if AOF is valid (Pieter Noordhuis)
2010-05-02 included fmacros.h in linenose.c to avoid compilation warnings on Linux (antirez)
2010-05-02 compilation fix for mac os x (antirez)
2010-05-02 Merge branch 'master' of git@github.com:antirez/redis (antirez)
2010-05-02 On Linux now fdatasync() is used insetad of fsync() in order to flush the AOF file kernel buffers (antirez)
2010-04-30 More tests for APPEND and tests for SUBSTR (antirez)
2010-04-30 linenoise.c updated, now redis-cli can be used in a pipe (antirez)
2010-04-29 redis-cli minor fix (less segfault is better) (antirez)
2010-04-29 New MONITOR output format with timestamp, every command in a single line, string representations (antirez)

1
Makefile
View File

@ -2,6 +2,7 @@
# Copyright (C) 2009 Salvatore Sanfilippo <antirez at gmail dot com>
# This file is released under the BSD license, see the COPYING file
release_hdr := $(shell sh -c './mkreleasehdr.sh')
uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
OPTIMIZATION?=-O2
ifeq ($(uname_S),SunOS)

48
doc/AppendCommand.html Normal file
View File

@ -0,0 +1,48 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>AppendCommand: Contents</b><br>&nbsp;&nbsp;<a href="#APPEND _key_ _value_">APPEND _key_ _value_</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Examples">Examples</a>
</div>
<h1 class="wikiname">AppendCommand</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="StringCommandsSidebar.html">StringCommandsSidebar</a><h1><a name="APPEND _key_ _value_">APPEND _key_ _value_</a></h1>
<i>Time complexity: O(1). The amortized time complexity is O(1) assuming the appended value is small and the already present value is of any size, since the dynamic string library used by Redis will double the free space available on every reallocation.</i><blockquote>If the <i>key</i> already exists and is a string, this command appends theprovided value at the end of the string.If the <i>key</i> does not exist it is created and set as an empty string, soAPPEND will be very similar to SET in this special case.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Integer reply</a>, specifically the total length of the string after the append operation.<h2><a name="Examples">Examples</a></h2><pre class="codeblock python" name="code">
redis&gt; exists mykey
(integer) 0
redis&gt; append mykey &quot;Hello &quot;
(integer) 6
redis&gt; append mykey &quot;World&quot;
(integer) 11
redis&gt; get mykey
&quot;Hello World&quot;
</pre>
</div>
</div>
</div>
</body>
</html>

49
doc/BlpopCommand.html Normal file
View File

@ -0,0 +1,49 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>BlpopCommand: Contents</b><br>&nbsp;&nbsp;<a href="#BLPOP _key1_ _key2_ ... _keyN_ _timeout_ (Redis &gt;">BLPOP _key1_ _key2_ ... _keyN_ _timeout_ (Redis &gt;</a><br>&nbsp;&nbsp;<a href="#BRPOP _key1_ _key2_ ... _keyN_ _timeout_ (Redis &gt;">BRPOP _key1_ _key2_ ... _keyN_ _timeout_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Non blocking behavior">Non blocking behavior</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Blocking behavior">Blocking behavior</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Multiple clients blocking for the same keys">Multiple clients blocking for the same keys</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">BlpopCommand</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="ListCommandsSidebar.html">ListCommandsSidebar</a><h1><a name="BLPOP _key1_ _key2_ ... _keyN_ _timeout_ (Redis &gt;">BLPOP _key1_ _key2_ ... _keyN_ _timeout_ (Redis &gt;</a></h1> 1.3.1) =
<h1><a name="BRPOP _key1_ _key2_ ... _keyN_ _timeout_ (Redis &gt;">BRPOP _key1_ _key2_ ... _keyN_ _timeout_ (Redis &gt;</a></h1> 1.3.1) =
<i>Time complexity: O(1)</i><blockquote>BLPOP (and BRPOP) is a blocking list pop primitive. You can see this commandsas blocking versions of <a href="LpopCommand.html">LPOP</a> and <a href="LpopCommand.html">RPOP</a> able toblock if the specified keys don't exist or contain empty lists.</blockquote>
<blockquote>The following is a description of the exact semantic. We describe BLPOP butthe two commands are identical, the only difference is that BLPOP pops theelement from the left (head) of the list, and BRPOP pops from the right (tail).</blockquote>
<h2><a name="Non blocking behavior">Non blocking behavior</a></h2><blockquote>When BLPOP is called, if at least one of the specified keys contain a nonempty list, an element is popped from the head of the list and returned tothe caller together with the name of the key (BLPOP returns a two elementsarray, the first element is the key, the second the popped value).</blockquote>
<blockquote>Keys are scanned from left to right, so for instance if youissue <b>BLPOP list1 list2 list3 0</b> against a dataset where <b>list1</b> does notexist but <b>list2</b> and <b>list3</b> contain non empty lists, BLPOP guaranteesto return an element from the list stored at <b>list2</b> (since it is the firstnon empty list starting from the left).</blockquote>
<h2><a name="Blocking behavior">Blocking behavior</a></h2><blockquote>If none of the specified keys exist or contain non empty lists, BLPOPblocks until some other client performs a <a href="RpushCommand.html">LPUSH</a> oran <a href="RpushCommand.html">RPUSH</a> operation against one of the lists.</blockquote>
<blockquote>Once new data is present on one of the lists, the client finally returnswith the name of the key unblocking it and the popped value.</blockquote>
<blockquote>When blocking, if a non-zero timeout is specified, the client will unblockreturning a nil special value if the specified amount of seconds passedwithout a push operation against at least one of the specified keys.</blockquote>
<blockquote>A timeout of zero means instead to block forever.</blockquote>
<h2><a name="Multiple clients blocking for the same keys">Multiple clients blocking for the same keys</a></h2><blockquote>Multiple clients can block for the same key. They are put intoa queue, so the first to be served will be the one that started to waitearlier, in a first-blpopping first-served fashion.</blockquote>
<h2><a name="Return value">Return value</a></h2><blockquote>BLPOP returns a two-elements array via a multi bulk reply in order to returnboth the unblocking key and the popped value.</blockquote>
<blockquote>When a non-zero timeout is specified, and the BLPOP operation timed out,the return value is a nil multi bulk reply. Most client values will returnfalse or nil accordingly to the programming language used.</blockquote>
<a href="ReplyTypes.html">Multi bulk reply</a>
</div>
</div>
</div>
</body>
</html>

2
doc/CommandReference.html
View File

@ -37,7 +37,7 @@
<h2><a name="Transactions">Transactions</a></h2><ul><li> <a href="MultiExecCommand.html">MULTI/EXEC/DISCARD</a> <code name="code" class="python">Redis atomic transactions</code></li></ul>
<h2><a name="Publish/Subscribe">Publish/Subscribe</a></h2><ul><li> <a href="PublishSubscribe.html">SUBSCRIBE/UNSUBSCRIBE/PUBLISH</a> <code name="code" class="python">Redis Public/Subscribe messaging paradigm implementation</code></li></ul>
<h2><a name="Persistence control commands">Persistence control commands</a></h2><ul><li> <a href="SaveCommand.html">SAVE</a> <code name="code" class="python">Synchronously save the DB on disk</code></li><li> <a href="BgsaveCommand.html">BGSAVE</a> <code name="code" class="python">Asynchronously save the DB on disk</code></li><li> <a href="LastsaveCommand.html">LASTSAVE</a> <code name="code" class="python">Return the UNIX time stamp of the last successfully saving of the dataset on disk</code></li><li> <a href="ShutdownCommand.html">SHUTDOWN</a> <code name="code" class="python">Synchronously save the DB on disk, then shutdown the server</code></li><li> <a href="BgrewriteaofCommand.html">BGREWRITEAOF</a> <code name="code" class="python">Rewrite the append only file in background when it gets too big</code></li></ul>
<h2><a name="Remote server control commands">Remote server control commands</a></h2><ul><li> <a href="InfoCommand.html">INFO</a> <code name="code" class="python">Provide information and statistics about the server</code></li><li> <a href="MonitorCommand.html">MONITOR</a> <code name="code" class="python">Dump all the received requests in real time</code></li><li> <a href="SlaveofCommand.html">SLAVEOF</a> <code name="code" class="python">Change the replication settings</code></li></ul>
<h2><a name="Remote server control commands">Remote server control commands</a></h2><ul><li> <a href="InfoCommand.html">INFO</a> <code name="code" class="python">Provide information and statistics about the server</code></li><li> <a href="MonitorCommand.html">MONITOR</a> <code name="code" class="python">Dump all the received requests in real time</code></li><li> <a href="SlaveofCommand.html">SLAVEOF</a> <code name="code" class="python">Change the replication settings</code></li><li> <a href="ConfigCommand.html">CONFIG</a> <code name="code" class="python">Configure a Redis server at runtime</code></li></ul>
</div>
</div>

76
doc/ConfigCommand.html Normal file
View File

@ -0,0 +1,76 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>ConfigCommand: Contents</b><br>&nbsp;&nbsp;<a href="#CONFIG GET _pattern_ (Redis &gt;">CONFIG GET _pattern_ (Redis &gt;</a><br>&nbsp;&nbsp;<a href="#CONFIG SET _parameter_ _value_ (Redis &gt;">CONFIG SET _parameter_ _value_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#CONFIG GET _pattern_">CONFIG GET _pattern_</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#CONFIG SET _parameter_ _value_">CONFIG SET _parameter_ _value_</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Parameters value format">Parameters value format</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#See Also">See Also</a>
</div>
<h1 class="wikiname">ConfigCommand</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="ControlCommandsSidebar.html">ControlCommandsSidebar</a><h1><a name="CONFIG GET _pattern_ (Redis &gt;">CONFIG GET _pattern_ (Redis &gt;</a></h1> 2.0)=
<h1><a name="CONFIG SET _parameter_ _value_ (Redis &gt;">CONFIG SET _parameter_ _value_ (Redis &gt;</a></h1> 2.0)=<br/><br/><blockquote>The CONFIG command is able to retrieve or alter the configuration of a runningRedis server. Not all the configuration parameters are supported.</blockquote>
<blockquote>CONFIG has two sub commands, GET and SET. The GET command is used to readthe configuration, while the SET command is used to alter the configuration.</blockquote>
<h2><a name="CONFIG GET _pattern_">CONFIG GET _pattern_</a></h2><blockquote>CONFIG GET returns the current configuration parameters. This sub commandonly accepts a single argument, that is glob style pattern. All theconfiguration parameters matching this parameter are reported as alist of key-value pairs. Example:</blockquote><pre class="codeblock python" name="code">
$ redis-cli config get '*'
1. &quot;dbfilename&quot;
2. &quot;dump.rdb&quot;
3. &quot;requirepass&quot;
4. (nil)
5. &quot;masterauth&quot;
6. (nil)
7. &quot;maxmemory&quot;
8. &quot;0\n&quot;
9. &quot;appendfsync&quot;
10. &quot;everysec&quot;
11. &quot;save&quot;
12. &quot;3600 1 300 100 60 10000&quot;
$ redis-cli config get 'm*'
1. &quot;masterauth&quot;
2. (nil)
3. &quot;maxmemory&quot;
4. &quot;0\n&quot;
</pre>The return type of the command is a <a href="ReplyTypes.html">bulk reply</a>.<h2><a name="CONFIG SET _parameter_ _value_">CONFIG SET _parameter_ _value_</a></h2><blockquote>CONFIG SET is used in order to reconfigure the server, setting a specificconfiguration parameter to a new value.</blockquote>
<blockquote>The list of configuration parameters supported by CONFIG SET can beobtained issuing a <code name="code" class="python">CONFIG GET *</code> command.</blockquote>
<blockquote>The configuration set using CONFIG SET is immediately loaded by the Redisserver that will start acting as specified starting from the next command.</blockquote>
<blockquote>Example:</blockquote><pre class="codeblock python python" name="code">
$ ./redis-cli
redis&gt; set x 10
OK
redis&gt; config set maxmemory 200
OK
redis&gt; set y 20
(error) ERR command not allowed when used memory &gt; 'maxmemory'
redis&gt; config set maxmemory 0
OK
redis&gt; set y 20
OK
</pre><h2><a name="Parameters value format">Parameters value format</a></h2><blockquote>The value of the configuration parameter is the same as the one of thesame parameter in the Redis configuration file, with the following exceptions:</blockquote>
<ul><li> The <code name="code" class="python">save</code> paramter is a list of space-separated integers. Every pair of integers specify the time and number of changes limit to trigger a save. For instance the command <code name="code" class="python">CONFIG SET save &quot;3600 10 60 10000&quot;</code> will configure the server to issue a background saving of the RDB file every 3600 seconds if there are at least 10 changes in the dataset, and every 60 seconds if there are at least 10000 changes. To completely disable automatic snapshots just set the parameter as an empty string.</li><li> All the integer parameters representing memory are returned and accepted only using bytes as unit.</li></ul>
<h2><a name="See Also">See Also</a></h2>The <a href="InfoCommand.html">INFO</a> command can be used in order to read configuriaton parameters that are not available in the CONFIG command.
</div>
</div>
</div>
</body>
</html>

44
doc/EventLibray.html Normal file
View File

@ -0,0 +1,44 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>EventLibray: Contents</b><br>&nbsp;&nbsp;<a href="#Event Library">Event Library</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Why is an Event Library needed at all?">Why is an Event Library needed at all?</a>
</div>
<h1 class="wikiname">EventLibray</h1>
<div class="summary">
</div>
<div class="narrow">
<h1><a name="Event Library">Event Library</a></h1><h2><a name="Why is an Event Library needed at all?">Why is an Event Library needed at all?</a></h2>Let us figure it out through a series of Q&amp;As.<br/><br/>Q: What do you expect a network server to be doing all the time? &lt;br/&gt;
A: Watch for inbound connections on the port its listening and accept them.<br/><br/>Q: Calling <a href="http://man.cx/accept%282%29" target="_blank">accept</a> yields a descriptor. What do I do with it?&lt;br/&gt;
A: Save the descriptor and do a non-blocking read/write operation on it.<br/><br/>Q: Why does the read/write have to be non-blocking?&lt;br/&gt;
A: If the file operation ( even a socket in Unix is a file ) is blocking how could the server for example accept other connection requests when its blocked in a file I/O operation.<br/><br/>Q: I guess I have to do many such non-blocking operations on the socket to see when it's ready. Am I right?&lt;br/&gt;
A: Yes. That is what an event library does for you. Now you get it.<br/><br/>Q: How do Event Libraries do what they do?&lt;br/&gt;
A: They use the operating system's <a href="http://www.devshed.com/c/a/BrainDump/Linux-Files-and-the-Event-Poll-Interface/" target="_blank">polling</a> facility along with timers.<br/><br/>Q: So are there any open source event libraries that do what you just described? &lt;br/&gt;
A: Yes. Libevent and Libev are two such event libraries that I can recall off the top of my head.<br/><br/>Q: Does Redis use such open source event libraries for handling socket I/O?&lt;br/&gt;
A: No. For various <a href="http://groups.google.com/group/redis-db/browse_thread/thread/b52814e9ef15b8d0/" target="_blank">reasons</a> Redis uses its own event library.
</div>
</div>
</div>
</body>
</html>

83
doc/HackingStrings.html Normal file
View File

@ -0,0 +1,83 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>HackingStrings: Contents</b><br>&nbsp;&nbsp;<a href="#Hacking Strings">Hacking Strings</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Creating Redis Strings">Creating Redis Strings</a>
</div>
<h1 class="wikiname">HackingStrings</h1>
<div class="summary">
</div>
<div class="narrow">
<h1><a name="Hacking Strings">Hacking Strings</a></h1>The implementation of Redis strings is contained in <b></b>sds.c<b></b> ( sds stands for Simple Dynamic Strings ).<br/><br/>The C structure <i>sdshdr</i> declared in <b>sds.h</b> represents a Redis string:<br/><br/><pre class="codeblock python" name="code">
struct sdshdr {
long len;
long free;
char buf[];
};
</pre>The <i>buf</i> character array stores the actual string.<br/><br/>The <i>len</i> field stores the length of <i>buf</i>. This makes obtaining the length
of a Redis string an O(1) operation.<br/><br/>The <i>free</i> field stores the number of additional bytes available for use.<br/><br/>Together the <i>len</i> and <i>free</i> field can be thought of as holding the metadata of the
<i>buf</i> character array.<h2><a name="Creating Redis Strings">Creating Redis Strings</a></h2>A new data type named <code name="code" class="python">sds</code> is defined in <b>sds.h</b> to be a synonymn for a character pointer:<br/><br/><pre class="codeblock python python" name="code">
typedef char *sds;
</pre><code name="code" class="python">sdsnewlen</code> function defined in <b>sds.c</b> creates a new Redis String: <br/><br/><pre class="codeblock python python python" name="code">
sds sdsnewlen(const void *init, size_t initlen) {
struct sdshdr *sh;
sh = zmalloc(sizeof(struct sdshdr)+initlen+1);
#ifdef SDS_ABORT_ON_OOM
if (sh == NULL) sdsOomAbort();
#else
if (sh == NULL) return NULL;
#endif
sh-&gt;len = initlen;
sh-&gt;free = 0;
if (initlen) {
if (init) memcpy(sh-&gt;buf, init, initlen);
else memset(sh-&gt;buf,0,initlen);
}
sh-&gt;buf[initlen] = '\0';
return (char*)sh-&gt;buf;
}
</pre>Remember a Redis string is a variable of type <code name="code" class="python">struct sdshdr</code>. But <code name="code" class="python">sdsnewlen</code> returns a character pointer!!<br/><br/>That's a trick and needs some explanation.<br/><br/>Suppose I create a Redis string using <code name="code" class="python">sdsnewlen</code> like below:<br/><br/><pre class="codeblock python python python python" name="code">
sdsnewlen(&quot;redis&quot;, 5);
</pre>This creates a new variable of type <code name="code" class="python">struct sdshdr</code> allocating memory for <i>len</i> and <i>free</i>
fields as well as for the <i>buf</i> character array.<br/><br/><pre class="codeblock python python python python python" name="code">
sh = zmalloc(sizeof(struct sdshdr)+initlen+1); // initlen is length of init argument.
</pre>After <code name="code" class="python">sdsnewlen</code> succesfully creates a Redis string the result is something like:<br/><br/><pre class="codeblock python python python python python python" name="code">
-----------
|5|0|redis|
-----------
^ ^
sh sh-&gt;buf
</pre><code name="code" class="python">sdsnewlen</code> returns sh-&gt;buf to the caller.<br/><br/>What do you do if you need to free the Redis string pointed by <code name="code" class="python">sh</code>?<br/><br/>You want the pointer <code name="code" class="python">sh</code> but you only have the pointer <code name="code" class="python">sh-&gt;buf</code>.<br/><br/>Can you get the pointer <code name="code" class="python">sh</code> from <code name="code" class="python">sh-&gt;buf</code>?<br/><br/>Yes. Pointer arithmetic. Notice from the above ASCII art that if you subtract
the size of two longs from <code name="code" class="python">sh-&gt;buf</code> you get the pointer <code name="code" class="python">sh</code>. <br/><br/>The sizeof two longs happens to be the size of <code name="code" class="python">struct sdshdr</code>.<br/><br/>Look at <code name="code" class="python">sdslen</code> function and see this trick at work:<br/><br/><pre class="codeblock python python python python python python python" name="code">
size_t sdslen(const sds s) {
struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));
return sh-&gt;len;
}
</pre>Knowing this trick you could easily go through the rest of the functions in <b>sds.c</b>.<br/><br/>The Redis string implementation is hidden behind an interface that accepts only character pointers. The users of Redis strings need not care about how its implemented and treat Redis strings as a character pointer.
</div>
</div>
</div>
</body>
</html>

36
doc/HashCommandsSidebar.html Normal file
View File

@ -0,0 +1,36 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>HashCommandsSidebar: Contents</b>
</div>
<h1 class="wikiname">HashCommandsSidebar</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;== Hash Commands ==<br/><br/><ul><li> <a href="HsetCommand.html">HSET</a></li><li> <a href="HgetCommand.html">HGET</a></li><li> <a href="HsetnxCommand.html">HSETNX</a></li><li> <a href="HmsetCommand.html">HMSET</a></li><li> <a href="HincrbyCommand.html">HINCRBY</a></li><li> <a href="HexistsCommand.html">HEXISTS</a></li><li> <a href="HdelCommand.html">HDEL</a></li><li> <a href="HlenCommand.html">HLEN</a></li><li> <a href="HgetallCommand.html">HKEYS</a></li><li> <a href="HgetallCommand.html">HVALS</a></li><li> <a href="HgetallCommand.html">HGETALL</a></li></ul>
</div>
</div>
</div>
</body>
</html>

37
doc/Hashes.html Normal file
View File

@ -0,0 +1,37 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>Hashes: Contents</b><br>&nbsp;&nbsp;<a href="#Redis Hash Type">Redis Hash Type</a><br>&nbsp;&nbsp;<a href="#Implementation details">Implementation details</a>
</div>
<h1 class="wikiname">Hashes</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="HashCommandsSidebar.html">HashCommandsSidebar</a><h1><a name="Redis Hash Type">Redis Hash Type</a></h1>Redis Hashes are unordered maps of <a href="String.html">Redis Strings</a> between fields and values. It is possible to add, remove, test for existence of fields in O(1) amortized time. It is also possible to enumerate all the keys, values, or both, in O(N) (where N is the number of fields inside the hash).<br/><br/>Redis Hashes are interesting because they are very well suited to represent objects. For instance web applications users can be represented by a Redis Hash containing fields such username, encrpypted_password, lastlogin, and so forth.<br/><br/>Another very important property of Redis Hashes is that they use very little memory for hashes composed of a small number of fields (configurable, check redis.conf for details), compared to storing every field as a top level Redis key. This is obtained using a different specialized representation for small hashes. See the implementation details paragraph below for more information.<br/><br/>Commands operating on hashes try to make a good use of the return value in order to signal the application about previous existence of fields. For instance the <a href="HsetCommand.html">HSET</a> command will return 1 if the field set was not already present in the hash, otherwise will return 0 (and the user knows this was just an update operation).<br/><br/>The max number of fields in a set is 232-1 (4294967295, more than 4 billion of members per hash).<h1><a name="Implementation details">Implementation details</a></h1>The obvious internal representation of hashes is indeed an hash table, as the name of the data structure itself suggests. Still the drawback of this representation is that there is a lot of space overhead for hash table metadata.<br/><br/>Because one of the most interesting uses of Hashes is object encoding, and objects are often composed of a few fields each, Redis uses a different internal representation for small hashes (for Redis to consider a hash small, this must be composed a limited number of fields, and each field and value can't exceed a given number of bytes. All this is user-configurable).<br/><br/>Small hashes are thus encoded using a data structure called zipmap (is not something you can find in a CS book, the name is a Redis invention), that is a very memory efficient data structure to represent string to string maps, at the cost of being O(N) instead of O(1) for most operations. Since the constant times of this data structure are very small, and the zipmaps are converted into real hash tables once they are big enough, the amortized time of Redis hashes is still O(1), and in the practice small zipmaps are not slower than small hash tables because they are designed for good cache locality and fast access.<br/><br/>The result is that small hashes are both memory efficient and fast, while bigger hashes are fast but not as memory efficient than small hashes.
</div>
</div>
</div>
</body>
</html>

39
doc/HdelCommand.html Normal file
View File

@ -0,0 +1,39 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>HdelCommand: Contents</b><br>&nbsp;&nbsp;<a href="#HDEL _key_ _field_ (Redis &gt;">HDEL _key_ _field_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">HdelCommand</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="HashCommandsSidebar.html">HashCommandsSidebar</a><h1><a name="HDEL _key_ _field_ (Redis &gt;">HDEL _key_ _field_ (Redis &gt;</a></h1> 1.3.10)=
<i>Time complexity: O(1)</i><blockquote>Remove the specified <i>field</i> from an hash stored at <i>key</i>.</blockquote>
<blockquote>If the <i>field</i> was present in the hash it is returned and 1 is returned, otherwise 0 is returned and no operation is performed.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Integer reply</a>
</div>
</div>
</div>
</body>
</html>

39
doc/HexistsCommand.html Normal file
View File

@ -0,0 +1,39 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>HexistsCommand: Contents</b><br>&nbsp;&nbsp;<a href="#HEXISTS _key_ _field_ (Redis &gt;">HEXISTS _key_ _field_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">HexistsCommand</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="HashCommandsSidebar.html">HashCommandsSidebar</a><h1><a name="HEXISTS _key_ _field_ (Redis &gt;">HEXISTS _key_ _field_ (Redis &gt;</a></h1> 1.3.10)=
<i>Time complexity: O(1)</i><blockquote>Return 1 if the hash stored at <i>key</i> contains the specified <i>field</i>.</blockquote>
<blockquote>Return 0 if the <i>key</i> is not found or the <i>field</i> is not present.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Integer reply</a>
</div>
</div>
</div>
</body>
</html>

39
doc/HgetCommand.html Normal file
View File

@ -0,0 +1,39 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>HgetCommand: Contents</b><br>&nbsp;&nbsp;<a href="#HGET _key_ _field_ (Redis &gt;">HGET _key_ _field_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">HgetCommand</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="HashCommandsSidebar.html">HashCommandsSidebar</a><h1><a name="HGET _key_ _field_ (Redis &gt;">HGET _key_ _field_ (Redis &gt;</a></h1> 1.3.10)=
<i>Time complexity: O(1)</i><blockquote>If <i>key</i> holds a hash, retrieve the value associated to the specified <i>field</i>.</blockquote>
<blockquote>If the <i>field</i> is not found or the <i>key</i> does not exist, a special 'nil' value is returned.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Bulk reply</a>
</div>
</div>
</div>
</body>
</html>

40
doc/HgetallCommand.html Normal file
View File

@ -0,0 +1,40 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>HgetallCommand: Contents</b><br>&nbsp;&nbsp;<a href="#HKEYS _key_ (Redis &gt;">HKEYS _key_ (Redis &gt;</a><br>&nbsp;&nbsp;<a href="#HVALS _key_ (Redis &gt;">HVALS _key_ (Redis &gt;</a><br>&nbsp;&nbsp;<a href="#HGETALL _key_ (Redis &gt;">HGETALL _key_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">HgetallCommand</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="HashCommandsSidebar.html">HashCommandsSidebar</a><h1><a name="HKEYS _key_ (Redis &gt;">HKEYS _key_ (Redis &gt;</a></h1> 1.3.10)=
<h1><a name="HVALS _key_ (Redis &gt;">HVALS _key_ (Redis &gt;</a></h1> 1.3.10)=
<h1><a name="HGETALL _key_ (Redis &gt;">HGETALL _key_ (Redis &gt;</a></h1> 1.3.10)=
<i>Time complexity: O(N), where N is the total number of entries</i><blockquote>HKEYS returns all the fields names contained into a hash, HVALS all the associated values, while HGETALL returns both the fields and values in the form of <i>field1</i>, <i>value1</i>, <i>field2</i>, <i>value2</i>, ..., <i>fieldN</i>, <i>valueN</i>.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Multi Bulk Reply</a>
</div>
</div>
</div>
</body>
</html>

45
doc/HincrbyCommand.html Normal file
View File

@ -0,0 +1,45 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>HincrbyCommand: Contents</b><br>&nbsp;&nbsp;<a href="#HINCRBY _key_ _field_ _value_ (Redis &gt;">HINCRBY _key_ _field_ _value_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Examples">Examples</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">HincrbyCommand</h1>
<div class="summary">
</div>
<div class="narrow">
<h1><a name="HINCRBY _key_ _field_ _value_ (Redis &gt;">HINCRBY _key_ _field_ _value_ (Redis &gt;</a></h1> 1.3.10)=
<i>Time complexity: O(1)</i><blockquote>Increment the number stored at <i>field</i> in the hash at <i>key</i> by <i>value</i>. If <i>key</i> does not exist, a new key holding a hash is created. If <i>field</i> does not exist or holds a string, the value is set to 0 before applying the operation.</blockquote>
<blockquote>The range of values supported by HINCRBY is limited to 64 bit signed integers.</blockquote><h2><a name="Examples">Examples</a></h2>
Since the <i>value</i> argument is signed you can use this command to perform both increments and decrements:<br/><br/><pre class="codeblock python" name="code">
HINCRBY key field 1 (increment by one)
HINCRBY key field -1 (decrement by one, just like the DECR command)
HINCRBY key field -10 (decrement by 10)
</pre>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Integer reply</a> The new value at <i>field</i> after the increment operation.
</div>
</div>
</div>
</body>
</html>

38
doc/HlenCommand.html Normal file
View File

@ -0,0 +1,38 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>HlenCommand: Contents</b><br>&nbsp;&nbsp;<a href="#HLEN _key_ (Redis &gt;">HLEN _key_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">HlenCommand</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="HashCommandsSidebar.html">HashCommandsSidebar</a><h1><a name="HLEN _key_ (Redis &gt;">HLEN _key_ (Redis &gt;</a></h1> 1.3.10)=
<i>Time complexity: O(1)</i><blockquote>Return the number of entries (fields) contained in the hash stored at <i>key</i>. If the specified <i>key</i> does not exist, 0 is returned assuming an empty hash. </blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Integer reply</a>
</div>
</div>
</div>
</body>
</html>

40
doc/HmsetCommand.html Normal file
View File

@ -0,0 +1,40 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>HmsetCommand: Contents</b><br>&nbsp;&nbsp;<a href="#HMSET _key_ _field1_ _value1_ ... _fieldN_ _valueN_ (Redis &gt;">HMSET _key_ _field1_ _value1_ ... _fieldN_ _valueN_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">HmsetCommand</h1>
<div class="summary">
</div>
<div class="narrow">
<h1><a name="HMSET _key_ _field1_ _value1_ ... _fieldN_ _valueN_ (Redis &gt;">HMSET _key_ _field1_ _value1_ ... _fieldN_ _valueN_ (Redis &gt;</a></h1> 1.3.10) =
<i>Time complexity: O(N) (with N being the number of fields)</i><blockquote>Set the respective fields to the respective values. HMSET replaces old values with new values.</blockquote>
<blockquote>If <i>key</i> does not exist, a new key holding a hash is created.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Status code reply</a> Always +OK because HMSET can't fail
</div>
</div>
</div>
</body>
</html>

40
doc/HsetCommand.html Normal file
View File

@ -0,0 +1,40 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>HsetCommand: Contents</b><br>&nbsp;&nbsp;<a href="#HSET _key_ _field_ _value_ (Redis &gt;">HSET _key_ _field_ _value_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">HsetCommand</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="HashCommandsSidebar.html">HashCommandsSidebar</a><h1><a name="HSET _key_ _field_ _value_ (Redis &gt;">HSET _key_ _field_ _value_ (Redis &gt;</a></h1> 1.3.10)=
<i>Time complexity: O(1)</i><blockquote>Set the specified hash <i>field</i> to the specified <i>value</i>.</blockquote>
<blockquote>If <i>key</i> does not exist, a new key holding a hash is created.</blockquote>
<blockquote>If the field already exists, and the HSET just produced an update of thevalue, 0 is returned, otherwise if a new field is created 1 is returned.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Integer reply</a>
</div>
</div>
</div>
</body>
</html>

41
doc/HsetnxCommand.html Normal file
View File

@ -0,0 +1,41 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>HsetnxCommand: Contents</b><br>&nbsp;&nbsp;<a href="#HSETNX _key_ _field_ _value_ (Redis &gt;">HSETNX _key_ _field_ _value_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">HsetnxCommand</h1>
<div class="summary">
</div>
<div class="narrow">
<h1><a name="HSETNX _key_ _field_ _value_ (Redis &gt;">HSETNX _key_ _field_ _value_ (Redis &gt;</a></h1> 1.3.10)=
<i>Time complexity: O(1)</i><blockquote>Set the specified hash <i>field</i> to the specified <i>value</i>, if <i>field</i> does not exist yet.</blockquote>
<blockquote>If <i>key</i> does not exist, a new key holding a hash is created.</blockquote>
<blockquote>If the field already exists, this operation has no effect and returns 0.Otherwise, the field is set to <i>value</i> and the operation returns 1.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Integer reply</a>
</div>
</div>
</div>
</body>
</html>

96
doc/MultiExecCommand.html Normal file
View File

@ -0,0 +1,96 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>MultiExecCommand: Contents</b><br>&nbsp;&nbsp;<a href="#MULTI">MULTI</a><br>&nbsp;&nbsp;<a href="#COMMAND_1 ...">COMMAND_1 ...</a><br>&nbsp;&nbsp;<a href="#COMMAND_2 ...">COMMAND_2 ...</a><br>&nbsp;&nbsp;<a href="#COMMAND_N ...">COMMAND_N ...</a><br>&nbsp;&nbsp;<a href="#EXEC or DISCARD">EXEC or DISCARD</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Usage">Usage</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#The DISCARD command">The DISCARD command</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">MultiExecCommand</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="GenericCommandsSidebar.html">GenericCommandsSidebar</a><h1><a name="MULTI">MULTI</a></h1>
<h1><a name="COMMAND_1 ...">COMMAND_1 ...</a></h1>
<h1><a name="COMMAND_2 ...">COMMAND_2 ...</a></h1>
<h1><a name="COMMAND_N ...">COMMAND_N ...</a></h1>
<h1><a name="EXEC or DISCARD">EXEC or DISCARD</a></h1><blockquote>MULTI, EXEC and DISCARD commands are the fundation of Redis Transactions.A Redis Transaction allows to execute a group of Redis commands in a singlestep, with two important guarantees:</blockquote>
<ul><li> All the commands in a transaction are serialized and executed sequentially. It can never happen that a request issued by another client is served <b>in the middle</b> of the execution of a Redis transaction. This guarantees that the commands are executed as a single atomic operation.</li><li> Either all of the commands or none are processed. The EXEC command triggers the execution of all the commands in the transaction, so if a client loses the connection to the server in the context of a transaction before calling the MULTI command none of the operations are performed, instead if the EXEC command is called, all the operations are performed. An exception to this rule is when the Append Only File is enabled: every command that is part of a Redis transaction will log in the AOF as long as the operation is completed, so if the Redis server crashes or is killed by the system administrator in some hard way it is possible that only a partial number of operations are registered.</li></ul>
<h2><a name="Usage">Usage</a></h2><blockquote>A Redis transaction is entered using the MULTI command. The command alwaysreplies with OK. At this point the user can issue multiple commands. Insteadto execute this commands Redis will &quot;queue&quot; them. All the commands areexecuted once EXEC is called.</blockquote>
<blockquote>Calling DISCARD instead will flush the transaction queue and will exitthe transaction.</blockquote>
<blockquote>The following is an example using the Ruby client:</blockquote><pre class="codeblock python" name="code">
?&gt; r.multi
=&gt; &quot;OK&quot;
&gt;&gt; r.incr &quot;foo&quot;
=&gt; &quot;QUEUED&quot;
&gt;&gt; r.incr &quot;bar&quot;
=&gt; &quot;QUEUED&quot;
&gt;&gt; r.incr &quot;bar&quot;
=&gt; &quot;QUEUED&quot;
&gt;&gt; r.exec
=&gt; [1, 1, 2]
</pre>
<blockquote>As it is possible to see from the session above, MULTI returns an &quot;array&quot; ofreplies, where every element is the reply of a single command in thetransaction, in the same order the commands were queued.</blockquote>
<blockquote>When a Redis connection is in the context of a MULTI request, all the commandswill reply with a simple string &quot;QUEUED&quot; if they are correct from thepoint of view of the syntax and arity (number of arguments) of the commaand.Some command is still allowed to fail during execution time.</blockquote>
<blockquote>This is more clear if at protocol level: in the following example one commandwill fail when executed even if the syntax is right:</blockquote><pre class="codeblock python python" name="code">
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
MULTI
+OK
SET a 3
abc
+QUEUED
LPOP a
+QUEUED
EXEC
*2
+OK
-ERR Operation against a key holding the wrong kind of value
</pre>
<blockquote>MULTI returned a two elements bulk reply in witch one of this is a +OKcode and one is a -ERR reply. It's up to the client lib to find a sensibleway to provide the error to the user.</blockquote>
<blockquote>IMPORTANT: even when a command will raise an error, all the other commandsin the queue will be processed. Redis will NOT stop the processing ofcommands once an error is found.</blockquote>
<blockquote>Another example, again using the write protocol with telnet, shows howsyntax errors are reported ASAP instead:</blockquote><pre class="codeblock python python python" name="code">
MULTI
+OK
INCR a b c
-ERR wrong number of arguments for 'incr' command
</pre>
<blockquote>This time due to the syntax error the &quot;bad&quot; INCR command is not queuedat all.</blockquote>
<h2><a name="The DISCARD command">The DISCARD command</a></h2><blockquote>DISCARD can be used in order to abort a transaction. No command will beexecuted, and the state of the client is again the normal one, outsideof a transaction. Example using the Ruby client:</blockquote><pre class="codeblock python python python python" name="code">
?&gt; r.set(&quot;foo&quot;,1)
=&gt; true
&gt;&gt; r.multi
=&gt; &quot;OK&quot;
&gt;&gt; r.incr(&quot;foo&quot;)
=&gt; &quot;QUEUED&quot;
&gt;&gt; r.discard
=&gt; &quot;OK&quot;
&gt;&gt; r.get(&quot;foo&quot;)
=&gt; &quot;1&quot;
</pre><h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Multi bulk reply</a>, specifically:<br/><br/><pre class="codeblock python python python python python" name="code">
The result of a MULTI/EXEC command is a multi bulk reply where every element is the return value of every command in the atomic transaction.
</pre>
</div>
</div>
</div>
</body>
</html>

116
doc/PublishSubscribe.html Normal file
View File

@ -0,0 +1,116 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>PublishSubscribe: Contents</b><br>&nbsp;&nbsp;<a href="#UNSUBSCRIBE channel_1 channel_2 ... channel_N">UNSUBSCRIBE channel_1 channel_2 ... channel_N</a><br>&nbsp;&nbsp;<a href="#UNSUBSCRIBE (unsubscribe from all channels)">UNSUBSCRIBE (unsubscribe from all channels)</a><br>&nbsp;&nbsp;<a href="#PSUBSCRIBE pattern_1 pattern_2 ... pattern_N">PSUBSCRIBE pattern_1 pattern_2 ... pattern_N</a><br>&nbsp;&nbsp;<a href="#PUNSUBSCRIBE pattern_1 pattern_2 ... pattern_N">PUNSUBSCRIBE pattern_1 pattern_2 ... pattern_N</a><br>&nbsp;&nbsp;<a href="#PUNSUBSCRIBE (unsubscribe from all patterns)">PUNSUBSCRIBE (unsubscribe from all patterns)</a><br>&nbsp;&nbsp;<a href="#PUBLISH channel message">PUBLISH channel message</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Format of pushed messages">Format of pushed messages</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Unsubscribing from all the channels at once">Unsubscribing from all the channels at once</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Wire protocol example">Wire protocol example</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#PSUBSCRIBE and PUNSUBSCRIBE: pattern matching subscriptions">PSUBSCRIBE and PUNSUBSCRIBE: pattern matching subscriptions</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Messages matching both a pattern and a channel subscription">Messages matching both a pattern and a channel subscription</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#The meaning of the count of subscriptions with pattern matching">The meaning of the count of subscriptions with pattern matching</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#More details on the PUBLISH command">More details on the PUBLISH command</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Programming Example">Programming Example</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Client library implementations hints">Client library implementations hints</a>
</div>
<h1 class="wikiname">PublishSubscribe</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;=SUBSCRIBE channel_1 channel_2 ... channel_N=
<h1><a name="UNSUBSCRIBE channel_1 channel_2 ... channel_N">UNSUBSCRIBE channel_1 channel_2 ... channel_N</a></h1>
<h1><a name="UNSUBSCRIBE (unsubscribe from all channels)">UNSUBSCRIBE (unsubscribe from all channels)</a></h1>
<h1><a name="PSUBSCRIBE pattern_1 pattern_2 ... pattern_N">PSUBSCRIBE pattern_1 pattern_2 ... pattern_N</a></h1>
<h1><a name="PUNSUBSCRIBE pattern_1 pattern_2 ... pattern_N">PUNSUBSCRIBE pattern_1 pattern_2 ... pattern_N</a></h1>
<h1><a name="PUNSUBSCRIBE (unsubscribe from all patterns)">PUNSUBSCRIBE (unsubscribe from all patterns)</a></h1>
<h1><a name="PUBLISH channel message">PUBLISH channel message</a></h1>Time complexity: subscribe is O(1), unsubscribe is O(N) where N is the number of clients already subscribed to a channel, publish is O(N+M) where N is the number of clients subscribed to the receiving channel, and M is the total number of subscribed patterns (by any client). Psubscribe is O(N) where N is the number of patterns the Psubscribing client is already subscribed to. Punsubscribe is O(N+M) where N is the number of patterns the Punsubscribing client is already subscribed and M is the number of total patterns subscribed in the system (by any client).<br/><br/><blockquote>SUBSCRIBE, UNSUBSCRIBE and PUBLISH commands implement the<a href="http://en.wikipedia.org/wiki/Publish/subscribe" target="_blank">Publish/Subscribe messaging paradigm</a> where (citing Wikipedia) senders (publishers) are not programmed to send their messages to specific receivers (subscribers). Rather, published messages are characterized into channels, without knowledge of what (if any) subscribers there may be. Subscribers express interest in one or more channels, and only receive messages that are of interest, without knowledge of what (if any) publishers there are. This decoupling of publishers and subscribers can allow for greater scalability and a more dynamic network topology.</blockquote>
<blockquote>For instance in order to subscribe to the channels foo and bar the clientwill issue the SUBSCRIBE command followed by the names of the channels.</blockquote><pre class="codeblock python" name="code">
SUBSCRIBE foo bar
</pre>
<blockquote>All the messages sent by other clients to this channels will be pushed bythe Redis server to all the subscribed clients, in the form of a threeelements bulk reply, where the first element is the message type, thesecond the originating channel, and the third argument the message payload.</blockquote>
<blockquote>A client subscribed to 1 or more channels should NOT issue other commandsother than SUBSCRIBE and UNSUBSCRIBE, but can subscribe or unsubscribeto other channels dynamically.</blockquote>
<blockquote>The reply of the SUBSCRIBE and UNSUBSCRIBE operations are sent in the formof messages, so that the client can just read a coherent stream of messageswhere the first element indicates the kind of message.</blockquote><h2><a name="Format of pushed messages">Format of pushed messages</a></h2>
<blockquote>Messages are in the form of multi bulk replies with three elements.The first element is the kind of message:</blockquote><ul><li> &quot;subscribe&quot;: means that we successfully subscribed to the channel given as second element of the multi bulk reply. The third argument represents the number of channels we are currently subscribed to.</li><li> &quot;unsubscribe&quot;: means that we successfully unsubscribed from the channel given as second element of the multi bulk reply. The third argument represents the number of channels we are currently subscribed to. If this latest argument is zero, we are no longer subscribed to any channel, and the client can issue any kind of Redis command as we are outside the Pub/sub state.</li><li> &quot;message&quot;: it is a message received as result of a PUBLISH command issued by another client. The second element is the name of the originating channel, and the third the actual message payload.</li></ul><h2><a name="Unsubscribing from all the channels at once">Unsubscribing from all the channels at once</a></h2>
If the UNSUBSCRIBE command is issued without additional arguments, it is equivalent to unsubscribing to all the channels we are currently subscribed. A message for every unsubscribed channel will be received.
<h2><a name="Wire protocol example">Wire protocol example</a></h2>
<pre class="codeblock python python" name="code">
SUBSCRIBE first second
*3
$9
subscribe
$5
first
:1
*3
$9
subscribe
$6
second
:2
</pre>
at this point from another client we issue a PUBLISH operation against the channel named &quot;second&quot;. This is what the first client receives:
<pre class="codeblock python python python" name="code">
*3
$7
message
$6
second
$5
Hello
</pre>
Now the client unsubscribes itself from all the channels using the UNSUBSCRIBE command without additional arguments:
<pre class="codeblock python python python python" name="code">
UNSUBSCRIBE
*3
$11
unsubscribe
$6
second
:1
*3
$11
unsubscribe
$5
first
:0
</pre>
<h2><a name="PSUBSCRIBE and PUNSUBSCRIBE: pattern matching subscriptions">PSUBSCRIBE and PUNSUBSCRIBE: pattern matching subscriptions</a></h2>
Redis Pub/Sub implementation supports pattern matching. Clients may subscribe to glob style patterns in order to receive all the messages sent to channel names matching a given pattern.<br/><br/>For instance the command:
<pre class="codeblock python python python python python" name="code">
PSUBSCRIBE news.*
</pre>
Will receive all the messages sent to the channel news.art.figurative and news.music.jazz and so forth. All the glob style patterns as valid, so multiple wild cards are supported.<br/><br/>Messages received as a result of pattern matching are sent in a different format:
<ul><li> The type of the message is &quot;pmessage&quot;: it is a message received as result of a PUBLISH command issued by another client, matching a pattern matching subscription. The second element is the original pattern matched, the third element is the name of the originating channel, and the last element the actual message payload.</li></ul>
Similarly to SUBSCRIBE and UNSUBSCRIBE, PSUBSCRIBE and PUNSUBSCRIBE commands are acknowledged by the system sending a message of type &quot;psubscribe&quot; and &quot;punsubscribe&quot; using the same format as the &quot;subscribe&quot; and &quot;unsubscribe&quot; message format.
<h2><a name="Messages matching both a pattern and a channel subscription">Messages matching both a pattern and a channel subscription</a></h2>
A client may receive a single message multiple time if it's subscribed to multiple patterns matching a published message, or it is subscribed to both patterns and channels matching the message. Like in the following example:
<pre class="codeblock python python python python python python" name="code">
SUBSCRIBE foo
PSUBSCRIBE f*
</pre>
In the above example, if a message is sent to the <b>foo</b> channel, the client will receive two messages, one of type &quot;message&quot; and one of type &quot;pmessage&quot;.
<h2><a name="The meaning of the count of subscriptions with pattern matching">The meaning of the count of subscriptions with pattern matching</a></h2>
In <b>subscribe</b>, <b>unsubscribe</b>, <b>psubscribe</b> and <b>punsubscribe</b> message types, the last argument is the count of subscriptions still active. This number is actually the total number of channels and patterns the client is still subscribed to. So the client will exit the Pub/Sub state only when this count will drop to zero as a result of unsubscription from all the channels and patterns.
<h2><a name="More details on the PUBLISH command">More details on the PUBLISH command</a></h2>
The Publish command is a bulk command where the first argument is the target class, and the second argument the data to send. It returns an Integer Reply representing the number of clients that received the message (that is, the number of clients that were listening for this class).
<h2><a name="Programming Example">Programming Example</a></h2>
Pieter Noordhuis provided a great example using Event-machine and Redis to create <a href="http://chat.redis-db.com" target="_blank">a multi user high performance web chat</a>, with source code included of course!
<h2><a name="Client library implementations hints">Client library implementations hints</a></h2>
Because all the messages received contain the original subscription causing the message delivery (the channel in the case of &quot;message&quot; type, and the original pattern in the case of &quot;pmessage&quot; type) clinet libraries may bind the original subscription to callbacks (that can be anonymous functions, blocks, function pointers, and so forth), using an hash table.<br/><br/>When a message is received an O(1) lookup can be done in order to deliver the message to the registered callback.
</div>
</div>
</div>
</body>
</html>

61
doc/RedisBigData.html Normal file
View File

@ -0,0 +1,61 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>RedisBigData: Contents</b><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#BGSAVE and BGREWRITEAOF blocking fork() call">BGSAVE and BGREWRITEAOF blocking fork() call</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Using multiple cores">Using multiple cores</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Splitting data into multiple instances">Splitting data into multiple instances</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#BGSAVE / AOFSAVE memory usage, and copy on write">BGSAVE / AOFSAVE memory usage, and copy on write</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#BGSAVE / AOFSAVE time for big datasets">BGSAVE / AOFSAVE time for big datasets</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Non blocking hash table">Non blocking hash table</a>
</div>
<h1 class="wikiname">RedisBigData</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;=Redis Big Data: facts and guidelines=<h2><a name="BGSAVE and BGREWRITEAOF blocking fork() call">BGSAVE and BGREWRITEAOF blocking fork() call</a></h2>
<pre class="codeblock python" name="code">
fork.c &amp;&amp; ./a.out
allocated: 1 MB, fork() took 0.000
allocated: 10 MB, fork() took 0.001
allocated: 100 MB, fork() took 0.007
allocated: 1000 MB, fork() took 0.059
allocated: 10000 MB, fork() took 0.460
allocated: 20000 MB, fork() took 0.895
allocated: 30000 MB, fork() took 1.327
allocated: 40000 MB, fork() took 1.759
allocated: 50000 MB, fork() took 2.190
allocated: 60000 MB, fork() took 2.621
allocated: 70000 MB, fork() took 3.051
allocated: 80000 MB, fork() took 3.483
allocated: 90000 MB, fork() took 3.911
allocated: 100000 MB, fork() took 4.340
allocated: 110000 MB, fork() took 4.770
allocated: 120000 MB, fork() took 5.202
</pre>
<h2><a name="Using multiple cores">Using multiple cores</a></h2>
<h2><a name="Splitting data into multiple instances">Splitting data into multiple instances</a></h2>
<h2><a name="BGSAVE / AOFSAVE memory usage, and copy on write">BGSAVE / AOFSAVE memory usage, and copy on write</a></h2>
<h2><a name="BGSAVE / AOFSAVE time for big datasets">BGSAVE / AOFSAVE time for big datasets</a></h2>
<h2><a name="Non blocking hash table">Non blocking hash table</a></h2>
</div>
</div>
</div>
</body>
</html>

70
doc/RedisEventLibrary.html Normal file
View File

@ -0,0 +1,70 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>RedisEventLibrary: Contents</b><br>&nbsp;&nbsp;<a href="#Redis Event Library">Redis Event Library</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Event Loop Initialization">Event Loop Initialization</a><br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="#aeCreateEventLoop">aeCreateEventLoop</a><br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="#aeCreateTimeEvent">aeCreateTimeEvent</a><br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="#aeCreateFileEvent">aeCreateFileEvent</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Event Loop Processing">Event Loop Processing</a><br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="#aeProcessEvents">aeProcessEvents</a><br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href="#processTimeEvents">processTimeEvents</a>
</div>
<h1 class="wikiname">RedisEventLibrary</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="RedisInternals.html">RedisInternals</a><h1><a name="Redis Event Library">Redis Event Library</a></h1>Redis implements its own event library. The event library is implemented in <b>ae.c</b>.<br/><br/>The best way to understand how the Redis event library works is to understand how Redis uses it.<h2><a name="Event Loop Initialization">Event Loop Initialization</a></h2>
<code name="code" class="python">initServer</code> function defined in <b>redis.c</b> initializes the numerous fields of the <code name="code" class="python">redisServer</code> structure variable. One such field is the Redis event loop <code name="code" class="python">el</code>:<br/><br/><pre class="codeblock python" name="code">
aeEventLoop *el
</pre><code name="code" class="python">initServer</code> initializes <code name="code" class="python">server.el</code> field by calling <code name="code" class="python">aeCreateEventLoop</code> defined in <b>ae.c</b>. The definition of <code name="code" class="python">aeEventLoop</code> is below:
<pre class="codeblock python python" name="code">
typedef struct aeEventLoop
{
int maxfd;
long long timeEventNextId;
aeFileEvent events[AE_SETSIZE]; /* Registered events */
aeFiredEvent fired[AE_SETSIZE]; /* Fired events */
aeTimeEvent *timeEventHead;
int stop;
void *apidata; /* This is used for polling API specific data */
aeBeforeSleepProc *beforesleep;
} aeEventLoop;
</pre><h3><a name="aeCreateEventLoop">aeCreateEventLoop</a></h3><code name="code" class="python">aeCreateEventLoop</code> first mallocs aeEventLoop structure then calls ae_epoll.c:aeApiCreate<code name="code" class="python">.
</code>aeApiCreate<code name="code" class="python"> mallocs </code>aeApiState<code name="code" class="python"> that has two fields - </code>epfd<code name="code" class="python"> that holds the epoll file descriptor returned by a call from [http://man.cx/epoll_create%282%29 epoll_create] and </code>events<code name="code" class="python"> that is of type </code>struct epoll_event<code name="code" class="python"> define by the Linux epoll library. The use of the </code>events<code name="code" class="python"> field will be described later.
Next is 'ae.c:aeCreateTimeEvent</code>. But before that <code name="code" class="python">initServer</code> call <code name="code" class="python">anet.c:anetTcpServer</code> that creates and returns a <i>listening descriptor</i>. The descriptor is listens to <b>port 6379</b> by default. The returned <i>listening descriptor</i> is stored in <code name="code" class="python">server.fd</code> field.<h3><a name="aeCreateTimeEvent">aeCreateTimeEvent</a></h3><code name="code" class="python">aeCreateTimeEvent</code> accepts the following as parameters:<br/><br/><ul><li> eventLoop: This is <code name="code" class="python">server.el</code> in <b>redis.c</b></li><li> milliseconds: The number of milliseconds from the curent time after which the timer expires.</li><li> proc: Function pointer. Stores the address of the function that has to be called after the timer expires.</li><li> clientData: Mostly NULL.</li><li> finalizerProc: Pointer to the function that has to be called before the timed event is removed from the list of timed events.</li></ul>
<code name="code" class="python">initServer</code> calls <code name="code" class="python">aeCreateTimeEvent</code> to add a timed event to <code name="code" class="python">timeEventHead</code> field of <code name="code" class="python">server.el</code>. <code name="code" class="python">timeEventHead</code> is a pointer to a list of such timed events. The call to <code name="code" class="python">aeCreateTimeEvent</code> from <code name="code" class="python">redis.c:initServer</code> function is given below:<br/><br/><pre class="codeblock python python python" name="code">
aeCreateTimeEvent(server.el /*eventLoop*/, 1 /*milliseconds*/, serverCron /*proc*/, NULL /*clientData*/, NULL /*finalizerProc*/);
</pre><code name="code" class="python">redis.c:serverCron</code> performs many operations that helps keep Redis running properly.<h3><a name="aeCreateFileEvent">aeCreateFileEvent</a></h3>The essence of <code name="code" class="python">aeCreateFileEvent</code> function is to execute <a href="http://man.cx/epoll_ctl" target="_blank">epoll_ctl</a> system call which adds a watch for <code name="code" class="python">EPOLLIN</code> event on the <i>listening descriptor</i> create by <code name="code" class="python">anetTcpServer</code> and associate it with the epoll descriptor created by a call to <code name="code" class="python">aeCreateEventLoop</code>. <br/><br/>Following is an explanation of what precisely <code name="code" class="python">aeCreateFileEvent</code> does when called from <code name="code" class="python">redis.c:initServer</code>.<br/><br/><code name="code" class="python">initServer</code> passes the following arguments to <code name="code" class="python">aeCreateFileEvent</code>:
<ul><li> server.el: The event loop created by <code name="code" class="python">aeCreateEventLoop</code>. The epoll descriptor is got from server.el. </li><li> server.fd: The <i>listening descriptor</i> that also serves as an index to access the relevant file event structure from the <code name="code" class="python">eventLoop-&gt;events</code> table and store extra information like the callback function.</li><li> AE_READABLE: Signifies that server.fd has to be watched for EPOLLIN event.</li><li> acceptHandler: The function that has to be executed when the event being watched for is ready. This function pointer is stored in <code name="code" class="python">eventLoop-&gt;events[server.fd]-&gt;rfileProc</code>. </li></ul>
This completes the initialization of Redis event loop.<h2><a name="Event Loop Processing">Event Loop Processing</a></h2><code name="code" class="python">ae.c:aeMain</code> called from <code name="code" class="python">redis.c:main</code> does the job of processing the event loop that is initialized in the previous phase.<br/><br/><code name="code" class="python">ae.c:aeMain</code> calls <code name="code" class="python">ae.c:aeProcessEvents</code> in a while loop that processes pending time and file events.<h3><a name="aeProcessEvents">aeProcessEvents</a></h3><code name="code" class="python">ae.c:aeProcessEvents</code> looks for the time event that will be pending in the smallest amount of time by calling <code name="code" class="python">ae.c:aeSearchNearestTimer</code> on the event loop. In our case there is only one timer event in the event loop that was created by <code name="code" class="python">ae.c:aeCreateTimeEvent</code>. <br/><br/>Remember, that timer event created by <code name="code" class="python">aeCreateTimeEvent</code> has by now probably elapsed because it had a expiry time of one millisecond. Since, the timer has already expired the seconds and microseconds fields of the <code name="code" class="python">tvp</code> timeval structure variable is initialized to zero. <br/><br/>The <code name="code" class="python">tvp</code> structure variable along with the event loop variable is passed to <code name="code" class="python">ae_epoll.c:aeApiPoll</code>.<br/><br/><code name="code" class="python">aeApiPoll</code> functions does a <a href="http://man.cx/epoll_wait" target="_blank">epoll_wait</a> on the epoll descriptor and populates the <code name="code" class="python">eventLoop-&gt;fired</code> table with the details:
<ul><li> fd: The descriptor that is now ready to do a read/write operation depending on the mask value. The </li><li> mask: The read/write event that can now be performed on the corresponding descriptor.</li></ul>
<code name="code" class="python">aeApiPoll</code> returns the number of such file events ready for operation. Now to put things in context, if any client has requested for a connection then aeApiPoll would have noticed it and populated the <code name="code" class="python">eventLoop-&gt;fired</code> table with an entry of the descriptor being the <i>listening descriptor</i> and mask being <code name="code" class="python">AE_READABLE</code>.<br/><br/>Now, <code name="code" class="python">aeProcessEvents</code> calls the <code name="code" class="python">redis.c:acceptHandler</code> registered as the callback. <code name="code" class="python">acceptHandler</code> executes [<a href="http://man.cx/accept(2" target="_blank">http://man.cx/accept(2</a>) accept] on the <i>listening descriptor</i> returning a <i>connected descriptor</i> with the client. <code name="code" class="python">redis.c:createClient</code> adds a file event on the <i>connected descriptor</i> through a call to <code name="code" class="python">ae.c:aeCreateFileEvent</code> like below:<br/><br/><pre class="codeblock python python python python" name="code">
if (aeCreateFileEvent(server.el, c-&gt;fd, AE_READABLE,
readQueryFromClient, c) == AE_ERR) {
freeClient(c);
return NULL;
}
</pre><code name="code" class="python">c</code> is the <code name="code" class="python">redisClient</code> structure variable and <code name="code" class="python">c-&gt;fd</code> is the connected descriptor.<br/><br/>Next the <code name="code" class="python">ae.c:aeProcessEvent</code> calls <code name="code" class="python">ae.c:processTimeEvents</code><h3><a name="processTimeEvents">processTimeEvents</a></h3><code name="code" class="python">ae.processTimeEvents</code> iterates over list of time events starting at <code name="code" class="python">eventLoop-&gt;timeEventHead</code>.<br/><br/>For every timed event that has elapsed <code name="code" class="python">processTimeEvents</code> calls the registered callback. In this case it calls the only timed event callback registered, that is, <code name="code" class="python">redis.c:serverCron</code>. The callback returns the time in milliseconds after which the callback must be called again. This change is recorded via a call to <code name="code" class="python">ae.c:aeAddMilliSeconds</code> and will be handled on the next iteration of <code name="code" class="python">ae.c:aeMain</code> while loop.<br/><br/>That's all.
</div>
</div>
</div>
</body>
</html>

37
doc/RedisGuides.html Normal file
View File

@ -0,0 +1,37 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>RedisGuides: Contents</b>
</div>
<h1 class="wikiname">RedisGuides</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;= Redis Guides and Howtos=
<ul><li> <a href="QuickStart.html">Redis Quick Start</a></li><li> <a href="VirtualMemoryUserGuide.html">Virutal Memory User Guide</a></li><li> <a href="IntroductionToRedisDataTypes.html">A Fifteen Minutes Introduction to the Redis Data Types</a></li><li> <a href="ReplicationHowto.html">The Redis Replication HOWTO</a></li><li> <a href="AppendOnlyFileHowto.html">The Append Only File HOWTO</a></li></ul>
</div>
</div>
</div>
</body>
</html>

38
doc/RedisInternals.html Normal file
View File

@ -0,0 +1,38 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>RedisInternals: Contents</b><br>&nbsp;&nbsp;<a href="#Redis Internals">Redis Internals</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Redis STRINGS">Redis STRINGS</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Redis Virtual Memory">Redis Virtual Memory</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Redis Event Library">Redis Event Library</a>
</div>
<h1 class="wikiname">RedisInternals</h1>
<div class="summary">
</div>
<div class="narrow">
<h1><a name="Redis Internals">Redis Internals</a></h1>This is a source code level documentation of Redis.<h2><a name="Redis STRINGS">Redis STRINGS</a></h2>String is the basic building block of Redis types. <br/><br/>Redis is a key-value store.
All Redis keys are strings and its also the simplest value type.<br/><br/><blockquote></blockquote>Lists, sets, sorted sets and hashes are other more complex value types and even
these are composed of strings.<br/><br/><a href="HackingStrings.html">Hacking Strings</a> documents the Redis String implementation details.<h2><a name="Redis Virtual Memory">Redis Virtual Memory</a></h2>A technical specification full of details about the <a href="VirtualMemorySpecification.html">Redis Virtual Memory subsystem</a><h2><a name="Redis Event Library">Redis Event Library</a></h2>Read <a href="EventLibray.html">event library</a> to understand what an event library does and why its needed.<br/><br/><a href="RedisEventLibrary.html">Redis event library</a> documents the implementation details of the event library used by Redis
</div>
</div>
</div>
</body>
</html>

42
doc/SetexCommand.html Normal file
View File

@ -0,0 +1,42 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>SetexCommand: Contents</b><br>&nbsp;&nbsp;<a href="#SETEX _key_ _time_ _value_">SETEX _key_ _time_ _value_</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">SetexCommand</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="StringCommandsSidebar.html">StringCommandsSidebar</a><h1><a name="SETEX _key_ _time_ _value_">SETEX _key_ _time_ _value_</a></h1>
<i>Time complexity: O(1)</i><blockquote>The command is exactly equivalent to the following group of commands:</blockquote><pre class="codeblock python" name="code">
SET _key_ _value_
EXPIRE _key_ _time_
</pre>
<blockquote>The operation is atomic. An atomic <a href="SetCommand.html">SET</a>+<a href="ExpireCommand.html">EXPIRE</a> operation was already providedusing <a href="MultiExecCommand.html">MULTI/EXEC</a>, but SETEX is a faster alternative providedbecause this operation is very common when Redis is used as a Cache.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Status code reply</a>
</div>
</div>
</div>
</body>
</html>

51
doc/SubstrCommand.html Normal file
View File

@ -0,0 +1,51 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>SubstrCommand: Contents</b><br>&nbsp;&nbsp;<a href="#SUBSTR _key_ _start_ _end_">SUBSTR _key_ _start_ _end_</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Examples">Examples</a>
</div>
<h1 class="wikiname">SubstrCommand</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="StringCommandsSidebar.html">StringCommandsSidebar</a><h1><a name="SUBSTR _key_ _start_ _end_">SUBSTR _key_ _start_ _end_</a></h1>
<i>Time complexity: O(start+n) (with start being the start index and n the total length of the requested range). Note that the lookup part of this command is O(1) so for small strings this is actually an O(1) command.</i><blockquote>Return a subset of the string from offset <i>start</i> to offset <i>end</i>(both offsets are inclusive).Negative offsets can be used in order to provide an offset starting fromthe end of the string. So -1 means the last char, -2 the penultimate andso forth.</blockquote>
<blockquote>The function handles out of range requests without raising an error, butjust limiting the resulting range to the actual length of the string.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Bulk reply</a><h2><a name="Examples">Examples</a></h2><pre class="codeblock python" name="code">
redis&gt; set s &quot;This is a string&quot;
OK
redis&gt; substr s 0 3
&quot;This&quot;
redis&gt; substr s -3 -1
&quot;ing&quot;
redis&gt; substr s 0 -1
&quot;This is a string&quot;
redis&gt; substr s 9 100000
&quot; string&quot;
</pre>
</div>
</div>
</div>
</body>
</html>

156
doc/VirtualMemorySpecification.html Normal file
View File

@ -0,0 +1,156 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>VirtualMemorySpecification: Contents</b><br>&nbsp;&nbsp;<a href="#Virtual Memory technical specification">Virtual Memory technical specification</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Keys vs Values: what is swapped out?">Keys vs Values: what is swapped out?</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#How does a swapped value looks like internally">How does a swapped value looks like internally</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#The Swap File">The Swap File</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Transfering objects from memory to swap">Transfering objects from memory to swap</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Loading objects back in memory">Loading objects back in memory</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#How blocking VM works">How blocking VM works</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Blocking VM swapping">Blocking VM swapping</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#What values to swap when we are out of memory?">What values to swap when we are out of memory?</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Blocking VM loading">Blocking VM loading</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Background saving when VM is active">Background saving when VM is active</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#The problem with the blocking VM">The problem with the blocking VM</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Threaded VM">Threaded VM</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#I/O Threads">I/O Threads</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Non blocking VM as probabilistic enhancement of blocking VM">Non blocking VM as probabilistic enhancement of blocking VM</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Blocking clients on swapped keys">Blocking clients on swapped keys</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Aborting I/O jobs">Aborting I/O jobs</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Questions?">Questions?</a>
</div>
<h1 class="wikiname">VirtualMemorySpecification</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="RedisInternals.html">RedisInternals</a><h1><a name="Virtual Memory technical specification">Virtual Memory technical specification</a></h1>This document details the internals of the Redis Virtual Memory subsystem. The intended audience is not the final user but programmers willing to understand or modify the Virtual Memory implementation.<h2><a name="Keys vs Values: what is swapped out?">Keys vs Values: what is swapped out?</a></h2>
The goal of the VM subsystem is to free memory transferring Redis Objects from memory to disk. This is a very generic command, but specifically, Redis transfers only objects associated with <i>values</i>. In order to understand better this concept we'll show, using the DEBUG command, how a key holding a value looks from the point of view of the Redis internals:
<pre class="codeblock python" name="code">
redis&gt; set foo bar
OK
redis&gt; debug object foo
Key at:0x100101d00 refcount:1, value at:0x100101ce0 refcount:1 encoding:raw serializedlength:4
</pre>
As you can see from the above output, the Redis top level hash table maps Redis Objects (keys) to other Redis Objects (values). The Virtual Memory is only able to swap <i>values</i> on disk, the objects associated to <i>keys</i> are always taken in memory: this trade off guarantees very good lookup performances, as one of the main design goals of the Redis VM is to have performances similar to Redis with VM disabled when the part of the dataset frequently used fits in RAM.<h2><a name="How does a swapped value looks like internally">How does a swapped value looks like internally</a></h2>
When an object is swapped out, this is what happens in the hash table entry:
<ul><li> The key continues to hold a Redis Object representing the key.</li><li> The value is set to NULL</li></ul>So you may wonder where we store the information that a given value (associated to a given key) was swapped out. Just in the key object!<br/><br/>This is how the Redis Object structure <i>robj</i> looks like:
<pre class="codeblock python python" name="code">
/* The actual Redis Object */
typedef struct redisObject {
void *ptr;
unsigned char type;
unsigned char encoding;
unsigned char storage; /* If this object is a key, where is the value?
* REDIS_VM_MEMORY, REDIS_VM_SWAPPED, ... */
unsigned char vtype; /* If this object is a key, and value is swapped out,
* this is the type of the swapped out object. */
int refcount;
/* VM fields, this are only allocated if VM is active, otherwise the
* object allocation function will just allocate
* sizeof(redisObjct) minus sizeof(redisObjectVM), so using
* Redis without VM active will not have any overhead. */
struct redisObjectVM vm;
} robj;
</pre>
As you can see there are a few fields about VM. The most important one is <i>storage</i>, that can be one of this values:
<ul><li> REDIS_VM_MEMORY: the associated value is in memory.</li><li> REDIS_VM_SWAPPED: the associated values is swapped, and the value entry of the hash table is just set to NULL.</li><li> REDIS_VM_LOADING: the value is swapped on disk, the entry is NULL, but there is a job to load the object from the swap to the memory (this field is only used when threaded VM is active).</li><li> REDIS_VM_SWAPPING: the value is in memory, the entry is a pointer to the actual Redis Object, but there is an I/O job in order to transfer this value to the swap file.</li></ul>If an object is swapped on disk (REDIS_VM_SWAPPED or REDIS_VM_LOADING), how do we know where it is stored, what type it is, and so forth? That's simple: the <i>vtype</i> field is set to the original type of the Redis object swapped, while the <i>vm</i> field (that is a <i>redisObjectVM</i> structure) holds information about the location of the object. This is the definition of this additional structure:
<pre class="codeblock python python python" name="code">
/* The VM object structure */
struct redisObjectVM {
off_t page; /* the page at which the object is stored on disk */
off_t usedpages; /* number of pages used on disk */
time_t atime; /* Last access time */
} vm;
</pre>
As you can see the structure contains the page at which the object is located in the swap file, the number of pages used, and the last access time of the object (this is very useful for the algorithm that select what object is a good candidate for swapping, as we want to transfer on disk objects that are rarely accessed).<br/><br/>As you can see, while all the other fields are using unused bytes in the old Redis Object structure (we had some free bit due to natural memory alignment concerns), the <i>vm</i> field is new, and indeed uses additional memory. Should we pay such a memory cost even when VM is disabled? No! This is the code to create a new Redis Object:
<pre class="codeblock python python python python" name="code">
... some code ...
if (server.vm_enabled) {
pthread_mutex_unlock(&amp;server.obj_freelist_mutex);
o = zmalloc(sizeof(*o));
} else {
o = zmalloc(sizeof(*o)-sizeof(struct redisObjectVM));
}
... some code ...
</pre>
As you can see if the VM system is not enabled we allocate just <code name="code" class="python">sizeof(*o)-sizeof(struct redisObjectVM)</code> of memory. Given that the <i>vm</i> field is the last in the object structure, and that this fields are never accessed if VM is disabled, we are safe and Redis without VM does not pay the memory overhead.<h2><a name="The Swap File">The Swap File</a></h2>
The next step in order to understand how the VM subsystem works is understanding how objects are stored inside the swap file. The good news is that's not some kind of special format, we just use the same format used to store the objects in .rdb files, that are the usual dump files produced by Redis using the SAVE command.<br/><br/>The swap file is composed of a given number of pages, where every page size is a given number of bytes. This parameters can be changed in redis.conf, since different Redis instances may work better with different values: it depends on the actual data you store inside it. The following are the default values:
<pre class="codeblock python python python python python" name="code">
vm-page-size 32
vm-pages 134217728
</pre>
Redis takes a &quot;bitmap&quot; (an contiguous array of bits set to zero or one) in memory, every bit represent a page of the swap file on disk: if a given bit is set to 1, it represents a page that is already used (there is some Redis Object stored there), while if the corresponding bit is zero, the page is free.<br/><br/>Taking this bitmap (that will call the page table) in memory is a huge win in terms of performances, and the memory used is small: we just need 1 bit for every page on disk. For instance in the example below 134217728 pages of 32 bytes each (4GB swap file) is using just 16 MB of RAM for the page table.<h2><a name="Transfering objects from memory to swap">Transfering objects from memory to swap</a></h2>
In order to transfer an object from memory to disk we need to perform the following steps (assuming non threaded VM, just a simple blocking approach):<br/><br/><ul><li> Find how many pages are needed in order to store this object on the swap file. This is trivially accomplished just calling the function <code name="code" class="python">rdbSavedObjectPages</code> that returns the number of pages used by an object on disk. Note that this function does not duplicate the .rdb saving code just to understand what will be the length <b>after</b> an object will be saved on disk, we use the trick of opening /dev/null and writing the object there, finally calling <code name="code" class="python">ftello</code> in order check the amount of bytes required. What we do basically is to save the object on a virtual very fast file, that is, /dev/null.</li><li> Now that we know how many pages are required in the swap file, we need to find this number of contiguous free pages inside the swap file. This task is accomplished by the <code name="code" class="python">vmFindContiguousPages</code> function. As you can guess this function may fail if the swap is full, or so fragmented that we can't easily find the required number of contiguous free pages. When this happens we just abort the swapping of the object, that will continue to live in memory.</li><li> Finally we can write the object on disk, at the specified position, just calling the function <code name="code" class="python">vmWriteObjectOnSwap</code>.</li></ul>As you can guess once the object was correctly written in the swap file, it is freed from memory, the storage field in the associated key is set to REDIS_VM_SWAPPED, and the used pages are marked as used in the page table.
<h2><a name="Loading objects back in memory">Loading objects back in memory</a></h2>
Loading an object from swap to memory is simpler, as we already know where the object is located and how many pages it is using. We also know the type of the object (the loading functions are required to know this information, as there is no header or any other information about the object type on disk), but this is stored in the <i>vtype</i> field of the associated key as already seen above.<br/><br/>Calling the function <code name="code" class="python">vmLoadObject</code> passing the key object associated to the value object we want to load back is enough. The function will also take care of fixing the storage type of the key (that will be REDIS_VM_MEMORY), marking the pages as freed in the page table, and so forth.<br/><br/>The return value of the function is the loaded Redis Object itself, that we'll have to set again as value in the main hash table (instead of the NULL value we put in place of the object pointer when the value was originally swapped out).
<h2><a name="How blocking VM works">How blocking VM works</a></h2>
Now we have all the building blocks in order to describe how the blocking VM works. First of all, an important detail about configuration. In order to enable blocking VM in Redis <code name="code" class="python">server.vm_max_threads</code> must be set to zero.
We'll see later how this max number of threads info is used in the threaded VM, for now all it's needed to now is that Redis reverts to fully blocking VM when this is set to zero.<br/><br/>We also need to introduce another important VM parameter, that is, <code name="code" class="python">server.vm_max_memory</code>. This parameter is very important as it is used in order to trigger swapping: Redis will try to swap objects only if it is using more memory than the max memory setting, otherwise there is no need to swap as we are matching the user requested memory usage.
<h2><a name="Blocking VM swapping">Blocking VM swapping</a></h2>
Swapping of object from memory to disk happens in the cron function. This function used to be called every second, while in the recent Redis versions on git it is called every 100 milliseconds (that is, 10 times per second).
If this function detects we are out of memory, that is, the memory used is greater than the vm-max-memory setting, it starts transferring objects from memory to disk in a loop calling the function <code name="code" class="python">vmSwapOneObect</code>. This function takes just one argument, if 0 it will swap objects in a blocking way, otherwise if it is 1, I/O threads are used. In the blocking scenario we just call it with zero as argument.<br/><br/>vmSwapOneObject acts performing the following steps:
<ul><li> The key space in inspected in order to find a good candidate for swapping (we'll see later what a good candidate for swapping is).</li><li> The associated value is transfered to disk, in a blocking way.</li><li> The key storage field is set to REDIS_VM_SWAPPED, while the <i>vm</i> fields of the object are set to the right values (the page index where the object was swapped, and the number of pages used to swap it).</li><li> Finally the value object is freed and the value entry of the hash table is set to NULL.</li></ul>The function is called again and again until one of the following happens: there is no way to swap more objects because either the swap file is full or nearly all the objects are already transfered on disk, or simply the memory usage is already under the vm-max-memory parameter.
<h2><a name="What values to swap when we are out of memory?">What values to swap when we are out of memory?</a></h2>
Understanding what's a good candidate for swapping is not too hard. A few objects at random are sampled, and for each their <i>swappability</i> is commuted as:
<pre class="codeblock python python python python python python" name="code">
swappability = age*log(size_in_memory)
</pre>
The age is the number of seconds the key was not requested, while size_in_memory is a fast estimation of the amount of memory (in bytes) used by the object in memory. So we try to swap out objects that are rarely accessed, and we try to swap bigger objects over smaller one, but the latter is a less important factor (because of the logarithmic function used). This is because we don't want bigger objects to be swapped out and in too often as the bigger the object the more I/O and CPU is required in order to transfer it.
<h2><a name="Blocking VM loading">Blocking VM loading</a></h2>
What happens if an operation against a key associated with a swapped out object is requested? For instance Redis may just happen to process the following command:
<pre class="codeblock python python python python python python python" name="code">
GET foo
</pre>
If the value object of the <code name="code" class="python">foo</code> key is swapped we need to load it back in memory before processing the operation. In Redis the key lookup process is centralized in the <code name="code" class="python">lookupKeyRead</code> and <code name="code" class="python">lookupKeyWrite</code> functions, this two functions are used in the implementation of all the Redis commands accessing the keyspace, so we have a single point in the code where to handle the loading of the key from the swap file to memory.<br/><br/>So this is what happens:
<ul><li> The user calls some command having as argumenet a swapped key</li><li> The command implementation calls the lookup function</li><li> The lookup function search for the key in the top level hash table. If the value associated with the requested key is swapped (we can see that checking the <i>storage</i> field of the key object), we load it back in memory in a blocking way before to return to the user.</li></ul>
This is pretty straightforward, but things will get more <i>interesting</i> with the threads. From the point of view of the blocking VM the only real problem is the saving of the dataset using another process, that is, handling BGSAVE and BGREWRITEAOF commands.
<h2><a name="Background saving when VM is active">Background saving when VM is active</a></h2>
The default Redis way to persist on disk is to create .rdb files using a child process. Redis calls the fork() system call in order to create a child, that has the exact copy of the in memory dataset, since fork duplicates the whole program memory space (actually thanks to a technique called Copy on Write memory pages are shared between the parent and child process, so the fork() call will not require too much memory).<br/><br/>In the child process we have a copy of the dataset in a given point in the time. Other commands issued by clients will just be served by the parent process and will not modify the child data.<br/><br/>The child process will just store the whole dataset into the dump.rdb file and finally will exit. But what happens when the VM is active? Values can be swapped out so we don't have all the data in memory, and we need to access the swap file in order to retrieve the swapped values. While child process is saving the swap file is shared between the parent and child process, since:
<ul><li> The parent process needs to access the swap file in order to load values back into memory if an operation against swapped out values are performed.</li><li> The child process needs to access the swap file in order to retrieve the full dataset while saving the data set on disk.</li></ul>In order to avoid problems while both the processes are accessing the same swap file we do a simple thing, that is, not allowing values to be swapped out in the parent process while a background saving is in progress. This way both the processes will access the swap file in read only. This approach has the problem that while the child process is saving no new values can be transfered on the swap file even if Redis is using more memory than the max memory parameters dictates. This is usually not a problem as the background saving will terminate in a short amount of time and if still needed a percentage of values will be swapped on disk ASAP.<br/><br/>An alternative to this scenario is to enable the Append Only File that will have this problem only when a log rewrite is performed using the BGREWRITEAOF command.
<h2><a name="The problem with the blocking VM">The problem with the blocking VM</a></h2>
The problem of blocking VM is that... it's blocking :)
This is not a problem when Redis is used in batch processing activities, but for real-time usage one of the good points of Redis is the low latency. The blocking VM will have bad latency behaviors as when a client is accessing a swapped out value, or when Redis needs to swap out values, no other clients will be served in the meantime.<br/><br/>Swapping out keys should happen in background. Similarly when a client is accessing a swapped out value other clients accessing in memory values should be served mostly as fast as when VM is disabled. Only the clients dealing with swapped out keys should be delayed.<br/><br/>All this limitations called for a non-blocking VM implementation.
<h2><a name="Threaded VM">Threaded VM</a></h2>
There are basically three main ways to turn the blocking VM into a non blocking one.
<ul><li> 1: One way is obvious, and in my opionion, not a good idea at all, that is, turning Redis itself into a theaded server: if every request is served by a different thread automatically other clients don't need to wait for blocked ones. Redis is fast, exports atomic operations, has no locks, and is just 10k lines of code, <b>because</b> it is single threaded, so this was not an option for me.</li><li> 2: Using non-blocking I/O against the swap file. After all you can think Redis already event-loop based, why don't just handle disk I/O in a non-blocking fashion? I also discarded this possiblity because of two main reasons. One is that non blocking file operations, unlike sockets, are an incompatibility nightmare. It's not just like calling select, you need to use OS-specific things. The other problem is that the I/O is just one part of the time consumed to handle VM, another big part is the CPU used in order to encode/decode data to/from the swap file. This is I picked option three, that is...</li><li> 3: Using I/O threads, that is, a pool of threads handling the swap I/O operations. This is what the Redis VM is using, so let's detail how this works.</li></ul><h2><a name="I/O Threads">I/O Threads</a></h2>
The threaded VM design goals where the following, in order of importance:
<ul><li> Simple implementation, little room for race condtions, simple locking, VM system more or less completeley decoupled from the rest of Redis code.</li><li> Good performances, no locks for clients accessing values in memory.</li><li> Ability to decode/encode objects in the I/O threads.</li></ul>The above goals resulted in an implementation where the Redis main thread (the one serving actual clients) and the I/O threads communicate using a queue of jobs, with a single mutex.
Basically when main thread requires some work done in the background by some I/O thread, it pushes an I/O job structure in the <code name="code" class="python">server.io_newjobs</code> queue (that is, just a linked list). If there are no active I/O threads, one is started. At this point some I/O thread will process the I/O job, and the result of the processing is pushed in the <code name="code" class="python">server.io_processed</code> queue. The I/O thread will send a byte using an UNIX pipe to the main thread in order to signal that a new job was processed and the result is ready to be processed.<br/><br/>This is how the <code name="code" class="python">iojob</code> structure looks like:
<pre class="codeblock python python python python python python python python" name="code">
typedef struct iojob {
int type; /* Request type, REDIS_IOJOB_* */
redisDb *db;/* Redis database */
robj *key; /* This I/O request is about swapping this key */
robj *val; /* the value to swap for REDIS_IOREQ_*_SWAP, otherwise this
* field is populated by the I/O thread for REDIS_IOREQ_LOAD. */
off_t page; /* Swap page where to read/write the object */
off_t pages; /* Swap pages needed to save object. PREPARE_SWAP return val */
int canceled; /* True if this command was canceled by blocking side of VM */
pthread_t thread; /* ID of the thread processing this entry */
} iojob;
</pre>
There are just three type of jobs that an I/O thread can perform (the type is specified by the <code name="code" class="python">type</code> field of the structure):
<ul><li> REDIS_IOJOB_LOAD: load the value associated to a given key from swap to memory. The object offset inside the swap file is <code name="code" class="python">page</code>, the object type is <code name="code" class="python">key-&gt;vtype</code>. The result of this operation will populate the <code name="code" class="python">val</code> field of the structure.</li><li> REDIS_IOJOB_PREPARE_SWAP: compute the number of pages needed in order to save the object pointed by <code name="code" class="python">val</code> into the swap. The result of this operation will populate the <code name="code" class="python">pages</code> field.</li><li> REDIS_IOJOB_DO_SWAP: Transfer the object pointed by <code name="code" class="python">val</code> to the swap file, at page offset <code name="code" class="python">page</code>.</li></ul>The main thread delegates just the above three tasks. All the rest is handled by the main thread itself, for instance finding a suitable range of free pages in the swap file page table (that is a fast operation), deciding what object to swap, altering the storage field of a Redis object to reflect the current state of a value.
<h2><a name="Non blocking VM as probabilistic enhancement of blocking VM">Non blocking VM as probabilistic enhancement of blocking VM</a></h2>
So now we have a way to request background jobs dealing with slow VM operations. How to add this to the mix of the rest of the work done by the main thread? While blocking VM was aware that an object was swapped out just when the object was looked up, this is too late for us: in C it is not trivial to start a background job in the middle of the command, leave the function, and re-enter in the same point the computation when the I/O thread finished what we requested (that is, no co-routines or continuations or alike).<br/><br/>Fortunately there was a much, much simpler way to do this. And we love simple things: basically consider the VM implementation a blocking one, but add an optimization (using non the no blocking VM operations we are able to perform) to make the blocking <b>very</b> unlikely.<br/><br/>This is what we do:
<ul><li> Every time a client sends us a command, <b>before</b> the command is executed, we examine the argument vector of the command in search for swapped keys. After all we know for every command what arguments are keys, as the Redis command format is pretty simple.</li><li> If we detect that at least a key in the requested command is swapped on disk, we block the client instead of really issuing the command. For every swapped value associated to a requested key, an I/O job is created, in order to bring the values back in memory. The main thread continues the execution of the event loop, without caring about the blocked client.</li><li> In the meanwhile, I/O threads are loading values in memory. Every time an I/O thread finished loading a value, it sends a byte to the main thread using an UNIX pipe. The pipe file descriptor has a readable event associated in the main thread event loop, that is the function <code name="code" class="python">vmThreadedIOCompletedJob</code>. If this function detects that all the values needed for a blocked client were loaded, the client is restarted and the original command called.</li></ul>
So you can think at this as a blocked VM that almost always happen to have the right keys in memory, since we pause clients that are going to issue commands about swapped out values until this values are loaded.<br/><br/>If the function checking what argument is a key fails in some way, there is no problem: the lookup function will see that a given key is associated to a swapped out value and will block loading it. So our non blocking VM reverts to a blocking one when it is not possible to anticipate what keys are touched.<br/><br/>For instance in the case of the SORT command used together with the GET or BY options, it is not trivial to know beforehand what keys will be requested, so at least in the first implementation, SORT BY/GET resorts to the blocking VM implementation.
<h2><a name="Blocking clients on swapped keys">Blocking clients on swapped keys</a></h2>
How to block clients? To suspend a client in an event-loop based server is pretty trivial. All we do is cancelling its read handler. Sometimes we do something different (for instance for BLPOP) that is just marking the client as blocked, but not processing new data (just accumulating the new data into input buffers).
<h2><a name="Aborting I/O jobs">Aborting I/O jobs</a></h2>
There is something hard to solve about the interactions between our blocking and non blocking VM, that is, what happens if a blocking operation starts about a key that is also &quot;interested&quot; by a non blocking operation at the same time?<br/><br/>For instance while SORT BY is executed, a few keys are being loaded in a blocking manner by the sort command. At the same time, another client may request the same keys with a simple <i>GET key</i> command, that will trigger the creation of an I/O job to load the key in background.<br/><br/>The only simple way to deal with this problem is to be able to kill I/O jobs in the main thread, so that if a key that we want to load or swap in a blocking way is in the REDIS_VM_LOADING or REDIS_VM_SWAPPING state (that is, there is an I/O job about this key), we can just kill the I/O job about this key, and go ahead with the blocking operation we want to perform.<br/><br/>This is not as trivial as it is. In a given moment an I/O job can be in one of the following three queues:
<ul><li> server.io_newjobs: the job was already queued but no thread is handling it.</li><li> server.io_processing: the job is being processed by an I/O thread.</li><li> server.io_processed: the job was already processed.</li></ul>The function able to kill an I/O job is <code name="code" class="python">vmCancelThreadedIOJob</code>, and this is what it does:
<ul><li> If the job is in the newjobs queue, that's simple, removing the iojob structure from the queue is enough as no thread is still executing any operation.</li><li> If the job is in the processing queue, a thread is messing with our job (and possibly with the associated object!). The only thing we can do is waiting for the item to move to the next queue in a <b>blocking way</b>. Fortunately this condition happens very rarely so it's not a performance problem.</li><li> If the job is in the processed queue, we just mark it as <i>canceled</i> marking setting the <code name="code" class="python">canceled</code> field to 1 in the iojob structure. The function processing completed jobs will just ignored and free the job instead of really processing it.</li></ul>
<h2><a name="Questions?">Questions?</a></h2>
This document is in no way complete, the only way to get the whole picture is reading the source code, but it should be a good introduction in order to make the code review / understanding a lot simpler.<br/><br/>Something is not clear about this page? Please leave a comment and I'll try to address the issue possibly integrating the answer in this document.
</div>
</div>
</div>
</body>
</html>

67
doc/VirtualMemoryUserGuide.html Normal file
View File

@ -0,0 +1,67 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>VirtualMemoryUserGuide: Contents</b><br>&nbsp;&nbsp;<a href="#Virtual Memory User Guide">Virtual Memory User Guide</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Virtual Memory explained in simple words">Virtual Memory explained in simple words</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#When using Virtual Memory is a good idea">When using Virtual Memory is a good idea</a><br>&nbsp;&nbsp;<a href="#VM Configuration">VM Configuration</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#The vm-max-memory setting">The vm-max-memory setting</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Configuring the swap file">Configuring the swap file</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Threaded VM vs Blocking VM">Threaded VM vs Blocking VM</a><br>&nbsp;&nbsp;<a href="#Random things to know">Random things to know</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#A good place for the swap file">A good place for the swap file</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Monitoring the VM">Monitoring the VM</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Redis with VM enabled: better .rdb files or Append Only File?">Redis with VM enabled: better .rdb files or Append Only File?</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Using as little memory as possible">Using as little memory as possible</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#VM Stability">VM Stability</a>
</div>
<h1 class="wikiname">VirtualMemoryUserGuide</h1>
<div class="summary">
</div>
<div class="narrow">
&iuml;&raquo;&iquest;#sidebar <a href="RedisGuides.html">RedisGuides</a><h1><a name="Virtual Memory User Guide">Virtual Memory User Guide</a></h1>Redis Virtual Memory is a feature that will appear for the first time in a stable Redis distribution in Redis 2.0. However Virtual Memory (called VM starting from now) is already available and stable enough to be tests in the unstable branch of Redis available <a href="http://github.com/antirez/redis" target="_blank">on Git</a>.<h2><a name="Virtual Memory explained in simple words">Virtual Memory explained in simple words</a></h2>Redis follows a Key-Value model. You have keys associated with some values.
Usually Redis takes both Keys and associated Values in memory. Sometimes this is not the best option, and while Keys <b>must</b> be taken in memory for the way Redis is designed (and in order to ensure fast lookups), Values can be swapped out to disk when they are rarely used.<br/><br/>In practical terms this means that if you have a dataset of 100,000 keys in memory, but only 10% of this keys are often used, Redis with Virtual Memory enabled will try to transfer the values associated to the rarely used keys on disk.<br/><br/>When this values are requested, as a result of a command issued by a client, the values are loaded back from the swap file to the main memory.<h2><a name="When using Virtual Memory is a good idea">When using Virtual Memory is a good idea</a></h2>Before using VM you should ask yourself if you really need it. Redis is a disk backed, in memory database. The right way to use Redis is almost always to have enough RAM to fit all the data in memory. Still there are a scenarios where to hold all the data in memory is not possible:
<ul><li> Data access is very biased. Only a small percentage of keys (for instance related to active users in your web site) gets the vast majority of accesses. At the same time there is too much data per key to take everything in memory.</li><li> There is simply not enough memory available to hold all the data in memory, regardless of the data access pattern, and values are large. In this configuration Redis can be used as an on-disk DB where keys are in memory, so the key lookup is fast, but the access to the actual values require accessing the (slower) disk.</li></ul>
An important concept to take in mind is that Redis <b>is not able to swap the keys</b>, so if your memory problems are related to the fact you have too much keys with very small values, VM is not the solution.<br/><br/>Instead if a good amount of memory is used because values are pretty large (for example large strings, lists, sets or hashes with many elements), then VM can be a good idea.<br/><br/>Sometimes you can turn your &quot;many keys with small values&quot; problem into a &quot;less keys but with very large values&quot; one just using Hashes in order to group related data into fields of a single key. For instance instead of having a key for every attribute of your object you have a single key per object where Hash fields represent the different attributes.<h1><a name="VM Configuration">VM Configuration</a></h1>Configuring the VM is not hard but requires some care to set the best parameters accordingly to the requirements.<br/><br/>The VM is enabled and configured editing redis.conf, the first step is switching it on with:<br/><br/><pre class="codeblock python" name="code">
vm-enabled yes
</pre>Many other configuration options are able to change the behavior of VM. The rule is that you don't want to run with the default configuration, as every problem and dataset requires some tuning in order to get the maximum advantages.<h2><a name="The vm-max-memory setting">The vm-max-memory setting</a></h2>The <b>vm-max-memory</b> setting specifies how much memory Redis is free to use before starting swapping values on disk.<br/><br/>Basically if this memory limit is still not reached, no object will be swapped, Redis will work all in memory as usually. Once this limit is hit, enough objects are swapped out in order to return just under the limit.<br/><br/>The swapped objects are the one with the highest &quot;age&quot; (that is, the number of seconds since they are not used in any way) mainly, but the &quot;swappability&quot; of an object is also proportional to the logarithm of it's size in memory. So basically older objects are preferred, but when they are about the same size, bigger objects are preferred.<br/><br/><b>WARNING:</b> Because keys can't be swapped out, Redis will not be able to honour the <b>vm-max-memory</b> setting if the keys alone are using more space than the limit.<br/><br/>The best value for this setting is enough RAM in order to hold the &quot;working set&quot; of data. In practical terms, just give Redis as much memory as you can, and swapping will work better.<h2><a name="Configuring the swap file">Configuring the swap file</a></h2>In order to transfer data from memory to disk, Redis uses a swap file. The swap file has nothing to do with durability of data, and can be removed when a Redis instance is terminated. Still the swap file should not be moved, deleted, or altered in any other way while Redis is running.<br/><br/>Because the Redis swap file is used mostly in a random access fashion, to put the swap file into a Solid State Disk will lead to better performances.<br/><br/>The swap file is divided into &quot;pages&quot;. A value can be swapped into one or multiple pages, but a single page can't hold more than a value.<br/><br/>There is no direct way to tell Redis how much bytes of swap file it should be using. Instead two different values are configured, that multiplied together will produce the total number of bytes used. This two values are the number of pages inside the swap file, and the page size. It is possible to configure this two parameters in redis.conf.<br/><br/><ul><li> The <b>vm-pages</b> configuration directive is used to set the total number of pages in the swap file.</li><li> the <b>vm-page-size</b> configuration directive is used in order to set the page size in bytes.</li></ul>
So for instance if the page size is set to the value of 32 bytes, and the total number of pages is set to 10000000 (10 millions), the swap file can hold a total of 320 MB of data.<br/><br/>Because a single page can't be used to hold more than a value (but a value can be stored into multiple pages), care must be taken in setting this parameters.
Usually the best idea is setting the page size so that the majority of the values can be swapped using a few pages.<h2><a name="Threaded VM vs Blocking VM">Threaded VM vs Blocking VM</a></h2>Another very important configuration parameter is <b>vm-max-threads</b>:<br/><br/><pre class="codeblock python python" name="code">
# The default vm-max-threads configuration
vm-max-threads 4
</pre>This is the maximum number of threads used in order to perform I/O from/to the swap file. A good value is just to match the number of cores in your system.<br/><br/>However the special value of &quot;0&quot; will enable blocking VM. When VM is configured to be blocking it performs the I/O in a synchronous blocking way. This is what you can expect from blocking VM:
<ul><li> Clients accessing swapped out keys will block other clients while reading from disk, so the latency experimented by clients can be larger, especially if the disk is slow or busy and/or if there are big values swapped on disk.</li><li> The blocking VM performances are <b>overall</b> better, as there is no time lost in synchronization, spawning of threads, resuming blocked clients waiting for values.</li></ul>So if you are willing to accept an higher latency from time to time, blocking VM can be a good pick, especially if swapping happens rarely as most of your often accessed data happens to fit in your memory.<br/><br/>If instead you have a lot of swap in and swap out operations and you have many cores that you want to exploit, and in general when you don't want that clients dealing with swapped values will block other clients for a few milliseconds (or more if the swapped value is very big), then it's better to use threaded VM.<br/><br/>To experiment with your dataset and different configurations is warmly encouraged...<h1><a name="Random things to know">Random things to know</a></h1>
<h2><a name="A good place for the swap file">A good place for the swap file</a></h2>In many configurations the swap file can be fairly large, even 40GB or more.
Not all the kind of file systems are able to deal with large files in a good way, especially Mac OS X file system tends to be really lame about it.<br/><br/>The suggestion is to use Linux ext3 file system, or any other file system with good support for <b>sparse files</b>. What are sparse files?<br/><br/>Sparse files are files where a lot of the content happen to be empty. Advanced file systems like ext2, ext3, ext4, RaiserFS, Raiser4, and many others, are able to encode this files in a more efficient way and will allocate more space for the file when needed, that is, when more actual blocks of the file will be used.<br/><br/>The swap file is obviously pretty sparse, especially if the server is running since little time or it is much bigger compared to the amount of data swapped out. A file system not supporting sparse files can at some point block the Redis process while creating a very big file at once.<br/><br/>For a list of file systems supporting spare files <a href="http://en.wikipedia.org/wiki/Comparison_of_file_systems" target="_blank">check this Wikipedia page comparing different files systems</a>.<h2><a name="Monitoring the VM">Monitoring the VM</a></h2>Once you have a Redis system with VM enabled up and running, you may be very interested in knowing how it's working: how many objects are swapped in total, the number of objects swapped and loaded every second, and so forth.<br/><br/>There is an utility that is very handy in checking how the VM is working, that is part of <a href="http://code.google.com/p/redis-tools" target="_blank">Redis Tools</a>. This tool is called redis-stat, and using it is pretty straightforward:<br/><br/><pre class="codeblock python python python" name="code">
$ ./redis-stat vmstat
--------------- objects --------------- ------ pages ------ ----- memory -----
load-in swap-out swapped delta used delta used delta
138837 1078936 800402 +800402 807620 +807620 209.50M +209.50M
4277 38011 829802 +29400 837441 +29821 206.47M -3.03M
3347 39508 862619 +32817 870340 +32899 202.96M -3.51M
4445 36943 890646 +28027 897925 +27585 199.92M -3.04M
10391 16902 886783 -3863 894104 -3821 200.22M +309.56K
8888 19507 888371 +1588 895678 +1574 200.05M -171.81K
8377 20082 891664 +3293 899850 +4172 200.10M +53.55K
9671 20210 892586 +922 899917 +67 199.82M -285.30K
10861 16723 887638 -4948 895003 -4914 200.13M +312.35K
9541 21945 890618 +2980 898004 +3001 199.94M -197.11K
9689 17257 888345 -2273 896405 -1599 200.27M +337.77K
10087 18784 886771 -1574 894577 -1828 200.36M +91.60K
9330 19350 887411 +640 894817 +240 200.17M -189.72K
</pre>The above output is about a redis-server with VM enable, around 1 million of keys inside, and a lot of simulated load using the redis-load utility.<br/><br/>As you can see from the output a number of load-in and swap-out operations are happening every second. Note that the first line reports the actual values since the server was started, while the next lines are differences compared to the previous reading.<br/><br/>If you assigned enough memory to hold your working set of data, probably you should see a lot less dramatic swapping happening, so redis-stat can be a really valuable tool in order to understand if you need to shop for RAM ;)<h2><a name="Redis with VM enabled: better .rdb files or Append Only File?">Redis with VM enabled: better .rdb files or Append Only File?</a></h2>When VM is enabled, saving and loading the database are <b>much slower</b> operations. A DB that usually loads in 2 seconds takes 13 seconds with VM enabled if the server is configured to use the smallest memory possible (that is, vm-max-memory set to 0).<br/><br/>So you probably want to switch to a configuration using the Append Only File for persistence, so that you can perform the BGREWRITEAOF from time to time.<br/><br/>It is important to note that while a BGSAVE or BGREWRITEAOF is in progress Redis does <b>not</b> swap new values on disk. The VM will be read-only while there is another child accessing it. So if you have a lot of writes while there is a child working, the memory usage may grow.<h2><a name="Using as little memory as possible">Using as little memory as possible</a></h2>An interesting setup to turn Redis into an on-disk DB with just keys in memory is setting vm-max-memory to 0. If you don't mind some latency more and poorer performances but want to use very little memory for very big values, this is a good setup.<br/><br/>In this setup you should first try setting the VM as blocking (vm-max-threads 0) as with this configuration and high traffic the number of swap in and swap out operations will be huge, and threading will consume a lot of resources compared to a simple blocking implementation.<h2><a name="VM Stability">VM Stability</a></h2>VM is still experimental code, but in the latest weeks it was tested in many ways in development environments, and even in some production environment. No bugs were noticed during this testing period. Still the more obscure bugs may happen in non controlled environments where there are setups that we are not able to reproduce for some reason.<br/><br/>In this stage you are encouraged to try VM in your development environment, and even in production if your DB is not mission critical, but for instance just a big persistent cache of data that may go away without too much problems.<br/><br/>Please report any problem you will notice to the Redis Google Group or by IRC joining the #redis IRC channel on freenode.
</div>
</div>
</div>
</body>
</html>

42
doc/ZrankCommand.html Normal file
View File

@ -0,0 +1,42 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>ZrankCommand: Contents</b><br>&nbsp;&nbsp;<a href="#ZRANK _key_ _member_ (Redis &gt;">ZRANK _key_ _member_ (Redis &gt;</a><br>&nbsp;&nbsp;<a href="#ZREVRANK _key_ _member_ (Redis &gt;">ZREVRANK _key_ _member_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">ZrankCommand</h1>
<div class="summary">
</div>
<div class="narrow">
<h1><a name="ZRANK _key_ _member_ (Redis &gt;">ZRANK _key_ _member_ (Redis &gt;</a></h1> 1.3.4) =
<h1><a name="ZREVRANK _key_ _member_ (Redis &gt;">ZREVRANK _key_ _member_ (Redis &gt;</a></h1> 1.3.4) =
<i>Time complexity: O(log(N))</i><blockquote>ZRANK returns the rank of the member in the sorted set, with scores ordered from low to high. ZREVRANK returns the rank with scores ordered from high to low. When the given member does not exist in the sorted set, the special value 'nil' is returned. The returned rank (or index) of the member is 0-based for both commands.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Bulk reply</a>, specifically:<br/><br/><pre class="codeblock python" name="code">
the rank (an integer number) represented as an string.
</pre>
</div>
</div>
</div>
</body>
</html>

39
doc/ZremrangebyrankCommand.html Normal file
View File

@ -0,0 +1,39 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>ZremrangebyrankCommand: Contents</b><br>&nbsp;&nbsp;<a href="#ZREMRANGEBYRANK _key_ _start_ _end_ (Redis &gt;">ZREMRANGEBYRANK _key_ _start_ _end_ (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">ZremrangebyrankCommand</h1>
<div class="summary">
</div>
<div class="narrow">
<h1><a name="ZREMRANGEBYRANK _key_ _start_ _end_ (Redis &gt;">ZREMRANGEBYRANK _key_ _start_ _end_ (Redis &gt;</a></h1> 1.3.4) =
<i>Time complexity: O(log(N))+O(M) with N being the number of elements in the sorted set and M the number of elements removed by the operation</i><blockquote>Remove all elements in the sorted set at <i>key</i> with rank between <i>start</i> and <i>end</i>. Start and end are 0-based with rank 0 being the element with the lowest score. Both start and end can be negative numbers, where they indicate offsets starting at the element with the highest rank. For example: -1 is the element with the highest score, -2 the element with the second highest score and so forth.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Integer reply</a>, specifically the number of elements removed.
</div>
</div>
</div>
</body>
</html>

41
doc/ZunionCommand.html Normal file
View File

@ -0,0 +1,41 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>ZunionCommand: Contents</b><br>&nbsp;&nbsp;<a href="#ZUNION / ZINTER _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis &gt;">ZUNION / ZINTER _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">ZunionCommand</h1>
<div class="summary">
</div>
<div class="narrow">
<h1><a name="ZUNION / ZINTER _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis &gt;">ZUNION / ZINTER _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis &gt;</a></h1> 1.3.5) =<br/><br/><i>Time complexity: O(N) + O(M log(M)) with N being the sum of the sizes of the input sorted sets, and M being the number of elements in the resulting sorted set</i><blockquote>Creates a union or intersection of <i>N</i> sorted sets given by keys <i>k1</i> through <i>kN</i>, and stores it at <i>dstkey</i>. It is mandatory to provide the number of input keys <i>N</i>, before passing the input keys and the other (optional) arguments.</blockquote>
<blockquote>As the terms imply, the ZINTER command requires an element to be present in each of the given inputs to be inserted in the result. The ZUNION command inserts all elements across all inputs.</blockquote>
<blockquote>Using the WEIGHTS option, it is possible to add weight to each input sorted set. This means that the score of each element in the sorted set is first multiplied by this weight before being passed to the aggregation. When this option is not given, all weights default to 1.</blockquote>
<blockquote>With the AGGREGATE option, it's possible to specify how the results of the union or intersection are aggregated. This option defaults to SUM, where the score of an element is summed across the inputs where it exists. When this option is set to be either MIN or MAX, the resulting set will contain the minimum or maximum score of an element across the inputs where it exists.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Integer reply</a>, specifically the number of elements in the sorted set at <i>dstkey</i>.
</div>
</div>
</div>
</body>
</html>

41
doc/ZunionstoreCommand.html Normal file
View File

@ -0,0 +1,41 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<link type="text/css" rel="stylesheet" href="style.css" />
</head>
<body>
<div id="page">
<div id='header'>
<a href="index.html">
<img style="border:none" alt="Redis Documentation" src="redis.png">
</a>
</div>
<div id="pagecontent">
<div class="index">
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>ZunionstoreCommand: Contents</b><br>&nbsp;&nbsp;<a href="#ZUNION / ZINTER _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis &gt;">ZUNION / ZINTER _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis &gt;</a><br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="#Return value">Return value</a>
</div>
<h1 class="wikiname">ZunionstoreCommand</h1>
<div class="summary">
</div>
<div class="narrow">
<h1><a name="ZUNION / ZINTER _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis &gt;">ZUNION / ZINTER _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis &gt;</a></h1> 1.3.5) =<br/><br/><i>Time complexity: O(N) + O(M log(M)) with N being the sum of the sizes of the input sorted sets, and M being the number of elements in the resulting sorted set</i><blockquote>Creates a union or intersection of <i>N</i> sorted sets given by keys <i>k1</i> through <i>kN</i>, and stores it at <i>dstkey</i>. It is mandatory to provide the number of input keys <i>N</i>, before passing the input keys and the other (optional) arguments.</blockquote>
<blockquote>As the terms imply, the ZINTER command requires an element to be present in each of the given inputs to be inserted in the result. The ZUNION command inserts all elements across all inputs.</blockquote>
<blockquote>Using the WEIGHTS option, it is possible to add weight to each input sorted set. This means that the score of each element in the sorted set is first multiplied by this weight before being passed to the aggregation. When this option is not given, all weights default to 1.</blockquote>
<blockquote>With the AGGREGATE option, it's possible to specify how the results of the union or intersection are aggregated. This option defaults to SUM, where the score of an element is summed across the inputs where it exists. When this option is set to be either MIN or MAX, the resulting set will contain the minimum or maximum score of an element across the inputs where it exists.</blockquote>
<h2><a name="Return value">Return value</a></h2><a href="ReplyTypes.html">Integer reply</a>, specifically the number of elements in the sorted set at <i>dstkey</i>.
</div>
</div>
</div>
</body>
</html>

9
mkreleasehdr.sh Executable file
View File

@ -0,0 +1,9 @@
#!/bin/sh
GIT_SHA1=$( (git show-ref --head --hash=8 2> /dev/null || echo 00000000) | head -n1)
GIT_DIRTY=$(git diff 2> /dev/null | wc -l)
test -f release.h || touch release.h
(cat release.h | grep SHA1 | grep $GIT_SHA1) && \
(cat release.h | grep DIRTY | grep $GIT_DIRTY) && exit 0 # Already uptodate
echo "#define REDIS_GIT_SHA1 \"$GIT_SHA1\"" > release.h
echo "#define REDIS_GIT_DIRTY \"$GIT_DIRTY\"" >> release.h
touch redis.c # force recompile of redis.c

13
redis-cli.c
View File

@ -54,6 +54,7 @@ static struct config {
long repeat;
int dbnum;
int interactive;
int shutdown;
int monitor_mode;
int pubsub_mode;
int raw_output;
@ -313,7 +314,10 @@ static int cliReadMultiBulkReply(int fd) {
static int cliReadReply(int fd) {
char type;
if (anetRead(fd,&type,1) <= 0) exit(1);
if (anetRead(fd,&type,1) <= 0) {
if (config.shutdown) return 0;
exit(1);
}
switch(type) {
case '-':
printf("(error) ");
@ -356,7 +360,6 @@ static int selectDb(int fd) {
static int cliSendCommand(int argc, char **argv, int repeat) {
struct redisCommand *rc = lookupCommand(argv[0]);
int shutdown = 0;
int fd, j, retval = 0;
sds cmd;
@ -372,7 +375,7 @@ static int cliSendCommand(int argc, char **argv, int repeat) {
return 1;
}
if (!strcasecmp(rc->name,"shutdown")) shutdown = 1;
if (!strcasecmp(rc->name,"shutdown")) config.shutdown = 1;
if (!strcasecmp(rc->name,"monitor")) config.monitor_mode = 1;
if (!strcasecmp(rc->name,"subscribe") ||
!strcasecmp(rc->name,"psubscribe")) config.pubsub_mode = 1;
@ -410,8 +413,9 @@ static int cliSendCommand(int argc, char **argv, int repeat) {
}
retval = cliReadReply(fd);
if (retval) {
return shutdown ? 0 : retval;
return retval;
}
}
return 0;
@ -594,6 +598,7 @@ int main(int argc, char **argv) {
config.hostport = 6379;
config.repeat = 1;
config.dbnum = 0;
config.shutdown = 0;
config.interactive = 0;
config.monitor_mode = 0;
config.pubsub_mode = 0;

159
redis.c
View File

@ -27,7 +27,7 @@
* POSSIBILITY OF SUCH DAMAGE.
*/
#define REDIS_VERSION "1.3.10"
#define REDIS_VERSION "1.3.12"
#include "fmacros.h"
#include "config.h"
@ -76,6 +76,7 @@
#include "pqsort.h" /* Partial qsort for SORT+LIMIT */
#include "zipmap.h" /* Compact dictionary-alike data structure */
#include "sha1.h" /* SHA1 is used for DEBUG DIGEST */
#include "release.h" /* Release and/or git repository information */
/* Error codes */
#define REDIS_OK 0
@ -2760,21 +2761,6 @@ static void addReplyDouble(redisClient *c, double d) {
(unsigned long) strlen(buf),buf));
}
static void addReplyLong(redisClient *c, long l) {
char buf[128];
size_t len;
if (l == 0) {
addReply(c,shared.czero);
return;
} else if (l == 1) {
addReply(c,shared.cone);
return;
}
len = snprintf(buf,sizeof(buf),":%ld\r\n",l);
addReplySds(c,sdsnewlen(buf,len));
}
static void addReplyLongLong(redisClient *c, long long ll) {
char buf[128];
size_t len;
@ -2786,8 +2772,11 @@ static void addReplyLongLong(redisClient *c, long long ll) {
addReply(c,shared.cone);
return;
}
len = snprintf(buf,sizeof(buf),":%lld\r\n",ll);
addReplySds(c,sdsnewlen(buf,len));
buf[0] = ':';
len = ll2string(buf+1,sizeof(buf)-1,ll);
buf[len+1] = '\r';
buf[len+2] = '\n';
addReplySds(c,sdsnewlen(buf,len+3));
}
static void addReplyUlong(redisClient *c, unsigned long ul) {
@ -2806,7 +2795,8 @@ static void addReplyUlong(redisClient *c, unsigned long ul) {
}
static void addReplyBulkLen(redisClient *c, robj *obj) {
size_t len;
size_t len, intlen;
char buf[128];
if (obj->encoding == REDIS_ENCODING_RAW) {
len = sdslen(obj->ptr);
@ -2823,7 +2813,11 @@ static void addReplyBulkLen(redisClient *c, robj *obj) {
len++;
}
}
addReplySds(c,sdscatprintf(sdsempty(),"$%lu\r\n",(unsigned long)len));
buf[0] = '$';
intlen = ll2string(buf+1,sizeof(buf)-1,(long long)len);
buf[intlen+1] = '\r';
buf[intlen+2] = '\n';
addReplySds(c,sdsnewlen(buf,intlen+3));
}
static void addReplyBulk(redisClient *c, robj *obj) {
@ -4334,8 +4328,7 @@ static void incrDecrCommand(redisClient *c, long long incr) {
if (getLongLongFromObjectOrReply(c,o,&value,NULL) != REDIS_OK) return;
value += incr;
o = createObject(REDIS_STRING,sdscatprintf(sdsempty(),"%lld",value));
o = tryObjectEncoding(o);
o = createStringObjectFromLongLong(value);
retval = dictAdd(c->db->dict,c->argv[1],o);
if (retval == DICT_ERR) {
dictReplace(c->db->dict,c->argv[1],o);
@ -4465,7 +4458,7 @@ static void delCommand(redisClient *c) {
deleted++;
}
}
addReplyLong(c,deleted);
addReplyLongLong(c,deleted);
}
static void existsCommand(redisClient *c) {
@ -4750,7 +4743,7 @@ static void pushGenericCommand(redisClient *c, int where) {
incrRefCount(c->argv[2]);
}
server.dirty++;
addReplySds(c,sdscatprintf(sdsempty(),":%d\r\n",listLength(list)));
addReplyLongLong(c,listLength(list));
}
static void lpushCommand(redisClient *c) {
@ -5252,7 +5245,7 @@ static void sinterGenericCommand(redisClient *c, robj **setskeys, unsigned long
if (dictSize((dict*)dstset->ptr) > 0) {
dictAdd(c->db->dict,dstkey,dstset);
incrRefCount(dstkey);
addReplyLong(c,dictSize((dict*)dstset->ptr));
addReplyLongLong(c,dictSize((dict*)dstset->ptr));
} else {
decrRefCount(dstset);
addReply(c,shared.czero);
@ -5355,7 +5348,7 @@ static void sunionDiffGenericCommand(redisClient *c, robj **setskeys, int setsnu
if (dictSize((dict*)dstset->ptr) > 0) {
dictAdd(c->db->dict,dstkey,dstset);
incrRefCount(dstkey);
addReplyLong(c,dictSize((dict*)dstset->ptr));
addReplyLongLong(c,dictSize((dict*)dstset->ptr));
} else {
decrRefCount(dstset);
addReply(c,shared.czero);
@ -5834,7 +5827,7 @@ static void zremrangebyscoreCommand(redisClient *c) {
if (htNeedsResize(zs->dict)) dictResize(zs->dict);
if (dictSize(zs->dict) == 0) deleteKey(c->db,c->argv[1]);
server.dirty += deleted;
addReplyLong(c,deleted);
addReplyLongLong(c,deleted);
}
static void zremrangebyrankCommand(redisClient *c) {
@ -5872,7 +5865,7 @@ static void zremrangebyrankCommand(redisClient *c) {
if (htNeedsResize(zs->dict)) dictResize(zs->dict);
if (dictSize(zs->dict) == 0) deleteKey(c->db,c->argv[1]);
server.dirty += deleted;
addReplyLong(c, deleted);
addReplyLongLong(c, deleted);
}
typedef struct {
@ -6058,7 +6051,7 @@ static void zunionInterGenericCommand(redisClient *c, robj *dstkey, int op) {
if (dstzset->zsl->length) {
dictAdd(c->db->dict,dstkey,dstobj);
incrRefCount(dstkey);
addReplyLong(c, dstzset->zsl->length);
addReplyLongLong(c, dstzset->zsl->length);
server.dirty++;
} else {
decrRefCount(dstobj);
@ -6254,7 +6247,7 @@ static void genericZrangebyscoreCommand(redisClient *c, int justcount) {
if (limit > 0) limit--;
}
if (justcount) {
addReplyLong(c,(long)rangelen);
addReplyLongLong(c,(long)rangelen);
} else {
lenobj->ptr = sdscatprintf(sdsempty(),"*%lu\r\n",
withscores ? (rangelen*2) : rangelen);
@ -6324,9 +6317,9 @@ static void zrankGenericCommand(redisClient *c, int reverse) {
rank = zslGetRank(zsl, *score, c->argv[2]);
if (rank) {
if (reverse) {
addReplyLong(c, zsl->length - rank);
addReplyLongLong(c, zsl->length - rank);
} else {
addReplyLong(c, rank-1);
addReplyLongLong(c, rank-1);
}
} else {
addReply(c,shared.nullbulk);
@ -7183,6 +7176,8 @@ static sds genRedisInfoString(void) {
bytesToHuman(hmem,zmalloc_used_memory());
info = sdscatprintf(sdsempty(),
"redis_version:%s\r\n"
"redis_git_sha1:%s\r\n"
"redis_git_dirty:%d\r\n"
"arch_bits:%s\r\n"
"multiplexing_api:%s\r\n"
"process_id:%ld\r\n"
@ -7200,13 +7195,15 @@ static sds genRedisInfoString(void) {
"total_connections_received:%lld\r\n"
"total_commands_processed:%lld\r\n"
"expired_keys:%lld\r\n"
"hash_max_zipmap_entries:%ld\r\n"
"hash_max_zipmap_value:%ld\r\n"
"hash_max_zipmap_entries:%zu\r\n"
"hash_max_zipmap_value:%zu\r\n"
"pubsub_channels:%ld\r\n"
"pubsub_patterns:%u\r\n"
"vm_enabled:%d\r\n"
"role:%s\r\n"
,REDIS_VERSION,
REDIS_GIT_SHA1,
strtol(REDIS_GIT_DIRTY,NULL,10) > 0,
(sizeof(long) == 8) ? "64" : "32",
aeGetApiName(),
(long) getpid(),
@ -8727,6 +8724,48 @@ static void aofRemoveTempFile(pid_t childpid) {
* as a fully non-blocking VM.
*/
/* Called when the user switches from "appendonly yes" to "appendonly no"
* at runtime using the CONFIG command. */
static void stopAppendOnly(void) {
flushAppendOnlyFile();
fsync(server.appendfd);
close(server.appendfd);
server.appendfd = -1;
server.appendseldb = -1;
server.appendonly = 0;
/* rewrite operation in progress? kill it, wait child exit */
if (server.bgsavechildpid != -1) {
int statloc;
if (kill(server.bgsavechildpid,SIGKILL) != -1)
wait3(&statloc,0,NULL);
/* reset the buffer accumulating changes while the child saves */
sdsfree(server.bgrewritebuf);
server.bgrewritebuf = sdsempty();
server.bgsavechildpid = -1;
}
}
/* Called when the user switches from "appendonly no" to "appendonly yes"
* at runtime using the CONFIG command. */
static int startAppendOnly(void) {
server.appendonly = 1;
server.lastfsync = time(NULL);
server.appendfd = open(server.appendfilename,O_WRONLY|O_APPEND|O_CREAT,0644);
if (server.appendfd == -1) {
redisLog(REDIS_WARNING,"Used tried to switch on AOF via CONFIG, but I can't open the AOF file: %s",strerror(errno));
return REDIS_ERR;
}
if (rewriteAppendOnlyFileBackground() == REDIS_ERR) {
server.appendonly = 0;
close(server.appendfd);
redisLog(REDIS_WARNING,"Used tried to switch on AOF via CONFIG, I can't trigger a background AOF rewrite operation. Check the above logs for more info about the error.",strerror(errno));
return REDIS_ERR;
}
return REDIS_OK;
}
/* =================== Virtual Memory - Blocking Side ====================== */
static void vmInit(void) {
@ -9829,6 +9868,8 @@ static void handleClientsBlockedOnSwappedKey(redisDb *db, robj *key) {
static void configSetCommand(redisClient *c) {
robj *o = getDecodedObject(c->argv[3]);
long long ll;
if (!strcasecmp(c->argv[2]->ptr,"dbfilename")) {
zfree(server.dbfilename);
server.dbfilename = zstrdup(o->ptr);
@ -9839,7 +9880,13 @@ static void configSetCommand(redisClient *c) {
zfree(server.masterauth);
server.masterauth = zstrdup(o->ptr);
} else if (!strcasecmp(c->argv[2]->ptr,"maxmemory")) {
server.maxmemory = strtoll(o->ptr, NULL, 10);
if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
ll < 0) goto badfmt;
server.maxmemory = ll;
} else if (!strcasecmp(c->argv[2]->ptr,"timeout")) {
if (getLongLongFromObject(o,&ll) == REDIS_ERR ||
ll < 0 || ll > LONG_MAX) goto badfmt;
server.maxidletime = ll;
} else if (!strcasecmp(c->argv[2]->ptr,"appendfsync")) {
if (!strcasecmp(o->ptr,"no")) {
server.appendfsync = APPENDFSYNC_NO;
@ -9850,6 +9897,23 @@ static void configSetCommand(redisClient *c) {
} else {
goto badfmt;
}
} else if (!strcasecmp(c->argv[2]->ptr,"appendonly")) {
int old = server.appendonly;
int new = yesnotoi(o->ptr);
if (new == -1) goto badfmt;
if (old != new) {
if (new == 0) {
stopAppendOnly();
} else {
if (startAppendOnly() == REDIS_ERR) {
addReplySds(c,sdscatprintf(sdsempty(),
"-ERR Unable to turn on AOF. Check server logs.\r\n"));
decrRefCount(o);
return;
}
}
}
} else if (!strcasecmp(c->argv[2]->ptr,"save")) {
int vlen, j;
sds *v = sdssplitlen(o->ptr,sdslen(o->ptr)," ",1,&vlen);
@ -9930,11 +9994,24 @@ static void configGetCommand(redisClient *c) {
if (stringmatch(pattern,"maxmemory",0)) {
char buf[128];
snprintf(buf,128,"%llu\n",server.maxmemory);
ll2string(buf,128,server.maxmemory);
addReplyBulkCString(c,"maxmemory");
addReplyBulkCString(c,buf);
matches++;
}
if (stringmatch(pattern,"timeout",0)) {
char buf[128];
ll2string(buf,128,server.maxidletime);
addReplyBulkCString(c,"timeout");
addReplyBulkCString(c,buf);
matches++;
}
if (stringmatch(pattern,"appendonly",0)) {
addReplyBulkCString(c,"appendonly");
addReplyBulkCString(c,server.appendonly ? "yes" : "no");
matches++;
}
if (stringmatch(pattern,"appendfsync",0)) {
char *policy;
@ -10036,7 +10113,7 @@ static int pubsubSubscribeChannel(redisClient *c, robj *channel) {
addReply(c,shared.mbulk3);
addReply(c,shared.subscribebulk);
addReplyBulk(c,channel);
addReplyLong(c,dictSize(c->pubsub_channels)+listLength(c->pubsub_patterns));
addReplyLongLong(c,dictSize(c->pubsub_channels)+listLength(c->pubsub_patterns));
return retval;
}
@ -10072,7 +10149,7 @@ static int pubsubUnsubscribeChannel(redisClient *c, robj *channel, int notify) {
addReply(c,shared.mbulk3);
addReply(c,shared.unsubscribebulk);
addReplyBulk(c,channel);
addReplyLong(c,dictSize(c->pubsub_channels)+
addReplyLongLong(c,dictSize(c->pubsub_channels)+
listLength(c->pubsub_patterns));
}
@ -10098,7 +10175,7 @@ static int pubsubSubscribePattern(redisClient *c, robj *pattern) {
addReply(c,shared.mbulk3);
addReply(c,shared.psubscribebulk);
addReplyBulk(c,pattern);
addReplyLong(c,dictSize(c->pubsub_channels)+listLength(c->pubsub_patterns));
addReplyLongLong(c,dictSize(c->pubsub_channels)+listLength(c->pubsub_patterns));
return retval;
}
@ -10123,7 +10200,7 @@ static int pubsubUnsubscribePattern(redisClient *c, robj *pattern, int notify) {
addReply(c,shared.mbulk3);
addReply(c,shared.punsubscribebulk);
addReplyBulk(c,pattern);
addReplyLong(c,dictSize(c->pubsub_channels)+
addReplyLongLong(c,dictSize(c->pubsub_channels)+
listLength(c->pubsub_patterns));
}
decrRefCount(pattern);
@ -10251,7 +10328,7 @@ static void punsubscribeCommand(redisClient *c) {
static void publishCommand(redisClient *c) {
int receivers = pubsubPublishMessage(c->argv[1],c->argv[2]);
addReplyLong(c,receivers);
addReplyLongLong(c,receivers);
}
/* ================================= Debugging ============================== */

4
sha1.c
View File

@ -30,7 +30,7 @@ A million repetitions of "a"
#if (BSD >= 199103)
# include <machine/endian.h>
#else
#ifdef linux
#if defined(linux) || defined(__linux__)
# include <endian.h>
#else
#define LITTLE_ENDIAN 1234 /* least-significant byte first (vax, pc) */
@ -49,7 +49,7 @@ A million repetitions of "a"
defined(apollo) || defined(__convex__) || defined(_CRAY) || \
defined(__hppa) || defined(__hp9000) || \
defined(__hp9000s300) || defined(__hp9000s700) || \
defined (BIT_ZERO_ON_LEFT) || defined(m68k)
defined (BIT_ZERO_ON_LEFT) || defined(m68k) || defined(__sparc)
#define BYTE_ORDER BIG_ENDIAN
#endif
#endif /* linux */

2
solarisfixes.h
View File

@ -15,4 +15,6 @@
#define isinf(x) \
__extension__ ({ __typeof (x) __x_i = (x); \
__builtin_expect(!isnan(__x_i) && !isfinite(__x_i), 0); })
#define u_int uint
#endif /* __GNUC__ */

1
staticsymbols.h
View File

@ -8,7 +8,6 @@ static struct redisFunctionSym symsTable[] = {
{"addReplyBulkCString",(unsigned long)addReplyBulkCString},
{"addReplyBulkLen",(unsigned long)addReplyBulkLen},
{"addReplyDouble",(unsigned long)addReplyDouble},
{"addReplyLong",(unsigned long)addReplyLong},
{"addReplyLongLong",(unsigned long)addReplyLongLong},
{"addReplySds",(unsigned long)addReplySds},
{"addReplyUlong",(unsigned long)addReplyUlong},

2
tests/support/server.tcl
View File

@ -13,7 +13,7 @@ proc kill_server config {
# check for leaks
catch {
if {[string match {*Darwin*} [exec uname -a]]} {
test {Check for memory leaks} {
test "Check for memory leaks (pid $pid)" {
exec leaks $pid
} {*0 leaks*}
}