opensourcemirror
/
postgresql
mirror of git://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Releases Wiki Activity
postgresql/src/backend/postmaster/postmaster.c

6612 lines
184 KiB
C
Raw Normal View History

Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
1999-02-14 00:22:53 +01:00
* postmaster.c
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* This program acts as a clearing house for requests to the
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* POSTGRES system. Frontend programs send a startup message
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* to the Postmaster and the postmaster uses the info in the
* message to setup a backend process.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
* The postmaster also manages system-wide operations such as
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* startup and shutdown. The postmaster itself doesn't do those
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
* operations, mind you --- it just forks off a subprocess to do them
* at the right times. It also takes care of resetting the system
* if a backend crashes.
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
*
* The postmaster process creates the shared memory and semaphore
* pools during startup, but as a rule does not touch them itself.
Katherine Ward wrote: > Changes to avoid collisions with WIN32 & MFC names... > 1. Renamed: > a. PROC => PGPROC > b. GetUserName() => GetUserNameFromId() > c. GetCurrentTime() => GetCurrentDateTime() > d. IGNORE => IGNORE_DTF in include/utils/datetime.h & utils/adt/datetim > > 2. Added _P to some lex/yacc tokens: > CONST, CHAR, DELETE, FLOAT, GROUP, IN, OUT Jan
2002-06-11 15:40:53 +02:00
* In particular, it is not a member of the PGPROC array of backends
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* and so it cannot participate in lock-manager operations. Keeping
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
* the postmaster away from shared memory operations makes it simpler
* and more reliable. The postmaster is almost always able to recover
* from crashes of individual backends by resetting shared memory;
* if it did much with shared memory then it would be prone to crashing
* along with the backends.
*
A bit of code beautification/cleanup of obsolete comments. Rethink ordering of startup operations in one or two places.
2001-06-21 18:43:24 +02:00
* When a request message is received, we now fork() immediately.
* The child process performs authentication of the request, and
* then becomes a backend if successful. This allows the auth code
* to be written in a simple single-threaded style (as opposed to the
* crufty "poor man's multitasking" code that used to be needed).
* More importantly, it ensures that blockages in non-multithreaded
* libraries like SSL or PAM cannot cause denial of service to other
* clients.
*
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
*
Update copyrights for 2020 Backpatch-through: update all files in master, backpatch legal files through 9.4
2020-01-01 18:21:45 +01:00
* Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
Add: * Portions Copyright (c) 1996-2000, PostgreSQL, Inc to all files copyright Regents of Berkeley. Man, that's a lot of files.
2000-01-26 06:58:53 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*
* IDENTIFICATION
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/backend/postmaster/postmaster.c
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
* NOTES
*
* Initialization:
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
* The Postmaster sets up shared memory data structures
* for the backends.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
* Synchronization:
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
* The Postmaster shares memory with the backends but should avoid
* touching shared memory, so as not to become stuck if a crashing
* backend screws up locks or shared memory. Likewise, the Postmaster
* should never block on messages from frontend clients.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* Garbage Collection:
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* The Postmaster cleans up after backends if they have an emergency
* exit and/or core dump.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Back out pg_autovacuum commit after cvs clean failure causes commit.
2004-07-21 22:34:50 +02:00
* Error Reporting:
* Use write_stderr() only for reporting "interactive" errors
* (essentially, bogus arguments on the command line). Once the
Remove silent_mode. You get the same functionality with "pg_ctl -l postmaster.log", or nohup. There was a small issue with LINUX_OOM_ADJ and silent_mode, namely that with silent_mode the postmaster process incorrectly used the OOM settings meant for backend processes. We certainly could've fixed that directly, but since silent_mode was redundant anyway, we might as well just remove it.
2011-07-04 13:35:44 +02:00
* postmaster is launched, use ereport().
Back out pg_autovacuum commit after cvs clean failure causes commit.
2004-07-21 22:34:50 +02:00
*
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*-------------------------------------------------------------------------
*/
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
New ps display code, works on more platforms. Install a default configuration file. Clean up some funny business in the config file code.
2000-06-04 03:44:38 +02:00
#include "postgres.h"
Re-add getopt.h check, remove NT-specific tests for it.
1999-07-19 04:27:16 +02:00
#include <unistd.h>
#include <signal.h>
Adjust our timezone library to use pg_time_t (typedef'd as int64) in place of time_t, as per prior discussion. The behavior does not change on machines without a 64-bit-int type, but on machines with one, which is most, we are rid of the bizarre boundary behavior at the edges of the 32-bit-time_t range (1901 and 2038). The system will now treat times over the full supported timestamp range as being in your local time zone. It may seem a little bizarre to consider that times in 4000 BC are PST or EST, but this is surely at least as reasonable as propagating Gregorian calendar rules back that far. I did not modify the format of the zic timezone database files, which means that for the moment the system will not know about daylight-savings periods outside the range 1901-2038. Given the way the files are set up, it's not a simple decision like 'widen to 64 bits'; we have to actually think about the range of years that need to be supported. We should probably inquire what the plans of the upstream zic people are before making any decisions of our own.
2004-06-03 04:08:07 +02:00
#include <time.h>
Re-add getopt.h check, remove NT-specific tests for it.
1999-07-19 04:27:16 +02:00
#include <sys/wait.h>
#include <ctype.h>
#include <sys/stat.h>
#include <sys/socket.h>
#include <fcntl.h>
Update #include cleanups
1999-07-16 05:14:30 +02:00
#include <sys/param.h>
#include <netdb.h>
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
#include <limits.h>
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Another switch for configure: NEED_SYS_SELECT_H to HAVE_SYS_SELECT_H
1997-01-24 19:27:32 +01:00
#ifdef HAVE_SYS_SELECT_H
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
#include <sys/select.h>
#endif
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Rename Rendezvous to Bonjour to match OS/X renaming.
2005-05-15 02:26:19 +02:00
#ifdef USE_BONJOUR
Replace use of the long-deprecated Bonjour API DNSServiceRegistrationCreate with the not-so-deprecated DNSServiceRegister. This patch shouldn't change any user-visible behavior, it just gets rid of a deprecation warning in --with-bonjour builds. The new code will fail on OS X releases before 10.3, but it seems unlikely that anyone will want to run Postgres 8.5 on 10.2.
2009-09-08 18:08:26 +02:00
#include <dns_sd.h>
Add Rendezvous support to postmaster, from Chris Campbell
2003-06-11 08:56:07 +02:00
#endif
Add support for systemd service notifications Insert sd_notify() calls at server start and stop for integration with systemd. This allows the use of systemd service units of type "notify", which greatly simplifies the systemd configuration. Reviewed-by: Pavel Stěhule <pavel.stehule@gmail.com>
2015-11-17 12:46:17 +01:00
#ifdef USE_SYSTEMD
#include <systemd/sd-daemon.h>
#endif
On Darwin, detect and report a multithreaded postmaster. Darwin --enable-nls builds use a substitute setlocale() that may start a thread. Buildfarm member orangutan experienced BackendList corruption on account of different postmaster threads executing signal handlers simultaneously. Furthermore, a multithreaded postmaster risks undefined behavior from sigprocmask() and fork(). Emit LOG messages about the problem and its workaround. Back-patch to 9.0 (all supported versions).
2015-01-08 04:35:44 +01:00
#ifdef HAVE_PTHREAD_IS_THREADED_NP
#include <pthread.h>
#endif
Put back some more not-so-unused-as-all-that #includes. This un-breaks the EXEC_BACKEND code on my machines, so hopefully it will fix the Windows buildfarm members.
2006-07-15 17:47:17 +02:00
#include "access/transam.h"
Change xlog.h to xlogdefs.h in bufpage.h, and fix fallout.
2008-06-07 00:35:22 +02:00
#include "access/xlog.h"
Refactor fork()-related code. We need to do various housekeeping tasks before we can invoke fork() -- flush stdio buffers, save and restore the profiling timer on Linux with LINUX_PROFILE, and handle BeOS stuff. This patch moves that code into a single function, fork_process(), instead of duplicating it at the various callsites of fork(). This patch doesn't address the EXEC_BACKEND case; there is room for further cleanup there.
2005-03-10 08:14:03 +01:00
#include "bootstrap/bootstrap.h"
Put back some more not-so-unused-as-all-that #includes. This un-breaks the EXEC_BACKEND code on my machines, so hopefully it will fix the Windows buildfarm members.
2006-07-15 17:47:17 +02:00
#include "catalog/pg_control.h"
Refactor dir/file permissions Consolidate directory and file create permissions for tools which work with the PG data directory by adding a new module (common/file_perm.c) that contains variables (pg_file_create_mode, pg_dir_create_mode) and constants to initialize them (0600 for files and 0700 for directories). Convert mkdir() calls in the backend to MakePGDirectory() if the original call used default permissions (always the case for regular PG directories). Add tests to make sure permissions in PGDATA are set correctly by the tools which modify the PG data directory. Authors: David Steele <david@pgmasters.net>, Adam Brightwell <adam.brightwell@crunchydata.com> Reviewed-By: Michael Paquier, with discussion amongst many others. Discussion: https://postgr.es/m/ad346fe6-b23e-59f1-ecb7-0e08390ad629%40pgmasters.net
2018-04-07 23:45:39 +02:00
#include "common/file_perm.h"
Move code shared between libpq and backend from backend/libpq/ to common/. When building libpq, ip.c and md5.c were symlinked or copied from src/backend/libpq into src/interfaces/libpq, but now that we have a directory specifically for routines that are shared between the server and client binaries, src/common/, move them there. Some routines in ip.c were only used in the backend. Keep those in src/backend/libpq, but rename to ifaddr.c to avoid confusion with the file that's now in common. Fix the comment in src/common/Makefile to reflect how libpq actually links those files. There are two more files that libpq symlinks directly from src/backend: encnames.c and wchar.c. I don't feel compelled to move those right now, though. Patch by Michael Paquier, with some changes by me. Discussion: <69938195-9c76-8523-0af8-eb718ea5b36e@iki.fi>
2016-09-02 12:49:59 +02:00
#include "common/ip.h"
Add application_name to connection authorized msg The connection authorized message has quite a bit of useful information in it, but didn't include the application_name (when provided), so let's add that as it can be very useful. Note that at the point where we're emitting the connection authorized message, we haven't processed GUCs, so it's not possible to get this by using log_line_prefix (which pulls from the GUC). There's also something to be said for having this included in the connection authorized message and then not needing to repeat it for every line, as having it in log_line_prefix would do. The GUC cleans the application name to pure-ascii, so do that here too, but pull out the logic for cleaning up a string into its own function in common and re-use it from those places, and check_cluster_name which was doing the same thing. Author: Don Seiler <don@seiler.us> Discussion: https://postgr.es/m/CAHJZqBB_Pxv8HRfoh%2BAB4KxSQQuPVvtYCzMg7woNR3r7dfmopw%40mail.gmail.com
2018-09-29 01:04:50 +02:00
#include "common/string.h"
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
#include "lib/ilist.h"
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
#include "libpq/auth.h"
Final cleanup.
1999-07-16 07:00:38 +02:00
#include "libpq/libpq.h"
Provide for forward compatibility with future minor protocol versions. Previously, any attempt to request a 3.x protocol version other than 3.0 would lead to a hard connection failure, which made the minor protocol version really no different from the major protocol version and precluded gentle protocol version breaks. Instead, when the client requests a 3.x protocol version where x is greater than 0, send the new NegotiateProtocolVersion message to convey that we support only 3.0. This makes it possible to introduce new minor protocol versions without requiring a connection retry when the server is older. In addition, if the startup packet includes name/value pairs where the name starts with "_pq_.", assume that those are protocol options, not GUCs. Include those we don't support (i.e. all of them, at present) in the NegotiateProtocolVersion message so that the client knows they were not understood. This makes it possible for the client to request previously-unsupported features without bumping the protocol version at all; the client can tell from the server's response whether the option was understood. It will take some time before servers that support these new facilities become common in the wild; to speed things up and make things easier for a future 3.1 protocol version, back-patch to all supported releases. Robert Haas and Badrul Chowdhury Discussion: http://postgr.es/m/BN6PR21MB0772FFA0CBD298B76017744CD1730@BN6PR21MB0772.namprd21.prod.outlook.com Discussion: http://postgr.es/m/30788.1498672033@sss.pgh.pa.us
2017-11-21 19:56:24 +01:00
#include "libpq/pqformat.h"
pq/signal() portability patch. Also psql copy prompt fix.
1996-12-26 23:08:34 +01:00
#include "libpq/pqsignal.h"
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
#include "miscadmin.h"
Centralize getopt-related declarations in a new header file pg_getopt.h. We used to have externs for getopt() and its API variables scattered all over the place. Now that we find we're going to need to tweak the variable declarations for Cygwin, it seems like a good idea to have just one place to tweak. In this commit, the variables are declared "#ifndef HAVE_GETOPT_H". That may or may not work everywhere, but we'll soon find out. Andres Freund
2014-02-15 20:31:30 +01:00
#include "pg_getopt.h"
Refactor fork()-related code. We need to do various housekeeping tasks before we can invoke fork() -- flush stdio buffers, save and restore the profiling timer on Linux with LINUX_PROFILE, and handle BeOS stuff. This patch moves that code into a single function, fork_process(), instead of duplicating it at the various callsites of fork(). This patch doesn't address the EXEC_BACKEND case; there is room for further cleanup there.
2005-03-10 08:14:03 +01:00
#include "pgstat.h"
Replace most usages of ntoh[ls] and hton[sl] with pg_bswap.h. All postgres internal usages are replaced, it's just libpq example usages that haven't been converted. External users of libpq can't generally rely on including postgres internal headers. Note that this includes replacing open-coded byte swapping of 64bit integers (using two 32 bit swaps) with a single 64bit swap. Where it looked applicable, I have removed netinet/in.h and arpa/inet.h usage, which previously provided the relevant functionality. It's perfectly possible that I missed other reasons for including those, the buildfarm will tell. Author: Andres Freund Discussion: https://postgr.es/m/20170927172019.gheidqy6xvlxb325@alap3.anarazel.de
2017-10-02 00:36:14 +02:00
#include "port/pg_bswap.h"
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
#include "postmaster/autovacuum.h"
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
#include "postmaster/bgworker_internals.h"
Refactor fork()-related code. We need to do various housekeeping tasks before we can invoke fork() -- flush stdio buffers, save and restore the profiling timer on Linux with LINUX_PROFILE, and handle BeOS stuff. This patch moves that code into a single function, fork_process(), instead of duplicating it at the various callsites of fork(). This patch doesn't address the EXEC_BACKEND case; there is room for further cleanup there.
2005-03-10 08:14:03 +01:00
#include "postmaster/fork_process.h"
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
#include "postmaster/pgarch.h"
Refactor fork()-related code. We need to do various housekeeping tasks before we can invoke fork() -- flush stdio buffers, save and restore the profiling timer on Linux with LINUX_PROFILE, and handle BeOS stuff. This patch moves that code into a single function, fork_process(), instead of duplicating it at the various callsites of fork(). This patch doesn't address the EXEC_BACKEND case; there is room for further cleanup there.
2005-03-10 08:14:03 +01:00
#include "postmaster/postmaster.h"
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
#include "postmaster/syslogger.h"
Logical replication - Add PUBLICATION catalogs and DDL - Add SUBSCRIPTION catalog and DDL - Define logical replication protocol and output plugin - Add logical replication workers From: Petr Jelinek <petr@2ndquadrant.com> Reviewed-by: Steve Singer <steve@ssinger.info> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Erik Rijkers <er@xs4all.nl> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2017-01-19 18:00:00 +01:00
#include "replication/logicallauncher.h"
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
#include "replication/walsender.h"
Major code cleanup following the pg_password insertion... ...malloc/free -> palloc/pfree ...fopen/fclose -> AllocateFile/FreeFile
1997-12-09 04:11:25 +01:00
#include "storage/fd.h"
Final cleanup.
1999-07-16 07:00:38 +02:00
#include "storage/ipc.h"
Put back some more not-so-unused-as-all-that #includes. This un-breaks the EXEC_BACKEND code on my machines, so hopefully it will fix the Windows buildfarm members.
2006-07-15 17:47:17 +02:00
#include "storage/pg_shmem.h"
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
#include "storage/pmsignal.h"
Put back storage/proc.h in postmaster.c. I took this out thinking it wasn't needed anymore, but the EXEC_BACKEND code still needs it. Per buildfarm.
2012-07-17 16:14:06 +02:00
#include "storage/proc.h"
Final cleanup.
1999-07-16 07:00:38 +02:00
#include "tcop/tcopprot.h"
Make listen_addresses be a comma-separated list instead of a space-separated list. More consistent with our other list-containing GUC variables.
2004-08-08 22:17:36 +02:00
#include "utils/builtins.h"
Clean up the rather historically encumbered interface to now() and current time: provide a GetCurrentTimestamp() function that returns current time in the form of a TimestampTz, instead of separate time_t and microseconds fields. This is what all the callers really want anyway, and it eliminates low-level dependencies on AbsoluteTime, which is a deprecated datatype that will have to disappear eventually.
2005-06-30 00:51:57 +02:00
#include "utils/datetime.h"
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
#include "utils/memutils.h"
Change pg_ctl to detect server-ready by watching status in postmaster.pid. Traditionally, "pg_ctl start -w" has waited for the server to become ready to accept connections by attempting a connection once per second. That has the major problem that connection issues (for instance, a kernel packet filter blocking traffic) can't be reliably told apart from server startup issues, and the minor problem that if server startup isn't quick, we accumulate "the database system is starting up" spam in the server log. We've hacked around many of the possible connection issues, but it resulted in ugly and complicated code in pg_ctl.c. In commit c61559ec3, I changed the probe rate to every tenth of a second. That prompted Jeff Janes to complain that the log-spam problem had become much worse. In the ensuing discussion, Andres Freund pointed out that we could dispense with connection attempts altogether if the postmaster were changed to report its status in postmaster.pid, which "pg_ctl start" already relies on being able to read. This patch implements that, teaching postmaster.c to report a status string into the pidfile at the same state-change points already identified as being of interest for systemd status reporting (cf commit 7d17e683f). pg_ctl no longer needs to link with libpq at all; all its functions now depend on reading server files. In support of this, teach AddToDataDirLockFile() to allow addition of postmaster.pid lines in not-necessarily-sequential order. This is needed on Windows where the SHMEM_KEY line will never be written at all. We still have the restriction that we don't want to truncate the pidfile; document the reasons for that a bit better. Also, fix the pg_ctl TAP tests so they'll notice if "start -w" mode is broken --- before, they'd just wait out the sixty seconds until the loop gives up, and then report success anyway. (Yes, I found that out the hard way.) While at it, arrange for pg_ctl to not need to #include miscadmin.h; as a rather low-level backend header, requiring that to be compilable client-side is pretty dubious. This requires moving the #define's associated with the pidfile into a new header file, and moving PG_BACKEND_VERSIONSTR someplace else. For lack of a clearly better "someplace else", I put it into port.h, beside the declaration of find_other_exec(), since most users of that macro are passing the value to find_other_exec(). (initdb still depends on miscadmin.h, but at least pg_ctl and pg_upgrade no longer do.) In passing, fix main.c so that PG_BACKEND_VERSIONSTR actually defines the output of "postgres -V", which remarkably it had never done before. Discussion: https://postgr.es/m/CAMkU=1xJW8e+CTotojOMBd-yzUvD0e_JZu2xHo=MnuZ4__m7Pg@mail.gmail.com
2017-06-28 23:31:24 +02:00
#include "utils/pidfile.h"
Move init_ps_display from postgres.c to postmaster.c, putting it just after receipt of the startup packet. Now, postmaster children that are waiting for client authentication response will show as 'postgres: user database host authentication'. Also, do an init_ps_display for startup/shutdown/checkpoint subprocesses, so that they are readily identifiable as well. Fix an obscure race condition that could lead to Assert failure in the postmaster --- attempting to start a checkpoint process before any connections have been received led to calling PostmasterRandom before setting random_seed.
2001-10-19 02:44:08 +02:00
#include "utils/ps_status.h"
Introduce timeout handling framework Management of timeouts was getting a little cumbersome; what we originally had was more than enough back when we were only concerned about deadlocks and query cancel; however, when we added timeouts for standby processes, the code got considerably messier. Since there are plans to add more complex timeouts, this seems a good time to introduce a central timeout handling module. External modules register their timeout handlers during process initialization, and later enable and disable them as they see fit using a simple API; timeout.c is in charge of keeping track of which timeouts are in effect at any time, installing a common SIGALRM signal handler, and calling setitimer() as appropriate to ensure timely firing of external handlers. timeout.c additionally supports pluggable modules to add their own timeouts, though this capability isn't exercised anywhere yet. Additionally, as of this commit, walsender processes are aware of timeouts; we had a preexisting bug there that made those ignore SIGALRM, thus being subject to unhandled deadlocks, particularly during the authentication phase. This has already been fixed in back branches in commit 0bf8eb2a, which see for more details. Main author: Zoltán Böszörményi Some review and cleanup by Álvaro Herrera Extensive reworking by Tom Lane
2012-07-17 00:43:21 +02:00
#include "utils/timeout.h"
Refactor pid, random seed and start time initialization. Background workers, including parallel workers, were generating the same sequence of numbers in random(). This showed up as DSM handle collisions when Parallel Hash created multiple segments, but any code that calls random() in background workers could be affected if it cares about different backends generating different numbers. Repair by making sure that all new processes initialize the seed at the same time as they set MyProcPid and MyStartTime in a new function InitProcessGlobals(), called by the postmaster, its children and also standalone processes. Also add a new high resolution MyStartTimestamp as a potentially useful by-product, and remove SessionStartTime from struct Port as it is now redundant. No back-patch for now, as the known consequences so far are just a bunch of harmless shm_open(O_EXCL) collisions. Author: Thomas Munro Reviewed-by: Tom Lane Discussion: https://postgr.es/m/CAEepm%3D2eJj_6%3DB%2B2tEpGu2nf1BjthCf9nXXUouYvJJ4C5WSwhg%40mail.gmail.com
2018-10-19 02:59:14 +02:00
#include "utils/timestamp.h"
Move some things from builtins.h to new header files This avoids that builtins.h has to include additional header files.
2017-01-21 02:29:53 +01:00
#include "utils/varlena.h"
Statistical system views (yet without the config stuff, but it's hard to keep such massive changes in sync with the tree so I need to get it in and work from there now). Jan
2001-06-22 21:16:24 +02:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#ifdef EXEC_BACKEND
#include "storage/spin.h"
#endif
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
* Possible types of a backend. Beyond being the possible bkend_type values in
* struct bkend, these are OR-able request flag bits for SignalSomeChildren()
* and CountChildren().
*/
#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
A bit of code beautification/cleanup of obsolete comments. Rethink ordering of startup operations in one or two places.
2001-06-21 18:43:24 +02:00
* List of active backends (or child processes anyway; we don't actually
* know whether a given child has become a backend or is still in the
* authorization phase). This is used mainly to keep track of how many
* children we have and send them appropriate signals when necessary.
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
*
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
* "Special" children such as the startup, bgwriter and autovacuum launcher
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* tasks are not in this list. Autovacuum worker and walsender are in it.
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
* Also, "dead_end" children are in it: these are children launched just for
* the purpose of sending a friendly rejection message to a would-be client.
* We must track them because they are attached to shared memory, but we know
* they will never become live backends. dead_end children are not assigned a
* PMChildSlot.
*
Allow notifications to bgworkers without database connections. Previously, if one background worker registered another background worker and set bgw_notify_pid while for the second background worker, it would not receive notifications from the postmaster unless, at the time the "parent" was registered, BGWORKER_BACKEND_DATABASE_CONNECTION was set. To fix, instead instead of including only those background workers that requested database connections in the postmater's BackendList, include them all. There doesn't seem to be any reason not do this, and indeed it removes a significant amount of duplicated code. The other option is to make PostmasterMarkPIDForWorkerNotify look at BackgroundWorkerList in addition to BackendList, but that adds more code duplication instead of getting rid of it. Patch by me. Review and testing by Ashutosh Bapat.
2015-09-01 21:30:19 +02:00
* Background workers are in this list, too.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
typedef struct bkend
{
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
pid_t pid; /* process id of backend */
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
int32 cancel_key; /* cancel key for cancels for this backend */
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
int child_slot; /* PMChildSlot for this backend, if any */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
* backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
* bkend_type is normal, you should check for a recent transition.
*/
int bkend_type;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
bool dead_end; /* is it going to send an error and quit? */
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
bool bgworker_notify; /* gets bgworker start/stop notifications */
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
dlist_node elem; /* list link in BackendList */
Add typdefs to pgindent run.
1997-09-08 22:59:27 +02:00
} Backend;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
#ifdef EXEC_BACKEND
static Backend *ShmemBackendArray;
#endif
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
BackgroundWorker *MyBgworkerEntry = NULL;
Keep track of the last active slot in the shared ProcState array, so that search loops only have to scan that far and not through all maxBackends entries. This eliminates a performance penalty for setting maxBackends much higher than the average number of active backends. Also, eliminate no-longer-used 'backend tag' concept. Remove setting of environment variables at backend start (except for CYR_RECODE), since none of them are being examined by the backend any longer.
2000-11-12 21:51:52 +01:00
/* The socket number we are listening for connections on */
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
int PostPortNumber;
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
Support having multiple Unix-domain sockets per postmaster. Replace unix_socket_directory with unix_socket_directories, which is a list of socket directories, and adjust postmaster's code to allow zero or more Unix-domain sockets to be created. This is mostly a straightforward change, but since the Unix sockets ought to be created after the TCP/IP sockets for safety reasons (better chance of detecting a port number conflict), AddToDataDirLockFile needs to be fixed to support out-of-order updates of data directory lockfile lines. That's a change that had been foreseen to be necessary someday anyway. Honza Horak, reviewed and revised by Tom Lane
2012-08-10 23:26:44 +02:00
/* The directory names for Unix socket(s) */
char *Unix_socket_directories;
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
Support having multiple Unix-domain sockets per postmaster. Replace unix_socket_directory with unix_socket_directories, which is a list of socket directories, and adjust postmaster's code to allow zero or more Unix-domain sockets to be created. This is mostly a straightforward change, but since the Unix sockets ought to be created after the TCP/IP sockets for safety reasons (better chance of detecting a port number conflict), AddToDataDirLockFile needs to be fixed to support out-of-order updates of data directory lockfile lines. That's a change that had been foreseen to be necessary someday anyway. Honza Horak, reviewed and revised by Tom Lane
2012-08-10 23:26:44 +02:00
/* The TCP listen address(es) */
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
char *ListenAddresses;
pgindent run before 6.3 release, with Thomas' requested changes.
1998-02-26 05:46:47 +01:00
This patch reserves the last superuser_reserved_connections slots for connections by the superuser only. This patch replaces the last patch I sent a couple of days ago. It closes a connection that has not been authorised by a superuser if it would leave less than the GUC variable ReservedBackends (superuser_reserved_connections in postgres.conf) backend process slots free in the SISeg. This differs to the first patch which only reserved the last ReservedBackends slots in the procState array. This has made the free slot test more expensive due to the use of a lock. After thinking about a comment on the first patch I've also made it a fatal error if the number of reserved slots is not less than the maximum number of connections. Nigel J. Andrews
2002-08-29 23:02:12 +02:00
/*
* ReservedBackends is the number of backends reserved for superuser use.
Fix cross-checking of ReservedBackends/max_wal_senders/MaxConnections. We were independently checking ReservedBackends < MaxConnections and max_wal_senders < MaxConnections, but because walsenders aren't allowed to use superuser-reserved connections, that's really the wrong thing. Correct behavior is to insist on ReservedBackends + max_wal_senders being less than MaxConnections. Fix the code and associated documentation. This has been wrong for a long time, but since the situation probably hardly ever arises in the field (especially pre-v10, when the default for max_wal_senders was zero), no back-patch. Discussion: https://postgr.es/m/28271.1520195491@sss.pgh.pa.us
2018-03-08 17:25:26 +01:00
* This number is taken out of the pool size given by MaxConnections so
Code review for superuser_reserved_connections patch. Don't try to do database access outside a transaction; revert bogus performance improvement in SIBackendInit(); improve comments; add documentation (this part courtesy Neil Conway).
2002-11-21 07:36:08 +01:00
* number of backend slots available to non-superusers is
Fix cross-checking of ReservedBackends/max_wal_senders/MaxConnections. We were independently checking ReservedBackends < MaxConnections and max_wal_senders < MaxConnections, but because walsenders aren't allowed to use superuser-reserved connections, that's really the wrong thing. Correct behavior is to insist on ReservedBackends + max_wal_senders being less than MaxConnections. Fix the code and associated documentation. This has been wrong for a long time, but since the situation probably hardly ever arises in the field (especially pre-v10, when the default for max_wal_senders was zero), no back-patch. Discussion: https://postgr.es/m/28271.1520195491@sss.pgh.pa.us
2018-03-08 17:25:26 +01:00
* (MaxConnections - ReservedBackends). Note what this really means is
Code review for superuser_reserved_connections patch. Don't try to do database access outside a transaction; revert bogus performance improvement in SIBackendInit(); improve comments; add documentation (this part courtesy Neil Conway).
2002-11-21 07:36:08 +01:00
* "if there are <= ReservedBackends connections available, only superusers
* can make new connections" --- pre-existing superuser connections don't
* count against the limit.
This patch reserves the last superuser_reserved_connections slots for connections by the superuser only. This patch replaces the last patch I sent a couple of days ago. It closes a connection that has not been authorised by a superuser if it would leave less than the GUC variable ReservedBackends (superuser_reserved_connections in postgres.conf) backend process slots free in the SISeg. This differs to the first patch which only reserved the last ReservedBackends slots in the procState array. This has made the free slot test more expensive due to the use of a lock. After thinking about a comment on the first patch I've also made it a fatal error if the number of reserved slots is not less than the maximum number of connections. Nigel J. Andrews
2002-08-29 23:02:12 +02:00
*/
Generate pg_config.h.in by autoheader. Separate out manually editable parts. Standardize spelling of comments in pg_config.h.
2003-04-07 00:45:23 +02:00
int ReservedBackends;
This patch reserves the last superuser_reserved_connections slots for connections by the superuser only. This patch replaces the last patch I sent a couple of days ago. It closes a connection that has not been authorised by a superuser if it would leave less than the GUC variable ReservedBackends (superuser_reserved_connections in postgres.conf) backend process slots free in the SISeg. This differs to the first patch which only reserved the last ReservedBackends slots in the procState array. This has made the free slot test more expensive due to the use of a lock. After thinking about a comment on the first patch I've also made it a fatal error if the number of reserved slots is not less than the maximum number of connections. Nigel J. Andrews
2002-08-29 23:02:12 +02:00
Have a go at fixing various outstanding portability issues in code that was modified for IPv6. Use a robust definition of struct sockaddr_storage, do a proper configure test to see if ss_len exists, don't assume that getnameinfo() will handle AF_UNIX sockets, don't trust getaddrinfo to return the protocol we ask for, etc. This incorporates several outstanding patches from Kurt Roeckx, but I'm to blame for anything that doesn't work ...
2003-07-24 01:30:41 +02:00
/* The socket(s) we're listening to. */
Increase MAXLISTEN to a more generous value, and add an error message telling when it has been exceeded. Per trouble report from Jean-GÅrard Pailloncy.
2005-01-12 17:38:17 +01:00
#define MAXLISTEN 64
Create typedef pgsocket for storing socket descriptors. This silences some warnings on Win64. Not using the proper SOCKET datatype was actually wrong on Win32 as well, but didn't cause any warnings there. Also create define PGINVALID_SOCKET to indicate an invalid/non-existing socket, instead of using a hardcoded -1 value.
2010-01-10 15:16:08 +01:00
static pgsocket ListenSocket[MAXLISTEN];
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
* Set by the -o option
*/
Standardize on MAXPGPATH as the size of a file pathname buffer, eliminating some wildly inconsistent coding in various parts of the system. I set MAXPGPATH = 1024 in config.h.in. If anyone is really convinced that there ought to be a configure-time test to set the value, go right ahead ... but I think it's a waste of time.
1999-10-25 05:08:03 +02:00
static char ExtraOptions[MAXPGPATH];
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
* These globals control the behavior of the postmaster in case some
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* backend dumps core. Normally, it kills all peers of the dead backend
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* and reinitializes shared memory. By specifying -s or -n, we can have
* the postmaster stop (rather than kill) peers and not reinitialize
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* shared data structures. (Reinit is currently dead code, though.)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
static bool Reinit = true;
postmaster/postgres options cleanup.
1998-05-27 20:32:05 +02:00
static int SendStop = false;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
A bit of code beautification/cleanup of obsolete comments. Rethink ordering of startup operations in one or two places.
2001-06-21 18:43:24 +02:00
/* still more option variables */
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
bool EnableSSL = false;
Remove fork()/exec() and only do fork(). Small cleanups.
1998-05-29 19:00:34 +02:00
Add an overall timeout on the client authentication cycle, so that a hung client or lost connection can't indefinitely block a postmaster child (not to mention the possibility of deliberate DoS attacks). Timeout is controlled by new authentication_timeout GUC variable, which I set to 60 seconds by default ... does that seem reasonable?
2001-09-21 19:06:12 +02:00
int PreAuthDelay = 0;
int AuthenticationTimeout = 60;
Auto checkpoint creation.
2000-11-09 12:26:00 +01:00
Remove GUC log_statement, log_pid, log_timestamp, log_source_port. Functionality superceeded by log_line_prefix. Andrew Dunstan
2004-03-15 16:56:28 +01:00
bool log_hostname; /* for ps display and logging */
Move init_ps_display from postgres.c to postmaster.c, putting it just after receipt of the startup packet. Now, postmaster children that are waiting for client authentication response will show as 'postgres: user database host authentication'. Also, do an init_ps_display for startup/shutdown/checkpoint subprocesses, so that they are readily identifiable as well. Fix an obscure race condition that could lead to Assert failure in the postmaster --- attempting to start a checkpoint process before any connections have been received led to calling PostmasterRandom before setting random_seed.
2001-10-19 02:44:08 +02:00
bool Log_connections = false;
Add db-local user names, per discussion on hackers.
2002-08-18 05:03:26 +02:00
bool Db_user_namespace = false;
Add a boolean GUC parameter "bonjour" to control whether a Bonjour-enabled build actually attempts to advertise itself via Bonjour. Formerly it always did so, which meant that packagers had to decide for their users whether this behavior was wanted or not. The default is "off" to be on the safe side, though this represents a change in the default behavior of a Bonjour-enabled build. Per discussion.
2009-09-08 19:08:36 +02:00
bool enable_bonjour = false;
Rename Rendezvous to Bonjour to match OS/X renaming.
2005-05-15 02:26:19 +02:00
char *bonjour_name;
Add restart_after_crash GUC. Normally, we automatically restart after a backend crash, but in some cases when PostgreSQL is invoked by clusterware it may be desirable to suppress this behavior, so we provide an option which does this. Since no existing GUC group quite fits, create a new group called "error handling options" for this and the previously undocumented GUC exit_on_error, which is now documented. Review by Fujii Masao.
2010-07-20 02:47:53 +02:00
bool restart_after_crash = true;
Add GUC parameter to control rendezvous name.
2003-07-22 22:29:13 +02:00
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
/* PIDs of special child processes; 0 when not running */
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
static pid_t StartupPID = 0,
Arrange to explicitly stop the pgstat processes at the same time we begin the shutdown checkpoint; there isn't anything left for them to do, so we may as well ensure that they shut down sooner rather than later. Per discussion.
2004-06-14 20:08:19 +02:00
BgWriterPID = 0,
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
CheckpointerPID = 0,
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
WalWriterPID = 0,
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
WalReceiverPID = 0,
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
AutoVacPID = 0,
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
PgArchPID = 0,
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
PgStatPID = 0,
SysLoggerPID = 0;
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
/* Startup process's status */
typedef enum
{
STARTUP_NOT_RUNNING,
STARTUP_RUNNING,
STARTUP_SIGNALED, /* we sent it a SIGQUIT or SIGKILL */
STARTUP_CRASHED
} StartupStatusEnum;
static StartupStatusEnum StartupStatus = STARTUP_NOT_RUNNING;
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
/* Startup/shutdown state */
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
#define NoShutdown 0
#define SmartShutdown 1
#define FastShutdown 2
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
#define ImmediateShutdown 3
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
static int Shutdown = NoShutdown;
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
static bool FatalError = false; /* T if recovering from backend crash */
Remove fork()/exec() and only do fork(). Small cleanups.
1998-05-29 19:00:34 +02:00
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/*
* We use a simple state machine to control startup, shutdown, and
* crash recovery (which is rather like shutdown followed by startup).
*
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
* After doing all the postmaster initialization work, we enter PM_STARTUP
* state and the startup process is launched. The startup process begins by
Cleanup and code review for the patch that made bgwriter active during archive recovery. Invent a separate state variable and inquiry function for XLogInsertAllowed() to clarify some tests and make the management of writing the end-of-recovery checkpoint less klugy. Fix several places that were incorrectly testing InRecovery when they should be looking at RecoveryInProgress or XLogInsertAllowed (because they will now be executed in the bgwriter not startup process). Clarify handling of bad LSNs passed to XLogFlush during recovery. Use a spinlock for setting/testing SharedRecoveryInProgress. Improve quite a lot of comments. Heikki and Tom
2009-06-26 22:29:04 +02:00
* reading the control file and other preliminary initialization steps.
* In a normal startup, or after crash recovery, the startup process exits
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* with exit code 0 and we switch to PM_RUN state. However, archive recovery
Cleanup and code review for the patch that made bgwriter active during archive recovery. Invent a separate state variable and inquiry function for XLogInsertAllowed() to clarify some tests and make the management of writing the end-of-recovery checkpoint less klugy. Fix several places that were incorrectly testing InRecovery when they should be looking at RecoveryInProgress or XLogInsertAllowed (because they will now be executed in the bgwriter not startup process). Clarify handling of bad LSNs passed to XLogFlush during recovery. Use a spinlock for setting/testing SharedRecoveryInProgress. Improve quite a lot of comments. Heikki and Tom
2009-06-26 22:29:04 +02:00
* is handled specially since it takes much longer and we would like to support
* hot standby during archive recovery.
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
*
Cleanup and code review for the patch that made bgwriter active during archive recovery. Invent a separate state variable and inquiry function for XLogInsertAllowed() to clarify some tests and make the management of writing the end-of-recovery checkpoint less klugy. Fix several places that were incorrectly testing InRecovery when they should be looking at RecoveryInProgress or XLogInsertAllowed (because they will now be executed in the bgwriter not startup process). Clarify handling of bad LSNs passed to XLogFlush during recovery. Use a spinlock for setting/testing SharedRecoveryInProgress. Improve quite a lot of comments. Heikki and Tom
2009-06-26 22:29:04 +02:00
* When the startup process is ready to start archive recovery, it signals the
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
* postmaster, and we switch to PM_RECOVERY state. The background writer and
* checkpointer are launched, while the startup process continues applying WAL.
* If Hot Standby is enabled, then, after reaching a consistent point in WAL
* redo, startup process signals us again, and we switch to PM_HOT_STANDBY
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* state and begin accepting connections to perform read-only queries. When
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
* archive recovery is finished, the startup process exits with exit code 0
* and we switch to PM_RUN state.
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
*
Allow read only connections during recovery, known as Hot Standby. Enabled by recovery_connections = on (default) and forcing archive recovery using a recovery.conf. Recovery processing now emulates the original transactions as they are replayed, providing full locking and MVCC behaviour for read only queries. Recovery must enter consistent state before connections are allowed, so there is a delay, typically short, before connections succeed. Replay of recovering transactions can conflict and in some cases deadlock with queries during recovery; these result in query cancellation after max_standby_delay seconds have expired. Infrastructure changes have minor effects on normal running, though introduce four new types of WAL record. New test mode "make standbycheck" allows regression tests of static command behaviour on a standby server while in recovery. Typical and extreme dynamic behaviours have been checked via code inspection and manual testing. Few port specific behaviours have been utilised, though primary testing has been on Linux only so far. This commit is the basic patch. Additional changes will follow in this release to enhance some aspects of behaviour, notably improved handling of conflicts, deadlock detection and query cancellation. Changes to VACUUM FULL are also required. Simon Riggs, with significant and lengthy review by Heikki Linnakangas, including streamlined redesign of snapshot creation and two-phase commit. Important contributions from Florian Pflug, Mark Kirkwood, Merlin Moncure, Greg Stark, Gianni Ciolli, Gabriele Bartolini, Hannu Krosing, Robert Haas, Tatsuo Ishii, Hiroyuki Yamada plus support and feedback from many other community members.
2009-12-19 02:32:45 +01:00
* Normal child backends can only be launched when we are in PM_RUN or
Rename PM_RECOVERY_CONSISTENT and PMSIGNAL_RECOVERY_CONSISTENT. The new names PM_HOT_STANDBY and PMSIGNAL_BEGIN_HOT_STANDBY more accurately reflect their actual function.
2010-05-15 22:01:32 +02:00
* PM_HOT_STANDBY state. (We also allow launch of normal
Allow read only connections during recovery, known as Hot Standby. Enabled by recovery_connections = on (default) and forcing archive recovery using a recovery.conf. Recovery processing now emulates the original transactions as they are replayed, providing full locking and MVCC behaviour for read only queries. Recovery must enter consistent state before connections are allowed, so there is a delay, typically short, before connections succeed. Replay of recovering transactions can conflict and in some cases deadlock with queries during recovery; these result in query cancellation after max_standby_delay seconds have expired. Infrastructure changes have minor effects on normal running, though introduce four new types of WAL record. New test mode "make standbycheck" allows regression tests of static command behaviour on a standby server while in recovery. Typical and extreme dynamic behaviours have been checked via code inspection and manual testing. Few port specific behaviours have been utilised, though primary testing has been on Linux only so far. This commit is the basic patch. Additional changes will follow in this release to enhance some aspects of behaviour, notably improved handling of conflicts, deadlock detection and query cancellation. Changes to VACUUM FULL are also required. Simon Riggs, with significant and lengthy review by Heikki Linnakangas, including streamlined redesign of snapshot creation and two-phase commit. Important contributions from Florian Pflug, Mark Kirkwood, Merlin Moncure, Greg Stark, Gianni Ciolli, Gabriele Bartolini, Hannu Krosing, Robert Haas, Tatsuo Ishii, Hiroyuki Yamada plus support and feedback from many other community members.
2009-12-19 02:32:45 +01:00
* child backends in PM_WAIT_BACKUP state, but only for superusers.)
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
* In other states we handle connection requests by launching "dead_end"
* child processes, which will simply send the client an error message and
* quit. (We track these in the BackendList so that we can know when they
* are all gone; this is important because they're still connected to shared
* memory, and would interfere with an attempt to destroy the shmem segment,
* possibly leading to SHMALL failure when we try to make a new one.)
* In PM_WAIT_DEAD_END state we are waiting for all the dead_end children
* to drain out of the system, and therefore stop accepting connection
* requests at all until the last existing child has quit (which hopefully
* will not be very long).
*
* Notice that this state variable does not distinguish *why* we entered
Code review for recent patch to terminate online backup during shutdown: do CancelBackup at a sane place, fix some oversights in the state transitions, allow only superusers to connect while we are waiting for backup mode to end.
2008-04-27 00:47:40 +02:00
* states later than PM_RUN --- Shutdown and FatalError must be consulted
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
* to find that out. FatalError is never true in PM_RECOVERY_* or PM_RUN
* states, nor in PM_SHUTDOWN states (because we don't enter those states
* when trying to recover from a crash). It can be true in PM_STARTUP state,
* because we don't clear it until we've successfully started WAL redo.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
typedef enum
{
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
PM_INIT, /* postmaster starting */
PM_STARTUP, /* waiting for startup subprocess */
Cleanup and code review for the patch that made bgwriter active during archive recovery. Invent a separate state variable and inquiry function for XLogInsertAllowed() to clarify some tests and make the management of writing the end-of-recovery checkpoint less klugy. Fix several places that were incorrectly testing InRecovery when they should be looking at RecoveryInProgress or XLogInsertAllowed (because they will now be executed in the bgwriter not startup process). Clarify handling of bad LSNs passed to XLogFlush during recovery. Use a spinlock for setting/testing SharedRecoveryInProgress. Improve quite a lot of comments. Heikki and Tom
2009-06-26 22:29:04 +02:00
PM_RECOVERY, /* in archive recovery mode */
Rename PM_RECOVERY_CONSISTENT and PMSIGNAL_RECOVERY_CONSISTENT. The new names PM_HOT_STANDBY and PMSIGNAL_BEGIN_HOT_STANDBY more accurately reflect their actual function.
2010-05-15 22:01:32 +02:00
PM_HOT_STANDBY, /* in hot standby mode */
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
PM_RUN, /* normal "database is alive" state */
Prevent shutdown in normal mode if online backup is running, and have pg_ctl warn about this. Cancel running online backups (by renaming the backup_label file, thus rendering the backup useless) when shutting down in fast mode. Laurenz Albe
2008-04-23 15:44:59 +02:00
PM_WAIT_BACKUP, /* waiting for online backup mode to end */
Make smart shutdown work in combination with Hot Standby/Streaming Replication. At present, killing the startup process does not release any locks it holds, so we must wait to stop the startup and walreceiver processes until all read-only backends have exited. Without this patch, the startup and walreceiver processes never exit, so the server gets permanently stuck in a half-shutdown state. Fujii Masao, with review, docs, and comment adjustments by me.
2010-04-08 03:39:37 +02:00
PM_WAIT_READONLY, /* waiting for read only backends to exit */
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
PM_WAIT_BACKENDS, /* waiting for live backends to exit */
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
PM_SHUTDOWN, /* waiting for checkpointer to do shutdown
* ckpt */
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
PM_SHUTDOWN_2, /* waiting for archiver and walsenders to
* finish */
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
PM_WAIT_DEAD_END, /* waiting for dead_end children to exit */
PM_NO_CHILDREN /* all important children have exited */
Re-run pgindent with updated list of typedefs. (Updated README should avoid this problem in the future.)
2007-11-15 23:25:18 +01:00
} PMState;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
static PMState pmState = PM_INIT;
In immediate shutdown, postmaster should not exit till children are gone. This adjusts commit 82233ce7ea42d6ba519aaec63008aff49da6c7af so that the postmaster does not exit until all its child processes have exited, even if the 5-second timeout elapses and we have to send SIGKILL. There is no great value in having the postmaster process quit sooner, and doing so can mislead onlookers into thinking that the cluster is fully terminated when actually some child processes still survive. This effect might explain recent test failures on buildfarm member hamster, wherein we failed to restart a cluster just after shutting it down with "pg_ctl stop -m immediate". I also did a bit of code review/beautification, including fixing a faulty use of the Max() macro on a volatile expression. Back-patch to 9.4. In older branches, the postmaster never waited for children to exit during immediate shutdowns, and changing that would be too much of a behavioral change.
2015-06-19 20:23:39 +02:00
/* Start time of SIGKILL timeout during immediate shutdown or child crash */
/* Zero means timeout is not running */
static time_t AbortStartTime = 0;
Initial pgindent run with pg_bsd_indent version 2.0. The new indent version includes numerous fixes thanks to Piotr Stefaniak. The main changes visible in this commit are: * Nicer formatting of function-pointer declarations. * No longer unexpectedly removes spaces in expressions using casts, sizeof, or offsetof. * No longer wants to add a space in "struct structname *varname", as well as some similar cases for const- or volatile-qualified pointers. * Declarations using PG_USED_FOR_ASSERTS_ONLY are formatted more nicely. * Fixes bug where comments following declarations were sometimes placed with no space separating them from the code. * Fixes some odd decisions for comments following case labels. * Fixes some cases where comments following code were indented to less than the expected column 33. On the less good side, it now tends to put more whitespace around typedef names that are not listed in typedefs.list. This might encourage us to put more effort into typedef name collection; it's not really a bug in indent itself. There are more changes coming after this round, having to do with comment indentation and alignment of lines appearing within parentheses. I wanted to limit the size of the diffs to something that could be reviewed without one's eyes completely glazing over, so it seemed better to split up the changes as much as practical. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 20:39:04 +02:00
In immediate shutdown, postmaster should not exit till children are gone. This adjusts commit 82233ce7ea42d6ba519aaec63008aff49da6c7af so that the postmaster does not exit until all its child processes have exited, even if the 5-second timeout elapses and we have to send SIGKILL. There is no great value in having the postmaster process quit sooner, and doing so can mislead onlookers into thinking that the cluster is fully terminated when actually some child processes still survive. This effect might explain recent test failures on buildfarm member hamster, wherein we failed to restart a cluster just after shutting it down with "pg_ctl stop -m immediate". I also did a bit of code review/beautification, including fixing a faulty use of the Max() macro on a volatile expression. Back-patch to 9.4. In older branches, the postmaster never waited for children to exit during immediate shutdowns, and changing that would be too much of a behavioral change.
2015-06-19 20:23:39 +02:00
/* Length of said timeout */
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
#define SIGKILL_CHILDREN_AFTER_SECS 5
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
static bool ReachedNormalRunning = false; /* T if we've reached PM_RUN */
More fixes for shutdown during recovery. 1. If we receive a fast shutdown request while in the PM_STARTUP state, process it just as we would in PM_RECOVERY, PM_HOT_STANDBY, or PM_RUN. Without this change, an early fast shutdown followed by Hot Standby causes the database to get stuck in a state where a shutdown is pending (so no new connections are allowed) but the shutdown request is never processed unless we end Hot Standby and enter normal running. 2. Avoid removing the backup label file when a smart or fast shutdown occurs during recovery. It makes sense to do this once we've reached normal running, since we must be taking a backup which now won't be valid. But during recovery we must be recovering from a previously taken backup, and any backup label file is needed to restart recovery from the right place. Fujii Masao and Robert Haas
2010-05-26 14:32:41 +02:00
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
bool ClientAuthInProgress = false; /* T during new-client
* authentication */
Further work on elog cleanup: fix some bogosities in elog's logic about when to send what to which, prevent recursion by introducing new COMMERROR elog level for client-communication problems, get rid of direct writes to stderr in backend/libpq files, prevent non-error elogs from going to client during the authentication cycle.
2002-03-04 02:46:04 +01:00
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
bool redirection_done = false; /* stderr redirected for syslogger? */
Only use the pipe chunking protocol if we know the syslogger should be catching stderr output, and we are not ourselves the syslogger. Otherwise, go directly to stderr. Bug noticed by Tom Lane. Backpatch as far as 8.0.
2007-07-19 21:13:43 +02:00
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
/* received START_AUTOVAC_LAUNCHER signal */
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
static volatile sig_atomic_t start_autovac_launcher = false;
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
Avoid calling kill() in a postmaster signal handler. This causes problems when the system load is high, per report from Zdenek Kotala in <1250860954.1239.114.camel@localhost>; instead of calling kill directly, have the signal handler set a flag which is checked in ServerLoop. This way, the handler can return before being called again by a subsequent signal sent from the autovacuum launcher. Also, increase the sleep in the launcher in this failure path to 1 second. Backpatch to 8.3, which is when the signalling between autovacuum launcher/postmaster was introduced. Also, add a couple of ReleasePostmasterChildSlot calls in error paths; this part backpatched to 8.4 which is when the child slot stuff was introduced.
2009-08-24 19:23:02 +02:00
/* the launcher needs to be signalled to communicate some condition */
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
static volatile bool avlauncher_needs_signal = false;
Fix recently-understood problems with handling of XID freezing, particularly in PITR scenarios. We now WAL-log the replacement of old XIDs with FrozenTransactionId, so that such replacement is guaranteed to propagate to PITR slave databases. Also, rather than relying on hint-bit updates to be preserved, pg_clog is not truncated until all instances of an XID are known to have been replaced by FrozenTransactionId. Add new GUC variables and pg_autovacuum columns to allow management of the freezing policy, so that users can trade off the size of pg_clog against the amount of freezing work done. Revise the already-existing code that forces autovacuum of tables approaching the wraparound point to make it more bulletproof; also, revise the autovacuum logic so that anti-wraparound vacuuming is done per-table rather than per-database. initdb forced because of changes in pg_class, pg_database, and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
Don't lose walreceiver start requests due to race condition in postmaster. When a walreceiver dies, the startup process will notice that and send a PMSIGNAL_START_WALRECEIVER signal to the postmaster, asking for a new walreceiver to be launched. There's a race condition, which at least in HEAD is very easy to hit, whereby the postmaster might see that signal before it processes the SIGCHLD from the walreceiver process. In that situation, sigusr1_handler() just dropped the start request on the floor, reasoning that it must be redundant. Eventually, after 10 seconds (WALRCV_STARTUP_TIMEOUT), the startup process would make a fresh request --- but that's a long time if the connection could have been re-established almost immediately. Fix it by setting a state flag inside the postmaster that we won't clear until we do launch a walreceiver. In cases where that results in an extra walreceiver launch, it's up to the walreceiver to realize it's unwanted and go away --- but we have, and need, that logic anyway for the opposite race case. I came across this through investigating unexpected delays in the src/test/recovery TAP tests: it manifests there in test cases where a master server is stopped and restarted while leaving streaming slaves active. This logic has been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/21344.1498494720@sss.pgh.pa.us
2017-06-26 23:31:56 +02:00
/* received START_WALRECEIVER signal */
static volatile sig_atomic_t WalReceiverRequested = false;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/* set when there's a worker that needs to be started up */
static volatile bool StartWorkerNeeded = true;
static volatile bool HaveCrashedWorker = false;
Allow SSL configuration to be updated at SIGHUP. It is no longer necessary to restart the server to enable, disable, or reconfigure SSL. Instead, we just create a new SSL_CTX struct (by re-reading all relevant files) whenever we get SIGHUP. Testing shows that this is fast enough that it shouldn't be a problem. In conjunction with that, downgrade the logic that complains about pg_hba.conf "hostssl" lines when SSL isn't active: now that's just a warning condition not an error. An issue that still needs to be addressed is what shall we do with passphrase-protected server keys? As this stands, the server would demand the passphrase again on every SIGHUP, which is certainly impractical. But the case was only barely supported before, so that does not seem a sufficient reason to hold up committing this patch. Andreas Karlsson, reviewed by Michael Banck and Michael Paquier Discussion: https://postgr.es/m/556A6E8A.9030400@proxel.se
2017-01-03 03:37:12 +01:00
#ifdef USE_SSL
/* Set when and if SSL has been initialized properly */
static bool LoadedSSL = false;
#endif
Replace use of the long-deprecated Bonjour API DNSServiceRegistrationCreate with the not-so-deprecated DNSServiceRegister. This patch shouldn't change any user-visible behavior, it just gets rid of a deprecation warning in --with-bonjour builds. The new code will fail on OS X releases before 10.3, but it seems unlikely that anyone will want to run Postgres 8.5 on 10.2.
2009-09-08 18:08:26 +02:00
#ifdef USE_BONJOUR
static DNSServiceRef bonjour_sdref = NULL;
#endif
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* postmaster.c - function prototypes
*/
Fix incorrect order of lock file removal and failure to close() sockets. Commit c9b0cbe98bd783e24a8c4d8d8ac472a494b81292 accidentally broke the order of operations during postmaster shutdown: it resulted in removing the per-socket lockfiles after, not before, postmaster.pid. This creates a race-condition hazard for a new postmaster that's started immediately after observing that postmaster.pid has disappeared; if it sees the socket lockfile still present, it will quite properly refuse to start. This error appears to be the explanation for at least some of the intermittent buildfarm failures we've seen in the pg_upgrade test. Another problem, which has been there all along, is that the postmaster has never bothered to close() its listen sockets, but has just allowed them to close at process death. This creates a different race condition for an incoming postmaster: it might be unable to bind to the desired listen address because the old postmaster is still incumbent. This might explain some odd failures we've seen in the past, too. (Note: this is not related to the fact that individual backends don't close their client communication sockets. That behavior is intentional and is not changed by this patch.) Fix by adding an on_proc_exit function that closes the postmaster's ports explicitly, and (in 9.3 and up) reshuffling the responsibility for where to unlink the Unix socket files. Lock file unlinking can stay where it is, but teach it to unlink the lock files in reverse order of creation.
2015-08-02 20:54:44 +02:00
static void CloseServerPorts(int status, Datum arg);
Remove external PID file on postmaster exit
2012-08-21 05:47:11 +02:00
static void unlink_external_pid_file(int status, Datum arg);
Install some simple defenses in postmaster startup to help ensure a useful error message if the installation directory layout is messed up (or at least, something more useful than the behavior exhibited in bug #4787). During postmaster startup, check that get_pkglib_path resolves as a readable directory; and if ParseTzFile() fails to open the expected timezone abbreviation file, check the possibility that the directory is missing rather than just the specified file. In case of either failure, issue a hint suggesting that the installation is broken. These two checks cover the lib/ and share/ trees of a full installation, which should take care of most scenarios where a sysadmin decides to get cute.
2009-05-03 00:02:37 +02:00
static void getInstallationPaths(const char *argv0);
Allow group access on PGDATA Allow the cluster to be optionally init'd with read access for the group. This means a relatively non-privileged user can perform a backup of the cluster without requiring write privileges, which enhances security. The mode of PGDATA is used to determine whether group permissions are enabled for directory and file creates. This method was chosen as it's simple and works well for the various utilities that write into PGDATA. Changing the mode of PGDATA manually will not automatically change the mode of all the files contained therein. If the user would like to enable group access on an existing cluster then changing the mode of all the existing files will be required. Note that pg_upgrade will automatically change the mode of all migrated files if the new cluster is init'd with the -g option. Tests are included for the backend and all the utilities which operate on the PG data directory to ensure that the correct mode is set based on the data directory permissions. Author: David Steele <david@pgmasters.net> Reviewed-By: Michael Paquier, with discussion amongst many others. Discussion: https://postgr.es/m/ad346fe6-b23e-59f1-ecb7-0e08390ad629%40pgmasters.net
2018-04-07 23:45:39 +02:00
static void checkControlFile(void);
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
static Port *ConnCreate(int serverFd);
static void ConnFree(Port *port);
Use data directory inode number, not port, to select SysV resource keys. This approach provides a much tighter binding between a data directory and the associated SysV shared memory block (and SysV or named-POSIX semaphores, if we're using those). Key collisions are still possible, but only between data directories stored on different filesystems, so the situation should be negligible in practice. More importantly, restarting the postmaster with a different port number no longer risks failing to identify a relevant shared memory block, even when postmaster.pid has been removed. A standalone backend is likewise much more certain to detect conflicting leftover backends. (In the longer term, we might now think about deprecating the port as a cluster-wide value, so that one postmaster could support sockets with varying port numbers. But that's for another day.) The hazards fixed here apply only on Unix systems; our Windows code paths already use identifiers derived from the data directory path name rather than the port. src/test/recovery/t/017_shm.pl, which intends to test key-collision cases, has been substantially rewritten since it can no longer use two postmasters with identical port numbers to trigger the case. Instead, use Perl's IPC::SharedMem module to create a conflicting shmem segment directly. The test script will be skipped if that module is not available. (This means that some older buildfarm members won't run it, but I don't think that that results in any meaningful coverage loss.) Patch by me; thanks to Noah Misch and Peter Eisentraut for discussion and review. Discussion: https://postgr.es/m/16908.1557521200@sss.pgh.pa.us
2019-09-05 19:31:41 +02:00
static void reset_shared(void);
Revert removal of signed, volatile, and signal handler arg type tests.
2000-08-29 11:36:51 +02:00
static void SIGHUP_handler(SIGNAL_ARGS);
static void pmdie(SIGNAL_ARGS);
static void reaper(SIGNAL_ARGS);
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
static void sigusr1_handler(SIGNAL_ARGS);
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
static void startup_die(SIGNAL_ARGS);
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
static void dummy_handler(SIGNAL_ARGS);
Introduce timeout handling framework Management of timeouts was getting a little cumbersome; what we originally had was more than enough back when we were only concerned about deadlocks and query cancel; however, when we added timeouts for standby processes, the code got considerably messier. Since there are plans to add more complex timeouts, this seems a good time to introduce a central timeout handling module. External modules register their timeout handlers during process initialization, and later enable and disable them as they see fit using a simple API; timeout.c is in charge of keeping track of which timeouts are in effect at any time, installing a common SIGALRM signal handler, and calling setitimer() as appropriate to ensure timely firing of external handlers. timeout.c additionally supports pluggable modules to add their own timeouts, though this capability isn't exercised anywhere yet. Additionally, as of this commit, walsender processes are aware of timeouts; we had a preexisting bug there that made those ignore SIGALRM, thus being subject to unhandled deadlocks, particularly during the authentication phase. This has already been fixed in back branches in commit 0bf8eb2a, which see for more details. Main author: Zoltán Böszörményi Some review and cleanup by Álvaro Herrera Extensive reworking by Tom Lane
2012-07-17 00:43:21 +02:00
static void StartupPacketTimeoutHandler(void);
Tweak postmaster code to avoid double reporting when bgwriter crashes.
2004-08-04 22:09:47 +02:00
static void CleanupBackend(int pid, int exitstatus);
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
static bool CleanupBackgroundWorker(int pid, int exitstatus);
Tweak postmaster code to avoid double reporting when bgwriter crashes.
2004-08-04 22:09:47 +02:00
static void HandleChildCrash(int pid, int exitstatus, const char *procname);
Further work on elog cleanup: fix some bogosities in elog's logic about when to send what to which, prevent recursion by introducing new COMMERROR elog level for client-communication problems, get rid of direct writes to stderr in backend/libpq files, prevent non-error elogs from going to client during the authentication cycle.
2002-03-04 02:46:04 +01:00
static void LogChildExit(int lev, const char *procname,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
int pid, int exitstatus);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
static void PostmasterStateMachine(void);
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
static void BackendInitialize(Port *port);
Tweak __attribute__-wrapping macros for better pgindent results. This improves on commit bbfd7edae5aa5ad5553d3c7e102f2e450d4380d4 by making two simple changes: * pg_attribute_noreturn now takes parentheses, ie pg_attribute_noreturn(). Likewise pg_attribute_unused(), pg_attribute_packed(). This reduces pgindent's tendency to misformat declarations involving them. * attributes are now always attached to function declarations, not definitions. Previously some places were taking creative shortcuts, which were not merely candidates for bad misformatting by pgindent but often were outright wrong anyway. (It does little good to put a noreturn annotation where callers can't see it.) In any case, if we would like to believe that these macros can be used with non-gcc compilers, we should avoid gratuitous variance in usage patterns. I also went through and manually improved the formatting of a lot of declarations, and got rid of excessively repetitive (and now obsolete anyway) comments informing the reader what pg_attribute_printf is for.
2015-03-26 19:03:19 +01:00
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
static int ServerLoop(void);
static int BackendStartup(Port *port);
Fix assorted inconsistencies. There were a number of issues in the recent commits which include typos, code and comments mismatch, leftover function declarations. Fix them. Reported-by: Alexander Lakhin Author: Alexander Lakhin, Amit Kapila and Amit Langote Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/ef0c0232-0c1d-3a35-63d4-0ebd06e31387@gmail.com
2019-06-08 04:46:38 +02:00
static int ProcessStartupPacket(Port *port, bool secure_done);
Provide for forward compatibility with future minor protocol versions. Previously, any attempt to request a 3.x protocol version other than 3.0 would lead to a hard connection failure, which made the minor protocol version really no different from the major protocol version and precluded gentle protocol version breaks. Instead, when the client requests a 3.x protocol version where x is greater than 0, send the new NegotiateProtocolVersion message to convey that we support only 3.0. This makes it possible to introduce new minor protocol versions without requiring a connection retry when the server is older. In addition, if the startup packet includes name/value pairs where the name starts with "_pq_.", assume that those are protocol options, not GUCs. Include those we don't support (i.e. all of them, at present) in the NegotiateProtocolVersion message so that the client knows they were not understood. This makes it possible for the client to request previously-unsupported features without bumping the protocol version at all; the client can tell from the server's response whether the option was understood. It will take some time before servers that support these new facilities become common in the wild; to speed things up and make things easier for a future 3.1 protocol version, back-patch to all supported releases. Robert Haas and Badrul Chowdhury Discussion: http://postgr.es/m/BN6PR21MB0772FFA0CBD298B76017744CD1730@BN6PR21MB0772.namprd21.prod.outlook.com Discussion: http://postgr.es/m/30788.1498672033@sss.pgh.pa.us
2017-11-21 19:56:24 +01:00
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
static void processCancelRequest(Port *port, void *pkt);
Have a go at fixing various outstanding portability issues in code that was modified for IPv6. Use a robust definition of struct sockaddr_storage, do a proper configure test to see if ss_len exists, don't assume that getnameinfo() will handle AF_UNIX sockets, don't trust getaddrinfo to return the protocol we ask for, etc. This incorporates several outstanding patches from Kurt Roeckx, but I'm to blame for anything that doesn't work ...
2003-07-24 01:30:41 +02:00
static int initMasks(fd_set *rmask);
If we fail to fork a new backend process, (try to) report the failure to the client before closing the connection. Before 7.2 this was done correctly, but new code would simply close the connection with no report to the client.
2002-01-06 22:40:02 +01:00
static void report_fork_failure_to_client(Port *port, int errnum);
Check for too many postmaster children before spawning a bgworker. The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-07 18:39:09 +02:00
static CAC_state canAcceptConnections(int backend_type);
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
static bool RandomCancelKey(int32 *cancel_key);
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
static void signal_child(pid_t pid, int signal);
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
static bool SignalSomeChildren(int signal, int targets);
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
static void TerminateChildren(int signal);
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
static int CountChildren(int target);
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
static void maybe_start_bgworkers(void);
Reorganize backend code to more cleanly manage executable names and backend startup.
2004-05-14 00:45:04 +02:00
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
static pid_t StartChildProcess(AuxProcType type);
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
static void StartAutovacuumWorker(void);
Don't lose walreceiver start requests due to race condition in postmaster. When a walreceiver dies, the startup process will notice that and send a PMSIGNAL_START_WALRECEIVER signal to the postmaster, asking for a new walreceiver to be launched. There's a race condition, which at least in HEAD is very easy to hit, whereby the postmaster might see that signal before it processes the SIGCHLD from the walreceiver process. In that situation, sigusr1_handler() just dropped the start request on the floor, reasoning that it must be redundant. Eventually, after 10 seconds (WALRCV_STARTUP_TIMEOUT), the startup process would make a fresh request --- but that's a long time if the connection could have been re-established almost immediately. Fix it by setting a state flag inside the postmaster that we won't clear until we do launch a walreceiver. In cases where that results in an extra walreceiver launch, it's up to the walreceiver to realize it's unwanted and go away --- but we have, and need, that logic anyway for the opposite race case. I came across this through investigating unexpected delays in the src/test/recovery TAP tests: it manifests there in test cases where a master server is stopped and restarted while leaving streaming slaves active. This logic has been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/21344.1498494720@sss.pgh.pa.us
2017-06-26 23:31:56 +02:00
static void MaybeStartWalReceiver(void);
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
static void InitPostmasterDeathWatchHandle(void);
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
Make postmaster restart archiver soon after it dies, even during recovery. After the archiver dies, postmaster tries to start a new one immediately. But previously this could happen only while server was running normally even though archiving was enabled always (i.e., archive_mode was set to always). So the archiver running during recovery could not restart soon after it died. This is an oversight in commit ffd3774. This commit changes reaper(), postmaster's signal handler to cleanup after a child process dies, so that it tries to a new archiver even during recovery if necessary. Patch by me. Review by Alvaro Herrera.
2015-06-12 16:11:51 +02:00
/*
* Archiver is allowed to start up at the current postmaster state?
*
* If WAL archiving is enabled always, we are allowed to start archiver
* even during recovery.
*/
#define PgArchStartupAllowed() \
((XLogArchivingActive() && pmState == PM_RUN) || \
(XLogArchivingAlways() && \
(pmState == PM_RECOVERY || pmState == PM_HOT_STANDBY)))
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
#ifdef EXEC_BACKEND
Reorganize backend code to more cleanly manage executable names and backend startup.
2004-05-14 00:45:04 +02:00
Drops in the CreateProcess calls for Win32 (essentially wrapping up the fork/exec portion of the port), and fixes a handful of whitespace issues Claudio Natoli
2004-01-11 04:49:31 +01:00
#ifdef WIN32
Remove support for using wait3() in place of waitpid(). All Unix-oid platforms that we currently support should have waitpid(), since it's in V2 of the Single Unix Spec. Our git history shows that the wait3 code was added to support NextStep, which we officially dropped support for as of 9.2. So get rid of the configure test, and simplify the macro spaghetti in reaper(). Per suggestion from Fujii Masao.
2012-07-05 20:00:40 +02:00
#define WNOHANG 0 /* ignored, so any integer value will do */
static pid_t waitpid(pid_t pid, int *exitstatus, int options);
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
static void WINAPI pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired);
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
static HANDLE win32ChildQueue;
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
typedef struct
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
HANDLE waitHandle;
HANDLE procHandle;
DWORD procId;
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
} win32_deadchild_waitinfo;
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
#endif /* WIN32 */
Drops in the CreateProcess calls for Win32 (essentially wrapping up the fork/exec portion of the port), and fixes a handful of whitespace issues Claudio Natoli
2004-01-11 04:49:31 +01:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
static pid_t backend_forkexec(Port *port);
static pid_t internal_forkexec(int argc, char *argv[], Port *port);
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
typedef struct
{
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
* if not a socket */
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
WSAPROTOCOL_INFO wsainfo;
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
} InheritableSocket;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#else
typedef int InheritableSocket;
#endif
/*
* Structure contains all variables passed to exec:ed backends
*/
typedef struct
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Port port;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
InheritableSocket portsocket;
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
char DataDir[MAXPGPATH];
Create typedef pgsocket for storing socket descriptors. This silences some warnings on Win64. Not using the proper SOCKET datatype was actually wrong on Win32 as well, but didn't cause any warnings there. Also create define PGINVALID_SOCKET to indicate an invalid/non-existing socket, instead of using a hardcoded -1 value.
2010-01-10 15:16:08 +01:00
pgsocket ListenSocket[MAXLISTEN];
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
int32 MyCancelKey;
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
int MyPMChildSlot;
Fix one more cast for _open_osfhandle(). Tsutomu Yamada
2010-01-02 13:01:29 +01:00
#ifndef WIN32
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
unsigned long UsedShmemSegID;
Fix one more cast for _open_osfhandle(). Tsutomu Yamada
2010-01-02 13:01:29 +01:00
#else
Avoid "could not reattach" by providing space for concurrent allocation. We've long had reports of intermittent "could not reattach to shared memory" errors on Windows. Buildfarm member dory fails that way when PGSharedMemoryReAttach() execution overlaps with creation of a thread for the process's "default thread pool". Fix that by providing a second region to receive asynchronous allocations that would otherwise intrude into UsedShmemSegAddr. In pgwin32_ReserveSharedMemoryRegion(), stop trying to free reservations landing at incorrect addresses; the caller's next step has been to terminate the affected process. Back-patch to 9.4 (all supported versions). Reviewed by Tom Lane. He also did much of the prerequisite research; see commit bcbf2346d69f6006f126044864dd9383d50d87b4. Discussion: https://postgr.es/m/20190402135442.GA1173872@rfd.leadboat.com
2019-04-09 06:39:00 +02:00
void *ShmemProtectiveRegion;
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
HANDLE UsedShmemSegID;
Fix one more cast for _open_osfhandle(). Tsutomu Yamada
2010-01-02 13:01:29 +01:00
#endif
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
void *UsedShmemSegAddr;
slock_t *ShmemLock;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
VariableCache ShmemVariableCache;
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Backend *ShmemBackendArray;
Reduce the number of semaphores used under --disable-spinlocks. Instead of allocating a semaphore from the operating system for every spinlock, allocate a fixed number of semaphores (by default, 1024) from the operating system and multiplex all the spinlocks that get created onto them. This could self-deadlock if a process attempted to acquire more than one spinlock at a time, but since processes aren't supposed to execute anything other than short stretches of straight-line code while holding a spinlock, that shouldn't happen. One motivation for this change is that, with the introduction of dynamic shared memory, it may be desirable to create spinlocks that last for less than the lifetime of the server. Without this change, attempting to use such facilities under --disable-spinlocks would quickly exhaust any supply of available semaphores. Quite apart from that, it's desirable to contain the quantity of semaphores needed to run the server simply on convenience grounds, since using too many may make it harder to get PostgreSQL running on a new platform, which is mostly the point of --disable-spinlocks in the first place. Patch by me; review by Tom Lane.
2014-01-09 00:49:14 +01:00
#ifndef HAVE_SPINLOCKS
Make the different Unix-y semaphore implementations ABI-compatible. Previously, the "sem" field of PGPROC varied in size depending on which kernel semaphore API we were using. That was okay as long as there was only one likely choice per platform, but in the wake of commit ecb0d20a9, that assumption seems rather shaky. It doesn't seem out of the question anymore that an extension compiled against one API choice might be loaded into a postmaster built with another choice. Moreover, this prevents any possibility of selecting the semaphore API at postmaster startup, which might be something we want to do in future. Hence, change PGPROC.sem to be PGSemaphore (i.e. a pointer) for all Unix semaphore APIs, and turn the pointed-to data into an opaque struct whose contents are only known within the responsible modules. For the SysV and unnamed-POSIX APIs, the pointed-to data has to be allocated elsewhere in shared memory, which takes a little bit of rejiggering of the InitShmemAllocation code sequence. (I invented a ShmemAllocUnlocked() function to make that a little cleaner than it used to be. That function is not meant for any uses other than the ones it has now, but it beats having InitShmemAllocation() know explicitly about allocation of space for semaphores and spinlocks.) This change means an extra indirection to access the semaphore data, but since we only touch that when blocking or awakening a process, there shouldn't be any meaningful performance penalty. Moreover, at least for the unnamed-POSIX case on Linux, the sem_t type is quite a bit wider than a pointer, so this reduces sizeof(PGPROC) which seems like a good thing. For the named-POSIX API, there's effectively no change: the PGPROC.sem field was and still is a pointer to something returned by sem_open() in the postmaster's memory space. Document and check the pre-existing limitation that this case can't work in EXEC_BACKEND mode. It did not seem worth unifying the Windows semaphore ABI with the Unix cases, since there's no likelihood of needing ABI compatibility much less runtime switching across those cases. However, we can simplify the Windows code a bit if we define PGSemaphore as being directly a HANDLE, rather than pointer to HANDLE, so let's do that while we're here. (This also ends up being no change in what's physically stored in PGPROC.sem. We're just moving the HANDLE fetch from callees to callers.) It would take a bunch of additional code shuffling to get to the point of actually choosing a semaphore API at postmaster start, but the effects of that would now be localized in the port/XXX_sema.c files, so it seems like fit material for a separate patch. The need for it is unproven as yet, anyhow, whereas the ABI risk to extensions seems real enough. Discussion: https://postgr.es/m/4029.1481413370@sss.pgh.pa.us
2016-12-12 19:32:10 +01:00
PGSemaphore *SpinlockSemaArray;
Reduce the number of semaphores used under --disable-spinlocks. Instead of allocating a semaphore from the operating system for every spinlock, allocate a fixed number of semaphores (by default, 1024) from the operating system and multiplex all the spinlocks that get created onto them. This could self-deadlock if a process attempted to acquire more than one spinlock at a time, but since processes aren't supposed to execute anything other than short stretches of straight-line code while holding a spinlock, that shouldn't happen. One motivation for this change is that, with the introduction of dynamic shared memory, it may be desirable to create spinlocks that last for less than the lifetime of the server. Without this change, attempting to use such facilities under --disable-spinlocks would quickly exhaust any supply of available semaphores. Quite apart from that, it's desirable to contain the quantity of semaphores needed to run the server simply on convenience grounds, since using too many may make it harder to get PostgreSQL running on a new platform, which is mostly the point of --disable-spinlocks in the first place. Patch by me; review by Tom Lane.
2014-01-09 00:49:14 +01:00
#endif
Change the way that LWLocks for extensions are allocated. The previous RequestAddinLWLocks() method had several disadvantages. First, the locks would be in the main tranche; we've recently decided that it's useful for LWLocks used for separate purposes to have separate tranche IDs. Second, there wasn't any correlation between what code called RequestAddinLWLocks() and what code called LWLockAssign(); when multiple modules are in use, it could become quite difficult to troubleshoot problems where LWLockAssign() ran out of locks. To fix, create a concept of named LWLock tranches which can be used either by extension or by core code. Amit Kapila and Robert Haas
2016-02-04 22:43:04 +01:00
int NamedLWLockTrancheRequests;
NamedLWLockTranche *NamedLWLockTrancheArray;
Fix compiler warning in EXEC_BACKEND builds. Per a report by Rajeev Rastogi.
2014-01-29 05:35:50 +01:00
LWLockPadded *MainLWLockArray;
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
slock_t *ProcStructLock;
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
PROC_HDR *ProcGlobal;
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
PGPROC *AuxiliaryProcs;
Move "hot" members of PGPROC into a separate PGXACT array. This speeds up snapshot-taking and reduces ProcArrayLock contention. Also, the PGPROC (and PGXACT) structures used by two-phase commit are now allocated as part of the main array, rather than in a separate array, and we keep ProcArray sorted in pointer order. These changes are intended to minimize the number of cache lines that must be pulled in to take a snapshot, and testing shows a substantial increase in performance on both read and write workloads at high concurrencies. Pavan Deolasee, Heikki Linnakangas, Robert Haas
2011-11-25 14:02:10 +01:00
PGPROC *PreparedXactProcs;
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
PMSignalData *PMSignalState;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
InheritableSocket pgStatSock;
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
pid_t PostmasterPid;
Clean up the rather historically encumbered interface to now() and current time: provide a GetCurrentTimestamp() function that returns current time in the form of a TimestampTz, instead of separate time_t and microseconds fields. This is what all the callers really want anyway, and it eliminates low-level dependencies on AbsoluteTime, which is a deprecated datatype that will have to disappear eventually.
2005-06-30 00:51:57 +02:00
TimestampTz PgStartTime;
Add pg_conf_load_time() function to report when the Postgres configuration files were last loaded. George Gensure
2008-05-04 23:13:36 +02:00
TimestampTz PgReloadTime;
Fix syslogger so that log_truncate_on_rotation works in the first rotation. In the original coding of the log rotation stuff, we did not bother to make the truncation logic work for the very first rotation after postmaster start (or after a syslogger crash and restart). It just always appended in that case. It did not seem terribly important at the time, but we've recently had two separate complaints from people who expected it to work unsurprisingly. (Both users tend to restart the postmaster about as often as a log rotation is configured to happen, which is maybe not typical use, but still...) Since the initial log file is opened in the postmaster, fixing this requires passing down some more state to the syslogger child process. It's always been like this, so back-patch to all supported branches.
2012-07-31 20:36:54 +02:00
pg_time_t first_syslogger_file_time;
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
bool redirection_done;
Fix pg_upgrade for EXEC_BACKEND builds (e.g. Windows) by properly passing the -b/binary-upgrade flag. Backpatch to 9.1.X.
2011-09-29 23:20:53 +02:00
bool IsBinaryUpgrade;
Inherit max_safe_fds to child processes in EXEC_BACKEND mode. Postmaster sets max_safe_fds by testing how many open file descriptors it can open, and that is normally inherited by all child processes at fork(). Not so on EXEC_BACKEND, ie. Windows, however. Because of that, we effectively ignored max_files_per_process on Windows, and always assumed a conservative default of 32 simultaneous open files. That could have an impact on performance, if you need to access a lot of different files in a query. After this patch, the value is passed to child processes by save/restore_backend_variables() among many other global variables. It has been like this forever, but given the lack of complaints about it, I'm not backpatching this.
2012-03-29 07:19:11 +02:00
int max_safe_fds;
Fix background workers for EXEC_BACKEND Commit da07a1e8 was broken for EXEC_BACKEND because I failed to realize that the MaxBackends recomputation needed to be duplicated by subprocesses in SubPostmasterMain. However, instead of having the value be recomputed at all, it's better to assign the correct value at postmaster initialization time, and have it be propagated to exec'ed backends via BackendParameters. MaxBackends stays as zero until after modules in shared_preload_libraries have had a chance to register bgworkers, since the value is going to be untrustworthy till that's finished. Heikki Linnakangas and Álvaro Herrera
2013-01-02 16:01:14 +01:00
int MaxBackends;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#ifdef WIN32
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
HANDLE PostmasterHandle;
HANDLE initial_signal_pipe;
HANDLE syslogPipe[2];
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#else
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
int postmaster_alive_fds[2];
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
int syslogPipe[2];
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#endif
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
char my_exec_path[MAXPGPATH];
char pkglib_path[MAXPGPATH];
char ExtraOptions[MAXPGPATH];
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
} BackendParameters;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
static void read_backend_variables(char *id, Port *port);
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
static void restore_backend_variables(BackendParameters *param, Port *port);
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#ifndef WIN32
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
static bool save_backend_variables(BackendParameters *param, Port *port);
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#else
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
static bool save_backend_variables(BackendParameters *param, Port *port,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
HANDLE childProcess, pid_t childPid);
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#endif
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
static void ShmemBackendArrayAdd(Backend *bn);
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
static void ShmemBackendArrayRemove(Backend *bn);
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
#endif /* EXEC_BACKEND */
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
#define StartupDataBase() StartChildProcess(StartupProcess)
#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
#define StartWalWriter() StartChildProcess(WalWriterProcess)
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
Adjust elog.c so that elog(FATAL) exits (including cases where ERROR is promoted to FATAL) end in exit(1) not exit(0). Then change the postmaster to allow exit(1) without a system-wide panic, but not for the startup subprocess or the bgwriter. There were a couple of places that were using exit(1) to deliberately force a system-wide panic; adjust these to be exit(2) instead. This fixes the problem noted back in July that if the startup process exits with elog(ERROR), the postmaster would think everything is hunky-dory and proceed to start up. Alternative solutions such as trying to run the entire startup process as a critical section seem less clean, primarily because of the fact that a fair amount of startup code is shared by all postmaster children in the EXEC_BACKEND case. We'd need an ugly special case somewhere near the head of main.c to make it work if it's the child process's responsibility to determine what happens; and what's the point when the postmaster already treats different children differently?
2006-11-21 01:49:55 +01:00
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
#define EXIT_STATUS_1(st) (WIFEXITED(st) && WEXITSTATUS(st) == 1)
action_at_recovery_target recovery config option action_at_recovery_target = pause | promote | shutdown Petr Jelinek Reviewed by Muhammad Asif Naeem, Fujji Masao and Simon Riggs
2014-11-25 21:13:30 +01:00
#define EXIT_STATUS_3(st) (WIFEXITED(st) && WEXITSTATUS(st) == 3)
Adjust elog.c so that elog(FATAL) exits (including cases where ERROR is promoted to FATAL) end in exit(1) not exit(0). Then change the postmaster to allow exit(1) without a system-wide panic, but not for the startup subprocess or the bgwriter. There were a couple of places that were using exit(1) to deliberately force a system-wide panic; adjust these to be exit(2) instead. This fixes the problem noted back in July that if the startup process exits with elog(ERROR), the postmaster would think everything is hunky-dory and proceed to start up. Alternative solutions such as trying to run the entire startup process as a critical section seem less clean, primarily because of the fact that a fair amount of startup code is shared by all postmaster children in the EXEC_BACKEND case. We'd need an ugly special case somewhere near the head of main.c to make it work if it's the child process's responsibility to determine what happens; and what's the point when the postmaster already treats different children differently?
2006-11-21 01:49:55 +01:00
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
#ifndef WIN32
/*
* File descriptors for pipe used to monitor if postmaster is alive.
* First is POSTMASTER_FD_WATCH, second is POSTMASTER_FD_OWN.
*/
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
int postmaster_alive_fds[2] = {-1, -1};
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
#else
/* Process handle of postmaster used for the same purpose on Windows */
HANDLE PostmasterHandle;
#endif
Add Rendezvous support to postmaster, from Chris Campbell
2003-06-11 08:56:07 +02:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/*
* Postmaster main entry point
*/
Unify calling conventions for postgres/postmaster sub-main functions There was a wild mix of calling conventions: Some were declared to return void and didn't return, some returned an int exit code, some claimed to return an exit code, which the callers checked, but actually never returned, and so on. Now all of these functions are declared to return void and decorated with attribute noreturn and don't return. That's easiest, and most code already worked that way.
2012-06-25 20:25:26 +02:00
void
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
PostmasterMain(int argc, char *argv[])
{
pgindent run.
2003-08-04 02:43:34 +02:00
int opt;
int status;
Whack some sense into the configuration-file-location patch. Refactor code into something reasonably understandable, cause use of the feature to not fail in standalone backends or in EXEC_BACKEND case, fix sloppy guc.c table entries, make the documentation minimally usable.
2004-10-08 03:36:36 +02:00
char *userDoption = NULL;
Code review for postmaster.pid contents changes. Fix broken test for pre-existing postmaster, caused by wrong code for appending lines to the lockfile; don't write a failed listen_address setting into the lockfile; don't arbitrarily change the location of the data directory in the lockfile compared to previous releases; provide more consistent and useful definitions of the socket path and listen_address entries; avoid assuming that pg_ctl has the same DEFAULT_PGSOCKET_DIR as the postmaster; assorted code style improvements.
2011-01-14 01:01:28 +01:00
bool listen_addr_saved = false;
pgindent run.
2003-08-04 02:43:34 +02:00
int i;
Mark some more variables as static or include the appropriate header Detected by clang's -Wmissing-variable-declarations. From: Andres Freund <andres@anarazel.de>
2014-02-09 03:21:46 +01:00
char *output_config_variable = NULL;
Code review for postmaster.pid contents changes. Fix broken test for pre-existing postmaster, caused by wrong code for appending lines to the lockfile; don't write a failed listen_address setting into the lockfile; don't arbitrarily change the location of the data directory in the lockfile compared to previous releases; provide more consistent and useful definitions of the socket path and listen_address entries; avoid assuming that pg_ctl has the same DEFAULT_PGSOCKET_DIR as the postmaster; assorted code style improvements.
2011-01-14 01:01:28 +01:00
Refactor pid, random seed and start time initialization. Background workers, including parallel workers, were generating the same sequence of numbers in random(). This showed up as DSM handle collisions when Parallel Hash created multiple segments, but any code that calls random() in background workers could be affected if it cares about different backends generating different numbers. Repair by making sure that all new processes initialize the seed at the same time as they set MyProcPid and MyStartTime in a new function InitProcessGlobals(), called by the postmaster, its children and also standalone processes. Also add a new high resolution MyStartTimestamp as a potentially useful by-product, and remove SessionStartTime from struct Port as it is now redundant. No back-patch for now, as the known consequences so far are just a bunch of harmless shm_open(O_EXCL) collisions. Author: Thomas Munro Reviewed-by: Tom Lane Discussion: https://postgr.es/m/CAEepm%3D2eJj_6%3DB%2B2tEpGu2nf1BjthCf9nXXUouYvJJ4C5WSwhg%40mail.gmail.com
2018-10-19 02:59:14 +02:00
InitProcessGlobals();
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
Refactor pid, random seed and start time initialization. Background workers, including parallel workers, were generating the same sequence of numbers in random(). This showed up as DSM handle collisions when Parallel Hash created multiple segments, but any code that calls random() in background workers could be affected if it cares about different backends generating different numbers. Repair by making sure that all new processes initialize the seed at the same time as they set MyProcPid and MyStartTime in a new function InitProcessGlobals(), called by the postmaster, its children and also standalone processes. Also add a new high resolution MyStartTimestamp as a potentially useful by-product, and remove SessionStartTime from struct Port as it is now redundant. No back-patch for now, as the known consequences so far are just a bunch of harmless shm_open(O_EXCL) collisions. Author: Thomas Munro Reviewed-by: Tom Lane Discussion: https://postgr.es/m/CAEepm%3D2eJj_6%3DB%2B2tEpGu2nf1BjthCf9nXXUouYvJJ4C5WSwhg%40mail.gmail.com
2018-10-19 02:59:14 +02:00
PostmasterPid = MyProcPid;
Move session_start out of MyProcPort stucture and make it a global called MyStartTime, so that we will be able to create a cookie for all processes for CSVlogs. It is set wherever MyProcPid is set. Take the opportunity to remove the now unnecessary session-only restriction on the %s and %c escapes in log_line_prefix.
2007-08-03 01:39:45 +02:00
In bootstrap and standalone-backend modes, do not sort LOG elevel out of order; the 'server log' output is actually client output in these scenarios and we ought to treat elevels the same way as in the client case. This allows initdb to not send backend stderr to /dev/null anymore, which makes it much more likely that people will notice problems during initdb.
2003-05-28 19:25:02 +02:00
IsPostmasterEnvironment = true;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Allow group access on PGDATA Allow the cluster to be optionally init'd with read access for the group. This means a relatively non-privileged user can perform a backup of the cluster without requiring write privileges, which enhances security. The mode of PGDATA is used to determine whether group permissions are enabled for directory and file creates. This method was chosen as it's simple and works well for the various utilities that write into PGDATA. Changing the mode of PGDATA manually will not automatically change the mode of all the files contained therein. If the user would like to enable group access on an existing cluster then changing the mode of all the existing files will be required. Note that pg_upgrade will automatically change the mode of all migrated files if the new cluster is init'd with the -g option. Tests are included for the backend and all the utilities which operate on the PG data directory to ensure that the correct mode is set based on the data directory permissions. Author: David Steele <david@pgmasters.net> Reviewed-By: Michael Paquier, with discussion amongst many others. Discussion: https://postgr.es/m/ad346fe6-b23e-59f1-ecb7-0e08390ad629%40pgmasters.net
2018-04-07 23:45:39 +02:00
* We should not be creating any files or directories before we check the
* data directory (see checkDataDir()), but just in case set the umask to
* the most restrictive (owner-only) permissions.
*
* checkDataDir() will reset the umask based on the data directory
* permissions.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Refactor dir/file permissions Consolidate directory and file create permissions for tools which work with the PG data directory by adding a new module (common/file_perm.c) that contains variables (pg_file_create_mode, pg_dir_create_mode) and constants to initialize them (0600 for files and 0700 for directories). Convert mkdir() calls in the backend to MakePGDirectory() if the original call used default permissions (always the case for regular PG directories). Add tests to make sure permissions in PGDATA are set correctly by the tools which modify the PG data directory. Authors: David Steele <david@pgmasters.net>, Adam Brightwell <adam.brightwell@crunchydata.com> Reviewed-By: Michael Paquier, with discussion amongst many others. Discussion: https://postgr.es/m/ad346fe6-b23e-59f1-ecb7-0e08390ad629%40pgmasters.net
2018-04-07 23:45:39 +02:00
umask(PG_MODE_MASK_OWNER);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* By default, palloc() requests in the postmaster will be allocated in
* the PostmasterContext, which is space that can be recycled by backends.
* Allocated data that needs to be available to backends should be
* allocated in TopMemoryContext.
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
*/
PostmasterContext = AllocSetContextCreate(TopMemoryContext,
"Postmaster",
Add macros to make AllocSetContextCreate() calls simpler and safer. I found that half a dozen (nearly 5%) of our AllocSetContextCreate calls had typos in the context-sizing parameters. While none of these led to especially significant problems, they did create minor inefficiencies, and it's now clear that expecting people to copy-and-paste those calls accurately is not a great idea. Let's reduce the risk of future errors by introducing single macros that encapsulate the common use-cases. Three such macros are enough to cover all but two special-purpose contexts; those two calls can be left as-is, I think. While this patch doesn't in itself improve matters for third-party extensions, it doesn't break anything for them either, and they can gradually adopt the simplified notation over time. In passing, change TopMemoryContext to use the default allocation parameters. Formerly it could only be extended 8K at a time. That was probably reasonable when this code was written; but nowadays we create many more contexts than we did then, so that it's not unusual to have a couple hundred K in TopMemoryContext, even without considering various dubious code that sticks other things there. There seems no good reason not to let it use growing blocks like most other contexts. Back-patch to 9.6, mostly because that's still close enough to HEAD that it's easy to do so, and keeping the branches in sync can be expected to avoid some future back-patching pain. The bugs fixed by these changes don't seem to be significant enough to justify fixing them further back. Discussion: <21072.1472321324@sss.pgh.pa.us>
2016-08-27 23:50:38 +02:00
ALLOCSET_DEFAULT_SIZES);
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
MemoryContextSwitchTo(PostmasterContext);
Install some simple defenses in postmaster startup to help ensure a useful error message if the installation directory layout is messed up (or at least, something more useful than the behavior exhibited in bug #4787). During postmaster startup, check that get_pkglib_path resolves as a readable directory; and if ParseTzFile() fails to open the expected timezone abbreviation file, check the possibility that the directory is missing rather than just the specified file. In case of either failure, issue a hint suggesting that the installation is broken. These two checks cover the lib/ and share/ trees of a full installation, which should take care of most scenarios where a sysadmin decides to get cute.
2009-05-03 00:02:37 +02:00
/* Initialize paths to installation files */
getInstallationPaths(argv[0]);
Add get_pkglib_path to postmaster.c
2004-05-19 21:11:25 +02:00
Block signals earlier during postmaster startup. Formerly, we set up the postmaster's signal handling only when we were about to start launching subprocesses. This is a bad idea though, as it means that for example a SIGINT arriving before that will kill the postmaster instantly, perhaps leaving lockfiles, socket files, shared memory, etc laying about. We'd rather that such a signal caused orderly postmaster termination including releasing of those resources. A simple fix is to move the PostmasterMain stanza that initializes signal handling to an earlier point, before we've created any such resources. Then, an early-arriving signal will be blocked until we're ready to deal with it in the usual way. (The only part that really needs to be moved up is blocking of signals, but it seems best to keep the signal handler installation calls together with that; for one thing this ensures the kernel won't drop any signals we wished to get. The handlers won't get invoked in any case until we unblock signals in ServerLoop.) Per a report from MauMau. He proposed changing the way "pg_ctl stop" works to deal with this, but that'd just be masking one symptom not fixing the core issue. It's been like this since forever, so back-patch to all supported branches.
2014-04-06 00:16:08 +02:00
/*
* Set up signal handlers for the postmaster process.
*
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
* In the postmaster, we use pqsignal_pm() rather than pqsignal() (which
* is used by all child processes and client processes). That has a
* couple of special behaviors:
*
* 1. Except on Windows, we tell sigaction() to block all signals for the
* duration of the signal handler. This is faster than our old approach
* of blocking/unblocking explicitly in the signal handler, and it should
* also prevent excessive stack consumption if signals arrive quickly.
*
* 2. We do not set the SA_RESTART flag. This is because signals will be
* blocked at all times except when ServerLoop is waiting for something to
* happen, and during that window, we want signals to exit the select(2)
* wait so that ServerLoop can respond if anything interesting happened.
* On some platforms, signals marked SA_RESTART would not cause the
* select() wait to end.
*
* Child processes will generally want SA_RESTART, so pqsignal() sets that
* flag. We expect children to set up their own handlers before
* unblocking signals.
Run the postmaster's signal handlers without SA_RESTART. The postmaster keeps signals blocked everywhere except while waiting for something to happen in ServerLoop(). The code expects that the select(2) will be cancelled with EINTR if an interrupt occurs; without that, followup actions that should be performed by ServerLoop() itself will be delayed. However, some platforms interpret the SA_RESTART signal flag as meaning that they should restart rather than cancel the select(2). Worse yet, some of them restart it with the original timeout delay, meaning that a steady stream of signal interrupts can prevent ServerLoop() from iterating at all if there are no incoming connection requests. Observable symptoms of this, on an affected platform such as HPUX 10, include extremely slow parallel query startup (possibly as much as 30 seconds) and failure to update timestamps on the postmaster's sockets and lockfiles when no new connections arrive for a long time. We can fix this by running the postmaster's signal handlers without SA_RESTART. That would be quite a scary change if the range of code where signals are accepted weren't so tiny, but as it is, it seems safe enough. (Note that postmaster children do, and must, reset all the handlers before unblocking signals; so this change should not affect any child process.) There is talk of rewriting the postmaster to use a WaitEventSet and not do signal response work in signal handlers, at which point it might be appropriate to revert this patch. But that's not happening before v11 at the earliest. Back-patch to 9.6. The problem exists much further back, but the worst symptom arises only in connection with parallel query, so it does not seem worth taking any portability risks in older branches. Discussion: https://postgr.es/m/9205.1492833041@sss.pgh.pa.us
2017-04-24 19:00:23 +02:00
*
Block signals earlier during postmaster startup. Formerly, we set up the postmaster's signal handling only when we were about to start launching subprocesses. This is a bad idea though, as it means that for example a SIGINT arriving before that will kill the postmaster instantly, perhaps leaving lockfiles, socket files, shared memory, etc laying about. We'd rather that such a signal caused orderly postmaster termination including releasing of those resources. A simple fix is to move the PostmasterMain stanza that initializes signal handling to an earlier point, before we've created any such resources. Then, an early-arriving signal will be blocked until we're ready to deal with it in the usual way. (The only part that really needs to be moved up is blocking of signals, but it seems best to keep the signal handler installation calls together with that; for one thing this ensures the kernel won't drop any signals we wished to get. The handlers won't get invoked in any case until we unblock signals in ServerLoop.) Per a report from MauMau. He proposed changing the way "pg_ctl stop" works to deal with this, but that'd just be masking one symptom not fixing the core issue. It's been like this since forever, so back-patch to all supported branches.
2014-04-06 00:16:08 +02:00
* CAUTION: when changing this list, check for side-effects on the signal
* handling setup of child processes. See tcop/postgres.c,
* bootstrap/bootstrap.c, postmaster/bgwriter.c, postmaster/walwriter.c,
* postmaster/autovacuum.c, postmaster/pgarch.c, postmaster/pgstat.c,
* postmaster/syslogger.c, postmaster/bgworker.c and
* postmaster/checkpointer.c.
*/
pqinitmask();
PG_SETMASK(&BlockSig);
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
pqsignal_pm(SIGHUP, SIGHUP_handler); /* reread config file and have
* children do same */
pqsignal_pm(SIGINT, pmdie); /* send SIGTERM and shut down */
pqsignal_pm(SIGQUIT, pmdie); /* send SIGQUIT and die */
pqsignal_pm(SIGTERM, pmdie); /* wait for children and shut down */
pqsignal_pm(SIGALRM, SIG_IGN); /* ignored */
pqsignal_pm(SIGPIPE, SIG_IGN); /* ignored */
pqsignal_pm(SIGUSR1, sigusr1_handler); /* message from child process */
pqsignal_pm(SIGUSR2, dummy_handler); /* unused, reserve for children */
pqsignal_pm(SIGCHLD, reaper); /* handle child termination */
Leave SIGTTIN/SIGTTOU signal handling alone in postmaster child processes. For reasons lost in the mists of time, most postmaster child processes reset SIGTTIN/SIGTTOU signal handling to SIG_DFL, with the major exception that backend sessions do not. It seems like a pretty bad idea for any postmaster children to do that: if stderr is connected to the terminal, and the user has put the postmaster in background, any log output would result in the child process freezing up. Hence, switch them all to doing what backends do, ie, nothing. This allows them to inherit the postmaster's SIG_IGN setting. On the other hand, manually-launched processes such as standalone backends will have default processing, which seems fine. In passing, also remove useless resets of SIGCONT and SIGWINCH signal processing. Perhaps the postmaster once changed those to something besides SIG_DFL, but it doesn't now, so these are just wasted (and confusing) syscalls. Basically, this propagates the changes made in commit 8e2998d8a from backends to other postmaster children. Probably the only reason these calls now exist elsewhere is that I missed changing pgstat.c along with postgres.c at the time. Given the lack of field complaints that can be traced to this, I don't presently feel a need to back-patch. Discussion: https://postgr.es/m/5627.1542477392@sss.pgh.pa.us
2018-11-17 22:23:55 +01:00
/*
* No other place in Postgres should touch SIGTTIN/SIGTTOU handling. We
* ignore those signals in a postmaster environment, so that there is no
* risk of a child process freezing up due to writing to stderr. But for
* a standalone backend, their default handling is reasonable. Hence, all
* child processes should just allow the inherited settings to stand.
*/
Avoid defining SIGTTIN/SIGTTOU on Windows. Setting them to SIG_IGN seems unlikely to have any beneficial effect on that platform, and given the signal numbering collision with SIGABRT, it could easily have bad effects. Given the lack of field complaints that can be traced to this, I don't presently feel a need to back-patch. Discussion: https://postgr.es/m/5627.1542477392@sss.pgh.pa.us
2018-11-17 22:31:07 +01:00
#ifdef SIGTTIN
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
pqsignal_pm(SIGTTIN, SIG_IGN); /* ignored */
Avoid defining SIGTTIN/SIGTTOU on Windows. Setting them to SIG_IGN seems unlikely to have any beneficial effect on that platform, and given the signal numbering collision with SIGABRT, it could easily have bad effects. Given the lack of field complaints that can be traced to this, I don't presently feel a need to back-patch. Discussion: https://postgr.es/m/5627.1542477392@sss.pgh.pa.us
2018-11-17 22:31:07 +01:00
#endif
#ifdef SIGTTOU
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
pqsignal_pm(SIGTTOU, SIG_IGN); /* ignored */
Avoid defining SIGTTIN/SIGTTOU on Windows. Setting them to SIG_IGN seems unlikely to have any beneficial effect on that platform, and given the signal numbering collision with SIGABRT, it could easily have bad effects. Given the lack of field complaints that can be traced to this, I don't presently feel a need to back-patch. Discussion: https://postgr.es/m/5627.1542477392@sss.pgh.pa.us
2018-11-17 22:31:07 +01:00
#endif
Leave SIGTTIN/SIGTTOU signal handling alone in postmaster child processes. For reasons lost in the mists of time, most postmaster child processes reset SIGTTIN/SIGTTOU signal handling to SIG_DFL, with the major exception that backend sessions do not. It seems like a pretty bad idea for any postmaster children to do that: if stderr is connected to the terminal, and the user has put the postmaster in background, any log output would result in the child process freezing up. Hence, switch them all to doing what backends do, ie, nothing. This allows them to inherit the postmaster's SIG_IGN setting. On the other hand, manually-launched processes such as standalone backends will have default processing, which seems fine. In passing, also remove useless resets of SIGCONT and SIGWINCH signal processing. Perhaps the postmaster once changed those to something besides SIG_DFL, but it doesn't now, so these are just wasted (and confusing) syscalls. Basically, this propagates the changes made in commit 8e2998d8a from backends to other postmaster children. Probably the only reason these calls now exist elsewhere is that I missed changing pgstat.c along with postgres.c at the time. Given the lack of field complaints that can be traced to this, I don't presently feel a need to back-patch. Discussion: https://postgr.es/m/5627.1542477392@sss.pgh.pa.us
2018-11-17 22:23:55 +01:00
Block signals earlier during postmaster startup. Formerly, we set up the postmaster's signal handling only when we were about to start launching subprocesses. This is a bad idea though, as it means that for example a SIGINT arriving before that will kill the postmaster instantly, perhaps leaving lockfiles, socket files, shared memory, etc laying about. We'd rather that such a signal caused orderly postmaster termination including releasing of those resources. A simple fix is to move the PostmasterMain stanza that initializes signal handling to an earlier point, before we've created any such resources. Then, an early-arriving signal will be blocked until we're ready to deal with it in the usual way. (The only part that really needs to be moved up is blocking of signals, but it seems best to keep the signal handler installation calls together with that; for one thing this ensures the kernel won't drop any signals we wished to get. The handlers won't get invoked in any case until we unblock signals in ServerLoop.) Per a report from MauMau. He proposed changing the way "pg_ctl stop" works to deal with this, but that'd just be masking one symptom not fixing the core issue. It's been like this since forever, so back-patch to all supported branches.
2014-04-06 00:16:08 +02:00
/* ignore SIGXFSZ, so that ulimit violations work like disk full */
#ifdef SIGXFSZ
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
pqsignal_pm(SIGXFSZ, SIG_IGN); /* ignored */
Block signals earlier during postmaster startup. Formerly, we set up the postmaster's signal handling only when we were about to start launching subprocesses. This is a bad idea though, as it means that for example a SIGINT arriving before that will kill the postmaster instantly, perhaps leaving lockfiles, socket files, shared memory, etc laying about. We'd rather that such a signal caused orderly postmaster termination including releasing of those resources. A simple fix is to move the PostmasterMain stanza that initializes signal handling to an earlier point, before we've created any such resources. Then, an early-arriving signal will be blocked until we're ready to deal with it in the usual way. (The only part that really needs to be moved up is blocking of signals, but it seems best to keep the signal handler installation calls together with that; for one thing this ensures the kernel won't drop any signals we wished to get. The handlers won't get invoked in any case until we unblock signals in ServerLoop.) Per a report from MauMau. He proposed changing the way "pg_ctl stop" works to deal with this, but that'd just be masking one symptom not fixing the core issue. It's been like this since forever, so back-patch to all supported branches.
2014-04-06 00:16:08 +02:00
#endif
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
/*
* Options setup
*/
Merge the last few variable.c configuration variables into the generic GUC support. It's now possible to set datestyle, timezone, and client_encoding from postgresql.conf and per-database or per-user settings. Also, implement rollback of SET commands that occur in a transaction that later fails. Create a SET LOCAL var = value syntax that sets the variable only for the duration of the current transaction. All per previous discussions in pghackers.
2002-05-17 03:19:19 +02:00
InitializeGUCOptions();
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
The heralded `Grand Unified Configuration scheme' (GUC) That means you can now set your options in either or all of $PGDATA/configuration, some postmaster option (--enable-fsync=off), or set a SET command. The list of options is in backend/utils/misc/guc.c, documentation will be written post haste. pg_options is gone, so is that pq_geqo config file. Also removed were backend -K, -Q, and -T options (no longer applicable, although -d0 does the same as -Q). Added to configure an --enable-syslog option. changed all callers from TPRINTF to elog(DEBUG)
2000-05-31 02:28:42 +02:00
opterr = 1;
Proper fix for glibc getopt() botch. Surprising we did not see this before.
2001-10-19 22:47:09 +02:00
Fix erroneous implementation of -s in postmaster.c (the switch doesn't take an optarg). Add some comments noting that code in three different files has to be kept in sync. Fix erroneous description of -S switch (it sets work_mem not silent_mode), and do some light copy-editing elsewhere in postgres-ref.
2007-01-04 01:57:51 +01:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Parse command-line options. CAUTION: keep this in sync with
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* tcop/postgres.c (the option sets should not conflict) and with the
* common help() function in main/main.c.
Fix erroneous implementation of -s in postmaster.c (the switch doesn't take an optarg). Add some comments noting that code in three different files has to be kept in sync. Fix erroneous description of -S switch (it sets work_mem not silent_mode), and do some light copy-editing elsewhere in postgres-ref.
2007-01-04 01:57:51 +01:00
*/
Don't allow to disable backend assertions via the debug_assertions GUC. The existance of the assert_enabled variable (backing the debug_assertions GUC) reduced the amount of knowledge some static code checkers (like coverity and various compilers) could infer from the existance of the assertion. That could have been solved by optionally removing the assertion_enabled variable from the Assert() et al macros at compile time when some special macro is defined, but the resulting complication doesn't seem to be worth the gain from having debug_assertions. Recompiling is fast enough. The debug_assertions GUC is still available, but readonly, as it's useful when diagnosing problems. The commandline/client startup option -A, which previously also allowed to enable/disable assertions, has been removed as it doesn't serve a purpose anymore. While at it, reduce code duplication in bufmgr.c and localbuf.c assertions checking for spurious buffer pins. That code had to be reindented anyway to cope with the assert_enabled removal.
2014-06-20 11:06:42 +02:00
while ((opt = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOo:Pp:r:S:sTt:W:-:")) != -1)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
switch (opt)
{
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
case 'B':
Save source of GUC settings, allowing different sources to be processed in any order without affecting results.
2002-02-23 02:31:37 +01:00
SetConfigOption("shared_buffers", optarg, PGC_POSTMASTER, PGC_S_ARGV);
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
break;
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
Add postmaster/postgres undocumented -b option for binary upgrades. This option turns off autovacuum, prevents non-super-user connections, and enables oid setting hooks in the backend. The code continues to use the old autoavacuum disable settings for servers with earlier catalog versions. This includes a catalog version bump to identify servers that support the -b option.
2011-04-25 18:00:21 +02:00
case 'b':
/* Undocumented flag used for binary upgrades */
IsBinaryUpgrade = true;
break;
Add postmaster -C option to query configuration parameters, and have pg_ctl use that to query the data directory for config-only installs. This fixes awkward or impossible pg_ctl operation for config-only installs.
2011-10-06 15:38:39 +02:00
case 'C':
In our source code, make a copy of getopt's 'optarg' string arguments, rather than just storing a pointer.
2012-10-12 19:35:40 +02:00
output_config_variable = strdup(optarg);
Add postmaster -C option to query configuration parameters, and have pg_ctl use that to query the data directory for config-only installs. This fixes awkward or impossible pg_ctl operation for config-only installs.
2011-10-06 15:38:39 +02:00
break;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
case 'D':
In our source code, make a copy of getopt's 'optarg' string arguments, rather than just storing a pointer.
2012-10-12 19:35:40 +02:00
userDoption = strdup(optarg);
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
break;
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
case 'd':
Remove GUC USERLIMIT variable category, making the affected variables plain SUSET instead. Also delay processing of options received in client connection request until after we know if the user is a superuser, so that SUSET values can be set that way by legitimate superusers. Per recent discussion.
2004-11-14 20:35:35 +01:00
set_debug_options(atoi(optarg), PGC_POSTMASTER, PGC_S_ARGV);
break;
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
case 'E':
SetConfigOption("log_statement", "all", PGC_POSTMASTER, PGC_S_ARGV);
break;
case 'e':
SetConfigOption("datestyle", "euro", PGC_POSTMASTER, PGC_S_ARGV);
break;
Remove NT-specific file open defines by defining our own open macros for "rb" and "wb".
2000-06-02 17:57:44 +02:00
case 'F':
Save source of GUC settings, allowing different sources to be processed in any order without affecting results.
2002-02-23 02:31:37 +01:00
SetConfigOption("fsync", "false", PGC_POSTMASTER, PGC_S_ARGV);
Remove NT-specific file open defines by defining our own open macros for "rb" and "wb".
2000-06-02 17:57:44 +02:00
break;
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
case 'f':
if (!set_plan_disabling_options(optarg, PGC_POSTMASTER, PGC_S_ARGV))
{
write_stderr("%s: invalid argument for option -f: \"%s\"\n",
progname, optarg);
ExitPostmaster(1);
}
break;
UUNET is looking into offering PostgreSQL as a part of a managed web hosting product, on both shared and dedicated machines. We currently offer Oracle and MySQL, and it would be a nice middle-ground. However, as shipped, PostgreSQL lacks the following features we need that MySQL has: 1. The ability to listen only on a particular IP address. Each hosting customer has their own IP address, on which all of their servers (http, ftp, real media, etc.) run. 2. The ability to place the Unix-domain socket in a mode 700 directory. This allows us to automatically create an empty database, with an empty DBA password, for new or upgrading customers without having to interactively set a DBA password and communicate it to (or from) the customer. This in turn cuts down our install and upgrade times. 3. The ability to connect to the Unix-domain socket from within a change-rooted environment. We run CGI programs chrooted to the user's home directory, which is another reason why we need to be able to specify where the Unix-domain socket is, instead of /tmp. 4. The ability to, if run as root, open a pid file in /var/run as root, and then setuid to the desired user. (mysqld -u can almost do this; I had to patch it, too). The patch below fixes problem 1-3. I plan to address #4, also, but haven't done so yet. These diffs are big enough that they should give the PG development team something to think about in the meantime :-) Also, I'm about to leave for 2 weeks' vacation, so I thought I'd get out what I have, which works (for the problems it tackles), now. With these changes, we can set up and run PostgreSQL with scripts the same way we can with apache or proftpd or mysql. In summary, this patch makes the following enhancements: 1. Adds an environment variable PGUNIXSOCKET, analogous to MYSQL_UNIX_PORT, and command line options -k --unix-socket to the relevant programs. 2. Adds a -h option to postmaster to set the hostname or IP address to listen on instead of the default INADDR_ANY. 3. Extends some library interfaces to support the above. 4. Fixes a few memory leaks in PQconnectdb(). The default behavior is unchanged from stock 7.0.2; if you don't use any of these new features, they don't change the operation. David J. MacKenzie
2000-11-13 16:18:15 +01:00
case 'h':
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
SetConfigOption("listen_addresses", optarg, PGC_POSTMASTER, PGC_S_ARGV);
UUNET is looking into offering PostgreSQL as a part of a managed web hosting product, on both shared and dedicated machines. We currently offer Oracle and MySQL, and it would be a nice middle-ground. However, as shipped, PostgreSQL lacks the following features we need that MySQL has: 1. The ability to listen only on a particular IP address. Each hosting customer has their own IP address, on which all of their servers (http, ftp, real media, etc.) run. 2. The ability to place the Unix-domain socket in a mode 700 directory. This allows us to automatically create an empty database, with an empty DBA password, for new or upgrading customers without having to interactively set a DBA password and communicate it to (or from) the customer. This in turn cuts down our install and upgrade times. 3. The ability to connect to the Unix-domain socket from within a change-rooted environment. We run CGI programs chrooted to the user's home directory, which is another reason why we need to be able to specify where the Unix-domain socket is, instead of /tmp. 4. The ability to, if run as root, open a pid file in /var/run as root, and then setuid to the desired user. (mysqld -u can almost do this; I had to patch it, too). The patch below fixes problem 1-3. I plan to address #4, also, but haven't done so yet. These diffs are big enough that they should give the PG development team something to think about in the meantime :-) Also, I'm about to leave for 2 weeks' vacation, so I thought I'd get out what I have, which works (for the problems it tackles), now. With these changes, we can set up and run PostgreSQL with scripts the same way we can with apache or proftpd or mysql. In summary, this patch makes the following enhancements: 1. Adds an environment variable PGUNIXSOCKET, analogous to MYSQL_UNIX_PORT, and command line options -k --unix-socket to the relevant programs. 2. Adds a -h option to postmaster to set the hostname or IP address to listen on instead of the default INADDR_ANY. 3. Extends some library interfaces to support the above. 4. Fixes a few memory leaks in PQconnectdb(). The default behavior is unchanged from stock 7.0.2; if you don't use any of these new features, they don't change the operation. David J. MacKenzie
2000-11-13 16:18:15 +01:00
break;
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
Fix case issues with quotes.
1997-11-10 06:10:50 +01:00
case 'i':
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
SetConfigOption("listen_addresses", "*", PGC_POSTMASTER, PGC_S_ARGV);
Cleanup -is flag to -l for SSL. Another PERL variable name fix. Clean up debugging options for postmaster and postgres programs. postmaster -d is no longer optional. Documentation updates.
1999-10-08 06:28:57 +02:00
break;
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
case 'j':
/* only used by interactive backend */
break;
UUNET is looking into offering PostgreSQL as a part of a managed web hosting product, on both shared and dedicated machines. We currently offer Oracle and MySQL, and it would be a nice middle-ground. However, as shipped, PostgreSQL lacks the following features we need that MySQL has: 1. The ability to listen only on a particular IP address. Each hosting customer has their own IP address, on which all of their servers (http, ftp, real media, etc.) run. 2. The ability to place the Unix-domain socket in a mode 700 directory. This allows us to automatically create an empty database, with an empty DBA password, for new or upgrading customers without having to interactively set a DBA password and communicate it to (or from) the customer. This in turn cuts down our install and upgrade times. 3. The ability to connect to the Unix-domain socket from within a change-rooted environment. We run CGI programs chrooted to the user's home directory, which is another reason why we need to be able to specify where the Unix-domain socket is, instead of /tmp. 4. The ability to, if run as root, open a pid file in /var/run as root, and then setuid to the desired user. (mysqld -u can almost do this; I had to patch it, too). The patch below fixes problem 1-3. I plan to address #4, also, but haven't done so yet. These diffs are big enough that they should give the PG development team something to think about in the meantime :-) Also, I'm about to leave for 2 weeks' vacation, so I thought I'd get out what I have, which works (for the problems it tackles), now. With these changes, we can set up and run PostgreSQL with scripts the same way we can with apache or proftpd or mysql. In summary, this patch makes the following enhancements: 1. Adds an environment variable PGUNIXSOCKET, analogous to MYSQL_UNIX_PORT, and command line options -k --unix-socket to the relevant programs. 2. Adds a -h option to postmaster to set the hostname or IP address to listen on instead of the default INADDR_ANY. 3. Extends some library interfaces to support the above. 4. Fixes a few memory leaks in PQconnectdb(). The default behavior is unchanged from stock 7.0.2; if you don't use any of these new features, they don't change the operation. David J. MacKenzie
2000-11-13 16:18:15 +01:00
case 'k':
Support having multiple Unix-domain sockets per postmaster. Replace unix_socket_directory with unix_socket_directories, which is a list of socket directories, and adjust postmaster's code to allow zero or more Unix-domain sockets to be created. This is mostly a straightforward change, but since the Unix sockets ought to be created after the TCP/IP sockets for safety reasons (better chance of detecting a port number conflict), AddToDataDirLockFile needs to be fixed to support out-of-order updates of data directory lockfile lines. That's a change that had been foreseen to be necessary someday anyway. Honza Horak, reviewed and revised by Tom Lane
2012-08-10 23:26:44 +02:00
SetConfigOption("unix_socket_directories", optarg, PGC_POSTMASTER, PGC_S_ARGV);
UUNET is looking into offering PostgreSQL as a part of a managed web hosting product, on both shared and dedicated machines. We currently offer Oracle and MySQL, and it would be a nice middle-ground. However, as shipped, PostgreSQL lacks the following features we need that MySQL has: 1. The ability to listen only on a particular IP address. Each hosting customer has their own IP address, on which all of their servers (http, ftp, real media, etc.) run. 2. The ability to place the Unix-domain socket in a mode 700 directory. This allows us to automatically create an empty database, with an empty DBA password, for new or upgrading customers without having to interactively set a DBA password and communicate it to (or from) the customer. This in turn cuts down our install and upgrade times. 3. The ability to connect to the Unix-domain socket from within a change-rooted environment. We run CGI programs chrooted to the user's home directory, which is another reason why we need to be able to specify where the Unix-domain socket is, instead of /tmp. 4. The ability to, if run as root, open a pid file in /var/run as root, and then setuid to the desired user. (mysqld -u can almost do this; I had to patch it, too). The patch below fixes problem 1-3. I plan to address #4, also, but haven't done so yet. These diffs are big enough that they should give the PG development team something to think about in the meantime :-) Also, I'm about to leave for 2 weeks' vacation, so I thought I'd get out what I have, which works (for the problems it tackles), now. With these changes, we can set up and run PostgreSQL with scripts the same way we can with apache or proftpd or mysql. In summary, this patch makes the following enhancements: 1. Adds an environment variable PGUNIXSOCKET, analogous to MYSQL_UNIX_PORT, and command line options -k --unix-socket to the relevant programs. 2. Adds a -h option to postmaster to set the hostname or IP address to listen on instead of the default INADDR_ANY. 3. Extends some library interfaces to support the above. 4. Fixes a few memory leaks in PQconnectdb(). The default behavior is unchanged from stock 7.0.2; if you don't use any of these new features, they don't change the operation. David J. MacKenzie
2000-11-13 16:18:15 +01:00
break;
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
Cleanup -is flag to -l for SSL. Another PERL variable name fix. Clean up debugging options for postmaster and postgres programs. postmaster -d is no longer optional. Documentation updates.
1999-10-08 06:28:57 +02:00
case 'l':
Save source of GUC settings, allowing different sources to be processed in any order without affecting results.
2002-02-23 02:31:37 +01:00
SetConfigOption("ssl", "true", PGC_POSTMASTER, PGC_S_ARGV);
Add Unix domain socket support, from Goran Thyni, goran@bildbasen.se
1997-11-07 21:52:15 +01:00
break;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Allow maximum number of backends to be set at configure time (--with-maxbackends). Add a postmaster switch (-N backends) that allows the limit to be reduced at postmaster start time. (You can't increase it, sorry to say, because there are still some fixed-size arrays.) Grab the number of semaphores indicated by min(MAXBACKENDS, -N) at postmaster startup, so that this particular form of bogus configuration is exposed immediately rather than under heavy load.
1999-02-19 07:06:39 +01:00
case 'N':
Save source of GUC settings, allowing different sources to be processed in any order without affecting results.
2002-02-23 02:31:37 +01:00
SetConfigOption("max_connections", optarg, PGC_POSTMASTER, PGC_S_ARGV);
Allow maximum number of backends to be set at configure time (--with-maxbackends). Add a postmaster switch (-N backends) that allows the limit to be reduced at postmaster start time. (You can't increase it, sorry to say, because there are still some fixed-size arrays.) Grab the number of semaphores indicated by min(MAXBACKENDS, -N) at postmaster startup, so that this particular form of bogus configuration is exposed immediately rather than under heavy load.
1999-02-19 07:06:39 +01:00
break;
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
case 'n':
/* Don't reinit shared mem after abnormal exit */
postmaster/postgres options cleanup.
1998-05-27 20:32:05 +02:00
Reinit = false;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
break;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
case 'O':
SetConfigOption("allow_system_table_mods", "true", PGC_POSTMASTER, PGC_S_ARGV);
break;
case 'o':
/* Other options to pass to the backend on the command line */
Defend against overrun of ExtraOptions array --- strictly paranoia, since the person or script starting the postmaster has to be trusted anyway.
2004-07-11 01:29:16 +02:00
snprintf(ExtraOptions + strlen(ExtraOptions),
sizeof(ExtraOptions) - strlen(ExtraOptions),
" %s", optarg);
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
break;
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
case 'P':
SetConfigOption("ignore_system_indexes", "true", PGC_POSTMASTER, PGC_S_ARGV);
break;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
case 'p':
Save source of GUC settings, allowing different sources to be processed in any order without affecting results.
2002-02-23 02:31:37 +01:00
SetConfigOption("port", optarg, PGC_POSTMASTER, PGC_S_ARGV);
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
break;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
case 'r':
/* only used by single-user backend */
break;
case 'S':
SetConfigOption("work_mem", optarg, PGC_POSTMASTER, PGC_S_ARGV);
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
break;
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
case 's':
Fix erroneous implementation of -s in postmaster.c (the switch doesn't take an optarg). Add some comments noting that code in three different files has to be kept in sync. Fix erroneous description of -S switch (it sets work_mem not silent_mode), and do some light copy-editing elsewhere in postgres-ref.
2007-01-04 01:57:51 +01:00
SetConfigOption("log_statement_stats", "true", PGC_POSTMASTER, PGC_S_ARGV);
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
break;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
case 'T':
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* In the event that some backend dumps core, send SIGSTOP,
* rather than SIGQUIT, to all its peers. This lets the wily
* post_hacker collect core dumps from everyone.
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
*/
postmaster/postgres options cleanup.
1998-05-27 20:32:05 +02:00
SendStop = true;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
break;
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
case 't':
{
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
const char *tmp = get_stats_option_name(optarg);
if (tmp)
{
SetConfigOption(tmp, "true", PGC_POSTMASTER, PGC_S_ARGV);
}
else
{
write_stderr("%s: invalid argument for option -t: \"%s\"\n",
progname, optarg);
ExitPostmaster(1);
}
break;
Make all command-line options of postmaster and postgres the same. See http://archives.postgresql.org/pgsql-hackers/2006-01/msg00151.php for the complete plan.
2006-01-05 11:07:46 +01:00
}
case 'W':
SetConfigOption("post_auth_delay", optarg, PGC_POSTMASTER, PGC_S_ARGV);
break;
Add '-c name=val' flag for setting run-time parameters.
2000-11-08 18:57:46 +01:00
case 'c':
Remove NT-specific file open defines by defining our own open macros for "rb" and "wb".
2000-06-02 17:57:44 +02:00
case '-':
Add '-c name=val' flag for setting run-time parameters.
2000-11-08 18:57:46 +01:00
{
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
char *name,
*value;
ParseLongOption(optarg, &name, &value);
if (!value)
{
if (opt == '-')
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("--%s requires a value",
optarg)));
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
else
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("-c %s requires a value",
optarg)));
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
}
Save source of GUC settings, allowing different sources to be processed in any order without affecting results.
2002-02-23 02:31:37 +01:00
SetConfigOption(name, value, PGC_POSTMASTER, PGC_S_ARGV);
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
free(name);
if (value)
free(value);
break;
Add '-c name=val' flag for setting run-time parameters.
2000-11-08 18:57:46 +01:00
}
Move PGPORT envar handling to ResetAllOptions(). Improve long options parsing to not clobber the optarg string -- so that we can bring SetOptsFile() up to speed.
2000-07-03 22:46:10 +02:00
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
default:
Back out pg_autovacuum commit after cvs clean failure causes commit.
2004-07-21 22:34:50 +02:00
write_stderr("Try \"%s --help\" for more information.\n",
progname);
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
ExitPostmaster(1);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
}
Add startup-time check that -B is not unreasonably small for given number of backends (-N), per recent discussion in pghackers list.
1999-06-04 23:14:46 +02:00
Save source of GUC settings, allowing different sources to be processed in any order without affecting results.
2002-02-23 02:31:37 +01:00
/*
* Postmaster accepts no non-option switch arguments.
*/
if (optind < argc)
{
Back out pg_autovacuum commit after cvs clean failure causes commit.
2004-07-21 22:34:50 +02:00
write_stderr("%s: invalid argument: \"%s\"\n",
progname, argv[optind]);
write_stderr("Try \"%s --help\" for more information.\n",
progname);
Save source of GUC settings, allowing different sources to be processed in any order without affecting results.
2002-02-23 02:31:37 +01:00
ExitPostmaster(1);
}
Whack some sense into the configuration-file-location patch. Refactor code into something reasonably understandable, cause use of the feature to not fail in standalone backends or in EXEC_BACKEND case, fix sloppy guc.c table entries, make the documentation minimally usable.
2004-10-08 03:36:36 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Locate the proper configuration files and data directory, and read
* postgresql.conf for the first time.
Whack some sense into the configuration-file-location patch. Refactor code into something reasonably understandable, cause use of the feature to not fail in standalone backends or in EXEC_BACKEND case, fix sloppy guc.c table entries, make the documentation minimally usable.
2004-10-08 03:36:36 +02:00
*/
if (!SelectConfigFiles(userDoption, progname))
ExitPostmaster(2);
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
Add postmaster -C option to query configuration parameters, and have pg_ctl use that to query the data directory for config-only installs. This fixes awkward or impossible pg_ctl operation for config-only installs.
2011-10-06 15:38:39 +02:00
if (output_config_variable != NULL)
{
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
/*
Avoid crash in "postgres -C guc" for a GUC with a null string value. Emit "(null)" instead, which was the behavior all along on platforms that don't crash, eg OS X. Per report from Jehan-Guillaume de Rorthais. Back-patch to 9.2 where -C option was introduced. Michael Paquier Report: <20160615204036.2d35d86a@firost>
2016-06-16 18:17:03 +02:00
* "-C guc" was specified, so print GUC's value and exit. No extra
* permission check is needed because the user is reading inside the
* data dir.
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
*/
Avoid crash in "postgres -C guc" for a GUC with a null string value. Emit "(null)" instead, which was the behavior all along on platforms that don't crash, eg OS X. Per report from Jehan-Guillaume de Rorthais. Back-patch to 9.2 where -C option was introduced. Michael Paquier Report: <20160615204036.2d35d86a@firost>
2016-06-16 18:17:03 +02:00
const char *config_val = GetConfigOption(output_config_variable,
false, false);
Make "postgres -C guc" print "" not "(null)" for null-valued GUCs. Commit 0b0baf262 et al made this case print "(null)" on the grounds that that's what happened on platforms that didn't crash. But neither behavior was actually intentional. What we should print is just an empty string, for compatibility with the behavior of SHOW and other ways of examining string GUCs. Those code paths don't distinguish NULL from empty strings, so we should not here either. Per gripe from Alain Radix. Like the previous patch, back-patch to 9.2 where -C option was introduced. Discussion: <CA+YdpwxPUADrmxSD7+Td=uOshMB1KkDN7G7cf+FGmNjjxMhjbw@mail.gmail.com>
2016-06-22 17:55:18 +02:00
puts(config_val ? config_val : "");
Add postmaster -C option to query configuration parameters, and have pg_ctl use that to query the data directory for config-only installs. This fixes awkward or impossible pg_ctl operation for config-only installs.
2011-10-06 15:38:39 +02:00
ExitPostmaster(0);
}
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
Whack some sense into the configuration-file-location patch. Refactor code into something reasonably understandable, cause use of the feature to not fail in standalone backends or in EXEC_BACKEND case, fix sloppy guc.c table entries, make the documentation minimally usable.
2004-10-08 03:36:36 +02:00
/* Verify that DataDir looks reasonable */
checkDataDir();
Integrate src/timezone library for all platforms. There is more we can and should do now that we control our own destiny for timezone handling, but this commit gets the bulk of the picayune diffs in place. Magnus Hagander and Tom Lane.
2004-05-21 07:08:06 +02:00
Allow group access on PGDATA Allow the cluster to be optionally init'd with read access for the group. This means a relatively non-privileged user can perform a backup of the cluster without requiring write privileges, which enhances security. The mode of PGDATA is used to determine whether group permissions are enabled for directory and file creates. This method was chosen as it's simple and works well for the various utilities that write into PGDATA. Changing the mode of PGDATA manually will not automatically change the mode of all the files contained therein. If the user would like to enable group access on an existing cluster then changing the mode of all the existing files will be required. Note that pg_upgrade will automatically change the mode of all migrated files if the new cluster is init'd with the -g option. Tests are included for the backend and all the utilities which operate on the PG data directory to ensure that the correct mode is set based on the data directory permissions. Author: David Steele <david@pgmasters.net> Reviewed-By: Michael Paquier, with discussion amongst many others. Discussion: https://postgr.es/m/ad346fe6-b23e-59f1-ecb7-0e08390ad629%40pgmasters.net
2018-04-07 23:45:39 +02:00
/* Check that pg_control exists */
checkControlFile();
Arrange for the postmaster (and standalone backends, initdb, etc) to chdir into PGDATA and subsequently use relative paths instead of absolute paths to access all files under PGDATA. This seems to give a small performance improvement, and it should make the system more robust against naive DBAs doing things like moving a database directory that has a live postmaster in it. Per recent discussion.
2005-07-04 06:51:52 +02:00
/* And switch working directory into it */
ChangeToDataDir();
Code review for superuser_reserved_connections patch. Don't try to do database access outside a transaction; revert bogus performance improvement in SIBackendInit(); improve comments; add documentation (this part courtesy Neil Conway).
2002-11-21 07:36:08 +01:00
/*
* Check for invalid combinations of GUC settings.
Add startup-time check that -B is not unreasonably small for given number of backends (-N), per recent discussion in pghackers list.
1999-06-04 23:14:46 +02:00
*/
Move max_wal_senders out of max_connections for connection slot handling Since its introduction, max_wal_senders is counted as part of max_connections when it comes to define how many connection slots can be used for replication connections with a WAL sender context. This can lead to confusion for some users, as it could be possible to block a base backup or replication from happening because other backend sessions are already taken for other purposes by an application, and superuser-only connection slots are not a correct solution to handle that case. This commit makes max_wal_senders independent of max_connections for its handling of PGPROC entries in ProcGlobal, meaning that connection slots for WAL senders are handled using their own free queue, like autovacuum workers and bgworkers. One compatibility issue that this change creates is that a standby now requires to have a value of max_wal_senders at least equal to its primary. So, if a standby created enforces the value of max_wal_senders to be lower than that, then this could break failovers. Normally this should not be an issue though, as any settings of a standby are inherited from its primary as postgresql.conf gets normally copied as part of a base backup, so parameters would be consistent. Author: Alexander Kukushkin Reviewed-by: Kyotaro Horiguchi, Petr Jelínek, Masahiko Sawada, Oleksii Kliukin Discussion: https://postgr.es/m/CAFh8B=nBzHQeYAu0b8fjK-AF1X4+_p6GRtwG+cCgs6Vci2uRuQ@mail.gmail.com
2019-02-12 02:07:56 +01:00
if (ReservedBackends >= MaxConnections)
Code review for superuser_reserved_connections patch. Don't try to do database access outside a transaction; revert bogus performance improvement in SIBackendInit(); improve comments; add documentation (this part courtesy Neil Conway).
2002-11-21 07:36:08 +01:00
{
Move max_wal_senders out of max_connections for connection slot handling Since its introduction, max_wal_senders is counted as part of max_connections when it comes to define how many connection slots can be used for replication connections with a WAL sender context. This can lead to confusion for some users, as it could be possible to block a base backup or replication from happening because other backend sessions are already taken for other purposes by an application, and superuser-only connection slots are not a correct solution to handle that case. This commit makes max_wal_senders independent of max_connections for its handling of PGPROC entries in ProcGlobal, meaning that connection slots for WAL senders are handled using their own free queue, like autovacuum workers and bgworkers. One compatibility issue that this change creates is that a standby now requires to have a value of max_wal_senders at least equal to its primary. So, if a standby created enforces the value of max_wal_senders to be lower than that, then this could break failovers. Normally this should not be an issue though, as any settings of a standby are inherited from its primary as postgresql.conf gets normally copied as part of a base backup, so parameters would be consistent. Author: Alexander Kukushkin Reviewed-by: Kyotaro Horiguchi, Petr Jelínek, Masahiko Sawada, Oleksii Kliukin Discussion: https://postgr.es/m/CAFh8B=nBzHQeYAu0b8fjK-AF1X4+_p6GRtwG+cCgs6Vci2uRuQ@mail.gmail.com
2019-02-12 02:07:56 +01:00
write_stderr("%s: superuser_reserved_connections (%d) must be less than max_connections (%d)\n",
Fix cross-checking of ReservedBackends/max_wal_senders/MaxConnections. We were independently checking ReservedBackends < MaxConnections and max_wal_senders < MaxConnections, but because walsenders aren't allowed to use superuser-reserved connections, that's really the wrong thing. Correct behavior is to insist on ReservedBackends + max_wal_senders being less than MaxConnections. Fix the code and associated documentation. This has been wrong for a long time, but since the situation probably hardly ever arises in the field (especially pre-v10, when the default for max_wal_senders was zero), no back-patch. Discussion: https://postgr.es/m/28271.1520195491@sss.pgh.pa.us
2018-03-08 17:25:26 +01:00
progname,
Move max_wal_senders out of max_connections for connection slot handling Since its introduction, max_wal_senders is counted as part of max_connections when it comes to define how many connection slots can be used for replication connections with a WAL sender context. This can lead to confusion for some users, as it could be possible to block a base backup or replication from happening because other backend sessions are already taken for other purposes by an application, and superuser-only connection slots are not a correct solution to handle that case. This commit makes max_wal_senders independent of max_connections for its handling of PGPROC entries in ProcGlobal, meaning that connection slots for WAL senders are handled using their own free queue, like autovacuum workers and bgworkers. One compatibility issue that this change creates is that a standby now requires to have a value of max_wal_senders at least equal to its primary. So, if a standby created enforces the value of max_wal_senders to be lower than that, then this could break failovers. Normally this should not be an issue though, as any settings of a standby are inherited from its primary as postgresql.conf gets normally copied as part of a base backup, so parameters would be consistent. Author: Alexander Kukushkin Reviewed-by: Kyotaro Horiguchi, Petr Jelínek, Masahiko Sawada, Oleksii Kliukin Discussion: https://postgr.es/m/CAFh8B=nBzHQeYAu0b8fjK-AF1X4+_p6GRtwG+cCgs6Vci2uRuQ@mail.gmail.com
2019-02-12 02:07:56 +01:00
ReservedBackends, MaxConnections);
Fix upper limit of superuser_reserved_connections, add limit for wal_senders Should be limited to the maximum number of connections excluding autovacuum workers, not including. Add similar check for max_wal_senders, which should never be higher than max_connections.
2012-08-10 14:49:03 +02:00
ExitPostmaster(1);
}
Add archive_mode='always' option. In 'always' mode, the standby independently archives all files it receives from the primary. Original patch by Fujii Masao, docs and review by me.
2015-05-15 17:55:24 +02:00
if (XLogArchiveMode > ARCHIVE_MODE_OFF && wal_level == WAL_LEVEL_MINIMAL)
Introduce wal_level GUC to explicitly control if information needed for archival or hot standby should be WAL-logged, instead of deducing that from other options like archive_mode. This replaces recovery_connections GUC in the primary, where it now has no effect, but it's still used in the standby to enable/disable hot standby. Remove the WAL-logging of "unlogged operations", like creating an index without WAL-logging and fsyncing it at the end. Instead, we keep a copy of the wal_mode setting and the settings that affect how much shared memory a hot standby server needs to track master transactions (max_connections, max_prepared_xacts, max_locks_per_xact) in pg_control. Whenever the settings change, at server restart, write a WAL record noting the new settings and update pg_control. This allows us to notice the change in those settings in the standby at the right moment, they used to be included in checkpoint records, but that meant that a changed value was not reflected in the standby until the first checkpoint after the change. Bump PG_CONTROL_VERSION and XLOG_PAGE_MAGIC. Whack XLOG_PAGE_MAGIC back to the sequence it used to follow, before hot standby and subsequent patches changed it to 0x9003.
2010-04-28 18:10:43 +02:00
ereport(ERROR,
Add archive_mode='always' option. In 'always' mode, the standby independently archives all files it receives from the primary. Original patch by Fujii Masao, docs and review by me.
2015-05-15 17:55:24 +02:00
(errmsg("WAL archival cannot be enabled when wal_level is \"minimal\"")));
Introduce wal_level GUC to explicitly control if information needed for archival or hot standby should be WAL-logged, instead of deducing that from other options like archive_mode. This replaces recovery_connections GUC in the primary, where it now has no effect, but it's still used in the standby to enable/disable hot standby. Remove the WAL-logging of "unlogged operations", like creating an index without WAL-logging and fsyncing it at the end. Instead, we keep a copy of the wal_mode setting and the settings that affect how much shared memory a hot standby server needs to track master transactions (max_connections, max_prepared_xacts, max_locks_per_xact) in pg_control. Whenever the settings change, at server restart, write a WAL record noting the new settings and update pg_control. This allows us to notice the change in those settings in the standby at the right moment, they used to be included in checkpoint records, but that meant that a changed value was not reflected in the standby until the first checkpoint after the change. Bump PG_CONTROL_VERSION and XLOG_PAGE_MAGIC. Whack XLOG_PAGE_MAGIC back to the sequence it used to follow, before hot standby and subsequent patches changed it to 0x9003.
2010-04-28 18:10:43 +02:00
if (max_wal_senders > 0 && wal_level == WAL_LEVEL_MINIMAL)
ereport(ERROR,
Merge wal_level "archive" and "hot_standby" into new name "replica" The distinction between "archive" and "hot_standby" existed only because at the time "hot_standby" was added, there was some uncertainty about stability. This is now a long time ago. We would like to move forward with simplifying the replication configuration, but this distinction is in the way, because a primary server cannot tell (without asking a standby or predicting the future) which one of these would be the appropriate level. Pick a new name for the combined setting to make it clearer that it covers all (non-logical) backup and replication uses. The old values are still accepted but are converted internally. Reviewed-by: Michael Paquier <michael.paquier@gmail.com> Reviewed-by: David Steele <david@pgmasters.net>
2016-03-01 02:01:54 +01:00
(errmsg("WAL streaming (max_wal_senders > 0) requires wal_level \"replica\" or \"logical\"")));
This patch reserves the last superuser_reserved_connections slots for connections by the superuser only. This patch replaces the last patch I sent a couple of days ago. It closes a connection that has not been authorised by a superuser if it would leave less than the GUC variable ReservedBackends (superuser_reserved_connections in postgres.conf) backend process slots free in the SISeg. This differs to the first patch which only reserved the last ReservedBackends slots in the procState array. This has made the free slot test more expensive due to the use of a lock. After thinking about a comment on the first patch I've also made it a fatal error if the number of reserved slots is not less than the maximum number of connections. Nigel J. Andrews
2002-08-29 23:02:12 +02:00
Repair an embarrassingly large number of alphabetization mistakes in the datetime token tables. Even more embarrassing, the regression tests revealed some of the problems --- but evidently the bogus output wasn't questioned. Add code to postmaster startup to directly check the tables for correct ordering, in hopes of not being embarrassed like this again.
2003-01-16 01:26:49 +01:00
/*
Postpone pg_timezone_initialize() until after creation of postmaster.pid, since it can take a fair amount of time and this can confuse boot scripts that expect postmaster.pid to appear quickly. Move initialization of SSL library and preloaded libraries to after that point, too, just for luck. Per reports from Tony Caduto and others.
2005-10-20 22:05:45 +02:00
* Other one-time internal sanity checks can go here, if they are fast.
* (Put any slow processing further down, after postmaster.pid creation.)
Repair an embarrassingly large number of alphabetization mistakes in the datetime token tables. Even more embarrassing, the regression tests revealed some of the problems --- but evidently the bogus output wasn't questioned. Add code to postmaster startup to directly check the tables for correct ordering, in hopes of not being embarrassed like this again.
2003-01-16 01:26:49 +01:00
*/
if (!CheckDateTokenTables())
{
Back out pg_autovacuum commit after cvs clean failure causes commit.
2004-07-21 22:34:50 +02:00
write_stderr("%s: invalid datetoken tables, please fix\n", progname);
Repair an embarrassingly large number of alphabetization mistakes in the datetime token tables. Even more embarrassing, the regression tests revealed some of the problems --- but evidently the bogus output wasn't questioned. Add code to postmaster startup to directly check the tables for correct ordering, in hopes of not being embarrassed like this again.
2003-01-16 01:26:49 +01:00
ExitPostmaster(1);
}
Proper fix for glibc getopt() botch. Surprising we did not see this before.
2001-10-19 22:47:09 +02:00
/*
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* Now that we are done processing the postmaster arguments, reset
* getopt(3) library so that it will work correctly in subprocesses.
Proper fix for glibc getopt() botch. Surprising we did not see this before.
2001-10-19 22:47:09 +02:00
*/
optind = 1;
Remove optreset from src/port/ implementations of getopt and getopt_long. We don't actually need optreset, because we can easily fix the code to ensure that it's cleanly restartable after having completed a scan over the argv array; which is the only case we need to restart in. Getting rid of it avoids a class of interactions with the system libraries and allows reversion of my change of yesterday in postmaster.c and postgres.c. Back-patch to 8.4. Before that the getopt code was a bit different anyway.
2010-12-16 22:22:05 +01:00
#ifdef HAVE_INT_OPTRESET
Fix getopt-vs-init_ps_display problem by copying original argv[] info, per suggestion from Peter. Simplify several APIs by transmitting the original argv location directly from main.c to ps_status.c, instead of passing it down through several levels of subroutines.
2001-10-21 05:25:36 +02:00
optreset = 1; /* some systems need this too */
Proper fix for glibc getopt() botch. Surprising we did not see this before.
2001-10-19 22:47:09 +02:00
#endif
/* For debugging: display postmaster environment */
Keep track of the last active slot in the shared ProcState array, so that search loops only have to scan that far and not through all maxBackends entries. This eliminates a performance penalty for setting maxBackends much higher than the average number of active backends. Also, eliminate no-longer-used 'backend tag' concept. Remove setting of environment variables at backend start (except for CYR_RECODE), since none of them are being examined by the backend any longer.
2000-11-12 21:51:52 +01:00
{
extern char **environ;
char **p;
Change some frequently-reached elog(DEBUG...) calls to ereport(DEBUG...) for speed reasons. (ereport falls out much more quickly when no output is needed than elog does.)
2003-08-12 20:23:21 +02:00
ereport(DEBUG3,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errmsg_internal("%s: PostmasterMain: initial environment dump:",
progname)));
Change some frequently-reached elog(DEBUG...) calls to ereport(DEBUG...) for speed reasons. (ereport falls out much more quickly when no output is needed than elog does.)
2003-08-12 20:23:21 +02:00
ereport(DEBUG3,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errmsg_internal("-----------------------------------------")));
Keep track of the last active slot in the shared ProcState array, so that search loops only have to scan that far and not through all maxBackends entries. This eliminates a performance penalty for setting maxBackends much higher than the average number of active backends. Also, eliminate no-longer-used 'backend tag' concept. Remove setting of environment variables at backend start (except for CYR_RECODE), since none of them are being examined by the backend any longer.
2000-11-12 21:51:52 +01:00
for (p = environ; *p; ++p)
Change some frequently-reached elog(DEBUG...) calls to ereport(DEBUG...) for speed reasons. (ereport falls out much more quickly when no output is needed than elog does.)
2003-08-12 20:23:21 +02:00
ereport(DEBUG3,
(errmsg_internal("\t%s", *p)));
ereport(DEBUG3,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errmsg_internal("-----------------------------------------")));
Keep track of the last active slot in the shared ProcState array, so that search loops only have to scan that far and not through all maxBackends entries. This eliminates a performance penalty for setting maxBackends much higher than the average number of active backends. Also, eliminate no-longer-used 'backend tag' concept. Remove setting of environment variables at backend start (except for CYR_RECODE), since none of them are being examined by the backend any longer.
2000-11-12 21:51:52 +01:00
}
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
/*
* Create lockfile for data directory.
*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We want to do this before we try to grab the input sockets, because the
* data directory interlock is more reliable than the socket-file
* interlock (thanks to whoever decided to put socket files in /tmp :-().
* For the same reason, it's best to grab the TCP socket(s) before the
Update overlooked comment.
2012-08-10 23:36:54 +02:00
* Unix socket(s).
Fix incorrect order of lock file removal and failure to close() sockets. Commit c9b0cbe98bd783e24a8c4d8d8ac472a494b81292 accidentally broke the order of operations during postmaster shutdown: it resulted in removing the per-socket lockfiles after, not before, postmaster.pid. This creates a race-condition hazard for a new postmaster that's started immediately after observing that postmaster.pid has disappeared; if it sees the socket lockfile still present, it will quite properly refuse to start. This error appears to be the explanation for at least some of the intermittent buildfarm failures we've seen in the pg_upgrade test. Another problem, which has been there all along, is that the postmaster has never bothered to close() its listen sockets, but has just allowed them to close at process death. This creates a different race condition for an incoming postmaster: it might be unable to bind to the desired listen address because the old postmaster is still incumbent. This might explain some odd failures we've seen in the past, too. (Note: this is not related to the fact that individual backends don't close their client communication sockets. That behavior is intentional and is not changed by this patch.) Fix by adding an on_proc_exit function that closes the postmaster's ports explicitly, and (in 9.3 and up) reshuffling the responsibility for where to unlink the Unix socket files. Lock file unlinking can stay where it is, but teach it to unlink the lock files in reverse order of creation.
2015-08-02 20:54:44 +02:00
*
* Also note that this internally sets up the on_proc_exit function that
* is responsible for removing both data directory and socket lockfiles;
* so it must happen before opening sockets so that at exit, the socket
* lockfiles go away after CloseServerPorts runs.
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
*/
Arrange for the postmaster (and standalone backends, initdb, etc) to chdir into PGDATA and subsequently use relative paths instead of absolute paths to access all files under PGDATA. This seems to give a small performance improvement, and it should make the system more robust against naive DBAs doing things like moving a database directory that has a live postmaster in it. Per recent discussion.
2005-07-04 06:51:52 +02:00
CreateDataDirLockFile(true);
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
Further improve code for probing the availability of ARM CRC instructions. Andrew Gierth pointed out that commit 1c72ec6f4 would yield the wrong answer on big-endian ARM systems, because the data being CRC'd would be different. To fix that, and avoid the rather unsightly hard-wired constant, simply compare the hardware and software implementations' results. While we're at it, also log the resulting decision at DEBUG1, and error out if the hw and sw results unexpectedly differ. Also, since this file must compile for both frontend and backend, avoid incorrect dependencies on backend-only headers. In passing, add a comment to postmaster.c about when the CRC function pointer will get initialized. Thomas Munro, based on complaints from Andrew Gierth and Tom Lane Discussion: https://postgr.es/m/HE1PR0801MB1323D171938EABC04FFE7FA9E3110@HE1PR0801MB1323.eurprd08.prod.outlook.com
2018-05-03 17:32:57 +02:00
/*
* Read the control file (for error checking and config info).
*
* Since we verify the control file's CRC, this has a useful side effect
* on machines where we need a run-time test for CRC support instructions.
* The postmaster will do the test once at startup, and then its child
* processes will inherit the correct function pointer and not need to
* repeat the test.
*/
Fix crash restart bug introduced in 8356753c212. The bug was caused by not re-reading the control file during crash recovery restarts, which lead to an attempt to pfree() shared memory contents. The fix is to re-read the control file, which seems good anyway. It's unclear as of this moment, whether we want to keep the refactoring introduced in the commit referenced above, or come up with an alternative approach. But fixing the bug in the mean time seems like a good idea regardless. A followup commit will introduce regression test coverage for crash restarts. Reported-By: Tom Lane Discussion: https://postgr.es/m/14134.1505572349@sss.pgh.pa.us
2017-09-17 10:00:39 +02:00
LocalProcessControlFile(false);
Perform only one ReadControlFile() during startup. Previously we read the control file in multiple places. But soon the segment size will be configurable and stored in the control file, and that needs to be available earlier than it currently is needed. Instead of adding yet another place where it's read, refactor things so there's a single processing of the control file during startup (in EXEC_BACKEND that's every individual backend's startup). Author: Andres Freund Discussion: http://postgr.es/m/20170913092828.aozd3gvvmw67gmyc@alap3.anarazel.de
2017-09-13 11:12:17 +02:00
Postpone pg_timezone_initialize() until after creation of postmaster.pid, since it can take a fair amount of time and this can confuse boot scripts that expect postmaster.pid to appear quickly. Move initialization of SSL library and preloaded libraries to after that point, too, just for luck. Per reports from Tony Caduto and others.
2005-10-20 22:05:45 +02:00
/*
* Initialize SSL library, if specified.
*/
#ifdef USE_SSL
if (EnableSSL)
Allow SSL configuration to be updated at SIGHUP. It is no longer necessary to restart the server to enable, disable, or reconfigure SSL. Instead, we just create a new SSL_CTX struct (by re-reading all relevant files) whenever we get SIGHUP. Testing shows that this is fast enough that it shouldn't be a problem. In conjunction with that, downgrade the logic that complains about pg_hba.conf "hostssl" lines when SSL isn't active: now that's just a warning condition not an error. An issue that still needs to be addressed is what shall we do with passphrase-protected server keys? As this stands, the server would demand the passphrase again on every SIGHUP, which is certainly impractical. But the case was only barely supported before, so that does not seem a sufficient reason to hold up committing this patch. Andreas Karlsson, reviewed by Michael Banck and Michael Paquier Discussion: https://postgr.es/m/556A6E8A.9030400@proxel.se
2017-01-03 03:37:12 +01:00
{
(void) secure_initialize(true);
LoadedSSL = true;
}
Postpone pg_timezone_initialize() until after creation of postmaster.pid, since it can take a fair amount of time and this can confuse boot scripts that expect postmaster.pid to appear quickly. Move initialization of SSL library and preloaded libraries to after that point, too, just for luck. Per reports from Tony Caduto and others.
2005-10-20 22:05:45 +02:00
#endif
Logical replication - Add PUBLICATION catalogs and DDL - Add SUBSCRIPTION catalog and DDL - Define logical replication protocol and output plugin - Add logical replication workers From: Petr Jelinek <petr@2ndquadrant.com> Reviewed-by: Steve Singer <steve@ssinger.info> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Erik Rijkers <er@xs4all.nl> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2017-01-19 18:00:00 +01:00
/*
* Register the apply launcher. Since it registers a background worker,
* it needs to be called before InitializeMaxBackends(), and it's probably
* a good idea to call it before any modules had chance to take the
* background worker slots.
*/
ApplyLauncherRegister();
Postpone pg_timezone_initialize() until after creation of postmaster.pid, since it can take a fair amount of time and this can confuse boot scripts that expect postmaster.pid to appear quickly. Move initialization of SSL library and preloaded libraries to after that point, too, just for luck. Per reports from Tony Caduto and others.
2005-10-20 22:05:45 +02:00
/*
Add a feature for automatic initialization and finalization of dynamically loaded libraries: call functions _PG_init() and _PG_fini() if the library defines such symbols. Hence we no longer need to specify an initialization function in preload_libraries: we can assume that the library used the _PG_init() convention, instead. This removes one source of pilot error in use of preloaded libraries. Original patch by Ralf Engelschall, preload_libraries changes by me.
2006-08-08 21:15:09 +02:00
* process any libraries that should be preloaded at postmaster start
Postpone pg_timezone_initialize() until after creation of postmaster.pid, since it can take a fair amount of time and this can confuse boot scripts that expect postmaster.pid to appear quickly. Move initialization of SSL library and preloaded libraries to after that point, too, just for luck. Per reports from Tony Caduto and others.
2005-10-20 22:05:45 +02:00
*/
Add server support for "plugin" libraries that can be used for add-on tasks such as debugging and performance measurement. This consists of two features: a table of "rendezvous variables" that allows separately-loaded shared libraries to communicate, and a new GUC setting "local_preload_libraries" that allows libraries to be loaded into specific sessions without explicit cooperation from the client application. To make local_preload_libraries as flexible as possible, we do not restrict its use to superusers; instead, it is restricted to load only libraries stored in $libdir/plugins/. The existing LOAD command has also been modified to allow non-superusers to LOAD libraries stored in this directory. This patch also renames the existing GUC variable preload_libraries to shared_preload_libraries (after a suggestion by Simon Riggs) and does some code refactoring in dfmgr.c to improve clarity. Korry Douglas, with a little help from Tom Lane.
2006-08-15 20:26:59 +02:00
process_shared_preload_libraries();
Postpone pg_timezone_initialize() until after creation of postmaster.pid, since it can take a fair amount of time and this can confuse boot scripts that expect postmaster.pid to appear quickly. Move initialization of SSL library and preloaded libraries to after that point, too, just for luck. Per reports from Tony Caduto and others.
2005-10-20 22:05:45 +02:00
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
Fix background workers for EXEC_BACKEND Commit da07a1e8 was broken for EXEC_BACKEND because I failed to realize that the MaxBackends recomputation needed to be duplicated by subprocesses in SubPostmasterMain. However, instead of having the value be recomputed at all, it's better to assign the correct value at postmaster initialization time, and have it be propagated to exec'ed backends via BackendParameters. MaxBackends stays as zero until after modules in shared_preload_libraries have had a chance to register bgworkers, since the value is going to be untrustworthy till that's finished. Heikki Linnakangas and Álvaro Herrera
2013-01-02 16:01:14 +01:00
* Now that loadable modules have had their chance to register background
Make sure MaxBackends is always set Auxiliary and bootstrap processes weren't getting it, causing initdb to fail completely.
2013-01-02 18:39:11 +01:00
* workers, calculate MaxBackends.
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
*/
Make sure MaxBackends is always set Auxiliary and bootstrap processes weren't getting it, causing initdb to fail completely.
2013-01-02 18:39:11 +01:00
InitializeMaxBackends();
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
Rearrange postmaster's startup sequence for better syslogger results. This is a second try at what commit 57431a911 tried to do, namely, launch the syslogger before we open postmaster sockets so that our messages about the sockets end up in the syslogger files. That commit fell foul of a bunch of subtle issues caused by trying to launch a postmaster child process before creating shared memory. Rather than messing with that interaction, let's postpone opening the sockets till after we launch the syslogger. This would not have been terribly safe before commit 7de19fbc0, because we relied on socket opening to detect whether any competing postmasters were using the same port number. But now that we choose IPC keys without regard to the port number, there's no interaction to worry about. Also delay creation of the external PID file (if requested) till after the sockets are open, since external code could plausibly be relying on that ordering of events. And postpone most of the work of RemovePgTempFiles() so that that potentially-slow processing still happens after we make the external PID file. We have to be a bit careful about that last though: as noted in the discussion subsequent to bug #15804, EXEC_BACKEND builds still have to clear the parameter-file temp dir before launching the syslogger. Patch by me; thanks to Michael Paquier for review/testing. Discussion: https://postgr.es/m/15804-3721117bf40fb654@postgresql.org
2019-09-11 17:43:01 +02:00
/*
* Set up shared memory and semaphores.
*/
reset_shared();
/*
* Estimate number of openable files. This must happen after setting up
* semaphores, because on some platforms semaphores count as open files.
*/
set_max_safe_fds();
/*
* Set reference point for stack-depth checking.
*/
set_stack_base();
/*
* Initialize pipe (or process handle on Windows) that allows children to
* wake up from sleep on postmaster death.
*/
InitPostmasterDeathWatchHandle();
#ifdef WIN32
/*
* Initialize I/O completion port used to deliver list of dead children.
*/
win32ChildQueue = CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, 0, 1);
if (win32ChildQueue == NULL)
ereport(FATAL,
(errmsg("could not create I/O completion port for child queue")));
#endif
#ifdef EXEC_BACKEND
/* Write out nondefault GUC settings for child processes to use */
write_nondefault_variables(PGC_POSTMASTER);
/*
* Clean out the temp directory used to transmit parameters to child
* processes (see internal_forkexec, below). We must do this before
* launching any child processes, else we have a race condition: we could
* remove a parameter file before the child can read it. It should be
* safe to do so now, because we verified earlier that there are no
* conflicting Postgres processes in this data directory.
*/
RemovePgTempFilesInDir(PG_TEMP_FILES_DIR, true, false);
#endif
/*
* Forcibly remove the files signaling a standby promotion request.
* Otherwise, the existence of those files triggers a promotion too early,
* whether a user wants that or not.
*
* This removal of files is usually unnecessary because they can exist
* only during a few moments during a standby promotion. However there is
* a race condition: if pg_ctl promote is executed and creates the files
* during a promotion, the files can stay around even after the server is
* brought up to new master. Then, if new standby starts by using the
* backup taken from that master, the files can exist at the server
* startup and should be removed in order to avoid an unexpected
* promotion.
*
* Note that promotion signal files need to be removed before the startup
* process is invoked. Because, after that, they can be used by
* postmaster's SIGUSR1 signal handler.
*/
RemovePromoteSignalFiles();
/* Do the same for logrotate signal file */
RemoveLogrotateSignalFiles();
/* Remove any outdated file holding the current log filenames. */
if (unlink(LOG_METAINFO_DATAFILE) < 0 && errno != ENOENT)
ereport(LOG,
(errcode_for_file_access(),
errmsg("could not remove file \"%s\": %m",
LOG_METAINFO_DATAFILE)));
/*
* If enabled, start up syslogger collection subprocess
*/
SysLoggerPID = SysLogger_Start();
/*
* Reset whereToSendOutput from DestDebug (its starting state) to
* DestNone. This stops ereport from sending log messages to stderr unless
* Log_destination permits. We don't do this until the postmaster is
* fully launched, since startup failures may as well be reported to
* stderr.
*
* If we are in fact disabling logging to stderr, first emit a log message
* saying so, to provide a breadcrumb trail for users who may not remember
* that their logging is configured to go somewhere else.
*/
if (!(Log_destination & LOG_DESTINATION_STDERR))
ereport(LOG,
(errmsg("ending log output to stderr"),
errhint("Future log output will go to log destination \"%s\".",
Log_destination_string)));
whereToSendOutput = DestNone;
/*
* Report server startup in log. While we could emit this much earlier,
* it seems best to do so after starting the log collector, if we intend
* to use one.
*/
Log PostgreSQL version number on startup Logging the PostgreSQL version on startup is useful for two reasons: There is a clear marker in the log file that a new postmaster is beginning, and it's useful for tracking the server version across startup while upgrading. Author: Christoph Berg <christoph.berg@credativ.de> Discussion: https://www.postgresql.org/message-id/flat/20181121144611.GJ15795@msg.credativ.de/
2019-01-30 23:26:10 +01:00
ereport(LOG,
(errmsg("starting %s", PG_VERSION_STR)));
Keep track of the last active slot in the shared ProcState array, so that search loops only have to scan that far and not through all maxBackends entries. This eliminates a performance penalty for setting maxBackends much higher than the average number of active backends. Also, eliminate no-longer-used 'backend tag' concept. Remove setting of environment variables at backend start (except for CYR_RECODE), since none of them are being examined by the backend any longer.
2000-11-12 21:51:52 +01:00
/*
* Establish input sockets.
Fix incorrect order of lock file removal and failure to close() sockets. Commit c9b0cbe98bd783e24a8c4d8d8ac472a494b81292 accidentally broke the order of operations during postmaster shutdown: it resulted in removing the per-socket lockfiles after, not before, postmaster.pid. This creates a race-condition hazard for a new postmaster that's started immediately after observing that postmaster.pid has disappeared; if it sees the socket lockfile still present, it will quite properly refuse to start. This error appears to be the explanation for at least some of the intermittent buildfarm failures we've seen in the pg_upgrade test. Another problem, which has been there all along, is that the postmaster has never bothered to close() its listen sockets, but has just allowed them to close at process death. This creates a different race condition for an incoming postmaster: it might be unable to bind to the desired listen address because the old postmaster is still incumbent. This might explain some odd failures we've seen in the past, too. (Note: this is not related to the fact that individual backends don't close their client communication sockets. That behavior is intentional and is not changed by this patch.) Fix by adding an on_proc_exit function that closes the postmaster's ports explicitly, and (in 9.3 and up) reshuffling the responsibility for where to unlink the Unix socket files. Lock file unlinking can stay where it is, but teach it to unlink the lock files in reverse order of creation.
2015-08-02 20:54:44 +02:00
*
* First, mark them all closed, and set up an on_proc_exit function that's
* charged with closing the sockets again at postmaster shutdown.
Keep track of the last active slot in the shared ProcState array, so that search loops only have to scan that far and not through all maxBackends entries. This eliminates a performance penalty for setting maxBackends much higher than the average number of active backends. Also, eliminate no-longer-used 'backend tag' concept. Remove setting of environment variables at backend start (except for CYR_RECODE), since none of them are being examined by the backend any longer.
2000-11-12 21:51:52 +01:00
*/
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
for (i = 0; i < MAXLISTEN; i++)
Create typedef pgsocket for storing socket descriptors. This silences some warnings on Win64. Not using the proper SOCKET datatype was actually wrong on Win32 as well, but didn't cause any warnings there. Also create define PGINVALID_SOCKET to indicate an invalid/non-existing socket, instead of using a hardcoded -1 value.
2010-01-10 15:16:08 +01:00
ListenSocket[i] = PGINVALID_SOCKET;
Have a go at fixing various outstanding portability issues in code that was modified for IPv6. Use a robust definition of struct sockaddr_storage, do a proper configure test to see if ss_len exists, don't assume that getnameinfo() will handle AF_UNIX sockets, don't trust getaddrinfo to return the protocol we ask for, etc. This incorporates several outstanding patches from Kurt Roeckx, but I'm to blame for anything that doesn't work ...
2003-07-24 01:30:41 +02:00
Fix incorrect order of lock file removal and failure to close() sockets. Commit c9b0cbe98bd783e24a8c4d8d8ac472a494b81292 accidentally broke the order of operations during postmaster shutdown: it resulted in removing the per-socket lockfiles after, not before, postmaster.pid. This creates a race-condition hazard for a new postmaster that's started immediately after observing that postmaster.pid has disappeared; if it sees the socket lockfile still present, it will quite properly refuse to start. This error appears to be the explanation for at least some of the intermittent buildfarm failures we've seen in the pg_upgrade test. Another problem, which has been there all along, is that the postmaster has never bothered to close() its listen sockets, but has just allowed them to close at process death. This creates a different race condition for an incoming postmaster: it might be unable to bind to the desired listen address because the old postmaster is still incumbent. This might explain some odd failures we've seen in the past, too. (Note: this is not related to the fact that individual backends don't close their client communication sockets. That behavior is intentional and is not changed by this patch.) Fix by adding an on_proc_exit function that closes the postmaster's ports explicitly, and (in 9.3 and up) reshuffling the responsibility for where to unlink the Unix socket files. Lock file unlinking can stay where it is, but teach it to unlink the lock files in reverse order of creation.
2015-08-02 20:54:44 +02:00
on_proc_exit(CloseServerPorts, 0);
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
if (ListenAddresses)
Fix case issues with quotes.
1997-11-10 06:10:50 +01:00
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
char *rawstring;
List *elemlist;
ListCell *l;
Issue fatal error if no TCP/IP sockets could be created
2005-06-30 12:02:22 +02:00
int success = 0;
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
Make listen_addresses be a comma-separated list instead of a space-separated list. More consistent with our other list-containing GUC variables.
2004-08-08 22:17:36 +02:00
/* Need a modifiable copy of ListenAddresses */
rawstring = pstrdup(ListenAddresses);
Support having multiple Unix-domain sockets per postmaster. Replace unix_socket_directory with unix_socket_directories, which is a list of socket directories, and adjust postmaster's code to allow zero or more Unix-domain sockets to be created. This is mostly a straightforward change, but since the Unix sockets ought to be created after the TCP/IP sockets for safety reasons (better chance of detecting a port number conflict), AddToDataDirLockFile needs to be fixed to support out-of-order updates of data directory lockfile lines. That's a change that had been foreseen to be necessary someday anyway. Honza Horak, reviewed and revised by Tom Lane
2012-08-10 23:26:44 +02:00
/* Parse string into list of hostnames */
Avoid using SplitIdentifierString to parse ListenAddresses, too. This gets rid of our former behavior of forcibly downcasing the postmaster's hostname list and truncating the elements to NAMEDATALEN. In principle, DNS hostnames are case-insensitive so the first behavior should be harmless, and server hostnames are seldom long enough for the second behavior to be an issue. But it's still dubious, and an easy fix is available: just use SplitGUCList instead. AFAICT, all other SplitIdentifierString calls in the backend are OK: either the items actually are SQL identifiers, or they are keywords that are short and case-insensitive. Per thinking about bug #16106. While this has been wrong for a very long time, the lack of field complaints means there's little reason to back-patch. Discussion: https://postgr.es/m/16106-7d319e4295d08e70@postgresql.org
2019-11-13 19:51:58 +01:00
if (!SplitGUCList(rawstring, ',', &elemlist))
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
{
Make listen_addresses be a comma-separated list instead of a space-separated list. More consistent with our other list-containing GUC variables.
2004-08-08 22:17:36 +02:00
/* syntax error in list */
ereport(FATAL,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
Make error wording more consistent
2013-08-29 18:33:50 +02:00
errmsg("invalid list syntax in parameter \"%s\"",
"listen_addresses")));
Make listen_addresses be a comma-separated list instead of a space-separated list. More consistent with our other list-containing GUC variables.
2004-08-08 22:17:36 +02:00
}
foreach(l, elemlist)
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
char *curhost = (char *) lfirst(l);
Make listen_addresses be a comma-separated list instead of a space-separated list. More consistent with our other list-containing GUC variables.
2004-08-08 22:17:36 +02:00
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
if (strcmp(curhost, "*") == 0)
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
status = StreamServerPort(AF_UNSPEC, NULL,
(unsigned short) PostPortNumber,
Support having multiple Unix-domain sockets per postmaster. Replace unix_socket_directory with unix_socket_directories, which is a list of socket directories, and adjust postmaster's code to allow zero or more Unix-domain sockets to be created. This is mostly a straightforward change, but since the Unix sockets ought to be created after the TCP/IP sockets for safety reasons (better chance of detecting a port number conflict), AddToDataDirLockFile needs to be fixed to support out-of-order updates of data directory lockfile lines. That's a change that had been foreseen to be necessary someday anyway. Honza Horak, reviewed and revised by Tom Lane
2012-08-10 23:26:44 +02:00
NULL,
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
ListenSocket, MAXLISTEN);
else
Have a go at fixing various outstanding portability issues in code that was modified for IPv6. Use a robust definition of struct sockaddr_storage, do a proper configure test to see if ss_len exists, don't assume that getnameinfo() will handle AF_UNIX sockets, don't trust getaddrinfo to return the protocol we ask for, etc. This incorporates several outstanding patches from Kurt Roeckx, but I'm to blame for anything that doesn't work ...
2003-07-24 01:30:41 +02:00
status = StreamServerPort(AF_UNSPEC, curhost,
(unsigned short) PostPortNumber,
Support having multiple Unix-domain sockets per postmaster. Replace unix_socket_directory with unix_socket_directories, which is a list of socket directories, and adjust postmaster's code to allow zero or more Unix-domain sockets to be created. This is mostly a straightforward change, but since the Unix sockets ought to be created after the TCP/IP sockets for safety reasons (better chance of detecting a port number conflict), AddToDataDirLockFile needs to be fixed to support out-of-order updates of data directory lockfile lines. That's a change that had been foreseen to be necessary someday anyway. Honza Horak, reviewed and revised by Tom Lane
2012-08-10 23:26:44 +02:00
NULL,
Have a go at fixing various outstanding portability issues in code that was modified for IPv6. Use a robust definition of struct sockaddr_storage, do a proper configure test to see if ss_len exists, don't assume that getnameinfo() will handle AF_UNIX sockets, don't trust getaddrinfo to return the protocol we ask for, etc. This incorporates several outstanding patches from Kurt Roeckx, but I'm to blame for anything that doesn't work ...
2003-07-24 01:30:41 +02:00
ListenSocket, MAXLISTEN);
Include the first valid listen address in pg_ctl to improve server start "wait" detection and add postmaster start time to help determine if the postmaster is actually using the specified data directory.
2010-12-31 23:24:26 +01:00
Issue fatal error if no TCP/IP sockets could be created
2005-06-30 12:02:22 +02:00
if (status == STATUS_OK)
Code review for postmaster.pid contents changes. Fix broken test for pre-existing postmaster, caused by wrong code for appending lines to the lockfile; don't write a failed listen_address setting into the lockfile; don't arbitrarily change the location of the data directory in the lockfile compared to previous releases; provide more consistent and useful definitions of the socket path and listen_address entries; avoid assuming that pg_ctl has the same DEFAULT_PGSOCKET_DIR as the postmaster; assorted code style improvements.
2011-01-14 01:01:28 +01:00
{
Issue fatal error if no TCP/IP sockets could be created
2005-06-30 12:02:22 +02:00
success++;
Code review for postmaster.pid contents changes. Fix broken test for pre-existing postmaster, caused by wrong code for appending lines to the lockfile; don't write a failed listen_address setting into the lockfile; don't arbitrarily change the location of the data directory in the lockfile compared to previous releases; provide more consistent and useful definitions of the socket path and listen_address entries; avoid assuming that pg_ctl has the same DEFAULT_PGSOCKET_DIR as the postmaster; assorted code style improvements.
2011-01-14 01:01:28 +01:00
/* record the first successful host addr in lockfile */
if (!listen_addr_saved)
{
AddToDataDirLockFile(LOCK_FILE_LINE_LISTEN_ADDR, curhost);
listen_addr_saved = true;
}
}
Issue fatal error if no TCP/IP sockets could be created
2005-06-30 12:02:22 +02:00
else
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
ereport(WARNING,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errmsg("could not create listen socket for \"%s\"",
curhost)));
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
}
Make listen_addresses be a comma-separated list instead of a space-separated list. More consistent with our other list-containing GUC variables.
2004-08-08 22:17:36 +02:00
Support having multiple Unix-domain sockets per postmaster. Replace unix_socket_directory with unix_socket_directories, which is a list of socket directories, and adjust postmaster's code to allow zero or more Unix-domain sockets to be created. This is mostly a straightforward change, but since the Unix sockets ought to be created after the TCP/IP sockets for safety reasons (better chance of detecting a port number conflict), AddToDataDirLockFile needs to be fixed to support out-of-order updates of data directory lockfile lines. That's a change that had been foreseen to be necessary someday anyway. Honza Horak, reviewed and revised by Tom Lane
2012-08-10 23:26:44 +02:00
if (!success && elemlist != NIL)
Issue fatal error if no TCP/IP sockets could be created
2005-06-30 12:02:22 +02:00
ereport(FATAL,
(errmsg("could not create any TCP/IP sockets")));
Make listen_addresses be a comma-separated list instead of a space-separated list. More consistent with our other list-containing GUC variables.
2004-08-08 22:17:36 +02:00
list_free(elemlist);
pfree(rawstring);
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
}
Have a go at fixing various outstanding portability issues in code that was modified for IPv6. Use a robust definition of struct sockaddr_storage, do a proper configure test to see if ss_len exists, don't assume that getnameinfo() will handle AF_UNIX sockets, don't trust getaddrinfo to return the protocol we ask for, etc. This incorporates several outstanding patches from Kurt Roeckx, but I'm to blame for anything that doesn't work ...
2003-07-24 01:30:41 +02:00
Rename Rendezvous to Bonjour to match OS/X renaming.
2005-05-15 02:26:19 +02:00
#ifdef USE_BONJOUR
/* Register for Bonjour only if we opened TCP socket(s) */
Create typedef pgsocket for storing socket descriptors. This silences some warnings on Win64. Not using the proper SOCKET datatype was actually wrong on Win32 as well, but didn't cause any warnings there. Also create define PGINVALID_SOCKET to indicate an invalid/non-existing socket, instead of using a hardcoded -1 value.
2010-01-10 15:16:08 +01:00
if (enable_bonjour && ListenSocket[0] != PGINVALID_SOCKET)
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
{
Replace use of the long-deprecated Bonjour API DNSServiceRegistrationCreate with the not-so-deprecated DNSServiceRegister. This patch shouldn't change any user-visible behavior, it just gets rid of a deprecation warning in --with-bonjour builds. The new code will fail on OS X releases before 10.3, but it seems unlikely that anyone will want to run Postgres 8.5 on 10.2.
2009-09-08 18:08:26 +02:00
DNSServiceErrorType err;
/*
* We pass 0 for interface_index, which will result in registering on
* all "applicable" interfaces. It's not entirely clear from the
* DNS-SD docs whether this would be appropriate if we have bound to
* just a subset of the available network interfaces.
*/
err = DNSServiceRegister(&bonjour_sdref,
0,
0,
bonjour_name,
"_postgresql._tcp.",
NULL,
NULL,
Replace most usages of ntoh[ls] and hton[sl] with pg_bswap.h. All postgres internal usages are replaced, it's just libpq example usages that haven't been converted. External users of libpq can't generally rely on including postgres internal headers. Note that this includes replacing open-coded byte swapping of 64bit integers (using two 32 bit swaps) with a single 64bit swap. Where it looked applicable, I have removed netinet/in.h and arpa/inet.h usage, which previously provided the relevant functionality. It's perfectly possible that I missed other reasons for including those, the buildfarm will tell. Author: Andres Freund Discussion: https://postgr.es/m/20170927172019.gheidqy6xvlxb325@alap3.anarazel.de
2017-10-02 00:36:14 +02:00
pg_hton16(PostPortNumber),
Replace use of the long-deprecated Bonjour API DNSServiceRegistrationCreate with the not-so-deprecated DNSServiceRegister. This patch shouldn't change any user-visible behavior, it just gets rid of a deprecation warning in --with-bonjour builds. The new code will fail on OS X releases before 10.3, but it seems unlikely that anyone will want to run Postgres 8.5 on 10.2.
2009-09-08 18:08:26 +02:00
0,
NULL,
NULL,
NULL);
if (err != kDNSServiceErr_NoError)
elog(LOG, "DNSServiceRegister() failed: error code %ld",
(long) err);
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
Replace use of the long-deprecated Bonjour API DNSServiceRegistrationCreate with the not-so-deprecated DNSServiceRegister. This patch shouldn't change any user-visible behavior, it just gets rid of a deprecation warning in --with-bonjour builds. The new code will fail on OS X releases before 10.3, but it seems unlikely that anyone will want to run Postgres 8.5 on 10.2.
2009-09-08 18:08:26 +02:00
/*
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* We don't bother to read the mDNS daemon's reply, and we expect that
* it will automatically terminate our registration when the socket is
* closed at postmaster termination. So there's nothing more to be
* done here. However, the bonjour_sdref is kept around so that
* forked children can close their copies of the socket.
Replace use of the long-deprecated Bonjour API DNSServiceRegistrationCreate with the not-so-deprecated DNSServiceRegister. This patch shouldn't change any user-visible behavior, it just gets rid of a deprecation warning in --with-bonjour builds. The new code will fail on OS X releases before 10.3, but it seems unlikely that anyone will want to run Postgres 8.5 on 10.2.
2009-09-08 18:08:26 +02:00
*/
Fix case issues with quotes.
1997-11-10 06:10:50 +01:00
}
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
#endif
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
Introduce HAVE_UNIX_SOCKETS symbol to replace repeatedly listing all the unsupported platforms.
2000-08-20 12:55:35 +02:00
#ifdef HAVE_UNIX_SOCKETS
Support having multiple Unix-domain sockets per postmaster. Replace unix_socket_directory with unix_socket_directories, which is a list of socket directories, and adjust postmaster's code to allow zero or more Unix-domain sockets to be created. This is mostly a straightforward change, but since the Unix sockets ought to be created after the TCP/IP sockets for safety reasons (better chance of detecting a port number conflict), AddToDataDirLockFile needs to be fixed to support out-of-order updates of data directory lockfile lines. That's a change that had been foreseen to be necessary someday anyway. Honza Horak, reviewed and revised by Tom Lane
2012-08-10 23:26:44 +02:00
if (Unix_socket_directories)
{
char *rawstring;
List *elemlist;
ListCell *l;
int success = 0;
/* Need a modifiable copy of Unix_socket_directories */
rawstring = pstrdup(Unix_socket_directories);
/* Parse string into list of directories */
if (!SplitDirectoriesString(rawstring, ',', &elemlist))
{
/* syntax error in list */
ereport(FATAL,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
Make error wording more consistent
2013-08-29 18:33:50 +02:00
errmsg("invalid list syntax in parameter \"%s\"",
"unix_socket_directories")));
Support having multiple Unix-domain sockets per postmaster. Replace unix_socket_directory with unix_socket_directories, which is a list of socket directories, and adjust postmaster's code to allow zero or more Unix-domain sockets to be created. This is mostly a straightforward change, but since the Unix sockets ought to be created after the TCP/IP sockets for safety reasons (better chance of detecting a port number conflict), AddToDataDirLockFile needs to be fixed to support out-of-order updates of data directory lockfile lines. That's a change that had been foreseen to be necessary someday anyway. Honza Horak, reviewed and revised by Tom Lane
2012-08-10 23:26:44 +02:00
}
foreach(l, elemlist)
{
char *socketdir = (char *) lfirst(l);
status = StreamServerPort(AF_UNIX, NULL,
(unsigned short) PostPortNumber,
socketdir,
ListenSocket, MAXLISTEN);
if (status == STATUS_OK)
{
success++;
/* record the first successful Unix socket in lockfile */
if (success == 1)
AddToDataDirLockFile(LOCK_FILE_LINE_SOCKET_DIR, socketdir);
}
else
ereport(WARNING,
(errmsg("could not create Unix-domain socket in directory \"%s\"",
socketdir)));
}
if (!success && elemlist != NIL)
ereport(FATAL,
(errmsg("could not create any Unix-domain sockets")));
list_free_deep(elemlist);
pfree(rawstring);
}
Apply Win32 patch from Horak Daniel.
1999-01-17 07:20:06 +01:00
#endif
Keep track of the last active slot in the shared ProcState array, so that search loops only have to scan that far and not through all maxBackends entries. This eliminates a performance penalty for setting maxBackends much higher than the average number of active backends. Also, eliminate no-longer-used 'backend tag' concept. Remove setting of environment variables at backend start (except for CYR_RECODE), since none of them are being examined by the backend any longer.
2000-11-12 21:51:52 +01:00
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
/*
* check that we have some socket to listen on
*/
Create typedef pgsocket for storing socket descriptors. This silences some warnings on Win64. Not using the proper SOCKET datatype was actually wrong on Win32 as well, but didn't cause any warnings there. Also create define PGINVALID_SOCKET to indicate an invalid/non-existing socket, instead of using a hardcoded -1 value.
2010-01-10 15:16:08 +01:00
if (ListenSocket[0] == PGINVALID_SOCKET)
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
ereport(FATAL,
Adjust error message wording per Andrew Dunstan's advice.
2004-03-24 16:20:54 +01:00
(errmsg("no socket created for listening")));
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter, as per recent discussion. The default behavior is now to listen on localhost, which eliminates the need for the -i postmaster switch in many scenarios. Andrew Dunstan
2004-03-23 02:23:48 +01:00
Code review for postmaster.pid contents changes. Fix broken test for pre-existing postmaster, caused by wrong code for appending lines to the lockfile; don't write a failed listen_address setting into the lockfile; don't arbitrarily change the location of the data directory in the lockfile compared to previous releases; provide more consistent and useful definitions of the socket path and listen_address entries; avoid assuming that pg_ctl has the same DEFAULT_PGSOCKET_DIR as the postmaster; assorted code style improvements.
2011-01-14 01:01:28 +01:00
/*
* If no valid TCP ports, write an empty line for listen address,
* indicating the Unix socket must be used. Note that this line is not
* added to the lock file until there is a socket backing it.
*/
if (!listen_addr_saved)
AddToDataDirLockFile(LOCK_FILE_LINE_LISTEN_ADDR, "");
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Record postmaster options. We delay this till now to avoid recording
Rearrange postmaster's startup sequence for better syslogger results. This is a second try at what commit 57431a911 tried to do, namely, launch the syslogger before we open postmaster sockets so that our messages about the sockets end up in the syslogger files. That commit fell foul of a bunch of subtle issues caused by trying to launch a postmaster child process before creating shared memory. Rather than messing with that interaction, let's postpone opening the sockets till after we launch the syslogger. This would not have been terribly safe before commit 7de19fbc0, because we relied on socket opening to detect whether any competing postmasters were using the same port number. But now that we choose IPC keys without regard to the port number, there's no interaction to worry about. Also delay creation of the external PID file (if requested) till after the sockets are open, since external code could plausibly be relying on that ordering of events. And postpone most of the work of RemovePgTempFiles() so that that potentially-slow processing still happens after we make the external PID file. We have to be a bit careful about that last though: as noted in the discussion subsequent to bug #15804, EXEC_BACKEND builds still have to clear the parameter-file temp dir before launching the syslogger. Patch by me; thanks to Michael Paquier for review/testing. Discussion: https://postgr.es/m/15804-3721117bf40fb654@postgresql.org
2019-09-11 17:43:01 +02:00
* bogus options (eg, unusable port number).
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
*/
Reorganize backend code to more cleanly manage executable names and backend startup.
2004-05-14 00:45:04 +02:00
if (!CreateOptsFile(argc, argv, my_exec_path))
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
ExitPostmaster(1);
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
Whack some sense into the configuration-file-location patch. Refactor code into something reasonably understandable, cause use of the feature to not fail in standalone backends or in EXEC_BACKEND case, fix sloppy guc.c table entries, make the documentation minimally usable.
2004-10-08 03:36:36 +02:00
/*
* Write the external PID file if requested
*/
Adjust configuration-files GUC behavior as per my recent proposal. The vars are renamed to data_directory, config_file, hba_file, and ident_file, and are guaranteed to be set to accurate absolute paths during postmaster startup. This commit does not yet do anything about hiding path values from non-superusers.
2004-10-10 01:13:22 +02:00
if (external_pid_file)
Whack some sense into the configuration-file-location patch. Refactor code into something reasonably understandable, cause use of the feature to not fail in standalone backends or in EXEC_BACKEND case, fix sloppy guc.c table entries, make the documentation minimally usable.
2004-10-08 03:36:36 +02:00
{
Adjust configuration-files GUC behavior as per my recent proposal. The vars are renamed to data_directory, config_file, hba_file, and ident_file, and are guaranteed to be set to accurate absolute paths during postmaster startup. This commit does not yet do anything about hiding path values from non-superusers.
2004-10-10 01:13:22 +02:00
FILE *fpidfile = fopen(external_pid_file, "w");
Whack some sense into the configuration-file-location patch. Refactor code into something reasonably understandable, cause use of the feature to not fail in standalone backends or in EXEC_BACKEND case, fix sloppy guc.c table entries, make the documentation minimally usable.
2004-10-08 03:36:36 +02:00
if (fpidfile)
{
fprintf(fpidfile, "%d\n", MyProcPid);
fclose(fpidfile);
Make external_pid_file world readable
2011-06-18 23:34:32 +02:00
/* Make PID file world readable */
if (chmod(external_pid_file, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH) != 0)
write_stderr("%s: could not change permissions of external PID file \"%s\": %s\n",
progname, external_pid_file, strerror(errno));
Whack some sense into the configuration-file-location patch. Refactor code into something reasonably understandable, cause use of the feature to not fail in standalone backends or in EXEC_BACKEND case, fix sloppy guc.c table entries, make the documentation minimally usable.
2004-10-08 03:36:36 +02:00
}
else
Message style revisions
2004-10-12 23:54:45 +02:00
write_stderr("%s: could not write external PID file \"%s\": %s\n",
Adjust configuration-files GUC behavior as per my recent proposal. The vars are renamed to data_directory, config_file, hba_file, and ident_file, and are guaranteed to be set to accurate absolute paths during postmaster startup. This commit does not yet do anything about hiding path values from non-superusers.
2004-10-10 01:13:22 +02:00
progname, external_pid_file, strerror(errno));
Remove external PID file on postmaster exit
2012-08-21 05:47:11 +02:00
on_proc_exit(unlink_external_pid_file, 0);
Whack some sense into the configuration-file-location patch. Refactor code into something reasonably understandable, cause use of the feature to not fail in standalone backends or in EXEC_BACKEND case, fix sloppy guc.c table entries, make the documentation minimally usable.
2004-10-08 03:36:36 +02:00
}
Make logging_collector=on work with non-windows EXEC_BACKEND again. Commit b94ce6e80 reordered postmaster's startup sequence so that the tempfile directory is only cleaned up after all the necessary state for pg_ctl is collected. Unfortunately the chosen location is after the syslogger has been started; which normally is fine, except for !WIN32 EXEC_BACKEND builds, which pass information to children via files in the temp directory. Move the call to RemovePgTempFiles() to just before the syslogger has started. That's the first child we fork. Luckily EXEC_BACKEND is pretty much only used by endusers on windows, which has a separate method to pass information to children. That means the real world impact of this bug is very small. Discussion: 20150113182344.GF12272@alap3.anarazel.de Backpatch to 9.1, just as the previous commit was.
2015-01-13 21:02:47 +01:00
/*
* Remove old temporary files. At this point there can be no other
* Postgres processes running in this directory, so this should be safe.
*/
RemovePgTempFiles();
Start the stats collector at a less randomly chosen time. Bad idea to start it before we have acquired the data directory lock; also a bad idea to start it before we have set up to catch SIGCHLD signals.
2001-07-03 18:52:12 +02:00
/*
Make the order of operations in PostmasterMain a bit saner ... some recent patches had added stuff in rather random spots.
2006-07-25 03:23:34 +02:00
* Initialize stats collection subsystem (this does NOT start the
* collector process!)
Start the stats collector at a less randomly chosen time. Bad idea to start it before we have acquired the data directory lock; also a bad idea to start it before we have set up to catch SIGCHLD signals.
2001-07-03 18:52:12 +02:00
*/
Tweak stats collector start logic so that we will not try to spawn a new stats collector oftener than once a minute. Per gripe from Erik Walthinsen 4/25/03.
2003-04-26 04:57:14 +02:00
pgstat_init();
Start the stats collector at a less randomly chosen time. Bad idea to start it before we have acquired the data directory lock; also a bad idea to start it before we have set up to catch SIGCHLD signals.
2001-07-03 18:52:12 +02:00
Fix pg_pwd caching mechanism, which was broken by changes to fork postmaster children before client auth step. Postmaster now rereads pg_pwd on receipt of SIGHUP, the same way that pg_hba.conf is handled. No cycles need be expended to validate password cache validity during connection startup.
2001-11-02 19:39:57 +01:00
/*
Make the order of operations in PostmasterMain a bit saner ... some recent patches had added stuff in rather random spots.
2006-07-25 03:23:34 +02:00
* Initialize the autovacuum subsystem (again, no process start yet)
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
*/
Make the order of operations in PostmasterMain a bit saner ... some recent patches had added stuff in rather random spots.
2006-07-25 03:23:34 +02:00
autovac_init();
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Try to make silent_mode behave somewhat reasonably. Instead of sending stdout/stderr to /dev/null after forking away from the terminal, send them to postmaster.log within the data directory. Since this opens the door to indefinite logfile bloat, recommend even more strongly that log output be redirected when using silent_mode. Move the postmaster's initial calls of load_hba() and load_ident() down to after we have started the log collector, if we are going to. This is so that errors reported by them will appear in the "usual" place. Reclassify silent_mode as a LOGGING_WHERE, not LOGGING_WHEN, parameter, since it's got absolutely nothing to do with the latter category. In passing, fix some obsolete references to -S ... this option hasn't had that switch letter for a long time. Back-patch to 8.4, since as of 8.4 load_hba() and load_ident() are more picky (and thus more likely to fail) than they used to be. This entire change was driven by a complaint about those errors disappearing into the bit bucket.
2009-08-24 22:08:32 +02:00
/*
* Load configuration files for client authentication.
*/
if (!load_hba())
{
/*
* It makes no sense to continue if we fail to load the HBA file,
* since there is no way to connect to the database in this case.
*/
ereport(FATAL,
(errmsg("could not load pg_hba.conf")));
}
Parse pg_ident.conf when it's loaded, keeping it in memory in parsed format. Similar changes were done to pg_hba.conf earlier already, this commit makes pg_ident.conf to behave the same as pg_hba.conf. This has two user-visible effects. First, if pg_ident.conf contains multiple errors, the whole file is parsed at postmaster startup time and all the errors are immediately reported. Before this patch, the file was parsed and the errors were reported only when someone tries to connect using an authentication method that uses the file, and the parsing stopped on first error. Second, if you SIGHUP to reload the config files, and the new pg_ident.conf file contains an error, the error is logged but the old file stays in effect. Also, regular expressions in pg_ident.conf are now compiled only once when the file is loaded, rather than every time the a user is authenticated. That should speed up authentication if you have a lot of regexps in the file. Amit Kapila
2012-09-21 16:41:22 +02:00
if (!load_ident())
{
/*
* We can start up without the IDENT file, although it means that you
* cannot log in using any of the authentication methods that need a
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
* user name mapping. load_ident() already logged the details of error
* to the log.
Parse pg_ident.conf when it's loaded, keeping it in memory in parsed format. Similar changes were done to pg_hba.conf earlier already, this commit makes pg_ident.conf to behave the same as pg_hba.conf. This has two user-visible effects. First, if pg_ident.conf contains multiple errors, the whole file is parsed at postmaster startup time and all the errors are immediately reported. Before this patch, the file was parsed and the errors were reported only when someone tries to connect using an authentication method that uses the file, and the parsing stopped on first error. Second, if you SIGHUP to reload the config files, and the new pg_ident.conf file contains an error, the error is logged but the old file stays in effect. Also, regular expressions in pg_ident.conf are now compiled only once when the file is loaded, rather than every time the a user is authenticated. That should speed up authentication if you have a lot of regexps in the file. Amit Kapila
2012-09-21 16:41:22 +02:00
*/
}
On Darwin, detect and report a multithreaded postmaster. Darwin --enable-nls builds use a substitute setlocale() that may start a thread. Buildfarm member orangutan experienced BackendList corruption on account of different postmaster threads executing signal handlers simultaneously. Furthermore, a multithreaded postmaster risks undefined behavior from sigprocmask() and fork(). Emit LOG messages about the problem and its workaround. Back-patch to 9.0 (all supported versions).
2015-01-08 04:35:44 +01:00
#ifdef HAVE_PTHREAD_IS_THREADED_NP
/*
Refer to OS X as "macOS", except for the port name which is still "darwin". We weren't terribly consistent about whether to call Apple's OS "OS X" or "Mac OS X", and the former is probably confusing to people who aren't Apple users. Now that Apple has rebranded it "macOS", follow their lead to establish a consistent naming pattern. Also, avoid the use of the ancient project name "Darwin", except as the port code name which does not seem desirable to change. (In short, this patch touches documentation and comments, but no actual code.) I didn't touch contrib/start-scripts/osx/, either. I suspect those are obsolete and due for a rewrite, anyway. I dithered about whether to apply this edit to old release notes, but those were responsible for quite a lot of the inconsistencies, so I ended up changing them too. Anyway, Apple's being ahistorical about this, so why shouldn't we be?
2016-09-25 21:40:57 +02:00
* On macOS, libintl replaces setlocale() with a version that calls
On Darwin, detect and report a multithreaded postmaster. Darwin --enable-nls builds use a substitute setlocale() that may start a thread. Buildfarm member orangutan experienced BackendList corruption on account of different postmaster threads executing signal handlers simultaneously. Furthermore, a multithreaded postmaster risks undefined behavior from sigprocmask() and fork(). Emit LOG messages about the problem and its workaround. Back-patch to 9.0 (all supported versions).
2015-01-08 04:35:44 +01:00
* CFLocaleCopyCurrent() when its second argument is "" and every relevant
* environment variable is unset or empty. CFLocaleCopyCurrent() makes
* the process multithreaded. The postmaster calls sigprocmask() and
* calls fork() without an immediate exec(), both of which have undefined
* behavior in a multithreaded program. A multithreaded postmaster is the
* normal case on Windows, which offers neither fork() nor sigprocmask().
*/
if (pthread_is_threaded_np() != 0)
On Darwin, refuse postmaster startup when multithreaded. The previous commit introduced its report at LOG level to avoid surprises at minor release upgrade time. Compel users deploying the next major release to also deploy the reported workaround.
2015-01-08 04:46:59 +01:00
ereport(FATAL,
On Darwin, detect and report a multithreaded postmaster. Darwin --enable-nls builds use a substitute setlocale() that may start a thread. Buildfarm member orangutan experienced BackendList corruption on account of different postmaster threads executing signal handlers simultaneously. Furthermore, a multithreaded postmaster risks undefined behavior from sigprocmask() and fork(). Emit LOG messages about the problem and its workaround. Back-patch to 9.0 (all supported versions).
2015-01-08 04:35:44 +01:00
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("postmaster became multithreaded during startup"),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errhint("Set the LC_ALL environment variable to a valid locale.")));
On Darwin, detect and report a multithreaded postmaster. Darwin --enable-nls builds use a substitute setlocale() that may start a thread. Buildfarm member orangutan experienced BackendList corruption on account of different postmaster threads executing signal handlers simultaneously. Furthermore, a multithreaded postmaster risks undefined behavior from sigprocmask() and fork(). Emit LOG messages about the problem and its workaround. Back-patch to 9.0 (all supported versions).
2015-01-08 04:35:44 +01:00
#endif
Add pg_postmaster_start_time() function. Euler Taveira de Oliveira Matthias Schmidt
2005-06-14 23:04:42 +02:00
/*
Clean up the rather historically encumbered interface to now() and current time: provide a GetCurrentTimestamp() function that returns current time in the form of a TimestampTz, instead of separate time_t and microseconds fields. This is what all the callers really want anyway, and it eliminates low-level dependencies on AbsoluteTime, which is a deprecated datatype that will have to disappear eventually.
2005-06-30 00:51:57 +02:00
* Remember postmaster startup time
Add pg_postmaster_start_time() function. Euler Taveira de Oliveira Matthias Schmidt
2005-06-14 23:04:42 +02:00
*/
Clean up the rather historically encumbered interface to now() and current time: provide a GetCurrentTimestamp() function that returns current time in the form of a TimestampTz, instead of separate time_t and microseconds fields. This is what all the callers really want anyway, and it eliminates low-level dependencies on AbsoluteTime, which is a deprecated datatype that will have to disappear eventually.
2005-06-30 00:51:57 +02:00
PgStartTime = GetCurrentTimestamp();
Add pg_postmaster_start_time() function. Euler Taveira de Oliveira Matthias Schmidt
2005-06-14 23:04:42 +02:00
Change pg_ctl to detect server-ready by watching status in postmaster.pid. Traditionally, "pg_ctl start -w" has waited for the server to become ready to accept connections by attempting a connection once per second. That has the major problem that connection issues (for instance, a kernel packet filter blocking traffic) can't be reliably told apart from server startup issues, and the minor problem that if server startup isn't quick, we accumulate "the database system is starting up" spam in the server log. We've hacked around many of the possible connection issues, but it resulted in ugly and complicated code in pg_ctl.c. In commit c61559ec3, I changed the probe rate to every tenth of a second. That prompted Jeff Janes to complain that the log-spam problem had become much worse. In the ensuing discussion, Andres Freund pointed out that we could dispense with connection attempts altogether if the postmaster were changed to report its status in postmaster.pid, which "pg_ctl start" already relies on being able to read. This patch implements that, teaching postmaster.c to report a status string into the pidfile at the same state-change points already identified as being of interest for systemd status reporting (cf commit 7d17e683f). pg_ctl no longer needs to link with libpq at all; all its functions now depend on reading server files. In support of this, teach AddToDataDirLockFile() to allow addition of postmaster.pid lines in not-necessarily-sequential order. This is needed on Windows where the SHMEM_KEY line will never be written at all. We still have the restriction that we don't want to truncate the pidfile; document the reasons for that a bit better. Also, fix the pg_ctl TAP tests so they'll notice if "start -w" mode is broken --- before, they'd just wait out the sixty seconds until the loop gives up, and then report success anyway. (Yes, I found that out the hard way.) While at it, arrange for pg_ctl to not need to #include miscadmin.h; as a rather low-level backend header, requiring that to be compilable client-side is pretty dubious. This requires moving the #define's associated with the pidfile into a new header file, and moving PG_BACKEND_VERSIONSTR someplace else. For lack of a clearly better "someplace else", I put it into port.h, beside the declaration of find_other_exec(), since most users of that macro are passing the value to find_other_exec(). (initdb still depends on miscadmin.h, but at least pg_ctl and pg_upgrade no longer do.) In passing, fix main.c so that PG_BACKEND_VERSIONSTR actually defines the output of "postgres -V", which remarkably it had never done before. Discussion: https://postgr.es/m/CAMkU=1xJW8e+CTotojOMBd-yzUvD0e_JZu2xHo=MnuZ4__m7Pg@mail.gmail.com
2017-06-28 23:31:24 +02:00
/*
* Report postmaster status in the postmaster.pid file, to allow pg_ctl to
* see what's happening.
*/
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STARTING);
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
/*
Make the order of operations in PostmasterMain a bit saner ... some recent patches had added stuff in rather random spots.
2006-07-25 03:23:34 +02:00
* We're ready to rock and roll...
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
*/
Make the order of operations in PostmasterMain a bit saner ... some recent patches had added stuff in rather random spots.
2006-07-25 03:23:34 +02:00
StartupPID = StartupDataBase();
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
Assert(StartupPID != 0);
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
StartupStatus = STARTUP_RUNNING;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
pmState = PM_STARTUP;
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/* Some workers may be scheduled to start now */
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
maybe_start_bgworkers();
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
status = ServerLoop();
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* ServerLoop probably shouldn't ever return, but if it does, close down.
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
*/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
ExitPostmaster(status != STATUS_OK);
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
Unify calling conventions for postgres/postmaster sub-main functions There was a wild mix of calling conventions: Some were declared to return void and didn't return, some returned an int exit code, some claimed to return an exit code, which the callers checked, but actually never returned, and so on. Now all of these functions are declared to return void and decorated with attribute noreturn and don't return. That's easiest, and most code already worked that way.
2012-06-25 20:25:26 +02:00
abort(); /* not reached */
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Fix incorrect order of lock file removal and failure to close() sockets. Commit c9b0cbe98bd783e24a8c4d8d8ac472a494b81292 accidentally broke the order of operations during postmaster shutdown: it resulted in removing the per-socket lockfiles after, not before, postmaster.pid. This creates a race-condition hazard for a new postmaster that's started immediately after observing that postmaster.pid has disappeared; if it sees the socket lockfile still present, it will quite properly refuse to start. This error appears to be the explanation for at least some of the intermittent buildfarm failures we've seen in the pg_upgrade test. Another problem, which has been there all along, is that the postmaster has never bothered to close() its listen sockets, but has just allowed them to close at process death. This creates a different race condition for an incoming postmaster: it might be unable to bind to the desired listen address because the old postmaster is still incumbent. This might explain some odd failures we've seen in the past, too. (Note: this is not related to the fact that individual backends don't close their client communication sockets. That behavior is intentional and is not changed by this patch.) Fix by adding an on_proc_exit function that closes the postmaster's ports explicitly, and (in 9.3 and up) reshuffling the responsibility for where to unlink the Unix socket files. Lock file unlinking can stay where it is, but teach it to unlink the lock files in reverse order of creation.
2015-08-02 20:54:44 +02:00
/*
* on_proc_exit callback to close server's listen sockets
*/
static void
CloseServerPorts(int status, Datum arg)
{
int i;
/*
* First, explicitly close all the socket FDs. We used to just let this
* happen implicitly at postmaster exit, but it's better to close them
* before we remove the postmaster.pid lockfile; otherwise there's a race
* condition if a new postmaster wants to re-use the TCP port number.
*/
for (i = 0; i < MAXLISTEN; i++)
{
if (ListenSocket[i] != PGINVALID_SOCKET)
{
StreamClose(ListenSocket[i]);
ListenSocket[i] = PGINVALID_SOCKET;
}
}
/*
* Next, remove any filesystem entries for Unix sockets. To avoid race
* conditions against incoming postmasters, this must happen after closing
* the sockets and before removing lock files.
*/
RemoveSocketFiles();
/*
* We don't do anything about socket lock files here; those will be
* removed in a later on_proc_exit callback.
*/
}
Remove external PID file on postmaster exit
2012-08-21 05:47:11 +02:00
/*
* on_proc_exit callback to delete external_pid_file
*/
static void
unlink_external_pid_file(int status, Datum arg)
{
if (external_pid_file)
unlink(external_pid_file);
}
Install some simple defenses in postmaster startup to help ensure a useful error message if the installation directory layout is messed up (or at least, something more useful than the behavior exhibited in bug #4787). During postmaster startup, check that get_pkglib_path resolves as a readable directory; and if ParseTzFile() fails to open the expected timezone abbreviation file, check the possibility that the directory is missing rather than just the specified file. In case of either failure, issue a hint suggesting that the installation is broken. These two checks cover the lib/ and share/ trees of a full installation, which should take care of most scenarios where a sysadmin decides to get cute.
2009-05-03 00:02:37 +02:00
/*
* Compute and check the directory paths to files that are part of the
* installation (as deduced from the postgres executable's own location)
*/
static void
getInstallationPaths(const char *argv0)
{
DIR *pdir;
/* Locate the postgres executable itself */
if (find_my_exec(argv0, my_exec_path) < 0)
elog(FATAL, "%s: could not locate my own executable path", argv0);
#ifdef EXEC_BACKEND
/* Locate executable backend before we change working directory */
if (find_other_exec(argv0, "postgres", PG_BACKEND_VERSIONSTR,
postgres_exec_path) < 0)
ereport(FATAL,
(errmsg("%s: could not locate matching postgres executable",
argv0)));
#endif
/*
* Locate the pkglib directory --- this has to be set early in case we try
* to load any modules from it in response to postgresql.conf entries.
*/
get_pkglib_path(my_exec_path, pkglib_path);
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* Verify that there's a readable directory there; otherwise the Postgres
* installation is incomplete or corrupt. (A typical cause of this
* failure is that the postgres executable has been moved or hardlinked to
* some directory that's not a sibling of the installation lib/
* directory.)
Install some simple defenses in postmaster startup to help ensure a useful error message if the installation directory layout is messed up (or at least, something more useful than the behavior exhibited in bug #4787). During postmaster startup, check that get_pkglib_path resolves as a readable directory; and if ParseTzFile() fails to open the expected timezone abbreviation file, check the possibility that the directory is missing rather than just the specified file. In case of either failure, issue a hint suggesting that the installation is broken. These two checks cover the lib/ and share/ trees of a full installation, which should take care of most scenarios where a sysadmin decides to get cute.
2009-05-03 00:02:37 +02:00
*/
pdir = AllocateDir(pkglib_path);
if (pdir == NULL)
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not open directory \"%s\": %m",
pkglib_path),
errhint("This may indicate an incomplete PostgreSQL installation, or that the file \"%s\" has been moved away from its proper location.",
my_exec_path)));
FreeDir(pdir);
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* XXX is it worth similarly checking the share/ directory? If the lib/
* directory is there, then share/ probably is too.
Install some simple defenses in postmaster startup to help ensure a useful error message if the installation directory layout is messed up (or at least, something more useful than the behavior exhibited in bug #4787). During postmaster startup, check that get_pkglib_path resolves as a readable directory; and if ParseTzFile() fails to open the expected timezone abbreviation file, check the possibility that the directory is missing rather than just the specified file. In case of either failure, issue a hint suggesting that the installation is broken. These two checks cover the lib/ and share/ trees of a full installation, which should take care of most scenarios where a sysadmin decides to get cute.
2009-05-03 00:02:37 +02:00
*/
}
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/*
Allow group access on PGDATA Allow the cluster to be optionally init'd with read access for the group. This means a relatively non-privileged user can perform a backup of the cluster without requiring write privileges, which enhances security. The mode of PGDATA is used to determine whether group permissions are enabled for directory and file creates. This method was chosen as it's simple and works well for the various utilities that write into PGDATA. Changing the mode of PGDATA manually will not automatically change the mode of all the files contained therein. If the user would like to enable group access on an existing cluster then changing the mode of all the existing files will be required. Note that pg_upgrade will automatically change the mode of all migrated files if the new cluster is init'd with the -g option. Tests are included for the backend and all the utilities which operate on the PG data directory to ensure that the correct mode is set based on the data directory permissions. Author: David Steele <david@pgmasters.net> Reviewed-By: Michael Paquier, with discussion amongst many others. Discussion: https://postgr.es/m/ad346fe6-b23e-59f1-ecb7-0e08390ad629%40pgmasters.net
2018-04-07 23:45:39 +02:00
* Check that pg_control exists in the correct location in the data directory.
*
* No attempt is made to validate the contents of pg_control here. This is
* just a sanity check to see if we are looking at a real data directory.
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
*/
static void
Allow group access on PGDATA Allow the cluster to be optionally init'd with read access for the group. This means a relatively non-privileged user can perform a backup of the cluster without requiring write privileges, which enhances security. The mode of PGDATA is used to determine whether group permissions are enabled for directory and file creates. This method was chosen as it's simple and works well for the various utilities that write into PGDATA. Changing the mode of PGDATA manually will not automatically change the mode of all the files contained therein. If the user would like to enable group access on an existing cluster then changing the mode of all the existing files will be required. Note that pg_upgrade will automatically change the mode of all migrated files if the new cluster is init'd with the -g option. Tests are included for the backend and all the utilities which operate on the PG data directory to ensure that the correct mode is set based on the data directory permissions. Author: David Steele <david@pgmasters.net> Reviewed-By: Michael Paquier, with discussion amongst many others. Discussion: https://postgr.es/m/ad346fe6-b23e-59f1-ecb7-0e08390ad629%40pgmasters.net
2018-04-07 23:45:39 +02:00
checkControlFile(void)
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
{
char path[MAXPGPATH];
FILE *fp;
Whack some sense into the configuration-file-location patch. Refactor code into something reasonably understandable, cause use of the feature to not fail in standalone backends or in EXEC_BACKEND case, fix sloppy guc.c table entries, make the documentation minimally usable.
2004-10-08 03:36:36 +02:00
snprintf(path, sizeof(path), "%s/global/pg_control", DataDir);
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
fp = AllocateFile(path, PG_BINARY_R);
if (fp == NULL)
{
Back out pg_autovacuum commit after cvs clean failure causes commit.
2004-07-21 22:34:50 +02:00
write_stderr("%s: could not find the database system\n"
"Expected to find it in the directory \"%s\",\n"
"but could not open file \"%s\": %s\n",
Whack some sense into the configuration-file-location patch. Refactor code into something reasonably understandable, cause use of the feature to not fail in standalone backends or in EXEC_BACKEND case, fix sloppy guc.c table entries, make the documentation minimally usable.
2004-10-08 03:36:36 +02:00
progname, DataDir, path, strerror(errno));
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
ExitPostmaster(2);
}
FreeFile(fp);
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
* Determine how long should we let ServerLoop sleep.
*
* In normal conditions we wait at most one minute, to ensure that the other
* background tasks handled by ServerLoop get done even when no requests are
* arriving. However, if there are background workers waiting to be started,
In immediate shutdown, postmaster should not exit till children are gone. This adjusts commit 82233ce7ea42d6ba519aaec63008aff49da6c7af so that the postmaster does not exit until all its child processes have exited, even if the 5-second timeout elapses and we have to send SIGKILL. There is no great value in having the postmaster process quit sooner, and doing so can mislead onlookers into thinking that the cluster is fully terminated when actually some child processes still survive. This effect might explain recent test failures on buildfarm member hamster, wherein we failed to restart a cluster just after shutting it down with "pg_ctl stop -m immediate". I also did a bit of code review/beautification, including fixing a faulty use of the Max() macro on a volatile expression. Back-patch to 9.4. In older branches, the postmaster never waited for children to exit during immediate shutdowns, and changing that would be too much of a behavioral change.
2015-06-19 20:23:39 +02:00
* we don't actually sleep so that they are quickly serviced. Other exception
* cases are as shown in the code.
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
*/
static void
Initial pgindent run with pg_bsd_indent version 2.0. The new indent version includes numerous fixes thanks to Piotr Stefaniak. The main changes visible in this commit are: * Nicer formatting of function-pointer declarations. * No longer unexpectedly removes spaces in expressions using casts, sizeof, or offsetof. * No longer wants to add a space in "struct structname *varname", as well as some similar cases for const- or volatile-qualified pointers. * Declarations using PG_USED_FOR_ASSERTS_ONLY are formatted more nicely. * Fixes bug where comments following declarations were sometimes placed with no space separating them from the code. * Fixes some odd decisions for comments following case labels. * Fixes some cases where comments following code were indented to less than the expected column 33. On the less good side, it now tends to put more whitespace around typedef names that are not listed in typedefs.list. This might encourage us to put more effort into typedef name collection; it's not really a bug in indent itself. There are more changes coming after this round, having to do with comment indentation and alignment of lines appearing within parentheses. I wanted to limit the size of the diffs to something that could be reviewed without one's eyes completely glazing over, so it seemed better to split up the changes as much as practical. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 20:39:04 +02:00
DetermineSleepTime(struct timeval *timeout)
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
{
TimestampTz next_wakeup = 0;
/*
* Normal case: either there are no background workers at all, or we're in
* a shutdown sequence (during which we ignore bgworkers altogether).
*/
if (Shutdown > NoShutdown ||
(!StartWorkerNeeded && !HaveCrashedWorker))
{
In immediate shutdown, postmaster should not exit till children are gone. This adjusts commit 82233ce7ea42d6ba519aaec63008aff49da6c7af so that the postmaster does not exit until all its child processes have exited, even if the 5-second timeout elapses and we have to send SIGKILL. There is no great value in having the postmaster process quit sooner, and doing so can mislead onlookers into thinking that the cluster is fully terminated when actually some child processes still survive. This effect might explain recent test failures on buildfarm member hamster, wherein we failed to restart a cluster just after shutting it down with "pg_ctl stop -m immediate". I also did a bit of code review/beautification, including fixing a faulty use of the Max() macro on a volatile expression. Back-patch to 9.4. In older branches, the postmaster never waited for children to exit during immediate shutdowns, and changing that would be too much of a behavioral change.
2015-06-19 20:23:39 +02:00
if (AbortStartTime != 0)
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
{
Fix various bugs in postmaster SIGKILL processing Clamp the minimum sleep time during immediate shutdown or crash to a minimum of zero, not a maximum of one second. The previous code could result in a negative sleep time, leading to failure in select() calls. Also, on crash recovery, reset AbortStartTime as soon as SIGKILL is sent or abort processing has commenced instead of waiting until the startup process completes. Reset AbortStartTime as soon as SIGKILL is sent, too, to avoid doing that repeatedly. Per trouble report from Jeff Janes on CAMkU=1xd3=wFqZwwuXPWe4BQs3h1seYo8LV9JtSjW5RodoPxMg@mail.gmail.com Author: MauMau
2013-10-06 04:24:50 +02:00
/* time left to abort; clamp to 0 in case it already expired */
In immediate shutdown, postmaster should not exit till children are gone. This adjusts commit 82233ce7ea42d6ba519aaec63008aff49da6c7af so that the postmaster does not exit until all its child processes have exited, even if the 5-second timeout elapses and we have to send SIGKILL. There is no great value in having the postmaster process quit sooner, and doing so can mislead onlookers into thinking that the cluster is fully terminated when actually some child processes still survive. This effect might explain recent test failures on buildfarm member hamster, wherein we failed to restart a cluster just after shutting it down with "pg_ctl stop -m immediate". I also did a bit of code review/beautification, including fixing a faulty use of the Max() macro on a volatile expression. Back-patch to 9.4. In older branches, the postmaster never waited for children to exit during immediate shutdowns, and changing that would be too much of a behavioral change.
2015-06-19 20:23:39 +02:00
timeout->tv_sec = SIGKILL_CHILDREN_AFTER_SECS -
(time(NULL) - AbortStartTime);
timeout->tv_sec = Max(timeout->tv_sec, 0);
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
timeout->tv_usec = 0;
}
else
{
timeout->tv_sec = 60;
timeout->tv_usec = 0;
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
return;
}
if (StartWorkerNeeded)
{
timeout->tv_sec = 0;
timeout->tv_usec = 0;
return;
}
if (HaveCrashedWorker)
{
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
slist_mutable_iter siter;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
* When there are crashed bgworkers, we sleep just long enough that
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* they are restarted when they request to be. Scan the list to
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
* determine the minimum of all wakeup times according to most recent
* crash time and requested restart interval.
*/
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
slist_foreach_modify(siter, &BackgroundWorkerList)
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
{
RegisteredBgWorker *rw;
TimestampTz this_wakeup;
rw = slist_container(RegisteredBgWorker, rw_lnode, siter.cur);
if (rw->rw_crashed_at == 0)
continue;
Provide a reliable mechanism for terminating a background worker. Although previously-introduced APIs allow the process that registers a background worker to obtain the worker's PID, there's no way to prevent a worker that is not currently running from being restarted. This patch introduces a new API TerminateBackgroundWorker() that prevents the background worker from being restarted, terminates it if it is currently running, and causes it to be unregistered if or when it is not running. Patch by me. Review by Michael Paquier and KaiGai Kohei.
2013-10-18 16:21:25 +02:00
if (rw->rw_worker.bgw_restart_time == BGW_NEVER_RESTART
|| rw->rw_terminate)
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
{
Improve ilist.h's support for deletion of slist elements during iteration. Previously one had to use slist_delete(), implying an additional scan of the list, making this infrastructure considerably less efficient than traditional Lists when deletion of element(s) in a long list is needed. Modify the slist_foreach_modify() macro to support deleting the current element in O(1) time, by keeping a "prev" pointer in addition to "cur" and "next". Although this makes iteration with this macro a bit slower, no real harm is done, since in any scenario where you're not going to delete the current list element you might as well just use slist_foreach instead. Improve the comments about when to use each macro. Back-patch to 9.3 so that we'll have consistent semantics in all branches that provide ilist.h. Note this is an ABI break for callers of slist_foreach_modify(). Andres Freund and Tom Lane
2013-07-24 23:41:55 +02:00
ForgetBackgroundWorker(&siter);
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
continue;
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
this_wakeup = TimestampTzPlusMilliseconds(rw->rw_crashed_at,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
1000L * rw->rw_worker.bgw_restart_time);
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
if (next_wakeup == 0 || this_wakeup < next_wakeup)
next_wakeup = this_wakeup;
}
}
if (next_wakeup != 0)
{
Fix unportable coding in DetermineSleepTime(). We should not assume that struct timeval.tv_sec is a long, because it ain't necessarily. (POSIX says that it's a time_t, which might well be 64 bits now or in the future; or for that matter might be 32 bits on machines with 64-bit longs.) Per buildfarm member panther. Back-patch to 9.3 where the dubious coding was introduced.
2014-02-15 23:09:50 +01:00
long secs;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
int microsecs;
TimestampDifference(GetCurrentTimestamp(), next_wakeup,
Fix unportable coding in DetermineSleepTime(). We should not assume that struct timeval.tv_sec is a long, because it ain't necessarily. (POSIX says that it's a time_t, which might well be 64 bits now or in the future; or for that matter might be 32 bits on machines with 64-bit longs.) Per buildfarm member panther. Back-patch to 9.3 where the dubious coding was introduced.
2014-02-15 23:09:50 +01:00
&secs, &microsecs);
timeout->tv_sec = secs;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
timeout->tv_usec = microsecs;
/* Ensure we don't exceed one minute */
if (timeout->tv_sec > 60)
{
timeout->tv_sec = 60;
timeout->tv_usec = 0;
}
}
else
{
timeout->tv_sec = 60;
timeout->tv_usec = 0;
}
}
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/*
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
* Main idle loop of postmaster
Block signals while computing the sleep time in postmaster's main loop. DetermineSleepTime() was previously called without blocked signals. That's not good, because it allows signal handlers to interrupt its workings. DetermineSleepTime() was added in 9.3 with the addition of background workers (da07a1e856511), where it only read from BackgroundWorkerList. Since 9.4, where dynamic background workers were added (7f7485a0cde), the list is also manipulated in DetermineSleepTime(). That's bad because the list now can be persistently corrupted if modified by both a signal handler and DetermineSleepTime(). This was discovered during the investigation of hangs on buildfarm member anole. It's unclear whether this bug is the source of these hangs or not, but it's worth fixing either way. I have confirmed that it can cause crashes. It luckily looks like this only can cause problems when bgworkers are actively used. Discussion: 20140929193733.GB14400@awork2.anarazel.de Backpatch to 9.3 where background workers were introduced.
2014-10-01 14:23:43 +02:00
*
* NB: Needs to be called with signals blocked
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
*/
Make functions static where possible, enclose unused functions in #ifdef NOT_USED.
1997-08-19 23:40:56 +02:00
static int
The remainder of D'Arcy's changes, most notibly the usage of SIGNAL_ARGS
1996-10-04 22:17:11 +02:00
ServerLoop(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
pgindent run.
2003-08-04 02:43:34 +02:00
fd_set readmask;
pgindent run before 6.3 release, with Thomas' requested changes.
1998-02-26 05:46:47 +01:00
int nSockets;
Fix uninitialized-variable bug. For some reason, neither of the compilers I usually use noticed the uninitialized-variable problem I introduced in commit 7e2a18a9161fee7e. That's hardly a good enough excuse though. Committing with brown paper bag on head. In addition to putting the operations in the right order, move the declaration of "now" inside the loop; there's no need for it to be outside, and that does wake up older gcc enough to notice any similar future problem. Back-patch to 9.4; earlier versions lack the time-to-SIGKILL stanza so there's no bug.
2015-10-09 16:12:03 +02:00
time_t last_lockfile_recheck_time,
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
last_touch_time;
Add real random() call to postmaster for use in cancel.
1998-06-08 06:27:59 +02:00
Perform an immediate shutdown if the postmaster.pid file is removed. The postmaster now checks every minute or so (worst case, at most two minutes) that postmaster.pid is still there and still contains its own PID. If not, it performs an immediate shutdown, as though it had received SIGQUIT. The original goal behind this change was to ensure that failed buildfarm runs would get fully cleaned up, even if the test scripts had left a postmaster running, which is not an infrequent occurrence. When the buildfarm script removes a test postmaster's $PGDATA directory, its next check on postmaster.pid will fail and cause it to exit. Previously, manual intervention was often needed to get rid of such orphaned postmasters, since they'd block new test postmasters from obtaining the expected socket address. However, by checking postmaster.pid and not something else, we can provide additional robustness: manual removal of postmaster.pid is a frequent DBA mistake, and now we can at least limit the damage that will ensue if a new postmaster is started while the old one is still alive. Back-patch to all supported branches, since we won't get the desired improvement in buildfarm reliability otherwise.
2015-10-06 23:15:27 +02:00
last_lockfile_recheck_time = last_touch_time = time(NULL);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Have a go at fixing various outstanding portability issues in code that was modified for IPv6. Use a robust definition of struct sockaddr_storage, do a proper configure test to see if ss_len exists, don't assume that getnameinfo() will handle AF_UNIX sockets, don't trust getaddrinfo to return the protocol we ask for, etc. This incorporates several outstanding patches from Kurt Roeckx, but I'm to blame for anything that doesn't work ...
2003-07-24 01:30:41 +02:00
nSockets = initMasks(&readmask);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
for (;;)
{
Have a go at fixing various outstanding portability issues in code that was modified for IPv6. Use a robust definition of struct sockaddr_storage, do a proper configure test to see if ss_len exists, don't assume that getnameinfo() will handle AF_UNIX sockets, don't trust getaddrinfo to return the protocol we ask for, etc. This incorporates several outstanding patches from Kurt Roeckx, but I'm to blame for anything that doesn't work ...
2003-07-24 01:30:41 +02:00
fd_set rmask;
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
int selres;
Fix uninitialized-variable bug. For some reason, neither of the compilers I usually use noticed the uninitialized-variable problem I introduced in commit 7e2a18a9161fee7e. That's hardly a good enough excuse though. Committing with brown paper bag on head. In addition to putting the operations in the right order, move the declaration of "now" inside the loop; there's no need for it to be outside, and that does wake up older gcc enough to notice any similar future problem. Back-patch to 9.4; earlier versions lack the time-to-SIGKILL stanza so there's no bug.
2015-10-09 16:12:03 +02:00
time_t now;
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
/*
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
* Wait for a connection request to arrive.
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
*
Block signals while computing the sleep time in postmaster's main loop. DetermineSleepTime() was previously called without blocked signals. That's not good, because it allows signal handlers to interrupt its workings. DetermineSleepTime() was added in 9.3 with the addition of background workers (da07a1e856511), where it only read from BackgroundWorkerList. Since 9.4, where dynamic background workers were added (7f7485a0cde), the list is also manipulated in DetermineSleepTime(). That's bad because the list now can be persistently corrupted if modified by both a signal handler and DetermineSleepTime(). This was discovered during the investigation of hangs on buildfarm member anole. It's unclear whether this bug is the source of these hangs or not, but it's worth fixing either way. I have confirmed that it can cause crashes. It luckily looks like this only can cause problems when bgworkers are actively used. Discussion: 20140929193733.GB14400@awork2.anarazel.de Backpatch to 9.3 where background workers were introduced.
2014-10-01 14:23:43 +02:00
* We block all signals except while sleeping. That makes it safe for
* signal handlers, which again block all signals while executing, to
* do nontrivial work.
Revert "Use pselect(2) not select(2), if available, to wait in postmaster's loop." This reverts commit 81069a9efc5a374dd39874a161f456f0fb3afba4. Buildfarm results suggest that some platforms have versions of pselect(2) that are not merely non-atomic, but flat out non-functional. Revert the use-pselect patch to confirm this diagnosis (and exclude the no-SA_RESTART patch as the source of trouble). If it's so, we should probably look into blacklisting specific platforms that have broken pselect. Discussion: https://postgr.es/m/9696.1493072081@sss.pgh.pa.us
2017-04-25 00:29:03 +02:00
*
* If we are in PM_WAIT_DEAD_END state, then we don't want to accept
* any new connections, so we don't call select(), and just sleep.
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
*/
Revert "Use pselect(2) not select(2), if available, to wait in postmaster's loop." This reverts commit 81069a9efc5a374dd39874a161f456f0fb3afba4. Buildfarm results suggest that some platforms have versions of pselect(2) that are not merely non-atomic, but flat out non-functional. Revert the use-pselect patch to confirm this diagnosis (and exclude the no-SA_RESTART patch as the source of trouble). If it's so, we should probably look into blacklisting specific platforms that have broken pselect. Discussion: https://postgr.es/m/9696.1493072081@sss.pgh.pa.us
2017-04-25 00:29:03 +02:00
memcpy((char *) &rmask, (char *) &readmask, sizeof(fd_set));
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (pmState == PM_WAIT_DEAD_END)
{
Revert "Use pselect(2) not select(2), if available, to wait in postmaster's loop." This reverts commit 81069a9efc5a374dd39874a161f456f0fb3afba4. Buildfarm results suggest that some platforms have versions of pselect(2) that are not merely non-atomic, but flat out non-functional. Revert the use-pselect patch to confirm this diagnosis (and exclude the no-SA_RESTART patch as the source of trouble). If it's so, we should probably look into blacklisting specific platforms that have broken pselect. Discussion: https://postgr.es/m/9696.1493072081@sss.pgh.pa.us
2017-04-25 00:29:03 +02:00
PG_SETMASK(&UnBlockSig);
pg_usleep(100000L); /* 100 msec seems reasonable */
selres = 0;
PG_SETMASK(&BlockSig);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
}
else
{
Revert "Use pselect(2) not select(2), if available, to wait in postmaster's loop." This reverts commit 81069a9efc5a374dd39874a161f456f0fb3afba4. Buildfarm results suggest that some platforms have versions of pselect(2) that are not merely non-atomic, but flat out non-functional. Revert the use-pselect patch to confirm this diagnosis (and exclude the no-SA_RESTART patch as the source of trouble). If it's so, we should probably look into blacklisting specific platforms that have broken pselect. Discussion: https://postgr.es/m/9696.1493072081@sss.pgh.pa.us
2017-04-25 00:29:03 +02:00
/* must set timeout each time; some OSes change it! */
struct timeval timeout;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
Block signals while computing the sleep time in postmaster's main loop. DetermineSleepTime() was previously called without blocked signals. That's not good, because it allows signal handlers to interrupt its workings. DetermineSleepTime() was added in 9.3 with the addition of background workers (da07a1e856511), where it only read from BackgroundWorkerList. Since 9.4, where dynamic background workers were added (7f7485a0cde), the list is also manipulated in DetermineSleepTime(). That's bad because the list now can be persistently corrupted if modified by both a signal handler and DetermineSleepTime(). This was discovered during the investigation of hangs on buildfarm member anole. It's unclear whether this bug is the source of these hangs or not, but it's worth fixing either way. I have confirmed that it can cause crashes. It luckily looks like this only can cause problems when bgworkers are actively used. Discussion: 20140929193733.GB14400@awork2.anarazel.de Backpatch to 9.3 where background workers were introduced.
2014-10-01 14:23:43 +02:00
/* Needs to run with blocked signals! */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
DetermineSleepTime(&timeout);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
Block signals while computing the sleep time in postmaster's main loop. DetermineSleepTime() was previously called without blocked signals. That's not good, because it allows signal handlers to interrupt its workings. DetermineSleepTime() was added in 9.3 with the addition of background workers (da07a1e856511), where it only read from BackgroundWorkerList. Since 9.4, where dynamic background workers were added (7f7485a0cde), the list is also manipulated in DetermineSleepTime(). That's bad because the list now can be persistently corrupted if modified by both a signal handler and DetermineSleepTime(). This was discovered during the investigation of hangs on buildfarm member anole. It's unclear whether this bug is the source of these hangs or not, but it's worth fixing either way. I have confirmed that it can cause crashes. It luckily looks like this only can cause problems when bgworkers are actively used. Discussion: 20140929193733.GB14400@awork2.anarazel.de Backpatch to 9.3 where background workers were introduced.
2014-10-01 14:23:43 +02:00
PG_SETMASK(&UnBlockSig);
Revert "Use pselect(2) not select(2), if available, to wait in postmaster's loop." This reverts commit 81069a9efc5a374dd39874a161f456f0fb3afba4. Buildfarm results suggest that some platforms have versions of pselect(2) that are not merely non-atomic, but flat out non-functional. Revert the use-pselect patch to confirm this diagnosis (and exclude the no-SA_RESTART patch as the source of trouble). If it's so, we should probably look into blacklisting specific platforms that have broken pselect. Discussion: https://postgr.es/m/9696.1493072081@sss.pgh.pa.us
2017-04-25 00:29:03 +02:00
selres = select(nSockets, &rmask, NULL, NULL, &timeout);
Background writer process This first part of the background writer does no syncing at all. It's only purpose is to keep the LRU heads clean so that regular backends seldom to never have to call write(). Jan
2003-11-19 16:55:08 +01:00
Block signals while computing the sleep time in postmaster's main loop. DetermineSleepTime() was previously called without blocked signals. That's not good, because it allows signal handlers to interrupt its workings. DetermineSleepTime() was added in 9.3 with the addition of background workers (da07a1e856511), where it only read from BackgroundWorkerList. Since 9.4, where dynamic background workers were added (7f7485a0cde), the list is also manipulated in DetermineSleepTime(). That's bad because the list now can be persistently corrupted if modified by both a signal handler and DetermineSleepTime(). This was discovered during the investigation of hangs on buildfarm member anole. It's unclear whether this bug is the source of these hangs or not, but it's worth fixing either way. I have confirmed that it can cause crashes. It luckily looks like this only can cause problems when bgworkers are actively used. Discussion: 20140929193733.GB14400@awork2.anarazel.de Backpatch to 9.3 where background workers were introduced.
2014-10-01 14:23:43 +02:00
PG_SETMASK(&BlockSig);
}
Check for SIGHUP and process config file updates just after waiting for input, not just before.
2000-10-24 23:33:52 +02:00
Revert "Use pselect(2) not select(2), if available, to wait in postmaster's loop." This reverts commit 81069a9efc5a374dd39874a161f456f0fb3afba4. Buildfarm results suggest that some platforms have versions of pselect(2) that are not merely non-atomic, but flat out non-functional. Revert the use-pselect patch to confirm this diagnosis (and exclude the no-SA_RESTART patch as the source of trouble). If it's so, we should probably look into blacklisting specific platforms that have broken pselect. Discussion: https://postgr.es/m/9696.1493072081@sss.pgh.pa.us
2017-04-25 00:29:03 +02:00
/* Now check the select() result */
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
if (selres < 0)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Fix oversight: in case where SIGTERM is received while there are live backends, the archiver and stats processes never got sent a kill signal. They'd eventually exit on their own, but not for awhile, which is a bit annoying when you are trying to replace the executable file on a platform that doesn't allow removal of busy executables. Also, tweak main loop logic so that we will perform the background tasks after select() returns EINTR.
2004-07-27 03:46:03 +02:00
if (errno != EINTR && errno != EWOULDBLOCK)
{
ereport(LOG,
(errcode_for_socket_access(),
errmsg("select() failed in postmaster: %m")));
return STATUS_ERROR;
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Check for SIGHUP and process config file updates just after waiting for input, not just before.
2000-10-24 23:33:52 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* New connection pending on any of our sockets? If so, fork a child
* process to deal with it.
Attached is a patch that fixes these leaks, and does a couple other things as well: * Computes and saves a cancel key for each backend. * fflush before forking, to eliminate double-buffering problems between postmaster and backends. Other cleanups. Tom Lane
1998-06-09 06:06:12 +02:00
*/
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
if (selres > 0)
Add real random() call to postmaster for use in cancel.
1998-06-08 06:27:59 +02:00
{
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
int i;
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
for (i = 0; i < MAXLISTEN; i++)
A bit of code beautification/cleanup of obsolete comments. Rethink ordering of startup operations in one or two places.
2001-06-21 18:43:24 +02:00
{
Create typedef pgsocket for storing socket descriptors. This silences some warnings on Win64. Not using the proper SOCKET datatype was actually wrong on Win32 as well, but didn't cause any warnings there. Also create define PGINVALID_SOCKET to indicate an invalid/non-existing socket, instead of using a hardcoded -1 value.
2010-01-10 15:16:08 +01:00
if (ListenSocket[i] == PGINVALID_SOCKET)
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
break;
if (FD_ISSET(ListenSocket[i], &rmask))
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
{
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
Port *port;
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
port = ConnCreate(ListenSocket[i]);
if (port)
{
BackendStartup(port);
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We no longer need the open socket or port structure
* in this process
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
*/
StreamClose(port->sock);
ConnFree(port);
}
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
}
A bit of code beautification/cleanup of obsolete comments. Rethink ordering of startup operations in one or two places.
2001-06-21 18:43:24 +02:00
}
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
}
Tweak stats collector start logic so that we will not try to spawn a new stats collector oftener than once a minute. Per gripe from Erik Walthinsen 4/25/03.
2003-04-26 04:57:14 +02:00
Provide for logfiles in machine readable CSV format. In consequence, rename redirect_stderr to logging_collector. Original patch from Arul Shaji, subsequently modified by Greg Smith, and then heavily modified by me.
2007-08-19 03:41:25 +02:00
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
SysLoggerPID = SysLogger_Start();
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
/*
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* If no background writer process is running, and we are not in a
* state that prevents it, start one. It doesn't matter if this
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
* fails, we'll just try again later. Likewise for the checkpointer.
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
*/
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
if (pmState == PM_RUN || pmState == PM_RECOVERY ||
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
pmState == PM_HOT_STANDBY)
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
{
if (CheckpointerPID == 0)
CheckpointerPID = StartCheckpointer();
Checkpointer starts before bgwriter to avoid missing fsync requests. Noted while testing Hot Standby startup.
2012-06-01 09:25:17 +02:00
if (BgWriterPID == 0)
BgWriterPID = StartBackgroundWriter();
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
}
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* Likewise, if we have lost the walwriter process, try to start a new
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
* one. But this is needed only in normal operation (else we cannot
* be writing any new WAL).
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
*/
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (WalWriterPID == 0 && pmState == PM_RUN)
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
WalWriterPID = StartWalWriter();
Add postmaster/postgres undocumented -b option for binary upgrades. This option turns off autovacuum, prevents non-super-user connections, and enables oid setting hooks in the backend. The code continues to use the old autoavacuum disable settings for servers with earlier catalog versions. This includes a catalog version bump to identify servers that support the -b option.
2011-04-25 18:00:21 +02:00
/*
Pgindent run before 9.1 beta2.
2011-06-09 20:32:50 +02:00
* If we have lost the autovacuum launcher, try to start a new one. We
* don't want autovacuum to run in binary upgrade mode because
* autovacuum might update relfrozenxid for empty tables before the
* physical files are put in place.
Add postmaster/postgres undocumented -b option for binary upgrades. This option turns off autovacuum, prevents non-super-user connections, and enables oid setting hooks in the backend. The code continues to use the old autoavacuum disable settings for servers with earlier catalog versions. This includes a catalog version bump to identify servers that support the -b option.
2011-04-25 18:00:21 +02:00
*/
if (!IsBinaryUpgrade && AutoVacPID == 0 &&
Treat the autovac launcher more like a regular backend, in that we wait for it to die before telling the bgwriter to initiate shutdown checkpoint. Since it's connected to shared memory, this seems more prudent than the alternative of letting it quit asynchronously. Resolves my complaint of yesterday about repeated shutdown checkpoints in CVS HEAD.
2007-07-01 20:28:41 +02:00
(AutoVacuumingActive() || start_autovac_launcher) &&
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
pmState == PM_RUN)
Fix recently-understood problems with handling of XID freezing, particularly in PITR scenarios. We now WAL-log the replacement of old XIDs with FrozenTransactionId, so that such replacement is guaranteed to propagate to PITR slave databases. Also, rather than relying on hint-bit updates to be preserved, pg_clog is not truncated until all instances of an XID are known to have been replaced by FrozenTransactionId. Add new GUC variables and pg_autovacuum columns to allow management of the freezing policy, so that users can trade off the size of pg_clog against the amount of freezing work done. Revise the already-existing code that forces autovacuum of tables approaching the wraparound point to make it more bulletproof; also, revise the autovacuum logic so that anti-wraparound vacuuming is done per-table rather than per-database. initdb forced because of changes in pg_class, pg_database, and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
{
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
AutoVacPID = StartAutoVacLauncher();
Fix recently-understood problems with handling of XID freezing, particularly in PITR scenarios. We now WAL-log the replacement of old XIDs with FrozenTransactionId, so that such replacement is guaranteed to propagate to PITR slave databases. Also, rather than relying on hint-bit updates to be preserved, pg_clog is not truncated until all instances of an XID are known to have been replaced by FrozenTransactionId. Add new GUC variables and pg_autovacuum columns to allow management of the freezing policy, so that users can trade off the size of pg_clog against the amount of freezing work done. Revise the already-existing code that forces autovacuum of tables approaching the wraparound point to make it more bulletproof; also, revise the autovacuum logic so that anti-wraparound vacuuming is done per-table rather than per-database. initdb forced because of changes in pg_class, pg_database, and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
if (AutoVacPID != 0)
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
start_autovac_launcher = false; /* signal processed */
Fix recently-understood problems with handling of XID freezing, particularly in PITR scenarios. We now WAL-log the replacement of old XIDs with FrozenTransactionId, so that such replacement is guaranteed to propagate to PITR slave databases. Also, rather than relying on hint-bit updates to be preserved, pg_clog is not truncated until all instances of an XID are known to have been replaced by FrozenTransactionId. Add new GUC variables and pg_autovacuum columns to allow management of the freezing policy, so that users can trade off the size of pg_clog against the amount of freezing work done. Revise the already-existing code that forces autovacuum of tables approaching the wraparound point to make it more bulletproof; also, revise the autovacuum logic so that anti-wraparound vacuuming is done per-table rather than per-database. initdb forced because of changes in pg_class, pg_database, and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
}
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
Put back stats-collector restarting code, removed accidentally. Removed that code snippet accidentally in the archive_mode='always' patch. Also, use varname-tags for archive_command in the docs. Fujii Masao
2015-05-18 09:18:46 +02:00
/* If we have lost the stats collector, try to start a new one */
If the stats collector dies during Hot Standby, restart it. This bug exists as far back as 9.0, when Hot Standby was introduced, so back-patch to all supported branches. Report and patch by Takayuki Tsunakawa, reviewed by Michael Paquier and Kuntal Ghosh.
2016-10-27 20:27:40 +02:00
if (PgStatPID == 0 &&
(pmState == PM_RUN || pmState == PM_HOT_STANDBY))
Put back stats-collector restarting code, removed accidentally. Removed that code snippet accidentally in the archive_mode='always' patch. Also, use varname-tags for archive_command in the docs. Fujii Masao
2015-05-18 09:18:46 +02:00
PgStatPID = pgstat_start();
Make postmaster restart archiver soon after it dies, even during recovery. After the archiver dies, postmaster tries to start a new one immediately. But previously this could happen only while server was running normally even though archiving was enabled always (i.e., archive_mode was set to always). So the archiver running during recovery could not restart soon after it died. This is an oversight in commit ffd3774. This commit changes reaper(), postmaster's signal handler to cleanup after a child process dies, so that it tries to a new archiver even during recovery if necessary. Patch by me. Review by Alvaro Herrera.
2015-06-12 16:11:51 +02:00
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
PgArchPID = pgarch_start();
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
Avoid calling kill() in a postmaster signal handler. This causes problems when the system load is high, per report from Zdenek Kotala in <1250860954.1239.114.camel@localhost>; instead of calling kill directly, have the signal handler set a flag which is checked in ServerLoop. This way, the handler can return before being called again by a subsequent signal sent from the autovacuum launcher. Also, increase the sleep in the launcher in this failure path to 1 second. Backpatch to 8.3, which is when the signalling between autovacuum launcher/postmaster was introduced. Also, add a couple of ReleasePostmasterChildSlot calls in error paths; this part backpatched to 8.4 which is when the child slot stuff was introduced.
2009-08-24 19:23:02 +02:00
/* If we need to signal the autovacuum launcher, do so now */
if (avlauncher_needs_signal)
{
avlauncher_needs_signal = false;
if (AutoVacPID != 0)
Change the autovacuum launcher to read pg_database directly, rather than via the "flat files" facility. This requires making it enough like a backend to be able to run transactions; it's no longer an "auxiliary process" but more like the autovacuum worker processes. Also, its signal handling has to be brought into line with backends/workers. In particular, since it now has to handle procsignal.c processing, the special autovac-launcher-only signal conditions are moved to SIGUSR2. Alvaro, with some cleanup from Tom
2009-08-31 21:41:00 +02:00
kill(AutoVacPID, SIGUSR2);
Avoid calling kill() in a postmaster signal handler. This causes problems when the system load is high, per report from Zdenek Kotala in <1250860954.1239.114.camel@localhost>; instead of calling kill directly, have the signal handler set a flag which is checked in ServerLoop. This way, the handler can return before being called again by a subsequent signal sent from the autovacuum launcher. Also, increase the sleep in the launcher in this failure path to 1 second. Backpatch to 8.3, which is when the signalling between autovacuum launcher/postmaster was introduced. Also, add a couple of ReleasePostmasterChildSlot calls in error paths; this part backpatched to 8.4 which is when the child slot stuff was introduced.
2009-08-24 19:23:02 +02:00
}
Don't lose walreceiver start requests due to race condition in postmaster. When a walreceiver dies, the startup process will notice that and send a PMSIGNAL_START_WALRECEIVER signal to the postmaster, asking for a new walreceiver to be launched. There's a race condition, which at least in HEAD is very easy to hit, whereby the postmaster might see that signal before it processes the SIGCHLD from the walreceiver process. In that situation, sigusr1_handler() just dropped the start request on the floor, reasoning that it must be redundant. Eventually, after 10 seconds (WALRCV_STARTUP_TIMEOUT), the startup process would make a fresh request --- but that's a long time if the connection could have been re-established almost immediately. Fix it by setting a state flag inside the postmaster that we won't clear until we do launch a walreceiver. In cases where that results in an extra walreceiver launch, it's up to the walreceiver to realize it's unwanted and go away --- but we have, and need, that logic anyway for the opposite race case. I came across this through investigating unexpected delays in the src/test/recovery TAP tests: it manifests there in test cases where a master server is stopped and restarted while leaving streaming slaves active. This logic has been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/21344.1498494720@sss.pgh.pa.us
2017-06-26 23:31:56 +02:00
/* If we need to start a WAL receiver, try to do that now */
if (WalReceiverRequested)
MaybeStartWalReceiver();
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/* Get other worker processes running, if needed */
if (StartWorkerNeeded || HaveCrashedWorker)
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
maybe_start_bgworkers();
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
On Darwin, detect and report a multithreaded postmaster. Darwin --enable-nls builds use a substitute setlocale() that may start a thread. Buildfarm member orangutan experienced BackendList corruption on account of different postmaster threads executing signal handlers simultaneously. Furthermore, a multithreaded postmaster risks undefined behavior from sigprocmask() and fork(). Emit LOG messages about the problem and its workaround. Back-patch to 9.0 (all supported versions).
2015-01-08 04:35:44 +01:00
#ifdef HAVE_PTHREAD_IS_THREADED_NP
/*
* With assertions enabled, check regularly for appearance of
* additional threads. All builds check at start and exit.
*/
Assert(pthread_is_threaded_np() == 0);
#endif
Fix uninitialized-variable bug. For some reason, neither of the compilers I usually use noticed the uninitialized-variable problem I introduced in commit 7e2a18a9161fee7e. That's hardly a good enough excuse though. Committing with brown paper bag on head. In addition to putting the operations in the right order, move the declaration of "now" inside the loop; there's no need for it to be outside, and that does wake up older gcc enough to notice any similar future problem. Back-patch to 9.4; earlier versions lack the time-to-SIGKILL stanza so there's no bug.
2015-10-09 16:12:03 +02:00
/*
* Lastly, check to see if it's time to do some things that we don't
* want to do every single time through the loop, because they're a
* bit expensive. Note that there's up to a minute of slop in when
* these tasks will be performed, since DetermineSleepTime() will let
* us sleep at most that long; except for SIGKILL timeout which has
* special-case logic there.
*/
now = time(NULL);
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
/*
* If we already sent SIGQUIT to children and they are slow to shut
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* down, it's time to send them SIGKILL. This doesn't happen
* normally, but under certain conditions backends can get stuck while
* shutting down. This is a last measure to get them unwedged.
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
*
* Note we also do this during recovery from a process crash.
*/
if ((Shutdown >= ImmediateShutdown || (FatalError && !SendStop)) &&
In immediate shutdown, postmaster should not exit till children are gone. This adjusts commit 82233ce7ea42d6ba519aaec63008aff49da6c7af so that the postmaster does not exit until all its child processes have exited, even if the 5-second timeout elapses and we have to send SIGKILL. There is no great value in having the postmaster process quit sooner, and doing so can mislead onlookers into thinking that the cluster is fully terminated when actually some child processes still survive. This effect might explain recent test failures on buildfarm member hamster, wherein we failed to restart a cluster just after shutting it down with "pg_ctl stop -m immediate". I also did a bit of code review/beautification, including fixing a faulty use of the Max() macro on a volatile expression. Back-patch to 9.4. In older branches, the postmaster never waited for children to exit during immediate shutdowns, and changing that would be too much of a behavioral change.
2015-06-19 20:23:39 +02:00
AbortStartTime != 0 &&
(now - AbortStartTime) >= SIGKILL_CHILDREN_AFTER_SECS)
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
{
/* We were gentle with them before. Not anymore */
TerminateChildren(SIGKILL);
Fix various bugs in postmaster SIGKILL processing Clamp the minimum sleep time during immediate shutdown or crash to a minimum of zero, not a maximum of one second. The previous code could result in a negative sleep time, leading to failure in select() calls. Also, on crash recovery, reset AbortStartTime as soon as SIGKILL is sent or abort processing has commenced instead of waiting until the startup process completes. Reset AbortStartTime as soon as SIGKILL is sent, too, to avoid doing that repeatedly. Per trouble report from Jeff Janes on CAMkU=1xd3=wFqZwwuXPWe4BQs3h1seYo8LV9JtSjW5RodoPxMg@mail.gmail.com Author: MauMau
2013-10-06 04:24:50 +02:00
/* reset flag so we don't SIGKILL again */
AbortStartTime = 0;
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
}
Perform an immediate shutdown if the postmaster.pid file is removed. The postmaster now checks every minute or so (worst case, at most two minutes) that postmaster.pid is still there and still contains its own PID. If not, it performs an immediate shutdown, as though it had received SIGQUIT. The original goal behind this change was to ensure that failed buildfarm runs would get fully cleaned up, even if the test scripts had left a postmaster running, which is not an infrequent occurrence. When the buildfarm script removes a test postmaster's $PGDATA directory, its next check on postmaster.pid will fail and cause it to exit. Previously, manual intervention was often needed to get rid of such orphaned postmasters, since they'd block new test postmasters from obtaining the expected socket address. However, by checking postmaster.pid and not something else, we can provide additional robustness: manual removal of postmaster.pid is a frequent DBA mistake, and now we can at least limit the damage that will ensue if a new postmaster is started while the old one is still alive. Back-patch to all supported branches, since we won't get the desired improvement in buildfarm reliability otherwise.
2015-10-06 23:15:27 +02:00
/*
* Once a minute, verify that postmaster.pid hasn't been removed or
* overwritten. If it has, we force a shutdown. This avoids having
* postmasters and child processes hanging around after their database
* is gone, and maybe causing problems if a new database cluster is
* created in the same place. It also provides some protection
* against a DBA foolishly removing postmaster.pid and manually
* starting a new postmaster. Data corruption is likely to ensue from
* that anyway, but we can minimize the damage by aborting ASAP.
*/
if (now - last_lockfile_recheck_time >= 1 * SECS_PER_MINUTE)
{
if (!RecheckDataDirLockFile())
{
ereport(LOG,
(errmsg("performing immediate shutdown because data directory lock file is invalid")));
kill(MyProcPid, SIGQUIT);
}
last_lockfile_recheck_time = now;
}
/*
* Touch Unix socket and lock files every 58 minutes, to ensure that
* they are not removed by overzealous /tmp-cleaning tasks. We assume
* no one runs cleaners with cutoff times of less than an hour ...
*/
if (now - last_touch_time >= 58 * SECS_PER_MINUTE)
{
TouchSocketFiles();
TouchSocketLockFiles();
last_touch_time = now;
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
New host-based authentication -- send error message when authentication fails
1996-10-12 09:48:49 +02:00
/*
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
* Initialise the masks for select() for the ports we are listening on.
* Return the number of sockets to listen on.
From: Phil Thompson <phil@river-bank.demon.co.uk> I've completed the patch to fix the protocol and authentication issues I was discussing a couple of weeks ago. The particular changes are: - the protocol has a version number - network byte order is used throughout - the pg_hba.conf file is used to specify what method is used to authenticate a frontend (either password, ident, trust, reject, krb4 or krb5) - support for multiplexed backends is removed - appropriate changes to man pages - the -a switch to many programs to specify an authentication service no longer has any effect - the libpq.so version number has changed to 1.1 The new backend still supports the old protocol so old interfaces won't break.
1998-01-26 02:42:53 +01:00
*/
pgindent run before 6.3 release, with Thomas' requested changes.
1998-02-26 05:46:47 +01:00
static int
Have a go at fixing various outstanding portability issues in code that was modified for IPv6. Use a robust definition of struct sockaddr_storage, do a proper configure test to see if ss_len exists, don't assume that getnameinfo() will handle AF_UNIX sockets, don't trust getaddrinfo to return the protocol we ask for, etc. This incorporates several outstanding patches from Kurt Roeckx, but I'm to blame for anything that doesn't work ...
2003-07-24 01:30:41 +02:00
initMasks(fd_set *rmask)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
int maxsock = -1;
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
int i;
From: Phil Thompson <phil@river-bank.demon.co.uk> I've completed the patch to fix the protocol and authentication issues I was discussing a couple of weeks ago. The particular changes are: - the protocol has a version number - network byte order is used throughout - the pg_hba.conf file is used to specify what method is used to authenticate a frontend (either password, ident, trust, reject, krb4 or krb5) - support for multiplexed backends is removed - appropriate changes to man pages - the -a switch to many programs to specify an authentication service no longer has any effect - the libpq.so version number has changed to 1.1 The new backend still supports the old protocol so old interfaces won't break.
1998-01-26 02:42:53 +01:00
FD_ZERO(rmask);
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
for (i = 0; i < MAXLISTEN; i++)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
pgindent run.
2003-08-04 02:43:34 +02:00
int fd = ListenSocket[i];
Have a go at fixing various outstanding portability issues in code that was modified for IPv6. Use a robust definition of struct sockaddr_storage, do a proper configure test to see if ss_len exists, don't assume that getnameinfo() will handle AF_UNIX sockets, don't trust getaddrinfo to return the protocol we ask for, etc. This incorporates several outstanding patches from Kurt Roeckx, but I'm to blame for anything that doesn't work ...
2003-07-24 01:30:41 +02:00
Create typedef pgsocket for storing socket descriptors. This silences some warnings on Win64. Not using the proper SOCKET datatype was actually wrong on Win32 as well, but didn't cause any warnings there. Also create define PGINVALID_SOCKET to indicate an invalid/non-existing socket, instead of using a hardcoded -1 value.
2010-01-10 15:16:08 +01:00
if (fd == PGINVALID_SOCKET)
Have a go at fixing various outstanding portability issues in code that was modified for IPv6. Use a robust definition of struct sockaddr_storage, do a proper configure test to see if ss_len exists, don't assume that getnameinfo() will handle AF_UNIX sockets, don't trust getaddrinfo to return the protocol we ask for, etc. This incorporates several outstanding patches from Kurt Roeckx, but I'm to blame for anything that doesn't work ...
2003-07-24 01:30:41 +02:00
break;
pgindent run for 9.0, second run
2010-07-06 21:19:02 +02:00
FD_SET(fd, rmask);
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (fd > maxsock)
maxsock = fd;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
From: Phil Thompson <phil@river-bank.demon.co.uk> I've completed the patch to fix the protocol and authentication issues I was discussing a couple of weeks ago. The particular changes are: - the protocol has a version number - network byte order is used throughout - the pg_hba.conf file is used to specify what method is used to authenticate a frontend (either password, ident, trust, reject, krb4 or krb5) - support for multiplexed backends is removed - appropriate changes to man pages - the -a switch to many programs to specify an authentication service no longer has any effect - the libpq.so version number has changed to 1.1 The new backend still supports the old protocol so old interfaces won't break.
1998-01-26 02:42:53 +01:00
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
return maxsock + 1;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
New host-based authentication -- send error message when authentication fails
1996-10-12 09:48:49 +02:00
/*
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
* Read a client's startup packet and do something according to it.
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
*
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
* Returns STATUS_OK or STATUS_ERROR, or might call ereport(FATAL) and
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
* not return at all.
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
*
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
* (Note that ereport(FATAL) stuff is sent to the client, so only use it
Move init_ps_display from postgres.c to postmaster.c, putting it just after receipt of the startup packet. Now, postmaster children that are waiting for client authentication response will show as 'postgres: user database host authentication'. Also, do an init_ps_display for startup/shutdown/checkpoint subprocesses, so that they are readily identifiable as well. Fix an obscure race condition that could lead to Assert failure in the postmaster --- attempting to start a checkpoint process before any connections have been received led to calling PostmasterRandom before setting random_seed.
2001-10-19 02:44:08 +02:00
* if that's what you want. Return STATUS_ERROR if you don't want to
* send anything to the client, which would typically be appropriate
* if we detect a communications failure.)
GSSAPI encryption support On both the frontend and backend, prepare for GSSAPI encryption support by moving common code for error handling into a separate file. Fix a TODO for handling multiple status messages in the process. Eliminate the OIDs, which have not been needed for some time. Add frontend and backend encryption support functions. Keep the context initiation for authentication-only separate on both the frontend and backend in order to avoid concerns about changing the requested flags to include encryption support. In postmaster, pull GSSAPI authorization checking into a shared function. Also share the initiator name between the encryption and non-encryption codepaths. For HBA, add "hostgssenc" and "hostnogssenc" entries that behave similarly to their SSL counterparts. "hostgssenc" requires either "gss", "trust", or "reject" for its authentication. Similarly, add a "gssencmode" parameter to libpq. Supported values are "disable", "require", and "prefer". Notably, negotiation will only be attempted if credentials can be acquired. Move credential acquisition into its own function to support this behavior. Add a simple pg_stat_gssapi view similar to pg_stat_ssl, for monitoring if GSSAPI authentication was used, what principal was used, and if encryption is being used on the connection. Finally, add documentation for everything new, and update existing documentation on connection security. Thanks to Michael Paquier for the Windows fixes. Author: Robbie Harwood, with changes to the read/write functions by me. Reviewed in various forms and at different times by: Michael Paquier, Andres Freund, David Steele. Discussion: https://www.postgresql.org/message-id/flat/jlg1tgq1ktm.fsf@thriss.redhat.com
2019-04-03 21:02:33 +02:00
*
* Set secure_done when negotiation of an encrypted layer (currently, TLS or
* GSSAPI) is already completed.
From: Phil Thompson <phil@river-bank.demon.co.uk> I've completed the patch to fix the protocol and authentication issues I was discussing a couple of weeks ago. The particular changes are: - the protocol has a version number - network byte order is used throughout - the pg_hba.conf file is used to specify what method is used to authenticate a frontend (either password, ident, trust, reject, krb4 or krb5) - support for multiplexed backends is removed - appropriate changes to man pages - the -a switch to many programs to specify an authentication service no longer has any effect - the libpq.so version number has changed to 1.1 The new backend still supports the old protocol so old interfaces won't break.
1998-01-26 02:42:53 +01:00
*/
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
static int
GSSAPI encryption support On both the frontend and backend, prepare for GSSAPI encryption support by moving common code for error handling into a separate file. Fix a TODO for handling multiple status messages in the process. Eliminate the OIDs, which have not been needed for some time. Add frontend and backend encryption support functions. Keep the context initiation for authentication-only separate on both the frontend and backend in order to avoid concerns about changing the requested flags to include encryption support. In postmaster, pull GSSAPI authorization checking into a shared function. Also share the initiator name between the encryption and non-encryption codepaths. For HBA, add "hostgssenc" and "hostnogssenc" entries that behave similarly to their SSL counterparts. "hostgssenc" requires either "gss", "trust", or "reject" for its authentication. Similarly, add a "gssencmode" parameter to libpq. Supported values are "disable", "require", and "prefer". Notably, negotiation will only be attempted if credentials can be acquired. Move credential acquisition into its own function to support this behavior. Add a simple pg_stat_gssapi view similar to pg_stat_ssl, for monitoring if GSSAPI authentication was used, what principal was used, and if encryption is being used on the connection. Finally, add documentation for everything new, and update existing documentation on connection security. Thanks to Michael Paquier for the Windows fixes. Author: Robbie Harwood, with changes to the read/write functions by me. Reviewed in various forms and at different times by: Michael Paquier, Andres Freund, David Steele. Discussion: https://www.postgresql.org/message-id/flat/jlg1tgq1ktm.fsf@thriss.redhat.com
2019-04-03 21:02:33 +02:00
ProcessStartupPacket(Port *port, bool secure_done)
New host-based authentication -- send error message when authentication fails
1996-10-12 09:48:49 +02:00
{
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
int32 len;
void *buf;
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
ProtocolVersion proto;
MemoryContext oldcontext;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Be more careful to not lose sync in the FE/BE protocol. If any error occurred while we were in the middle of reading a protocol message from the client, we could lose sync, and incorrectly try to interpret a part of another message as a new protocol message. That will usually lead to an "invalid frontend message" error that terminates the connection. However, this is a security issue because an attacker might be able to deliberately cause an error, inject a Query message in what's supposed to be just user data, and have the server execute it. We were quite careful to not have CHECK_FOR_INTERRUPTS() calls or other operations that could ereport(ERROR) in the middle of processing a message, but a query cancel interrupt or statement timeout could nevertheless cause it to happen. Also, the V2 fastpath and COPY handling were not so careful. It's very difficult to recover in the V2 COPY protocol, so we will just terminate the connection on error. In practice, that's what happened previously anyway, as we lost protocol sync. To fix, add a new variable in pqcomm.c, PqCommReadingMsg, that is set whenever we're in the middle of reading a message. When it's set, we cannot safely ERROR out and continue running, because we might've read only part of a message. PqCommReadingMsg acts somewhat similarly to critical sections in that if an error occurs while it's set, the error handler will force the connection to be terminated, as if the error was FATAL. It's not implemented by promoting ERROR to FATAL in elog.c, like ERROR is promoted to PANIC in critical sections, because we want to be able to use PG_TRY/CATCH to recover and regain protocol sync. pq_getmessage() takes advantage of that to prevent an OOM error from terminating the connection. To prevent unnecessary connection terminations, add a holdoff mechanism similar to HOLD/RESUME_INTERRUPTS() that can be used hold off query cancel interrupts, but still allow die interrupts. The rules on which interrupts are processed when are now a bit more complicated, so refactor ProcessInterrupts() and the calls to it in signal handlers so that the signal handlers always call it if ImmediateInterruptOK is set, and ProcessInterrupts() can decide to not do anything if the other conditions are not met. Reported by Emil Lenngren. Patch reviewed by Noah Misch and Andres Freund. Backpatch to all supported versions. Security: CVE-2015-0244
2015-02-02 16:08:45 +01:00
pq_startmsgread();
Don't log incomplete startup packet if it's empty This will stop logging cases where, for example, a monitor opens a connection and immediately closes it. If the packet contains any data an incomplete packet will still be logged. Author: Tom Lane Discussion: https://postgr.es/m/a1379a72-2958-1ed0-ef51-09a21219b155@2ndQuadrant.com
2019-03-06 21:27:19 +01:00
/*
* Grab the first byte of the length word separately, so that we can tell
* whether we have no data at all or an incomplete packet. (This might
* sound inefficient, but it's not really, because of buffering in
* pqcomm.c.)
*/
if (pq_getbytes((char *) &len, 1) == EOF)
Move init_ps_display from postgres.c to postmaster.c, putting it just after receipt of the startup packet. Now, postmaster children that are waiting for client authentication response will show as 'postgres: user database host authentication'. Also, do an init_ps_display for startup/shutdown/checkpoint subprocesses, so that they are readily identifiable as well. Fix an obscure race condition that could lead to Assert failure in the postmaster --- attempting to start a checkpoint process before any connections have been received led to calling PostmasterRandom before setting random_seed.
2001-10-19 02:44:08 +02:00
{
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
/*
Don't log incomplete startup packet if it's empty This will stop logging cases where, for example, a monitor opens a connection and immediately closes it. If the packet contains any data an incomplete packet will still be logged. Author: Tom Lane Discussion: https://postgr.es/m/a1379a72-2958-1ed0-ef51-09a21219b155@2ndQuadrant.com
2019-03-06 21:27:19 +01:00
* If we get no data at all, don't clutter the log with a complaint;
* such cases often occur for legitimate reasons. An example is that
* we might be here after responding to NEGOTIATE_SSL_CODE, and if the
* client didn't like our response, it'll probably just drop the
* connection. Service-monitoring software also often just opens and
* closes a connection without sending anything. (So do port
* scanners, which may be less benign, but it's not really our job to
* notice those.)
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
*/
Don't log incomplete startup packet if it's empty This will stop logging cases where, for example, a monitor opens a connection and immediately closes it. If the packet contains any data an incomplete packet will still be logged. Author: Tom Lane Discussion: https://postgr.es/m/a1379a72-2958-1ed0-ef51-09a21219b155@2ndQuadrant.com
2019-03-06 21:27:19 +01:00
return STATUS_ERROR;
}
if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
{
/* Got a partial length word, so bleat about that */
GSSAPI encryption support On both the frontend and backend, prepare for GSSAPI encryption support by moving common code for error handling into a separate file. Fix a TODO for handling multiple status messages in the process. Eliminate the OIDs, which have not been needed for some time. Add frontend and backend encryption support functions. Keep the context initiation for authentication-only separate on both the frontend and backend in order to avoid concerns about changing the requested flags to include encryption support. In postmaster, pull GSSAPI authorization checking into a shared function. Also share the initiator name between the encryption and non-encryption codepaths. For HBA, add "hostgssenc" and "hostnogssenc" entries that behave similarly to their SSL counterparts. "hostgssenc" requires either "gss", "trust", or "reject" for its authentication. Similarly, add a "gssencmode" parameter to libpq. Supported values are "disable", "require", and "prefer". Notably, negotiation will only be attempted if credentials can be acquired. Move credential acquisition into its own function to support this behavior. Add a simple pg_stat_gssapi view similar to pg_stat_ssl, for monitoring if GSSAPI authentication was used, what principal was used, and if encryption is being used on the connection. Finally, add documentation for everything new, and update existing documentation on connection security. Thanks to Michael Paquier for the Windows fixes. Author: Robbie Harwood, with changes to the read/write functions by me. Reviewed in various forms and at different times by: Michael Paquier, Andres Freund, David Steele. Discussion: https://www.postgresql.org/message-id/flat/jlg1tgq1ktm.fsf@thriss.redhat.com
2019-04-03 21:02:33 +02:00
if (!secure_done)
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("incomplete startup packet")));
Move init_ps_display from postgres.c to postmaster.c, putting it just after receipt of the startup packet. Now, postmaster children that are waiting for client authentication response will show as 'postgres: user database host authentication'. Also, do an init_ps_display for startup/shutdown/checkpoint subprocesses, so that they are readily identifiable as well. Fix an obscure race condition that could lead to Assert failure in the postmaster --- attempting to start a checkpoint process before any connections have been received led to calling PostmasterRandom before setting random_seed.
2001-10-19 02:44:08 +02:00
return STATUS_ERROR;
}
Replace most usages of ntoh[ls] and hton[sl] with pg_bswap.h. All postgres internal usages are replaced, it's just libpq example usages that haven't been converted. External users of libpq can't generally rely on including postgres internal headers. Note that this includes replacing open-coded byte swapping of 64bit integers (using two 32 bit swaps) with a single 64bit swap. Where it looked applicable, I have removed netinet/in.h and arpa/inet.h usage, which previously provided the relevant functionality. It's perfectly possible that I missed other reasons for including those, the buildfarm will tell. Author: Andres Freund Discussion: https://postgr.es/m/20170927172019.gheidqy6xvlxb325@alap3.anarazel.de
2017-10-02 00:36:14 +02:00
len = pg_ntoh32(len);
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
len -= 4;
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
if (len < (int32) sizeof(ProtocolVersion) ||
len > MAX_STARTUP_PACKET_LENGTH)
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
{
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("invalid length of startup packet")));
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
return STATUS_ERROR;
}
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
/*
* Allocate at least the size of an old-style startup packet, plus one
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* extra byte, and make sure all are zeroes. This ensures we will have
* null termination of all strings, in both fixed- and variable-length
* packet layouts.
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
*/
if (len <= (int32) sizeof(StartupPacket))
buf = palloc0(sizeof(StartupPacket) + 1);
else
buf = palloc0(len + 1);
Move init_ps_display from postgres.c to postmaster.c, putting it just after receipt of the startup packet. Now, postmaster children that are waiting for client authentication response will show as 'postgres: user database host authentication'. Also, do an init_ps_display for startup/shutdown/checkpoint subprocesses, so that they are readily identifiable as well. Fix an obscure race condition that could lead to Assert failure in the postmaster --- attempting to start a checkpoint process before any connections have been received led to calling PostmasterRandom before setting random_seed.
2001-10-19 02:44:08 +02:00
if (pq_getbytes(buf, len) == EOF)
{
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("incomplete startup packet")));
Move init_ps_display from postgres.c to postmaster.c, putting it just after receipt of the startup packet. Now, postmaster children that are waiting for client authentication response will show as 'postgres: user database host authentication'. Also, do an init_ps_display for startup/shutdown/checkpoint subprocesses, so that they are readily identifiable as well. Fix an obscure race condition that could lead to Assert failure in the postmaster --- attempting to start a checkpoint process before any connections have been received led to calling PostmasterRandom before setting random_seed.
2001-10-19 02:44:08 +02:00
return STATUS_ERROR;
}
Be more careful to not lose sync in the FE/BE protocol. If any error occurred while we were in the middle of reading a protocol message from the client, we could lose sync, and incorrectly try to interpret a part of another message as a new protocol message. That will usually lead to an "invalid frontend message" error that terminates the connection. However, this is a security issue because an attacker might be able to deliberately cause an error, inject a Query message in what's supposed to be just user data, and have the server execute it. We were quite careful to not have CHECK_FOR_INTERRUPTS() calls or other operations that could ereport(ERROR) in the middle of processing a message, but a query cancel interrupt or statement timeout could nevertheless cause it to happen. Also, the V2 fastpath and COPY handling were not so careful. It's very difficult to recover in the V2 COPY protocol, so we will just terminate the connection on error. In practice, that's what happened previously anyway, as we lost protocol sync. To fix, add a new variable in pqcomm.c, PqCommReadingMsg, that is set whenever we're in the middle of reading a message. When it's set, we cannot safely ERROR out and continue running, because we might've read only part of a message. PqCommReadingMsg acts somewhat similarly to critical sections in that if an error occurs while it's set, the error handler will force the connection to be terminated, as if the error was FATAL. It's not implemented by promoting ERROR to FATAL in elog.c, like ERROR is promoted to PANIC in critical sections, because we want to be able to use PG_TRY/CATCH to recover and regain protocol sync. pq_getmessage() takes advantage of that to prevent an OOM error from terminating the connection. To prevent unnecessary connection terminations, add a holdoff mechanism similar to HOLD/RESUME_INTERRUPTS() that can be used hold off query cancel interrupts, but still allow die interrupts. The rules on which interrupts are processed when are now a bit more complicated, so refactor ProcessInterrupts() and the calls to it in signal handlers so that the signal handlers always call it if ImmediateInterruptOK is set, and ProcessInterrupts() can decide to not do anything if the other conditions are not met. Reported by Emil Lenngren. Patch reviewed by Noah Misch and Andres Freund. Backpatch to all supported versions. Security: CVE-2015-0244
2015-02-02 16:08:45 +01:00
pq_endmsgread();
Load pg_hba.conf and pg_ident.conf on startup and SIGHUP into List of Lists, and use that for user validation. Bruce Momjian
2001-07-30 16:50:24 +02:00
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
/*
* The first field is either a protocol version number or a special
* request code.
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
*/
Replace most usages of ntoh[ls] and hton[sl] with pg_bswap.h. All postgres internal usages are replaced, it's just libpq example usages that haven't been converted. External users of libpq can't generally rely on including postgres internal headers. Note that this includes replacing open-coded byte swapping of 64bit integers (using two 32 bit swaps) with a single 64bit swap. Where it looked applicable, I have removed netinet/in.h and arpa/inet.h usage, which previously provided the relevant functionality. It's perfectly possible that I missed other reasons for including those, the buildfarm will tell. Author: Andres Freund Discussion: https://postgr.es/m/20170927172019.gheidqy6xvlxb325@alap3.anarazel.de
2017-10-02 00:36:14 +02:00
port->proto = proto = pg_ntoh32(*((ProtocolVersion *) buf));
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
if (proto == CANCEL_REQUEST_CODE)
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
{
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
processCancelRequest(port, buf);
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
/* Not really an error, but we don't want to proceed further */
return STATUS_ERROR;
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
}
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
GSSAPI encryption support On both the frontend and backend, prepare for GSSAPI encryption support by moving common code for error handling into a separate file. Fix a TODO for handling multiple status messages in the process. Eliminate the OIDs, which have not been needed for some time. Add frontend and backend encryption support functions. Keep the context initiation for authentication-only separate on both the frontend and backend in order to avoid concerns about changing the requested flags to include encryption support. In postmaster, pull GSSAPI authorization checking into a shared function. Also share the initiator name between the encryption and non-encryption codepaths. For HBA, add "hostgssenc" and "hostnogssenc" entries that behave similarly to their SSL counterparts. "hostgssenc" requires either "gss", "trust", or "reject" for its authentication. Similarly, add a "gssencmode" parameter to libpq. Supported values are "disable", "require", and "prefer". Notably, negotiation will only be attempted if credentials can be acquired. Move credential acquisition into its own function to support this behavior. Add a simple pg_stat_gssapi view similar to pg_stat_ssl, for monitoring if GSSAPI authentication was used, what principal was used, and if encryption is being used on the connection. Finally, add documentation for everything new, and update existing documentation on connection security. Thanks to Michael Paquier for the Windows fixes. Author: Robbie Harwood, with changes to the read/write functions by me. Reviewed in various forms and at different times by: Michael Paquier, Andres Freund, David Steele. Discussion: https://www.postgresql.org/message-id/flat/jlg1tgq1ktm.fsf@thriss.redhat.com
2019-04-03 21:02:33 +02:00
if (proto == NEGOTIATE_SSL_CODE && !secure_done)
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
{
char SSLok;
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
#ifdef USE_SSL
Clean up broken test for whether to wait for input in SSL case. Per discussion with Magnus Hagander.
2000-10-26 00:27:25 +02:00
/* No SSL when disabled or on Unix sockets */
Allow SSL configuration to be updated at SIGHUP. It is no longer necessary to restart the server to enable, disable, or reconfigure SSL. Instead, we just create a new SSL_CTX struct (by re-reading all relevant files) whenever we get SIGHUP. Testing shows that this is fast enough that it shouldn't be a problem. In conjunction with that, downgrade the logic that complains about pg_hba.conf "hostssl" lines when SSL isn't active: now that's just a warning condition not an error. An issue that still needs to be addressed is what shall we do with passphrase-protected server keys? As this stands, the server would demand the passphrase again on every SIGHUP, which is certainly impractical. But the case was only barely supported before, so that does not seem a sufficient reason to hold up committing this patch. Andreas Karlsson, reviewed by Michael Banck and Michael Paquier Discussion: https://postgr.es/m/556A6E8A.9030400@proxel.se
2017-01-03 03:37:12 +01:00
if (!LoadedSSL || IS_AF_UNIX(port->laddr.addr.ss_family))
Clean up broken test for whether to wait for input in SSL case. Per discussion with Magnus Hagander.
2000-10-26 00:27:25 +02:00
SSLok = 'N';
SSL patch from Magnus
2000-08-30 16:54:24 +02:00
else
Clean up broken test for whether to wait for input in SSL case. Per discussion with Magnus Hagander.
2000-10-26 00:27:25 +02:00
SSLok = 'S'; /* Support for SSL */
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
#else
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
SSLok = 'N'; /* No support for SSL */
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
#endif
Ensure that we retry rather than erroring out when send() or recv() return EINTR; the stats code was failing to do this and so were a couple of places in the postmaster. The stats code assumed that recv() could not return EINTR if a preceding select() showed the socket to be read-ready, but this is demonstrably false with our Windows implementation of recv(), and it may not be the case on all Unix variants either. I think this explains the intermittent stats regression test failures we've been seeing, as well as reports of stats collector instability under high load on Windows. Backpatch as far as 8.0.
2006-07-16 20:17:14 +02:00
retry1:
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
if (send(port->sock, &SSLok, 1, 0) != 1)
{
Ensure that we retry rather than erroring out when send() or recv() return EINTR; the stats code was failing to do this and so were a couple of places in the postmaster. The stats code assumed that recv() could not return EINTR if a preceding select() showed the socket to be read-ready, but this is demonstrably false with our Windows implementation of recv(), and it may not be the case on all Unix variants either. I think this explains the intermittent stats regression test failures we've been seeing, as well as reports of stats collector instability under high load on Windows. Backpatch as far as 8.0.
2006-07-16 20:17:14 +02:00
if (errno == EINTR)
goto retry1; /* if interrupted, just retry */
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(COMMERROR,
(errcode_for_socket_access(),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
errmsg("failed to send SSL negotiation response: %m")));
Another pgindent run. Fixes enum indenting, and improves #endif spacing. Also adds space for one-line comments.
2001-10-28 07:26:15 +01:00
return STATUS_ERROR; /* close the connection */
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
}
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
#ifdef USE_SSL
UPDATED PATCH: Attached are a revised set of SSL patches. Many of these patches are motivated by security concerns, it's not just bug fixes. The key differences (from stock 7.2.1) are: *) almost all code that directly uses the OpenSSL library is in two new files, src/interfaces/libpq/fe-ssl.c src/backend/postmaster/be-ssl.c in the long run, it would be nice to merge these two files. *) the legacy code to read and write network data have been encapsulated into read_SSL() and write_SSL(). These functions should probably be renamed - they handle both SSL and non-SSL cases. the remaining code should eliminate the problems identified earlier, albeit not very cleanly. *) both front- and back-ends will send a SSL shutdown via the new close_SSL() function. This is necessary for sessions to work properly. (Sessions are not yet fully supported, but by cleanly closing the SSL connection instead of just sending a TCP FIN packet other SSL tools will be much happier.) *) The client certificate and key are now expected in a subdirectory of the user's home directory. Specifically, - the directory .postgresql must be owned by the user, and allow no access by 'group' or 'other.' - the file .postgresql/postgresql.crt must be a regular file owned by the user. - the file .postgresql/postgresql.key must be a regular file owned by the user, and allow no access by 'group' or 'other'. At the current time encrypted private keys are not supported. There should also be a way to support multiple client certs/keys. *) the front-end performs minimal validation of the back-end cert. Self-signed certs are permitted, but the common name *must* match the hostname used by the front-end. (The cert itself should always use a fully qualified domain name (FDQN) in its common name field.) This means that psql -h eris db will fail, but psql -h eris.example.com db will succeed. At the current time this must be an exact match; future patches may support any FQDN that resolves to the address returned by getpeername(2). Another common "problem" is expiring certs. For now, it may be a good idea to use a very-long-lived self-signed cert. As a compile-time option, the front-end can specify a file containing valid root certificates, but it is not yet required. *) the back-end performs minimal validation of the client cert. It allows self-signed certs. It checks for expiration. It supports a compile-time option specifying a file containing valid root certificates. *) both front- and back-ends default to TLSv1, not SSLv3/SSLv2. *) both front- and back-ends support DSA keys. DSA keys are moderately more expensive on startup, but many people consider them preferable than RSA keys. (E.g., SSH2 prefers DSA keys.) *) if /dev/urandom exists, both client and server will read 16k of randomization data from it. *) the server can read empheral DH parameters from the files $DataDir/dh512.pem $DataDir/dh1024.pem $DataDir/dh2048.pem $DataDir/dh4096.pem if none are provided, the server will default to hardcoded parameter files provided by the OpenSSL project. Remaining tasks: *) the select() clauses need to be revisited - the SSL abstraction layer may need to absorb more of the current code to avoid rare deadlock conditions. This also touches on a true solution to the pg_eof() problem. *) the SIGPIPE signal handler may need to be revisited. *) support encrypted private keys. *) sessions are not yet fully supported. (SSL sessions can span multiple "connections," and allow the client and server to avoid costly renegotiations.) *) makecert - a script that creates back-end certs. *) pgkeygen - a tool that creates front-end certs. *) the whole protocol issue, SASL, etc. *) certs are fully validated - valid root certs must be available. This is a hassle, but it means that you *can* trust the identity of the server. *) the client library can handle hardcoded root certificates, to avoid the need to copy these files. *) host name of server cert must resolve to IP address, or be a recognized alias. This is more liberal than the previous iteration. *) the number of bytes transferred is tracked, and the session key is periodically renegotiated. *) basic cert generation scripts (mkcert.sh, pgkeygen.sh). The configuration files have reasonable defaults for each type of use. Bear Giles
2002-06-14 06:23:17 +02:00
if (SSLok == 'S' && secure_open_server(port) == -1)
pgindent run.
2002-09-04 22:31:48 +02:00
return STATUS_ERROR;
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
#endif
A bit of code beautification/cleanup of obsolete comments. Rethink ordering of startup operations in one or two places.
2001-06-21 18:43:24 +02:00
/* regular startup packet, cancel, etc packet should follow... */
/* but not another SSL negotiation request */
return ProcessStartupPacket(port, true);
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
}
GSSAPI encryption support On both the frontend and backend, prepare for GSSAPI encryption support by moving common code for error handling into a separate file. Fix a TODO for handling multiple status messages in the process. Eliminate the OIDs, which have not been needed for some time. Add frontend and backend encryption support functions. Keep the context initiation for authentication-only separate on both the frontend and backend in order to avoid concerns about changing the requested flags to include encryption support. In postmaster, pull GSSAPI authorization checking into a shared function. Also share the initiator name between the encryption and non-encryption codepaths. For HBA, add "hostgssenc" and "hostnogssenc" entries that behave similarly to their SSL counterparts. "hostgssenc" requires either "gss", "trust", or "reject" for its authentication. Similarly, add a "gssencmode" parameter to libpq. Supported values are "disable", "require", and "prefer". Notably, negotiation will only be attempted if credentials can be acquired. Move credential acquisition into its own function to support this behavior. Add a simple pg_stat_gssapi view similar to pg_stat_ssl, for monitoring if GSSAPI authentication was used, what principal was used, and if encryption is being used on the connection. Finally, add documentation for everything new, and update existing documentation on connection security. Thanks to Michael Paquier for the Windows fixes. Author: Robbie Harwood, with changes to the read/write functions by me. Reviewed in various forms and at different times by: Michael Paquier, Andres Freund, David Steele. Discussion: https://www.postgresql.org/message-id/flat/jlg1tgq1ktm.fsf@thriss.redhat.com
2019-04-03 21:02:33 +02:00
else if (proto == NEGOTIATE_GSS_CODE && !secure_done)
{
char GSSok = 'N';
#ifdef ENABLE_GSS
/* No GSSAPI encryption when on Unix socket */
if (!IS_AF_UNIX(port->laddr.addr.ss_family))
GSSok = 'G';
#endif
while (send(port->sock, &GSSok, 1, 0) != 1)
{
if (errno == EINTR)
continue;
ereport(COMMERROR,
(errcode_for_socket_access(),
More message style fixes Discussion: https://postgr.es/m/20190515183005.GA26486@alvherre.pgsql
2019-05-17 00:50:56 +02:00
errmsg("failed to send GSSAPI negotiation response: %m")));
GSSAPI encryption support On both the frontend and backend, prepare for GSSAPI encryption support by moving common code for error handling into a separate file. Fix a TODO for handling multiple status messages in the process. Eliminate the OIDs, which have not been needed for some time. Add frontend and backend encryption support functions. Keep the context initiation for authentication-only separate on both the frontend and backend in order to avoid concerns about changing the requested flags to include encryption support. In postmaster, pull GSSAPI authorization checking into a shared function. Also share the initiator name between the encryption and non-encryption codepaths. For HBA, add "hostgssenc" and "hostnogssenc" entries that behave similarly to their SSL counterparts. "hostgssenc" requires either "gss", "trust", or "reject" for its authentication. Similarly, add a "gssencmode" parameter to libpq. Supported values are "disable", "require", and "prefer". Notably, negotiation will only be attempted if credentials can be acquired. Move credential acquisition into its own function to support this behavior. Add a simple pg_stat_gssapi view similar to pg_stat_ssl, for monitoring if GSSAPI authentication was used, what principal was used, and if encryption is being used on the connection. Finally, add documentation for everything new, and update existing documentation on connection security. Thanks to Michael Paquier for the Windows fixes. Author: Robbie Harwood, with changes to the read/write functions by me. Reviewed in various forms and at different times by: Michael Paquier, Andres Freund, David Steele. Discussion: https://www.postgresql.org/message-id/flat/jlg1tgq1ktm.fsf@thriss.redhat.com
2019-04-03 21:02:33 +02:00
return STATUS_ERROR; /* close the connection */
}
#ifdef ENABLE_GSS
if (GSSok == 'G' && secure_open_gssapi(port) == -1)
return STATUS_ERROR;
#endif
/* Won't ever see more than one negotiation request */
return ProcessStartupPacket(port, true);
}
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
/* Could add additional special packet types here */
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Set FrontendProtocol now so that ereport() knows what format to send if
* we fail during startup.
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
*/
FrontendProtocol = proto;
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
Provide for forward compatibility with future minor protocol versions. Previously, any attempt to request a 3.x protocol version other than 3.0 would lead to a hard connection failure, which made the minor protocol version really no different from the major protocol version and precluded gentle protocol version breaks. Instead, when the client requests a 3.x protocol version where x is greater than 0, send the new NegotiateProtocolVersion message to convey that we support only 3.0. This makes it possible to introduce new minor protocol versions without requiring a connection retry when the server is older. In addition, if the startup packet includes name/value pairs where the name starts with "_pq_.", assume that those are protocol options, not GUCs. Include those we don't support (i.e. all of them, at present) in the NegotiateProtocolVersion message so that the client knows they were not understood. This makes it possible for the client to request previously-unsupported features without bumping the protocol version at all; the client can tell from the server's response whether the option was understood. It will take some time before servers that support these new facilities become common in the wild; to speed things up and make things easier for a future 3.1 protocol version, back-patch to all supported releases. Robert Haas and Badrul Chowdhury Discussion: http://postgr.es/m/BN6PR21MB0772FFA0CBD298B76017744CD1730@BN6PR21MB0772.namprd21.prod.outlook.com Discussion: http://postgr.es/m/30788.1498672033@sss.pgh.pa.us
2017-11-21 19:56:24 +01:00
/* Check that the major protocol version is in range. */
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
if (PG_PROTOCOL_MAJOR(proto) < PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST) ||
Provide for forward compatibility with future minor protocol versions. Previously, any attempt to request a 3.x protocol version other than 3.0 would lead to a hard connection failure, which made the minor protocol version really no different from the major protocol version and precluded gentle protocol version breaks. Instead, when the client requests a 3.x protocol version where x is greater than 0, send the new NegotiateProtocolVersion message to convey that we support only 3.0. This makes it possible to introduce new minor protocol versions without requiring a connection retry when the server is older. In addition, if the startup packet includes name/value pairs where the name starts with "_pq_.", assume that those are protocol options, not GUCs. Include those we don't support (i.e. all of them, at present) in the NegotiateProtocolVersion message so that the client knows they were not understood. This makes it possible for the client to request previously-unsupported features without bumping the protocol version at all; the client can tell from the server's response whether the option was understood. It will take some time before servers that support these new facilities become common in the wild; to speed things up and make things easier for a future 3.1 protocol version, back-patch to all supported releases. Robert Haas and Badrul Chowdhury Discussion: http://postgr.es/m/BN6PR21MB0772FFA0CBD298B76017744CD1730@BN6PR21MB0772.namprd21.prod.outlook.com Discussion: http://postgr.es/m/30788.1498672033@sss.pgh.pa.us
2017-11-21 19:56:24 +01:00
PG_PROTOCOL_MAJOR(proto) > PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST))
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(FATAL,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("unsupported frontend protocol %u.%u: server supports %u.0 to %u.%u",
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
PG_PROTOCOL_MAJOR(proto), PG_PROTOCOL_MINOR(proto),
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST),
PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST),
PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST))));
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Now fetch parameters out of startup packet and save them into the Port
* structure. All data structures attached to the Port struct must be
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* allocated in TopMemoryContext so that they will remain available in a
* running backend (even after PostmasterContext is destroyed). We need
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
* not worry about leaking this storage on failure, since we aren't in the
* postmaster process anymore.
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
*/
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
oldcontext = MemoryContextSwitchTo(TopMemoryContext);
if (PG_PROTOCOL_MAJOR(proto) >= 3)
{
pgindent run.
2003-08-04 02:43:34 +02:00
int32 offset = sizeof(ProtocolVersion);
Provide for forward compatibility with future minor protocol versions. Previously, any attempt to request a 3.x protocol version other than 3.0 would lead to a hard connection failure, which made the minor protocol version really no different from the major protocol version and precluded gentle protocol version breaks. Instead, when the client requests a 3.x protocol version where x is greater than 0, send the new NegotiateProtocolVersion message to convey that we support only 3.0. This makes it possible to introduce new minor protocol versions without requiring a connection retry when the server is older. In addition, if the startup packet includes name/value pairs where the name starts with "_pq_.", assume that those are protocol options, not GUCs. Include those we don't support (i.e. all of them, at present) in the NegotiateProtocolVersion message so that the client knows they were not understood. This makes it possible for the client to request previously-unsupported features without bumping the protocol version at all; the client can tell from the server's response whether the option was understood. It will take some time before servers that support these new facilities become common in the wild; to speed things up and make things easier for a future 3.1 protocol version, back-patch to all supported releases. Robert Haas and Badrul Chowdhury Discussion: http://postgr.es/m/BN6PR21MB0772FFA0CBD298B76017744CD1730@BN6PR21MB0772.namprd21.prod.outlook.com Discussion: http://postgr.es/m/30788.1498672033@sss.pgh.pa.us
2017-11-21 19:56:24 +01:00
List *unrecognized_protocol_options = NIL;
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Scan packet body for name/option pairs. We can assume any string
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* beginning within the packet body is null-terminated, thanks to
* zeroing extra byte above.
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
*/
port->guc_options = NIL;
while (offset < len)
{
pgindent run.
2003-08-04 02:43:34 +02:00
char *nameptr = ((char *) buf) + offset;
int32 valoffset;
char *valptr;
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
if (*nameptr == '\0')
break; /* found packet terminator */
valoffset = offset + strlen(nameptr) + 1;
if (valoffset >= len)
break; /* missing value, will complain below */
valptr = ((char *) buf) + valoffset;
if (strcmp(nameptr, "database") == 0)
port->database_name = pstrdup(valptr);
else if (strcmp(nameptr, "user") == 0)
port->user_name = pstrdup(valptr);
else if (strcmp(nameptr, "options") == 0)
port->cmdline_options = pstrdup(valptr);
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
else if (strcmp(nameptr, "replication") == 0)
{
Allow logical decoding via the walsender interface. In order for this to work, walsenders need the optional ability to connect to a database, so the "replication" keyword now allows true or false, for backward-compatibility, and the new value "database" (which causes the "dbname" parameter to be respected). walsender needs to loop not only when idle but also when sending decoded data to the user and when waiting for more xlog data to decode. This means that there are now three separate loops inside walsender.c; although some refactoring has been done here, this is still a bit ugly. Andres Freund, with contributions from Álvaro Herrera, and further review by me.
2014-03-10 18:50:28 +01:00
/*
* Due to backward compatibility concerns the replication
* parameter is a hybrid beast which allows the value to be
* either boolean or the string 'database'. The latter
* connects to a specific database which is e.g. required for
* logical decoding while.
*/
if (strcmp(valptr, "database") == 0)
{
am_walsender = true;
am_db_walsender = true;
}
else if (!parse_bool(valptr, &am_walsender))
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
ereport(FATAL,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("invalid value for parameter \"%s\": \"%s\"",
"replication",
valptr),
Message improvements
2015-11-17 03:16:42 +01:00
errhint("Valid values are: \"false\", 0, \"true\", 1, \"database\".")));
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
}
Provide for forward compatibility with future minor protocol versions. Previously, any attempt to request a 3.x protocol version other than 3.0 would lead to a hard connection failure, which made the minor protocol version really no different from the major protocol version and precluded gentle protocol version breaks. Instead, when the client requests a 3.x protocol version where x is greater than 0, send the new NegotiateProtocolVersion message to convey that we support only 3.0. This makes it possible to introduce new minor protocol versions without requiring a connection retry when the server is older. In addition, if the startup packet includes name/value pairs where the name starts with "_pq_.", assume that those are protocol options, not GUCs. Include those we don't support (i.e. all of them, at present) in the NegotiateProtocolVersion message so that the client knows they were not understood. This makes it possible for the client to request previously-unsupported features without bumping the protocol version at all; the client can tell from the server's response whether the option was understood. It will take some time before servers that support these new facilities become common in the wild; to speed things up and make things easier for a future 3.1 protocol version, back-patch to all supported releases. Robert Haas and Badrul Chowdhury Discussion: http://postgr.es/m/BN6PR21MB0772FFA0CBD298B76017744CD1730@BN6PR21MB0772.namprd21.prod.outlook.com Discussion: http://postgr.es/m/30788.1498672033@sss.pgh.pa.us
2017-11-21 19:56:24 +01:00
else if (strncmp(nameptr, "_pq_.", 5) == 0)
{
/*
* Any option beginning with _pq_. is reserved for use as a
* protocol-level option, but at present no such options are
* defined.
*/
unrecognized_protocol_options =
lappend(unrecognized_protocol_options, pstrdup(nameptr));
}
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
else
{
/* Assume it's a generic GUC option */
port->guc_options = lappend(port->guc_options,
pstrdup(nameptr));
port->guc_options = lappend(port->guc_options,
pstrdup(valptr));
Add application_name to connection authorized msg The connection authorized message has quite a bit of useful information in it, but didn't include the application_name (when provided), so let's add that as it can be very useful. Note that at the point where we're emitting the connection authorized message, we haven't processed GUCs, so it's not possible to get this by using log_line_prefix (which pulls from the GUC). There's also something to be said for having this included in the connection authorized message and then not needing to repeat it for every line, as having it in log_line_prefix would do. The GUC cleans the application name to pure-ascii, so do that here too, but pull out the logic for cleaning up a string into its own function in common and re-use it from those places, and check_cluster_name which was doing the same thing. Author: Don Seiler <don@seiler.us> Discussion: https://postgr.es/m/CAHJZqBB_Pxv8HRfoh%2BAB4KxSQQuPVvtYCzMg7woNR3r7dfmopw%40mail.gmail.com
2018-09-29 01:04:50 +02:00
/*
* Copy application_name to port if we come across it. This
* is done so we can log the application_name in the
* connection authorization message. Note that the GUC would
* be used but we haven't gone through GUC setup yet.
*/
if (strcmp(nameptr, "application_name") == 0)
{
char *tmp_app_name = pstrdup(valptr);
pg_clean_ascii(tmp_app_name);
port->application_name = tmp_app_name;
}
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
}
offset = valoffset + strlen(valptr) + 1;
}
pgindent run.
2003-08-04 02:43:34 +02:00
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
/*
* If we didn't find a packet terminator exactly at the end of the
* given packet length, complain.
*/
pgindent run.
2003-08-04 02:43:34 +02:00
if (offset != len - 1)
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(FATAL,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("invalid startup packet layout: expected terminator as last byte")));
Provide for forward compatibility with future minor protocol versions. Previously, any attempt to request a 3.x protocol version other than 3.0 would lead to a hard connection failure, which made the minor protocol version really no different from the major protocol version and precluded gentle protocol version breaks. Instead, when the client requests a 3.x protocol version where x is greater than 0, send the new NegotiateProtocolVersion message to convey that we support only 3.0. This makes it possible to introduce new minor protocol versions without requiring a connection retry when the server is older. In addition, if the startup packet includes name/value pairs where the name starts with "_pq_.", assume that those are protocol options, not GUCs. Include those we don't support (i.e. all of them, at present) in the NegotiateProtocolVersion message so that the client knows they were not understood. This makes it possible for the client to request previously-unsupported features without bumping the protocol version at all; the client can tell from the server's response whether the option was understood. It will take some time before servers that support these new facilities become common in the wild; to speed things up and make things easier for a future 3.1 protocol version, back-patch to all supported releases. Robert Haas and Badrul Chowdhury Discussion: http://postgr.es/m/BN6PR21MB0772FFA0CBD298B76017744CD1730@BN6PR21MB0772.namprd21.prod.outlook.com Discussion: http://postgr.es/m/30788.1498672033@sss.pgh.pa.us
2017-11-21 19:56:24 +01:00
/*
* If the client requested a newer protocol version or if the client
* requested any protocol options we didn't recognize, let them know
Update typedefs.list and re-run pgindent Discussion: http://postgr.es/m/CA+TgmoaA9=1RWKtBWpDaj+sF3Stgc8sHgf5z=KGtbjwPLQVDMA@mail.gmail.com
2017-11-29 15:24:24 +01:00
* the newest minor protocol version we do support and the names of
* any unrecognized options.
Provide for forward compatibility with future minor protocol versions. Previously, any attempt to request a 3.x protocol version other than 3.0 would lead to a hard connection failure, which made the minor protocol version really no different from the major protocol version and precluded gentle protocol version breaks. Instead, when the client requests a 3.x protocol version where x is greater than 0, send the new NegotiateProtocolVersion message to convey that we support only 3.0. This makes it possible to introduce new minor protocol versions without requiring a connection retry when the server is older. In addition, if the startup packet includes name/value pairs where the name starts with "_pq_.", assume that those are protocol options, not GUCs. Include those we don't support (i.e. all of them, at present) in the NegotiateProtocolVersion message so that the client knows they were not understood. This makes it possible for the client to request previously-unsupported features without bumping the protocol version at all; the client can tell from the server's response whether the option was understood. It will take some time before servers that support these new facilities become common in the wild; to speed things up and make things easier for a future 3.1 protocol version, back-patch to all supported releases. Robert Haas and Badrul Chowdhury Discussion: http://postgr.es/m/BN6PR21MB0772FFA0CBD298B76017744CD1730@BN6PR21MB0772.namprd21.prod.outlook.com Discussion: http://postgr.es/m/30788.1498672033@sss.pgh.pa.us
2017-11-21 19:56:24 +01:00
*/
if (PG_PROTOCOL_MINOR(proto) > PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST) ||
unrecognized_protocol_options != NIL)
SendNegotiateProtocolVersion(unrecognized_protocol_options);
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
}
else
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Get the parameters from the old-style, fixed-width-fields startup
* packet as C strings. The packet destination was cleared first so a
* short packet has zeros silently added. We have to be prepared to
* truncate the pstrdup result for oversize fields, though.
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
*/
StartupPacket *packet = (StartupPacket *) buf;
port->database_name = pstrdup(packet->database);
if (strlen(port->database_name) > sizeof(packet->database))
port->database_name[sizeof(packet->database)] = '\0';
port->user_name = pstrdup(packet->user);
if (strlen(port->user_name) > sizeof(packet->user))
port->user_name[sizeof(packet->user)] = '\0';
port->cmdline_options = pstrdup(packet->options);
if (strlen(port->cmdline_options) > sizeof(packet->options))
port->cmdline_options[sizeof(packet->options)] = '\0';
port->guc_options = NIL;
}
Truncate incoming username and database name to NAMEDATALEN-1 characters so that we don't reject overlength names unnecessarily.
2001-02-20 02:34:40 +01:00
From: Phil Thompson <phil@river-bank.demon.co.uk> I've completed the patch to fix the protocol and authentication issues I was discussing a couple of weeks ago. The particular changes are: - the protocol has a version number - network byte order is used throughout - the pg_hba.conf file is used to specify what method is used to authenticate a frontend (either password, ident, trust, reject, krb4 or krb5) - support for multiplexed backends is removed - appropriate changes to man pages - the -a switch to many programs to specify an authentication service no longer has any effect - the libpq.so version number has changed to 1.1 The new backend still supports the old protocol so old interfaces won't break.
1998-01-26 02:42:53 +01:00
/* Check a user name was given. */
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
if (port->user_name == NULL || port->user_name[0] == '\0')
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(FATAL,
(errcode(ERRCODE_INVALID_AUTHORIZATION_SPECIFICATION),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("no PostgreSQL user name specified in startup packet")));
From: Phil Thompson <phil@river-bank.demon.co.uk> I've completed the patch to fix the protocol and authentication issues I was discussing a couple of weeks ago. The particular changes are: - the protocol has a version number - network byte order is used throughout - the pg_hba.conf file is used to specify what method is used to authenticate a frontend (either password, ident, trust, reject, krb4 or krb5) - support for multiplexed backends is removed - appropriate changes to man pages - the -a switch to many programs to specify an authentication service no longer has any effect - the libpq.so version number has changed to 1.1 The new backend still supports the old protocol so old interfaces won't break.
1998-01-26 02:42:53 +01:00
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
/* The database defaults to the user name. */
if (port->database_name == NULL || port->database_name[0] == '\0')
port->database_name = pstrdup(port->user_name);
Add db-local user names, per discussion on hackers.
2002-08-18 05:03:26 +02:00
if (Db_user_namespace)
pgindent run.
2002-09-04 22:31:48 +02:00
{
Add db-local user names, per discussion on hackers.
2002-08-18 05:03:26 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If user@, it is a global user, remove '@'. We only want to do this
* if there is an '@' at the end and no earlier in the user string or
* they may fake as a local user of another database attaching to this
* database.
Add db-local user names, per discussion on hackers.
2002-08-18 05:03:26 +02:00
*/
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
if (strchr(port->user_name, '@') ==
port->user_name + strlen(port->user_name) - 1)
*strchr(port->user_name, '@') = '\0';
Add db-local user names, per discussion on hackers.
2002-08-18 05:03:26 +02:00
else
{
/* Append '@' and dbname */
Add use of asprintf() Add asprintf(), pg_asprintf(), and psprintf() to simplify string allocation and composition. Replacement implementations taken from NetBSD. Reviewed-by: Álvaro Herrera <alvherre@2ndquadrant.com> Reviewed-by: Asif Naeem <anaeem.it@gmail.com>
2013-10-13 06:09:18 +02:00
port->user_name = psprintf("%s@%s", port->user_name, port->database_name);
Add db-local user names, per discussion on hackers.
2002-08-18 05:03:26 +02:00
}
}
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Truncate given database and user names to length of a Postgres name.
* This avoids lookup failures when overlength names are given.
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
*/
if (strlen(port->database_name) >= NAMEDATALEN)
port->database_name[NAMEDATALEN - 1] = '\0';
if (strlen(port->user_name) >= NAMEDATALEN)
port->user_name[NAMEDATALEN - 1] = '\0';
Allow logical decoding via the walsender interface. In order for this to work, walsenders need the optional ability to connect to a database, so the "replication" keyword now allows true or false, for backward-compatibility, and the new value "database" (which causes the "dbname" parameter to be respected). walsender needs to loop not only when idle but also when sending decoded data to the user and when waiting for more xlog data to decode. This means that there are now three separate loops inside walsender.c; although some refactoring has been done here, this is still a bit ugly. Andres Freund, with contributions from Álvaro Herrera, and further review by me.
2014-03-10 18:50:28 +01:00
/*
* Normal walsender backends, e.g. for streaming replication, are not
* connected to a particular database. But walsenders used for logical
* replication need to connect to a specific database. We allow streaming
* replication commands to be issued even if connected to a database as it
* can make sense to first make a basebackup and then stream changes
* starting from that.
*/
if (am_walsender && !am_db_walsender)
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
port->database_name[0] = '\0';
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
/*
* Done putting stuff in TopMemoryContext.
*/
MemoryContextSwitchTo(oldcontext);
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If we're going to reject the connection due to database state, say so
* now instead of wasting cycles on an authentication exchange. (This also
* allows a pg_ping utility to be written.)
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
*/
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
switch (port->canAcceptConnections)
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
{
case CAC_STARTUP:
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is starting up")));
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
break;
case CAC_SHUTDOWN:
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is shutting down")));
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
break;
case CAC_RECOVERY:
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(FATAL,
(errcode(ERRCODE_CANNOT_CONNECT_NOW),
errmsg("the database system is in recovery mode")));
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
break;
case CAC_TOOMANY:
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(FATAL,
(errcode(ERRCODE_TOO_MANY_CONNECTIONS),
errmsg("sorry, too many clients already")));
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
break;
Code review for recent patch to terminate online backup during shutdown: do CancelBackup at a sane place, fix some oversights in the state transitions, allow only superusers to connect while we are waiting for backup mode to end.
2008-04-27 00:47:40 +02:00
case CAC_WAITBACKUP:
/* OK for now, will check in InitPostgres */
break;
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
case CAC_OK:
Rearrange LOG_CONNECTIONS code so that two log messages are made: one immediately upon forking to handle a new connection, and one after the authentication cycle is finished. Per today's pggeneral discussion.
2002-05-29 01:56:51 +02:00
break;
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
}
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
return STATUS_OK;
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
}
Provide for forward compatibility with future minor protocol versions. Previously, any attempt to request a 3.x protocol version other than 3.0 would lead to a hard connection failure, which made the minor protocol version really no different from the major protocol version and precluded gentle protocol version breaks. Instead, when the client requests a 3.x protocol version where x is greater than 0, send the new NegotiateProtocolVersion message to convey that we support only 3.0. This makes it possible to introduce new minor protocol versions without requiring a connection retry when the server is older. In addition, if the startup packet includes name/value pairs where the name starts with "_pq_.", assume that those are protocol options, not GUCs. Include those we don't support (i.e. all of them, at present) in the NegotiateProtocolVersion message so that the client knows they were not understood. This makes it possible for the client to request previously-unsupported features without bumping the protocol version at all; the client can tell from the server's response whether the option was understood. It will take some time before servers that support these new facilities become common in the wild; to speed things up and make things easier for a future 3.1 protocol version, back-patch to all supported releases. Robert Haas and Badrul Chowdhury Discussion: http://postgr.es/m/BN6PR21MB0772FFA0CBD298B76017744CD1730@BN6PR21MB0772.namprd21.prod.outlook.com Discussion: http://postgr.es/m/30788.1498672033@sss.pgh.pa.us
2017-11-21 19:56:24 +01:00
/*
* Send a NegotiateProtocolVersion to the client. This lets the client know
* that they have requested a newer minor protocol version than we are able
* to speak. We'll speak the highest version we know about; the client can,
* of course, abandon the connection if that's a problem.
*
* We also include in the response a list of protocol options we didn't
* understand. This allows clients to include optional parameters that might
* be present either in newer protocol versions or third-party protocol
* extensions without fear of having to reconnect if those options are not
* understood, while at the same time making certain that the client is aware
* of which options were actually accepted.
*/
static void
SendNegotiateProtocolVersion(List *unrecognized_protocol_options)
{
StringInfoData buf;
ListCell *lc;
pq_beginmessage(&buf, 'v'); /* NegotiateProtocolVersion */
pq_sendint32(&buf, PG_PROTOCOL_LATEST);
pq_sendint32(&buf, list_length(unrecognized_protocol_options));
foreach(lc, unrecognized_protocol_options)
pq_sendstring(&buf, lfirst(lc));
pq_endmessage(&buf);
/* no need to flush, some other message will follow */
}
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
/*
* The client has sent a cancel request packet, not a normal
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
* start-a-new-connection packet. Perform the necessary processing.
* Nothing is sent back to the client.
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
*/
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
static void
processCancelRequest(Port *port, void *pkt)
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
{
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
CancelRequestPacket *canc = (CancelRequestPacket *) pkt;
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
int backendPID;
Fix query cancellation. In commit fe0a0b59, the datatype used for MyCancelKey and other variables that store cancel keys were changed from long to uint32, but I missed this one. That broke query cancellation on platforms where long is wider than 32 bits. Report by Andres Freund, fix by Michael Paquier.
2016-12-07 08:47:43 +01:00
int32 cancelAuthCode;
Back out win32 patch so we can apply it separately.
2004-01-26 23:54:58 +01:00
Backend *bp;
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
#ifndef EXEC_BACKEND
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
dlist_iter iter;
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
#else
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
int i;
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
#endif
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
Replace most usages of ntoh[ls] and hton[sl] with pg_bswap.h. All postgres internal usages are replaced, it's just libpq example usages that haven't been converted. External users of libpq can't generally rely on including postgres internal headers. Note that this includes replacing open-coded byte swapping of 64bit integers (using two 32 bit swaps) with a single 64bit swap. Where it looked applicable, I have removed netinet/in.h and arpa/inet.h usage, which previously provided the relevant functionality. It's perfectly possible that I missed other reasons for including those, the buildfarm will tell. Author: Andres Freund Discussion: https://postgr.es/m/20170927172019.gheidqy6xvlxb325@alap3.anarazel.de
2017-10-02 00:36:14 +02:00
backendPID = (int) pg_ntoh32(canc->backendPID);
cancelAuthCode = (int32) pg_ntoh32(canc->cancelAuthCode);
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* See if we have a matching backend. In the EXEC_BACKEND case, we can no
* longer access the postmaster's own backend list, and must rely on the
* duplicate array in shared memory.
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
*/
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
#ifndef EXEC_BACKEND
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
dlist_foreach(iter, &BackendList)
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
{
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
bp = dlist_container(Backend, elem, iter.cur);
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
#else
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
for (i = MaxLivePostmasterChildren() - 1; i >= 0; i--)
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
{
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
bp = (Backend *) &ShmemBackendArray[i];
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
#endif
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
if (bp->pid == backendPID)
{
if (bp->cancel_key == cancelAuthCode)
{
/* Found a match; signal that backend to cancel current op */
Change some frequently-reached elog(DEBUG...) calls to ereport(DEBUG...) for speed reasons. (ereport falls out much more quickly when no output is needed than elog does.)
2003-08-12 20:23:21 +02:00
ereport(DEBUG2,
(errmsg_internal("processing cancel request: sending SIGINT to process %d",
backendPID)));
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
signal_child(bp->pid, SIGINT);
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
}
else
/* Right PID, wrong key: no way, Jose */
Use LOG instead of DEBUG2 for logging invalid cancel requests. (That is, cancel requests with an incorrect key, or with a non-existent PID). Per recent discussion on -hackers.
2007-11-05 01:00:34 +01:00
ereport(LOG,
Clarify log messages
2007-12-13 12:55:44 +01:00
(errmsg("wrong key in cancel request for process %d",
Use LOG instead of DEBUG2 for logging invalid cancel requests. (That is, cancel requests with an incorrect key, or with a non-existent PID). Per recent discussion on -hackers.
2007-11-05 01:00:34 +01:00
backendPID)));
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
return;
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
}
Use preprocessor conditions compatible with Emacs indent. Emacs wrongly indented hundreds of subsequent lines.
2019-04-28 21:56:53 +02:00
#ifndef EXEC_BACKEND /* make GNU Emacs 26.1 see brace balance */
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
}
Use preprocessor conditions compatible with Emacs indent. Emacs wrongly indented hundreds of subsequent lines.
2019-04-28 21:56:53 +02:00
#else
}
#endif
From: Tom Lane <tgl@sss.pgh.pa.us> Making PQrequestCancel safe to call in a signal handler turned out to be much easier than I feared. So here are the diffs. Some notes: * I modified the postmaster's packet "iodone" callback interface to allow the callback routine to return a continue-or-drop-connection return code; this was necessary to allow the connection to be closed after receiving a Cancel, rather than proceeding to launch a new backend... Being a neatnik, I also made the iodone proc have a typechecked parameter list. * I deleted all code I could find that had to do with OOB. * I made some edits to ensure that all signals mentioned in the code are referred to symbolically not by numbers ("SIGUSR2" not "2"). I think Bruce may have already done at least some of the same edits; I hope that merging these patches is not too painful.
1998-07-09 05:29:11 +02:00
/* No matching backend */
Use LOG instead of DEBUG2 for logging invalid cancel requests. (That is, cancel requests with an incorrect key, or with a non-existent PID). Per recent discussion on -hackers.
2007-11-05 01:00:34 +01:00
ereport(LOG,
Clarify log messages
2007-12-13 12:55:44 +01:00
(errmsg("PID %d in cancel request did not match any process",
Use LOG instead of DEBUG2 for logging invalid cancel requests. (That is, cancel requests with an incorrect key, or with a non-existent PID). Per recent discussion on -hackers.
2007-11-05 01:00:34 +01:00
backendPID)));
New host-based authentication -- send error message when authentication fails
1996-10-12 09:48:49 +02:00
}
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
/*
Check for too many postmaster children before spawning a bgworker. The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-07 18:39:09 +02:00
* canAcceptConnections --- check to see if database state allows connections
* of the specified type. backend_type can be BACKEND_TYPE_NORMAL,
* BACKEND_TYPE_AUTOVAC, or BACKEND_TYPE_BGWORKER. (Note that we don't yet
* know whether a NORMAL connection might turn into a walsender.)
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
*/
Fix canAcceptConnections() bugs introduced by replication-related patches. We must not return any "okay to proceed" result code without having checked for too many children, else we might fail later on when trying to add the new child to one of the per-child state arrays. It's not clear whether this oversight explains Stefan Kaltenbrunner's recent report, but it could certainly produce a similar symptom. Back-patch to 8.4; the logic was not broken before that.
2010-11-14 21:57:37 +01:00
static CAC_state
Check for too many postmaster children before spawning a bgworker. The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-07 18:39:09 +02:00
canAcceptConnections(int backend_type)
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
{
Fix canAcceptConnections() bugs introduced by replication-related patches. We must not return any "okay to proceed" result code without having checked for too many children, else we might fail later on when trying to add the new child to one of the per-child state arrays. It's not clear whether this oversight explains Stefan Kaltenbrunner's recent report, but it could certainly produce a similar symptom. Back-patch to 8.4; the logic was not broken before that.
2010-11-14 21:57:37 +01:00
CAC_state result = CAC_OK;
Prevent shutdown in normal mode if online backup is running, and have pg_ctl warn about this. Cancel running online backups (by renaming the backup_label file, thus rendering the backup useless) when shutting down in fast mode. Laurenz Albe
2008-04-23 15:44:59 +02:00
/*
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* Can't start backends when in startup/shutdown/inconsistent recovery
Check for too many postmaster children before spawning a bgworker. The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-07 18:39:09 +02:00
* state. We treat autovac workers the same as user backends for this
* purpose. However, bgworkers are excluded from this test; we expect
* bgworker_should_start_now() decided whether the DB state allows them.
Code review for recent patch to terminate online backup during shutdown: do CancelBackup at a sane place, fix some oversights in the state transitions, allow only superusers to connect while we are waiting for backup mode to end.
2008-04-27 00:47:40 +02:00
*
* In state PM_WAIT_BACKUP only superusers can connect (this must be
* allowed so that a superuser can end online backup mode); we return
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* CAC_WAITBACKUP code to indicate that this must be checked later. Note
* that neither CAC_OK nor CAC_WAITBACKUP can safely be returned until we
* have checked for too many children.
Prevent shutdown in normal mode if online backup is running, and have pg_ctl warn about this. Cancel running online backups (by renaming the backup_label file, thus rendering the backup useless) when shutting down in fast mode. Laurenz Albe
2008-04-23 15:44:59 +02:00
*/
Check for too many postmaster children before spawning a bgworker. The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-07 18:39:09 +02:00
if (pmState != PM_RUN &&
backend_type != BACKEND_TYPE_BGWORKER)
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
{
Code review for recent patch to terminate online backup during shutdown: do CancelBackup at a sane place, fix some oversights in the state transitions, allow only superusers to connect while we are waiting for backup mode to end.
2008-04-27 00:47:40 +02:00
if (pmState == PM_WAIT_BACKUP)
Fix canAcceptConnections() bugs introduced by replication-related patches. We must not return any "okay to proceed" result code without having checked for too many children, else we might fail later on when trying to add the new child to one of the per-child state arrays. It's not clear whether this oversight explains Stefan Kaltenbrunner's recent report, but it could certainly produce a similar symptom. Back-patch to 8.4; the logic was not broken before that.
2010-11-14 21:57:37 +01:00
result = CAC_WAITBACKUP; /* allow superusers only */
else if (Shutdown > NoShutdown)
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
return CAC_SHUTDOWN; /* shutdown is pending */
Fix canAcceptConnections() bugs introduced by replication-related patches. We must not return any "okay to proceed" result code without having checked for too many children, else we might fail later on when trying to add the new child to one of the per-child state arrays. It's not clear whether this oversight explains Stefan Kaltenbrunner's recent report, but it could certainly produce a similar symptom. Back-patch to 8.4; the logic was not broken before that.
2010-11-14 21:57:37 +01:00
else if (!FatalError &&
(pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
return CAC_STARTUP; /* normal startup */
Fix canAcceptConnections() bugs introduced by replication-related patches. We must not return any "okay to proceed" result code without having checked for too many children, else we might fail later on when trying to add the new child to one of the per-child state arrays. It's not clear whether this oversight explains Stefan Kaltenbrunner's recent report, but it could certainly produce a similar symptom. Back-patch to 8.4; the logic was not broken before that.
2010-11-14 21:57:37 +01:00
else if (!FatalError &&
pmState == PM_HOT_STANDBY)
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
result = CAC_OK; /* connection OK during hot standby */
Fix canAcceptConnections() bugs introduced by replication-related patches. We must not return any "okay to proceed" result code without having checked for too many children, else we might fail later on when trying to add the new child to one of the per-child state arrays. It's not clear whether this oversight explains Stefan Kaltenbrunner's recent report, but it could certainly produce a similar symptom. Back-patch to 8.4; the logic was not broken before that.
2010-11-14 21:57:37 +01:00
else
return CAC_RECOVERY; /* else must be crash recovery */
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
}
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Tweak startup sequence so that running out of PROC array slots is detected sooner in backend startup, and is treated as an expected error (it gives 'Sorry, too many clients already' now). This allows us not to have to enforce the MaxBackends limit exactly in the postmaster. Also, remove ProcRemove() and fold its functionality into ProcKill(). There's no good reason for a backend not to be responsible for removing its PROC entry, and there are lots of good reasons for the postmaster not to be touching shared-memory data structures.
2001-06-17 00:58:17 +02:00
/*
* Don't start too many children.
*
Check for too many postmaster children before spawning a bgworker. The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-07 18:39:09 +02:00
* We allow more connections here than we can have backends because some
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* might still be authenticating; they might fail auth, or some existing
Check for too many postmaster children before spawning a bgworker. The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-07 18:39:09 +02:00
* backend might exit before the auth cycle is completed. The exact
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* MaxBackends limit is enforced when a new backend tries to join the
* shared-inval backend array.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
* The limit here must match the sizes of the per-child-process arrays;
* see comments for MaxLivePostmasterChildren().
Tweak startup sequence so that running out of PROC array slots is detected sooner in backend startup, and is treated as an expected error (it gives 'Sorry, too many clients already' now). This allows us not to have to enforce the MaxBackends limit exactly in the postmaster. Also, remove ProcRemove() and fold its functionality into ProcKill(). There's no good reason for a backend not to be responsible for removing its PROC entry, and there are lots of good reasons for the postmaster not to be touching shared-memory data structures.
2001-06-17 00:58:17 +02:00
*/
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
if (CountChildren(BACKEND_TYPE_ALL) >= MaxLivePostmasterChildren())
Fix canAcceptConnections() bugs introduced by replication-related patches. We must not return any "okay to proceed" result code without having checked for too many children, else we might fail later on when trying to add the new child to one of the per-child state arrays. It's not clear whether this oversight explains Stefan Kaltenbrunner's recent report, but it could certainly produce a similar symptom. Back-patch to 8.4; the logic was not broken before that.
2010-11-14 21:57:37 +01:00
result = CAC_TOOMANY;
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
Fix canAcceptConnections() bugs introduced by replication-related patches. We must not return any "okay to proceed" result code without having checked for too many children, else we might fail later on when trying to add the new child to one of the per-child state arrays. It's not clear whether this oversight explains Stefan Kaltenbrunner's recent report, but it could certainly produce a similar symptom. Back-patch to 8.4; the logic was not broken before that.
2010-11-14 21:57:37 +01:00
return result;
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
}
New host-based authentication -- send error message when authentication fails
1996-10-12 09:48:49 +02:00
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
* ConnCreate -- create a local connection data structure
Previous patch had no detectable virtue other than being a one-liner. Try to make the code look self-consistent again, so it doesn't confuse future developers.
2010-10-27 21:26:24 +02:00
*
* Returns NULL on failure, other than out-of-memory which is fatal.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
From: Phil Thompson <phil@river-bank.demon.co.uk> I've completed the patch to fix the protocol and authentication issues I was discussing a couple of weeks ago. The particular changes are: - the protocol has a version number - network byte order is used throughout - the pg_hba.conf file is used to specify what method is used to authenticate a frontend (either password, ident, trust, reject, krb4 or krb5) - support for multiplexed backends is removed - appropriate changes to man pages - the -a switch to many programs to specify an authentication service no longer has any effect - the libpq.so version number has changed to 1.1 The new backend still supports the old protocol so old interfaces won't break.
1998-01-26 02:42:53 +01:00
static Port *
ConnCreate(int serverFd)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
Port *port;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (!(port = (Port *) calloc(1, sizeof(Port))))
{
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(LOG,
(errcode(ERRCODE_OUT_OF_MEMORY),
errmsg("out of memory")));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
ExitPostmaster(1);
}
From: Phil Thompson <phil@river-bank.demon.co.uk> I've completed the patch to fix the protocol and authentication issues I was discussing a couple of weeks ago. The particular changes are: - the protocol has a version number - network byte order is used throughout - the pg_hba.conf file is used to specify what method is used to authenticate a frontend (either password, ident, trust, reject, krb4 or krb5) - support for multiplexed backends is removed - appropriate changes to man pages - the -a switch to many programs to specify an authentication service no longer has any effect - the libpq.so version number has changed to 1.1 The new backend still supports the old protocol so old interfaces won't break.
1998-01-26 02:42:53 +01:00
if (StreamConnection(serverFd, port) != STATUS_OK)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
check socket creation errors against PGINVALID_SOCKET Previously, in some places, socket creation errors were checked for negative values, which is not true for Windows because sockets are unsigned. This masked socket creation errors on Windows. Backpatch through 9.0. 8.4 doesn't have the infrastructure to fix this.
2014-04-16 16:45:48 +02:00
if (port->sock != PGINVALID_SOCKET)
Improve postmaster's behavior if an accept() call fails. Because the server socket is still read-ready, the code was a tight loop, wasting lots of CPU. We can't do anything to clear the failure, other than wait, but we should give other processes more chance to finish and release FDs; so insert a small sleep. Also, avoid bogus "close(-1)" in this case. Per report from Jim Nasby.
2007-02-13 20:18:54 +01:00
StreamClose(port->sock);
More paranoia about global variables containing references to long- since-closed file descriptors...
2000-05-26 03:38:08 +02:00
ConnFree(port);
Fix long-standing segfault when accept() or one of the calls made right after accepting a connection fails, and the server is compiled with GSSAPI support. Report and patch by Alexander V. Chernikov, bug #5731.
2010-10-27 19:03:00 +02:00
return NULL;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Previous patch had no detectable virtue other than being a one-liner. Try to make the code look self-consistent again, so it doesn't confuse future developers.
2010-10-27 21:26:24 +02:00
Add support for GSSAPI authentication. Documentation still being written, will be committed later. Henry B. Hotz and Magnus Hagander
2007-07-10 15:14:22 +02:00
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* Allocate GSSAPI specific state struct
Add support for GSSAPI authentication. Documentation still being written, will be committed later. Henry B. Hotz and Magnus Hagander
2007-07-10 15:14:22 +02:00
*/
SSPI authentication on Windows. GSSAPI compatible client when doing Kerberos against a Unix server, and Windows-specific server-side authentication using SSPI "negotiate" method (Kerberos or NTLM). Only builds properly with MSVC for now.
2007-07-23 12:16:54 +02:00
#ifndef EXEC_BACKEND
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
#if defined(ENABLE_GSS) || defined(ENABLE_SSPI)
port->gss = (pg_gssinfo *) calloc(1, sizeof(pg_gssinfo));
A bunch of GSSAPI fixes per comments from Tom: * use elog not ereport for debug * fix debug levels for some output * properly check for memory allocation errors in a couple of missed places
2007-07-11 10:27:33 +02:00
if (!port->gss)
{
ereport(LOG,
(errcode(ERRCODE_OUT_OF_MEMORY),
errmsg("out of memory")));
ExitPostmaster(1);
}
SSPI authentication on Windows. GSSAPI compatible client when doing Kerberos against a Unix server, and Windows-specific server-side authentication using SSPI "negotiate" method (Kerberos or NTLM). Only builds properly with MSVC for now.
2007-07-23 12:16:54 +02:00
#endif
Add support for GSSAPI authentication. Documentation still being written, will be committed later. Henry B. Hotz and Magnus Hagander
2007-07-10 15:14:22 +02:00
#endif
From: Phil Thompson <phil@river-bank.demon.co.uk> I've completed the patch to fix the protocol and authentication issues I was discussing a couple of weeks ago. The particular changes are: - the protocol has a version number - network byte order is used throughout - the pg_hba.conf file is used to specify what method is used to authenticate a frontend (either password, ident, trust, reject, krb4 or krb5) - support for multiplexed backends is removed - appropriate changes to man pages - the -a switch to many programs to specify an authentication service no longer has any effect - the libpq.so version number has changed to 1.1 The new backend still supports the old protocol so old interfaces won't break.
1998-01-26 02:42:53 +01:00
return port;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
/*
More paranoia about global variables containing references to long- since-closed file descriptors...
2000-05-26 03:38:08 +02:00
* ConnFree -- free a local connection data structure
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
*/
Fix a bunch of minor portability problems and maybe-bugs revealed by running gcc and HP's cc with warnings cranked way up. Signed vs unsigned comparisons, routines declared static and then defined not-static, that kind of thing. Tedious, but perhaps useful...
2000-03-17 03:36:41 +01:00
static void
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
ConnFree(Port *conn)
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
{
#ifdef USE_SSL
UPDATED PATCH: Attached are a revised set of SSL patches. Many of these patches are motivated by security concerns, it's not just bug fixes. The key differences (from stock 7.2.1) are: *) almost all code that directly uses the OpenSSL library is in two new files, src/interfaces/libpq/fe-ssl.c src/backend/postmaster/be-ssl.c in the long run, it would be nice to merge these two files. *) the legacy code to read and write network data have been encapsulated into read_SSL() and write_SSL(). These functions should probably be renamed - they handle both SSL and non-SSL cases. the remaining code should eliminate the problems identified earlier, albeit not very cleanly. *) both front- and back-ends will send a SSL shutdown via the new close_SSL() function. This is necessary for sessions to work properly. (Sessions are not yet fully supported, but by cleanly closing the SSL connection instead of just sending a TCP FIN packet other SSL tools will be much happier.) *) The client certificate and key are now expected in a subdirectory of the user's home directory. Specifically, - the directory .postgresql must be owned by the user, and allow no access by 'group' or 'other.' - the file .postgresql/postgresql.crt must be a regular file owned by the user. - the file .postgresql/postgresql.key must be a regular file owned by the user, and allow no access by 'group' or 'other'. At the current time encrypted private keys are not supported. There should also be a way to support multiple client certs/keys. *) the front-end performs minimal validation of the back-end cert. Self-signed certs are permitted, but the common name *must* match the hostname used by the front-end. (The cert itself should always use a fully qualified domain name (FDQN) in its common name field.) This means that psql -h eris db will fail, but psql -h eris.example.com db will succeed. At the current time this must be an exact match; future patches may support any FQDN that resolves to the address returned by getpeername(2). Another common "problem" is expiring certs. For now, it may be a good idea to use a very-long-lived self-signed cert. As a compile-time option, the front-end can specify a file containing valid root certificates, but it is not yet required. *) the back-end performs minimal validation of the client cert. It allows self-signed certs. It checks for expiration. It supports a compile-time option specifying a file containing valid root certificates. *) both front- and back-ends default to TLSv1, not SSLv3/SSLv2. *) both front- and back-ends support DSA keys. DSA keys are moderately more expensive on startup, but many people consider them preferable than RSA keys. (E.g., SSH2 prefers DSA keys.) *) if /dev/urandom exists, both client and server will read 16k of randomization data from it. *) the server can read empheral DH parameters from the files $DataDir/dh512.pem $DataDir/dh1024.pem $DataDir/dh2048.pem $DataDir/dh4096.pem if none are provided, the server will default to hardcoded parameter files provided by the OpenSSL project. Remaining tasks: *) the select() clauses need to be revisited - the SSL abstraction layer may need to absorb more of the current code to avoid rare deadlock conditions. This also touches on a true solution to the pg_eof() problem. *) the SIGPIPE signal handler may need to be revisited. *) support encrypted private keys. *) sessions are not yet fully supported. (SSL sessions can span multiple "connections," and allow the client and server to avoid costly renegotiations.) *) makecert - a script that creates back-end certs. *) pgkeygen - a tool that creates front-end certs. *) the whole protocol issue, SASL, etc. *) certs are fully validated - valid root certs must be available. This is a hassle, but it means that you *can* trust the identity of the server. *) the client library can handle hardcoded root certificates, to avoid the need to copy these files. *) host name of server cert must resolve to IP address, or be a recognized alias. This is more liberal than the previous iteration. *) the number of bytes transferred is tracked, and the session key is periodically renegotiated. *) basic cert generation scripts (mkcert.sh, pgkeygen.sh). The configuration files have reasonable defaults for each type of use. Bear Giles
2002-06-14 06:23:17 +02:00
secure_close(conn);
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
#endif
Add support for GSSAPI authentication. Documentation still being written, will be committed later. Henry B. Hotz and Magnus Hagander
2007-07-10 15:14:22 +02:00
if (conn->gss)
free(conn->gss);
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
free(conn);
}
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
When launching a child backend, take care to close file descriptors for any other client connections that may exist (which would only happen if another client is currently in the authentication cycle). This avoids wastage of open descriptors in a child. It might also explain peculiar behaviors like not closing connections when expected, since the kernel will probably not signal EOF as long as some other backend is randomly holding open a reference to the connection, even if the client went away long since ...
2001-02-08 01:35:10 +01:00
/*
* ClosePostmasterPorts -- close all the postmaster's open sockets
*
* This is called during child process startup to release file descriptors
A bit of code beautification/cleanup of obsolete comments. Rethink ordering of startup operations in one or two places.
2001-06-21 18:43:24 +02:00
* that are not needed by that child process. The postmaster still has
* them open, of course.
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
*
* Note: we pass am_syslogger as a boolean because we don't want to set
* the global variable yet when this is called.
When launching a child backend, take care to close file descriptors for any other client connections that may exist (which would only happen if another client is currently in the authentication cycle). This avoids wastage of open descriptors in a child. It might also explain peculiar behaviors like not closing connections when expected, since the kernel will probably not signal EOF as long as some other backend is randomly holding open a reference to the connection, even if the client went away long since ...
2001-02-08 01:35:10 +01:00
*/
Endeavor to make pgstats buffer process (a) safe and (b) useful. Make sure it exits immediately when collector process dies --- in old code, buffer process would hang around and compete with the new buffer process for packets. Make sure it doesn't block on writing the pipe when the collector falls more than a pipeload behind. Avoid leaking pgstats FDs into every backend.
2001-08-05 04:06:50 +02:00
void
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
ClosePostmasterPorts(bool am_syslogger)
When launching a child backend, take care to close file descriptors for any other client connections that may exist (which would only happen if another client is currently in the authentication cycle). This avoids wastage of open descriptors in a child. It might also explain peculiar behaviors like not closing connections when expected, since the kernel will probably not signal EOF as long as some other backend is randomly holding open a reference to the connection, even if the client went away long since ...
2001-02-08 01:35:10 +01:00
{
pgindent run.
2003-08-04 02:43:34 +02:00
int i;
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
#ifndef WIN32
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
/*
* Close the write end of postmaster death watch pipe. It's important to
* do this as early as possible, so that if postmaster dies, others won't
* think that it's still running because we're holding the pipe open.
*/
Use consistent style for checking return from system calls Use if (something() != 0) error ... instead of just if (something) error ... The latter is not incorrect, but it's a bit confusing and not the common style. Discussion: https://www.postgresql.org/message-id/flat/5de61b6b-8be9-7771-0048-860328efe027%402ndquadrant.com
2019-07-06 23:18:46 +02:00
if (close(postmaster_alive_fds[POSTMASTER_FD_OWN]) != 0)
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
ereport(FATAL,
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
(errcode_for_file_access(),
errmsg_internal("could not close postmaster death monitoring pipe in child process: %m")));
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
postmaster_alive_fds[POSTMASTER_FD_OWN] = -1;
#endif
When launching a child backend, take care to close file descriptors for any other client connections that may exist (which would only happen if another client is currently in the authentication cycle). This avoids wastage of open descriptors in a child. It might also explain peculiar behaviors like not closing connections when expected, since the kernel will probably not signal EOF as long as some other backend is randomly holding open a reference to the connection, even if the client went away long since ...
2001-02-08 01:35:10 +01:00
/* Close the listen sockets */
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
for (i = 0; i < MAXLISTEN; i++)
{
Create typedef pgsocket for storing socket descriptors. This silences some warnings on Win64. Not using the proper SOCKET datatype was actually wrong on Win32 as well, but didn't cause any warnings there. Also create define PGINVALID_SOCKET to indicate an invalid/non-existing socket, instead of using a hardcoded -1 value.
2010-01-10 15:16:08 +01:00
if (ListenSocket[i] != PGINVALID_SOCKET)
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
{
StreamClose(ListenSocket[i]);
Create typedef pgsocket for storing socket descriptors. This silences some warnings on Win64. Not using the proper SOCKET datatype was actually wrong on Win32 as well, but didn't cause any warnings there. Also create define PGINVALID_SOCKET to indicate an invalid/non-existing socket, instead of using a hardcoded -1 value.
2010-01-10 15:16:08 +01:00
ListenSocket[i] = PGINVALID_SOCKET;
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
}
}
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
/* If using syslogger, close the read side of the pipe */
if (!am_syslogger)
{
#ifndef WIN32
if (syslogPipe[0] >= 0)
close(syslogPipe[0]);
syslogPipe[0] = -1;
#else
if (syslogPipe[0])
CloseHandle(syslogPipe[0]);
syslogPipe[0] = 0;
#endif
}
Replace use of the long-deprecated Bonjour API DNSServiceRegistrationCreate with the not-so-deprecated DNSServiceRegister. This patch shouldn't change any user-visible behavior, it just gets rid of a deprecation warning in --with-bonjour builds. The new code will fail on OS X releases before 10.3, but it seems unlikely that anyone will want to run Postgres 8.5 on 10.2.
2009-09-08 18:08:26 +02:00
#ifdef USE_BONJOUR
/* If using Bonjour, close the connection to the mDNS daemon */
if (bonjour_sdref)
close(DNSServiceRefSockFD(bonjour_sdref));
#endif
When launching a child backend, take care to close file descriptors for any other client connections that may exist (which would only happen if another client is currently in the authentication cycle). This avoids wastage of open descriptors in a child. It might also explain peculiar behaviors like not closing connections when expected, since the kernel will probably not signal EOF as long as some other backend is randomly holding open a reference to the connection, even if the client went away long since ...
2001-02-08 01:35:10 +01:00
}
UUNET is looking into offering PostgreSQL as a part of a managed web hosting product, on both shared and dedicated machines. We currently offer Oracle and MySQL, and it would be a nice middle-ground. However, as shipped, PostgreSQL lacks the following features we need that MySQL has: 1. The ability to listen only on a particular IP address. Each hosting customer has their own IP address, on which all of their servers (http, ftp, real media, etc.) run. 2. The ability to place the Unix-domain socket in a mode 700 directory. This allows us to automatically create an empty database, with an empty DBA password, for new or upgrading customers without having to interactively set a DBA password and communicate it to (or from) the customer. This in turn cuts down our install and upgrade times. 3. The ability to connect to the Unix-domain socket from within a change-rooted environment. We run CGI programs chrooted to the user's home directory, which is another reason why we need to be able to specify where the Unix-domain socket is, instead of /tmp. 4. The ability to, if run as root, open a pid file in /var/run as root, and then setuid to the desired user. (mysqld -u can almost do this; I had to patch it, too). The patch below fixes problem 1-3. I plan to address #4, also, but haven't done so yet. These diffs are big enough that they should give the PG development team something to think about in the meantime :-) Also, I'm about to leave for 2 weeks' vacation, so I thought I'd get out what I have, which works (for the problems it tackles), now. With these changes, we can set up and run PostgreSQL with scripts the same way we can with apache or proftpd or mysql. In summary, this patch makes the following enhancements: 1. Adds an environment variable PGUNIXSOCKET, analogous to MYSQL_UNIX_PORT, and command line options -k --unix-socket to the relevant programs. 2. Adds a -h option to postmaster to set the hostname or IP address to listen on instead of the default INADDR_ANY. 3. Extends some library interfaces to support the above. 4. Fixes a few memory leaks in PQconnectdb(). The default behavior is unchanged from stock 7.0.2; if you don't use any of these new features, they don't change the operation. David J. MacKenzie
2000-11-13 16:18:15 +01:00
Refactor pid, random seed and start time initialization. Background workers, including parallel workers, were generating the same sequence of numbers in random(). This showed up as DSM handle collisions when Parallel Hash created multiple segments, but any code that calls random() in background workers could be affected if it cares about different backends generating different numbers. Repair by making sure that all new processes initialize the seed at the same time as they set MyProcPid and MyStartTime in a new function InitProcessGlobals(), called by the postmaster, its children and also standalone processes. Also add a new high resolution MyStartTimestamp as a potentially useful by-product, and remove SessionStartTime from struct Port as it is now redundant. No back-patch for now, as the known consequences so far are just a bunch of harmless shm_open(O_EXCL) collisions. Author: Thomas Munro Reviewed-by: Tom Lane Discussion: https://postgr.es/m/CAEepm%3D2eJj_6%3DB%2B2tEpGu2nf1BjthCf9nXXUouYvJJ4C5WSwhg%40mail.gmail.com
2018-10-19 02:59:14 +02:00
/*
* InitProcessGlobals -- set MyProcPid, MyStartTime[stamp], random seeds
*
Use pg_strong_random() to select each server process's random seed. Previously we just set the seed based on process ID and start timestamp. Both those values are directly available within the session, and can be found out or guessed by other users too, making the session's series of random(3) values fairly predictable. Up to now, our backend-internal uses of random(3) haven't seemed security-critical, but commit 88bdbd3f7 added one that potentially is: when using log_statement_sample_rate, a user might be able to predict which of his SQL statements will get logged. To improve this situation, upgrade the per-process seed initialization method to use pg_strong_random() if available, greatly reducing the predictability of the initial seed value. This adds a few tens of microseconds to process start time, but since backend startup time is at least a couple of milliseconds, that seems an acceptable price. This means that pg_strong_random() needs to be able to run without reliance on any backend infrastructure, since it will be invoked before any of that is up. It was safe for that already, but adjust comments and #include commands to make it clearer. Discussion: https://postgr.es/m/3859.1545849900@sss.pgh.pa.us
2018-12-29 23:56:06 +01:00
* Called early in the postmaster and every backend.
Refactor pid, random seed and start time initialization. Background workers, including parallel workers, were generating the same sequence of numbers in random(). This showed up as DSM handle collisions when Parallel Hash created multiple segments, but any code that calls random() in background workers could be affected if it cares about different backends generating different numbers. Repair by making sure that all new processes initialize the seed at the same time as they set MyProcPid and MyStartTime in a new function InitProcessGlobals(), called by the postmaster, its children and also standalone processes. Also add a new high resolution MyStartTimestamp as a potentially useful by-product, and remove SessionStartTime from struct Port as it is now redundant. No back-patch for now, as the known consequences so far are just a bunch of harmless shm_open(O_EXCL) collisions. Author: Thomas Munro Reviewed-by: Tom Lane Discussion: https://postgr.es/m/CAEepm%3D2eJj_6%3DB%2B2tEpGu2nf1BjthCf9nXXUouYvJJ4C5WSwhg%40mail.gmail.com
2018-10-19 02:59:14 +02:00
*/
void
InitProcessGlobals(void)
{
Use pg_strong_random() to select each server process's random seed. Previously we just set the seed based on process ID and start timestamp. Both those values are directly available within the session, and can be found out or guessed by other users too, making the session's series of random(3) values fairly predictable. Up to now, our backend-internal uses of random(3) haven't seemed security-critical, but commit 88bdbd3f7 added one that potentially is: when using log_statement_sample_rate, a user might be able to predict which of his SQL statements will get logged. To improve this situation, upgrade the per-process seed initialization method to use pg_strong_random() if available, greatly reducing the predictability of the initial seed value. This adds a few tens of microseconds to process start time, but since backend startup time is at least a couple of milliseconds, that seems an acceptable price. This means that pg_strong_random() needs to be able to run without reliance on any backend infrastructure, since it will be invoked before any of that is up. It was safe for that already, but adjust comments and #include commands to make it clearer. Discussion: https://postgr.es/m/3859.1545849900@sss.pgh.pa.us
2018-12-29 23:56:06 +01:00
unsigned int rseed;
Refactor pid, random seed and start time initialization. Background workers, including parallel workers, were generating the same sequence of numbers in random(). This showed up as DSM handle collisions when Parallel Hash created multiple segments, but any code that calls random() in background workers could be affected if it cares about different backends generating different numbers. Repair by making sure that all new processes initialize the seed at the same time as they set MyProcPid and MyStartTime in a new function InitProcessGlobals(), called by the postmaster, its children and also standalone processes. Also add a new high resolution MyStartTimestamp as a potentially useful by-product, and remove SessionStartTime from struct Port as it is now redundant. No back-patch for now, as the known consequences so far are just a bunch of harmless shm_open(O_EXCL) collisions. Author: Thomas Munro Reviewed-by: Tom Lane Discussion: https://postgr.es/m/CAEepm%3D2eJj_6%3DB%2B2tEpGu2nf1BjthCf9nXXUouYvJJ4C5WSwhg%40mail.gmail.com
2018-10-19 02:59:14 +02:00
MyProcPid = getpid();
MyStartTimestamp = GetCurrentTimestamp();
MyStartTime = timestamptz_to_time_t(MyStartTimestamp);
Increase the number of possible random seeds per time period. Commit 197e4af9 refactored the initialization of the libc random() seed, but reduced the number of possible seeds values that could be chosen in a given time period. This negation of the effects of commit 98c50656c was unintentional. Replace with code that shifts the fast moving timestamp bits left, similar to the original algorithm (though not the previous float-tolerating coding, which is no longer necessary). Author: Thomas Munro Reported-by: Noah Misch Reviewed-by: Tom Lane Discussion: https://postgr.es/m/20181112083358.GA1073796%40rfd.leadboat.com
2018-11-15 04:25:30 +01:00
/*
Use pg_strong_random() to select each server process's random seed. Previously we just set the seed based on process ID and start timestamp. Both those values are directly available within the session, and can be found out or guessed by other users too, making the session's series of random(3) values fairly predictable. Up to now, our backend-internal uses of random(3) haven't seemed security-critical, but commit 88bdbd3f7 added one that potentially is: when using log_statement_sample_rate, a user might be able to predict which of his SQL statements will get logged. To improve this situation, upgrade the per-process seed initialization method to use pg_strong_random() if available, greatly reducing the predictability of the initial seed value. This adds a few tens of microseconds to process start time, but since backend startup time is at least a couple of milliseconds, that seems an acceptable price. This means that pg_strong_random() needs to be able to run without reliance on any backend infrastructure, since it will be invoked before any of that is up. It was safe for that already, but adjust comments and #include commands to make it clearer. Discussion: https://postgr.es/m/3859.1545849900@sss.pgh.pa.us
2018-12-29 23:56:06 +01:00
* Set a different seed for random() in every process. We want something
* unpredictable, so if possible, use high-quality random bits for the
* seed. Otherwise, fall back to a seed based on timestamp and PID.
Increase the number of possible random seeds per time period. Commit 197e4af9 refactored the initialization of the libc random() seed, but reduced the number of possible seeds values that could be chosen in a given time period. This negation of the effects of commit 98c50656c was unintentional. Replace with code that shifts the fast moving timestamp bits left, similar to the original algorithm (though not the previous float-tolerating coding, which is no longer necessary). Author: Thomas Munro Reported-by: Noah Misch Reviewed-by: Tom Lane Discussion: https://postgr.es/m/20181112083358.GA1073796%40rfd.leadboat.com
2018-11-15 04:25:30 +01:00
*/
Use pg_strong_random() to select each server process's random seed. Previously we just set the seed based on process ID and start timestamp. Both those values are directly available within the session, and can be found out or guessed by other users too, making the session's series of random(3) values fairly predictable. Up to now, our backend-internal uses of random(3) haven't seemed security-critical, but commit 88bdbd3f7 added one that potentially is: when using log_statement_sample_rate, a user might be able to predict which of his SQL statements will get logged. To improve this situation, upgrade the per-process seed initialization method to use pg_strong_random() if available, greatly reducing the predictability of the initial seed value. This adds a few tens of microseconds to process start time, but since backend startup time is at least a couple of milliseconds, that seems an acceptable price. This means that pg_strong_random() needs to be able to run without reliance on any backend infrastructure, since it will be invoked before any of that is up. It was safe for that already, but adjust comments and #include commands to make it clearer. Discussion: https://postgr.es/m/3859.1545849900@sss.pgh.pa.us
2018-12-29 23:56:06 +01:00
if (!pg_strong_random(&rseed, sizeof(rseed)))
{
/*
* Since PIDs and timestamps tend to change more frequently in their
* least significant bits, shift the timestamp left to allow a larger
* total number of seeds in a given time period. Since that would
* leave only 20 bits of the timestamp that cycle every ~1 second,
* also mix in some higher bits.
*/
rseed = ((uint64) MyProcPid) ^
Further adjustment to random() seed initialization. Per complaint from Tom Lane, don't chomp the timestamp at 32 bits, so we can shift in some of its higher bits. Discussion: https://postgr.es/m/14712.1542253115%40sss.pgh.pa.us
2018-11-15 05:10:43 +01:00
((uint64) MyStartTimestamp << 12) ^
Use pg_strong_random() to select each server process's random seed. Previously we just set the seed based on process ID and start timestamp. Both those values are directly available within the session, and can be found out or guessed by other users too, making the session's series of random(3) values fairly predictable. Up to now, our backend-internal uses of random(3) haven't seemed security-critical, but commit 88bdbd3f7 added one that potentially is: when using log_statement_sample_rate, a user might be able to predict which of his SQL statements will get logged. To improve this situation, upgrade the per-process seed initialization method to use pg_strong_random() if available, greatly reducing the predictability of the initial seed value. This adds a few tens of microseconds to process start time, but since backend startup time is at least a couple of milliseconds, that seems an acceptable price. This means that pg_strong_random() needs to be able to run without reliance on any backend infrastructure, since it will be invoked before any of that is up. It was safe for that already, but adjust comments and #include commands to make it clearer. Discussion: https://postgr.es/m/3859.1545849900@sss.pgh.pa.us
2018-12-29 23:56:06 +01:00
((uint64) MyStartTimestamp >> 20);
}
srandom(rseed);
Refactor pid, random seed and start time initialization. Background workers, including parallel workers, were generating the same sequence of numbers in random(). This showed up as DSM handle collisions when Parallel Hash created multiple segments, but any code that calls random() in background workers could be affected if it cares about different backends generating different numbers. Repair by making sure that all new processes initialize the seed at the same time as they set MyProcPid and MyStartTime in a new function InitProcessGlobals(), called by the postmaster, its children and also standalone processes. Also add a new high resolution MyStartTimestamp as a potentially useful by-product, and remove SessionStartTime from struct Port as it is now redundant. No back-patch for now, as the known consequences so far are just a bunch of harmless shm_open(O_EXCL) collisions. Author: Thomas Munro Reviewed-by: Tom Lane Discussion: https://postgr.es/m/CAEepm%3D2eJj_6%3DB%2B2tEpGu2nf1BjthCf9nXXUouYvJJ4C5WSwhg%40mail.gmail.com
2018-10-19 02:59:14 +02:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
* reset_shared -- reset shared memory and semaphores
*/
static void
Use data directory inode number, not port, to select SysV resource keys. This approach provides a much tighter binding between a data directory and the associated SysV shared memory block (and SysV or named-POSIX semaphores, if we're using those). Key collisions are still possible, but only between data directories stored on different filesystems, so the situation should be negligible in practice. More importantly, restarting the postmaster with a different port number no longer risks failing to identify a relevant shared memory block, even when postmaster.pid has been removed. A standalone backend is likewise much more certain to detect conflicting leftover backends. (In the longer term, we might now think about deprecating the port as a cluster-wide value, so that one postmaster could support sockets with varying port numbers. But that's for another day.) The hazards fixed here apply only on Unix systems; our Windows code paths already use identifiers derived from the data directory path name rather than the port. src/test/recovery/t/017_shm.pl, which intends to test key-collision cases, has been substantially rewritten since it can no longer use two postmasters with identical port numbers to trigger the case. Instead, use Perl's IPC::SharedMem module to create a conflicting shmem segment directly. The test script will be skipped if that module is not available. (This means that some older buildfarm members won't run it, but I don't think that that results in any meaningful coverage loss.) Patch by me; thanks to Noah Misch and Peter Eisentraut for discussion and review. Discussion: https://postgr.es/m/16908.1557521200@sss.pgh.pa.us
2019-09-05 19:31:41 +02:00
reset_shared(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Significant cleanups in SysV IPC handling (shared mem and semaphores). IPC key assignment will now work correctly even when multiple postmasters are using same logical port number (which is possible given -k switch). There is only one shared-mem segment per postmaster now, not 3. Rip out broken code for non-TAS case in bufmgr and xlog, substitute a complete S_LOCK emulation using semaphores in spin.c. TAS and non-TAS logic is now exactly the same. When deadlock is detected, "Deadlock detected" is now the elog(ERROR) message, rather than a NOTICE that comes out before an unhelpful ERROR.
2000-11-29 00:27:57 +01:00
/*
* Create or re-create shared memory and semaphores.
Create an internal semaphore API that is not tied to SysV semaphores. As proof of concept, provide an alternate implementation based on POSIX semaphores. Also push the SysV shared-memory implementation into a separate file so that it can be replaced conveniently.
2002-05-05 02:03:29 +02:00
*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Note: in each "cycle of life" we will normally assign the same IPC keys
Use data directory inode number, not port, to select SysV resource keys. This approach provides a much tighter binding between a data directory and the associated SysV shared memory block (and SysV or named-POSIX semaphores, if we're using those). Key collisions are still possible, but only between data directories stored on different filesystems, so the situation should be negligible in practice. More importantly, restarting the postmaster with a different port number no longer risks failing to identify a relevant shared memory block, even when postmaster.pid has been removed. A standalone backend is likewise much more certain to detect conflicting leftover backends. (In the longer term, we might now think about deprecating the port as a cluster-wide value, so that one postmaster could support sockets with varying port numbers. But that's for another day.) The hazards fixed here apply only on Unix systems; our Windows code paths already use identifiers derived from the data directory path name rather than the port. src/test/recovery/t/017_shm.pl, which intends to test key-collision cases, has been substantially rewritten since it can no longer use two postmasters with identical port numbers to trigger the case. Instead, use Perl's IPC::SharedMem module to create a conflicting shmem segment directly. The test script will be skipped if that module is not available. (This means that some older buildfarm members won't run it, but I don't think that that results in any meaningful coverage loss.) Patch by me; thanks to Noah Misch and Peter Eisentraut for discussion and review. Discussion: https://postgr.es/m/16908.1557521200@sss.pgh.pa.us
2019-09-05 19:31:41 +02:00
* (if using SysV shmem and/or semas). This helps ensure that we will
* clean up dead IPC objects if the postmaster crashes and is restarted.
Significant cleanups in SysV IPC handling (shared mem and semaphores). IPC key assignment will now work correctly even when multiple postmasters are using same logical port number (which is possible given -k switch). There is only one shared-mem segment per postmaster now, not 3. Rip out broken code for non-TAS case in bufmgr and xlog, substitute a complete S_LOCK emulation using semaphores in spin.c. TAS and non-TAS logic is now exactly the same. When deadlock is detected, "Deadlock detected" is now the elog(ERROR) message, rather than a NOTICE that comes out before an unhelpful ERROR.
2000-11-29 00:27:57 +01:00
*/
Use data directory inode number, not port, to select SysV resource keys. This approach provides a much tighter binding between a data directory and the associated SysV shared memory block (and SysV or named-POSIX semaphores, if we're using those). Key collisions are still possible, but only between data directories stored on different filesystems, so the situation should be negligible in practice. More importantly, restarting the postmaster with a different port number no longer risks failing to identify a relevant shared memory block, even when postmaster.pid has been removed. A standalone backend is likewise much more certain to detect conflicting leftover backends. (In the longer term, we might now think about deprecating the port as a cluster-wide value, so that one postmaster could support sockets with varying port numbers. But that's for another day.) The hazards fixed here apply only on Unix systems; our Windows code paths already use identifiers derived from the data directory path name rather than the port. src/test/recovery/t/017_shm.pl, which intends to test key-collision cases, has been substantially rewritten since it can no longer use two postmasters with identical port numbers to trigger the case. Instead, use Perl's IPC::SharedMem module to create a conflicting shmem segment directly. The test script will be skipped if that module is not available. (This means that some older buildfarm members won't run it, but I don't think that that results in any meaningful coverage loss.) Patch by me; thanks to Noah Misch and Peter Eisentraut for discussion and review. Discussion: https://postgr.es/m/16908.1557521200@sss.pgh.pa.us
2019-09-05 19:31:41 +02:00
CreateSharedMemoryAndSemaphores();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
The heralded `Grand Unified Configuration scheme' (GUC) That means you can now set your options in either or all of $PGDATA/configuration, some postmaster option (--enable-fsync=off), or set a SET command. The list of options is in backend/utils/misc/guc.c, documentation will be written post haste. pg_options is gone, so is that pq_geqo config file. Also removed were backend -K, -Q, and -T options (no longer applicable, although -d0 does the same as -Q). Added to configure an --enable-syslog option. changed all callers from TPRINTF to elog(DEBUG)
2000-05-31 02:28:42 +02:00
/*
Fix now-obsolete comment.
2001-11-04 21:12:57 +01:00
* SIGHUP -- reread config files, and tell children to do same
The heralded `Grand Unified Configuration scheme' (GUC) That means you can now set your options in either or all of $PGDATA/configuration, some postmaster option (--enable-fsync=off), or set a SET command. The list of options is in backend/utils/misc/guc.c, documentation will be written post haste. pg_options is gone, so is that pq_geqo config file. Also removed were backend -K, -Q, and -T options (no longer applicable, although -d0 does the same as -Q). Added to configure an --enable-syslog option. changed all callers from TPRINTF to elog(DEBUG)
2000-05-31 02:28:42 +02:00
*/
static void
Revert removal of signed, volatile, and signal handler arg type tests.
2000-08-29 11:36:51 +02:00
SIGHUP_handler(SIGNAL_ARGS)
The heralded `Grand Unified Configuration scheme' (GUC) That means you can now set your options in either or all of $PGDATA/configuration, some postmaster option (--enable-fsync=off), or set a SET command. The list of options is in backend/utils/misc/guc.c, documentation will be written post haste. pg_options is gone, so is that pq_geqo config file. Also removed were backend -K, -Q, and -T options (no longer applicable, although -d0 does the same as -Q). Added to configure an --enable-syslog option. changed all callers from TPRINTF to elog(DEBUG)
2000-05-31 02:28:42 +02:00
{
Ensure that 'errno' is saved and restored by all signal handlers that might change it. Experimentation shows that the signal handler call mechanism does not save/restore errno for you, at least not on Linux or HPUX, so this is definitely a real risk.
2000-12-18 18:33:42 +01:00
int save_errno = errno;
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
/*
* We rely on the signal mechanism to have blocked all signals ... except
* on Windows, which lacks sigaction(), so we have to do it manually.
*/
#ifdef WIN32
Add missing PG_SETMASK(&BlockSig) to SIGHUP_handler().
2001-06-14 21:59:24 +02:00
PG_SETMASK(&BlockSig);
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
#endif
Add missing PG_SETMASK(&BlockSig) to SIGHUP_handler().
2001-06-14 21:59:24 +02:00
if (Shutdown <= SmartShutdown)
{
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(LOG,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errmsg("received SIGHUP, reloading configuration files")));
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
ProcessConfigFile(PGC_SIGHUP);
Write non-defaults file before signalling the children to reread it.
2003-05-08 22:43:07 +02:00
SignalChildren(SIGHUP);
Reload config file in startup process on SIGHUP. Fujii Masao
2009-03-04 14:56:40 +01:00
if (StartupPID != 0)
signal_child(StartupPID, SIGHUP);
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
if (BgWriterPID != 0)
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
signal_child(BgWriterPID, SIGHUP);
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
if (CheckpointerPID != 0)
signal_child(CheckpointerPID, SIGHUP);
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
if (WalWriterPID != 0)
signal_child(WalWriterPID, SIGHUP);
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
if (WalReceiverPID != 0)
signal_child(WalReceiverPID, SIGHUP);
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
if (AutoVacPID != 0)
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
signal_child(AutoVacPID, SIGHUP);
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
if (PgArchPID != 0)
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
signal_child(PgArchPID, SIGHUP);
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
if (SysLoggerPID != 0)
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
signal_child(SysLoggerPID, SIGHUP);
Make stats_temp_directory PGC_SIGHUP, and document how it may cause a temporary "outage" of the statistics views. This requires making the stats collector respond to SIGHUP, like the other utility processes already did.
2008-08-25 17:11:01 +02:00
if (PgStatPID != 0)
signal_child(PgStatPID, SIGHUP);
Add code to prevent transaction ID wraparound by enforcing a safe limit in GetNewTransactionId(). Since the limit value has to be computed before we run any real transactions, this requires adding code to database startup to scan pg_database and determine the oldest datfrozenxid. This can conveniently be combined with the first stage of an attack on the problem that the 'flat file' copies of pg_shadow and pg_group are not properly updated during WAL recovery. The code I've added to startup resides in a new file src/backend/utils/init/flatfiles.c, and it is responsible for rewriting the flat files as well as initializing the XID wraparound limit value. This will eventually allow us to get rid of GetRawDatabaseInfo too, but we'll need an initdb so we can add a trigger to pg_database.
2005-02-20 03:22:07 +01:00
/* Reload authentication config files too */
Parse pg_hba.conf in postmaster, instead of once in each backend for each connection. This makes it possible to catch errors in the pg_hba file when it's being reloaded, instead of silently reloading a broken file and failing only when a user tries to connect. This patch also makes the "sameuser" argument to ident authentication optional.
2008-09-15 14:32:57 +02:00
if (!load_hba())
Allow SSL configuration to be updated at SIGHUP. It is no longer necessary to restart the server to enable, disable, or reconfigure SSL. Instead, we just create a new SSL_CTX struct (by re-reading all relevant files) whenever we get SIGHUP. Testing shows that this is fast enough that it shouldn't be a problem. In conjunction with that, downgrade the logic that complains about pg_hba.conf "hostssl" lines when SSL isn't active: now that's just a warning condition not an error. An issue that still needs to be addressed is what shall we do with passphrase-protected server keys? As this stands, the server would demand the passphrase again on every SIGHUP, which is certainly impractical. But the case was only barely supported before, so that does not seem a sufficient reason to hold up committing this patch. Andreas Karlsson, reviewed by Michael Banck and Michael Paquier Discussion: https://postgr.es/m/556A6E8A.9030400@proxel.se
2017-01-03 03:37:12 +01:00
ereport(LOG,
More message style fixes Discussion: https://postgr.es/m/20190515183005.GA26486@alvherre.pgsql
2019-05-17 00:50:56 +02:00
/* translator: %s is a configuration file */
(errmsg("%s was not reloaded", "pg_hba.conf")));
Parse pg_hba.conf in postmaster, instead of once in each backend for each connection. This makes it possible to catch errors in the pg_hba file when it's being reloaded, instead of silently reloading a broken file and failing only when a user tries to connect. This patch also makes the "sameuser" argument to ident authentication optional.
2008-09-15 14:32:57 +02:00
Parse pg_ident.conf when it's loaded, keeping it in memory in parsed format. Similar changes were done to pg_hba.conf earlier already, this commit makes pg_ident.conf to behave the same as pg_hba.conf. This has two user-visible effects. First, if pg_ident.conf contains multiple errors, the whole file is parsed at postmaster startup time and all the errors are immediately reported. Before this patch, the file was parsed and the errors were reported only when someone tries to connect using an authentication method that uses the file, and the parsing stopped on first error. Second, if you SIGHUP to reload the config files, and the new pg_ident.conf file contains an error, the error is logged but the old file stays in effect. Also, regular expressions in pg_ident.conf are now compiled only once when the file is loaded, rather than every time the a user is authenticated. That should speed up authentication if you have a lot of regexps in the file. Amit Kapila
2012-09-21 16:41:22 +02:00
if (!load_ident())
Allow SSL configuration to be updated at SIGHUP. It is no longer necessary to restart the server to enable, disable, or reconfigure SSL. Instead, we just create a new SSL_CTX struct (by re-reading all relevant files) whenever we get SIGHUP. Testing shows that this is fast enough that it shouldn't be a problem. In conjunction with that, downgrade the logic that complains about pg_hba.conf "hostssl" lines when SSL isn't active: now that's just a warning condition not an error. An issue that still needs to be addressed is what shall we do with passphrase-protected server keys? As this stands, the server would demand the passphrase again on every SIGHUP, which is certainly impractical. But the case was only barely supported before, so that does not seem a sufficient reason to hold up committing this patch. Andreas Karlsson, reviewed by Michael Banck and Michael Paquier Discussion: https://postgr.es/m/556A6E8A.9030400@proxel.se
2017-01-03 03:37:12 +01:00
ereport(LOG,
More message style fixes Discussion: https://postgr.es/m/20190515183005.GA26486@alvherre.pgsql
2019-05-17 00:50:56 +02:00
(errmsg("%s was not reloaded", "pg_ident.conf")));
Background writer process This first part of the background writer does no syncing at all. It's only purpose is to keep the LRU heads clean so that regular backends seldom to never have to call write(). Jan
2003-11-19 16:55:08 +01:00
Allow SSL configuration to be updated at SIGHUP. It is no longer necessary to restart the server to enable, disable, or reconfigure SSL. Instead, we just create a new SSL_CTX struct (by re-reading all relevant files) whenever we get SIGHUP. Testing shows that this is fast enough that it shouldn't be a problem. In conjunction with that, downgrade the logic that complains about pg_hba.conf "hostssl" lines when SSL isn't active: now that's just a warning condition not an error. An issue that still needs to be addressed is what shall we do with passphrase-protected server keys? As this stands, the server would demand the passphrase again on every SIGHUP, which is certainly impractical. But the case was only barely supported before, so that does not seem a sufficient reason to hold up committing this patch. Andreas Karlsson, reviewed by Michael Banck and Michael Paquier Discussion: https://postgr.es/m/556A6E8A.9030400@proxel.se
2017-01-03 03:37:12 +01:00
#ifdef USE_SSL
/* Reload SSL configuration as well */
if (EnableSSL)
{
if (secure_initialize(false) == 0)
LoadedSSL = true;
else
ereport(LOG,
Re-allow SSL passphrase prompt at server start, but not thereafter. Leave OpenSSL's default passphrase collection callback in place during the first call of secure_initialize() in server startup. Although that doesn't work terribly well in daemon contexts, some people feel we should not break it for anyone who was successfully using it before. We still block passphrase demands during SIGHUP, meaning that you can't adjust SSL configuration on-the-fly if you used a passphrase, but this is no worse than what it was before commit de41869b6. And we block passphrase demands during EXEC_BACKEND reloads; that behavior wasn't useful either, but at least now it's documented. Tweak some related log messages for more readability, and avoid issuing essentially duplicate messages about reload failure caused by a passphrase. Discussion: https://postgr.es/m/29982.1483412575@sss.pgh.pa.us
2017-01-04 18:43:52 +01:00
(errmsg("SSL configuration was not reloaded")));
Allow SSL configuration to be updated at SIGHUP. It is no longer necessary to restart the server to enable, disable, or reconfigure SSL. Instead, we just create a new SSL_CTX struct (by re-reading all relevant files) whenever we get SIGHUP. Testing shows that this is fast enough that it shouldn't be a problem. In conjunction with that, downgrade the logic that complains about pg_hba.conf "hostssl" lines when SSL isn't active: now that's just a warning condition not an error. An issue that still needs to be addressed is what shall we do with passphrase-protected server keys? As this stands, the server would demand the passphrase again on every SIGHUP, which is certainly impractical. But the case was only barely supported before, so that does not seem a sufficient reason to hold up committing this patch. Andreas Karlsson, reviewed by Michael Banck and Michael Paquier Discussion: https://postgr.es/m/556A6E8A.9030400@proxel.se
2017-01-03 03:37:12 +01:00
}
else
{
secure_destroy();
LoadedSSL = false;
}
#endif
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
#ifdef EXEC_BACKEND
/* Update the starting-point file for future children */
write_nondefault_variables(PGC_SIGHUP);
#endif
Add missing PG_SETMASK(&BlockSig) to SIGHUP_handler().
2001-06-14 21:59:24 +02:00
}
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
#ifdef WIN32
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
PG_SETMASK(&UnBlockSig);
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
#endif
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
Ensure that 'errno' is saved and restored by all signal handlers that might change it. Experimentation shows that the signal handler call mechanism does not save/restore errno for you, at least not on Linux or HPUX, so this is definitely a real risk.
2000-12-18 18:33:42 +01:00
errno = save_errno;
The heralded `Grand Unified Configuration scheme' (GUC) That means you can now set your options in either or all of $PGDATA/configuration, some postmaster option (--enable-fsync=off), or set a SET command. The list of options is in backend/utils/misc/guc.c, documentation will be written post haste. pg_options is gone, so is that pq_geqo config file. Also removed were backend -K, -Q, and -T options (no longer applicable, although -d0 does the same as -Q). Added to configure an --enable-syslog option. changed all callers from TPRINTF to elog(DEBUG)
2000-05-31 02:28:42 +02:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
* pmdie -- signal handler for processing various postmaster signals.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
static void
Revert removal of signed, volatile, and signal handler arg type tests.
2000-08-29 11:36:51 +02:00
pmdie(SIGNAL_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Ensure that 'errno' is saved and restored by all signal handlers that might change it. Experimentation shows that the signal handler call mechanism does not save/restore errno for you, at least not on Linux or HPUX, so this is definitely a real risk.
2000-12-18 18:33:42 +01:00
int save_errno = errno;
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
/*
* We rely on the signal mechanism to have blocked all signals ... except
* on Windows, which lacks sigaction(), so we have to do it manually.
*/
#ifdef WIN32
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
PG_SETMASK(&BlockSig);
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
#endif
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
Change some frequently-reached elog(DEBUG...) calls to ereport(DEBUG...) for speed reasons. (ereport falls out much more quickly when no output is needed than elog does.)
2003-08-12 20:23:21 +02:00
ereport(DEBUG2,
(errmsg_internal("postmaster received signal %d",
postgres_signal_arg)));
From: Massimo Dal Zotto <dz@cs.unitn.it> > tprintf.patch > > tprintf.patch > > adds functions and macros which implement a conditional trace package > with the ability to change flags and numeric options of running > backends at runtime. > Options/flags can be specified in the command line and/or read from > the file pg_options in the data directory.
1998-08-25 23:34:10 +02:00
Revert removal of signed, volatile, and signal handler arg type tests.
2000-08-29 11:36:51 +02:00
switch (postgres_signal_arg)
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
{
From: Massimo Dal Zotto <dz@cs.unitn.it> > tprintf.patch > > tprintf.patch > > adds functions and macros which implement a conditional trace package > with the ability to change flags and numeric options of running > backends at runtime. > Options/flags can be specified in the command line and/or read from > the file pg_options in the data directory.
1998-08-25 23:34:10 +02:00
case SIGTERM:
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
/*
* Smart Shutdown:
*
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
* Wait for children to end their work, then shut down.
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
*/
if (Shutdown >= SmartShutdown)
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
break;
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
Shutdown = SmartShutdown;
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(LOG,
Revert "Temporarily make pg_ctl and server shutdown a whole lot chattier." This reverts commit 3971f64843b02e4a55d854156bd53e46a0588e45 and a couple of followon debugging commits; I think we've learned what we can from them.
2016-02-10 22:01:04 +01:00
(errmsg("received smart shutdown request")));
Change pg_ctl to detect server-ready by watching status in postmaster.pid. Traditionally, "pg_ctl start -w" has waited for the server to become ready to accept connections by attempting a connection once per second. That has the major problem that connection issues (for instance, a kernel packet filter blocking traffic) can't be reliably told apart from server startup issues, and the minor problem that if server startup isn't quick, we accumulate "the database system is starting up" spam in the server log. We've hacked around many of the possible connection issues, but it resulted in ugly and complicated code in pg_ctl.c. In commit c61559ec3, I changed the probe rate to every tenth of a second. That prompted Jeff Janes to complain that the log-spam problem had become much worse. In the ensuing discussion, Andres Freund pointed out that we could dispense with connection attempts altogether if the postmaster were changed to report its status in postmaster.pid, which "pg_ctl start" already relies on being able to read. This patch implements that, teaching postmaster.c to report a status string into the pidfile at the same state-change points already identified as being of interest for systemd status reporting (cf commit 7d17e683f). pg_ctl no longer needs to link with libpq at all; all its functions now depend on reading server files. In support of this, teach AddToDataDirLockFile() to allow addition of postmaster.pid lines in not-necessarily-sequential order. This is needed on Windows where the SHMEM_KEY line will never be written at all. We still have the restriction that we don't want to truncate the pidfile; document the reasons for that a bit better. Also, fix the pg_ctl TAP tests so they'll notice if "start -w" mode is broken --- before, they'd just wait out the sixty seconds until the loop gives up, and then report success anyway. (Yes, I found that out the hard way.) While at it, arrange for pg_ctl to not need to #include miscadmin.h; as a rather low-level backend header, requiring that to be compilable client-side is pretty dubious. This requires moving the #define's associated with the pidfile into a new header file, and moving PG_BACKEND_VERSIONSTR someplace else. For lack of a clearly better "someplace else", I put it into port.h, beside the declaration of find_other_exec(), since most users of that macro are passing the value to find_other_exec(). (initdb still depends on miscadmin.h, but at least pg_ctl and pg_upgrade no longer do.) In passing, fix main.c so that PG_BACKEND_VERSIONSTR actually defines the output of "postgres -V", which remarkably it had never done before. Discussion: https://postgr.es/m/CAMkU=1xJW8e+CTotojOMBd-yzUvD0e_JZu2xHo=MnuZ4__m7Pg@mail.gmail.com
2017-06-28 23:31:24 +02:00
/* Report status */
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STOPPING);
Add support for systemd service notifications Insert sd_notify() calls at server start and stop for integration with systemd. This allows the use of systemd service units of type "notify", which greatly simplifies the systemd configuration. Reviewed-by: Pavel Stěhule <pavel.stehule@gmail.com>
2015-11-17 12:46:17 +01:00
#ifdef USE_SYSTEMD
sd_notify(0, "STOPPING=1");
#endif
Repair some problems in bgwriter start/stop logic. In particular, don't allow the bgwriter to start before the startup subprocess has finished ... it tends to crash otherwise. (The same problem may have existed for the checkpointer, I'm not entirely sure.) Remove some code that was redundant because the bgwriter is handled as a member of the backend list.
2004-02-11 23:25:02 +01:00
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
if (pmState == PM_RUN || pmState == PM_RECOVERY ||
More fixes for shutdown during recovery. 1. If we receive a fast shutdown request while in the PM_STARTUP state, process it just as we would in PM_RECOVERY, PM_HOT_STANDBY, or PM_RUN. Without this change, an early fast shutdown followed by Hot Standby causes the database to get stuck in a state where a shutdown is pending (so no new connections are allowed) but the shutdown request is never processed unless we end Hot Standby and enter normal running. 2. Avoid removing the backup label file when a smart or fast shutdown occurs during recovery. It makes sense to do this once we've reached normal running, since we must be taking a backup which now won't be valid. But during recovery we must be recovering from a previously taken backup, and any backup label file is needed to restart recovery from the right place. Fujii Masao and Robert Haas
2010-05-26 14:32:41 +02:00
pmState == PM_HOT_STANDBY || pmState == PM_STARTUP)
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
{
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/* autovac workers are told to shut down immediately */
/* and bgworkers too; does this need tweaking? */
SignalSomeChildren(SIGTERM,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/* and the autovac launcher too */
if (AutoVacPID != 0)
signal_child(AutoVacPID, SIGTERM);
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
/* and the bgwriter too */
if (BgWriterPID != 0)
signal_child(BgWriterPID, SIGTERM);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/* and the walwriter too */
if (WalWriterPID != 0)
signal_child(WalWriterPID, SIGTERM);
pgindent run for 9.0, second run
2010-07-06 21:19:02 +02:00
Make smart shutdown work in combination with Hot Standby/Streaming Replication. At present, killing the startup process does not release any locks it holds, so we must wait to stop the startup and walreceiver processes until all read-only backends have exited. Without this patch, the startup and walreceiver processes never exit, so the server gets permanently stuck in a half-shutdown state. Fujii Masao, with review, docs, and comment adjustments by me.
2010-04-08 03:39:37 +02:00
/*
* If we're in recovery, we can't kill the startup process
* right away, because at present doing so does not release
* its locks. We might want to change this in a future
* release. For the time being, the PM_WAIT_READONLY state
* indicates that we're waiting for the regular (read only)
* backends to die off; once they do, we'll kill the startup
* and walreceiver processes.
*/
pmState = (pmState == PM_RUN) ?
PM_WAIT_BACKUP : PM_WAIT_READONLY;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
}
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
/*
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* Now wait for online backup mode to end and backends to exit. If
* that is already the case, PostmasterStateMachine will take the
* next step.
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
*/
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
PostmasterStateMachine();
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
break;
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
case SIGINT:
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
/*
* Fast Shutdown:
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* Abort all children with SIGTERM (rollback active transactions
* and exit) and shut down when they are gone.
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
*/
if (Shutdown >= FastShutdown)
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
break;
Repair some problems in bgwriter start/stop logic. In particular, don't allow the bgwriter to start before the startup subprocess has finished ... it tends to crash otherwise. (The same problem may have existed for the checkpointer, I'm not entirely sure.) Remove some code that was redundant because the bgwriter is handled as a member of the backend list.
2004-02-11 23:25:02 +01:00
Shutdown = FastShutdown;
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(LOG,
Revert "Temporarily make pg_ctl and server shutdown a whole lot chattier." This reverts commit 3971f64843b02e4a55d854156bd53e46a0588e45 and a couple of followon debugging commits; I think we've learned what we can from them.
2016-02-10 22:01:04 +01:00
(errmsg("received fast shutdown request")));
Change pg_ctl to detect server-ready by watching status in postmaster.pid. Traditionally, "pg_ctl start -w" has waited for the server to become ready to accept connections by attempting a connection once per second. That has the major problem that connection issues (for instance, a kernel packet filter blocking traffic) can't be reliably told apart from server startup issues, and the minor problem that if server startup isn't quick, we accumulate "the database system is starting up" spam in the server log. We've hacked around many of the possible connection issues, but it resulted in ugly and complicated code in pg_ctl.c. In commit c61559ec3, I changed the probe rate to every tenth of a second. That prompted Jeff Janes to complain that the log-spam problem had become much worse. In the ensuing discussion, Andres Freund pointed out that we could dispense with connection attempts altogether if the postmaster were changed to report its status in postmaster.pid, which "pg_ctl start" already relies on being able to read. This patch implements that, teaching postmaster.c to report a status string into the pidfile at the same state-change points already identified as being of interest for systemd status reporting (cf commit 7d17e683f). pg_ctl no longer needs to link with libpq at all; all its functions now depend on reading server files. In support of this, teach AddToDataDirLockFile() to allow addition of postmaster.pid lines in not-necessarily-sequential order. This is needed on Windows where the SHMEM_KEY line will never be written at all. We still have the restriction that we don't want to truncate the pidfile; document the reasons for that a bit better. Also, fix the pg_ctl TAP tests so they'll notice if "start -w" mode is broken --- before, they'd just wait out the sixty seconds until the loop gives up, and then report success anyway. (Yes, I found that out the hard way.) While at it, arrange for pg_ctl to not need to #include miscadmin.h; as a rather low-level backend header, requiring that to be compilable client-side is pretty dubious. This requires moving the #define's associated with the pidfile into a new header file, and moving PG_BACKEND_VERSIONSTR someplace else. For lack of a clearly better "someplace else", I put it into port.h, beside the declaration of find_other_exec(), since most users of that macro are passing the value to find_other_exec(). (initdb still depends on miscadmin.h, but at least pg_ctl and pg_upgrade no longer do.) In passing, fix main.c so that PG_BACKEND_VERSIONSTR actually defines the output of "postgres -V", which remarkably it had never done before. Discussion: https://postgr.es/m/CAMkU=1xJW8e+CTotojOMBd-yzUvD0e_JZu2xHo=MnuZ4__m7Pg@mail.gmail.com
2017-06-28 23:31:24 +02:00
/* Report status */
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STOPPING);
Add support for systemd service notifications Insert sd_notify() calls at server start and stop for integration with systemd. This allows the use of systemd service units of type "notify", which greatly simplifies the systemd configuration. Reviewed-by: Pavel Stěhule <pavel.stehule@gmail.com>
2015-11-17 12:46:17 +01:00
#ifdef USE_SYSTEMD
sd_notify(0, "STOPPING=1");
#endif
Repair some problems in bgwriter start/stop logic. In particular, don't allow the bgwriter to start before the startup subprocess has finished ... it tends to crash otherwise. (The same problem may have existed for the checkpointer, I'm not entirely sure.) Remove some code that was redundant because the bgwriter is handled as a member of the backend list.
2004-02-11 23:25:02 +01:00
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (StartupPID != 0)
signal_child(StartupPID, SIGTERM);
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
if (BgWriterPID != 0)
signal_child(BgWriterPID, SIGTERM);
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
if (WalReceiverPID != 0)
signal_child(WalReceiverPID, SIGTERM);
Stop bgworkers during fast shutdown with postmaster in startup phase When a postmaster gets into its phase PM_STARTUP, it would start background workers using BgWorkerStart_PostmasterStart mode immediately, which would cause problems for a fast shutdown as the postmaster forgets to send SIGTERM to already-started background workers. With smart and immediate shutdowns, this correctly happened, and fast shutdown is the only mode missing the shot. Author: Alexander Kukushkin Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/CAFh8B=mvnD8+DZUfzpi50DoaDfZRDfd7S=gwj5vU9GYn8UvHkA@mail.gmail.com Backpatch-through: 9.5
2018-08-30 02:10:02 +02:00
if (pmState == PM_STARTUP || pmState == PM_RECOVERY)
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
{
Allow notifications to bgworkers without database connections. Previously, if one background worker registered another background worker and set bgw_notify_pid while for the second background worker, it would not receive notifications from the postmaster unless, at the time the "parent" was registered, BGWORKER_BACKEND_DATABASE_CONNECTION was set. To fix, instead instead of including only those background workers that requested database connections in the postmater's BackendList, include them all. There doesn't seem to be any reason not do this, and indeed it removes a significant amount of duplicated code. The other option is to make PostmasterMarkPIDForWorkerNotify look at BackgroundWorkerList in addition to BackendList, but that adds more code duplication instead of getting rid of it. Patch by me. Review and testing by Ashutosh Bapat.
2015-09-01 21:30:19 +02:00
SignalSomeChildren(SIGTERM, BACKEND_TYPE_BGWORKER);
pgindent run for 9.6
2016-06-10 00:02:36 +02:00
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
/*
Allow notifications to bgworkers without database connections. Previously, if one background worker registered another background worker and set bgw_notify_pid while for the second background worker, it would not receive notifications from the postmaster unless, at the time the "parent" was registered, BGWORKER_BACKEND_DATABASE_CONNECTION was set. To fix, instead instead of including only those background workers that requested database connections in the postmater's BackendList, include them all. There doesn't seem to be any reason not do this, and indeed it removes a significant amount of duplicated code. The other option is to make PostmasterMarkPIDForWorkerNotify look at BackgroundWorkerList in addition to BackendList, but that adds more code duplication instead of getting rid of it. Patch by me. Review and testing by Ashutosh Bapat.
2015-09-01 21:30:19 +02:00
* Only startup, bgwriter, walreceiver, possibly bgworkers,
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
* and/or checkpointer should be active in this state; we just
* signaled the first four, and we don't want to kill
* checkpointer yet.
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
*/
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
pmState = PM_WAIT_BACKENDS;
}
Add stray "else" that seems to have gone missing.
2010-06-24 18:40:45 +02:00
else if (pmState == PM_RUN ||
pmState == PM_WAIT_BACKUP ||
pmState == PM_WAIT_READONLY ||
pmState == PM_WAIT_BACKENDS ||
pmState == PM_HOT_STANDBY)
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
{
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
ereport(LOG,
(errmsg("aborting any active transactions")));
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/* shut down all backends and workers */
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
SignalSomeChildren(SIGTERM,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
BACKEND_TYPE_NORMAL | BACKEND_TYPE_AUTOVAC |
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
BACKEND_TYPE_BGWORKER);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/* and the autovac launcher too */
if (AutoVacPID != 0)
signal_child(AutoVacPID, SIGTERM);
/* and the walwriter too */
if (WalWriterPID != 0)
signal_child(WalWriterPID, SIGTERM);
pmState = PM_WAIT_BACKENDS;
From: Massimo Dal Zotto <dz@cs.unitn.it> > tprintf.patch > > tprintf.patch > > adds functions and macros which implement a conditional trace package > with the ability to change flags and numeric options of running > backends at runtime. > Options/flags can be specified in the command line and/or read from > the file pg_options in the data directory.
1998-08-25 23:34:10 +02:00
}
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
/*
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
* Now wait for backends to exit. If there are none,
* PostmasterStateMachine will take the next step.
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
*/
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
PostmasterStateMachine();
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
break;
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
case SIGQUIT:
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
/*
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
* Immediate Shutdown:
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
*
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
* abort all children with SIGQUIT, wait for them to exit,
* terminate remaining ones with SIGKILL, then exit without
* attempt to properly shut down the data base system.
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
*/
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
if (Shutdown >= ImmediateShutdown)
break;
Shutdown = ImmediateShutdown;
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(LOG,
Revert "Temporarily make pg_ctl and server shutdown a whole lot chattier." This reverts commit 3971f64843b02e4a55d854156bd53e46a0588e45 and a couple of followon debugging commits; I think we've learned what we can from them.
2016-02-10 22:01:04 +01:00
(errmsg("received immediate shutdown request")));
Change pg_ctl to detect server-ready by watching status in postmaster.pid. Traditionally, "pg_ctl start -w" has waited for the server to become ready to accept connections by attempting a connection once per second. That has the major problem that connection issues (for instance, a kernel packet filter blocking traffic) can't be reliably told apart from server startup issues, and the minor problem that if server startup isn't quick, we accumulate "the database system is starting up" spam in the server log. We've hacked around many of the possible connection issues, but it resulted in ugly and complicated code in pg_ctl.c. In commit c61559ec3, I changed the probe rate to every tenth of a second. That prompted Jeff Janes to complain that the log-spam problem had become much worse. In the ensuing discussion, Andres Freund pointed out that we could dispense with connection attempts altogether if the postmaster were changed to report its status in postmaster.pid, which "pg_ctl start" already relies on being able to read. This patch implements that, teaching postmaster.c to report a status string into the pidfile at the same state-change points already identified as being of interest for systemd status reporting (cf commit 7d17e683f). pg_ctl no longer needs to link with libpq at all; all its functions now depend on reading server files. In support of this, teach AddToDataDirLockFile() to allow addition of postmaster.pid lines in not-necessarily-sequential order. This is needed on Windows where the SHMEM_KEY line will never be written at all. We still have the restriction that we don't want to truncate the pidfile; document the reasons for that a bit better. Also, fix the pg_ctl TAP tests so they'll notice if "start -w" mode is broken --- before, they'd just wait out the sixty seconds until the loop gives up, and then report success anyway. (Yes, I found that out the hard way.) While at it, arrange for pg_ctl to not need to #include miscadmin.h; as a rather low-level backend header, requiring that to be compilable client-side is pretty dubious. This requires moving the #define's associated with the pidfile into a new header file, and moving PG_BACKEND_VERSIONSTR someplace else. For lack of a clearly better "someplace else", I put it into port.h, beside the declaration of find_other_exec(), since most users of that macro are passing the value to find_other_exec(). (initdb still depends on miscadmin.h, but at least pg_ctl and pg_upgrade no longer do.) In passing, fix main.c so that PG_BACKEND_VERSIONSTR actually defines the output of "postgres -V", which remarkably it had never done before. Discussion: https://postgr.es/m/CAMkU=1xJW8e+CTotojOMBd-yzUvD0e_JZu2xHo=MnuZ4__m7Pg@mail.gmail.com
2017-06-28 23:31:24 +02:00
/* Report status */
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STOPPING);
Add support for systemd service notifications Insert sd_notify() calls at server start and stop for integration with systemd. This allows the use of systemd service units of type "notify", which greatly simplifies the systemd configuration. Reviewed-by: Pavel Stěhule <pavel.stehule@gmail.com>
2015-11-17 12:46:17 +01:00
#ifdef USE_SYSTEMD
sd_notify(0, "STOPPING=1");
#endif
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
TerminateChildren(SIGQUIT);
pmState = PM_WAIT_BACKENDS;
/* set stopwatch for them to die */
AbortStartTime = time(NULL);
/*
* Now wait for backends to exit. If there are none,
* PostmasterStateMachine will take the next step.
*/
PostmasterStateMachine();
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
break;
From: Massimo Dal Zotto <dz@cs.unitn.it> > tprintf.patch > > tprintf.patch > > adds functions and macros which implement a conditional trace package > with the ability to change flags and numeric options of running > backends at runtime. > Options/flags can be specified in the command line and/or read from > the file pg_options in the data directory.
1998-08-25 23:34:10 +02:00
}
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
#ifdef WIN32
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
PG_SETMASK(&UnBlockSig);
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
#endif
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
errno = save_errno;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/*
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
* Reaper -- signal handler to cleanup after a child process dies.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
static void
Revert removal of signed, volatile, and signal handler arg type tests.
2000-08-29 11:36:51 +02:00
reaper(SIGNAL_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Ensure that 'errno' is saved and restored by all signal handlers that might change it. Experimentation shows that the signal handler call mechanism does not save/restore errno for you, at least not on Linux or HPUX, so this is definitely a real risk.
2000-12-18 18:33:42 +01:00
int save_errno = errno;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
int pid; /* process id of dead child process */
int exitstatus; /* its exit status */
New pgindent run with fixes suggested by Tom. Patch manually reviewed, initdb/regression tests pass.
2001-11-05 18:46:40 +01:00
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
/*
* We rely on the signal mechanism to have blocked all signals ... except
* on Windows, which lacks sigaction(), so we have to do it manually.
*/
#ifdef WIN32
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
PG_SETMASK(&BlockSig);
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
#endif
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
Change some frequently-reached elog(DEBUG...) calls to ereport(DEBUG...) for speed reasons. (ereport falls out much more quickly when no output is needed than elog does.)
2003-08-12 20:23:21 +02:00
ereport(DEBUG4,
(errmsg_internal("reaping dead processes")));
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
Remove support for using wait3() in place of waitpid(). All Unix-oid platforms that we currently support should have waitpid(), since it's in V2 of the Single Unix Spec. Our git history shows that the wait3 code was added to support NextStep, which we officially dropped support for as of 9.2. So get rid of the configure test, and simplify the macro spaghetti in reaper(). Per suggestion from Fujii Masao.
2012-07-05 20:00:40 +02:00
while ((pid = waitpid(-1, &exitstatus, WNOHANG)) > 0)
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
{
Clean up formatting of child process exit-status reports so that they are correct, consistent, and complete ... motivated by gripe from Oliver Elphick, but I see someone had already made an incomplete stab at this.
2001-11-06 19:02:48 +01:00
/*
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
* Check if this child was a startup process.
Clean up formatting of child process exit-status reports so that they are correct, consistent, and complete ... motivated by gripe from Oliver Elphick, but I see someone had already made an incomplete stab at this.
2001-11-06 19:02:48 +01:00
*/
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (pid == StartupPID)
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
{
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
StartupPID = 0;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
Don't launch new child processes after we've been told to shut down. Once we've received a shutdown signal (SIGINT or SIGTERM), we should not launch any more child processes, even if we get signals requesting such. The normal code path for spawning backends has always understood that, but the postmaster's infrastructure for hot standby and autovacuum didn't get the memo. As reported by Hari Babu in bug #7643, this could lead to failure to shut down at all in some cases, such as when SIGINT is received just before the startup process sends PMSIGNAL_RECOVERY_STARTED: we'd launch a bgwriter and checkpointer, and then those processes would have no idea that they ought to quit. Similarly, launching a new autovacuum worker would result in waiting till it finished before shutting down. Also, switch the order of the code blocks in reaper() that detect startup process crash versus shutdown termination. Once we've sent it a signal, we should not consider that exit(1) is surprising. This is just a cosmetic fix since shutdown occurs correctly anyway, but better not to log a phony complaint about startup process crash. Back-patch to 9.0. Some parts of this might be applicable before that, but given the lack of prior complaints I'm not going to worry too much about older branches.
2012-11-21 21:18:38 +01:00
/*
* Startup process exited in response to a shutdown request (or it
* completed normally regardless of the shutdown request).
*/
if (Shutdown > NoShutdown &&
(EXIT_STATUS_0(exitstatus) || EXIT_STATUS_1(exitstatus)))
{
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
StartupStatus = STARTUP_NOT_RUNNING;
Don't launch new child processes after we've been told to shut down. Once we've received a shutdown signal (SIGINT or SIGTERM), we should not launch any more child processes, even if we get signals requesting such. The normal code path for spawning backends has always understood that, but the postmaster's infrastructure for hot standby and autovacuum didn't get the memo. As reported by Hari Babu in bug #7643, this could lead to failure to shut down at all in some cases, such as when SIGINT is received just before the startup process sends PMSIGNAL_RECOVERY_STARTED: we'd launch a bgwriter and checkpointer, and then those processes would have no idea that they ought to quit. Similarly, launching a new autovacuum worker would result in waiting till it finished before shutting down. Also, switch the order of the code blocks in reaper() that detect startup process crash versus shutdown termination. Once we've sent it a signal, we should not consider that exit(1) is surprising. This is just a cosmetic fix since shutdown occurs correctly anyway, but better not to log a phony complaint about startup process crash. Back-patch to 9.0. Some parts of this might be applicable before that, but given the lack of prior complaints I'm not going to worry too much about older branches.
2012-11-21 21:18:38 +01:00
pmState = PM_WAIT_BACKENDS;
action_at_recovery_target recovery config option action_at_recovery_target = pause | promote | shutdown Petr Jelinek Reviewed by Muhammad Asif Naeem, Fujji Masao and Simon Riggs
2014-11-25 21:13:30 +01:00
/* PostmasterStateMachine logic does the rest */
continue;
}
if (EXIT_STATUS_3(exitstatus))
{
ereport(LOG,
pgindent run for 9.5
2015-05-24 03:35:49 +02:00
(errmsg("shutdown at recovery target")));
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
StartupStatus = STARTUP_NOT_RUNNING;
action_at_recovery_target recovery config option action_at_recovery_target = pause | promote | shutdown Petr Jelinek Reviewed by Muhammad Asif Naeem, Fujji Masao and Simon Riggs
2014-11-25 21:13:30 +01:00
Shutdown = SmartShutdown;
TerminateChildren(SIGTERM);
pmState = PM_WAIT_BACKENDS;
Don't launch new child processes after we've been told to shut down. Once we've received a shutdown signal (SIGINT or SIGTERM), we should not launch any more child processes, even if we get signals requesting such. The normal code path for spawning backends has always understood that, but the postmaster's infrastructure for hot standby and autovacuum didn't get the memo. As reported by Hari Babu in bug #7643, this could lead to failure to shut down at all in some cases, such as when SIGINT is received just before the startup process sends PMSIGNAL_RECOVERY_STARTED: we'd launch a bgwriter and checkpointer, and then those processes would have no idea that they ought to quit. Similarly, launching a new autovacuum worker would result in waiting till it finished before shutting down. Also, switch the order of the code blocks in reaper() that detect startup process crash versus shutdown termination. Once we've sent it a signal, we should not consider that exit(1) is surprising. This is just a cosmetic fix since shutdown occurs correctly anyway, but better not to log a phony complaint about startup process crash. Back-patch to 9.0. Some parts of this might be applicable before that, but given the lack of prior complaints I'm not going to worry too much about older branches.
2012-11-21 21:18:38 +01:00
/* PostmasterStateMachine logic does the rest */
continue;
}
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
/*
* Unexpected exit of startup process (including FATAL exit)
Cleanup and code review for the patch that made bgwriter active during archive recovery. Invent a separate state variable and inquiry function for XLogInsertAllowed() to clarify some tests and make the management of writing the end-of-recovery checkpoint less klugy. Fix several places that were incorrectly testing InRecovery when they should be looking at RecoveryInProgress or XLogInsertAllowed (because they will now be executed in the bgwriter not startup process). Clarify handling of bad LSNs passed to XLogFlush during recovery. Use a spinlock for setting/testing SharedRecoveryInProgress. Improve quite a lot of comments. Heikki and Tom
2009-06-26 22:29:04 +02:00
* during PM_STARTUP is treated as catastrophic. There are no
* other processes running yet, so we can just exit.
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
*/
Fix postmaster state machine to handle dead_end child crashes better. A report from Alvaro Herrera shows that if we're in PM_STARTUP state, and we spawn a dead_end child to reject some incoming connection request, and that child dies with an unexpected exit code, the postmaster does not respond well. We correctly send SIGQUIT to the startup process, but then: * if the startup process exits with nonzero exit code, as expected, we thought that that indicated a crash and aborted startup. * if the startup process exits with zero exit code, which is possible due to the inherent race condition, we'd advance to PM_RUN state which is fine --- but the code forgot that AbortStartTime would be nonzero in this situation. We'd either die on the Asserts saying that it was zero, or perhaps misbehave later on. (A quick look suggests that the only misbehavior might be busy-waiting due to DetermineSleepTime doing the wrong thing.) To fix the first point, adjust the state-machine logic to recognize that a nonzero exit code is expected after sending SIGQUIT, and have it transition to a state where we can restart the startup process. To fix the second point, change the Asserts to clear the variable rather than just claiming it should be clear already. Perhaps we could improve this further by not treating a crash of a dead_end child as a reason for panic'ing the database. However, since those child processes are connected to shared memory, that seems a bit risky. There are few good reasons for a dead_end child to report failure anyway (the cause of this in Alvaro's report is quite unclear). On balance, therefore, a minimal fix seems best. This is an oversight in commit 45811be94. While that was back-patched, I'm hesitant to back-patch this change. The lack of reasons for a dead_end child to fail suggests that the case should be very rare in the field, which squares with the lack of reports; so it seems like this might not be worth the risk of introducing new issues. In any case we can let it bake awhile in HEAD before considering a back-patch. Discussion: https://postgr.es/m/20190615160950.GA31378@alvherre.pgsql
2019-08-26 21:59:44 +02:00
if (pmState == PM_STARTUP &&
StartupStatus != STARTUP_SIGNALED &&
!EXIT_STATUS_0(exitstatus))
More startup/shutdown log messages.
1999-10-08 04:16:22 +02:00
{
Use _() macro consistently rather than gettext(). Add translation macros around strings that were missing them.
2005-02-22 05:43:23 +01:00
LogChildExit(LOG, _("startup process"),
Restructure child-exit logging messages for easier translation, per suggestion from Peter.
2001-11-11 00:06:12 +01:00
pid, exitstatus);
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(LOG,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errmsg("aborting startup due to startup process failure")));
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
ExitPostmaster(1);
More startup/shutdown log messages.
1999-10-08 04:16:22 +02:00
}
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
If a shutdown request comes in while we're still starting up, don't service it until after we execute SetThisStartUpID(). Else shutdown process will write the wrong SUI into the shutdown checkpoint, which seems likely to be trouble --- although I've not quite figured out how significant it really is.
2003-02-23 05:48:19 +01:00
/*
Fix postmaster to attempt restart after a hot-standby crash. The postmaster was coded to treat any unexpected exit of the startup process (i.e., the WAL replay process) as a catastrophic crash, and not try to restart it. This was OK so long as the startup process could not have any sibling postmaster children. However, if a hot-standby backend crashes, we SIGQUIT the startup process along with everything else, and the resulting exit is hardly "unexpected". Treating it as such meant we failed to restart a standby server after any child crash at all, not only a crash of the WAL replay process as intended. Adjust that. Back-patch to 9.0 where hot standby was introduced.
2012-02-06 21:29:26 +01:00
* After PM_STARTUP, any unexpected exit (including FATAL exit) of
* the startup process is catastrophic, so kill other children,
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
* and set StartupStatus so we don't try to reinitialize after
* they're gone. Exception: if StartupStatus is STARTUP_SIGNALED,
* then we previously sent the startup process a SIGQUIT; so
Fix postmaster to attempt restart after a hot-standby crash. The postmaster was coded to treat any unexpected exit of the startup process (i.e., the WAL replay process) as a catastrophic crash, and not try to restart it. This was OK so long as the startup process could not have any sibling postmaster children. However, if a hot-standby backend crashes, we SIGQUIT the startup process along with everything else, and the resulting exit is hardly "unexpected". Treating it as such meant we failed to restart a standby server after any child crash at all, not only a crash of the WAL replay process as intended. Adjust that. Back-patch to 9.0 where hot standby was introduced.
2012-02-06 21:29:26 +01:00
* that's probably the reason it died, and we do want to try to
* restart in that case.
Fix postmaster state machine to handle dead_end child crashes better. A report from Alvaro Herrera shows that if we're in PM_STARTUP state, and we spawn a dead_end child to reject some incoming connection request, and that child dies with an unexpected exit code, the postmaster does not respond well. We correctly send SIGQUIT to the startup process, but then: * if the startup process exits with nonzero exit code, as expected, we thought that that indicated a crash and aborted startup. * if the startup process exits with zero exit code, which is possible due to the inherent race condition, we'd advance to PM_RUN state which is fine --- but the code forgot that AbortStartTime would be nonzero in this situation. We'd either die on the Asserts saying that it was zero, or perhaps misbehave later on. (A quick look suggests that the only misbehavior might be busy-waiting due to DetermineSleepTime doing the wrong thing.) To fix the first point, adjust the state-machine logic to recognize that a nonzero exit code is expected after sending SIGQUIT, and have it transition to a state where we can restart the startup process. To fix the second point, change the Asserts to clear the variable rather than just claiming it should be clear already. Perhaps we could improve this further by not treating a crash of a dead_end child as a reason for panic'ing the database. However, since those child processes are connected to shared memory, that seems a bit risky. There are few good reasons for a dead_end child to report failure anyway (the cause of this in Alvaro's report is quite unclear). On balance, therefore, a minimal fix seems best. This is an oversight in commit 45811be94. While that was back-patched, I'm hesitant to back-patch this change. The lack of reasons for a dead_end child to fail suggests that the case should be very rare in the field, which squares with the lack of reports; so it seems like this might not be worth the risk of introducing new issues. In any case we can let it bake awhile in HEAD before considering a back-patch. Discussion: https://postgr.es/m/20190615160950.GA31378@alvherre.pgsql
2019-08-26 21:59:44 +02:00
*
* This stanza also handles the case where we sent a SIGQUIT
* during PM_STARTUP due to some dead_end child crashing: in that
* situation, if the startup process dies on the SIGQUIT, we need
* to transition to PM_WAIT_BACKENDS state which will allow
* PostmasterStateMachine to restart the startup process. (On the
* other hand, the startup process might complete normally, if we
* were too late with the SIGQUIT. In that case we'll fall
* through and commence normal operations.)
If a shutdown request comes in while we're still starting up, don't service it until after we execute SetThisStartUpID(). Else shutdown process will write the wrong SUI into the shutdown checkpoint, which seems likely to be trouble --- although I've not quite figured out how significant it really is.
2003-02-23 05:48:19 +01:00
*/
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
if (!EXIT_STATUS_0(exitstatus))
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
{
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
if (StartupStatus == STARTUP_SIGNALED)
Fix postmaster state machine to handle dead_end child crashes better. A report from Alvaro Herrera shows that if we're in PM_STARTUP state, and we spawn a dead_end child to reject some incoming connection request, and that child dies with an unexpected exit code, the postmaster does not respond well. We correctly send SIGQUIT to the startup process, but then: * if the startup process exits with nonzero exit code, as expected, we thought that that indicated a crash and aborted startup. * if the startup process exits with zero exit code, which is possible due to the inherent race condition, we'd advance to PM_RUN state which is fine --- but the code forgot that AbortStartTime would be nonzero in this situation. We'd either die on the Asserts saying that it was zero, or perhaps misbehave later on. (A quick look suggests that the only misbehavior might be busy-waiting due to DetermineSleepTime doing the wrong thing.) To fix the first point, adjust the state-machine logic to recognize that a nonzero exit code is expected after sending SIGQUIT, and have it transition to a state where we can restart the startup process. To fix the second point, change the Asserts to clear the variable rather than just claiming it should be clear already. Perhaps we could improve this further by not treating a crash of a dead_end child as a reason for panic'ing the database. However, since those child processes are connected to shared memory, that seems a bit risky. There are few good reasons for a dead_end child to report failure anyway (the cause of this in Alvaro's report is quite unclear). On balance, therefore, a minimal fix seems best. This is an oversight in commit 45811be94. While that was back-patched, I'm hesitant to back-patch this change. The lack of reasons for a dead_end child to fail suggests that the case should be very rare in the field, which squares with the lack of reports; so it seems like this might not be worth the risk of introducing new issues. In any case we can let it bake awhile in HEAD before considering a back-patch. Discussion: https://postgr.es/m/20190615160950.GA31378@alvherre.pgsql
2019-08-26 21:59:44 +02:00
{
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
StartupStatus = STARTUP_NOT_RUNNING;
Fix postmaster state machine to handle dead_end child crashes better. A report from Alvaro Herrera shows that if we're in PM_STARTUP state, and we spawn a dead_end child to reject some incoming connection request, and that child dies with an unexpected exit code, the postmaster does not respond well. We correctly send SIGQUIT to the startup process, but then: * if the startup process exits with nonzero exit code, as expected, we thought that that indicated a crash and aborted startup. * if the startup process exits with zero exit code, which is possible due to the inherent race condition, we'd advance to PM_RUN state which is fine --- but the code forgot that AbortStartTime would be nonzero in this situation. We'd either die on the Asserts saying that it was zero, or perhaps misbehave later on. (A quick look suggests that the only misbehavior might be busy-waiting due to DetermineSleepTime doing the wrong thing.) To fix the first point, adjust the state-machine logic to recognize that a nonzero exit code is expected after sending SIGQUIT, and have it transition to a state where we can restart the startup process. To fix the second point, change the Asserts to clear the variable rather than just claiming it should be clear already. Perhaps we could improve this further by not treating a crash of a dead_end child as a reason for panic'ing the database. However, since those child processes are connected to shared memory, that seems a bit risky. There are few good reasons for a dead_end child to report failure anyway (the cause of this in Alvaro's report is quite unclear). On balance, therefore, a minimal fix seems best. This is an oversight in commit 45811be94. While that was back-patched, I'm hesitant to back-patch this change. The lack of reasons for a dead_end child to fail suggests that the case should be very rare in the field, which squares with the lack of reports; so it seems like this might not be worth the risk of introducing new issues. In any case we can let it bake awhile in HEAD before considering a back-patch. Discussion: https://postgr.es/m/20190615160950.GA31378@alvherre.pgsql
2019-08-26 21:59:44 +02:00
if (pmState == PM_STARTUP)
pmState = PM_WAIT_BACKENDS;
}
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
else
StartupStatus = STARTUP_CRASHED;
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
HandleChildCrash(pid, exitstatus,
_("startup process"));
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
continue;
}
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/*
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
* Startup succeeded, commence normal operations
If a shutdown request comes in while we're still starting up, don't service it until after we execute SetThisStartUpID(). Else shutdown process will write the wrong SUI into the shutdown checkpoint, which seems likely to be trouble --- although I've not quite figured out how significant it really is.
2003-02-23 05:48:19 +01:00
*/
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
StartupStatus = STARTUP_NOT_RUNNING;
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
FatalError = false;
Fix postmaster state machine to handle dead_end child crashes better. A report from Alvaro Herrera shows that if we're in PM_STARTUP state, and we spawn a dead_end child to reject some incoming connection request, and that child dies with an unexpected exit code, the postmaster does not respond well. We correctly send SIGQUIT to the startup process, but then: * if the startup process exits with nonzero exit code, as expected, we thought that that indicated a crash and aborted startup. * if the startup process exits with zero exit code, which is possible due to the inherent race condition, we'd advance to PM_RUN state which is fine --- but the code forgot that AbortStartTime would be nonzero in this situation. We'd either die on the Asserts saying that it was zero, or perhaps misbehave later on. (A quick look suggests that the only misbehavior might be busy-waiting due to DetermineSleepTime doing the wrong thing.) To fix the first point, adjust the state-machine logic to recognize that a nonzero exit code is expected after sending SIGQUIT, and have it transition to a state where we can restart the startup process. To fix the second point, change the Asserts to clear the variable rather than just claiming it should be clear already. Perhaps we could improve this further by not treating a crash of a dead_end child as a reason for panic'ing the database. However, since those child processes are connected to shared memory, that seems a bit risky. There are few good reasons for a dead_end child to report failure anyway (the cause of this in Alvaro's report is quite unclear). On balance, therefore, a minimal fix seems best. This is an oversight in commit 45811be94. While that was back-patched, I'm hesitant to back-patch this change. The lack of reasons for a dead_end child to fail suggests that the case should be very rare in the field, which squares with the lack of reports; so it seems like this might not be worth the risk of introducing new issues. In any case we can let it bake awhile in HEAD before considering a back-patch. Discussion: https://postgr.es/m/20190615160950.GA31378@alvherre.pgsql
2019-08-26 21:59:44 +02:00
AbortStartTime = 0;
More fixes for shutdown during recovery. 1. If we receive a fast shutdown request while in the PM_STARTUP state, process it just as we would in PM_RECOVERY, PM_HOT_STANDBY, or PM_RUN. Without this change, an early fast shutdown followed by Hot Standby causes the database to get stuck in a state where a shutdown is pending (so no new connections are allowed) but the shutdown request is never processed unless we end Hot Standby and enter normal running. 2. Avoid removing the backup label file when a smart or fast shutdown occurs during recovery. It makes sense to do this once we've reached normal running, since we must be taking a backup which now won't be valid. But during recovery we must be recovering from a previously taken backup, and any backup label file is needed to restart recovery from the right place. Fujii Masao and Robert Haas
2010-05-26 14:32:41 +02:00
ReachedNormalRunning = true;
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
pmState = PM_RUN;
/*
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
* Crank up the background tasks, if we didn't do that already
Cleanup and code review for the patch that made bgwriter active during archive recovery. Invent a separate state variable and inquiry function for XLogInsertAllowed() to clarify some tests and make the management of writing the end-of-recovery checkpoint less klugy. Fix several places that were incorrectly testing InRecovery when they should be looking at RecoveryInProgress or XLogInsertAllowed (because they will now be executed in the bgwriter not startup process). Clarify handling of bad LSNs passed to XLogFlush during recovery. Use a spinlock for setting/testing SharedRecoveryInProgress. Improve quite a lot of comments. Heikki and Tom
2009-06-26 22:29:04 +02:00
* when we entered consistent recovery state. It doesn't matter
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
* if this fails, we'll just try again later.
*/
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
if (CheckpointerPID == 0)
CheckpointerPID = StartCheckpointer();
Checkpointer starts before bgwriter to avoid missing fsync requests. Noted while testing Hot Standby startup.
2012-06-01 09:25:17 +02:00
if (BgWriterPID == 0)
BgWriterPID = StartBackgroundWriter();
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
if (WalWriterPID == 0)
WalWriterPID = StartWalWriter();
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
/*
* Likewise, start other special children as needed. In a restart
* situation, some of them may be alive already.
*/
Add postmaster/postgres undocumented -b option for binary upgrades. This option turns off autovacuum, prevents non-super-user connections, and enables oid setting hooks in the backend. The code continues to use the old autoavacuum disable settings for servers with earlier catalog versions. This includes a catalog version bump to identify servers that support the -b option.
2011-04-25 18:00:21 +02:00
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
AutoVacPID = StartAutoVacLauncher();
Make postmaster restart archiver soon after it dies, even during recovery. After the archiver dies, postmaster tries to start a new one immediately. But previously this could happen only while server was running normally even though archiving was enabled always (i.e., archive_mode was set to always). So the archiver running during recovery could not restart soon after it died. This is an oversight in commit ffd3774. This commit changes reaper(), postmaster's signal handler to cleanup after a child process dies, so that it tries to a new archiver even during recovery if necessary. Patch by me. Review by Alvaro Herrera.
2015-06-12 16:11:51 +02:00
if (PgArchStartupAllowed() && PgArchPID == 0)
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
PgArchPID = pgarch_start();
if (PgStatPID == 0)
PgStatPID = pgstat_start();
Restart bgworkers immediately after a crash-and-restart cycle. Just as we would start bgworkers immediately after an initial startup of the server, we should restart them immediately when reinitializing. Petr Jelinek and Robert Haas
2014-05-07 22:04:47 +02:00
/* workers may be scheduled to start now */
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
maybe_start_bgworkers();
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
/* at this point we are really open for business */
ereport(LOG,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errmsg("database system is ready to accept connections")));
Put back a "continue" that went missing in the changes to start background writer in WAL recovery.
2009-02-25 12:07:43 +01:00
Change pg_ctl to detect server-ready by watching status in postmaster.pid. Traditionally, "pg_ctl start -w" has waited for the server to become ready to accept connections by attempting a connection once per second. That has the major problem that connection issues (for instance, a kernel packet filter blocking traffic) can't be reliably told apart from server startup issues, and the minor problem that if server startup isn't quick, we accumulate "the database system is starting up" spam in the server log. We've hacked around many of the possible connection issues, but it resulted in ugly and complicated code in pg_ctl.c. In commit c61559ec3, I changed the probe rate to every tenth of a second. That prompted Jeff Janes to complain that the log-spam problem had become much worse. In the ensuing discussion, Andres Freund pointed out that we could dispense with connection attempts altogether if the postmaster were changed to report its status in postmaster.pid, which "pg_ctl start" already relies on being able to read. This patch implements that, teaching postmaster.c to report a status string into the pidfile at the same state-change points already identified as being of interest for systemd status reporting (cf commit 7d17e683f). pg_ctl no longer needs to link with libpq at all; all its functions now depend on reading server files. In support of this, teach AddToDataDirLockFile() to allow addition of postmaster.pid lines in not-necessarily-sequential order. This is needed on Windows where the SHMEM_KEY line will never be written at all. We still have the restriction that we don't want to truncate the pidfile; document the reasons for that a bit better. Also, fix the pg_ctl TAP tests so they'll notice if "start -w" mode is broken --- before, they'd just wait out the sixty seconds until the loop gives up, and then report success anyway. (Yes, I found that out the hard way.) While at it, arrange for pg_ctl to not need to #include miscadmin.h; as a rather low-level backend header, requiring that to be compilable client-side is pretty dubious. This requires moving the #define's associated with the pidfile into a new header file, and moving PG_BACKEND_VERSIONSTR someplace else. For lack of a clearly better "someplace else", I put it into port.h, beside the declaration of find_other_exec(), since most users of that macro are passing the value to find_other_exec(). (initdb still depends on miscadmin.h, but at least pg_ctl and pg_upgrade no longer do.) In passing, fix main.c so that PG_BACKEND_VERSIONSTR actually defines the output of "postgres -V", which remarkably it had never done before. Discussion: https://postgr.es/m/CAMkU=1xJW8e+CTotojOMBd-yzUvD0e_JZu2xHo=MnuZ4__m7Pg@mail.gmail.com
2017-06-28 23:31:24 +02:00
/* Report status */
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_READY);
Add support for systemd service notifications Insert sd_notify() calls at server start and stop for integration with systemd. This allows the use of systemd service units of type "notify", which greatly simplifies the systemd configuration. Reviewed-by: Pavel Stěhule <pavel.stehule@gmail.com>
2015-11-17 12:46:17 +01:00
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
Put back a "continue" that went missing in the changes to start background writer in WAL recovery.
2009-02-25 12:07:43 +01:00
continue;
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
}
/*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* Was it the bgwriter? Normal exit can be ignored; we'll start a new
* one at the next iteration of the postmaster's main loop, if
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
* necessary. Any other exit condition is treated as a crash.
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
*/
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (pid == BgWriterPID)
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
{
Arrange to explicitly stop the pgstat processes at the same time we begin the shutdown checkpoint; there isn't anything left for them to do, so we may as well ensure that they shut down sooner rather than later. Per discussion.
2004-06-14 20:08:19 +02:00
BgWriterPID = 0;
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
if (!EXIT_STATUS_0(exitstatus))
HandleChildCrash(pid, exitstatus,
_("background writer process"));
continue;
}
/*
* Was it the checkpointer?
*/
if (pid == CheckpointerPID)
{
CheckpointerPID = 0;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (EXIT_STATUS_0(exitstatus) && pmState == PM_SHUTDOWN)
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
{
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
/*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* OK, we saw normal exit of the checkpointer after it's been
* told to shut down. We expect that it wrote a shutdown
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* checkpoint. (If for some reason it didn't, recovery will
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
* occur on next postmaster start.)
Fix oversight: in case where SIGTERM is received while there are live backends, the archiver and stats processes never got sent a kill signal. They'd eventually exit on their own, but not for awhile, which is a bit annoying when you are trying to replace the executable file on a platform that doesn't allow removal of busy executables. Also, tweak main loop logic so that we will perform the background tasks after select() returns EINTR.
2004-07-27 03:46:03 +02:00
*
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
* At this point we should have no normal backend children
* left (else we'd not be in PM_SHUTDOWN state) but we might
* have dead_end children to wait for.
*
* If we have an archiver subprocess, tell it to do a last
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
* archive cycle and quit. Likewise, if we have walsender
* processes, tell them to send any remaining WAL and quit.
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
*/
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
Assert(Shutdown > NoShutdown);
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
/* Waken archiver for the last time */
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
if (PgArchPID != 0)
signal_child(PgArchPID, SIGUSR2);
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
/*
* Waken walsenders for the last time. No regular backends
* should be around anymore.
*/
Revert "Prevent panic during shutdown checkpoint" This reverts commit 086221cf6b1727c2baed4703c582f657b7c5350e, which was made to master only. The approach implemented in the above commit has some issues. While those could easily be fixed incrementally, doing so would make backpatching considerably harder, so instead first revert this patch. Discussion: https://postgr.es/m/20170602002912.tqlwn4gymzlxpvs2@alap3.anarazel.de
2017-06-06 03:53:41 +02:00
SignalChildren(SIGUSR2);
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
pmState = PM_SHUTDOWN_2;
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
/*
* We can also shut down the stats collector now; there's
* nothing left for it to do.
*/
if (PgStatPID != 0)
signal_child(PgStatPID, SIGQUIT);
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
}
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
else
Minor adjustments to make failures in startup/shutdown behave more cleanly. StartupXLOG and ShutdownXLOG no longer need to be critical sections, because in all contexts where they are invoked, elog(ERROR) would be translated to elog(FATAL) anyway. (One change in bgwriter.c is needed to make this true: set ExitOnAnyError before trying to exit. This is a good fix anyway since the existing code would have gone into an infinite loop on elog(ERROR) during shutdown.) That avoids a misleading report of PANIC during semi-orderly failures. Modify the postmaster to include the startup process in the set of processes that get SIGTERM when a fast shutdown is requested, and also fix it to not try to restart the bgwriter if the bgwriter fails while trying to write the shutdown checkpoint. Net result is that "pg_ctl stop -m fast" does something reasonable for a system in warm standby mode, and so should Unix system shutdown (ie, universal SIGTERM). Per gripe from Stephen Harris and some corner-case testing of my own.
2006-11-30 19:29:12 +01:00
{
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* Any unexpected exit of the checkpointer (including FATAL
* exit) is treated as a crash.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
HandleChildCrash(pid, exitstatus,
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
_("checkpointer process"));
Minor adjustments to make failures in startup/shutdown behave more cleanly. StartupXLOG and ShutdownXLOG no longer need to be critical sections, because in all contexts where they are invoked, elog(ERROR) would be translated to elog(FATAL) anyway. (One change in bgwriter.c is needed to make this true: set ExitOnAnyError before trying to exit. This is a good fix anyway since the existing code would have gone into an infinite loop on elog(ERROR) during shutdown.) That avoids a misleading report of PANIC during semi-orderly failures. Modify the postmaster to include the startup process in the set of processes that get SIGTERM when a fast shutdown is requested, and also fix it to not try to restart the bgwriter if the bgwriter fails while trying to write the shutdown checkpoint. Net result is that "pg_ctl stop -m fast" does something reasonable for a system in warm standby mode, and so should Unix system shutdown (ie, universal SIGTERM). Per gripe from Stephen Harris and some corner-case testing of my own.
2006-11-30 19:29:12 +01:00
}
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
continue;
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
}
Statistical system views (yet without the config stuff, but it's hard to keep such massive changes in sync with the tree so I need to get it in and work from there now). Jan
2001-06-22 21:16:24 +02:00
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* Was it the wal writer? Normal exit can be ignored; we'll start a
* new one at the next iteration of the postmaster's main loop, if
* necessary. Any other exit condition is treated as a crash.
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
*/
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (pid == WalWriterPID)
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
{
WalWriterPID = 0;
if (!EXIT_STATUS_0(exitstatus))
HandleChildCrash(pid, exitstatus,
Wording changes
2007-11-08 15:47:51 +01:00
_("WAL writer process"));
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
continue;
}
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
/*
* Was it the wal receiver? If exit status is zero (normal) or one
* (FATAL exit), we assume everything is all right just like normal
Don't lose walreceiver start requests due to race condition in postmaster. When a walreceiver dies, the startup process will notice that and send a PMSIGNAL_START_WALRECEIVER signal to the postmaster, asking for a new walreceiver to be launched. There's a race condition, which at least in HEAD is very easy to hit, whereby the postmaster might see that signal before it processes the SIGCHLD from the walreceiver process. In that situation, sigusr1_handler() just dropped the start request on the floor, reasoning that it must be redundant. Eventually, after 10 seconds (WALRCV_STARTUP_TIMEOUT), the startup process would make a fresh request --- but that's a long time if the connection could have been re-established almost immediately. Fix it by setting a state flag inside the postmaster that we won't clear until we do launch a walreceiver. In cases where that results in an extra walreceiver launch, it's up to the walreceiver to realize it's unwanted and go away --- but we have, and need, that logic anyway for the opposite race case. I came across this through investigating unexpected delays in the src/test/recovery TAP tests: it manifests there in test cases where a master server is stopped and restarted while leaving streaming slaves active. This logic has been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/21344.1498494720@sss.pgh.pa.us
2017-06-26 23:31:56 +02:00
* backends. (If we need a new wal receiver, we'll start one at the
* next iteration of the postmaster's main loop.)
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
*/
if (pid == WalReceiverPID)
{
WalReceiverPID = 0;
if (!EXIT_STATUS_0(exitstatus) && !EXIT_STATUS_1(exitstatus))
HandleChildCrash(pid, exitstatus,
_("WAL receiver process"));
continue;
}
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* Was it the autovacuum launcher? Normal exit can be ignored; we'll
* start a new one at the next iteration of the postmaster's main
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* loop, if necessary. Any other exit condition is treated as a
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* crash.
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
*/
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (pid == AutoVacPID)
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
{
AutoVacPID = 0;
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
if (!EXIT_STATUS_0(exitstatus))
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
HandleChildCrash(pid, exitstatus,
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
_("autovacuum launcher process"));
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
continue;
}
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Was it the archiver? If so, just try to start a new one; no need
* to force reset of the rest of the system. (If fail, we'll try
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* again in future cycles of the main loop.). Unless we were waiting
Fix some typos Josh Kupershmidt
2012-04-22 18:23:47 +02:00
* for it to shut down; don't restart it in that case, and
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* PostmasterStateMachine() will advance to the next shutdown step.
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
*/
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (pid == PgArchPID)
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
{
PgArchPID = 0;
Adjust elog.c so that elog(FATAL) exits (including cases where ERROR is promoted to FATAL) end in exit(1) not exit(0). Then change the postmaster to allow exit(1) without a system-wide panic, but not for the startup subprocess or the bgwriter. There were a couple of places that were using exit(1) to deliberately force a system-wide panic; adjust these to be exit(2) instead. This fixes the problem noted back in July that if the startup process exits with elog(ERROR), the postmaster would think everything is hunky-dory and proceed to start up. Alternative solutions such as trying to run the entire startup process as a critical section seem less clean, primarily because of the fact that a fair amount of startup code is shared by all postmaster children in the EXEC_BACKEND case. We'd need an ugly special case somewhere near the head of main.c to make it work if it's the child process's responsibility to determine what happens; and what's the point when the postmaster already treats different children differently?
2006-11-21 01:49:55 +01:00
if (!EXIT_STATUS_0(exitstatus))
Use _() macro consistently rather than gettext(). Add translation macros around strings that were missing them.
2005-02-22 05:43:23 +01:00
LogChildExit(LOG, _("archiver process"),
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
pid, exitstatus);
Make postmaster restart archiver soon after it dies, even during recovery. After the archiver dies, postmaster tries to start a new one immediately. But previously this could happen only while server was running normally even though archiving was enabled always (i.e., archive_mode was set to always). So the archiver running during recovery could not restart soon after it died. This is an oversight in commit ffd3774. This commit changes reaper(), postmaster's signal handler to cleanup after a child process dies, so that it tries to a new archiver even during recovery if necessary. Patch by me. Review by Alvaro Herrera.
2015-06-12 16:11:51 +02:00
if (PgArchStartupAllowed())
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
PgArchPID = pgarch_start();
continue;
}
Arrange to explicitly stop the pgstat processes at the same time we begin the shutdown checkpoint; there isn't anything left for them to do, so we may as well ensure that they shut down sooner rather than later. Per discussion.
2004-06-14 20:08:19 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Was it the statistics collector? If so, just try to start a new
* one; no need to force reset of the rest of the system. (If fail,
* we'll try again in future cycles of the main loop.)
Arrange to explicitly stop the pgstat processes at the same time we begin the shutdown checkpoint; there isn't anything left for them to do, so we may as well ensure that they shut down sooner rather than later. Per discussion.
2004-06-14 20:08:19 +02:00
*/
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (pid == PgStatPID)
Arrange to explicitly stop the pgstat processes at the same time we begin the shutdown checkpoint; there isn't anything left for them to do, so we may as well ensure that they shut down sooner rather than later. Per discussion.
2004-06-14 20:08:19 +02:00
{
PgStatPID = 0;
Adjust elog.c so that elog(FATAL) exits (including cases where ERROR is promoted to FATAL) end in exit(1) not exit(0). Then change the postmaster to allow exit(1) without a system-wide panic, but not for the startup subprocess or the bgwriter. There were a couple of places that were using exit(1) to deliberately force a system-wide panic; adjust these to be exit(2) instead. This fixes the problem noted back in July that if the startup process exits with elog(ERROR), the postmaster would think everything is hunky-dory and proceed to start up. Alternative solutions such as trying to run the entire startup process as a critical section seem less clean, primarily because of the fact that a fair amount of startup code is shared by all postmaster children in the EXEC_BACKEND case. We'd need an ugly special case somewhere near the head of main.c to make it work if it's the child process's responsibility to determine what happens; and what's the point when the postmaster already treats different children differently?
2006-11-21 01:49:55 +01:00
if (!EXIT_STATUS_0(exitstatus))
Use _() macro consistently rather than gettext(). Add translation macros around strings that were missing them.
2005-02-22 05:43:23 +01:00
LogChildExit(LOG, _("statistics collector process"),
Arrange to explicitly stop the pgstat processes at the same time we begin the shutdown checkpoint; there isn't anything left for them to do, so we may as well ensure that they shut down sooner rather than later. Per discussion.
2004-06-14 20:08:19 +02:00
pid, exitstatus);
If the stats collector dies during Hot Standby, restart it. This bug exists as far back as 9.0, when Hot Standby was introduced, so back-patch to all supported branches. Report and patch by Takayuki Tsunakawa, reviewed by Michael Paquier and Kuntal Ghosh.
2016-10-27 20:27:40 +02:00
if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
Arrange to explicitly stop the pgstat processes at the same time we begin the shutdown checkpoint; there isn't anything left for them to do, so we may as well ensure that they shut down sooner rather than later. Per discussion.
2004-06-14 20:08:19 +02:00
PgStatPID = pgstat_start();
continue;
}
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/* Was it the system logger? If so, try to start a new one */
if (pid == SysLoggerPID)
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
{
SysLoggerPID = 0;
/* for safety's sake, launch new logger *first* */
SysLoggerPID = SysLogger_Start();
Adjust elog.c so that elog(FATAL) exits (including cases where ERROR is promoted to FATAL) end in exit(1) not exit(0). Then change the postmaster to allow exit(1) without a system-wide panic, but not for the startup subprocess or the bgwriter. There were a couple of places that were using exit(1) to deliberately force a system-wide panic; adjust these to be exit(2) instead. This fixes the problem noted back in July that if the startup process exits with elog(ERROR), the postmaster would think everything is hunky-dory and proceed to start up. Alternative solutions such as trying to run the entire startup process as a critical section seem less clean, primarily because of the fact that a fair amount of startup code is shared by all postmaster children in the EXEC_BACKEND case. We'd need an ugly special case somewhere near the head of main.c to make it work if it's the child process's responsibility to determine what happens; and what's the point when the postmaster already treats different children differently?
2006-11-21 01:49:55 +01:00
if (!EXIT_STATUS_0(exitstatus))
Use _() macro consistently rather than gettext(). Add translation macros around strings that were missing them.
2005-02-22 05:43:23 +01:00
LogChildExit(LOG, _("system logger process"),
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
pid, exitstatus);
continue;
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/* Was it one of our background workers? */
if (CleanupBackgroundWorker(pid, exitstatus))
{
/* have it be restarted */
HaveCrashedWorker = true;
continue;
}
In child-crashed log messages, identify a checkpoint process as such instead of calling it a 'server process'.
2003-05-10 20:15:42 +02:00
/*
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
* Else do standard backend child cleanup.
In child-crashed log messages, identify a checkpoint process as such instead of calling it a 'server process'.
2003-05-10 20:15:42 +02:00
*/
Tweak postmaster code to avoid double reporting when bgwriter crashes.
2004-08-04 22:09:47 +02:00
CleanupBackend(pid, exitstatus);
pgindent run.
2003-08-04 02:43:34 +02:00
} /* loop over pending child-death reports */
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/*
* After cleaning out the SIGCHLD queue, see if we have any state changes
* or actions to make.
*/
PostmasterStateMachine();
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/* Done with signal handler */
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
#ifdef WIN32
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
PG_SETMASK(&UnBlockSig);
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
#endif
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
Ensure that 'errno' is saved and restored by all signal handlers that might change it. Experimentation shows that the signal handler call mechanism does not save/restore errno for you, at least not on Linux or HPUX, so this is definitely a real risk.
2000-12-18 18:33:42 +01:00
errno = save_errno;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
* Scan the bgworkers list and see if the given PID (which has just stopped
* or crashed) is in it. Handle its shutdown if so, and return true. If not a
* bgworker, return false.
*
* This is heavily based on CleanupBackend. One important difference is that
* we don't know yet that the dying process is a bgworker, so we must be silent
* until we're sure it is.
*/
static bool
CleanupBackgroundWorker(int pid,
int exitstatus) /* child's exit status */
{
char namebuf[MAXPGPATH];
Post-PG 10 beta1 pgindent run perltidy run not included.
2017-05-17 22:31:56 +02:00
slist_mutable_iter iter;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
Notify bgworker registrant after freeing worker slot. Tom Lane observed buildfarm failures caused by the select_parallel regression test trying to launch new parallel queries before the worker slots used by the previous ones were freed. Try to fix this by having the postmaster free the worker slots before it sends the SIGUSR1 notifications to the registering process. This doesn't completely eliminate the possibility that the user backend might (correctly) observe the worker as dead before the slot is free, but I believe it should make the window significantly narrower. Patch by me, per complaint from Tom Lane. Reviewed by Amit Kapila. Discussion: http://postgr.es/m/30673.1487310734@sss.pgh.pa.us
2017-03-03 04:44:49 +01:00
slist_foreach_modify(iter, &BackgroundWorkerList)
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
{
RegisteredBgWorker *rw;
rw = slist_container(RegisteredBgWorker, rw_lnode, iter.cur);
if (rw->rw_pid != pid)
continue;
#ifdef WIN32
/* see CleanupBackend */
if (exitstatus == ERROR_WAIT_NO_CHILDREN)
exitstatus = 0;
#endif
Add background worker type Add bgw_type field to background worker structure. It is intended to be set to the same value for all workers of the same type, so they can be grouped in pg_stat_activity, for example. The backend_type column in pg_stat_activity now shows bgw_type for a background worker. The ps listing also no longer calls out that a process is a background worker but just show the bgw_type. That way, being a background worker is more of an implementation detail now that is not shown to the user. However, most log messages still refer to 'background worker "%s"'; otherwise constructing sensible and translatable log messages would become tricky. Reviewed-by: Michael Paquier <michael.paquier@gmail.com> Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
2017-08-31 18:24:47 +02:00
snprintf(namebuf, MAXPGPATH, _("background worker \"%s\""),
rw->rw_worker.bgw_type);
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
if (!EXIT_STATUS_0(exitstatus))
When a background worker exists with code 0, unregister it. The previous behavior was to restart immediately, which was generally viewed as less useful. Petr Jelinek, with some adjustments by me.
2014-05-07 23:43:39 +02:00
{
/* Record timestamp, so we know when to restart the worker. */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
rw->rw_crashed_at = GetCurrentTimestamp();
When a background worker exists with code 0, unregister it. The previous behavior was to restart immediately, which was generally viewed as less useful. Petr Jelinek, with some adjustments by me.
2014-05-07 23:43:39 +02:00
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
else
When a background worker exists with code 0, unregister it. The previous behavior was to restart immediately, which was generally viewed as less useful. Petr Jelinek, with some adjustments by me.
2014-05-07 23:43:39 +02:00
{
/* Zero exit status means terminate */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
rw->rw_crashed_at = 0;
When a background worker exists with code 0, unregister it. The previous behavior was to restart immediately, which was generally viewed as less useful. Petr Jelinek, with some adjustments by me.
2014-05-07 23:43:39 +02:00
rw->rw_terminate = true;
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
* Additionally, for shared-memory-connected workers, just like a
* backend, any exit status other than 0 or 1 is considered a crash
* and causes a system-wide restart.
*/
When a bgworker exits, always call ReleasePostmasterChildSlot. Commit e2ce9aa27bf20eff2d991d0267a15ea5f7024cd7 was insufficiently well thought out. Repair.
2014-05-07 22:30:23 +02:00
if ((rw->rw_worker.bgw_flags & BGWORKER_SHMEM_ACCESS) != 0)
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
{
if (!EXIT_STATUS_0(exitstatus) && !EXIT_STATUS_1(exitstatus))
{
HandleChildCrash(pid, exitstatus, namebuf);
return true;
}
When a bgworker exits, always call ReleasePostmasterChildSlot. Commit e2ce9aa27bf20eff2d991d0267a15ea5f7024cd7 was insufficiently well thought out. Repair.
2014-05-07 22:30:23 +02:00
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
When a bgworker exits, always call ReleasePostmasterChildSlot. Commit e2ce9aa27bf20eff2d991d0267a15ea5f7024cd7 was insufficiently well thought out. Repair.
2014-05-07 22:30:23 +02:00
/*
pgindent run for 9.5
2015-05-24 03:35:49 +02:00
* We must release the postmaster child slot whether this worker is
* connected to shared memory or not, but we only treat it as a crash
* if it is in fact connected.
When a bgworker exits, always call ReleasePostmasterChildSlot. Commit e2ce9aa27bf20eff2d991d0267a15ea5f7024cd7 was insufficiently well thought out. Repair.
2014-05-07 22:30:23 +02:00
*/
if (!ReleasePostmasterChildSlot(rw->rw_child_slot) &&
(rw->rw_worker.bgw_flags & BGWORKER_SHMEM_ACCESS) != 0)
{
HandleChildCrash(pid, exitstatus, namebuf);
return true;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
}
/* Get it out of the BackendList and clear out remaining data */
Allow notifications to bgworkers without database connections. Previously, if one background worker registered another background worker and set bgw_notify_pid while for the second background worker, it would not receive notifications from the postmaster unless, at the time the "parent" was registered, BGWORKER_BACKEND_DATABASE_CONNECTION was set. To fix, instead instead of including only those background workers that requested database connections in the postmater's BackendList, include them all. There doesn't seem to be any reason not do this, and indeed it removes a significant amount of duplicated code. The other option is to make PostmasterMarkPIDForWorkerNotify look at BackgroundWorkerList in addition to BackendList, but that adds more code duplication instead of getting rid of it. Patch by me. Review and testing by Ashutosh Bapat.
2015-09-01 21:30:19 +02:00
dlist_delete(&rw->rw_backend->elem);
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
#ifdef EXEC_BACKEND
Allow notifications to bgworkers without database connections. Previously, if one background worker registered another background worker and set bgw_notify_pid while for the second background worker, it would not receive notifications from the postmaster unless, at the time the "parent" was registered, BGWORKER_BACKEND_DATABASE_CONNECTION was set. To fix, instead instead of including only those background workers that requested database connections in the postmater's BackendList, include them all. There doesn't seem to be any reason not do this, and indeed it removes a significant amount of duplicated code. The other option is to make PostmasterMarkPIDForWorkerNotify look at BackgroundWorkerList in addition to BackendList, but that adds more code duplication instead of getting rid of it. Patch by me. Review and testing by Ashutosh Bapat.
2015-09-01 21:30:19 +02:00
ShmemBackendArrayRemove(rw->rw_backend);
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
#endif
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
Allow notifications to bgworkers without database connections. Previously, if one background worker registered another background worker and set bgw_notify_pid while for the second background worker, it would not receive notifications from the postmaster unless, at the time the "parent" was registered, BGWORKER_BACKEND_DATABASE_CONNECTION was set. To fix, instead instead of including only those background workers that requested database connections in the postmater's BackendList, include them all. There doesn't seem to be any reason not do this, and indeed it removes a significant amount of duplicated code. The other option is to make PostmasterMarkPIDForWorkerNotify look at BackgroundWorkerList in addition to BackendList, but that adds more code duplication instead of getting rid of it. Patch by me. Review and testing by Ashutosh Bapat.
2015-09-01 21:30:19 +02:00
/*
* It's possible that this background worker started some OTHER
pgindent run for 9.6
2016-06-10 00:02:36 +02:00
* background worker and asked to be notified when that worker started
* or stopped. If so, cancel any notifications destined for the
* now-dead backend.
Allow notifications to bgworkers without database connections. Previously, if one background worker registered another background worker and set bgw_notify_pid while for the second background worker, it would not receive notifications from the postmaster unless, at the time the "parent" was registered, BGWORKER_BACKEND_DATABASE_CONNECTION was set. To fix, instead instead of including only those background workers that requested database connections in the postmater's BackendList, include them all. There doesn't seem to be any reason not do this, and indeed it removes a significant amount of duplicated code. The other option is to make PostmasterMarkPIDForWorkerNotify look at BackgroundWorkerList in addition to BackendList, but that adds more code duplication instead of getting rid of it. Patch by me. Review and testing by Ashutosh Bapat.
2015-09-01 21:30:19 +02:00
*/
if (rw->rw_backend->bgworker_notify)
BackgroundWorkerStopNotifications(rw->rw_pid);
free(rw->rw_backend);
rw->rw_backend = NULL;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
rw->rw_pid = 0;
rw->rw_child_slot = 0;
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
ReportBackgroundWorkerExit(&iter); /* report child death */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
Reduce log level for background worker events from LOG to DEBUG1. Per discussion, LOG is just too chatty for something that will happen as routinely as this. Pavel Stehule
2015-06-26 17:23:32 +02:00
LogChildExit(EXIT_STATUS_0(exitstatus) ? DEBUG1 : LOG,
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
namebuf, pid, exitstatus);
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
return true;
}
return false;
}
Here's a patch implementing the "thread method" to workaround the bug with socket calls in signal handlers (APC) on Win32. See details in mail to pgsql-hackers-win32 a couple of minutes ago. Magnus Hagander
2004-03-24 05:04:51 +01:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
Tweak postmaster code to avoid double reporting when bgwriter crashes.
2004-08-04 22:09:47 +02:00
* CleanupBackend -- cleanup after terminated backend.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
* Remove all local state associated with backend.
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
*
* If you change this, see also CleanupBackgroundWorker.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
static void
Tweak postmaster code to avoid double reporting when bgwriter crashes.
2004-08-04 22:09:47 +02:00
CleanupBackend(int pid,
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
int exitstatus) /* child's exit status. */
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
dlist_mutable_iter iter;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Use _() macro consistently rather than gettext(). Add translation macros around strings that were missing them.
2005-02-22 05:43:23 +01:00
LogChildExit(DEBUG2, _("server process"), pid, exitstatus);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Adjust elog.c so that elog(FATAL) exits (including cases where ERROR is promoted to FATAL) end in exit(1) not exit(0). Then change the postmaster to allow exit(1) without a system-wide panic, but not for the startup subprocess or the bgwriter. There were a couple of places that were using exit(1) to deliberately force a system-wide panic; adjust these to be exit(2) instead. This fixes the problem noted back in July that if the startup process exits with elog(ERROR), the postmaster would think everything is hunky-dory and proceed to start up. Alternative solutions such as trying to run the entire startup process as a critical section seem less clean, primarily because of the fact that a fair amount of startup code is shared by all postmaster children in the EXEC_BACKEND case. We'd need an ugly special case somewhere near the head of main.c to make it work if it's the child process's responsibility to determine what happens; and what's the point when the postmaster already treats different children differently?
2006-11-21 01:49:55 +01:00
* If a backend dies in an ugly way then we must signal all other backends
* to quickdie. If exit status is zero (normal) or one (FATAL exit), we
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
* assume everything is all right and proceed to remove the backend from
* the active backend list.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
C comments: remove odd blank lines after #ifdef WIN32 lines
2014-03-13 06:34:42 +01:00
#ifdef WIN32
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
Treat exit code 128 (ERROR_WAIT_NO_CHILDREN) as non-fatal on Win32, since it can happen when a process fails to start when the system is under high load. Per several bug reports and many peoples investigation. Back-patch to 8.4, which is as far back as the "deadman-switch" for shared memory access exists.
2010-09-16 22:37:13 +02:00
/*
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* On win32, also treat ERROR_WAIT_NO_CHILDREN (128) as nonfatal case,
* since that sometimes happens under load when the process fails to start
* properly (long before it starts using shared memory). Microsoft reports
* it is related to mutex failure:
* http://archives.postgresql.org/pgsql-hackers/2010-09/msg00790.php
Treat exit code 128 (ERROR_WAIT_NO_CHILDREN) as non-fatal on Win32, since it can happen when a process fails to start when the system is under high load. Per several bug reports and many peoples investigation. Back-patch to 8.4, which is as far back as the "deadman-switch" for shared memory access exists.
2010-09-16 22:37:13 +02:00
*/
if (exitstatus == ERROR_WAIT_NO_CHILDREN)
{
LogChildExit(LOG, _("server process"), pid, exitstatus);
exitstatus = 0;
}
#endif
Adjust elog.c so that elog(FATAL) exits (including cases where ERROR is promoted to FATAL) end in exit(1) not exit(0). Then change the postmaster to allow exit(1) without a system-wide panic, but not for the startup subprocess or the bgwriter. There were a couple of places that were using exit(1) to deliberately force a system-wide panic; adjust these to be exit(2) instead. This fixes the problem noted back in July that if the startup process exits with elog(ERROR), the postmaster would think everything is hunky-dory and proceed to start up. Alternative solutions such as trying to run the entire startup process as a critical section seem less clean, primarily because of the fact that a fair amount of startup code is shared by all postmaster children in the EXEC_BACKEND case. We'd need an ugly special case somewhere near the head of main.c to make it work if it's the child process's responsibility to determine what happens; and what's the point when the postmaster already treats different children differently?
2006-11-21 01:49:55 +01:00
if (!EXIT_STATUS_0(exitstatus) && !EXIT_STATUS_1(exitstatus))
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
{
Use _() macro consistently rather than gettext(). Add translation macros around strings that were missing them.
2005-02-22 05:43:23 +01:00
HandleChildCrash(pid, exitstatus, _("server process"));
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
return;
}
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
dlist_foreach_modify(iter, &BackendList)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
Backend *bp = dlist_container(Backend, elem, iter.cur);
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
if (bp->pid == pid)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (!bp->dead_end)
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
{
if (!ReleasePostmasterChildSlot(bp->child_slot))
{
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Uh-oh, the child failed to clean itself up. Treat as a
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* crash after all.
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
*/
HandleChildCrash(pid, exitstatus, _("server process"));
return;
}
#ifdef EXEC_BACKEND
ShmemBackendArrayRemove(bp);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
#endif
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
}
Allow discovery of whether a dynamic background worker is running. Using the infrastructure provided by this patch, it's possible either to wait for the startup of a dynamically-registered background worker, or to poll the status of such a worker without waiting. In either case, the current PID of the worker process can also be obtained. As usual, worker_spi is updated to demonstrate the new functionality. Patch by me. Review by Andres Freund.
2013-08-28 20:08:13 +02:00
if (bp->bgworker_notify)
{
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* This backend may have been slated to receive SIGUSR1 when
* some background worker started or stopped. Cancel those
* notifications, as we don't want to signal PIDs that are not
* PostgreSQL backends. This gets skipped in the (probably
* very common) case where the backend has never requested any
* such notifications.
Allow discovery of whether a dynamic background worker is running. Using the infrastructure provided by this patch, it's possible either to wait for the startup of a dynamically-registered background worker, or to poll the status of such a worker without waiting. In either case, the current PID of the worker process can also be obtained. As usual, worker_spi is updated to demonstrate the new functionality. Patch by me. Review by Andres Freund.
2013-08-28 20:08:13 +02:00
*/
BackgroundWorkerStopNotifications(bp->pid);
}
Remove unnecessary "head" arguments from some dlist/slist functions. dlist_delete, dlist_insert_after, dlist_insert_before, slist_insert_after do not need access to the list header, and indeed insisting on that negates one of the main advantages of a doubly-linked list. In consequence, revert addition of "cache_bucket" field to CatCTup.
2012-10-19 01:04:20 +02:00
dlist_delete(iter.cur);
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
free(bp);
break;
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
/*
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
* HandleChildCrash -- cleanup after failed backend, bgwriter, checkpointer,
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
* walwriter, autovacuum, or background worker.
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
*
* The objectives here are to clean up our local state about the child
* process, and to signal all other remaining children to quickdie.
*/
static void
Tweak postmaster code to avoid double reporting when bgwriter crashes.
2004-08-04 22:09:47 +02:00
HandleChildCrash(int pid, int exitstatus, const char *procname)
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
{
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
dlist_mutable_iter iter;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
slist_iter siter;
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
Backend *bp;
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
bool take_action;
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* We only log messages and send signals if this is the first process
* crash and we're not doing an immediate shutdown; otherwise, we're only
* here to update postmaster's idea of live processes. If we have already
* signalled children, nonzero exit status is to be expected, so don't
* clutter log.
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
*/
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
take_action = !FatalError && Shutdown != ImmediateShutdown;
if (take_action)
More startup/shutdown log messages.
1999-10-08 04:16:22 +02:00
{
Tweak postmaster code to avoid double reporting when bgwriter crashes.
2004-08-04 22:09:47 +02:00
LogChildExit(LOG, procname, pid, exitstatus);
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(LOG,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errmsg("terminating any other active server processes")));
More startup/shutdown log messages.
1999-10-08 04:16:22 +02:00
}
Prevent freshly-started backend from ignoring SIGUSR1, per race condition observed by Inoue. Also, don't call ProcRemove() from postmaster if we have detected a backend crash --- too risky if shared memory is corrupted. It's not needed anyway, considering we are going to reinitialize shared memory and semaphores as soon as the last child is dead.
2000-12-20 22:51:52 +01:00
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/* Process background workers. */
slist_foreach(siter, &BackgroundWorkerList)
{
RegisteredBgWorker *rw;
rw = slist_container(RegisteredBgWorker, rw_lnode, siter.cur);
if (rw->rw_pid == 0)
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
continue; /* not running */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
if (rw->rw_pid == pid)
{
/*
* Found entry for freshly-dead worker, so remove it.
*/
(void) ReleasePostmasterChildSlot(rw->rw_child_slot);
Allow notifications to bgworkers without database connections. Previously, if one background worker registered another background worker and set bgw_notify_pid while for the second background worker, it would not receive notifications from the postmaster unless, at the time the "parent" was registered, BGWORKER_BACKEND_DATABASE_CONNECTION was set. To fix, instead instead of including only those background workers that requested database connections in the postmater's BackendList, include them all. There doesn't seem to be any reason not do this, and indeed it removes a significant amount of duplicated code. The other option is to make PostmasterMarkPIDForWorkerNotify look at BackgroundWorkerList in addition to BackendList, but that adds more code duplication instead of getting rid of it. Patch by me. Review and testing by Ashutosh Bapat.
2015-09-01 21:30:19 +02:00
dlist_delete(&rw->rw_backend->elem);
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
#ifdef EXEC_BACKEND
Allow notifications to bgworkers without database connections. Previously, if one background worker registered another background worker and set bgw_notify_pid while for the second background worker, it would not receive notifications from the postmaster unless, at the time the "parent" was registered, BGWORKER_BACKEND_DATABASE_CONNECTION was set. To fix, instead instead of including only those background workers that requested database connections in the postmater's BackendList, include them all. There doesn't seem to be any reason not do this, and indeed it removes a significant amount of duplicated code. The other option is to make PostmasterMarkPIDForWorkerNotify look at BackgroundWorkerList in addition to BackendList, but that adds more code duplication instead of getting rid of it. Patch by me. Review and testing by Ashutosh Bapat.
2015-09-01 21:30:19 +02:00
ShmemBackendArrayRemove(rw->rw_backend);
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
#endif
Allow notifications to bgworkers without database connections. Previously, if one background worker registered another background worker and set bgw_notify_pid while for the second background worker, it would not receive notifications from the postmaster unless, at the time the "parent" was registered, BGWORKER_BACKEND_DATABASE_CONNECTION was set. To fix, instead instead of including only those background workers that requested database connections in the postmater's BackendList, include them all. There doesn't seem to be any reason not do this, and indeed it removes a significant amount of duplicated code. The other option is to make PostmasterMarkPIDForWorkerNotify look at BackgroundWorkerList in addition to BackendList, but that adds more code duplication instead of getting rid of it. Patch by me. Review and testing by Ashutosh Bapat.
2015-09-01 21:30:19 +02:00
free(rw->rw_backend);
rw->rw_backend = NULL;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
rw->rw_pid = 0;
rw->rw_child_slot = 0;
/* don't reset crashed_at */
Allow discovery of whether a dynamic background worker is running. Using the infrastructure provided by this patch, it's possible either to wait for the startup of a dynamically-registered background worker, or to poll the status of such a worker without waiting. In either case, the current PID of the worker process can also be obtained. As usual, worker_spi is updated to demonstrate the new functionality. Patch by me. Review by Andres Freund.
2013-08-28 20:08:13 +02:00
/* don't report child stop, either */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/* Keep looping so we can signal remaining workers */
}
else
{
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* This worker is still alive. Unless we did so already, tell it
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
* to commit hara-kiri.
*
* SIGQUIT is the special signal that says exit without proc_exit
* and let the user know what's going on. But if SendStop is set
* (-s on command line), then we send SIGSTOP instead, so that we
* can get core dumps from all backends by hand.
*/
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
if (take_action)
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
{
ereport(DEBUG2,
(errmsg_internal("sending %s to process %d",
(SendStop ? "SIGSTOP" : "SIGQUIT"),
(int) rw->rw_pid)));
signal_child(rw->rw_pid, (SendStop ? SIGSTOP : SIGQUIT));
}
}
}
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
/* Process regular backends */
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
dlist_foreach_modify(iter, &BackendList)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
bp = dlist_container(Backend, elem, iter.cur);
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
if (bp->pid == pid)
{
/*
* Found entry for freshly-dead backend, so remove it.
*/
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (!bp->dead_end)
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
{
(void) ReleasePostmasterChildSlot(bp->child_slot);
#ifdef EXEC_BACKEND
ShmemBackendArrayRemove(bp);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
#endif
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
}
Remove unnecessary "head" arguments from some dlist/slist functions. dlist_delete, dlist_insert_after, dlist_insert_before, slist_insert_after do not need access to the list header, and indeed insisting on that negates one of the main advantages of a doubly-linked list. In consequence, revert addition of "cache_bucket" field to CatCTup.
2012-10-19 01:04:20 +02:00
dlist_delete(iter.cur);
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
free(bp);
/* Keep looping so we can signal remaining backends */
}
else
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Prevent freshly-started backend from ignoring SIGUSR1, per race condition observed by Inoue. Also, don't call ProcRemove() from postmaster if we have detected a backend crash --- too risky if shared memory is corrupted. It's not needed anyway, considering we are going to reinitialize shared memory and semaphores as soon as the last child is dead.
2000-12-20 22:51:52 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* This backend is still alive. Unless we did so already, tell it
* to commit hara-kiri.
Prevent freshly-started backend from ignoring SIGUSR1, per race condition observed by Inoue. Also, don't call ProcRemove() from postmaster if we have detected a backend crash --- too risky if shared memory is corrupted. It's not needed anyway, considering we are going to reinitialize shared memory and semaphores as soon as the last child is dead.
2000-12-20 22:51:52 +01:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* SIGQUIT is the special signal that says exit without proc_exit
* and let the user know what's going on. But if SendStop is set
* (-s on command line), then we send SIGSTOP instead, so that we
* can get core dumps from all backends by hand.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*
* We could exclude dead_end children here, but at least in the
* SIGSTOP case it seems better to include them.
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
*
* Background workers were already processed above; ignore them
* here.
Prevent freshly-started backend from ignoring SIGUSR1, per race condition observed by Inoue. Also, don't call ProcRemove() from postmaster if we have detected a backend crash --- too risky if shared memory is corrupted. It's not needed anyway, considering we are going to reinitialize shared memory and semaphores as soon as the last child is dead.
2000-12-20 22:51:52 +01:00
*/
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
if (bp->bkend_type == BACKEND_TYPE_BGWORKER)
continue;
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
if (take_action)
Prevent freshly-started backend from ignoring SIGUSR1, per race condition observed by Inoue. Also, don't call ProcRemove() from postmaster if we have detected a backend crash --- too risky if shared memory is corrupted. It's not needed anyway, considering we are going to reinitialize shared memory and semaphores as soon as the last child is dead.
2000-12-20 22:51:52 +01:00
{
Change some frequently-reached elog(DEBUG...) calls to ereport(DEBUG...) for speed reasons. (ereport falls out much more quickly when no output is needed than elog does.)
2003-08-12 20:23:21 +02:00
ereport(DEBUG2,
(errmsg_internal("sending %s to process %d",
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(SendStop ? "SIGSTOP" : "SIGQUIT"),
Change some frequently-reached elog(DEBUG...) calls to ereport(DEBUG...) for speed reasons. (ereport falls out much more quickly when no output is needed than elog does.)
2003-08-12 20:23:21 +02:00
(int) bp->pid)));
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
signal_child(bp->pid, (SendStop ? SIGSTOP : SIGQUIT));
Prevent freshly-started backend from ignoring SIGUSR1, per race condition observed by Inoue. Also, don't call ProcRemove() from postmaster if we have detected a backend crash --- too risky if shared memory is corrupted. It's not needed anyway, considering we are going to reinitialize shared memory and semaphores as soon as the last child is dead.
2000-12-20 22:51:52 +01:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
}
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
/* Take care of the startup process too */
if (pid == StartupPID)
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
{
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
StartupPID = 0;
Fix postmaster state machine to handle dead_end child crashes better. A report from Alvaro Herrera shows that if we're in PM_STARTUP state, and we spawn a dead_end child to reject some incoming connection request, and that child dies with an unexpected exit code, the postmaster does not respond well. We correctly send SIGQUIT to the startup process, but then: * if the startup process exits with nonzero exit code, as expected, we thought that that indicated a crash and aborted startup. * if the startup process exits with zero exit code, which is possible due to the inherent race condition, we'd advance to PM_RUN state which is fine --- but the code forgot that AbortStartTime would be nonzero in this situation. We'd either die on the Asserts saying that it was zero, or perhaps misbehave later on. (A quick look suggests that the only misbehavior might be busy-waiting due to DetermineSleepTime doing the wrong thing.) To fix the first point, adjust the state-machine logic to recognize that a nonzero exit code is expected after sending SIGQUIT, and have it transition to a state where we can restart the startup process. To fix the second point, change the Asserts to clear the variable rather than just claiming it should be clear already. Perhaps we could improve this further by not treating a crash of a dead_end child as a reason for panic'ing the database. However, since those child processes are connected to shared memory, that seems a bit risky. There are few good reasons for a dead_end child to report failure anyway (the cause of this in Alvaro's report is quite unclear). On balance, therefore, a minimal fix seems best. This is an oversight in commit 45811be94. While that was back-patched, I'm hesitant to back-patch this change. The lack of reasons for a dead_end child to fail suggests that the case should be very rare in the field, which squares with the lack of reports; so it seems like this might not be worth the risk of introducing new issues. In any case we can let it bake awhile in HEAD before considering a back-patch. Discussion: https://postgr.es/m/20190615160950.GA31378@alvherre.pgsql
2019-08-26 21:59:44 +02:00
/* Caller adjusts StartupStatus, so don't touch it here */
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
}
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
else if (StartupPID != 0 && take_action)
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
{
ereport(DEBUG2,
(errmsg_internal("sending %s to process %d",
(SendStop ? "SIGSTOP" : "SIGQUIT"),
(int) StartupPID)));
Fix copy-pasto in the patch to allow background writer to run during recovery: if background writer or pgstat process dies during recovery (or any other child process, but those two are the only ones running), send SIGQUIT to the startup process using correct pid.
2009-03-03 11:42:05 +01:00
signal_child(StartupPID, (SendStop ? SIGSTOP : SIGQUIT));
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
StartupStatus = STARTUP_SIGNALED;
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
}
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
/* Take care of the bgwriter too */
if (pid == BgWriterPID)
Background writer process This first part of the background writer does no syncing at all. It's only purpose is to keep the LRU heads clean so that regular backends seldom to never have to call write(). Jan
2003-11-19 16:55:08 +01:00
BgWriterPID = 0;
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
else if (BgWriterPID != 0 && take_action)
Statistical system views (yet without the config stuff, but it's hard to keep such massive changes in sync with the tree so I need to get it in and work from there now). Jan
2001-06-22 21:16:24 +02:00
{
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
ereport(DEBUG2,
(errmsg_internal("sending %s to process %d",
(SendStop ? "SIGSTOP" : "SIGQUIT"),
(int) BgWriterPID)));
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
signal_child(BgWriterPID, (SendStop ? SIGSTOP : SIGQUIT));
Statistical system views (yet without the config stuff, but it's hard to keep such massive changes in sync with the tree so I need to get it in and work from there now). Jan
2001-06-22 21:16:24 +02:00
}
Prevent freshly-started backend from ignoring SIGUSR1, per race condition observed by Inoue. Also, don't call ProcRemove() from postmaster if we have detected a backend crash --- too risky if shared memory is corrupted. It's not needed anyway, considering we are going to reinitialize shared memory and semaphores as soon as the last child is dead.
2000-12-20 22:51:52 +01:00
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
/* Take care of the checkpointer too */
if (pid == CheckpointerPID)
CheckpointerPID = 0;
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
else if (CheckpointerPID != 0 && take_action)
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
{
ereport(DEBUG2,
(errmsg_internal("sending %s to process %d",
(SendStop ? "SIGSTOP" : "SIGQUIT"),
(int) CheckpointerPID)));
signal_child(CheckpointerPID, (SendStop ? SIGSTOP : SIGQUIT));
}
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
/* Take care of the walwriter too */
if (pid == WalWriterPID)
WalWriterPID = 0;
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
else if (WalWriterPID != 0 && take_action)
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
{
ereport(DEBUG2,
(errmsg_internal("sending %s to process %d",
(SendStop ? "SIGSTOP" : "SIGQUIT"),
(int) WalWriterPID)));
signal_child(WalWriterPID, (SendStop ? SIGSTOP : SIGQUIT));
}
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
/* Take care of the walreceiver too */
if (pid == WalReceiverPID)
WalReceiverPID = 0;
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
else if (WalReceiverPID != 0 && take_action)
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
{
ereport(DEBUG2,
(errmsg_internal("sending %s to process %d",
(SendStop ? "SIGSTOP" : "SIGQUIT"),
(int) WalReceiverPID)));
signal_child(WalReceiverPID, (SendStop ? SIGSTOP : SIGQUIT));
}
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
/* Take care of the autovacuum launcher too */
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
if (pid == AutoVacPID)
AutoVacPID = 0;
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
else if (AutoVacPID != 0 && take_action)
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
{
ereport(DEBUG2,
(errmsg_internal("sending %s to process %d",
(SendStop ? "SIGSTOP" : "SIGQUIT"),
(int) AutoVacPID)));
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
signal_child(AutoVacPID, (SendStop ? SIGSTOP : SIGQUIT));
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
}
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/*
* Force a power-cycle of the pgarch process too. (This isn't absolutely
* necessary, but it seems like a good idea for robustness, and it
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* simplifies the state-machine logic in the case where a shutdown request
* arrives during crash processing.)
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
if (PgArchPID != 0 && take_action)
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
{
ereport(DEBUG2,
(errmsg_internal("sending %s to process %d",
"SIGQUIT",
(int) PgArchPID)));
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
signal_child(PgArchPID, SIGQUIT);
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
}
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/*
* Force a power-cycle of the pgstat process too. (This isn't absolutely
* necessary, but it seems like a good idea for robustness, and it
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* simplifies the state-machine logic in the case where a shutdown request
* arrives during crash processing.)
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
if (PgStatPID != 0 && take_action)
Arrange to explicitly stop the pgstat processes at the same time we begin the shutdown checkpoint; there isn't anything left for them to do, so we may as well ensure that they shut down sooner rather than later. Per discussion.
2004-06-14 20:08:19 +02:00
{
ereport(DEBUG2,
(errmsg_internal("sending %s to process %d",
"SIGQUIT",
(int) PgStatPID)));
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
signal_child(PgStatPID, SIGQUIT);
Allow the pgstat process to restart immediately after a receiving SIGQUIT signal, rather than waiting for PGSTAT_RESTART_INTERVAL.
2007-03-22 20:53:31 +01:00
allow_immediate_pgstat_restart();
Arrange to explicitly stop the pgstat processes at the same time we begin the shutdown checkpoint; there isn't anything left for them to do, so we may as well ensure that they shut down sooner rather than later. Per discussion.
2004-06-14 20:08:19 +02:00
}
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
/* We do NOT restart the syslogger */
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
if (Shutdown != ImmediateShutdown)
FatalError = true;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/* We now transit into a state of waiting for children to die */
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
if (pmState == PM_RECOVERY ||
Rename PM_RECOVERY_CONSISTENT and PMSIGNAL_RECOVERY_CONSISTENT. The new names PM_HOT_STANDBY and PMSIGNAL_BEGIN_HOT_STANDBY more accurately reflect their actual function.
2010-05-15 22:01:32 +02:00
pmState == PM_HOT_STANDBY ||
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
pmState == PM_RUN ||
Code review for recent patch to terminate online backup during shutdown: do CancelBackup at a sane place, fix some oversights in the state transitions, allow only superusers to connect while we are waiting for backup mode to end.
2008-04-27 00:47:40 +02:00
pmState == PM_WAIT_BACKUP ||
Make smart shutdown work in combination with Hot Standby/Streaming Replication. At present, killing the startup process does not release any locks it holds, so we must wait to stop the startup and walreceiver processes until all read-only backends have exited. Without this patch, the startup and walreceiver processes never exit, so the server gets permanently stuck in a half-shutdown state. Fujii Masao, with review, docs, and comment adjustments by me.
2010-04-08 03:39:37 +02:00
pmState == PM_WAIT_READONLY ||
Code review for recent patch to terminate online backup during shutdown: do CancelBackup at a sane place, fix some oversights in the state transitions, allow only superusers to connect while we are waiting for backup mode to end.
2008-04-27 00:47:40 +02:00
pmState == PM_SHUTDOWN)
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
pmState = PM_WAIT_BACKENDS;
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
/*
* .. and if this doesn't happen quickly enough, now the clock is ticking
* for us to kill them without mercy.
*/
if (AbortStartTime == 0)
AbortStartTime = time(NULL);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Clean up formatting of child process exit-status reports so that they are correct, consistent, and complete ... motivated by gripe from Oliver Elphick, but I see someone had already made an incomplete stab at this.
2001-11-06 19:02:48 +01:00
/*
Restructure child-exit logging messages for easier translation, per suggestion from Peter.
2001-11-11 00:06:12 +01:00
* Log the death of a child process.
Clean up formatting of child process exit-status reports so that they are correct, consistent, and complete ... motivated by gripe from Oliver Elphick, but I see someone had already made an incomplete stab at this.
2001-11-06 19:02:48 +01:00
*/
Restructure child-exit logging messages for easier translation, per suggestion from Peter.
2001-11-11 00:06:12 +01:00
static void
Further work on elog cleanup: fix some bogosities in elog's logic about when to send what to which, prevent recursion by introducing new COMMERROR elog level for client-communication problems, get rid of direct writes to stderr in backend/libpq files, prevent non-error elogs from going to client during the authentication cycle.
2002-03-04 02:46:04 +01:00
LogChildExit(int lev, const char *procname, int pid, int exitstatus)
Clean up formatting of child process exit-status reports so that they are correct, consistent, and complete ... motivated by gripe from Oliver Elphick, but I see someone had already made an incomplete stab at this.
2001-11-06 19:02:48 +01:00
{
Code review for pgstat_get_crashed_backend_activity patch. Avoid possibly dumping core when pgstat_track_activity_query_size has a less-than-default value; avoid uselessly searching for the query string of a successfully-exited backend; don't bother putting out an ERRDETAIL if we don't have a query to show; some other minor stylistic improvements.
2011-10-21 22:36:04 +02:00
/*
* size of activity_buffer is arbitrary, but set equal to default
* track_activity_query_size
*/
char activity_buffer[1024];
const char *activity = NULL;
Try to log current the query string when a backend crashes. To avoid minimize risk inside the postmaster, we subject this feature to a number of significant limitations. We very much wish to avoid doing any complex processing inside the postmaster, due to the posssibility that the crashed backend has completely corrupted shared memory. To that end, no encoding conversion is done; instead, we just replace anything that doesn't look like an ASCII character with a question mark. We limit the amount of data copied to 1024 characters, and carefully sanity check the source of that data. While these restrictions would doubtless be unacceptable in a general-purpose logging facility, even this limited facility seems like an improvement over the status quo ante. Marti Raudsepp, reviewed by PDXPUG and myself
2011-10-21 19:26:40 +02:00
Code review for pgstat_get_crashed_backend_activity patch. Avoid possibly dumping core when pgstat_track_activity_query_size has a less-than-default value; avoid uselessly searching for the query string of a successfully-exited backend; don't bother putting out an ERRDETAIL if we don't have a query to show; some other minor stylistic improvements.
2011-10-21 22:36:04 +02:00
if (!EXIT_STATUS_0(exitstatus))
activity = pgstat_get_crashed_backend_activity(pid,
activity_buffer,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
sizeof(activity_buffer));
Try to log current the query string when a backend crashes. To avoid minimize risk inside the postmaster, we subject this feature to a number of significant limitations. We very much wish to avoid doing any complex processing inside the postmaster, due to the posssibility that the crashed backend has completely corrupted shared memory. To that end, no encoding conversion is done; instead, we just replace anything that doesn't look like an ASCII character with a question mark. We limit the amount of data copied to 1024 characters, and carefully sanity check the source of that data. While these restrictions would doubtless be unacceptable in a general-purpose logging facility, even this limited facility seems like an improvement over the status quo ante. Marti Raudsepp, reviewed by PDXPUG and myself
2011-10-21 19:26:40 +02:00
Clean up formatting of child process exit-status reports so that they are correct, consistent, and complete ... motivated by gripe from Oliver Elphick, but I see someone had already made an incomplete stab at this.
2001-11-06 19:02:48 +01:00
if (WIFEXITED(exitstatus))
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(lev,
pgindent run.
2003-08-04 02:43:34 +02:00
Fix some translator comments so that xgettext finds them and pgindent does not destroy them. Maybe we can adjust pgindent sometime.
2006-11-28 13:54:42 +01:00
/*------
translator: %s is a noun phrase describing a child process, such as
"server process" */
Message editing: remove gratuitous variations in message wording, standardize terms, add some clarifications, fix some untranslatable attempts at dynamic message building.
2003-09-25 08:58:07 +02:00
(errmsg("%s (PID %d) exited with exit code %d",
Try to log current the query string when a backend crashes. To avoid minimize risk inside the postmaster, we subject this feature to a number of significant limitations. We very much wish to avoid doing any complex processing inside the postmaster, due to the posssibility that the crashed backend has completely corrupted shared memory. To that end, no encoding conversion is done; instead, we just replace anything that doesn't look like an ASCII character with a question mark. We limit the amount of data copied to 1024 characters, and carefully sanity check the source of that data. While these restrictions would doubtless be unacceptable in a general-purpose logging facility, even this limited facility seems like an improvement over the status quo ante. Marti Raudsepp, reviewed by PDXPUG and myself
2011-10-21 19:26:40 +02:00
procname, pid, WEXITSTATUS(exitstatus)),
Code review for pgstat_get_crashed_backend_activity patch. Avoid possibly dumping core when pgstat_track_activity_query_size has a less-than-default value; avoid uselessly searching for the query string of a successfully-exited backend; don't bother putting out an ERRDETAIL if we don't have a query to show; some other minor stylistic improvements.
2011-10-21 22:36:04 +02:00
activity ? errdetail("Failed process was running: %s", activity) : 0));
Clean up formatting of child process exit-status reports so that they are correct, consistent, and complete ... motivated by gripe from Oliver Elphick, but I see someone had already made an incomplete stab at this.
2001-11-06 19:02:48 +01:00
else if (WIFSIGNALED(exitstatus))
Modernize our code for looking up descriptive strings for Unix signals. At least as far back as the 2008 spec, POSIX has defined strsignal(3) for looking up descriptive strings for signal numbers. We hadn't gotten the word though, and were still using the crufty old sys_siglist array, which is in no standard even though most Unixen provide it. Aside from not being formally standards-compliant, this was just plain ugly because it involved #ifdef's at every place using the code. To eliminate the #ifdef's, create a portability function pg_strsignal, which wraps strsignal(3) if available and otherwise falls back to sys_siglist[] if available. The set of Unixen with neither API is probably empty these days, but on any platform with neither, you'll just get "unrecognized signal". All extant callers print the numeric signal number too, so no need to work harder than that. Along the way, upgrade pg_basebackup's child-error-exit reporting to match the rest of the system. Discussion: https://postgr.es/m/25758.1544983503@sss.pgh.pa.us
2018-12-17 01:38:57 +01:00
{
Use sys_siglist[] to print out signal names for signal exits, rather than just numbers.
2007-01-28 02:12:05 +01:00
#if defined(WIN32)
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(lev,
pgindent run.
2003-08-04 02:43:34 +02:00
Fix some translator comments so that xgettext finds them and pgindent does not destroy them. Maybe we can adjust pgindent sometime.
2006-11-28 13:54:42 +01:00
/*------
translator: %s is a noun phrase describing a child process, such as
"server process" */
Clean up broken usage of HAVE_DECL_SYS_SIGLIST and inconsistent/poorly formatted error messages.
2007-01-28 07:32:03 +01:00
(errmsg("%s (PID %d) was terminated by exception 0x%X",
Use sys_siglist[] to print out signal names for signal exits, rather than just numbers.
2007-01-28 02:12:05 +01:00
procname, pid, WTERMSIG(exitstatus)),
Try to log current the query string when a backend crashes. To avoid minimize risk inside the postmaster, we subject this feature to a number of significant limitations. We very much wish to avoid doing any complex processing inside the postmaster, due to the posssibility that the crashed backend has completely corrupted shared memory. To that end, no encoding conversion is done; instead, we just replace anything that doesn't look like an ASCII character with a question mark. We limit the amount of data copied to 1024 characters, and carefully sanity check the source of that data. While these restrictions would doubtless be unacceptable in a general-purpose logging facility, even this limited facility seems like an improvement over the status quo ante. Marti Raudsepp, reviewed by PDXPUG and myself
2011-10-21 19:26:40 +02:00
errhint("See C include file \"ntstatus.h\" for a description of the hexadecimal value."),
Code review for pgstat_get_crashed_backend_activity patch. Avoid possibly dumping core when pgstat_track_activity_query_size has a less-than-default value; avoid uselessly searching for the query string of a successfully-exited backend; don't bother putting out an ERRDETAIL if we don't have a query to show; some other minor stylistic improvements.
2011-10-21 22:36:04 +02:00
activity ? errdetail("Failed process was running: %s", activity) : 0));
Modernize our code for looking up descriptive strings for Unix signals. At least as far back as the 2008 spec, POSIX has defined strsignal(3) for looking up descriptive strings for signal numbers. We hadn't gotten the word though, and were still using the crufty old sys_siglist array, which is in no standard even though most Unixen provide it. Aside from not being formally standards-compliant, this was just plain ugly because it involved #ifdef's at every place using the code. To eliminate the #ifdef's, create a portability function pg_strsignal, which wraps strsignal(3) if available and otherwise falls back to sys_siglist[] if available. The set of Unixen with neither API is probably empty these days, but on any platform with neither, you'll just get "unrecognized signal". All extant callers print the numeric signal number too, so no need to work harder than that. Along the way, upgrade pg_basebackup's child-error-exit reporting to match the rest of the system. Discussion: https://postgr.es/m/25758.1544983503@sss.pgh.pa.us
2018-12-17 01:38:57 +01:00
#else
Initial pgindent run with pg_bsd_indent version 2.0. The new indent version includes numerous fixes thanks to Piotr Stefaniak. The main changes visible in this commit are: * Nicer formatting of function-pointer declarations. * No longer unexpectedly removes spaces in expressions using casts, sizeof, or offsetof. * No longer wants to add a space in "struct structname *varname", as well as some similar cases for const- or volatile-qualified pointers. * Declarations using PG_USED_FOR_ASSERTS_ONLY are formatted more nicely. * Fixes bug where comments following declarations were sometimes placed with no space separating them from the code. * Fixes some odd decisions for comments following case labels. * Fixes some cases where comments following code were indented to less than the expected column 33. On the less good side, it now tends to put more whitespace around typedef names that are not listed in typedefs.list. This might encourage us to put more effort into typedef name collection; it's not really a bug in indent itself. There are more changes coming after this round, having to do with comment indentation and alignment of lines appearing within parentheses. I wanted to limit the size of the diffs to something that could be reviewed without one's eyes completely glazing over, so it seemed better to split up the changes as much as practical. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 20:39:04 +02:00
ereport(lev,
/*------
translator: %s is a noun phrase describing a child process, such as
"server process" */
(errmsg("%s (PID %d) was terminated by signal %d: %s",
procname, pid, WTERMSIG(exitstatus),
Modernize our code for looking up descriptive strings for Unix signals. At least as far back as the 2008 spec, POSIX has defined strsignal(3) for looking up descriptive strings for signal numbers. We hadn't gotten the word though, and were still using the crufty old sys_siglist array, which is in no standard even though most Unixen provide it. Aside from not being formally standards-compliant, this was just plain ugly because it involved #ifdef's at every place using the code. To eliminate the #ifdef's, create a portability function pg_strsignal, which wraps strsignal(3) if available and otherwise falls back to sys_siglist[] if available. The set of Unixen with neither API is probably empty these days, but on any platform with neither, you'll just get "unrecognized signal". All extant callers print the numeric signal number too, so no need to work harder than that. Along the way, upgrade pg_basebackup's child-error-exit reporting to match the rest of the system. Discussion: https://postgr.es/m/25758.1544983503@sss.pgh.pa.us
2018-12-17 01:38:57 +01:00
pg_strsignal(WTERMSIG(exitstatus))),
Code review for pgstat_get_crashed_backend_activity patch. Avoid possibly dumping core when pgstat_track_activity_query_size has a less-than-default value; avoid uselessly searching for the query string of a successfully-exited backend; don't bother putting out an ERRDETAIL if we don't have a query to show; some other minor stylistic improvements.
2011-10-21 22:36:04 +02:00
activity ? errdetail("Failed process was running: %s", activity) : 0));
When system() fails in Win32, report it as an exception, print the exception value in hex, and give a URL where the value can be looked-up.
2007-01-22 19:31:51 +01:00
#endif
Modernize our code for looking up descriptive strings for Unix signals. At least as far back as the 2008 spec, POSIX has defined strsignal(3) for looking up descriptive strings for signal numbers. We hadn't gotten the word though, and were still using the crufty old sys_siglist array, which is in no standard even though most Unixen provide it. Aside from not being formally standards-compliant, this was just plain ugly because it involved #ifdef's at every place using the code. To eliminate the #ifdef's, create a portability function pg_strsignal, which wraps strsignal(3) if available and otherwise falls back to sys_siglist[] if available. The set of Unixen with neither API is probably empty these days, but on any platform with neither, you'll just get "unrecognized signal". All extant callers print the numeric signal number too, so no need to work harder than that. Along the way, upgrade pg_basebackup's child-error-exit reporting to match the rest of the system. Discussion: https://postgr.es/m/25758.1544983503@sss.pgh.pa.us
2018-12-17 01:38:57 +01:00
}
Clean up formatting of child process exit-status reports so that they are correct, consistent, and complete ... motivated by gripe from Oliver Elphick, but I see someone had already made an incomplete stab at this.
2001-11-06 19:02:48 +01:00
else
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(lev,
pgindent run.
2003-08-04 02:43:34 +02:00
Fix some translator comments so that xgettext finds them and pgindent does not destroy them. Maybe we can adjust pgindent sometime.
2006-11-28 13:54:42 +01:00
/*------
translator: %s is a noun phrase describing a child process, such as
"server process" */
Clean up broken usage of HAVE_DECL_SYS_SIGLIST and inconsistent/poorly formatted error messages.
2007-01-28 07:32:03 +01:00
(errmsg("%s (PID %d) exited with unrecognized status %d",
Try to log current the query string when a backend crashes. To avoid minimize risk inside the postmaster, we subject this feature to a number of significant limitations. We very much wish to avoid doing any complex processing inside the postmaster, due to the posssibility that the crashed backend has completely corrupted shared memory. To that end, no encoding conversion is done; instead, we just replace anything that doesn't look like an ASCII character with a question mark. We limit the amount of data copied to 1024 characters, and carefully sanity check the source of that data. While these restrictions would doubtless be unacceptable in a general-purpose logging facility, even this limited facility seems like an improvement over the status quo ante. Marti Raudsepp, reviewed by PDXPUG and myself
2011-10-21 19:26:40 +02:00
procname, pid, exitstatus),
Code review for pgstat_get_crashed_backend_activity patch. Avoid possibly dumping core when pgstat_track_activity_query_size has a less-than-default value; avoid uselessly searching for the query string of a successfully-exited backend; don't bother putting out an ERRDETAIL if we don't have a query to show; some other minor stylistic improvements.
2011-10-21 22:36:04 +02:00
activity ? errdetail("Failed process was running: %s", activity) : 0));
Clean up formatting of child process exit-status reports so that they are correct, consistent, and complete ... motivated by gripe from Oliver Elphick, but I see someone had already made an incomplete stab at this.
2001-11-06 19:02:48 +01:00
}
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/*
* Advance the postmaster's state machine and take actions as appropriate
*
Avoid possible hang during smart shutdown. If a smart shutdown occurs just as a child is starting up, and the child subsequently becomes a walsender, there is a race condition: the postmaster might count the exstant backends, determine that there is one normal backend, and wait for it to die off. Had the walsender transition already occurred before the postmaster counted, it would have proceeded with the shutdown. To fix this, have each child that transforms into a walsender kick the postmaster just after doing so, so that the state machine is certain to advance. Fujii Masao
2011-04-04 01:42:00 +02:00
* This is common code for pmdie(), reaper() and sigusr1_handler(), which
* receive the signals that might mean we need to change state.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
static void
PostmasterStateMachine(void)
{
Prevent shutdown in normal mode if online backup is running, and have pg_ctl warn about this. Cancel running online backups (by renaming the backup_label file, thus rendering the backup useless) when shutting down in fast mode. Laurenz Albe
2008-04-23 15:44:59 +02:00
if (pmState == PM_WAIT_BACKUP)
{
/*
Code review for recent patch to terminate online backup during shutdown: do CancelBackup at a sane place, fix some oversights in the state transitions, allow only superusers to connect while we are waiting for backup mode to end.
2008-04-27 00:47:40 +02:00
* PM_WAIT_BACKUP state ends when online backup mode is not active.
Prevent shutdown in normal mode if online backup is running, and have pg_ctl warn about this. Cancel running online backups (by renaming the backup_label file, thus rendering the backup useless) when shutting down in fast mode. Laurenz Albe
2008-04-23 15:44:59 +02:00
*/
if (!BackupInProgress())
pmState = PM_WAIT_BACKENDS;
}
Make smart shutdown work in combination with Hot Standby/Streaming Replication. At present, killing the startup process does not release any locks it holds, so we must wait to stop the startup and walreceiver processes until all read-only backends have exited. Without this patch, the startup and walreceiver processes never exit, so the server gets permanently stuck in a half-shutdown state. Fujii Masao, with review, docs, and comment adjustments by me.
2010-04-08 03:39:37 +02:00
if (pmState == PM_WAIT_READONLY)
{
/*
* PM_WAIT_READONLY state ends when we have no regular backends that
* have been started during recovery. We kill the startup and
* walreceiver processes and transition to PM_WAIT_BACKENDS. Ideally,
* we might like to kill these processes first and then wait for
* backends to die off, but that doesn't work at present because
* killing the startup process doesn't release its locks.
*/
if (CountChildren(BACKEND_TYPE_NORMAL) == 0)
{
if (StartupPID != 0)
signal_child(StartupPID, SIGTERM);
if (WalReceiverPID != 0)
signal_child(WalReceiverPID, SIGTERM);
pmState = PM_WAIT_BACKENDS;
}
}
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* If we are in a state-machine state that implies waiting for backends to
* exit, see if they're all gone, and change state if so.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
if (pmState == PM_WAIT_BACKENDS)
{
/*
* PM_WAIT_BACKENDS state ends when we have no regular backends
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
* (including autovac workers), no bgworkers (including unconnected
* ones), and no walwriter, autovac launcher or bgwriter. If we are
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* doing crash recovery or an immediate shutdown then we expect the
* checkpointer to exit as well, otherwise not. The archiver, stats,
* and syslogger processes are disregarded since they are not
* connected to shared memory; we also disregard dead_end children
* here. Walsenders are also disregarded, they will be terminated
* later after writing the checkpoint record, like the archiver
* process.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
if (CountChildren(BACKEND_TYPE_NORMAL | BACKEND_TYPE_WORKER) == 0 &&
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
StartupPID == 0 &&
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
WalReceiverPID == 0 &&
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
BgWriterPID == 0 &&
Fix assertion failure by an immediate shutdown. In PM_WAIT_DEAD_END state, checkpointer process must be dead already. But an immediate shutdown could make postmaster's state machine transition to PM_WAIT_DEAD_END state even if checkpointer process is still running, and which caused assertion failure. This bug was introduced in commit 457d6cf049c57cabe9b46ea13f26138040a214ec. This patch ensures that postmaster's state machine doesn't transition to PM_WAIT_DEAD_END state in an immediate shutdown while checkpointer process is running.
2013-08-07 19:48:53 +02:00
(CheckpointerPID == 0 ||
(!FatalError && Shutdown < ImmediateShutdown)) &&
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
WalWriterPID == 0 &&
AutoVacPID == 0)
{
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
if (Shutdown >= ImmediateShutdown || FatalError)
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
{
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Start waiting for dead_end children to die. This state
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
* change causes ServerLoop to stop creating new ones.
*/
pmState = PM_WAIT_DEAD_END;
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* We already SIGQUIT'd the archiver and stats processes, if
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
* any, when we started immediate shutdown or entered
* FatalError state.
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
*/
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
}
else
{
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* If we get here, we are proceeding with normal shutdown. All
* the regular children are gone, and it's time to tell the
Revert "Prevent panic during shutdown checkpoint" This reverts commit 086221cf6b1727c2baed4703c582f657b7c5350e, which was made to master only. The approach implemented in the above commit has some issues. While those could easily be fixed incrementally, doing so would make backpatching considerably harder, so instead first revert this patch. Discussion: https://postgr.es/m/20170602002912.tqlwn4gymzlxpvs2@alap3.anarazel.de
2017-06-06 03:53:41 +02:00
* checkpointer to do a shutdown checkpoint.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
Assert(Shutdown > NoShutdown);
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
/* Start the checkpointer if not running */
if (CheckpointerPID == 0)
CheckpointerPID = StartCheckpointer();
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/* And tell it to shut down */
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
if (CheckpointerPID != 0)
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
{
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
signal_child(CheckpointerPID, SIGUSR2);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
pmState = PM_SHUTDOWN;
}
else
{
/*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* If we failed to fork a checkpointer, just shut down.
* Any required cleanup will happen at next restart. We
* set FatalError so that an "abnormal shutdown" message
* gets logged when we exit.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
FatalError = true;
pmState = PM_WAIT_DEAD_END;
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
/* Kill the walsenders, archiver and stats collector too */
Simplify SignalSomeChildren(BACKEND_TYPE_ALL) to SignalChildren()
2010-12-16 16:20:38 +01:00
SignalChildren(SIGQUIT);
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
if (PgArchPID != 0)
signal_child(PgArchPID, SIGQUIT);
if (PgStatPID != 0)
signal_child(PgStatPID, SIGQUIT);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
}
}
}
}
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
if (pmState == PM_SHUTDOWN_2)
{
/*
* PM_SHUTDOWN_2 state ends when there's no other children than
* dead_end children left. There shouldn't be any regular backends
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* left by now anyway; what we're really waiting for is walsenders and
* archiver.
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
*/
Fix some minor postmaster-state-machine issues. In sigusr1_handler, don't ignore PMSIGNAL_ADVANCE_STATE_MACHINE based on pmState. The restriction is unnecessary (PostmasterStateMachine should work in any state), not future-proof (since it makes too many assumptions about why the signal might be sent), and broken even today because a race condition can make it necessary to respond to the signal in PM_WAIT_READONLY state. The race condition seems unlikely, but if it did happen, a hot-standby postmaster could fail to shut down after receiving a smart-shutdown request. In MaybeStartWalReceiver, don't clear the WalReceiverRequested flag if the fork attempt fails. Leaving it set allows us to try again in future iterations of the postmaster idle loop. (The startup process would eventually send a fresh request signal, but this change may allow us to retry the fork sooner.) Remove an obsolete comment and unnecessary test in PostmasterStateMachine's handling of PM_SHUTDOWN_2 state. It's not possible to have a live walreceiver in that state, and AFAICT has not been possible since commit 5e85315ea. This isn't a live bug, but the false comment is quite confusing to readers. In passing, rearrange sigusr1_handler's CheckPromoteSignal tests so that we don't uselessly perform stat() calls that we're going to ignore the results of. Add some comments clarifying the behavior of MaybeStartWalReceiver; I very nearly rearranged it in a way that'd reintroduce the race condition fixed in e5d494d78. Mea culpa for not commenting that properly at the time. Back-patch to all supported branches. The PMSIGNAL_ADVANCE_STATE_MACHINE change is the only one of even minor significance, but we might as well keep this code in sync across branches. Discussion: https://postgr.es/m/9001.1556046681@sss.pgh.pa.us
2019-04-24 20:15:44 +02:00
if (PgArchPID == 0 && CountChildren(BACKEND_TYPE_ALL) == 0)
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
{
pmState = PM_WAIT_DEAD_END;
}
}
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (pmState == PM_WAIT_DEAD_END)
{
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* PM_WAIT_DEAD_END state ends when the BackendList is entirely empty
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
* (ie, no dead_end children remain), and the archiver and stats
* collector are gone too.
*
* The reason we wait for those two is to protect them against a new
* postmaster starting conflicting subprocesses; this isn't an
* ironclad protection, but it at least helps in the
* shutdown-and-immediately-restart scenario. Note that they have
* already been sent appropriate shutdown signals, either during a
* normal state transition leading up to PM_WAIT_DEAD_END, or during
* FatalError processing.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
if (dlist_is_empty(&BackendList) &&
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
PgArchPID == 0 && PgStatPID == 0)
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
{
/* These other guys should be dead already */
Assert(StartupPID == 0);
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
Assert(WalReceiverPID == 0);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
Assert(BgWriterPID == 0);
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
Assert(CheckpointerPID == 0);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
Assert(WalWriterPID == 0);
Assert(AutoVacPID == 0);
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
/* syslogger is not considered here */
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
pmState = PM_NO_CHILDREN;
}
}
/*
* If we've been told to shut down, we exit as soon as there are no
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* remaining children. If there was a crash, cleanup will occur at the
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
* next startup. (Before PostgreSQL 8.3, we tried to recover from the
* crash before exiting, but that seems unwise if we are quitting because
* we got SIGTERM from init --- there may well not be time for recovery
* before init decides to SIGKILL us.)
*
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
* Note that the syslogger continues to run. It will exit when it sees
* EOF on its input pipe, which happens when there are no more upstream
* processes.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
if (Shutdown > NoShutdown && pmState == PM_NO_CHILDREN)
{
if (FatalError)
{
ereport(LOG, (errmsg("abnormal database system shutdown")));
ExitPostmaster(1);
}
else
{
Code review for recent patch to terminate online backup during shutdown: do CancelBackup at a sane place, fix some oversights in the state transitions, allow only superusers to connect while we are waiting for backup mode to end.
2008-04-27 00:47:40 +02:00
/*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* Terminate exclusive backup mode to avoid recovery after a clean
* fast shutdown. Since an exclusive backup can only be taken
* during normal running (and not, for example, while running
* under Hot Standby) it only makes sense to do this if we reached
* normal running. If we're still in recovery, the backup file is
* one we're recovering *from*, and we must keep it around so that
* recovery restarts from the right place.
Code review for recent patch to terminate online backup during shutdown: do CancelBackup at a sane place, fix some oversights in the state transitions, allow only superusers to connect while we are waiting for backup mode to end.
2008-04-27 00:47:40 +02:00
*/
More fixes for shutdown during recovery. 1. If we receive a fast shutdown request while in the PM_STARTUP state, process it just as we would in PM_RECOVERY, PM_HOT_STANDBY, or PM_RUN. Without this change, an early fast shutdown followed by Hot Standby causes the database to get stuck in a state where a shutdown is pending (so no new connections are allowed) but the shutdown request is never processed unless we end Hot Standby and enter normal running. 2. Avoid removing the backup label file when a smart or fast shutdown occurs during recovery. It makes sense to do this once we've reached normal running, since we must be taking a backup which now won't be valid. But during recovery we must be recovering from a previously taken backup, and any backup label file is needed to restart recovery from the right place. Fujii Masao and Robert Haas
2010-05-26 14:32:41 +02:00
if (ReachedNormalRunning)
CancelBackup();
Code review for recent patch to terminate online backup during shutdown: do CancelBackup at a sane place, fix some oversights in the state transitions, allow only superusers to connect while we are waiting for backup mode to end.
2008-04-27 00:47:40 +02:00
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/* Normal exit from the postmaster is here */
ExitPostmaster(0);
}
}
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
/*
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
* If the startup process failed, or the user does not want an automatic
* restart after backend crashes, wait for all non-syslogger children to
* exit, and then exit postmaster. We don't try to reinitialize when the
* startup process fails, because more than likely it will just fail again
* and we will keep trying forever.
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
*/
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
if (pmState == PM_NO_CHILDREN &&
(StartupStatus == STARTUP_CRASHED || !restart_after_crash))
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
ExitPostmaster(1);
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* If we need to recover from a crash, wait for all non-syslogger children
* to exit, then reset shmem and StartupDataBase.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
if (FatalError && pmState == PM_NO_CHILDREN)
{
ereport(LOG,
(errmsg("all server processes terminated; reinitializing")));
Restart bgworkers immediately after a crash-and-restart cycle. Just as we would start bgworkers immediately after an initial startup of the server, we should restart them immediately when reinitializing. Petr Jelinek and Robert Haas
2014-05-07 22:04:47 +02:00
/* allow background workers to immediately restart */
ResetBackgroundWorkerCrashTimes();
Add contrib/pg_stat_statements for server-wide tracking of statement execution statistics. Takahiro Itagaki
2009-01-04 23:19:59 +01:00
shmem_exit(1);
Fix crash restart bug introduced in 8356753c212. The bug was caused by not re-reading the control file during crash recovery restarts, which lead to an attempt to pfree() shared memory contents. The fix is to re-read the control file, which seems good anyway. It's unclear as of this moment, whether we want to keep the refactoring introduced in the commit referenced above, or come up with an alternative approach. But fixing the bug in the mean time seems like a good idea regardless. A followup commit will introduce regression test coverage for crash restarts. Reported-By: Tom Lane Discussion: https://postgr.es/m/14134.1505572349@sss.pgh.pa.us
2017-09-17 10:00:39 +02:00
/* re-read control file into local memory */
LocalProcessControlFile(true);
Use data directory inode number, not port, to select SysV resource keys. This approach provides a much tighter binding between a data directory and the associated SysV shared memory block (and SysV or named-POSIX semaphores, if we're using those). Key collisions are still possible, but only between data directories stored on different filesystems, so the situation should be negligible in practice. More importantly, restarting the postmaster with a different port number no longer risks failing to identify a relevant shared memory block, even when postmaster.pid has been removed. A standalone backend is likewise much more certain to detect conflicting leftover backends. (In the longer term, we might now think about deprecating the port as a cluster-wide value, so that one postmaster could support sockets with varying port numbers. But that's for another day.) The hazards fixed here apply only on Unix systems; our Windows code paths already use identifiers derived from the data directory path name rather than the port. src/test/recovery/t/017_shm.pl, which intends to test key-collision cases, has been substantially rewritten since it can no longer use two postmasters with identical port numbers to trigger the case. Instead, use Perl's IPC::SharedMem module to create a conflicting shmem segment directly. The test script will be skipped if that module is not available. (This means that some older buildfarm members won't run it, but I don't think that that results in any meaningful coverage loss.) Patch by me; thanks to Noah Misch and Peter Eisentraut for discussion and review. Discussion: https://postgr.es/m/16908.1557521200@sss.pgh.pa.us
2019-09-05 19:31:41 +02:00
reset_shared();
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
StartupPID = StartupDataBase();
Assert(StartupPID != 0);
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
StartupStatus = STARTUP_RUNNING;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
pmState = PM_STARTUP;
Fix various bugs in postmaster SIGKILL processing Clamp the minimum sleep time during immediate shutdown or crash to a minimum of zero, not a maximum of one second. The previous code could result in a negative sleep time, leading to failure in select() calls. Also, on crash recovery, reset AbortStartTime as soon as SIGKILL is sent or abort processing has commenced instead of waiting until the startup process completes. Reset AbortStartTime as soon as SIGKILL is sent, too, to avoid doing that repeatedly. Per trouble report from Jeff Janes on CAMkU=1xd3=wFqZwwuXPWe4BQs3h1seYo8LV9JtSjW5RodoPxMg@mail.gmail.com Author: MauMau
2013-10-06 04:24:50 +02:00
/* crash recovery started, reset SIGKILL flag */
AbortStartTime = 0;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
}
}
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
/*
* Send a signal to a postmaster child process
*
* On systems that have setsid(), each child process sets itself up as a
* process group leader. For signals that are generally interpreted in the
* appropriate fashion, we signal the entire process group not just the
* direct child process. This allows us to, for example, SIGQUIT a blocked
* archive_recovery script, or SIGINT a script being run by a backend via
* system().
*
* There is a race condition for recently-forked children: they might not
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* have executed setsid() yet. So we signal the child directly as well as
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
* the group. We assume such a child will handle the signal before trying
* to spawn any grandchild processes. We also assume that signaling the
* child twice will not cause any problems.
*/
static void
signal_child(pid_t pid, int signal)
{
if (kill(pid, signal) < 0)
elog(DEBUG3, "kill(%ld,%d) failed: %m", (long) pid, signal);
#ifdef HAVE_SETSID
switch (signal)
{
case SIGINT:
case SIGTERM:
case SIGQUIT:
case SIGSTOP:
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
case SIGKILL:
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
if (kill(-pid, signal) < 0)
elog(DEBUG3, "kill(%ld,%d) failed: %m", (long) (-pid), signal);
break;
default:
break;
}
#endif
}
From: Massimo Dal Zotto <dz@cs.unitn.it> > tprintf.patch > > tprintf.patch > > adds functions and macros which implement a conditional trace package > with the ability to change flags and numeric options of running > backends at runtime. > Options/flags can be specified in the command line and/or read from > the file pg_options in the data directory.
1998-08-25 23:34:10 +02:00
/*
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
* Send a signal to the targeted children (but NOT special children;
* dead_end children are never signaled, either).
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
*/
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
static bool
SignalSomeChildren(int signal, int target)
From: Massimo Dal Zotto <dz@cs.unitn.it> > tprintf.patch > > tprintf.patch > > adds functions and macros which implement a conditional trace package > with the ability to change flags and numeric options of running > backends at runtime. > Options/flags can be specified in the command line and/or read from > the file pg_options in the data directory.
1998-08-25 23:34:10 +02:00
{
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
dlist_iter iter;
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
bool signaled = false;
From: Massimo Dal Zotto <dz@cs.unitn.it> > tprintf.patch > > tprintf.patch > > adds functions and macros which implement a conditional trace package > with the ability to change flags and numeric options of running > backends at runtime. > Options/flags can be specified in the command line and/or read from > the file pg_options in the data directory.
1998-08-25 23:34:10 +02:00
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
dlist_foreach(iter, &BackendList)
From: Massimo Dal Zotto <dz@cs.unitn.it> > tprintf.patch > > tprintf.patch > > adds functions and macros which implement a conditional trace package > with the ability to change flags and numeric options of running > backends at runtime. > Options/flags can be specified in the command line and/or read from > the file pg_options in the data directory.
1998-08-25 23:34:10 +02:00
{
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
Backend *bp = dlist_container(Backend, elem, iter.cur);
From: Massimo Dal Zotto <dz@cs.unitn.it> > tprintf.patch > > tprintf.patch > > adds functions and macros which implement a conditional trace package > with the ability to change flags and numeric options of running > backends at runtime. > Options/flags can be specified in the command line and/or read from > the file pg_options in the data directory.
1998-08-25 23:34:10 +02:00
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (bp->dead_end)
continue;
Avoid treating WAL senders as normal backends. The previous coding treated anything that wasn't an autovacuum launcher as a normal backend, which is wrong now that we also have WAL senders. Fujii Masao, reviewed by Robert Haas, Alvaro Herrera, Tom Lane, and Bernd Helmle.
2011-01-22 04:20:06 +01:00
/*
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* Since target == BACKEND_TYPE_ALL is the most common case, we test
* it first and avoid touching shared memory for every child.
Avoid treating WAL senders as normal backends. The previous coding treated anything that wasn't an autovacuum launcher as a normal backend, which is wrong now that we also have WAL senders. Fujii Masao, reviewed by Robert Haas, Alvaro Herrera, Tom Lane, and Bernd Helmle.
2011-01-22 04:20:06 +01:00
*/
if (target != BACKEND_TYPE_ALL)
{
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
* Assign bkend_type for any recently announced WAL Sender
* processes.
*/
if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
IsPostmasterChildWalSender(bp->child_slot))
bp->bkend_type = BACKEND_TYPE_WALSND;
Avoid treating WAL senders as normal backends. The previous coding treated anything that wasn't an autovacuum launcher as a normal backend, which is wrong now that we also have WAL senders. Fujii Masao, reviewed by Robert Haas, Alvaro Herrera, Tom Lane, and Bernd Helmle.
2011-01-22 04:20:06 +01:00
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
if (!(target & bp->bkend_type))
Avoid treating WAL senders as normal backends. The previous coding treated anything that wasn't an autovacuum launcher as a normal backend, which is wrong now that we also have WAL senders. Fujii Masao, reviewed by Robert Haas, Alvaro Herrera, Tom Lane, and Bernd Helmle.
2011-01-22 04:20:06 +01:00
continue;
}
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
ereport(DEBUG4,
(errmsg_internal("sending signal %d to process %d",
signal, (int) bp->pid)));
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
signal_child(bp->pid, signal);
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
signaled = true;
From: Massimo Dal Zotto <dz@cs.unitn.it> > tprintf.patch > > tprintf.patch > > adds functions and macros which implement a conditional trace package > with the ability to change flags and numeric options of running > backends at runtime. > Options/flags can be specified in the command line and/or read from > the file pg_options in the data directory.
1998-08-25 23:34:10 +02:00
}
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
return signaled;
From: Massimo Dal Zotto <dz@cs.unitn.it> > tprintf.patch > > tprintf.patch > > adds functions and macros which implement a conditional trace package > with the ability to change flags and numeric options of running > backends at runtime. > Options/flags can be specified in the command line and/or read from > the file pg_options in the data directory.
1998-08-25 23:34:10 +02:00
}
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
/*
* Send a termination signal to children. This considers all of our children
* processes, except syslogger and dead_end backends.
*/
static void
TerminateChildren(int signal)
{
SignalChildren(signal);
if (StartupPID != 0)
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
{
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
signal_child(StartupPID, signal);
Fix postmaster's handling of a startup-process crash. Ordinarily, a failure (unexpected exit status) of the startup subprocess should be considered fatal, so the postmaster should just close up shop and quit. However, if we sent the startup process a SIGQUIT or SIGKILL signal, the failure is hardly "unexpected", and we should attempt restart; this is necessary for recovery from ordinary backend crashes in hot-standby scenarios. I attempted to implement the latter rule with a two-line patch in commit 442231d7f71764b8c628044e7ce2225f9aa43b67, but it now emerges that that patch was a few bricks shy of a load: it failed to distinguish the case of a signaled startup process from the case where the new startup process crashes before reaching database consistency. That resulted in infinitely respawning a new startup process only to have it crash again. To handle this properly, we really must track whether we have sent the *current* startup process a kill signal. Rather than add yet another ad-hoc boolean to the postmaster's state, I chose to unify this with the existing RecoveryError flag into an enum tracking the startup process's state. That seems more consistent with the postmaster's general state machine design. Back-patch to 9.0, like the previous patch.
2015-07-09 19:22:22 +02:00
if (signal == SIGQUIT || signal == SIGKILL)
StartupStatus = STARTUP_SIGNALED;
}
Send SIGKILL to children if they don't die quickly in immediate shutdown On immediate shutdown, or during a restart-after-crash sequence, postmaster used to send SIGQUIT (and then abandon ship if shutdown); but this is not a good strategy if backends don't die because of that signal. (This might happen, for example, if a backend gets tangled trying to malloc() due to gettext(), as in an example illustrated by MauMau.) This causes problems when later trying to restart the server, because some processes are still attached to the shared memory segment. Instead of just abandoning such backends to their fates, we now have postmaster hang around for a little while longer, send a SIGKILL after some reasonable waiting period, and then exit. This makes immediate shutdown more reliable. There is disagreement on whether it's best for postmaster to exit after sending SIGKILL, or to stick around until all children have reported death. If this controversy is resolved differently than what this patch implements, it's an easy change to make. Bug reported by MauMau in message 20DAEA8949EC4E2289C6E8E58560DEC0@maumau MauMau and Álvaro Herrera
2013-06-28 23:20:53 +02:00
if (BgWriterPID != 0)
signal_child(BgWriterPID, signal);
if (CheckpointerPID != 0)
signal_child(CheckpointerPID, signal);
if (WalWriterPID != 0)
signal_child(WalWriterPID, signal);
if (WalReceiverPID != 0)
signal_child(WalReceiverPID, signal);
if (AutoVacPID != 0)
signal_child(AutoVacPID, signal);
if (PgArchPID != 0)
signal_child(PgArchPID, signal);
if (PgStatPID != 0)
signal_child(PgStatPID, signal);
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
* BackendStartup -- start backend process
*
Add code to allow profiling of backends on Linux: save and restore the profiling timer setting across fork(). The correct way to build a profilable backend on Linux is now gmake PROFILE="-pg -DLINUX_PROFILE"
2002-03-02 21:46:12 +01:00
* returns: STATUS_ERROR if the fork failed, STATUS_OK otherwise.
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
*
* Note: if you change this code, also consider StartAutovacuumWorker.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Make functions static where possible, enclose unused functions in #ifdef NOT_USED.
1997-08-19 23:40:56 +02:00
static int
From: Phil Thompson <phil@river-bank.demon.co.uk> I've completed the patch to fix the protocol and authentication issues I was discussing a couple of weeks ago. The particular changes are: - the protocol has a version number - network byte order is used throughout - the pg_hba.conf file is used to specify what method is used to authenticate a frontend (either password, ident, trust, reject, krb4 or krb5) - support for multiplexed backends is removed - appropriate changes to man pages - the -a switch to many programs to specify an authentication service no longer has any effect - the libpq.so version number has changed to 1.1 The new backend still supports the old protocol so old interfaces won't break.
1998-01-26 02:42:53 +01:00
BackendStartup(Port *port)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
Backend *bn; /* for backend cleanup */
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
pid_t pid;
pgindent run.
2002-09-04 22:31:48 +02:00
Attached is a patch that fixes these leaks, and does a couple other things as well: * Computes and saves a cancel key for each backend. * fflush before forking, to eliminate double-buffering problems between postmaster and backends. Other cleanups. Tom Lane
1998-06-09 06:06:12 +02:00
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* Create backend data structure. Better before the fork() so we can
* handle failure cleanly.
Fix getopt-vs-init_ps_display problem by copying original argv[] info, per suggestion from Peter. Simplify several APIs by transmitting the original argv location directly from main.c to ps_status.c, instead of passing it down through several levels of subroutines.
2001-10-21 05:25:36 +02:00
*/
bn = (Backend *) malloc(sizeof(Backend));
if (!bn)
{
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(LOG,
(errcode(ERRCODE_OUT_OF_MEMORY),
errmsg("out of memory")));
Fix getopt-vs-init_ps_display problem by copying original argv[] info, per suggestion from Peter. Simplify several APIs by transmitting the original argv location directly from main.c to ps_status.c, instead of passing it down through several levels of subroutines.
2001-10-21 05:25:36 +02:00
return STATUS_ERROR;
}
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
/*
* Compute the cancel key that will be assigned to this backend. The
* backend will have its own copy in the forked-off process' value of
* MyCancelKey, so that it can transmit the key to the frontend.
*/
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
if (!RandomCancelKey(&MyCancelKey))
{
Fix two thinkos related to strong random keys. pg_backend_random() is used for MD5 salt generation, but it can fail, and no checks were done on its status code. Fix memory leak, if generating a random number for a cancel key failed. Both issues were spotted by Coverity. Fix by Michael Paquier.
2016-12-12 08:58:32 +01:00
free(bn);
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
ereport(LOG,
Further cleanup from the strong-random patch. Also use the new facility for generating RADIUS authenticator requests, and salt in chkpass extension. Reword the error messages to be nicer. Fix bogus error code used in the message in BackendStartup.
2016-12-12 10:55:32 +01:00
(errcode(ERRCODE_INTERNAL_ERROR),
errmsg("could not generate random cancel key")));
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
return STATUS_ERROR;
}
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
bn->cancel_key = MyCancelKey;
Code review for recent patch to terminate online backup during shutdown: do CancelBackup at a sane place, fix some oversights in the state transitions, allow only superusers to connect while we are waiting for backup mode to end.
2008-04-27 00:47:40 +02:00
/* Pass down canAcceptConnections state */
Check for too many postmaster children before spawning a bgworker. The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-07 18:39:09 +02:00
port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
bn->dead_end = (port->canAcceptConnections != CAC_OK &&
port->canAcceptConnections != CAC_WAITBACKUP);
/*
* Unless it's a dead_end child, assign it a child slot number
*/
if (!bn->dead_end)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
else
bn->child_slot = 0;
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Allow discovery of whether a dynamic background worker is running. Using the infrastructure provided by this patch, it's possible either to wait for the startup of a dynamically-registered background worker, or to poll the status of such a worker without waiting. In either case, the current PID of the worker process can also be obtained. As usual, worker_spi is updated to demonstrate the new functionality. Patch by me. Review by Andres Freund.
2013-08-28 20:08:13 +02:00
/* Hasn't asked to be notified about any bgworkers yet */
bn->bgworker_notify = false;
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
#ifdef EXEC_BACKEND
pid = backend_forkexec(port);
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
#else /* !EXEC_BACKEND */
Refactor fork()-related code. We need to do various housekeeping tasks before we can invoke fork() -- flush stdio buffers, save and restore the profiling timer on Linux with LINUX_PROFILE, and handle BeOS stuff. This patch moves that code into a single function, fork_process(), instead of duplicating it at the various callsites of fork(). This patch doesn't address the EXEC_BACKEND case; there is room for further cleanup there.
2005-03-10 08:14:03 +01:00
pid = fork_process();
Handle reading of startup packet and authentication exchange after forking a new postmaster child process. This should eliminate problems with authentication blocking (e.g., ident, SSL init) and also reduce problems with the accept queue filling up under heavy load. The option to send elog output to a different file per backend (postgres -o) has been disabled for now because the initialization would have to happen in a different order and it's not clear we want to keep this anyway.
2001-06-20 20:07:56 +02:00
if (pid == 0) /* child */
Fix breakage of LINUX_PROFILE code due to recent Windows changes.
2003-05-28 21:36:28 +02:00
{
free(bn);
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
Commonalize process startup code. Move common code, that was duplicated in every postmaster child/every standalone process, into two functions in miscinit.c. Not only does that already result in a fair amount of net code reduction but it also makes it much easier to remove more duplication in the future. The prime motivation wasn't code deduplication though, but easier addition of new common code.
2015-01-13 13:12:37 +01:00
/* Detangle from postmaster */
InitPostmasterChild();
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
/* Close the postmaster's sockets */
ClosePostmasterPorts(false);
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
/* Perform additional initialization and collect startup packet */
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
BackendInitialize(port);
/* And run the backend */
Unify calling conventions for postgres/postmaster sub-main functions There was a wild mix of calling conventions: Some were declared to return void and didn't return, some returned an int exit code, some claimed to return an exit code, which the callers checked, but actually never returned, and so on. Now all of these functions are declared to return void and decorated with attribute noreturn and don't return. That's easiest, and most code already worked that way.
2012-06-25 20:25:26 +02:00
BackendRun(port);
Fix breakage of LINUX_PROFILE code due to recent Windows changes.
2003-05-28 21:36:28 +02:00
}
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
#endif /* EXEC_BACKEND */
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (pid < 0)
{
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/* in parent, fork failed */
If we fail to fork a new backend process, (try to) report the failure to the client before closing the connection. Before 7.2 this was done correctly, but new code would simply close the connection with no report to the client.
2002-01-06 22:40:02 +01:00
int save_errno = errno;
Small correction to previous patch: we shouldn't ReleasePostmasterChildSlot for a dead_end child, because we didn't AssignPostmasterChildSlot.
2009-08-24 20:09:37 +02:00
if (!bn->dead_end)
(void) ReleasePostmasterChildSlot(bn->child_slot);
If we fail to fork a new backend process, (try to) report the failure to the client before closing the connection. Before 7.2 this was done correctly, but new code would simply close the connection with no report to the client.
2002-01-06 22:40:02 +01:00
free(bn);
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
errno = save_errno;
ereport(LOG,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errmsg("could not fork new process for connection: %m")));
If we fail to fork a new backend process, (try to) report the failure to the client before closing the connection. Before 7.2 this was done correctly, but new code would simply close the connection with no report to the client.
2002-01-06 22:40:02 +01:00
report_fork_failure_to_client(port, save_errno);
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
return STATUS_ERROR;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/* in parent, successful fork */
Change some frequently-reached elog(DEBUG...) calls to ereport(DEBUG...) for speed reasons. (ereport falls out much more quickly when no output is needed than elog does.)
2003-08-12 20:23:21 +02:00
ereport(DEBUG2,
(errmsg_internal("forked new backend, pid=%d socket=%d",
Add some casts to try to silence most of the remaining format warnings on MinGW-W64.
2011-04-28 21:05:58 +02:00
(int) pid, (int) port->sock)));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Everything's been successful, it's safe to add this backend to our list
* of backends.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
bn->pid = pid;
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
bn->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
dlist_push_head(&BackendList, &bn->elem);
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
#ifdef EXEC_BACKEND
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
if (!bn->dead_end)
ShmemBackendArrayAdd(bn);
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
#endif
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
return STATUS_OK;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
If we fail to fork a new backend process, (try to) report the failure to the client before closing the connection. Before 7.2 this was done correctly, but new code would simply close the connection with no report to the client.
2002-01-06 22:40:02 +01:00
/*
* Try to report backend fork() failure to client before we close the
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* connection. Since we do not care to risk blocking the postmaster on
If we fail to fork a new backend process, (try to) report the failure to the client before closing the connection. Before 7.2 this was done correctly, but new code would simply close the connection with no report to the client.
2002-01-06 22:40:02 +01:00
* this connection, we set the connection to non-blocking and try only once.
*
* This is grungy special-purpose code; we cannot use backend libpq since
* it's not up and running.
*/
static void
report_fork_failure_to_client(Port *port, int errnum)
{
char buffer[1000];
Ensure that we retry rather than erroring out when send() or recv() return EINTR; the stats code was failing to do this and so were a couple of places in the postmaster. The stats code assumed that recv() could not return EINTR if a preceding select() showed the socket to be read-ready, but this is demonstrably false with our Windows implementation of recv(), and it may not be the case on all Unix variants either. I think this explains the intermittent stats regression test failures we've been seeing, as well as reports of stats collector instability under high load on Windows. Backpatch as far as 8.0.
2006-07-16 20:17:14 +02:00
int rc;
pgindent run.
2002-09-04 22:31:48 +02:00
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
/* Format the error message packet (always V2 protocol) */
If we fail to fork a new backend process, (try to) report the failure to the client before closing the connection. Before 7.2 this was done correctly, but new code would simply close the connection with no report to the client.
2002-01-06 22:40:02 +01:00
snprintf(buffer, sizeof(buffer), "E%s%s\n",
Use _() macro consistently rather than gettext(). Add translation macros around strings that were missing them.
2005-02-22 05:43:23 +01:00
_("could not fork new process for connection: "),
If we fail to fork a new backend process, (try to) report the failure to the client before closing the connection. Before 7.2 this was done correctly, but new code would simply close the connection with no report to the client.
2002-01-06 22:40:02 +01:00
strerror(errnum));
/* Set port to non-blocking. Don't do send() if this fails */
Kerberos fixes from Magnus Hagander --- in theory Kerberos 5 auth should work on Windows now. Also, rename set_noblock to pg_set_noblock; since it is included in libpq, the former name polluted application namespace.
2005-03-25 01:34:31 +01:00
if (!pg_set_noblock(port->sock))
If we fail to fork a new backend process, (try to) report the failure to the client before closing the connection. Before 7.2 this was done correctly, but new code would simply close the connection with no report to the client.
2002-01-06 22:40:02 +01:00
return;
Ensure that we retry rather than erroring out when send() or recv() return EINTR; the stats code was failing to do this and so were a couple of places in the postmaster. The stats code assumed that recv() could not return EINTR if a preceding select() showed the socket to be read-ready, but this is demonstrably false with our Windows implementation of recv(), and it may not be the case on all Unix variants either. I think this explains the intermittent stats regression test failures we've been seeing, as well as reports of stats collector instability under high load on Windows. Backpatch as far as 8.0.
2006-07-16 20:17:14 +02:00
/* We'll retry after EINTR, but ignore all other failures */
do
{
rc = send(port->sock, buffer, strlen(buffer) + 1, 0);
} while (rc < 0 && errno == EINTR);
If we fail to fork a new backend process, (try to) report the failure to the client before closing the connection. Before 7.2 this was done correctly, but new code would simply close the connection with no report to the client.
2002-01-06 22:40:02 +01:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
* BackendInitialize -- initialize an interactive (postmaster-child)
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
* backend process, and collect the client's startup packet.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
* returns: nothing. Will not return at all if there's any failure.
*
* Note: this code does not depend on having any access to shared memory.
* In the EXEC_BACKEND case, we are physically attached to shared memory
* but have not yet set up most of our local pointers to shmem structures.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
static void
BackendInitialize(Port *port)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
pgindent run.
2003-08-04 02:43:34 +02:00
int status;
Remove redundant code for getnameinfo() replacement Our getnameinfo() replacement implementation in getaddrinfo.c failed unless NI_NUMERICHOST and NI_NUMERICSERV were given as flags, because it doesn't resolve host names, only numeric IPs. But per standard, when those flags are not given, an implementation can still degrade to not returning host names, so this restriction is unnecessary. When we remove it, we can eliminate some code in postmaster.c that apparently tried to work around that.
2012-10-05 03:45:14 +02:00
int ret;
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
char remote_host[NI_MAXHOST];
char remote_port[NI_MAXSERV];
Remove GUC log_statement, log_pid, log_timestamp, log_source_port. Functionality superceeded by log_line_prefix. Andrew Dunstan
2004-03-15 16:56:28 +01:00
char remote_ps_data[NI_MAXHOST];
Remove fork()/exec() and only do fork(). Small cleanups.
1998-05-29 19:00:34 +02:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/* Save port etc. for ps status */
MyProcPort = port;
/*
* PreAuthDelay is a debugging aid for investigating problems in the
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* authentication cycle: it can be set in postgresql.conf to allow time to
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* attach to the newly-forked backend with a debugger. (See also
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* PostAuthDelay, which we allow clients to pass through PGOPTIONS, but it
* is not honored until after authentication.)
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
*/
if (PreAuthDelay > 0)
pg_usleep(PreAuthDelay * 1000000L);
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
/* This flag will remain set until InitPostgres finishes authentication */
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
ClientAuthInProgress = true; /* limit visibility of log messages */
This patch brings up to date what I did last year (now unfortunately bitrotted) to allow the logging of the end of a session, enabled by the config setting "log_disconnections". Andrew Dunstan
2004-02-17 04:54:57 +01:00
/* set these to empty in case they are needed before we set them up */
port->remote_host = "";
port->remote_port = "";
Add an overall timeout on the client authentication cycle, so that a hung client or lost connection can't indefinitely block a postmaster child (not to mention the possibility of deliberate DoS attacks). Timeout is controlled by new authentication_timeout GUC variable, which I set to 60 seconds by default ... does that seem reasonable?
2001-09-21 19:06:12 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Initialize libpq and enable reporting of ereport errors to the client.
* Must do this now because authentication uses libpq to send messages.
Add an overall timeout on the client authentication cycle, so that a hung client or lost connection can't indefinitely block a postmaster child (not to mention the possibility of deliberate DoS attacks). Timeout is controlled by new authentication_timeout GUC variable, which I set to 60 seconds by default ... does that seem reasonable?
2001-09-21 19:06:12 +02:00
*/
pq_init(); /* initialize libpq to talk to client */
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
whereToSendOutput = DestRemote; /* now safe to ereport to client */
Add an overall timeout on the client authentication cycle, so that a hung client or lost connection can't indefinitely block a postmaster child (not to mention the possibility of deliberate DoS attacks). Timeout is controlled by new authentication_timeout GUC variable, which I set to 60 seconds by default ... does that seem reasonable?
2001-09-21 19:06:12 +02:00
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
/*
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* We arrange for a simple exit(1) if we receive SIGTERM or SIGQUIT or
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* timeout while trying to collect the startup packet. Otherwise the
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* postmaster cannot shutdown the database FAST or IMMED cleanly if a
Prevent a double free by not reentering be_tls_close(). Reentering this function with the right timing caused a double free, typically crashing the backend. By synchronizing a disconnection with the authentication timeout, an unauthenticated attacker could achieve this somewhat consistently. Call be_tls_close() solely from within proc_exit_prepare(). Back-patch to 9.0 (all supported versions). Benkocs Norbert Attila Security: CVE-2015-3165
2015-05-18 16:02:31 +02:00
* buggy client fails to send the packet promptly. XXX it follows that
* the remainder of this function must tolerate losing control at any
* instant. Likewise, any pg_on_exit_callback registered before or during
* this function must be prepared to execute at any instant between here
* and the end of this function. Furthermore, affected callbacks execute
* partially or not at all when a second exit-inducing signal arrives
* after proc_exit_prepare() decrements on_proc_exit_index. (Thanks to
* that mechanic, callbacks need not anticipate more than one call.) This
* is fragile; it ought to instead follow the norm of handling interrupts
* at selected, safe opportunities.
Enable SIGTERM and SIGQUIT during client authentication so the postmaster can kill the forked off processes when shutdown is requested. Jan
2001-09-07 18:12:49 +02:00
*/
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
pqsignal(SIGTERM, startup_die);
pqsignal(SIGQUIT, startup_die);
Introduce timeout handling framework Management of timeouts was getting a little cumbersome; what we originally had was more than enough back when we were only concerned about deadlocks and query cancel; however, when we added timeouts for standby processes, the code got considerably messier. Since there are plans to add more complex timeouts, this seems a good time to introduce a central timeout handling module. External modules register their timeout handlers during process initialization, and later enable and disable them as they see fit using a simple API; timeout.c is in charge of keeping track of which timeouts are in effect at any time, installing a common SIGALRM signal handler, and calling setitimer() as appropriate to ensure timely firing of external handlers. timeout.c additionally supports pluggable modules to add their own timeouts, though this capability isn't exercised anywhere yet. Additionally, as of this commit, walsender processes are aware of timeouts; we had a preexisting bug there that made those ignore SIGALRM, thus being subject to unhandled deadlocks, particularly during the authentication phase. This has already been fixed in back branches in commit 0bf8eb2a, which see for more details. Main author: Zoltán Böszörményi Some review and cleanup by Álvaro Herrera Extensive reworking by Tom Lane
2012-07-17 00:43:21 +02:00
InitializeTimeouts(); /* establishes SIGALRM handler */
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
PG_SETMASK(&StartupBlockSig);
Enable SIGTERM and SIGQUIT during client authentication so the postmaster can kill the forked off processes when shutdown is requested. Jan
2001-09-07 18:12:49 +02:00
Cleanup code for preparsing pg_hba.conf and pg_ident.conf. Store line number in the data structure so that we can give at least a minimally useful idea of where the mistake is when we issue syntax error messages. Move the ClientAuthentication() call to where it should have been in the first place, so that postmaster memory releasing can happen in a reasonable place also. Update obsolete comments, correct one real bug (auth_argument was not picked up correctly).
2001-08-01 00:55:45 +02:00
/*
Rearrange LOG_CONNECTIONS code so that two log messages are made: one immediately upon forking to handle a new connection, and one after the authentication cycle is finished. Per today's pggeneral discussion.
2002-05-29 01:56:51 +02:00
* Get the remote host name and port for logging and status display.
Move init_ps_display from postgres.c to postmaster.c, putting it just after receipt of the startup packet. Now, postmaster children that are waiting for client authentication response will show as 'postgres: user database host authentication'. Also, do an init_ps_display for startup/shutdown/checkpoint subprocesses, so that they are readily identifiable as well. Fix an obscure race condition that could lead to Assert failure in the postmaster --- attempting to start a checkpoint process before any connections have been received led to calling PostmasterRandom before setting random_seed.
2001-10-19 02:44:08 +02:00
*/
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
remote_host[0] = '\0';
remote_port[0] = '\0';
Remove redundant code for getnameinfo() replacement Our getnameinfo() replacement implementation in getaddrinfo.c failed unless NI_NUMERICHOST and NI_NUMERICSERV were given as flags, because it doesn't resolve host names, only numeric IPs. But per standard, when those flags are not given, an implementation can still degrade to not returning host names, so this restriction is unnecessary. When we remove it, we can eliminate some code in postmaster.c that apparently tried to work around that.
2012-10-05 03:45:14 +02:00
if ((ret = pg_getnameinfo_all(&port->raddr.addr, port->raddr.salen,
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
remote_host, sizeof(remote_host),
remote_port, sizeof(remote_port),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(log_hostname ? 0 : NI_NUMERICHOST) | NI_NUMERICSERV)) != 0)
Remove redundant code for getnameinfo() replacement Our getnameinfo() replacement implementation in getaddrinfo.c failed unless NI_NUMERICHOST and NI_NUMERICSERV were given as flags, because it doesn't resolve host names, only numeric IPs. But per standard, when those flags are not given, an implementation can still degrade to not returning host names, so this restriction is unnecessary. When we remove it, we can eliminate some code in postmaster.c that apparently tried to work around that.
2012-10-05 03:45:14 +02:00
ereport(WARNING,
(errmsg_internal("pg_getnameinfo_all() failed: %s",
gai_strerror(ret))));
Refactor sprintf calls with computed format strings into multiple calls with constant format strings, so that the compiler can more easily check the formats for correctness.
2010-06-16 02:54:16 +02:00
if (remote_port[0] == '\0')
snprintf(remote_ps_data, sizeof(remote_ps_data), "%s", remote_host);
else
snprintf(remote_ps_data, sizeof(remote_ps_data), "%s(%s)", remote_host, remote_port);
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
Fix startup so that log prefix %h works for the log_connections message. We entirely randomly chose to initialize port->remote_host just after printing the log_connections message, when we could perfectly well do it just before, allowing %h and %r to work for that message. Per gripe from Artem Tomyuk.
2016-01-26 21:38:33 +01:00
/*
* Save remote_host and remote_port in port structure (after this, they
* will appear in log_line_prefix data for log messages).
*/
port->remote_host = strdup(remote_host);
port->remote_port = strdup(remote_port);
/* And now we can issue the Log_connections message, if wanted */
IPv6 cleanups. Kurt Roeckx Andrew Dunstan
2003-06-12 09:36:51 +02:00
if (Log_connections)
Modify some new and pre-existing messages for translatability.
2010-03-25 21:40:17 +01:00
{
if (remote_port[0])
ereport(LOG,
pgindent run for 9.0, second run
2010-07-06 21:19:02 +02:00
(errmsg("connection received: host=%s port=%s",
remote_host,
remote_port)));
Modify some new and pre-existing messages for translatability.
2010-03-25 21:40:17 +01:00
else
ereport(LOG,
pgindent run for 9.0, second run
2010-07-06 21:19:02 +02:00
(errmsg("connection received: host=%s",
remote_host)));
Modify some new and pre-existing messages for translatability.
2010-03-25 21:40:17 +01:00
}
Rearrange LOG_CONNECTIONS code so that two log messages are made: one immediately upon forking to handle a new connection, and one after the authentication cycle is finished. Per today's pggeneral discussion.
2002-05-29 01:56:51 +02:00
Fix assorted issues in client host name lookup. The code for matching clients to pg_hba.conf lines that specify host names (instead of IP address ranges) failed to complain if reverse DNS lookup failed; instead it silently didn't match, so that you might end up getting a surprising "no pg_hba.conf entry for ..." error, as seen in bug #9518 from Mike Blackwell. Since we don't want to make this a fatal error in situations where pg_hba.conf contains a mixture of host names and IP addresses (clients matching one of the numeric entries should not have to have rDNS data), remember the lookup failure and mention it as DETAIL if we get to "no pg_hba.conf entry". Apply the same approach to forward-DNS lookup failures, too, rather than treating them as immediate hard errors. Along the way, fix a couple of bugs that prevented us from detecting an rDNS lookup error reliably, and make sure that we make only one rDNS lookup attempt; formerly, if the lookup attempt failed, the code would try again for each host name entry in pg_hba.conf. Since more or less the whole point of this design is to ensure there's only one lookup attempt not one per entry, the latter point represents a performance bug that seems sufficient justification for back-patching. Also, adjust src/port/getaddrinfo.c so that it plays as well as it can with this code. Which is not all that well, since it does not have actual support for rDNS lookup, but at least it should return the expected (and required by spec) error codes so that the main code correctly perceives the lack of functionality as a lookup failure. It's unlikely that PG is still being used in production on any machines that require our getaddrinfo.c, so I'm not excited about working harder than this. To keep the code in the various branches similar, this includes back-patching commits c424d0d1052cb4053c8712ac44123f9b9a9aa3f2 and 1997f34db4687e671690ed054c8f30bb501b1168 into 9.2 and earlier. Back-patch to 9.1 where the facility for hostnames in pg_hba.conf was introduced.
2014-04-02 23:11:24 +02:00
/*
* If we did a reverse lookup to name, we might as well save the results
* rather than possibly repeating the lookup during authentication.
*
* Note that we don't want to specify NI_NAMEREQD above, because then we'd
* get nothing useful for a client without an rDNS entry. Therefore, we
* must check whether we got a numeric IPv4 or IPv6 address, and not save
* it into remote_hostname if so. (This test is conservative and might
* sometimes classify a hostname as numeric, but an error in that
* direction is safe; it only results in a possible extra lookup.)
*/
if (log_hostname &&
ret == 0 &&
strspn(remote_host, "0123456789.") < strlen(remote_host) &&
strspn(remote_host, "0123456789ABCDEFabcdef:") < strlen(remote_host))
port->remote_hostname = strdup(remote_host);
This patch brings up to date what I did last year (now unfortunately bitrotted) to allow the logging of the end of a session, enabled by the config setting "log_disconnections". Andrew Dunstan
2004-02-17 04:54:57 +01:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/*
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
* Ready to begin client interaction. We will give up and exit(1) after a
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* time delay, so that a broken client can't hog a connection
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
* indefinitely. PreAuthDelay and any DNS interactions above don't count
* against the time limit.
Introduce timeout handling framework Management of timeouts was getting a little cumbersome; what we originally had was more than enough back when we were only concerned about deadlocks and query cancel; however, when we added timeouts for standby processes, the code got considerably messier. Since there are plans to add more complex timeouts, this seems a good time to introduce a central timeout handling module. External modules register their timeout handlers during process initialization, and later enable and disable them as they see fit using a simple API; timeout.c is in charge of keeping track of which timeouts are in effect at any time, installing a common SIGALRM signal handler, and calling setitimer() as appropriate to ensure timely firing of external handlers. timeout.c additionally supports pluggable modules to add their own timeouts, though this capability isn't exercised anywhere yet. Additionally, as of this commit, walsender processes are aware of timeouts; we had a preexisting bug there that made those ignore SIGALRM, thus being subject to unhandled deadlocks, particularly during the authentication phase. This has already been fixed in back branches in commit 0bf8eb2a, which see for more details. Main author: Zoltán Böszörményi Some review and cleanup by Álvaro Herrera Extensive reworking by Tom Lane
2012-07-17 00:43:21 +02:00
*
* Note: AuthenticationTimeout is applied here while waiting for the
* startup packet, and then again in InitPostgres for the duration of any
* authentication operations. So a hostile client could tie up the
* process for nearly twice AuthenticationTimeout before we kick him off.
*
* Note: because PostgresMain will call InitializeTimeouts again, the
* registration of STARTUP_PACKET_TIMEOUT will be lost. This is okay
* since we never use it again after this function.
Rearrange LOG_CONNECTIONS code so that two log messages are made: one immediately upon forking to handle a new connection, and one after the authentication cycle is finished. Per today's pggeneral discussion.
2002-05-29 01:56:51 +02:00
*/
Introduce timeout handling framework Management of timeouts was getting a little cumbersome; what we originally had was more than enough back when we were only concerned about deadlocks and query cancel; however, when we added timeouts for standby processes, the code got considerably messier. Since there are plans to add more complex timeouts, this seems a good time to introduce a central timeout handling module. External modules register their timeout handlers during process initialization, and later enable and disable them as they see fit using a simple API; timeout.c is in charge of keeping track of which timeouts are in effect at any time, installing a common SIGALRM signal handler, and calling setitimer() as appropriate to ensure timely firing of external handlers. timeout.c additionally supports pluggable modules to add their own timeouts, though this capability isn't exercised anywhere yet. Additionally, as of this commit, walsender processes are aware of timeouts; we had a preexisting bug there that made those ignore SIGALRM, thus being subject to unhandled deadlocks, particularly during the authentication phase. This has already been fixed in back branches in commit 0bf8eb2a, which see for more details. Main author: Zoltán Böszörményi Some review and cleanup by Álvaro Herrera Extensive reworking by Tom Lane
2012-07-17 00:43:21 +02:00
RegisterTimeout(STARTUP_PACKET_TIMEOUT, StartupPacketTimeoutHandler);
enable_timeout_after(STARTUP_PACKET_TIMEOUT, AuthenticationTimeout * 1000);
Rearrange LOG_CONNECTIONS code so that two log messages are made: one immediately upon forking to handle a new connection, and one after the authentication cycle is finished. Per today's pggeneral discussion.
2002-05-29 01:56:51 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Receive the startup packet (which might turn out to be a cancel request
* packet).
Rearrange LOG_CONNECTIONS code so that two log messages are made: one immediately upon forking to handle a new connection, and one after the authentication cycle is finished. Per today's pggeneral discussion.
2002-05-29 01:56:51 +02:00
*/
status = ProcessStartupPacket(port, false);
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Stop here if it was bad or a cancel packet. ProcessStartupPacket
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
* already did any appropriate error reporting.
*/
Rearrange LOG_CONNECTIONS code so that two log messages are made: one immediately upon forking to handle a new connection, and one after the authentication cycle is finished. Per today's pggeneral discussion.
2002-05-29 01:56:51 +02:00
if (status != STATUS_OK)
Fix for cancel key restarting postmaster, from Claudio
2004-01-10 00:11:39 +01:00
proc_exit(0);
Rearrange LOG_CONNECTIONS code so that two log messages are made: one immediately upon forking to handle a new connection, and one after the authentication cycle is finished. Per today's pggeneral discussion.
2002-05-29 01:56:51 +02:00
/*
* Now that we have the user and database name, we can set the process
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* title for ps. It's good to do this as early as possible in startup.
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
*
* For a walsender, the ps display is set in the following form:
*
Sync process names between ps and pg_stat_activity Remove gratuitous differences in the process names shown in pg_stat_activity.backend_type and the ps output. Reviewed-by: Takayuki Tsunakawa <tsunakawa.takay@jp.fujitsu.com>
2017-09-20 03:15:09 +02:00
* postgres: walsender <user> <host> <activity>
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
*
Update typedefs.list and re-run pgindent Discussion: http://postgr.es/m/CA+TgmoaA9=1RWKtBWpDaj+sF3Stgc8sHgf5z=KGtbjwPLQVDMA@mail.gmail.com
2017-11-29 15:24:24 +01:00
* To achieve that, we pass "walsender" as username and username as dbname
* to init_ps_display(). XXX: should add a new variant of
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
* init_ps_display() to avoid abusing the parameters like this.
Move init_ps_display from postgres.c to postmaster.c, putting it just after receipt of the startup packet. Now, postmaster children that are waiting for client authentication response will show as 'postgres: user database host authentication'. Also, do an init_ps_display for startup/shutdown/checkpoint subprocesses, so that they are readily identifiable as well. Fix an obscure race condition that could lead to Assert failure in the postmaster --- attempting to start a checkpoint process before any connections have been received led to calling PostmasterRandom before setting random_seed.
2001-10-19 02:44:08 +02:00
*/
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
if (am_walsender)
Sync process names between ps and pg_stat_activity Remove gratuitous differences in the process names shown in pg_stat_activity.backend_type and the ps output. Reviewed-by: Takayuki Tsunakawa <tsunakawa.takay@jp.fujitsu.com>
2017-09-20 03:15:09 +02:00
init_ps_display(pgstat_get_backend_desc(B_WAL_SENDER), port->user_name, remote_ps_data,
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
update_process_title ? "authentication" : "");
else
init_ps_display(port->user_name, port->database_name, remote_ps_data,
update_process_title ? "authentication" : "");
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
Move init_ps_display from postgres.c to postmaster.c, putting it just after receipt of the startup packet. Now, postmaster children that are waiting for client authentication response will show as 'postgres: user database host authentication'. Also, do an init_ps_display for startup/shutdown/checkpoint subprocesses, so that they are readily identifiable as well. Fix an obscure race condition that could lead to Assert failure in the postmaster --- attempting to start a checkpoint process before any connections have been received led to calling PostmasterRandom before setting random_seed.
2001-10-19 02:44:08 +02:00
/*
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
* Disable the timeout, and prevent SIGTERM/SIGQUIT again.
Clean up some confusion about where and how to set whereToSendOutput. We will no longer try to send elog messages to the client before we have initialized backend libpq (oops); however, reporting bogus commandline switches via elog does work now (not irrelevant, because of PGOPTIONS). Fix problem with inappropriate sending of checkpoint-process messages to stderr.
2001-09-08 03:10:21 +02:00
*/
Introduce timeout handling framework Management of timeouts was getting a little cumbersome; what we originally had was more than enough back when we were only concerned about deadlocks and query cancel; however, when we added timeouts for standby processes, the code got considerably messier. Since there are plans to add more complex timeouts, this seems a good time to introduce a central timeout handling module. External modules register their timeout handlers during process initialization, and later enable and disable them as they see fit using a simple API; timeout.c is in charge of keeping track of which timeouts are in effect at any time, installing a common SIGALRM signal handler, and calling setitimer() as appropriate to ensure timely firing of external handlers. timeout.c additionally supports pluggable modules to add their own timeouts, though this capability isn't exercised anywhere yet. Additionally, as of this commit, walsender processes are aware of timeouts; we had a preexisting bug there that made those ignore SIGALRM, thus being subject to unhandled deadlocks, particularly during the authentication phase. This has already been fixed in back branches in commit 0bf8eb2a, which see for more details. Main author: Zoltán Böszörményi Some review and cleanup by Álvaro Herrera Extensive reworking by Tom Lane
2012-07-17 00:43:21 +02:00
disable_timeout(STARTUP_PACKET_TIMEOUT, false);
Enable SIGTERM and SIGQUIT during client authentication so the postmaster can kill the forked off processes when shutdown is requested. Jan
2001-09-07 18:12:49 +02:00
PG_SETMASK(&BlockSig);
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
}
/*
* BackendRun -- set up the backend's argument list and invoke PostgresMain()
*
* returns:
* Shouldn't return at all.
* If PostgresMain() fails, return status.
*/
Unify calling conventions for postgres/postmaster sub-main functions There was a wild mix of calling conventions: Some were declared to return void and didn't return, some returned an int exit code, some claimed to return an exit code, which the callers checked, but actually never returned, and so on. Now all of these functions are declared to return void and decorated with attribute noreturn and don't return. That's easiest, and most code already worked that way.
2012-06-25 20:25:26 +02:00
static void
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
BackendRun(Port *port)
{
char **av;
int maxac;
int ac;
int i;
Move init_ps_display from postgres.c to postmaster.c, putting it just after receipt of the startup packet. Now, postmaster children that are waiting for client authentication response will show as 'postgres: user database host authentication'. Also, do an init_ps_display for startup/shutdown/checkpoint subprocesses, so that they are readily identifiable as well. Fix an obscure race condition that could lead to Assert failure in the postmaster --- attempting to start a checkpoint process before any connections have been received led to calling PostmasterRandom before setting random_seed.
2001-10-19 02:44:08 +02:00
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
/*
Modify backend switch parsing to prevent 'insecure' switches from being accepted when they are passed from client connection request. Get rid of a couple that no longer do anything (like -P).
1999-05-22 19:47:54 +02:00
* Now, build the argv vector that will be given to PostgresMain.
*
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
* The maximum possible number of commandline arguments that could come
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
* from ExtraOptions is (strlen(ExtraOptions) + 1) / 2; see
* pg_split_opts().
Modify backend switch parsing to prevent 'insecure' switches from being accepted when they are passed from client connection request. Get rid of a couple that no longer do anything (like -P).
1999-05-22 19:47:54 +02:00
*/
Fix insecure parsing of server command-line switches. An oversight in commit e710b65c1c56ca7b91f662c63d37ff2e72862a94 allowed database names beginning with "-" to be treated as though they were secure command-line switches; and this switch processing occurs before client authentication, so that even an unprivileged remote attacker could exploit the bug, needing only connectivity to the postmaster's port. Assorted exploits for this are possible, some requiring a valid database login, some not. The worst known problem is that the "-r" switch can be invoked to redirect the process's stderr output, so that subsequent error messages will be appended to any file the server can write. This can for example be used to corrupt the server's configuration files, so that it will fail when next restarted. Complete destruction of database tables is also possible. Fix by keeping the database name extracted from a startup packet fully separate from command-line switches, as had already been done with the user name field. The Postgres project thanks Mitsumasa Kondo for discovering this bug, Kyotaro Horiguchi for drafting the fix, and Noah Misch for recognizing the full extent of the danger. Security: CVE-2013-1899
2013-04-01 20:00:51 +02:00
maxac = 2; /* for fixed args supplied below */
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
maxac += (strlen(ExtraOptions) + 1) / 2;
av = (char **) MemoryContextAlloc(TopMemoryContext,
maxac * sizeof(char *));
ac = 0;
Remove fork()/exec() and only do fork(). Small cleanups.
1998-05-29 19:00:34 +02:00
New ps display code, works on more platforms. Install a default configuration file. Clean up some funny business in the config file code.
2000-06-04 03:44:38 +02:00
av[ac++] = "postgres";
Remove fork()/exec() and only do fork(). Small cleanups.
1998-05-29 19:00:34 +02:00
Modify backend switch parsing to prevent 'insecure' switches from being accepted when they are passed from client connection request. Get rid of a couple that no longer do anything (like -P).
1999-05-22 19:47:54 +02:00
/*
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
* Pass any backend switches specified with -o on the postmaster's own
Allow escaping of option values for options passed at connection start. This is useful to allow to set GUCs to values that include spaces; something that wasn't previously possible. The primary case motivating this is the desire to set default_transaction_isolation to 'repeatable read' on a per connection basis, but other usecases like seach_path do also exist. This introduces a slight backward incompatibility: Previously a \ in an option value would have been passed on literally, now it'll be taken as an escape. The relevant mailing list discussion starts with 20140204125823.GJ12016@awork2.anarazel.de.
2014-08-28 13:59:29 +02:00
* command line. We assume these are secure.
Modify backend switch parsing to prevent 'insecure' switches from being accepted when they are passed from client connection request. Get rid of a couple that no longer do anything (like -P).
1999-05-22 19:47:54 +02:00
*/
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
pg_split_opts(av, &ac, ExtraOptions);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
More janitorial work: remove the explicit casting of NULL literals to a pointer type when it is not necessary to do so. For future reference, casting NULL to a pointer type is only necessary when (a) invoking a function AND either (b) the function has no prototype OR (c) the function is a varargs function.
2004-01-07 19:56:30 +01:00
av[ac] = NULL;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
First phase of FE/BE protocol modifications: new StartupPacket layout with variable-width fields. No more truncation of long user names. Also, libpq can now send its environment-variable-driven SET commands as part of the startup packet, saving round trips to server.
2003-04-18 00:26:02 +02:00
Assert(ac < maxac);
First phase of memory management rewrite (see backend/utils/mmgr/README for details). It doesn't really do that much yet, since there are no short-term memory contexts in the executor, but the infrastructure is in place and long-term contexts are handled reasonably. A few long- standing bugs have been fixed, such as 'VACUUM; anything' in a single query string crashing. Also, out-of-memory is now considered a recoverable ERROR, not FATAL. Eliminate a large amount of crufty, now-dead code in and around memory management. Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and backend startup.
2000-06-28 05:33:33 +02:00
/*
* Debug: print arguments being passed to backend
*/
Change some frequently-reached elog(DEBUG...) calls to ereport(DEBUG...) for speed reasons. (ereport falls out much more quickly when no output is needed than elog does.)
2003-08-12 20:23:21 +02:00
ereport(DEBUG3,
(errmsg_internal("%s child[%d]: starting with (",
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
progname, (int) getpid())));
Commit to match discussed elog() changes. Only update is that LOG is now just below FATAL in server_min_messages. Added more text to highlight ordering difference between it and client_min_messages. --------------------------------------------------------------------------- REALLYFATAL => PANIC STOP => PANIC New INFO level the prints to client by default New LOG level the prints to server log by default Cause VACUUM information to print only to the client NOTICE => INFO where purely information messages are sent DEBUG => LOG for purely server status messages DEBUG removed, kept as backward compatible DEBUG5, DEBUG4, DEBUG3, DEBUG2, DEBUG1 added DebugLvl removed in favor of new DEBUG[1-5] symbols New server_min_messages GUC parameter with values: DEBUG[5-1], INFO, NOTICE, ERROR, LOG, FATAL, PANIC New client_min_messages GUC parameter with values: DEBUG[5-1], LOG, INFO, NOTICE, ERROR, FATAL, PANIC Server startup now logged with LOG instead of DEBUG Remove debug_level GUC parameter elog() numbers now start at 10 Add test to print error message if older elog() values are passed to elog() Bootstrap mode now has a -d that requires an argument, like postmaster
2002-03-02 22:39:36 +01:00
for (i = 0; i < ac; ++i)
Change some frequently-reached elog(DEBUG...) calls to ereport(DEBUG...) for speed reasons. (ereport falls out much more quickly when no output is needed than elog does.)
2003-08-12 20:23:21 +02:00
ereport(DEBUG3,
(errmsg_internal("\t%s", av[i])));
ereport(DEBUG3,
(errmsg_internal(")")));
Further work on elog cleanup: fix some bogosities in elog's logic about when to send what to which, prevent recursion by introducing new COMMERROR elog level for client-communication problems, get rid of direct writes to stderr in backend/libpq files, prevent non-error elogs from going to client during the authentication cycle.
2002-03-04 02:46:04 +01:00
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
/*
* Make sure we aren't in PostmasterContext anymore. (We can't delete it
* just yet, though, because InitPostgres will need the HBA data.)
*/
MemoryContextSwitchTo(TopMemoryContext);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Fix insecure parsing of server command-line switches. An oversight in commit e710b65c1c56ca7b91f662c63d37ff2e72862a94 allowed database names beginning with "-" to be treated as though they were secure command-line switches; and this switch processing occurs before client authentication, so that even an unprivileged remote attacker could exploit the bug, needing only connectivity to the postmaster's port. Assorted exploits for this are possible, some requiring a valid database login, some not. The worst known problem is that the "-r" switch can be invoked to redirect the process's stderr output, so that subsequent error messages will be appended to any file the server can write. This can for example be used to corrupt the server's configuration files, so that it will fail when next restarted. Complete destruction of database tables is also possible. Fix by keeping the database name extracted from a startup packet fully separate from command-line switches, as had already been done with the user name field. The Postgres project thanks Mitsumasa Kondo for discovering this bug, Kyotaro Horiguchi for drafting the fix, and Noah Misch for recognizing the full extent of the danger. Security: CVE-2013-1899
2013-04-01 20:00:51 +02:00
PostgresMain(ac, av, port->database_name, port->user_name);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
#ifdef EXEC_BACKEND
/*
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
* postmaster_forkexec -- fork and exec a postmaster subprocess
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
*
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
* The caller must have set up the argv array already, except for argv[2]
* which will be filled with the name of the temp variable file.
*
* Returns the child process PID, or -1 on fork failure (a suitable error
* message has been logged on failure).
*
* All uses of this routine will dispatch to SubPostmasterMain in the
* child process.
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
*/
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
pid_t
postmaster_forkexec(int argc, char *argv[])
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
{
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
Port port;
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/* This entry point passes dummy values for the Port variables */
memset(&port, 0, sizeof(port));
return internal_forkexec(argc, argv, &port);
}
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/*
* backend_forkexec -- fork/exec off a backend process
*
Add comment about the use of EXEC_BACKEND.
2008-09-23 22:35:38 +02:00
* Some operating systems (WIN32) don't have fork() so we have to simulate
* it by storing parameters that need to be passed to the child and
* then create a new child process.
*
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
* returns the pid of the fork/exec'd process, or -1 on failure
*/
static pid_t
backend_forkexec(Port *port)
{
char *av[4];
int ac = 0;
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
av[ac++] = "postgres";
Merge postmaster and postgres command into just postgres. postmaster symlink is kept for now for compatibility. To call single-user mode, use postgres --single.
2006-06-18 17:38:37 +02:00
av[ac++] = "--forkbackend";
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
av[ac++] = NULL; /* filled in by internal_forkexec */
Review uses of IsUnderPostmaster, change some tests to look at whereToSendOutput instead because they are really inquiring about the correct client communication protocol. Update some comments. This is pointing towards supporting regular FE/BE client protocol in a standalone backend, per discussion a month or so back.
2004-01-28 22:02:40 +01:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
av[ac] = NULL;
Assert(ac < lengthof(av));
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
return internal_forkexec(ac, av, port);
}
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#ifndef WIN32
/*
* internal_forkexec non-win32 implementation
*
* - writes out backend variables to the parameter file
* - fork():s, and then exec():s the child process
*/
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
static pid_t
internal_forkexec(int argc, char *argv[], Port *port)
{
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
static unsigned long tmpBackendFileNum = 0;
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
pid_t pid;
char tmpfilename[MAXPGPATH];
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
BackendParameters param;
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
FILE *fp;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
if (!save_backend_variables(&param, port))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
Arrange for the postmaster (and standalone backends, initdb, etc) to chdir into PGDATA and subsequently use relative paths instead of absolute paths to access all files under PGDATA. This seems to give a small performance improvement, and it should make the system more robust against naive DBAs doing things like moving a database directory that has a live postmaster in it. Per recent discussion.
2005-07-04 06:51:52 +02:00
snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
MyProcPid, ++tmpBackendFileNum);
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/* Open file */
fp = AllocateFile(tmpfilename, PG_BINARY_W);
if (!fp)
{
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
* As in OpenTemporaryFileInTablespace, try to make the temp-file
Refactor dir/file permissions Consolidate directory and file create permissions for tools which work with the PG data directory by adding a new module (common/file_perm.c) that contains variables (pg_file_create_mode, pg_dir_create_mode) and constants to initialize them (0600 for files and 0700 for directories). Convert mkdir() calls in the backend to MakePGDirectory() if the original call used default permissions (always the case for regular PG directories). Add tests to make sure permissions in PGDATA are set correctly by the tools which modify the PG data directory. Authors: David Steele <david@pgmasters.net>, Adam Brightwell <adam.brightwell@crunchydata.com> Reviewed-By: Michael Paquier, with discussion amongst many others. Discussion: https://postgr.es/m/ad346fe6-b23e-59f1-ecb7-0e08390ad629%40pgmasters.net
2018-04-07 23:45:39 +02:00
* directory, ignoring errors.
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
*/
Refactor dir/file permissions Consolidate directory and file create permissions for tools which work with the PG data directory by adding a new module (common/file_perm.c) that contains variables (pg_file_create_mode, pg_dir_create_mode) and constants to initialize them (0600 for files and 0700 for directories). Convert mkdir() calls in the backend to MakePGDirectory() if the original call used default permissions (always the case for regular PG directories). Add tests to make sure permissions in PGDATA are set correctly by the tools which modify the PG data directory. Authors: David Steele <david@pgmasters.net>, Adam Brightwell <adam.brightwell@crunchydata.com> Reviewed-By: Michael Paquier, with discussion amongst many others. Discussion: https://postgr.es/m/ad346fe6-b23e-59f1-ecb7-0e08390ad629%40pgmasters.net
2018-04-07 23:45:39 +02:00
(void) MakePGDirectory(PG_TEMP_FILES_DIR);
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
fp = AllocateFile(tmpfilename, PG_BINARY_W);
if (!fp)
{
ereport(LOG,
(errcode_for_file_access(),
errmsg("could not create file \"%s\": %m",
tmpfilename)));
return -1;
}
}
if (fwrite(&param, sizeof(param), 1, fp) != 1)
{
ereport(LOG,
(errcode_for_file_access(),
errmsg("could not write to file \"%s\": %m", tmpfilename)));
FreeFile(fp);
return -1;
}
/* Release file */
if (FreeFile(fp))
{
ereport(LOG,
(errcode_for_file_access(),
errmsg("could not write to file \"%s\": %m", tmpfilename)));
return -1;
}
Corrects a typo, introduces missing variables, and rearranges the initialization of stats process under EXEC_BACKEND. [A cleaner, rationalized approach to stat/backend/SSDataBase child processes under EXEC_BACKEND is on my TODO list. However this patch takes care of immediate concerns (ie. stats test now passes under win32)] Claudio Natoli
2004-03-09 06:11:53 +01:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/* Make sure caller set up argv properly */
Assert(argc >= 3);
Assert(argv[argc] == NULL);
Merge postmaster and postgres command into just postgres. postmaster symlink is kept for now for compatibility. To call single-user mode, use postgres --single.
2006-06-18 17:38:37 +02:00
Assert(strncmp(argv[1], "--fork", 6) == 0);
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Assert(argv[2] == NULL);
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Merge postmaster and postgres command into just postgres. postmaster symlink is kept for now for compatibility. To call single-user mode, use postgres --single.
2006-06-18 17:38:37 +02:00
/* Insert temp file name after --fork argument */
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
argv[2] = tmpfilename;
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/* Fire off execv in child */
Refactor fork()-related code. We need to do various housekeeping tasks before we can invoke fork() -- flush stdio buffers, save and restore the profiling timer on Linux with LINUX_PROFILE, and handle BeOS stuff. This patch moves that code into a single function, fork_process(), instead of duplicating it at the various callsites of fork(). This patch doesn't address the EXEC_BACKEND case; there is room for further cleanup there.
2005-03-10 08:14:03 +01:00
if ((pid = fork_process()) == 0)
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
{
if (execv(postgres_exec_path, argv) < 0)
{
ereport(LOG,
Clarify some error messages
2004-11-09 14:01:27 +01:00
(errmsg("could not execute server process \"%s\": %m",
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
postgres_exec_path)));
/* We're already in the child process here, can't return */
exit(1);
}
}
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
return pid; /* Parent returns pid, or -1 on fork failure */
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
}
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
#else /* WIN32 */
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/*
* internal_forkexec win32 implementation
*
* - starts backend using CreateProcess(), in suspended state
* - writes out backend variables to the parameter file
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* - during this, duplicates handles and sockets required for
* inheritance into the new process
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
* - resumes execution of the new process once the backend parameter
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* file is complete.
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
*/
static pid_t
internal_forkexec(int argc, char *argv[], Port *port)
{
On Windows, retry process creation if we fail to reserve shared memory. We've heard occasional reports of backend launch failing because pgwin32_ReserveSharedMemoryRegion() fails, indicating that something has already used that address space in the child process. It's not very clear what, given that we disable ASLR in Windows builds, but suspicion falls on antivirus products. It'd be better if we didn't have to disable ASLR, anyway. So let's try to ameliorate the problem by retrying the process launch after such a failure, up to 100 times. Patch by me, based on previous work by Amit Kapila and others. This is a longstanding issue, so back-patch to all supported branches. Discussion: https://postgr.es/m/CAA4eK1+R6hSx6t_yvwtx+NRzneVp+MRqXAdGJZChcau8Uij-8g@mail.gmail.com
2017-07-10 17:00:09 +02:00
int retry_count = 0;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
STARTUPINFO si;
PROCESS_INFORMATION pi;
int i;
int j;
char cmdLine[MAXPGPATH * 2];
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
HANDLE paramHandle;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
BackendParameters *param;
SECURITY_ATTRIBUTES sa;
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
char paramHandleStr[32];
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
win32_deadchild_waitinfo *childinfo;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/* Make sure caller set up argv properly */
Assert(argc >= 3);
Assert(argv[argc] == NULL);
Merge postmaster and postgres command into just postgres. postmaster symlink is kept for now for compatibility. To call single-user mode, use postgres --single.
2006-06-18 17:38:37 +02:00
Assert(strncmp(argv[1], "--fork", 6) == 0);
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
Assert(argv[2] == NULL);
On Windows, retry process creation if we fail to reserve shared memory. We've heard occasional reports of backend launch failing because pgwin32_ReserveSharedMemoryRegion() fails, indicating that something has already used that address space in the child process. It's not very clear what, given that we disable ASLR in Windows builds, but suspicion falls on antivirus products. It'd be better if we didn't have to disable ASLR, anyway. So let's try to ameliorate the problem by retrying the process launch after such a failure, up to 100 times. Patch by me, based on previous work by Amit Kapila and others. This is a longstanding issue, so back-patch to all supported branches. Discussion: https://postgr.es/m/CAA4eK1+R6hSx6t_yvwtx+NRzneVp+MRqXAdGJZChcau8Uij-8g@mail.gmail.com
2017-07-10 17:00:09 +02:00
/* Resume here if we need to retry */
retry:
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/* Set up shared memory for parameter passing */
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
ZeroMemory(&sa, sizeof(sa));
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
sa.nLength = sizeof(sa);
sa.bInheritHandle = TRUE;
paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
&sa,
PAGE_READWRITE,
0,
sizeof(BackendParameters),
NULL);
if (paramHandle == INVALID_HANDLE_VALUE)
{
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
elog(LOG, "could not create backend parameter file mapping: error code %lu",
GetLastError());
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
return -1;
}
param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
if (!param)
{
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
elog(LOG, "could not map backend parameter memory: error code %lu",
GetLastError());
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
CloseHandle(paramHandle);
return -1;
}
Merge postmaster and postgres command into just postgres. postmaster symlink is kept for now for compatibility. To call single-user mode, use postgres --single.
2006-06-18 17:38:37 +02:00
/* Insert temp file name after --fork argument */
Send paramHandle to subprocesses as 64-bit on Win64 The handle to the shared memory segment containing startup parameters was sent as 32-bit even on 64-bit systems. Since HANDLEs appear to be allocated sequentially this shouldn't be a problem until we reach 2^32 open handles in the postmaster, but a 64-bit value should be sent across as 64-bit, and not zero out the top 32 bits. Noted by Tom Lane.
2010-11-16 12:40:56 +01:00
#ifdef _WIN64
Revert "Remove hard coded formats for INT64 and use configured settings instead." This reverts commit 9b1508af8971c1627cda5bb65f5e9eddb9a1a55e. As requested by Tom.
2011-04-27 17:28:14 +02:00
sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
Send paramHandle to subprocesses as 64-bit on Win64 The handle to the shared memory segment containing startup parameters was sent as 32-bit even on 64-bit systems. Since HANDLEs appear to be allocated sequentially this shouldn't be a problem until we reach 2^32 open handles in the postmaster, but a 64-bit value should be sent across as 64-bit, and not zero out the top 32 bits. Noted by Tom Lane.
2010-11-16 12:40:56 +01:00
#else
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
Send paramHandle to subprocesses as 64-bit on Win64 The handle to the shared memory segment containing startup parameters was sent as 32-bit even on 64-bit systems. Since HANDLEs appear to be allocated sequentially this shouldn't be a problem until we reach 2^32 open handles in the postmaster, but a 64-bit value should be sent across as 64-bit, and not zero out the top 32 bits. Noted by Tom Lane.
2010-11-16 12:40:56 +01:00
#endif
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
argv[2] = paramHandleStr;
/* Format the cmd line */
cmdLine[sizeof(cmdLine) - 1] = '\0';
cmdLine[sizeof(cmdLine) - 2] = '\0';
snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
i = 0;
while (argv[++i] != NULL)
{
j = strlen(cmdLine);
snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
}
if (cmdLine[sizeof(cmdLine) - 2] != '\0')
{
elog(LOG, "subprocess command line too long");
Fix some memory leaks and improve restricted token handling on Windows The leaks have been detected by a Coverity run on Windows. No backpatch is done as the leaks are minor. While on it, make restricted token creation more consistent in its error handling by logging an error instead of a warning if missing advapi32.dll, which was missing in the NT4 days. Any modern platform should have this DLL around. Now, if the library is not there, an error is still reported back to the caller, and nothing is done do there is no behavior change done in this commit. Author: Ranier Vilela Discussion: https://postgr.es/m/CAEudQApa9MG0foPkgPX87fipk=vhnF2Xfg+CfUyR08h4R7Mywg@mail.gmail.com
2020-01-27 03:02:05 +01:00
UnmapViewOfFile(param);
CloseHandle(paramHandle);
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
return -1;
}
memset(&pi, 0, sizeof(pi));
memset(&si, 0, sizeof(si));
si.cb = sizeof(si);
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Create the subprocess in a suspended state. This will be resumed later,
* once we have written out the parameter file.
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
*/
if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
NULL, NULL, &si, &pi))
{
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
elog(LOG, "CreateProcess call failed: %m (error code %lu)",
GetLastError());
Fix some memory leaks and improve restricted token handling on Windows The leaks have been detected by a Coverity run on Windows. No backpatch is done as the leaks are minor. While on it, make restricted token creation more consistent in its error handling by logging an error instead of a warning if missing advapi32.dll, which was missing in the NT4 days. Any modern platform should have this DLL around. Now, if the library is not there, an error is still reported back to the caller, and nothing is done do there is no behavior change done in this commit. Author: Ranier Vilela Discussion: https://postgr.es/m/CAEudQApa9MG0foPkgPX87fipk=vhnF2Xfg+CfUyR08h4R7Mywg@mail.gmail.com
2020-01-27 03:02:05 +01:00
UnmapViewOfFile(param);
CloseHandle(paramHandle);
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
return -1;
}
if (!save_backend_variables(param, port, pi.hProcess, pi.dwProcessId))
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* log made by save_backend_variables, but we have to clean up the
* mess with the half-started process
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
*/
if (!TerminateProcess(pi.hProcess, 255))
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
ereport(LOG,
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
(errmsg_internal("could not terminate unstarted process: error code %lu",
GetLastError())));
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
CloseHandle(pi.hProcess);
CloseHandle(pi.hThread);
Fix some memory leaks and improve restricted token handling on Windows The leaks have been detected by a Coverity run on Windows. No backpatch is done as the leaks are minor. While on it, make restricted token creation more consistent in its error handling by logging an error instead of a warning if missing advapi32.dll, which was missing in the NT4 days. Any modern platform should have this DLL around. Now, if the library is not there, an error is still reported back to the caller, and nothing is done do there is no behavior change done in this commit. Author: Ranier Vilela Discussion: https://postgr.es/m/CAEudQApa9MG0foPkgPX87fipk=vhnF2Xfg+CfUyR08h4R7Mywg@mail.gmail.com
2020-01-27 03:02:05 +01:00
UnmapViewOfFile(param);
CloseHandle(paramHandle);
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
return -1; /* log made by save_backend_variables */
}
Reserve the shared memory region during backend startup on Windows, so that memory allocated by starting third party DLLs doesn't end up conflicting with it. Hopefully this solves the long-time issue with "could not reattach to shared memory" errors on Win32. Patch from Tsutomu Yamada and me, based on idea from Trevor Talbot.
2009-07-24 22:12:42 +02:00
/* Drop the parameter shared memory that is now inherited to the backend */
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
if (!UnmapViewOfFile(param))
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
elog(LOG, "could not unmap view of backend parameter file: error code %lu",
GetLastError());
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
if (!CloseHandle(paramHandle))
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
elog(LOG, "could not close handle to backend parameter file: error code %lu",
GetLastError());
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
Reserve the shared memory region during backend startup on Windows, so that memory allocated by starting third party DLLs doesn't end up conflicting with it. Hopefully this solves the long-time issue with "could not reattach to shared memory" errors on Win32. Patch from Tsutomu Yamada and me, based on idea from Trevor Talbot.
2009-07-24 22:12:42 +02:00
/*
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* Reserve the memory region used by our main shared memory segment before
On Windows, retry process creation if we fail to reserve shared memory. We've heard occasional reports of backend launch failing because pgwin32_ReserveSharedMemoryRegion() fails, indicating that something has already used that address space in the child process. It's not very clear what, given that we disable ASLR in Windows builds, but suspicion falls on antivirus products. It'd be better if we didn't have to disable ASLR, anyway. So let's try to ameliorate the problem by retrying the process launch after such a failure, up to 100 times. Patch by me, based on previous work by Amit Kapila and others. This is a longstanding issue, so back-patch to all supported branches. Discussion: https://postgr.es/m/CAA4eK1+R6hSx6t_yvwtx+NRzneVp+MRqXAdGJZChcau8Uij-8g@mail.gmail.com
2017-07-10 17:00:09 +02:00
* we resume the child process. Normally this should succeed, but if ASLR
* is active then it might sometimes fail due to the stack or heap having
* gotten mapped into that range. In that case, just terminate the
* process and retry.
Reserve the shared memory region during backend startup on Windows, so that memory allocated by starting third party DLLs doesn't end up conflicting with it. Hopefully this solves the long-time issue with "could not reattach to shared memory" errors on Win32. Patch from Tsutomu Yamada and me, based on idea from Trevor Talbot.
2009-07-24 22:12:42 +02:00
*/
if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
{
On Windows, retry process creation if we fail to reserve shared memory. We've heard occasional reports of backend launch failing because pgwin32_ReserveSharedMemoryRegion() fails, indicating that something has already used that address space in the child process. It's not very clear what, given that we disable ASLR in Windows builds, but suspicion falls on antivirus products. It'd be better if we didn't have to disable ASLR, anyway. So let's try to ameliorate the problem by retrying the process launch after such a failure, up to 100 times. Patch by me, based on previous work by Amit Kapila and others. This is a longstanding issue, so back-patch to all supported branches. Discussion: https://postgr.es/m/CAA4eK1+R6hSx6t_yvwtx+NRzneVp+MRqXAdGJZChcau8Uij-8g@mail.gmail.com
2017-07-10 17:00:09 +02:00
/* pgwin32_ReserveSharedMemoryRegion already made a log entry */
Reserve the shared memory region during backend startup on Windows, so that memory allocated by starting third party DLLs doesn't end up conflicting with it. Hopefully this solves the long-time issue with "could not reattach to shared memory" errors on Win32. Patch from Tsutomu Yamada and me, based on idea from Trevor Talbot.
2009-07-24 22:12:42 +02:00
if (!TerminateProcess(pi.hProcess, 255))
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
ereport(LOG,
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
(errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
GetLastError())));
Reserve the shared memory region during backend startup on Windows, so that memory allocated by starting third party DLLs doesn't end up conflicting with it. Hopefully this solves the long-time issue with "could not reattach to shared memory" errors on Win32. Patch from Tsutomu Yamada and me, based on idea from Trevor Talbot.
2009-07-24 22:12:42 +02:00
CloseHandle(pi.hProcess);
CloseHandle(pi.hThread);
On Windows, retry process creation if we fail to reserve shared memory. We've heard occasional reports of backend launch failing because pgwin32_ReserveSharedMemoryRegion() fails, indicating that something has already used that address space in the child process. It's not very clear what, given that we disable ASLR in Windows builds, but suspicion falls on antivirus products. It'd be better if we didn't have to disable ASLR, anyway. So let's try to ameliorate the problem by retrying the process launch after such a failure, up to 100 times. Patch by me, based on previous work by Amit Kapila and others. This is a longstanding issue, so back-patch to all supported branches. Discussion: https://postgr.es/m/CAA4eK1+R6hSx6t_yvwtx+NRzneVp+MRqXAdGJZChcau8Uij-8g@mail.gmail.com
2017-07-10 17:00:09 +02:00
if (++retry_count < 100)
goto retry;
ereport(LOG,
(errmsg("giving up after too many tries to reserve shared memory"),
errhint("This might be caused by ASLR or antivirus software.")));
return -1;
Reserve the shared memory region during backend startup on Windows, so that memory allocated by starting third party DLLs doesn't end up conflicting with it. Hopefully this solves the long-time issue with "could not reattach to shared memory" errors on Win32. Patch from Tsutomu Yamada and me, based on idea from Trevor Talbot.
2009-07-24 22:12:42 +02:00
}
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Now that the backend variables are written out, we start the child
* thread so it can start initializing while we set up the rest of the
* parent state.
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
*/
if (ResumeThread(pi.hThread) == -1)
{
if (!TerminateProcess(pi.hProcess, 255))
{
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
ereport(LOG,
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
(errmsg_internal("could not terminate unstartable process: error code %lu",
GetLastError())));
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
CloseHandle(pi.hProcess);
CloseHandle(pi.hThread);
return -1;
}
CloseHandle(pi.hProcess);
CloseHandle(pi.hThread);
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
ereport(LOG,
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
(errmsg_internal("could not resume thread of unstarted process: error code %lu",
GetLastError())));
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
return -1;
}
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
/*
Update stale comments, and fix comment typos.
2019-06-08 19:12:26 +02:00
* Queue a waiter to signal when this child dies. The wait will be handled
* automatically by an operating system thread pool.
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* Note: use malloc instead of palloc, since it needs to be thread-safe.
* Struct will be free():d from the callback function that runs on a
* different thread.
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
*/
childinfo = malloc(sizeof(win32_deadchild_waitinfo));
if (!childinfo)
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
ereport(FATAL,
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
(errcode(ERRCODE_OUT_OF_MEMORY),
errmsg("out of memory")));
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
childinfo->procHandle = pi.hProcess;
childinfo->procId = pi.dwProcessId;
if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
pi.hProcess,
pgwin32_deadchild_callback,
childinfo,
INFINITE,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
ereport(FATAL,
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
(errmsg_internal("could not register process for wait: error code %lu",
GetLastError())));
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
/* Don't close pi.hProcess here - the wait thread needs access to it */
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
CloseHandle(pi.hThread);
return pi.dwProcessId;
}
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
#endif /* WIN32 */
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
/*
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
* SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
* to what it would be if we'd simply forked on Unix, and then
* dispatch to the appropriate place.
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
*
Merge postmaster and postgres command into just postgres. postmaster symlink is kept for now for compatibility. To call single-user mode, use postgres --single.
2006-06-18 17:38:37 +02:00
* The first two command line arguments are expected to be "--forkFOO"
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
* (where FOO indicates which postmaster child we are to become), and
* the name of a variables file that we can read to load data that would
* have been inherited by fork() on Unix. Remaining arguments go to the
* subprocess FooMain() routine.
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
*/
Unify calling conventions for postgres/postmaster sub-main functions There was a wild mix of calling conventions: Some were declared to return void and didn't return, some returned an int exit code, some claimed to return an exit code, which the callers checked, but actually never returned, and so on. Now all of these functions are declared to return void and decorated with attribute noreturn and don't return. That's easiest, and most code already worked that way.
2012-06-25 20:25:26 +02:00
void
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
SubPostmasterMain(int argc, char *argv[])
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
{
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Port port;
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
/* In EXEC_BACKEND case we will not have inherited these settings */
IsPostmasterEnvironment = true;
Fix one overlooked ocurrence of "None" in EXEC_BACKEND block.
2005-11-03 21:02:50 +01:00
whereToSendOutput = DestNone;
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
Commonalize process startup code. Move common code, that was duplicated in every postmaster child/every standalone process, into two functions in miscinit.c. Not only does that already result in a fair amount of net code reduction but it also makes it much easier to remove more duplication in the future. The prime motivation wasn't code deduplication though, but easier addition of new common code.
2015-01-13 13:12:37 +01:00
/* Setup as postmaster child */
InitPostmasterChild();
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
/* Setup essential subsystems (to ensure elog() behaves sanely) */
InitializeGUCOptions();
Do ClosePostmasterPorts() earlier in SubPostmasterMain(). In standard Unix builds, postmaster child processes do ClosePostmasterPorts immediately after InitPostmasterChild, that is almost immediately after being spawned. This is important because we don't want children holding open the postmaster's end of the postmaster death watch pipe. However, in EXEC_BACKEND builds, SubPostmasterMain was postponing this responsibility significantly, in order to make it slightly more convenient to pass the right flag value to ClosePostmasterPorts. This is bad, particularly seeing that process_shared_preload_libraries() might invoke nearly-arbitrary code. Rearrange so that we do it as soon as we've fetched the socket FDs via read_backend_variables(). Also move the comment explaining about randomize_va_space to before the call of PGSharedMemoryReAttach, which is where it's relevant. The old placement was appropriate when the reattach happened inside CreateSharedMemoryAndSemaphores, but that was a long time ago. Back-patch to 9.3; the patch doesn't apply cleanly before that, and it doesn't seem worth a lot of effort given that we've had no actual field complaints traceable to this. Discussion: <4157.1475178360@sss.pgh.pa.us>
2016-10-01 23:15:09 +02:00
/* Check we got appropriate args */
if (argc < 3)
elog(FATAL, "invalid subpostmaster invocation");
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
/* Read in the variables file */
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
memset(&port, 0, sizeof(Port));
read_backend_variables(argv[2], &port);
Do ClosePostmasterPorts() earlier in SubPostmasterMain(). In standard Unix builds, postmaster child processes do ClosePostmasterPorts immediately after InitPostmasterChild, that is almost immediately after being spawned. This is important because we don't want children holding open the postmaster's end of the postmaster death watch pipe. However, in EXEC_BACKEND builds, SubPostmasterMain was postponing this responsibility significantly, in order to make it slightly more convenient to pass the right flag value to ClosePostmasterPorts. This is bad, particularly seeing that process_shared_preload_libraries() might invoke nearly-arbitrary code. Rearrange so that we do it as soon as we've fetched the socket FDs via read_backend_variables(). Also move the comment explaining about randomize_va_space to before the call of PGSharedMemoryReAttach, which is where it's relevant. The old placement was appropriate when the reattach happened inside CreateSharedMemoryAndSemaphores, but that was a long time ago. Back-patch to 9.3; the patch doesn't apply cleanly before that, and it doesn't seem worth a lot of effort given that we've had no actual field complaints traceable to this. Discussion: <4157.1475178360@sss.pgh.pa.us>
2016-10-01 23:15:09 +02:00
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
Do stack-depth checking in all postmaster children. We used to only initialize the stack base pointer when starting up a regular backend, not in other processes. In particular, autovacuum workers can run arbitrary user code, and without stack-depth checking, infinite recursion in e.g an index expression will bring down the whole cluster. The comment about PL/Java using set_stack_base() is not yet true. As the code stands, PL/java still modifies the stack_base_ptr variable directly. However, it's been discussed in the PL/Java mailing list that it should be changed to use the function, because PL/Java is currently oblivious to the register stack used on Itanium. There's another issues with PL/Java, namely that the stack base pointer it sets is not really the base of the stack, it could be something close to the bottom of the stack. That's a separate issue that might need some further changes to this code, but that's a different story. Backpatch to all supported releases.
2012-04-08 17:28:12 +02:00
/*
* Set reference point for stack-depth checking
*/
set_stack_base();
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
/*
* Set up memory area for GSS information. Mirrors the code in ConnCreate
* for the non-exec case.
SSPI authentication on Windows. GSSAPI compatible client when doing Kerberos against a Unix server, and Windows-specific server-side authentication using SSPI "negotiate" method (Kerberos or NTLM). Only builds properly with MSVC for now.
2007-07-23 12:16:54 +02:00
*/
#if defined(ENABLE_GSS) || defined(ENABLE_SSPI)
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
port.gss = (pg_gssinfo *) calloc(1, sizeof(pg_gssinfo));
SSPI authentication on Windows. GSSAPI compatible client when doing Kerberos against a Unix server, and Windows-specific server-side authentication using SSPI "negotiate" method (Kerberos or NTLM). Only builds properly with MSVC for now.
2007-07-23 12:16:54 +02:00
if (!port.gss)
ereport(FATAL,
(errcode(ERRCODE_OUT_OF_MEMORY),
errmsg("out of memory")));
#endif
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If appropriate, physically re-attach to shared memory segment. We want
* to do this before going any further to ensure that we can attach at the
On Windows, ensure shared memory handle gets closed if not being used. Postmaster child processes that aren't supposed to be attached to shared memory were not bothering to close the shared memory mapping handle they inherit from the postmaster process. That's mostly harmless, since the handle vanishes anyway when the child process exits -- but the syslogger process, if used, doesn't get killed and restarted during recovery from a backend crash. That meant that Windows doesn't see the shared memory mapping as becoming free, so it doesn't delete it and the postmaster is unable to create a new one, resulting in failure to recover from crashes whenever logging_collector is turned on. Per report from Dmitry Vasilyev. It's a bit astonishing that we'd not figured this out long ago, since it's been broken from the very beginnings of out native Windows support; probably some previously-unexplained trouble reports trace to this. A secondary problem is that on Cygwin (perhaps only in older versions?), exec() may not detach from the shared memory segment after all, in which case these child processes did remain attached to shared memory, posing the risk of an unexpected shared memory clobber if they went off the rails somehow. That may be a long-gone bug, but we can deal with it now if it's still live, by detaching within the infrastructure introduced here to deal with closing the handle. Back-patch to all supported branches. Tom Lane and Amit Kapila
2015-10-13 17:21:33 +02:00
* same address the postmaster used. On the other hand, if we choose not
* to re-attach, we may have other cleanup to do.
Do ClosePostmasterPorts() earlier in SubPostmasterMain(). In standard Unix builds, postmaster child processes do ClosePostmasterPorts immediately after InitPostmasterChild, that is almost immediately after being spawned. This is important because we don't want children holding open the postmaster's end of the postmaster death watch pipe. However, in EXEC_BACKEND builds, SubPostmasterMain was postponing this responsibility significantly, in order to make it slightly more convenient to pass the right flag value to ClosePostmasterPorts. This is bad, particularly seeing that process_shared_preload_libraries() might invoke nearly-arbitrary code. Rearrange so that we do it as soon as we've fetched the socket FDs via read_backend_variables(). Also move the comment explaining about randomize_va_space to before the call of PGSharedMemoryReAttach, which is where it's relevant. The old placement was appropriate when the reattach happened inside CreateSharedMemoryAndSemaphores, but that was a long time ago. Back-patch to 9.3; the patch doesn't apply cleanly before that, and it doesn't seem worth a lot of effort given that we've had no actual field complaints traceable to this. Discussion: <4157.1475178360@sss.pgh.pa.us>
2016-10-01 23:15:09 +02:00
*
* If testing EXEC_BACKEND on Linux, you should run this as root before
* starting the postmaster:
*
* echo 0 >/proc/sys/kernel/randomize_va_space
*
* This prevents using randomized stack and code addresses that cause the
* child process's memory map to be different from the parent's, making it
* sometimes impossible to attach to shared memory at the desired address.
* Return the setting to its old value (usually '1' or '2') when finished.
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
*/
Merge postmaster and postgres command into just postgres. postmaster symlink is kept for now for compatibility. To call single-user mode, use postgres --single.
2006-06-18 17:38:37 +02:00
if (strcmp(argv[1], "--forkbackend") == 0 ||
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
strcmp(argv[1], "--forkavlauncher") == 0 ||
strcmp(argv[1], "--forkavworker") == 0 ||
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
strcmp(argv[1], "--forkboot") == 0 ||
strncmp(argv[1], "--forkbgworker=", 15) == 0)
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
PGSharedMemoryReAttach();
On Windows, ensure shared memory handle gets closed if not being used. Postmaster child processes that aren't supposed to be attached to shared memory were not bothering to close the shared memory mapping handle they inherit from the postmaster process. That's mostly harmless, since the handle vanishes anyway when the child process exits -- but the syslogger process, if used, doesn't get killed and restarted during recovery from a backend crash. That meant that Windows doesn't see the shared memory mapping as becoming free, so it doesn't delete it and the postmaster is unable to create a new one, resulting in failure to recover from crashes whenever logging_collector is turned on. Per report from Dmitry Vasilyev. It's a bit astonishing that we'd not figured this out long ago, since it's been broken from the very beginnings of out native Windows support; probably some previously-unexplained trouble reports trace to this. A secondary problem is that on Cygwin (perhaps only in older versions?), exec() may not detach from the shared memory segment after all, in which case these child processes did remain attached to shared memory, posing the risk of an unexpected shared memory clobber if they went off the rails somehow. That may be a long-gone bug, but we can deal with it now if it's still live, by detaching within the infrastructure introduced here to deal with closing the handle. Back-patch to all supported branches. Tom Lane and Amit Kapila
2015-10-13 17:21:33 +02:00
else
PGSharedMemoryNoReAttach();
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
Arrange for autovacuum to be killed when another operation wants to be alone accessing it, like DROP DATABASE. This allows the regression tests to pass with autovacuum enabled, which open the gates for finally enabling autovacuum by default.
2007-01-16 14:28:57 +01:00
/* autovacuum needs this set before calling InitProcess */
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
if (strcmp(argv[1], "--forkavlauncher") == 0)
AutovacuumLauncherIAm();
if (strcmp(argv[1], "--forkavworker") == 0)
AutovacuumWorkerIAm();
Arrange for autovacuum to be killed when another operation wants to be alone accessing it, like DROP DATABASE. This allows the regression tests to pass with autovacuum enabled, which open the gates for finally enabling autovacuum by default.
2007-01-16 14:28:57 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Start our win32 signal implementation. This has to be done after we
* read the backend variables, because we need to pick up the signal pipe
* from the parent process.
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
*/
#ifdef WIN32
pgwin32_signal_initialize();
#endif
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/* In EXEC_BACKEND case we will not have inherited these settings */
pqinitmask();
PG_SETMASK(&BlockSig);
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/* Read in remaining GUC variables */
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
read_nondefault_variables();
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Fix EXEC BACKEND + Windows builds for group privs Under EXEC BACKEND we also need to be going through the group privileges setup since we do support that on Unixy systems, so add that to SubPostmasterMain(). Under Windows, we need to simply return true from GetDataDirectoryCreatePerm(), but that wasn't happening due to a missing #else clause. Per buildfarm.
2018-04-08 01:01:43 +02:00
/*
* Check that the data directory looks valid, which will also check the
* privileges on the data directory and update our umask and file/group
* variables for creating files later. Note: this should really be done
* before we create any files or directories.
*/
checkDataDir();
Fix crash restart bug introduced in 8356753c212. The bug was caused by not re-reading the control file during crash recovery restarts, which lead to an attempt to pfree() shared memory contents. The fix is to re-read the control file, which seems good anyway. It's unclear as of this moment, whether we want to keep the refactoring introduced in the commit referenced above, or come up with an alternative approach. But fixing the bug in the mean time seems like a good idea regardless. A followup commit will introduce regression test coverage for crash restarts. Reported-By: Tom Lane Discussion: https://postgr.es/m/14134.1505572349@sss.pgh.pa.us
2017-09-17 10:00:39 +02:00
/*
* (re-)read control file, as it contains config. The postmaster will
* already have read this, but this process doesn't know about that.
*/
LocalProcessControlFile(false);
Perform only one ReadControlFile() during startup. Previously we read the control file in multiple places. But soon the segment size will be configurable and stored in the control file, and that needs to be available earlier than it currently is needed. Instead of adding yet another place where it's read, refactor things so there's a single processing of the control file during startup (in EXEC_BACKEND that's every individual backend's startup). Author: Andres Freund Discussion: http://postgr.es/m/20170913092828.aozd3gvvmw67gmyc@alap3.anarazel.de
2017-09-13 11:12:17 +02:00
Create a "shmem_startup_hook" to be called at the end of shared memory initialization, to give loadable modules a reasonable place to perform creation of any shared memory areas they need. This is the logical conclusion of our previous creation of RequestAddinShmemSpace() and RequestAddinLWLocks(). We don't need an explicit shmem_shutdown_hook, because the existing on_shmem_exit and on_proc_exit mechanisms serve that need. Also, adjust SubPostmasterMain so that libraries that got loaded into the postmaster will be loaded into all child processes, not only regular backends. This improves consistency with the non-EXEC_BACKEND behavior, and might be necessary for functionality for some types of add-ons.
2009-01-03 18:08:39 +01:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Reload any libraries that were preloaded by the postmaster. Since we
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* exec'd this process, those libraries didn't come along with us; but we
* should load them into all child processes to be consistent with the
* non-EXEC_BACKEND behavior.
Create a "shmem_startup_hook" to be called at the end of shared memory initialization, to give loadable modules a reasonable place to perform creation of any shared memory areas they need. This is the logical conclusion of our previous creation of RequestAddinShmemSpace() and RequestAddinLWLocks(). We don't need an explicit shmem_shutdown_hook, because the existing on_shmem_exit and on_proc_exit mechanisms serve that need. Also, adjust SubPostmasterMain so that libraries that got loaded into the postmaster will be loaded into all child processes, not only regular backends. This improves consistency with the non-EXEC_BACKEND behavior, and might be necessary for functionality for some types of add-ons.
2009-01-03 18:08:39 +01:00
*/
process_shared_preload_libraries();
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/* Run backend or appropriate child */
Merge postmaster and postgres command into just postgres. postmaster symlink is kept for now for compatibility. To call single-user mode, use postgres --single.
2006-06-18 17:38:37 +02:00
if (strcmp(argv[1], "--forkbackend") == 0)
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
{
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
Assert(argc == 3); /* shouldn't be any more args */
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
Here is a patch to fix win32 ssl builds. Summary of changes: * Links with -leay32 and -lssleay32 instead of crypto and ssl. On win32, "crypto and ssl" is only used for static linking. * Initializes SSL in the backend and not just in the postmaster. We cannot pass the SSL context from the postmaster through the parameter file, because it contains function pointers. * Split one error check in be-secure.c. Previously we could not tell which of three calls actually failed. The previous code also returned incorrect error messages if SSL_accept() failed - that function needs to use SSL_get_error() on the return value, can't just use the error queue. * Since the win32 implementation uses non-blocking sockets "behind the scenes" in order to deliver signals correctly, implements a version of SSL_accept() that can handle this. Also, add a wait function in case SSL_read or SSL_write() needs more data. Magnus Hagander
2004-10-06 11:35:23 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Need to reinitialize the SSL library in the backend, since the
* context structures contain function pointers and cannot be passed
* through the parameter file.
Create a "shmem_startup_hook" to be called at the end of shared memory initialization, to give loadable modules a reasonable place to perform creation of any shared memory areas they need. This is the logical conclusion of our previous creation of RequestAddinShmemSpace() and RequestAddinLWLocks(). We don't need an explicit shmem_shutdown_hook, because the existing on_shmem_exit and on_proc_exit mechanisms serve that need. Also, adjust SubPostmasterMain so that libraries that got loaded into the postmaster will be loaded into all child processes, not only regular backends. This improves consistency with the non-EXEC_BACKEND behavior, and might be necessary for functionality for some types of add-ons.
2009-01-03 18:08:39 +01:00
*
Allow SSL configuration to be updated at SIGHUP. It is no longer necessary to restart the server to enable, disable, or reconfigure SSL. Instead, we just create a new SSL_CTX struct (by re-reading all relevant files) whenever we get SIGHUP. Testing shows that this is fast enough that it shouldn't be a problem. In conjunction with that, downgrade the logic that complains about pg_hba.conf "hostssl" lines when SSL isn't active: now that's just a warning condition not an error. An issue that still needs to be addressed is what shall we do with passphrase-protected server keys? As this stands, the server would demand the passphrase again on every SIGHUP, which is certainly impractical. But the case was only barely supported before, so that does not seem a sufficient reason to hold up committing this patch. Andreas Karlsson, reviewed by Michael Banck and Michael Paquier Discussion: https://postgr.es/m/556A6E8A.9030400@proxel.se
2017-01-03 03:37:12 +01:00
* If for some reason reload fails (maybe the user installed broken
* key files), soldier on without SSL; that's better than all
* connections becoming impossible.
*
Create a "shmem_startup_hook" to be called at the end of shared memory initialization, to give loadable modules a reasonable place to perform creation of any shared memory areas they need. This is the logical conclusion of our previous creation of RequestAddinShmemSpace() and RequestAddinLWLocks(). We don't need an explicit shmem_shutdown_hook, because the existing on_shmem_exit and on_proc_exit mechanisms serve that need. Also, adjust SubPostmasterMain so that libraries that got loaded into the postmaster will be loaded into all child processes, not only regular backends. This improves consistency with the non-EXEC_BACKEND behavior, and might be necessary for functionality for some types of add-ons.
2009-01-03 18:08:39 +01:00
* XXX should we do this in all child processes? For the moment it's
* enough to do it in backend children.
Here is a patch to fix win32 ssl builds. Summary of changes: * Links with -leay32 and -lssleay32 instead of crypto and ssl. On win32, "crypto and ssl" is only used for static linking. * Initializes SSL in the backend and not just in the postmaster. We cannot pass the SSL context from the postmaster through the parameter file, because it contains function pointers. * Split one error check in be-secure.c. Previously we could not tell which of three calls actually failed. The previous code also returned incorrect error messages if SSL_accept() failed - that function needs to use SSL_get_error() on the return value, can't just use the error queue. * Since the win32 implementation uses non-blocking sockets "behind the scenes" in order to deliver signals correctly, implements a version of SSL_accept() that can handle this. Also, add a wait function in case SSL_read or SSL_write() needs more data. Magnus Hagander
2004-10-06 11:35:23 +02:00
*/
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
#ifdef USE_SSL
Here is a patch to fix win32 ssl builds. Summary of changes: * Links with -leay32 and -lssleay32 instead of crypto and ssl. On win32, "crypto and ssl" is only used for static linking. * Initializes SSL in the backend and not just in the postmaster. We cannot pass the SSL context from the postmaster through the parameter file, because it contains function pointers. * Split one error check in be-secure.c. Previously we could not tell which of three calls actually failed. The previous code also returned incorrect error messages if SSL_accept() failed - that function needs to use SSL_get_error() on the return value, can't just use the error queue. * Since the win32 implementation uses non-blocking sockets "behind the scenes" in order to deliver signals correctly, implements a version of SSL_accept() that can handle this. Also, add a wait function in case SSL_read or SSL_write() needs more data. Magnus Hagander
2004-10-06 11:35:23 +02:00
if (EnableSSL)
Allow SSL configuration to be updated at SIGHUP. It is no longer necessary to restart the server to enable, disable, or reconfigure SSL. Instead, we just create a new SSL_CTX struct (by re-reading all relevant files) whenever we get SIGHUP. Testing shows that this is fast enough that it shouldn't be a problem. In conjunction with that, downgrade the logic that complains about pg_hba.conf "hostssl" lines when SSL isn't active: now that's just a warning condition not an error. An issue that still needs to be addressed is what shall we do with passphrase-protected server keys? As this stands, the server would demand the passphrase again on every SIGHUP, which is certainly impractical. But the case was only barely supported before, so that does not seem a sufficient reason to hold up committing this patch. Andreas Karlsson, reviewed by Michael Banck and Michael Paquier Discussion: https://postgr.es/m/556A6E8A.9030400@proxel.se
2017-01-03 03:37:12 +01:00
{
if (secure_initialize(false) == 0)
LoadedSSL = true;
else
ereport(LOG,
Re-allow SSL passphrase prompt at server start, but not thereafter. Leave OpenSSL's default passphrase collection callback in place during the first call of secure_initialize() in server startup. Although that doesn't work terribly well in daemon contexts, some people feel we should not break it for anyone who was successfully using it before. We still block passphrase demands during SIGHUP, meaning that you can't adjust SSL configuration on-the-fly if you used a passphrase, but this is no worse than what it was before commit de41869b6. And we block passphrase demands during EXEC_BACKEND reloads; that behavior wasn't useful either, but at least now it's documented. Tweak some related log messages for more readability, and avoid issuing essentially duplicate messages about reload failure caused by a passphrase. Discussion: https://postgr.es/m/29982.1483412575@sss.pgh.pa.us
2017-01-04 18:43:52 +01:00
(errmsg("SSL configuration could not be loaded in child process")));
Allow SSL configuration to be updated at SIGHUP. It is no longer necessary to restart the server to enable, disable, or reconfigure SSL. Instead, we just create a new SSL_CTX struct (by re-reading all relevant files) whenever we get SIGHUP. Testing shows that this is fast enough that it shouldn't be a problem. In conjunction with that, downgrade the logic that complains about pg_hba.conf "hostssl" lines when SSL isn't active: now that's just a warning condition not an error. An issue that still needs to be addressed is what shall we do with passphrase-protected server keys? As this stands, the server would demand the passphrase again on every SIGHUP, which is certainly impractical. But the case was only barely supported before, so that does not seem a sufficient reason to hold up committing this patch. Andreas Karlsson, reviewed by Michael Banck and Michael Paquier Discussion: https://postgr.es/m/556A6E8A.9030400@proxel.se
2017-01-03 03:37:12 +01:00
}
Here is a patch to fix win32 ssl builds. Summary of changes: * Links with -leay32 and -lssleay32 instead of crypto and ssl. On win32, "crypto and ssl" is only used for static linking. * Initializes SSL in the backend and not just in the postmaster. We cannot pass the SSL context from the postmaster through the parameter file, because it contains function pointers. * Split one error check in be-secure.c. Previously we could not tell which of three calls actually failed. The previous code also returned incorrect error messages if SSL_accept() failed - that function needs to use SSL_get_error() on the return value, can't just use the error queue. * Since the win32 implementation uses non-blocking sockets "behind the scenes" in order to deliver signals correctly, implements a version of SSL_accept() that can handle this. Also, add a wait function in case SSL_read or SSL_write() needs more data. Magnus Hagander
2004-10-06 11:35:23 +02:00
#endif
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
/*
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
* Perform additional initialization and collect startup packet.
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* We want to do this before InitProcess() for a couple of reasons: 1.
* so that we aren't eating up a PGPROC slot while waiting on the
* client. 2. so that if InitProcess() fails due to being out of
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
*/
BackendInitialize(&port);
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
InitProcess();
Do ClosePostmasterPorts() earlier in SubPostmasterMain(). In standard Unix builds, postmaster child processes do ClosePostmasterPorts immediately after InitPostmasterChild, that is almost immediately after being spawned. This is important because we don't want children holding open the postmaster's end of the postmaster death watch pipe. However, in EXEC_BACKEND builds, SubPostmasterMain was postponing this responsibility significantly, in order to make it slightly more convenient to pass the right flag value to ClosePostmasterPorts. This is bad, particularly seeing that process_shared_preload_libraries() might invoke nearly-arbitrary code. Rearrange so that we do it as soon as we've fetched the socket FDs via read_backend_variables(). Also move the comment explaining about randomize_va_space to before the call of PGSharedMemoryReAttach, which is where it's relevant. The old placement was appropriate when the reattach happened inside CreateSharedMemoryAndSemaphores, but that was a long time ago. Back-patch to 9.3; the patch doesn't apply cleanly before that, and it doesn't seem worth a lot of effort given that we've had no actual field complaints traceable to this. Discussion: <4157.1475178360@sss.pgh.pa.us>
2016-10-01 23:15:09 +02:00
/* Attach process to shared data structures */
Use data directory inode number, not port, to select SysV resource keys. This approach provides a much tighter binding between a data directory and the associated SysV shared memory block (and SysV or named-POSIX semaphores, if we're using those). Key collisions are still possible, but only between data directories stored on different filesystems, so the situation should be negligible in practice. More importantly, restarting the postmaster with a different port number no longer risks failing to identify a relevant shared memory block, even when postmaster.pid has been removed. A standalone backend is likewise much more certain to detect conflicting leftover backends. (In the longer term, we might now think about deprecating the port as a cluster-wide value, so that one postmaster could support sockets with varying port numbers. But that's for another day.) The hazards fixed here apply only on Unix systems; our Windows code paths already use identifiers derived from the data directory path name rather than the port. src/test/recovery/t/017_shm.pl, which intends to test key-collision cases, has been substantially rewritten since it can no longer use two postmasters with identical port numbers to trigger the case. Instead, use Perl's IPC::SharedMem module to create a conflicting shmem segment directly. The test script will be skipped if that module is not available. (This means that some older buildfarm members won't run it, but I don't think that that results in any meaningful coverage loss.) Patch by me; thanks to Noah Misch and Peter Eisentraut for discussion and review. Discussion: https://postgr.es/m/16908.1557521200@sss.pgh.pa.us
2019-09-05 19:31:41 +02:00
CreateSharedMemoryAndSemaphores();
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
/* And run the backend */
Unify calling conventions for postgres/postmaster sub-main functions There was a wild mix of calling conventions: Some were declared to return void and didn't return, some returned an int exit code, some claimed to return an exit code, which the callers checked, but actually never returned, and so on. Now all of these functions are declared to return void and decorated with attribute noreturn and don't return. That's easiest, and most code already worked that way.
2012-06-25 20:25:26 +02:00
BackendRun(&port); /* does not return */
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
}
Merge postmaster and postgres command into just postgres. postmaster symlink is kept for now for compatibility. To call single-user mode, use postgres --single.
2006-06-18 17:38:37 +02:00
if (strcmp(argv[1], "--forkboot") == 0)
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
{
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
InitAuxiliaryProcess();
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
/* Attach process to shared data structures */
Use data directory inode number, not port, to select SysV resource keys. This approach provides a much tighter binding between a data directory and the associated SysV shared memory block (and SysV or named-POSIX semaphores, if we're using those). Key collisions are still possible, but only between data directories stored on different filesystems, so the situation should be negligible in practice. More importantly, restarting the postmaster with a different port number no longer risks failing to identify a relevant shared memory block, even when postmaster.pid has been removed. A standalone backend is likewise much more certain to detect conflicting leftover backends. (In the longer term, we might now think about deprecating the port as a cluster-wide value, so that one postmaster could support sockets with varying port numbers. But that's for another day.) The hazards fixed here apply only on Unix systems; our Windows code paths already use identifiers derived from the data directory path name rather than the port. src/test/recovery/t/017_shm.pl, which intends to test key-collision cases, has been substantially rewritten since it can no longer use two postmasters with identical port numbers to trigger the case. Instead, use Perl's IPC::SharedMem module to create a conflicting shmem segment directly. The test script will be skipped if that module is not available. (This means that some older buildfarm members won't run it, but I don't think that that results in any meaningful coverage loss.) Patch by me; thanks to Noah Misch and Peter Eisentraut for discussion and review. Discussion: https://postgr.es/m/16908.1557521200@sss.pgh.pa.us
2019-09-05 19:31:41 +02:00
CreateSharedMemoryAndSemaphores();
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
}
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
if (strcmp(argv[1], "--forkavlauncher") == 0)
{
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
Change the autovacuum launcher to read pg_database directly, rather than via the "flat files" facility. This requires making it enough like a backend to be able to run transactions; it's no longer an "auxiliary process" but more like the autovacuum worker processes. Also, its signal handling has to be brought into line with backends/workers. In particular, since it now has to handle procsignal.c processing, the special autovac-launcher-only signal conditions are moved to SIGUSR2. Alvaro, with some cleanup from Tom
2009-08-31 21:41:00 +02:00
InitProcess();
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
/* Attach process to shared data structures */
Use data directory inode number, not port, to select SysV resource keys. This approach provides a much tighter binding between a data directory and the associated SysV shared memory block (and SysV or named-POSIX semaphores, if we're using those). Key collisions are still possible, but only between data directories stored on different filesystems, so the situation should be negligible in practice. More importantly, restarting the postmaster with a different port number no longer risks failing to identify a relevant shared memory block, even when postmaster.pid has been removed. A standalone backend is likewise much more certain to detect conflicting leftover backends. (In the longer term, we might now think about deprecating the port as a cluster-wide value, so that one postmaster could support sockets with varying port numbers. But that's for another day.) The hazards fixed here apply only on Unix systems; our Windows code paths already use identifiers derived from the data directory path name rather than the port. src/test/recovery/t/017_shm.pl, which intends to test key-collision cases, has been substantially rewritten since it can no longer use two postmasters with identical port numbers to trigger the case. Instead, use Perl's IPC::SharedMem module to create a conflicting shmem segment directly. The test script will be skipped if that module is not available. (This means that some older buildfarm members won't run it, but I don't think that that results in any meaningful coverage loss.) Patch by me; thanks to Noah Misch and Peter Eisentraut for discussion and review. Discussion: https://postgr.es/m/16908.1557521200@sss.pgh.pa.us
2019-09-05 19:31:41 +02:00
CreateSharedMemoryAndSemaphores();
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
}
if (strcmp(argv[1], "--forkavworker") == 0)
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
{
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
InitProcess();
Clean up a number of autovacuum loose ends. Make the stats collector track shared relations in a separate hashtable, so that operations done from different databases are counted correctly. Add proper support for anti-XID-wraparound vacuuming, even in databases that are never connected to and so have no stats entries. Miscellaneous other bug fixes. Alvaro Herrera, some additional fixes by Tom Lane.
2005-07-29 21:30:09 +02:00
/* Attach process to shared data structures */
Use data directory inode number, not port, to select SysV resource keys. This approach provides a much tighter binding between a data directory and the associated SysV shared memory block (and SysV or named-POSIX semaphores, if we're using those). Key collisions are still possible, but only between data directories stored on different filesystems, so the situation should be negligible in practice. More importantly, restarting the postmaster with a different port number no longer risks failing to identify a relevant shared memory block, even when postmaster.pid has been removed. A standalone backend is likewise much more certain to detect conflicting leftover backends. (In the longer term, we might now think about deprecating the port as a cluster-wide value, so that one postmaster could support sockets with varying port numbers. But that's for another day.) The hazards fixed here apply only on Unix systems; our Windows code paths already use identifiers derived from the data directory path name rather than the port. src/test/recovery/t/017_shm.pl, which intends to test key-collision cases, has been substantially rewritten since it can no longer use two postmasters with identical port numbers to trigger the case. Instead, use Perl's IPC::SharedMem module to create a conflicting shmem segment directly. The test script will be skipped if that module is not available. (This means that some older buildfarm members won't run it, but I don't think that that results in any meaningful coverage loss.) Patch by me; thanks to Noah Misch and Peter Eisentraut for discussion and review. Discussion: https://postgr.es/m/16908.1557521200@sss.pgh.pa.us
2019-09-05 19:31:41 +02:00
CreateSharedMemoryAndSemaphores();
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
Integrate autovacuum functionality into the backend. There's still a few loose ends to be dealt with, but it seems to work. Alvaro Herrera, based on the contrib code by Matthew O'Connor.
2005-07-14 07:13:45 +02:00
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
{
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
int shmem_slot;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
Fix mishandling of background worker PGPROCs in EXEC_BACKEND builds. InitProcess() relies on IsBackgroundWorker to decide whether the PGPROC for a new backend should be taken from ProcGlobal's freeProcs or from bgworkerFreeProcs. In EXEC_BACKEND builds, InitProcess() is called sooner than in non-EXEC_BACKEND builds, and IsBackgroundWorker wasn't getting initialized soon enough. Report by Noah Misch. Diagnosis and fix by me.
2014-07-30 17:25:58 +02:00
/* do this as early as possible; in particular, before InitProcess() */
IsBackgroundWorker = true;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
InitProcess();
/* Attach process to shared data structures */
Use data directory inode number, not port, to select SysV resource keys. This approach provides a much tighter binding between a data directory and the associated SysV shared memory block (and SysV or named-POSIX semaphores, if we're using those). Key collisions are still possible, but only between data directories stored on different filesystems, so the situation should be negligible in practice. More importantly, restarting the postmaster with a different port number no longer risks failing to identify a relevant shared memory block, even when postmaster.pid has been removed. A standalone backend is likewise much more certain to detect conflicting leftover backends. (In the longer term, we might now think about deprecating the port as a cluster-wide value, so that one postmaster could support sockets with varying port numbers. But that's for another day.) The hazards fixed here apply only on Unix systems; our Windows code paths already use identifiers derived from the data directory path name rather than the port. src/test/recovery/t/017_shm.pl, which intends to test key-collision cases, has been substantially rewritten since it can no longer use two postmasters with identical port numbers to trigger the case. Instead, use Perl's IPC::SharedMem module to create a conflicting shmem segment directly. The test script will be skipped if that module is not available. (This means that some older buildfarm members won't run it, but I don't think that that results in any meaningful coverage loss.) Patch by me; thanks to Noah Misch and Peter Eisentraut for discussion and review. Discussion: https://postgr.es/m/16908.1557521200@sss.pgh.pa.us
2019-09-05 19:31:41 +02:00
CreateSharedMemoryAndSemaphores();
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
Remove duplicate InitPostmasterChild() call while starting a bgworker. This is apparently harmless on Windows, but on Unix it results in an assertion failure. We'd not noticed because this code doesn't get used on Unix unless you build with -DEXEC_BACKEND. Bug was evidently introduced by sloppy refactoring in commit 31c453165. Thomas Munro Discussion: <CAEepm=1VOnbVx4wsgQFvj94hu9jVt2nVabCr7QiooUSvPJXkgQ@mail.gmail.com>
2016-08-03 00:39:14 +02:00
/* Fetch MyBgworkerEntry from shared memory */
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
shmem_slot = atoi(argv[1] + 15);
MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
Remove duplicate InitPostmasterChild() call while starting a bgworker. This is apparently harmless on Windows, but on Unix it results in an assertion failure. We'd not noticed because this code doesn't get used on Unix unless you build with -DEXEC_BACKEND. Bug was evidently introduced by sloppy refactoring in commit 31c453165. Thomas Munro Discussion: <CAEepm=1VOnbVx4wsgQFvj94hu9jVt2nVabCr7QiooUSvPJXkgQ@mail.gmail.com>
2016-08-03 00:39:14 +02:00
Move more bgworker code to bgworker.c; also, some renaming. Per discussion on pgsql-hackers. Michael Paquier, slightly modified by me. Original suggestion from Amit Kapila.
2013-08-16 21:14:54 +02:00
StartBackgroundWorker();
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
}
Merge postmaster and postgres command into just postgres. postmaster symlink is kept for now for compatibility. To call single-user mode, use postgres --single.
2006-06-18 17:38:37 +02:00
if (strcmp(argv[1], "--forkarch") == 0)
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
{
/* Do not want to attach to shared memory */
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
PgArchiverMain(argc, argv); /* does not return */
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
}
Remove the separate 'stats buffer' process, letting backend stats messages be delivered directly to the collector process. The extra process context swaps required to transfer data through the buffer process seem to outweigh any value the buffering might have. Per recent discussion and tests. I modified Bruce's draft patch to use poll() rather than select() where available (this makes a noticeable difference on my system), and fixed up the EXEC_BACKEND case.
2006-06-29 22:00:08 +02:00
if (strcmp(argv[1], "--forkcol") == 0)
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
{
/* Do not want to attach to shared memory */
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
PgstatCollectorMain(argc, argv); /* does not return */
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
}
Merge postmaster and postgres command into just postgres. postmaster symlink is kept for now for compatibility. To call single-user mode, use postgres --single.
2006-06-18 17:38:37 +02:00
if (strcmp(argv[1], "--forklog") == 0)
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
{
/* Do not want to attach to shared memory */
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
SysLoggerMain(argc, argv); /* does not return */
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
}
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Unify calling conventions for postgres/postmaster sub-main functions There was a wild mix of calling conventions: Some were declared to return void and didn't return, some returned an int exit code, some claimed to return an exit code, which the callers checked, but actually never returned, and so on. Now all of these functions are declared to return void and decorated with attribute noreturn and don't return. That's easiest, and most code already worked that way.
2012-06-25 20:25:26 +02:00
abort(); /* shouldn't get here */
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
}
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
#endif /* EXEC_BACKEND */
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
* ExitPostmaster -- cleanup
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
*
* Do NOT call exit() directly --- always go through here!
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
elog mop-up: bring some straggling fprintf(stderr)'s into the elog world.
2003-07-27 23:49:55 +02:00
static void
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
ExitPostmaster(int status)
{
On Darwin, detect and report a multithreaded postmaster. Darwin --enable-nls builds use a substitute setlocale() that may start a thread. Buildfarm member orangutan experienced BackendList corruption on account of different postmaster threads executing signal handlers simultaneously. Furthermore, a multithreaded postmaster risks undefined behavior from sigprocmask() and fork(). Emit LOG messages about the problem and its workaround. Back-patch to 9.0 (all supported versions).
2015-01-08 04:35:44 +01:00
#ifdef HAVE_PTHREAD_IS_THREADED_NP
/*
* There is no known cause for a postmaster to become multithreaded after
* startup. Recheck to account for the possibility of unknown causes.
On Darwin, refuse postmaster startup when multithreaded. The previous commit introduced its report at LOG level to avoid surprises at minor release upgrade time. Compel users deploying the next major release to also deploy the reported workaround.
2015-01-08 04:46:59 +01:00
* This message uses LOG level, because an unclean shutdown at this point
* would usually not look much different from a clean shutdown.
On Darwin, detect and report a multithreaded postmaster. Darwin --enable-nls builds use a substitute setlocale() that may start a thread. Buildfarm member orangutan experienced BackendList corruption on account of different postmaster threads executing signal handlers simultaneously. Furthermore, a multithreaded postmaster risks undefined behavior from sigprocmask() and fork(). Emit LOG messages about the problem and its workaround. Back-patch to 9.0 (all supported versions).
2015-01-08 04:35:44 +01:00
*/
if (pthread_is_threaded_np() != 0)
ereport(LOG,
On Darwin, refuse postmaster startup when multithreaded. The previous commit introduced its report at LOG level to avoid surprises at minor release upgrade time. Compel users deploying the next major release to also deploy the reported workaround.
2015-01-08 04:46:59 +01:00
(errcode(ERRCODE_INTERNAL_ERROR),
errmsg_internal("postmaster became multithreaded"),
Replace @postgresql.org with @lists.postgresql.org for mailinglists Commit c0d0e54084 replaced the ones in the documentation, but missed out on the ones in the code. Replace those as well, but unlike c0d0e54084, don't backpatch the code changes to avoid breaking translations.
2019-01-19 19:06:35 +01:00
errdetail("Please report this to <pgsql-bugs@lists.postgresql.org>.")));
On Darwin, detect and report a multithreaded postmaster. Darwin --enable-nls builds use a substitute setlocale() that may start a thread. Buildfarm member orangutan experienced BackendList corruption on account of different postmaster threads executing signal handlers simultaneously. Furthermore, a multithreaded postmaster risks undefined behavior from sigprocmask() and fork(). Emit LOG messages about the problem and its workaround. Back-patch to 9.0 (all supported versions).
2015-01-08 04:35:44 +01:00
#endif
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/* should cleanup shared memory and kill all backends */
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Not sure of the semantics here. When the Postmaster dies, should the
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* backends all be killed? probably not.
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
*
* MUST -- vadim 05-10-1999
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
More paranoia about global variables containing references to long- since-closed file descriptors...
2000-05-26 03:38:08 +02:00
Fix for hang after postmaster restart. Add new proc_exit and shmem_exit to replace exitpg().
1998-06-27 06:53:49 +02:00
proc_exit(status);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
/*
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
* sigusr1_handler - handle signal conditions from child processes
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
*/
static void
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
sigusr1_handler(SIGNAL_ARGS)
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
{
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
int save_errno = errno;
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
/*
* We rely on the signal mechanism to have blocked all signals ... except
* on Windows, which lacks sigaction(), so we have to do it manually.
*/
#ifdef WIN32
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
PG_SETMASK(&BlockSig);
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
#endif
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
/* Process background worker state change. */
if (CheckPostmasterSignal(PMSIGNAL_BACKGROUND_WORKER_CHANGE))
{
BackgroundWorkerStateChange();
Eliminate one background-worker-related flag variable. Teach sigusr1_handler() to use the same test for whether a worker might need to be started as ServerLoop(). Aside from being perhaps a bit simpler, this prevents a potentially-unbounded delay when starting a background worker. On some platforms, select() doesn't return when interrupted by a signal, but is instead restarted, including a reset of the timeout to the originally-requested value. If signals arrive often enough, but no connection requests arrive, sigusr1_handler() will be executed repeatedly, but the body of ServerLoop() won't be reached. This change ensures that, even in that case, background workers will eventually get launched. This is far from a perfect fix; really, we need select() to return control to ServerLoop() after an interrupt, either via the self-pipe trick or some other mechanism. But that's going to require more work and discussion, so let's do this for now to at least mitigate the damage. Per investigation of test_shm_mq failures on buildfarm member anole.
2014-10-05 03:25:41 +02:00
StartWorkerNeeded = true;
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
}
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
/*
Rename PM_RECOVERY_CONSISTENT and PMSIGNAL_RECOVERY_CONSISTENT. The new names PM_HOT_STANDBY and PMSIGNAL_BEGIN_HOT_STANDBY more accurately reflect their actual function.
2010-05-15 22:01:32 +02:00
* RECOVERY_STARTED and BEGIN_HOT_STANDBY signals are ignored in
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
* unexpected states. If the startup process quickly starts up, completes
* recovery, exits, we might process the death of the startup process
* first. We don't want to go back to recovery in that case.
*/
if (CheckPostmasterSignal(PMSIGNAL_RECOVERY_STARTED) &&
Don't launch new child processes after we've been told to shut down. Once we've received a shutdown signal (SIGINT or SIGTERM), we should not launch any more child processes, even if we get signals requesting such. The normal code path for spawning backends has always understood that, but the postmaster's infrastructure for hot standby and autovacuum didn't get the memo. As reported by Hari Babu in bug #7643, this could lead to failure to shut down at all in some cases, such as when SIGINT is received just before the startup process sends PMSIGNAL_RECOVERY_STARTED: we'd launch a bgwriter and checkpointer, and then those processes would have no idea that they ought to quit. Similarly, launching a new autovacuum worker would result in waiting till it finished before shutting down. Also, switch the order of the code blocks in reaper() that detect startup process crash versus shutdown termination. Once we've sent it a signal, we should not consider that exit(1) is surprising. This is just a cosmetic fix since shutdown occurs correctly anyway, but better not to log a phony complaint about startup process crash. Back-patch to 9.0. Some parts of this might be applicable before that, but given the lack of prior complaints I'm not going to worry too much about older branches.
2012-11-21 21:18:38 +01:00
pmState == PM_STARTUP && Shutdown == NoShutdown)
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
{
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
/* WAL redo has started. We're out of reinitialization. */
FatalError = false;
Fix postmaster state machine to handle dead_end child crashes better. A report from Alvaro Herrera shows that if we're in PM_STARTUP state, and we spawn a dead_end child to reject some incoming connection request, and that child dies with an unexpected exit code, the postmaster does not respond well. We correctly send SIGQUIT to the startup process, but then: * if the startup process exits with nonzero exit code, as expected, we thought that that indicated a crash and aborted startup. * if the startup process exits with zero exit code, which is possible due to the inherent race condition, we'd advance to PM_RUN state which is fine --- but the code forgot that AbortStartTime would be nonzero in this situation. We'd either die on the Asserts saying that it was zero, or perhaps misbehave later on. (A quick look suggests that the only misbehavior might be busy-waiting due to DetermineSleepTime doing the wrong thing.) To fix the first point, adjust the state-machine logic to recognize that a nonzero exit code is expected after sending SIGQUIT, and have it transition to a state where we can restart the startup process. To fix the second point, change the Asserts to clear the variable rather than just claiming it should be clear already. Perhaps we could improve this further by not treating a crash of a dead_end child as a reason for panic'ing the database. However, since those child processes are connected to shared memory, that seems a bit risky. There are few good reasons for a dead_end child to report failure anyway (the cause of this in Alvaro's report is quite unclear). On balance, therefore, a minimal fix seems best. This is an oversight in commit 45811be94. While that was back-patched, I'm hesitant to back-patch this change. The lack of reasons for a dead_end child to fail suggests that the case should be very rare in the field, which squares with the lack of reports; so it seems like this might not be worth the risk of introducing new issues. In any case we can let it bake awhile in HEAD before considering a back-patch. Discussion: https://postgr.es/m/20190615160950.GA31378@alvherre.pgsql
2019-08-26 21:59:44 +02:00
AbortStartTime = 0;
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
/*
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
* Crank up the background tasks. It doesn't matter if this fails,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* we'll just try again later.
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
*/
Cosmetic adjustments for postmaster's handling of checkpointer. Correct some comments, order some operations a bit more consistently. No functional changes.
2012-05-11 23:46:08 +02:00
Assert(CheckpointerPID == 0);
CheckpointerPID = StartCheckpointer();
Checkpointer starts before bgwriter to avoid missing fsync requests. Noted while testing Hot Standby startup.
2012-06-01 09:25:17 +02:00
Assert(BgWriterPID == 0);
BgWriterPID = StartBackgroundWriter();
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
Add archive_mode='always' option. In 'always' mode, the standby independently archives all files it receives from the primary. Original patch by Fujii Masao, docs and review by me.
2015-05-15 17:55:24 +02:00
/*
* Start the archiver if we're responsible for (re-)archiving received
* files.
*/
Assert(PgArchPID == 0);
Make postmaster restart archiver soon after it dies, even during recovery. After the archiver dies, postmaster tries to start a new one immediately. But previously this could happen only while server was running normally even though archiving was enabled always (i.e., archive_mode was set to always). So the archiver running during recovery could not restart soon after it died. This is an oversight in commit ffd3774. This commit changes reaper(), postmaster's signal handler to cleanup after a child process dies, so that it tries to a new archiver even during recovery if necessary. Patch by me. Review by Alvaro Herrera.
2015-06-12 16:11:51 +02:00
if (XLogArchivingAlways())
Add archive_mode='always' option. In 'always' mode, the standby independently archives all files it receives from the primary. Original patch by Fujii Masao, docs and review by me.
2015-05-15 17:55:24 +02:00
PgArchPID = pgarch_start();
Change pg_ctl to detect server-ready by watching status in postmaster.pid. Traditionally, "pg_ctl start -w" has waited for the server to become ready to accept connections by attempting a connection once per second. That has the major problem that connection issues (for instance, a kernel packet filter blocking traffic) can't be reliably told apart from server startup issues, and the minor problem that if server startup isn't quick, we accumulate "the database system is starting up" spam in the server log. We've hacked around many of the possible connection issues, but it resulted in ugly and complicated code in pg_ctl.c. In commit c61559ec3, I changed the probe rate to every tenth of a second. That prompted Jeff Janes to complain that the log-spam problem had become much worse. In the ensuing discussion, Andres Freund pointed out that we could dispense with connection attempts altogether if the postmaster were changed to report its status in postmaster.pid, which "pg_ctl start" already relies on being able to read. This patch implements that, teaching postmaster.c to report a status string into the pidfile at the same state-change points already identified as being of interest for systemd status reporting (cf commit 7d17e683f). pg_ctl no longer needs to link with libpq at all; all its functions now depend on reading server files. In support of this, teach AddToDataDirLockFile() to allow addition of postmaster.pid lines in not-necessarily-sequential order. This is needed on Windows where the SHMEM_KEY line will never be written at all. We still have the restriction that we don't want to truncate the pidfile; document the reasons for that a bit better. Also, fix the pg_ctl TAP tests so they'll notice if "start -w" mode is broken --- before, they'd just wait out the sixty seconds until the loop gives up, and then report success anyway. (Yes, I found that out the hard way.) While at it, arrange for pg_ctl to not need to #include miscadmin.h; as a rather low-level backend header, requiring that to be compilable client-side is pretty dubious. This requires moving the #define's associated with the pidfile into a new header file, and moving PG_BACKEND_VERSIONSTR someplace else. For lack of a clearly better "someplace else", I put it into port.h, beside the declaration of find_other_exec(), since most users of that macro are passing the value to find_other_exec(). (initdb still depends on miscadmin.h, but at least pg_ctl and pg_upgrade no longer do.) In passing, fix main.c so that PG_BACKEND_VERSIONSTR actually defines the output of "postgres -V", which remarkably it had never done before. Discussion: https://postgr.es/m/CAMkU=1xJW8e+CTotojOMBd-yzUvD0e_JZu2xHo=MnuZ4__m7Pg@mail.gmail.com
2017-06-28 23:31:24 +02:00
/*
* If we aren't planning to enter hot standby mode later, treat
* RECOVERY_STARTED as meaning we're out of startup, and report status
* accordingly.
*/
Add support for systemd service notifications Insert sd_notify() calls at server start and stop for integration with systemd. This allows the use of systemd service units of type "notify", which greatly simplifies the systemd configuration. Reviewed-by: Pavel Stěhule <pavel.stehule@gmail.com>
2015-11-17 12:46:17 +01:00
if (!EnableHotStandby)
Change pg_ctl to detect server-ready by watching status in postmaster.pid. Traditionally, "pg_ctl start -w" has waited for the server to become ready to accept connections by attempting a connection once per second. That has the major problem that connection issues (for instance, a kernel packet filter blocking traffic) can't be reliably told apart from server startup issues, and the minor problem that if server startup isn't quick, we accumulate "the database system is starting up" spam in the server log. We've hacked around many of the possible connection issues, but it resulted in ugly and complicated code in pg_ctl.c. In commit c61559ec3, I changed the probe rate to every tenth of a second. That prompted Jeff Janes to complain that the log-spam problem had become much worse. In the ensuing discussion, Andres Freund pointed out that we could dispense with connection attempts altogether if the postmaster were changed to report its status in postmaster.pid, which "pg_ctl start" already relies on being able to read. This patch implements that, teaching postmaster.c to report a status string into the pidfile at the same state-change points already identified as being of interest for systemd status reporting (cf commit 7d17e683f). pg_ctl no longer needs to link with libpq at all; all its functions now depend on reading server files. In support of this, teach AddToDataDirLockFile() to allow addition of postmaster.pid lines in not-necessarily-sequential order. This is needed on Windows where the SHMEM_KEY line will never be written at all. We still have the restriction that we don't want to truncate the pidfile; document the reasons for that a bit better. Also, fix the pg_ctl TAP tests so they'll notice if "start -w" mode is broken --- before, they'd just wait out the sixty seconds until the loop gives up, and then report success anyway. (Yes, I found that out the hard way.) While at it, arrange for pg_ctl to not need to #include miscadmin.h; as a rather low-level backend header, requiring that to be compilable client-side is pretty dubious. This requires moving the #define's associated with the pidfile into a new header file, and moving PG_BACKEND_VERSIONSTR someplace else. For lack of a clearly better "someplace else", I put it into port.h, beside the declaration of find_other_exec(), since most users of that macro are passing the value to find_other_exec(). (initdb still depends on miscadmin.h, but at least pg_ctl and pg_upgrade no longer do.) In passing, fix main.c so that PG_BACKEND_VERSIONSTR actually defines the output of "postgres -V", which remarkably it had never done before. Discussion: https://postgr.es/m/CAMkU=1xJW8e+CTotojOMBd-yzUvD0e_JZu2xHo=MnuZ4__m7Pg@mail.gmail.com
2017-06-28 23:31:24 +02:00
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
#ifdef USE_SYSTEMD
Add support for systemd service notifications Insert sd_notify() calls at server start and stop for integration with systemd. This allows the use of systemd service units of type "notify", which greatly simplifies the systemd configuration. Reviewed-by: Pavel Stěhule <pavel.stehule@gmail.com>
2015-11-17 12:46:17 +01:00
sd_notify(0, "READY=1");
#endif
Change pg_ctl to detect server-ready by watching status in postmaster.pid. Traditionally, "pg_ctl start -w" has waited for the server to become ready to accept connections by attempting a connection once per second. That has the major problem that connection issues (for instance, a kernel packet filter blocking traffic) can't be reliably told apart from server startup issues, and the minor problem that if server startup isn't quick, we accumulate "the database system is starting up" spam in the server log. We've hacked around many of the possible connection issues, but it resulted in ugly and complicated code in pg_ctl.c. In commit c61559ec3, I changed the probe rate to every tenth of a second. That prompted Jeff Janes to complain that the log-spam problem had become much worse. In the ensuing discussion, Andres Freund pointed out that we could dispense with connection attempts altogether if the postmaster were changed to report its status in postmaster.pid, which "pg_ctl start" already relies on being able to read. This patch implements that, teaching postmaster.c to report a status string into the pidfile at the same state-change points already identified as being of interest for systemd status reporting (cf commit 7d17e683f). pg_ctl no longer needs to link with libpq at all; all its functions now depend on reading server files. In support of this, teach AddToDataDirLockFile() to allow addition of postmaster.pid lines in not-necessarily-sequential order. This is needed on Windows where the SHMEM_KEY line will never be written at all. We still have the restriction that we don't want to truncate the pidfile; document the reasons for that a bit better. Also, fix the pg_ctl TAP tests so they'll notice if "start -w" mode is broken --- before, they'd just wait out the sixty seconds until the loop gives up, and then report success anyway. (Yes, I found that out the hard way.) While at it, arrange for pg_ctl to not need to #include miscadmin.h; as a rather low-level backend header, requiring that to be compilable client-side is pretty dubious. This requires moving the #define's associated with the pidfile into a new header file, and moving PG_BACKEND_VERSIONSTR someplace else. For lack of a clearly better "someplace else", I put it into port.h, beside the declaration of find_other_exec(), since most users of that macro are passing the value to find_other_exec(). (initdb still depends on miscadmin.h, but at least pg_ctl and pg_upgrade no longer do.) In passing, fix main.c so that PG_BACKEND_VERSIONSTR actually defines the output of "postgres -V", which remarkably it had never done before. Discussion: https://postgr.es/m/CAMkU=1xJW8e+CTotojOMBd-yzUvD0e_JZu2xHo=MnuZ4__m7Pg@mail.gmail.com
2017-06-28 23:31:24 +02:00
}
Add support for systemd service notifications Insert sd_notify() calls at server start and stop for integration with systemd. This allows the use of systemd service units of type "notify", which greatly simplifies the systemd configuration. Reviewed-by: Pavel Stěhule <pavel.stehule@gmail.com>
2015-11-17 12:46:17 +01:00
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
pmState = PM_RECOVERY;
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
}
Rename PM_RECOVERY_CONSISTENT and PMSIGNAL_RECOVERY_CONSISTENT. The new names PM_HOT_STANDBY and PMSIGNAL_BEGIN_HOT_STANDBY more accurately reflect their actual function.
2010-05-15 22:01:32 +02:00
if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
Don't launch new child processes after we've been told to shut down. Once we've received a shutdown signal (SIGINT or SIGTERM), we should not launch any more child processes, even if we get signals requesting such. The normal code path for spawning backends has always understood that, but the postmaster's infrastructure for hot standby and autovacuum didn't get the memo. As reported by Hari Babu in bug #7643, this could lead to failure to shut down at all in some cases, such as when SIGINT is received just before the startup process sends PMSIGNAL_RECOVERY_STARTED: we'd launch a bgwriter and checkpointer, and then those processes would have no idea that they ought to quit. Similarly, launching a new autovacuum worker would result in waiting till it finished before shutting down. Also, switch the order of the code blocks in reaper() that detect startup process crash versus shutdown termination. Once we've sent it a signal, we should not consider that exit(1) is surprising. This is just a cosmetic fix since shutdown occurs correctly anyway, but better not to log a phony complaint about startup process crash. Back-patch to 9.0. Some parts of this might be applicable before that, but given the lack of prior complaints I'm not going to worry too much about older branches.
2012-11-21 21:18:38 +01:00
pmState == PM_RECOVERY && Shutdown == NoShutdown)
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
{
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
/*
* Likewise, start other special children as needed.
*/
Assert(PgStatPID == 0);
PgStatPID = pgstat_start();
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
Allow read only connections during recovery, known as Hot Standby. Enabled by recovery_connections = on (default) and forcing archive recovery using a recovery.conf. Recovery processing now emulates the original transactions as they are replayed, providing full locking and MVCC behaviour for read only queries. Recovery must enter consistent state before connections are allowed, so there is a delay, typically short, before connections succeed. Replay of recovering transactions can conflict and in some cases deadlock with queries during recovery; these result in query cancellation after max_standby_delay seconds have expired. Infrastructure changes have minor effects on normal running, though introduce four new types of WAL record. New test mode "make standbycheck" allows regression tests of static command behaviour on a standby server while in recovery. Typical and extreme dynamic behaviours have been checked via code inspection and manual testing. Few port specific behaviours have been utilised, though primary testing has been on Linux only so far. This commit is the basic patch. Additional changes will follow in this release to enhance some aspects of behaviour, notably improved handling of conflicts, deadlock detection and query cancellation. Changes to VACUUM FULL are also required. Simon Riggs, with significant and lengthy review by Heikki Linnakangas, including streamlined redesign of snapshot creation and two-phase commit. Important contributions from Florian Pflug, Mark Kirkwood, Merlin Moncure, Greg Stark, Gianni Ciolli, Gabriele Bartolini, Hannu Krosing, Robert Haas, Tatsuo Ishii, Hiroyuki Yamada plus support and feedback from many other community members.
2009-12-19 02:32:45 +01:00
ereport(LOG,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errmsg("database system is ready to accept read only connections")));
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
Change pg_ctl to detect server-ready by watching status in postmaster.pid. Traditionally, "pg_ctl start -w" has waited for the server to become ready to accept connections by attempting a connection once per second. That has the major problem that connection issues (for instance, a kernel packet filter blocking traffic) can't be reliably told apart from server startup issues, and the minor problem that if server startup isn't quick, we accumulate "the database system is starting up" spam in the server log. We've hacked around many of the possible connection issues, but it resulted in ugly and complicated code in pg_ctl.c. In commit c61559ec3, I changed the probe rate to every tenth of a second. That prompted Jeff Janes to complain that the log-spam problem had become much worse. In the ensuing discussion, Andres Freund pointed out that we could dispense with connection attempts altogether if the postmaster were changed to report its status in postmaster.pid, which "pg_ctl start" already relies on being able to read. This patch implements that, teaching postmaster.c to report a status string into the pidfile at the same state-change points already identified as being of interest for systemd status reporting (cf commit 7d17e683f). pg_ctl no longer needs to link with libpq at all; all its functions now depend on reading server files. In support of this, teach AddToDataDirLockFile() to allow addition of postmaster.pid lines in not-necessarily-sequential order. This is needed on Windows where the SHMEM_KEY line will never be written at all. We still have the restriction that we don't want to truncate the pidfile; document the reasons for that a bit better. Also, fix the pg_ctl TAP tests so they'll notice if "start -w" mode is broken --- before, they'd just wait out the sixty seconds until the loop gives up, and then report success anyway. (Yes, I found that out the hard way.) While at it, arrange for pg_ctl to not need to #include miscadmin.h; as a rather low-level backend header, requiring that to be compilable client-side is pretty dubious. This requires moving the #define's associated with the pidfile into a new header file, and moving PG_BACKEND_VERSIONSTR someplace else. For lack of a clearly better "someplace else", I put it into port.h, beside the declaration of find_other_exec(), since most users of that macro are passing the value to find_other_exec(). (initdb still depends on miscadmin.h, but at least pg_ctl and pg_upgrade no longer do.) In passing, fix main.c so that PG_BACKEND_VERSIONSTR actually defines the output of "postgres -V", which remarkably it had never done before. Discussion: https://postgr.es/m/CAMkU=1xJW8e+CTotojOMBd-yzUvD0e_JZu2xHo=MnuZ4__m7Pg@mail.gmail.com
2017-06-28 23:31:24 +02:00
/* Report status */
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_READY);
Add support for systemd service notifications Insert sd_notify() calls at server start and stop for integration with systemd. This allows the use of systemd service units of type "notify", which greatly simplifies the systemd configuration. Reviewed-by: Pavel Stěhule <pavel.stehule@gmail.com>
2015-11-17 12:46:17 +01:00
#ifdef USE_SYSTEMD
sd_notify(0, "READY=1");
#endif
Rename PM_RECOVERY_CONSISTENT and PMSIGNAL_RECOVERY_CONSISTENT. The new names PM_HOT_STANDBY and PMSIGNAL_BEGIN_HOT_STANDBY more accurately reflect their actual function.
2010-05-15 22:01:32 +02:00
pmState = PM_HOT_STANDBY;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/* Some workers may be scheduled to start now */
Eliminate one background-worker-related flag variable. Teach sigusr1_handler() to use the same test for whether a worker might need to be started as ServerLoop(). Aside from being perhaps a bit simpler, this prevents a potentially-unbounded delay when starting a background worker. On some platforms, select() doesn't return when interrupted by a signal, but is instead restarted, including a reset of the timeout to the originally-requested value. If signals arrive often enough, but no connection requests arrive, sigusr1_handler() will be executed repeatedly, but the body of ServerLoop() won't be reached. This change ensures that, even in that case, background workers will eventually get launched. This is far from a perfect fix; really, we need select() to return control to ServerLoop() after an interrupt, either via the self-pipe trick or some other mechanism. But that's going to require more work and discussion, so let's do this for now to at least mitigate the damage. Per investigation of test_shm_mq failures on buildfarm member anole.
2014-10-05 03:25:41 +02:00
StartWorkerNeeded = true;
Change the signaling of end-of-recovery. Startup process now indicates end of recovery by exiting with exit code 0, like in previous releases. Per Tom's suggestion.
2009-02-23 10:28:50 +01:00
}
Start background writer during archive recovery. Background writer now performs its usual buffer cleaning duties during archive recovery, and it's responsible for performing restartpoints. This requires some changes in postmaster. When the startup process has done all the initialization and is ready to start WAL redo, it signals the postmaster to launch the background writer. The postmaster is signaled again when the point in recovery is reached where we know that the database is in consistent state. Postmaster isn't interested in that at the moment, but that's the point where we could let other backends in to perform read-only queries. The postmaster is signaled third time when the recovery has ended, so that postmaster knows that it's safe to start accepting connections. The startup process now traps SIGTERM, and performs a "clean" shutdown. If you do a fast shutdown during recovery, a shutdown restartpoint is performed, like a shutdown checkpoint, and postmaster kills the processes cleanly. You still have to continue the recovery at next startup, though. Currently, the background writer is only launched during archive recovery. We could launch it during crash recovery as well, but it seems better to keep that codepath as simple as possible, for the sake of robustness. And it couldn't do any restartpoints during crash recovery anyway, so it wouldn't be that useful. log_restartpoints is gone. Use log_checkpoints instead. This is yet to be documented. This whole operation is a pre-requisite for Hot Standby, but has some value of its own whether the hot standby patch makes 8.4 or not. Simon Riggs, with lots of modifications by me.
2009-02-18 16:58:41 +01:00
Eliminate one background-worker-related flag variable. Teach sigusr1_handler() to use the same test for whether a worker might need to be started as ServerLoop(). Aside from being perhaps a bit simpler, this prevents a potentially-unbounded delay when starting a background worker. On some platforms, select() doesn't return when interrupted by a signal, but is instead restarted, including a reset of the timeout to the originally-requested value. If signals arrive often enough, but no connection requests arrive, sigusr1_handler() will be executed repeatedly, but the body of ServerLoop() won't be reached. This change ensures that, even in that case, background workers will eventually get launched. This is far from a perfect fix; really, we need select() to return control to ServerLoop() after an interrupt, either via the self-pipe trick or some other mechanism. But that's going to require more work and discussion, so let's do this for now to at least mitigate the damage. Per investigation of test_shm_mq failures on buildfarm member anole.
2014-10-05 03:25:41 +02:00
if (StartWorkerNeeded || HaveCrashedWorker)
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
maybe_start_bgworkers();
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
Code & docs review for server instrumentation patch. File timestamps should surely be timestamptz not timestamp; fix some but not all of the holes in check_and_make_absolute(); other minor cleanup. Also put in the missed catversion bump.
2005-08-12 20:23:56 +02:00
if (CheckPostmasterSignal(PMSIGNAL_WAKEN_ARCHIVER) &&
Restructure the shutdown procedure for the archiver process to allow it to finish archiving everything (when there's no error), and to eliminate various hazards as best we can. This fixes a previous 8.3 patch that caused the postmaster to kill and then restart the archiver during shutdown (!?). The new behavior is that the archiver is allowed to run unmolested until the bgwriter has exited; then it is sent SIGUSR2 to tell it to do a final archiving cycle and quit. We only SIGQUIT the archiver if we want a panic stop; this is important since SIGQUIT will also be sent to any active archive_command. The postmaster also now doesn't SIGQUIT the stats collector until the bgwriter is done, since the bgwriter can send stats messages in 8.3. The postmaster will not exit until both the archiver and stats collector are gone; this provides some defense (not too bulletproof) against conflicting archiver or stats collector processes being started by a new postmaster instance. We continue the prior practice that the archiver will check for postmaster death immediately before issuing any archive_command; that gives some additional protection against conflicting archivers. Also, modify the archiver process to notice SIGTERM and refuse to issue any more archive commands if it gets it. The postmaster doesn't ever send it SIGTERM; we assume that any such signal came from init and is a notice of impending whole-system shutdown. In this situation it seems imprudent to try to start new archive commands --- if they aren't extremely quick they're likely to get SIGKILL'd by init. All per discussion.
2008-01-11 01:54:09 +01:00
PgArchPID != 0)
XLOG file archiving and point-in-time recovery. There are still some loose ends and a glaring lack of documentation, but it basically works. Simon Riggs with some editorialization by Tom Lane.
2004-07-19 04:47:16 +02:00
{
Code & docs review for server instrumentation patch. File timestamps should surely be timestamptz not timestamp; fix some but not all of the holes in check_and_make_absolute(); other minor cleanup. Also put in the missed catversion bump.
2005-08-12 20:23:56 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Send SIGUSR1 to archiver process, to wake it up and begin archiving
Replace "transaction log" with "write-ahead log" This makes documentation and error messages match the renaming of "xlog" to "wal" in APIs and file naming.
2017-05-12 17:49:56 +02:00
* next WAL file.
Code & docs review for server instrumentation patch. File timestamps should surely be timestamptz not timestamp; fix some but not all of the holes in check_and_make_absolute(); other minor cleanup. Also put in the missed catversion bump.
2005-08-12 20:23:56 +02:00
*/
On systems that have setsid(2) (which should be just about everything except Windows), arrange for each postmaster child process to be its own process group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole process group not only the direct child process. This provides saner behavior for archive and recovery scripts; in particular, it's possible to shut down a warm-standby recovery server using "pg_ctl stop -m immediate", since delivery of SIGQUIT to the startup subprocess will result in killing the waiting recovery_command. Also, this makes Query Cancel and statement_timeout apply to scripts being run from backends via system(). (There is no support in the core backend for that, but it's widely done using untrusted PLs.) Per gripe from Stephen Harris and subsequent discussion.
2006-11-21 21:59:53 +01:00
signal_child(PgArchPID, SIGUSR1);
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
}
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
Implement "pg_ctl logrotate" command Currently there are two ways to trigger log rotation in logging collector process: call pg_rotate_logfile() SQL-function or send SIGUSR1 signal directly to logging collector process. However, it's nice to have more suitable way for external tools to do that, which wouldn't require SQL connection or knowledge of logging collector pid. This commit implements triggering log rotation by "pg_ctl logrotate" command. Discussion: https://postgr.es/m/20180416.115435.28153375.horiguchi.kyotaro%40lab.ntt.co.jp Author: Kyotaro Horiguchi, Alexander Kuzmenkov, Alexander Korotkov
2018-09-01 18:46:49 +02:00
/* Tell syslogger to rotate logfile if requested */
if (SysLoggerPID != 0)
Code & docs review for server instrumentation patch. File timestamps should surely be timestamptz not timestamp; fix some but not all of the holes in check_and_make_absolute(); other minor cleanup. Also put in the missed catversion bump.
2005-08-12 20:23:56 +02:00
{
Implement "pg_ctl logrotate" command Currently there are two ways to trigger log rotation in logging collector process: call pg_rotate_logfile() SQL-function or send SIGUSR1 signal directly to logging collector process. However, it's nice to have more suitable way for external tools to do that, which wouldn't require SQL connection or knowledge of logging collector pid. This commit implements triggering log rotation by "pg_ctl logrotate" command. Discussion: https://postgr.es/m/20180416.115435.28153375.horiguchi.kyotaro%40lab.ntt.co.jp Author: Kyotaro Horiguchi, Alexander Kuzmenkov, Alexander Korotkov
2018-09-01 18:46:49 +02:00
if (CheckLogrotateSignal())
{
signal_child(SysLoggerPID, SIGUSR1);
RemoveLogrotateSignalFiles();
}
else if (CheckPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE))
{
signal_child(SysLoggerPID, SIGUSR1);
}
Code & docs review for server instrumentation patch. File timestamps should surely be timestamptz not timestamp; fix some but not all of the holes in check_and_make_absolute(); other minor cleanup. Also put in the missed catversion bump.
2005-08-12 20:23:56 +02:00
}
Add files to do read I/O on the cluster directory: pg_stat_file() pg_read_file() pg_ls_dir() pg_reload_conf() pg_rotate_logfile() Dave Page Andreas Pflug
2005-08-12 05:25:13 +02:00
Don't launch new child processes after we've been told to shut down. Once we've received a shutdown signal (SIGINT or SIGTERM), we should not launch any more child processes, even if we get signals requesting such. The normal code path for spawning backends has always understood that, but the postmaster's infrastructure for hot standby and autovacuum didn't get the memo. As reported by Hari Babu in bug #7643, this could lead to failure to shut down at all in some cases, such as when SIGINT is received just before the startup process sends PMSIGNAL_RECOVERY_STARTED: we'd launch a bgwriter and checkpointer, and then those processes would have no idea that they ought to quit. Similarly, launching a new autovacuum worker would result in waiting till it finished before shutting down. Also, switch the order of the code blocks in reaper() that detect startup process crash versus shutdown termination. Once we've sent it a signal, we should not consider that exit(1) is surprising. This is just a cosmetic fix since shutdown occurs correctly anyway, but better not to log a phony complaint about startup process crash. Back-patch to 9.0. Some parts of this might be applicable before that, but given the lack of prior complaints I'm not going to worry too much about older branches.
2012-11-21 21:18:38 +01:00
if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_LAUNCHER) &&
Shutdown == NoShutdown)
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
{
Fix recently-understood problems with handling of XID freezing, particularly in PITR scenarios. We now WAL-log the replacement of old XIDs with FrozenTransactionId, so that such replacement is guaranteed to propagate to PITR slave databases. Also, rather than relying on hint-bit updates to be preserved, pg_clog is not truncated until all instances of an XID are known to have been replaced by FrozenTransactionId. Add new GUC variables and pg_autovacuum columns to allow management of the freezing policy, so that users can trade off the size of pg_clog against the amount of freezing work done. Revise the already-existing code that forces autovacuum of tables approaching the wraparound point to make it more bulletproof; also, revise the autovacuum logic so that anti-wraparound vacuuming is done per-table rather than per-database. initdb forced because of changes in pg_class, pg_database, and pg_autovacuum catalogs. Heikki Linnakangas, Simon Riggs, and Tom Lane.
2006-11-05 23:42:10 +01:00
/*
* Start one iteration of the autovacuum daemon, even if autovacuuming
* is nominally not enabled. This is so we can have an active defense
* against transaction ID wraparound. We set a flag for the main loop
* to do it rather than trying to do it here --- this is because the
* autovac process itself may send the signal, and we want to handle
* that by launching another iteration as soon as the current one
* completes.
*/
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
start_autovac_launcher = true;
Improve vacuum code to track minimum Xids per table instead of per database. To this end, add a couple of columns to pg_class, relminxid and relvacuumxid, based on which we calculate the pg_database columns after each vacuum. We now force all databases to be vacuumed, even template ones. A backend noticing too old a database (meaning pg_database.datminxid is in danger of falling behind Xid wraparound) will signal the postmaster, which in turn will start an autovacuum iteration to process the offending database. In principle this is only there to cope with frozen (non-connectable) databases without forcing users to set them to connectable, but it could force regular user database to go through a database-wide vacuum at any time. Maybe we should warn users about this somehow. Of course the real solution will be to use autovacuum all the time ;-) There are some additional improvements we could have in this area: for example the vacuum code could be smarter about not updating pg_database for each table when called by autovacuum, and do it only once the whole autovacuum iteration is done. I updated the system catalogs documentation, but I didn't modify the maintenance section. Also having some regression tests for this would be nice but it's not really a very straightforward thing to do. Catalog version bumped due to system catalog changes.
2006-07-10 18:20:52 +02:00
}
Don't launch new child processes after we've been told to shut down. Once we've received a shutdown signal (SIGINT or SIGTERM), we should not launch any more child processes, even if we get signals requesting such. The normal code path for spawning backends has always understood that, but the postmaster's infrastructure for hot standby and autovacuum didn't get the memo. As reported by Hari Babu in bug #7643, this could lead to failure to shut down at all in some cases, such as when SIGINT is received just before the startup process sends PMSIGNAL_RECOVERY_STARTED: we'd launch a bgwriter and checkpointer, and then those processes would have no idea that they ought to quit. Similarly, launching a new autovacuum worker would result in waiting till it finished before shutting down. Also, switch the order of the code blocks in reaper() that detect startup process crash versus shutdown termination. Once we've sent it a signal, we should not consider that exit(1) is surprising. This is just a cosmetic fix since shutdown occurs correctly anyway, but better not to log a phony complaint about startup process crash. Back-patch to 9.0. Some parts of this might be applicable before that, but given the lack of prior complaints I'm not going to worry too much about older branches.
2012-11-21 21:18:38 +01:00
if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_WORKER) &&
Shutdown == NoShutdown)
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
{
/* The autovacuum launcher wants us to start a worker process. */
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
StartAutovacuumWorker();
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
}
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
Don't lose walreceiver start requests due to race condition in postmaster. When a walreceiver dies, the startup process will notice that and send a PMSIGNAL_START_WALRECEIVER signal to the postmaster, asking for a new walreceiver to be launched. There's a race condition, which at least in HEAD is very easy to hit, whereby the postmaster might see that signal before it processes the SIGCHLD from the walreceiver process. In that situation, sigusr1_handler() just dropped the start request on the floor, reasoning that it must be redundant. Eventually, after 10 seconds (WALRCV_STARTUP_TIMEOUT), the startup process would make a fresh request --- but that's a long time if the connection could have been re-established almost immediately. Fix it by setting a state flag inside the postmaster that we won't clear until we do launch a walreceiver. In cases where that results in an extra walreceiver launch, it's up to the walreceiver to realize it's unwanted and go away --- but we have, and need, that logic anyway for the opposite race case. I came across this through investigating unexpected delays in the src/test/recovery TAP tests: it manifests there in test cases where a master server is stopped and restarted while leaving streaming slaves active. This logic has been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/21344.1498494720@sss.pgh.pa.us
2017-06-26 23:31:56 +02:00
if (CheckPostmasterSignal(PMSIGNAL_START_WALRECEIVER))
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
{
/* Startup Process wants us to start the walreceiver process. */
Don't lose walreceiver start requests due to race condition in postmaster. When a walreceiver dies, the startup process will notice that and send a PMSIGNAL_START_WALRECEIVER signal to the postmaster, asking for a new walreceiver to be launched. There's a race condition, which at least in HEAD is very easy to hit, whereby the postmaster might see that signal before it processes the SIGCHLD from the walreceiver process. In that situation, sigusr1_handler() just dropped the start request on the floor, reasoning that it must be redundant. Eventually, after 10 seconds (WALRCV_STARTUP_TIMEOUT), the startup process would make a fresh request --- but that's a long time if the connection could have been re-established almost immediately. Fix it by setting a state flag inside the postmaster that we won't clear until we do launch a walreceiver. In cases where that results in an extra walreceiver launch, it's up to the walreceiver to realize it's unwanted and go away --- but we have, and need, that logic anyway for the opposite race case. I came across this through investigating unexpected delays in the src/test/recovery TAP tests: it manifests there in test cases where a master server is stopped and restarted while leaving streaming slaves active. This logic has been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/21344.1498494720@sss.pgh.pa.us
2017-06-26 23:31:56 +02:00
/* Start immediately if possible, else remember request for later. */
WalReceiverRequested = true;
MaybeStartWalReceiver();
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
}
Fix some minor postmaster-state-machine issues. In sigusr1_handler, don't ignore PMSIGNAL_ADVANCE_STATE_MACHINE based on pmState. The restriction is unnecessary (PostmasterStateMachine should work in any state), not future-proof (since it makes too many assumptions about why the signal might be sent), and broken even today because a race condition can make it necessary to respond to the signal in PM_WAIT_READONLY state. The race condition seems unlikely, but if it did happen, a hot-standby postmaster could fail to shut down after receiving a smart-shutdown request. In MaybeStartWalReceiver, don't clear the WalReceiverRequested flag if the fork attempt fails. Leaving it set allows us to try again in future iterations of the postmaster idle loop. (The startup process would eventually send a fresh request signal, but this change may allow us to retry the fork sooner.) Remove an obsolete comment and unnecessary test in PostmasterStateMachine's handling of PM_SHUTDOWN_2 state. It's not possible to have a live walreceiver in that state, and AFAICT has not been possible since commit 5e85315ea. This isn't a live bug, but the false comment is quite confusing to readers. In passing, rearrange sigusr1_handler's CheckPromoteSignal tests so that we don't uselessly perform stat() calls that we're going to ignore the results of. Add some comments clarifying the behavior of MaybeStartWalReceiver; I very nearly rearranged it in a way that'd reintroduce the race condition fixed in e5d494d78. Mea culpa for not commenting that properly at the time. Back-patch to all supported branches. The PMSIGNAL_ADVANCE_STATE_MACHINE change is the only one of even minor significance, but we might as well keep this code in sync across branches. Discussion: https://postgr.es/m/9001.1556046681@sss.pgh.pa.us
2019-04-24 20:15:44 +02:00
/*
* Try to advance postmaster's state machine, if a child requests it.
*
* Be careful about the order of this action relative to sigusr1_handler's
* other actions. Generally, this should be after other actions, in case
* they have effects PostmasterStateMachine would need to know about.
* However, we should do it before the CheckPromoteSignal step, which
* cannot have any (immediate) effect on the state machine, but does
* depend on what state we're in now.
*/
if (CheckPostmasterSignal(PMSIGNAL_ADVANCE_STATE_MACHINE))
Avoid possible hang during smart shutdown. If a smart shutdown occurs just as a child is starting up, and the child subsequently becomes a walsender, there is a race condition: the postmaster might count the exstant backends, determine that there is one normal backend, and wait for it to die off. Had the walsender transition already occurred before the postmaster counted, it would have proceeded with the shutdown. To fix this, have each child that transforms into a walsender kick the postmaster just after doing so, so that the state machine is certain to advance. Fujii Masao
2011-04-04 01:42:00 +02:00
{
PostmasterStateMachine();
}
Fix some minor postmaster-state-machine issues. In sigusr1_handler, don't ignore PMSIGNAL_ADVANCE_STATE_MACHINE based on pmState. The restriction is unnecessary (PostmasterStateMachine should work in any state), not future-proof (since it makes too many assumptions about why the signal might be sent), and broken even today because a race condition can make it necessary to respond to the signal in PM_WAIT_READONLY state. The race condition seems unlikely, but if it did happen, a hot-standby postmaster could fail to shut down after receiving a smart-shutdown request. In MaybeStartWalReceiver, don't clear the WalReceiverRequested flag if the fork attempt fails. Leaving it set allows us to try again in future iterations of the postmaster idle loop. (The startup process would eventually send a fresh request signal, but this change may allow us to retry the fork sooner.) Remove an obsolete comment and unnecessary test in PostmasterStateMachine's handling of PM_SHUTDOWN_2 state. It's not possible to have a live walreceiver in that state, and AFAICT has not been possible since commit 5e85315ea. This isn't a live bug, but the false comment is quite confusing to readers. In passing, rearrange sigusr1_handler's CheckPromoteSignal tests so that we don't uselessly perform stat() calls that we're going to ignore the results of. Add some comments clarifying the behavior of MaybeStartWalReceiver; I very nearly rearranged it in a way that'd reintroduce the race condition fixed in e5d494d78. Mea culpa for not commenting that properly at the time. Back-patch to all supported branches. The PMSIGNAL_ADVANCE_STATE_MACHINE change is the only one of even minor significance, but we might as well keep this code in sync across branches. Discussion: https://postgr.es/m/9001.1556046681@sss.pgh.pa.us
2019-04-24 20:15:44 +02:00
if (StartupPID != 0 &&
pg_ctl promote Fujii Masao, reviewed by Robert Haas, Stephen Frost, and Magnus Hagander.
2011-02-16 03:28:48 +01:00
(pmState == PM_STARTUP || pmState == PM_RECOVERY ||
Fix some minor postmaster-state-machine issues. In sigusr1_handler, don't ignore PMSIGNAL_ADVANCE_STATE_MACHINE based on pmState. The restriction is unnecessary (PostmasterStateMachine should work in any state), not future-proof (since it makes too many assumptions about why the signal might be sent), and broken even today because a race condition can make it necessary to respond to the signal in PM_WAIT_READONLY state. The race condition seems unlikely, but if it did happen, a hot-standby postmaster could fail to shut down after receiving a smart-shutdown request. In MaybeStartWalReceiver, don't clear the WalReceiverRequested flag if the fork attempt fails. Leaving it set allows us to try again in future iterations of the postmaster idle loop. (The startup process would eventually send a fresh request signal, but this change may allow us to retry the fork sooner.) Remove an obsolete comment and unnecessary test in PostmasterStateMachine's handling of PM_SHUTDOWN_2 state. It's not possible to have a live walreceiver in that state, and AFAICT has not been possible since commit 5e85315ea. This isn't a live bug, but the false comment is quite confusing to readers. In passing, rearrange sigusr1_handler's CheckPromoteSignal tests so that we don't uselessly perform stat() calls that we're going to ignore the results of. Add some comments clarifying the behavior of MaybeStartWalReceiver; I very nearly rearranged it in a way that'd reintroduce the race condition fixed in e5d494d78. Mea culpa for not commenting that properly at the time. Back-patch to all supported branches. The PMSIGNAL_ADVANCE_STATE_MACHINE change is the only one of even minor significance, but we might as well keep this code in sync across branches. Discussion: https://postgr.es/m/9001.1556046681@sss.pgh.pa.us
2019-04-24 20:15:44 +02:00
pmState == PM_HOT_STANDBY || pmState == PM_WAIT_READONLY) &&
CheckPromoteSignal())
pg_ctl promote Fujii Masao, reviewed by Robert Haas, Stephen Frost, and Magnus Hagander.
2011-02-16 03:28:48 +01:00
{
/* Tell startup process to finish recovery */
signal_child(StartupPID, SIGUSR2);
}
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
#ifdef WIN32
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
PG_SETMASK(&UnBlockSig);
In the postmaster, rely on the signal infrastructure to block signals. POSIX sigaction(2) can be told to block a set of signals while a signal handler executes. Make use of that instead of manually blocking and unblocking signals in the postmaster's signal handlers. This should save a few cycles, and it also prevents recursive invocation of signal handlers when many signals arrive in close succession. We have seen buildfarm failures that seem to be due to postmaster stack overflow caused by such recursion (exacerbated by a Linux PPC64 kernel bug). This doesn't change anything about the way that it works on Windows. Somebody might consider adjusting port/win32/signal.c to let it work similarly, but I'm not in a position to do that. For the moment, just apply to HEAD. Possibly we should consider back-patching this, but it'd be good to let it age awhile first. Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2019-10-13 21:48:26 +02:00
#endif
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
errno = save_errno;
}
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
/*
Introduce timeout handling framework Management of timeouts was getting a little cumbersome; what we originally had was more than enough back when we were only concerned about deadlocks and query cancel; however, when we added timeouts for standby processes, the code got considerably messier. Since there are plans to add more complex timeouts, this seems a good time to introduce a central timeout handling module. External modules register their timeout handlers during process initialization, and later enable and disable them as they see fit using a simple API; timeout.c is in charge of keeping track of which timeouts are in effect at any time, installing a common SIGALRM signal handler, and calling setitimer() as appropriate to ensure timely firing of external handlers. timeout.c additionally supports pluggable modules to add their own timeouts, though this capability isn't exercised anywhere yet. Additionally, as of this commit, walsender processes are aware of timeouts; we had a preexisting bug there that made those ignore SIGALRM, thus being subject to unhandled deadlocks, particularly during the authentication phase. This has already been fixed in back branches in commit 0bf8eb2a, which see for more details. Main author: Zoltán Böszörményi Some review and cleanup by Álvaro Herrera Extensive reworking by Tom Lane
2012-07-17 00:43:21 +02:00
* SIGTERM or SIGQUIT while processing startup packet.
* Clean up and exit(1).
Remove the use of the pg_auth flat file for client authentication. (That flat file is now completely useless, but removal will come later.) To do this, postpone client authentication into the startup transaction that's run by InitPostgres. We still collect the startup packet and do SSL initialization (if needed) at the same time we did before. The AuthenticationTimeout is applied separately to startup packet collection and the actual authentication cycle. (This is a bit annoying, since it means a couple extra syscalls; but the signal handling requirements inside and outside a transaction are sufficiently different that it seems best to treat the timeouts as completely independent.) A small security disadvantage is that if the given database name is invalid, this will be reported to the client before any authentication happens. We could work around that by connecting to database "postgres" instead, but consensus seems to be that it's not worth introducing such surprising behavior. Processing of all command-line switches and GUC options received from the client is now postponed until after authentication. This means that PostAuthDelay is much less useful than it used to be --- if you need to investigate problems during InitPostgres you'll have to set PreAuthDelay instead. However, allowing an unauthenticated user to set any GUC options whatever seems a bit too risky, so we'll live with that.
2009-08-29 21:26:52 +02:00
*
* XXX: possible future improvement: try to send a message indicating
* why we are disconnecting. Problem is to be sure we don't block while
* doing so, nor mess up SSL initialization. In practice, if the client
* has wedged here, it probably couldn't do anything with the message anyway.
*/
static void
startup_die(SIGNAL_ARGS)
{
proc_exit(1);
}
From: todd brandys <brandys@eng3.hep.uiuc.edu> An extension to the code to allow for a pg_password authentication database that is *seperate* from the system password file
1997-12-04 01:28:15 +01:00
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
/*
* Dummy signal handler
*
* We use this for signals that we don't actually use in the postmaster,
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
* but we do use in backends. If we were to SIG_IGN such signals in the
* postmaster, then a newly started backend might drop a signal that arrives
* before it's able to reconfigure its signal processing. (See notes in
* tcop/postgres.c.)
Merge three existing ways of signaling postmaster from child processes, so that only one signal number is used not three. Flags in shared memory tell the reason(s) for the current signal. This method is extensible to handle more signal reasons without chewing up even more signal numbers, but the immediate reason is to keep pg_pwd reloads separate from SIGHUP processing in the postmaster. Also clean up some problems in the postmaster with delayed response to checkpoint status changes --- basically, it wouldn't schedule a checkpoint if it wasn't getting connection requests on a regular basis.
2001-11-04 20:55:31 +01:00
*/
static void
dummy_handler(SIGNAL_ARGS)
{
}
Introduce timeout handling framework Management of timeouts was getting a little cumbersome; what we originally had was more than enough back when we were only concerned about deadlocks and query cancel; however, when we added timeouts for standby processes, the code got considerably messier. Since there are plans to add more complex timeouts, this seems a good time to introduce a central timeout handling module. External modules register their timeout handlers during process initialization, and later enable and disable them as they see fit using a simple API; timeout.c is in charge of keeping track of which timeouts are in effect at any time, installing a common SIGALRM signal handler, and calling setitimer() as appropriate to ensure timely firing of external handlers. timeout.c additionally supports pluggable modules to add their own timeouts, though this capability isn't exercised anywhere yet. Additionally, as of this commit, walsender processes are aware of timeouts; we had a preexisting bug there that made those ignore SIGALRM, thus being subject to unhandled deadlocks, particularly during the authentication phase. This has already been fixed in back branches in commit 0bf8eb2a, which see for more details. Main author: Zoltán Böszörményi Some review and cleanup by Álvaro Herrera Extensive reworking by Tom Lane
2012-07-17 00:43:21 +02:00
/*
* Timeout while processing startup packet.
* As for startup_die(), we clean up and exit(1).
*/
static void
StartupPacketTimeoutHandler(void)
{
proc_exit(1);
}
Revert "Replace PostmasterRandom() with a stronger way of generating randomness." This reverts commit 9e083fd4683294f41544e6d0d72f6e258ff3a77c. That was a few bricks shy of a load: * Query cancel stopped working * Buildfarm member pademelon stopped working, because the box doesn't have /dev/urandom nor /dev/random. This clearly needs some more discussion, and a quite different patch, so revert for now.
2016-10-18 15:28:23 +02:00
/*
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
* Generate a random cancel key.
Revert "Replace PostmasterRandom() with a stronger way of generating randomness." This reverts commit 9e083fd4683294f41544e6d0d72f6e258ff3a77c. That was a few bricks shy of a load: * Query cancel stopped working * Buildfarm member pademelon stopped working, because the box doesn't have /dev/urandom nor /dev/random. This clearly needs some more discussion, and a quite different patch, so revert for now.
2016-10-18 15:28:23 +02:00
*/
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
static bool
RandomCancelKey(int32 *cancel_key)
Revert "Replace PostmasterRandom() with a stronger way of generating randomness." This reverts commit 9e083fd4683294f41544e6d0d72f6e258ff3a77c. That was a few bricks shy of a load: * Query cancel stopped working * Buildfarm member pademelon stopped working, because the box doesn't have /dev/urandom nor /dev/random. This clearly needs some more discussion, and a quite different patch, so revert for now.
2016-10-18 15:28:23 +02:00
{
Remove configure switch --disable-strong-random This removes a portion of infrastructure introduced by fe0a0b5 to allow compilation of Postgres in environments where no strong random source is available, meaning that there is no linking to OpenSSL and no /dev/urandom (Windows having its own CryptoAPI). No systems shipped this century lack /dev/urandom, and the buildfarm is actually not testing this switch at all, so just remove it. This simplifies particularly some backend code which included a fallback implementation using shared memory, and removes a set of alternate regression output files from pgcrypto. Author: Michael Paquier Reviewed-by: Tom Lane Discussion: https://postgr.es/m/20181230063219.GG608@paquier.xyz
2019-01-01 12:05:51 +01:00
return pg_strong_random(cancel_key, sizeof(int32));
Revert "Replace PostmasterRandom() with a stronger way of generating randomness." This reverts commit 9e083fd4683294f41544e6d0d72f6e258ff3a77c. That was a few bricks shy of a load: * Query cancel stopped working * Buildfarm member pademelon stopped working, because the box doesn't have /dev/urandom nor /dev/random. This clearly needs some more discussion, and a quite different patch, so revert for now.
2016-10-18 15:28:23 +02:00
}
Fix postmaster to not try to start more than MaxBackendId children, per patch from Tatsuo Ishii
1999-01-30 21:04:37 +01:00
/*
Fix typos in comments. Backpatch to all supported versions, where applicable, to make backpatching of future fixes go more smoothly. Josh Soref Discussion: https://www.postgresql.org/message-id/CACZqfqCf+5qRztLPgmmosr-B0Ye4srWzzw_mo4c_8_B_mtjmJQ@mail.gmail.com
2017-02-06 10:33:58 +01:00
* Count up number of child processes of specified types (dead_end children
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
* are always excluded).
Fix postmaster to not try to start more than MaxBackendId children, per patch from Tatsuo Ishii
1999-01-30 21:04:37 +01:00
*/
static int
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
CountChildren(int target)
Fix postmaster to not try to start more than MaxBackendId children, per patch from Tatsuo Ishii
1999-01-30 21:04:37 +01:00
{
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
dlist_iter iter;
Fix postmaster to not try to start more than MaxBackendId children, per patch from Tatsuo Ishii
1999-01-30 21:04:37 +01:00
int cnt = 0;
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
dlist_foreach(iter, &BackendList)
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
{
Embedded list interface Provide a common implementation of embedded singly-linked and doubly-linked lists. "Embedded" in the sense that the nodes' next/previous pointers exist within some larger struct; this design choice reduces memory allocation overhead. Most of the implementation uses inlineable functions (where supported), for performance. Some existing uses of both types of lists have been converted to the new code, for demonstration purposes. Other uses can (and probably will) be converted in the future. Since dllist.c is unused after this conversion, it has been removed. Author: Andres Freund Some tweaks by me Reviewed by Tom Lane, Peter Geoghegan
2012-10-16 22:36:30 +02:00
Backend *bp = dlist_container(Backend, elem, iter.cur);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
if (bp->dead_end)
continue;
Avoid treating WAL senders as normal backends. The previous coding treated anything that wasn't an autovacuum launcher as a normal backend, which is wrong now that we also have WAL senders. Fujii Masao, reviewed by Robert Haas, Alvaro Herrera, Tom Lane, and Bernd Helmle.
2011-01-22 04:20:06 +01:00
/*
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* Since target == BACKEND_TYPE_ALL is the most common case, we test
* it first and avoid touching shared memory for every child.
Avoid treating WAL senders as normal backends. The previous coding treated anything that wasn't an autovacuum launcher as a normal backend, which is wrong now that we also have WAL senders. Fujii Masao, reviewed by Robert Haas, Alvaro Herrera, Tom Lane, and Bernd Helmle.
2011-01-22 04:20:06 +01:00
*/
if (target != BACKEND_TYPE_ALL)
{
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
* Assign bkend_type for any recently announced WAL Sender
* processes.
*/
if (bp->bkend_type == BACKEND_TYPE_NORMAL &&
IsPostmasterChildWalSender(bp->child_slot))
bp->bkend_type = BACKEND_TYPE_WALSND;
Avoid treating WAL senders as normal backends. The previous coding treated anything that wasn't an autovacuum launcher as a normal backend, which is wrong now that we also have WAL senders. Fujii Masao, reviewed by Robert Haas, Alvaro Herrera, Tom Lane, and Bernd Helmle.
2011-01-22 04:20:06 +01:00
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
if (!(target & bp->bkend_type))
Avoid treating WAL senders as normal backends. The previous coding treated anything that wasn't an autovacuum launcher as a normal backend, which is wrong now that we also have WAL senders. Fujii Masao, reviewed by Robert Haas, Alvaro Herrera, Tom Lane, and Bernd Helmle.
2011-01-22 04:20:06 +01:00
continue;
}
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
cnt++;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
}
Fix postmaster to not try to start more than MaxBackendId children, per patch from Tatsuo Ishii
1999-01-30 21:04:37 +01:00
return cnt;
}
Lots of patches coming in from me today :-) When drawing up a very simple "text-drawing" of how the negotiation is done, I realised I had done this last part (fallback) in a very stupid way. Patch #4 fixes this, and does it in a much better way. Included is also the simple text-drawing of how the negotiation is done. //Magnus
1999-09-27 05:13:16 +02:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
/*
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
* StartChildProcess -- start an auxiliary process for the postmaster
Don't go belly-up if fork() fails for a routine checkpoint subprocess. Just try again later.
2001-03-14 18:58:46 +01:00
*
Do not let PostmasterContext survive into background workers. We don't want postmaster child processes to contain a copy of the postmaster's PostmasterContext. That would be a waste of memory at least, and at worst a security issue, since there are copies of the semi-sensitive pg_hba and pg_ident data in there. All other child process types delete the PostmasterContext after forking, but the original coding of the background worker patch (commit da07a1e85) did not do so. It appears that the only reason for that was to avoid copying the bgworker's MyBgworkerEntry out of that context; but the couple of additional statements needed to do so are hardly good justification for it. Hence, copy that data and then clear the context as other child processes do. Because this patch changes the memory context in which a bgworker function gains control, back-patching it would be a bit risky, so we won't fix this in back branches. The "security" complaint is pretty thin anyway for generic bgworkers; only with the introduction of parallel query is there any question of running untrusted code in a bgworker process. Discussion: <14111.1470082717@sss.pgh.pa.us>
2016-08-03 20:48:05 +02:00
* "type" determines what kind of child will be started. All child types
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
* initially go to AuxiliaryProcessMain, which will handle common setup.
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
*
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
* Return value of StartChildProcess is subprocess' PID, or 0 if failed
* to start subprocess.
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
*/
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
static pid_t
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
StartChildProcess(AuxProcType type)
Continued rearrangement to permit pgstat + BootstrapMain processes to be fork/exec'd, in the same mode as the previous patch for backends. Claudio Natoli
2003-12-25 04:52:51 +01:00
{
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
pid_t pid;
char *av[10];
int ac = 0;
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
char typebuf[32];
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/*
* Set up command-line arguments for subprocess
*/
av[ac++] = "postgres";
Continued rearrangement to permit pgstat + BootstrapMain processes to be fork/exec'd, in the same mode as the previous patch for backends. Claudio Natoli
2003-12-25 04:52:51 +01:00
Review uses of IsUnderPostmaster, change some tests to look at whereToSendOutput instead because they are really inquiring about the correct client communication protocol. Update some comments. This is pointing towards supporting regular FE/BE client protocol in a standalone backend, per discussion a month or so back.
2004-01-28 22:02:40 +01:00
#ifdef EXEC_BACKEND
Merge postmaster and postgres command into just postgres. postmaster symlink is kept for now for compatibility. To call single-user mode, use postgres --single.
2006-06-18 17:38:37 +02:00
av[ac++] = "--forkboot";
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
av[ac++] = NULL; /* filled in by postmaster_forkexec */
Review uses of IsUnderPostmaster, change some tests to look at whereToSendOutput instead because they are really inquiring about the correct client communication protocol. Update some comments. This is pointing towards supporting regular FE/BE client protocol in a standalone backend, per discussion a month or so back.
2004-01-28 22:02:40 +01:00
#endif
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
av[ac++] = typebuf;
Review uses of IsUnderPostmaster, change some tests to look at whereToSendOutput instead because they are really inquiring about the correct client communication protocol. Update some comments. This is pointing towards supporting regular FE/BE client protocol in a standalone backend, per discussion a month or so back.
2004-01-28 22:02:40 +01:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
av[ac] = NULL;
Assert(ac < lengthof(av));
Continued rearrangement to permit pgstat + BootstrapMain processes to be fork/exec'd, in the same mode as the previous patch for backends. Claudio Natoli
2003-12-25 04:52:51 +01:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
#ifdef EXEC_BACKEND
pid = postmaster_forkexec(ac, av);
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
#else /* !EXEC_BACKEND */
Refactor fork()-related code. We need to do various housekeeping tasks before we can invoke fork() -- flush stdio buffers, save and restore the profiling timer on Linux with LINUX_PROFILE, and handle BeOS stuff. This patch moves that code into a single function, fork_process(), instead of duplicating it at the various callsites of fork(). This patch doesn't address the EXEC_BACKEND case; there is room for further cleanup there.
2005-03-10 08:14:03 +01:00
pid = fork_process();
pgindent run.
2003-08-04 02:43:34 +02:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
if (pid == 0) /* child */
{
Commonalize process startup code. Move common code, that was duplicated in every postmaster child/every standalone process, into two functions in miscinit.c. Not only does that already result in a fair amount of net code reduction but it also makes it much easier to remove more duplication in the future. The prime motivation wasn't code deduplication though, but easier addition of new common code.
2015-01-13 13:12:37 +01:00
InitPostmasterChild();
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
When launching a child backend, take care to close file descriptors for any other client connections that may exist (which would only happen if another client is currently in the authentication cycle). This avoids wastage of open descriptors in a child. It might also explain peculiar behaviors like not closing connections when expected, since the kernel will probably not signal EOF as long as some other backend is randomly holding open a reference to the connection, even if the client went away long since ...
2001-02-08 01:35:10 +01:00
/* Close the postmaster's sockets */
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
ClosePostmasterPorts(false);
When launching a child backend, take care to close file descriptors for any other client connections that may exist (which would only happen if another client is currently in the authentication cycle). This avoids wastage of open descriptors in a child. It might also explain peculiar behaviors like not closing connections when expected, since the kernel will probably not signal EOF as long as some other backend is randomly holding open a reference to the connection, even if the client went away long since ...
2001-02-08 01:35:10 +01:00
Restructure error handling as recently discussed. It is now really possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-07-31 02:45:57 +02:00
/* Release postmaster's working memory context */
MemoryContextSwitchTo(TopMemoryContext);
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
AuxiliaryProcessMain(ac, av);
Get rid of not-very-portable fcntl(F_SETLK) mechanism for locking the Unix socket file, in favor of having an ordinary lockfile beside the socket file. Clean up a few robustness problems in the lockfile code. If postmaster is going to reject a connection request based on database state, it will now tell you so before authentication exchange not after. (Of course, a failure after is still possible if conditions change meanwhile, but this makes life easier for a yet-to-be-written pg_ping utility.)
2000-11-29 21:59:54 +01:00
ExitPostmaster(0);
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
}
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
#endif /* EXEC_BACKEND */
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
if (pid < 0)
{
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
/* in parent, fork failed */
int save_errno = errno;
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
errno = save_errno;
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
switch (type)
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
{
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
case StartupProcess:
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(LOG,
(errmsg("could not fork startup process: %m")));
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
break;
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
case BgWriterProcess:
Background writer process This first part of the background writer does no syncing at all. It's only purpose is to keep the LRU heads clean so that regular backends seldom to never have to call write(). Jan
2003-11-19 16:55:08 +01:00
ereport(LOG,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errmsg("could not fork background writer process: %m")));
Further work on elog cleanup: fix some bogosities in elog's logic about when to send what to which, prevent recursion by introducing new COMMERROR elog level for client-communication problems, get rid of direct writes to stderr in backend/libpq files, prevent non-error elogs from going to client during the authentication cycle.
2002-03-04 02:46:04 +01:00
break;
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
case CheckpointerProcess:
ereport(LOG,
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
(errmsg("could not fork checkpointer process: %m")));
Split work of bgwriter between 2 processes: bgwriter and checkpointer. bgwriter is now a much less important process, responsible for page cleaning duties only. checkpointer is now responsible for checkpoints and so has a key role in shutdown. Later patches will correct doc references to the now old idea that bgwriter performs checkpoints. Has beneficial effect on performance at high write rates, but mainly refactoring to more easily allow changes for power reduction by simplifying previously tortuous code around required to allow page cleaning and checkpointing to time slice in the same process. Patch by me, Review by Dickson Guedes
2011-11-01 18:14:47 +01:00
break;
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
case WalWriterProcess:
ereport(LOG,
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
(errmsg("could not fork WAL writer process: %m")));
Create a new dedicated Postgres process, "wal writer", which exists to write and fsync WAL at convenient intervals. For the moment it just tries to offload this work from backends, but soon it will be responsible for guaranteeing a maximum delay before asynchronously-committed transactions will be flushed to disk. This is a portion of Simon Riggs' async-commit patch, committed to CVS separately because a background WAL writer seems like it might be a good idea independently of the async-commit feature. I rebased walwriter.c on bgwriter.c because it seemed like a more appropriate way of handling signals; while the startup/shutdown logic in postmaster.c is more like autovac because we want walwriter to quit before we start the shutdown checkpoint.
2007-07-24 06:54:09 +02:00
break;
Introduce Streaming Replication. This includes two new kinds of postmaster processes, walsenders and walreceiver. Walreceiver is responsible for connecting to the primary server and streaming WAL to disk, while walsender runs in the primary server and streams WAL from disk to the client. Documentation still needs work, but the basics are there. We will probably pull the replication section to a new chapter later on, as well as the sections describing file-based replication. But let's do that as a separate patch, so that it's easier to see what has been added/changed. This patch also adds a new section to the chapter about FE/BE protocol, documenting the protocol used by walsender/walreceivxer. Bump catalog version because of two new functions, pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), for monitoring the progress of replication. Fujii Masao, with additional hacking by me
2010-01-15 10:19:10 +01:00
case WalReceiverProcess:
ereport(LOG,
(errmsg("could not fork WAL receiver process: %m")));
break;
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
default:
Error message editing in backend/libpq, backend/postmaster, backend/tcop. Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-22 21:00:12 +02:00
ereport(LOG,
(errmsg("could not fork process: %m")));
Convert some fprintf's to elog's.
2001-08-30 21:02:42 +02:00
break;
}
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Don't go belly-up if fork() fails for a routine checkpoint subprocess. Just try again later.
2001-03-14 18:58:46 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
Don't go belly-up if fork() fails for a routine checkpoint subprocess. Just try again later.
2001-03-14 18:58:46 +01:00
*/
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
if (type == StartupProcess)
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
ExitPostmaster(1);
return 0;
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
}
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
/*
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
* in parent, successful fork
XLOG (and related) changes: * Store two past checkpoint locations, not just one, in pg_control. On startup, we fall back to the older checkpoint if the newer one is unreadable. Also, a physical copy of the newest checkpoint record is kept in pg_control for possible use in disaster recovery (ie, complete loss of pg_xlog). Also add a version number for pg_control itself. Remove archdir from pg_control; it ought to be a GUC parameter, not a special case (not that it's implemented yet anyway). * Suppress successive checkpoint records when nothing has been entered in the WAL log since the last one. This is not so much to avoid I/O as to make it actually useful to keep track of the last two checkpoints. If the things are right next to each other then there's not a lot of redundancy gained... * Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs on alternate bytes. Polynomial borrowed from ECMA DLT1 standard. * Fix XLOG record length handling so that it will work at BLCKSZ = 32k. * Change XID allocation to work more like OID allocation. (This is of dubious necessity, but I think it's a good idea anyway.) * Fix a number of minor bugs, such as off-by-one logic for XLOG file wraparound at the 4 gig mark. * Add documentation and clean up some coding infelicities; move file format declarations out to include files where planned contrib utilities can get at them. * Checkpoint will now occur every CHECKPOINT_SEGMENTS log segments or every CHECKPOINT_TIMEOUT seconds, whichever comes first. It is also possible to force a checkpoint by sending SIGUSR1 to the postmaster (undocumented feature...) * Defend against kill -9 postmaster by storing shmem block's key and ID in postmaster.pid lockfile, and checking at startup to ensure that no processes are still connected to old shmem block (if it still exists). * Switch backends to accept SIGQUIT rather than SIGUSR1 for emergency stop, for symmetry with postmaster and xlog utilities. Clean up signal handling in bootstrap.c so that xlog utilities launched by postmaster will react to signals better. * Standalone bootstrap now grabs lockfile in target directory, as added insurance against running it in parallel with live postmaster.
2001-03-13 02:17:06 +01:00
*/
Don't go belly-up if fork() fails for a routine checkpoint subprocess. Just try again later.
2001-03-14 18:58:46 +01:00
return pid;
XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown. First step in cleaning up backend initialization code. Fix for FATAL: now FATAL is ERROR + exit.
1999-10-06 23:58:18 +02:00
}
Create postmaster.pid and postmaster.opts under $PGDATA
1999-12-03 07:26:34 +01:00
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
/*
* StartAutovacuumWorker
* Start an autovac worker process.
*
* This function is here because it enters the resulting PID into the
* postmaster's private backends list.
*
* NB -- this code very roughly matches BackendStartup.
*/
static void
StartAutovacuumWorker(void)
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Backend *bn;
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
/*
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
* If not in condition to run a process, don't try, but handle it like a
* fork failure. This does not normally happen, since the signal is only
* supposed to be sent by autovacuum launcher when it's OK to do it, but
* we have to check to avoid race-condition problems during DB state
* changes.
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
*/
Check for too many postmaster children before spawning a bgworker. The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-07 18:39:09 +02:00
if (canAcceptConnections(BACKEND_TYPE_AUTOVAC) == CAC_OK)
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
{
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
/*
Post-PG 10 beta1 pgindent run perltidy run not included.
2017-05-17 22:31:56 +02:00
* Compute the cancel key that will be assigned to this session. We
* probably don't need cancel keys for autovac workers, but we'd
* better have something random in the field to prevent unfriendly
* people from sending cancels to them.
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
*/
if (!RandomCancelKey(&MyCancelKey))
{
ereport(LOG,
(errcode(ERRCODE_INTERNAL_ERROR),
Further cleanup from the strong-random patch. Also use the new facility for generating RADIUS authenticator requests, and salt in chkpass extension. Reword the error messages to be nicer. Fix bogus error code used in the message in BackendStartup.
2016-12-12 10:55:32 +01:00
errmsg("could not generate random cancel key")));
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
return;
}
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
bn = (Backend *) malloc(sizeof(Backend));
if (bn)
Improve autovacuum launcher's ability to detect a problem in worker startup, by having the postmaster signal it when certain failures occur. This requires the postmaster setting a flag in shared memory, but should be as safe as the pmsignal.c code is. Also make sure the launcher honor's a postgresql.conf change turning it off on SIGHUP.
2007-06-25 18:09:03 +02:00
{
Revert "Replace PostmasterRandom() with a stronger way of generating randomness." This reverts commit 9e083fd4683294f41544e6d0d72f6e258ff3a77c. That was a few bricks shy of a load: * Query cancel stopped working * Buildfarm member pademelon stopped working, because the box doesn't have /dev/urandom nor /dev/random. This clearly needs some more discussion, and a quite different patch, so revert for now.
2016-10-18 15:28:23 +02:00
bn->cancel_key = MyCancelKey;
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
Revert "Replace PostmasterRandom() with a stronger way of generating randomness." This reverts commit 9e083fd4683294f41544e6d0d72f6e258ff3a77c. That was a few bricks shy of a load: * Query cancel stopped working * Buildfarm member pademelon stopped working, because the box doesn't have /dev/urandom nor /dev/random. This clearly needs some more discussion, and a quite different patch, so revert for now.
2016-10-18 15:28:23 +02:00
/* Autovac workers are not dead_end and need a child slot */
bn->dead_end = false;
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
Revert "Replace PostmasterRandom() with a stronger way of generating randomness." This reverts commit 9e083fd4683294f41544e6d0d72f6e258ff3a77c. That was a few bricks shy of a load: * Query cancel stopped working * Buildfarm member pademelon stopped working, because the box doesn't have /dev/urandom nor /dev/random. This clearly needs some more discussion, and a quite different patch, so revert for now.
2016-10-18 15:28:23 +02:00
bn->pid = StartAutoVacWorker();
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
dlist_push_head(&BackendList, &bn->elem);
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
#ifdef EXEC_BACKEND
Revert "Replace PostmasterRandom() with a stronger way of generating randomness." This reverts commit 9e083fd4683294f41544e6d0d72f6e258ff3a77c. That was a few bricks shy of a load: * Query cancel stopped working * Buildfarm member pademelon stopped working, because the box doesn't have /dev/urandom nor /dev/random. This clearly needs some more discussion, and a quite different patch, so revert for now.
2016-10-18 15:28:23 +02:00
ShmemBackendArrayAdd(bn);
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
#endif
Revert "Replace PostmasterRandom() with a stronger way of generating randomness." This reverts commit 9e083fd4683294f41544e6d0d72f6e258ff3a77c. That was a few bricks shy of a load: * Query cancel stopped working * Buildfarm member pademelon stopped working, because the box doesn't have /dev/urandom nor /dev/random. This clearly needs some more discussion, and a quite different patch, so revert for now.
2016-10-18 15:28:23 +02:00
/* all OK */
return;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
}
Improve autovacuum launcher's ability to detect a problem in worker startup, by having the postmaster signal it when certain failures occur. This requires the postmaster setting a flag in shared memory, but should be as safe as the pmsignal.c code is. Also make sure the launcher honor's a postgresql.conf change turning it off on SIGHUP.
2007-06-25 18:09:03 +02:00
Revert "Replace PostmasterRandom() with a stronger way of generating randomness." This reverts commit 9e083fd4683294f41544e6d0d72f6e258ff3a77c. That was a few bricks shy of a load: * Query cancel stopped working * Buildfarm member pademelon stopped working, because the box doesn't have /dev/urandom nor /dev/random. This clearly needs some more discussion, and a quite different patch, so revert for now.
2016-10-18 15:28:23 +02:00
/*
* fork failed, fall through to report -- actual error message was
* logged by StartAutoVacWorker
*/
(void) ReleasePostmasterChildSlot(bn->child_slot);
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
free(bn);
}
else
ereport(LOG,
(errcode(ERRCODE_OUT_OF_MEMORY),
errmsg("out of memory")));
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
}
Improve autovacuum launcher's ability to detect a problem in worker startup, by having the postmaster signal it when certain failures occur. This requires the postmaster setting a flag in shared memory, but should be as safe as the pmsignal.c code is. Also make sure the launcher honor's a postgresql.conf change turning it off on SIGHUP.
2007-06-25 18:09:03 +02:00
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* Report the failure to the launcher, if it's running. (If it's not, we
* might not even be connected to shared memory, so don't try to call
Avoid calling kill() in a postmaster signal handler. This causes problems when the system load is high, per report from Zdenek Kotala in <1250860954.1239.114.camel@localhost>; instead of calling kill directly, have the signal handler set a flag which is checked in ServerLoop. This way, the handler can return before being called again by a subsequent signal sent from the autovacuum launcher. Also, increase the sleep in the launcher in this failure path to 1 second. Backpatch to 8.3, which is when the signalling between autovacuum launcher/postmaster was introduced. Also, add a couple of ReleasePostmasterChildSlot calls in error paths; this part backpatched to 8.4 which is when the child slot stuff was introduced.
2009-08-24 19:23:02 +02:00
* AutoVacWorkerFailed.) Note that we also need to signal it so that it
* responds to the condition, but we don't do that here, instead waiting
* for ServerLoop to do it. This way we avoid a ping-pong signalling in
* quick succession between the autovac launcher and postmaster in case
* things get ugly.
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
*/
Improve autovacuum launcher's ability to detect a problem in worker startup, by having the postmaster signal it when certain failures occur. This requires the postmaster setting a flag in shared memory, but should be as safe as the pmsignal.c code is. Also make sure the launcher honor's a postgresql.conf change turning it off on SIGHUP.
2007-06-25 18:09:03 +02:00
if (AutoVacPID != 0)
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
{
AutoVacWorkerFailed();
Avoid calling kill() in a postmaster signal handler. This causes problems when the system load is high, per report from Zdenek Kotala in <1250860954.1239.114.camel@localhost>; instead of calling kill directly, have the signal handler set a flag which is checked in ServerLoop. This way, the handler can return before being called again by a subsequent signal sent from the autovacuum launcher. Also, increase the sleep in the launcher in this failure path to 1 second. Backpatch to 8.3, which is when the signalling between autovacuum launcher/postmaster was introduced. Also, add a couple of ReleasePostmasterChildSlot calls in error paths; this part backpatched to 8.4 which is when the child slot stuff was introduced.
2009-08-24 19:23:02 +02:00
avlauncher_needs_signal = true;
Revise postmaster startup/shutdown logic to eliminate the problem that a constant flow of new connection requests could prevent the postmaster from completing a shutdown or crash restart. This is done by labeling child processes that are "dead ends", that is, we know that they were launched only to tell a client that it can't connect. These processes are managed separately so that they don't confuse us into thinking that we can't advance to the next stage of a shutdown or restart sequence, until the very end where we must wait for them to drain out so we can delete the shmem segment. Per discussion of a misbehavior reported by Keaton Adams. Since this code was baroque already, and my first attempt at fixing the problem made it entirely impenetrable, I took the opportunity to rewrite it in a state-machine style. That eliminates some duplicated code sections and hopefully makes everything a bit clearer.
2007-08-09 03:18:43 +02:00
}
Restructure autovacuum in two processes: a dummy process, which runs continuously, and requests vacuum runs of "autovacuum workers" to postmaster. The workers do the actual vacuum work. This allows for future improvements, like allowing multiple autovacuum jobs running in parallel. For now, the code keeps the original behavior of having a single autovac process at any time by sleeping until the previous worker has finished.
2007-02-16 00:23:23 +01:00
}
Another round of those unportable config/build changes :-/ * Add option to build with OpenSSL out of the box. Fix thusly exposed bit rot. Although it compiles now, getting this to do something useful is left as an exercise. * Fix Kerberos options to defer checking for required libraries until all the other libraries are checked for. * Change default odbcinst.ini and krb5.srvtab path to PREFIX/etc. * Install work around for Autoconf's install-sh relative path anomaly. Get rid of old INSTL_*_OPTS variables, now that we don't need them anymore. * Use `gunzip -c' instead of g?zcat. Reportedly broke on AIX. * Look for only one of readline.h or readline/readline.h, not both. * Make check for PS_STRINGS cacheable. Don't test for the header files separately. * Disable fcntl(F_SETLK) test on Linux. * Substitute the standard GCC warnings set into CFLAGS in configure, don't add it on in Makefile.global. * Sweep through contrib tree to teach makefiles standard semantics. ... and in completely unrelated news: * Make postmaster.opts arbitrary options-aware. I still think we need to save the environment as well.
2000-07-09 15:14:19 +02:00
Don't lose walreceiver start requests due to race condition in postmaster. When a walreceiver dies, the startup process will notice that and send a PMSIGNAL_START_WALRECEIVER signal to the postmaster, asking for a new walreceiver to be launched. There's a race condition, which at least in HEAD is very easy to hit, whereby the postmaster might see that signal before it processes the SIGCHLD from the walreceiver process. In that situation, sigusr1_handler() just dropped the start request on the floor, reasoning that it must be redundant. Eventually, after 10 seconds (WALRCV_STARTUP_TIMEOUT), the startup process would make a fresh request --- but that's a long time if the connection could have been re-established almost immediately. Fix it by setting a state flag inside the postmaster that we won't clear until we do launch a walreceiver. In cases where that results in an extra walreceiver launch, it's up to the walreceiver to realize it's unwanted and go away --- but we have, and need, that logic anyway for the opposite race case. I came across this through investigating unexpected delays in the src/test/recovery TAP tests: it manifests there in test cases where a master server is stopped and restarted while leaving streaming slaves active. This logic has been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/21344.1498494720@sss.pgh.pa.us
2017-06-26 23:31:56 +02:00
/*
* MaybeStartWalReceiver
* Start the WAL receiver process, if not running and our state allows.
Fix some minor postmaster-state-machine issues. In sigusr1_handler, don't ignore PMSIGNAL_ADVANCE_STATE_MACHINE based on pmState. The restriction is unnecessary (PostmasterStateMachine should work in any state), not future-proof (since it makes too many assumptions about why the signal might be sent), and broken even today because a race condition can make it necessary to respond to the signal in PM_WAIT_READONLY state. The race condition seems unlikely, but if it did happen, a hot-standby postmaster could fail to shut down after receiving a smart-shutdown request. In MaybeStartWalReceiver, don't clear the WalReceiverRequested flag if the fork attempt fails. Leaving it set allows us to try again in future iterations of the postmaster idle loop. (The startup process would eventually send a fresh request signal, but this change may allow us to retry the fork sooner.) Remove an obsolete comment and unnecessary test in PostmasterStateMachine's handling of PM_SHUTDOWN_2 state. It's not possible to have a live walreceiver in that state, and AFAICT has not been possible since commit 5e85315ea. This isn't a live bug, but the false comment is quite confusing to readers. In passing, rearrange sigusr1_handler's CheckPromoteSignal tests so that we don't uselessly perform stat() calls that we're going to ignore the results of. Add some comments clarifying the behavior of MaybeStartWalReceiver; I very nearly rearranged it in a way that'd reintroduce the race condition fixed in e5d494d78. Mea culpa for not commenting that properly at the time. Back-patch to all supported branches. The PMSIGNAL_ADVANCE_STATE_MACHINE change is the only one of even minor significance, but we might as well keep this code in sync across branches. Discussion: https://postgr.es/m/9001.1556046681@sss.pgh.pa.us
2019-04-24 20:15:44 +02:00
*
* Note: if WalReceiverPID is already nonzero, it might seem that we should
* clear WalReceiverRequested. However, there's a race condition if the
* walreceiver terminates and the startup process immediately requests a new
* one: it's quite possible to get the signal for the request before reaping
* the dead walreceiver process. Better to risk launching an extra
* walreceiver than to miss launching one we need. (The walreceiver code
* has logic to recognize that it should go away if not needed.)
Don't lose walreceiver start requests due to race condition in postmaster. When a walreceiver dies, the startup process will notice that and send a PMSIGNAL_START_WALRECEIVER signal to the postmaster, asking for a new walreceiver to be launched. There's a race condition, which at least in HEAD is very easy to hit, whereby the postmaster might see that signal before it processes the SIGCHLD from the walreceiver process. In that situation, sigusr1_handler() just dropped the start request on the floor, reasoning that it must be redundant. Eventually, after 10 seconds (WALRCV_STARTUP_TIMEOUT), the startup process would make a fresh request --- but that's a long time if the connection could have been re-established almost immediately. Fix it by setting a state flag inside the postmaster that we won't clear until we do launch a walreceiver. In cases where that results in an extra walreceiver launch, it's up to the walreceiver to realize it's unwanted and go away --- but we have, and need, that logic anyway for the opposite race case. I came across this through investigating unexpected delays in the src/test/recovery TAP tests: it manifests there in test cases where a master server is stopped and restarted while leaving streaming slaves active. This logic has been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/21344.1498494720@sss.pgh.pa.us
2017-06-26 23:31:56 +02:00
*/
static void
MaybeStartWalReceiver(void)
{
if (WalReceiverPID == 0 &&
(pmState == PM_STARTUP || pmState == PM_RECOVERY ||
pmState == PM_HOT_STANDBY || pmState == PM_WAIT_READONLY) &&
Shutdown == NoShutdown)
{
WalReceiverPID = StartWalReceiver();
Fix some minor postmaster-state-machine issues. In sigusr1_handler, don't ignore PMSIGNAL_ADVANCE_STATE_MACHINE based on pmState. The restriction is unnecessary (PostmasterStateMachine should work in any state), not future-proof (since it makes too many assumptions about why the signal might be sent), and broken even today because a race condition can make it necessary to respond to the signal in PM_WAIT_READONLY state. The race condition seems unlikely, but if it did happen, a hot-standby postmaster could fail to shut down after receiving a smart-shutdown request. In MaybeStartWalReceiver, don't clear the WalReceiverRequested flag if the fork attempt fails. Leaving it set allows us to try again in future iterations of the postmaster idle loop. (The startup process would eventually send a fresh request signal, but this change may allow us to retry the fork sooner.) Remove an obsolete comment and unnecessary test in PostmasterStateMachine's handling of PM_SHUTDOWN_2 state. It's not possible to have a live walreceiver in that state, and AFAICT has not been possible since commit 5e85315ea. This isn't a live bug, but the false comment is quite confusing to readers. In passing, rearrange sigusr1_handler's CheckPromoteSignal tests so that we don't uselessly perform stat() calls that we're going to ignore the results of. Add some comments clarifying the behavior of MaybeStartWalReceiver; I very nearly rearranged it in a way that'd reintroduce the race condition fixed in e5d494d78. Mea culpa for not commenting that properly at the time. Back-patch to all supported branches. The PMSIGNAL_ADVANCE_STATE_MACHINE change is the only one of even minor significance, but we might as well keep this code in sync across branches. Discussion: https://postgr.es/m/9001.1556046681@sss.pgh.pa.us
2019-04-24 20:15:44 +02:00
if (WalReceiverPID != 0)
WalReceiverRequested = false;
/* else leave the flag set, so we'll try again later */
Don't lose walreceiver start requests due to race condition in postmaster. When a walreceiver dies, the startup process will notice that and send a PMSIGNAL_START_WALRECEIVER signal to the postmaster, asking for a new walreceiver to be launched. There's a race condition, which at least in HEAD is very easy to hit, whereby the postmaster might see that signal before it processes the SIGCHLD from the walreceiver process. In that situation, sigusr1_handler() just dropped the start request on the floor, reasoning that it must be redundant. Eventually, after 10 seconds (WALRCV_STARTUP_TIMEOUT), the startup process would make a fresh request --- but that's a long time if the connection could have been re-established almost immediately. Fix it by setting a state flag inside the postmaster that we won't clear until we do launch a walreceiver. In cases where that results in an extra walreceiver launch, it's up to the walreceiver to realize it's unwanted and go away --- but we have, and need, that logic anyway for the opposite race case. I came across this through investigating unexpected delays in the src/test/recovery TAP tests: it manifests there in test cases where a master server is stopped and restarted while leaving streaming slaves active. This logic has been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/21344.1498494720@sss.pgh.pa.us
2017-06-26 23:31:56 +02:00
}
}
Fix bugs regarding pid file.
1999-12-06 08:21:12 +01:00
/*
Move SetPidFile() and firends to utils/init/miscinit.c so that tcop/postgres.c can use them. Now we have an interlock between postmaster and postgres.
2000-01-09 13:13:24 +01:00
* Create the opts file
Fix bugs regarding pid file.
1999-12-06 08:21:12 +01:00
*/
Another round of those unportable config/build changes :-/ * Add option to build with OpenSSL out of the box. Fix thusly exposed bit rot. Although it compiles now, getting this to do something useful is left as an exercise. * Fix Kerberos options to defer checking for required libraries until all the other libraries are checked for. * Change default odbcinst.ini and krb5.srvtab path to PREFIX/etc. * Install work around for Autoconf's install-sh relative path anomaly. Get rid of old INSTL_*_OPTS variables, now that we don't need them anymore. * Use `gunzip -c' instead of g?zcat. Reportedly broke on AIX. * Look for only one of readline.h or readline/readline.h, not both. * Make check for PS_STRINGS cacheable. Don't test for the header files separately. * Disable fcntl(F_SETLK) test on Linux. * Substitute the standard GCC warnings set into CFLAGS in configure, don't add it on in Makefile.global. * Sweep through contrib tree to teach makefiles standard semantics. ... and in completely unrelated news: * Make postmaster.opts arbitrary options-aware. I still think we need to save the environment as well.
2000-07-09 15:14:19 +02:00
static bool
Reorganize backend code to more cleanly manage executable names and backend startup.
2004-05-14 00:45:04 +02:00
CreateOptsFile(int argc, char *argv[], char *fullprogname)
Create postmaster.pid and postmaster.opts under $PGDATA
1999-12-03 07:26:34 +01:00
{
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
FILE *fp;
elog mop-up: bring some straggling fprintf(stderr)'s into the elog world.
2003-07-27 23:49:55 +02:00
int i;
Create postmaster.pid and postmaster.opts under $PGDATA
1999-12-03 07:26:34 +01:00
Arrange for the postmaster (and standalone backends, initdb, etc) to chdir into PGDATA and subsequently use relative paths instead of absolute paths to access all files under PGDATA. This seems to give a small performance improvement, and it should make the system more robust against naive DBAs doing things like moving a database directory that has a live postmaster in it. Per recent discussion.
2005-07-04 06:51:52 +02:00
#define OPTS_FILE "postmaster.opts"
Create postmaster.pid and postmaster.opts under $PGDATA
1999-12-03 07:26:34 +01:00
Arrange for the postmaster (and standalone backends, initdb, etc) to chdir into PGDATA and subsequently use relative paths instead of absolute paths to access all files under PGDATA. This seems to give a small performance improvement, and it should make the system more robust against naive DBAs doing things like moving a database directory that has a live postmaster in it. Per recent discussion.
2005-07-04 06:51:52 +02:00
if ((fp = fopen(OPTS_FILE, "w")) == NULL)
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
{
Arrange for the postmaster (and standalone backends, initdb, etc) to chdir into PGDATA and subsequently use relative paths instead of absolute paths to access all files under PGDATA. This seems to give a small performance improvement, and it should make the system more robust against naive DBAs doing things like moving a database directory that has a live postmaster in it. Per recent discussion.
2005-07-04 06:51:52 +02:00
elog(LOG, "could not create file \"%s\": %m", OPTS_FILE);
Another round of those unportable config/build changes :-/ * Add option to build with OpenSSL out of the box. Fix thusly exposed bit rot. Although it compiles now, getting this to do something useful is left as an exercise. * Fix Kerberos options to defer checking for required libraries until all the other libraries are checked for. * Change default odbcinst.ini and krb5.srvtab path to PREFIX/etc. * Install work around for Autoconf's install-sh relative path anomaly. Get rid of old INSTL_*_OPTS variables, now that we don't need them anymore. * Use `gunzip -c' instead of g?zcat. Reportedly broke on AIX. * Look for only one of readline.h or readline/readline.h, not both. * Make check for PS_STRINGS cacheable. Don't test for the header files separately. * Disable fcntl(F_SETLK) test on Linux. * Substitute the standard GCC warnings set into CFLAGS in configure, don't add it on in Makefile.global. * Sweep through contrib tree to teach makefiles standard semantics. ... and in completely unrelated news: * Make postmaster.opts arbitrary options-aware. I still think we need to save the environment as well.
2000-07-09 15:14:19 +02:00
return false;
Create postmaster.pid and postmaster.opts under $PGDATA
1999-12-03 07:26:34 +01:00
}
Another round of those unportable config/build changes :-/ * Add option to build with OpenSSL out of the box. Fix thusly exposed bit rot. Although it compiles now, getting this to do something useful is left as an exercise. * Fix Kerberos options to defer checking for required libraries until all the other libraries are checked for. * Change default odbcinst.ini and krb5.srvtab path to PREFIX/etc. * Install work around for Autoconf's install-sh relative path anomaly. Get rid of old INSTL_*_OPTS variables, now that we don't need them anymore. * Use `gunzip -c' instead of g?zcat. Reportedly broke on AIX. * Look for only one of readline.h or readline/readline.h, not both. * Make check for PS_STRINGS cacheable. Don't test for the header files separately. * Disable fcntl(F_SETLK) test on Linux. * Substitute the standard GCC warnings set into CFLAGS in configure, don't add it on in Makefile.global. * Sweep through contrib tree to teach makefiles standard semantics. ... and in completely unrelated news: * Make postmaster.opts arbitrary options-aware. I still think we need to save the environment as well.
2000-07-09 15:14:19 +02:00
fprintf(fp, "%s", fullprogname);
for (i = 1; i < argc; i++)
Fix 'pg_ctl restart' to preserve command-line arguments.
2008-06-26 04:47:19 +02:00
fprintf(fp, " \"%s\"", argv[i]);
Another round of those unportable config/build changes :-/ * Add option to build with OpenSSL out of the box. Fix thusly exposed bit rot. Although it compiles now, getting this to do something useful is left as an exercise. * Fix Kerberos options to defer checking for required libraries until all the other libraries are checked for. * Change default odbcinst.ini and krb5.srvtab path to PREFIX/etc. * Install work around for Autoconf's install-sh relative path anomaly. Get rid of old INSTL_*_OPTS variables, now that we don't need them anymore. * Use `gunzip -c' instead of g?zcat. Reportedly broke on AIX. * Look for only one of readline.h or readline/readline.h, not both. * Make check for PS_STRINGS cacheable. Don't test for the header files separately. * Disable fcntl(F_SETLK) test on Linux. * Substitute the standard GCC warnings set into CFLAGS in configure, don't add it on in Makefile.global. * Sweep through contrib tree to teach makefiles standard semantics. ... and in completely unrelated news: * Make postmaster.opts arbitrary options-aware. I still think we need to save the environment as well.
2000-07-09 15:14:19 +02:00
fputs("\n", fp);
Create postmaster.pid and postmaster.opts under $PGDATA
1999-12-03 07:26:34 +01:00
Ensure that close() and fclose() are checked for errors, at least in cases involving writes. Per recent discussion about the possibility of close-time failures on some filesystems. There is a TODO item for this, too.
2004-01-26 23:35:32 +01:00
if (fclose(fp))
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
{
Arrange for the postmaster (and standalone backends, initdb, etc) to chdir into PGDATA and subsequently use relative paths instead of absolute paths to access all files under PGDATA. This seems to give a small performance improvement, and it should make the system more robust against naive DBAs doing things like moving a database directory that has a live postmaster in it. Per recent discussion.
2005-07-04 06:51:52 +02:00
elog(LOG, "could not write file \"%s\": %m", OPTS_FILE);
Another round of those unportable config/build changes :-/ * Add option to build with OpenSSL out of the box. Fix thusly exposed bit rot. Although it compiles now, getting this to do something useful is left as an exercise. * Fix Kerberos options to defer checking for required libraries until all the other libraries are checked for. * Change default odbcinst.ini and krb5.srvtab path to PREFIX/etc. * Install work around for Autoconf's install-sh relative path anomaly. Get rid of old INSTL_*_OPTS variables, now that we don't need them anymore. * Use `gunzip -c' instead of g?zcat. Reportedly broke on AIX. * Look for only one of readline.h or readline/readline.h, not both. * Make check for PS_STRINGS cacheable. Don't test for the header files separately. * Disable fcntl(F_SETLK) test on Linux. * Substitute the standard GCC warnings set into CFLAGS in configure, don't add it on in Makefile.global. * Sweep through contrib tree to teach makefiles standard semantics. ... and in completely unrelated news: * Make postmaster.opts arbitrary options-aware. I still think we need to save the environment as well.
2000-07-09 15:14:19 +02:00
return false;
Create postmaster.pid and postmaster.opts under $PGDATA
1999-12-03 07:26:34 +01:00
}
Another round of those unportable config/build changes :-/ * Add option to build with OpenSSL out of the box. Fix thusly exposed bit rot. Although it compiles now, getting this to do something useful is left as an exercise. * Fix Kerberos options to defer checking for required libraries until all the other libraries are checked for. * Change default odbcinst.ini and krb5.srvtab path to PREFIX/etc. * Install work around for Autoconf's install-sh relative path anomaly. Get rid of old INSTL_*_OPTS variables, now that we don't need them anymore. * Use `gunzip -c' instead of g?zcat. Reportedly broke on AIX. * Look for only one of readline.h or readline/readline.h, not both. * Make check for PS_STRINGS cacheable. Don't test for the header files separately. * Disable fcntl(F_SETLK) test on Linux. * Substitute the standard GCC warnings set into CFLAGS in configure, don't add it on in Makefile.global. * Sweep through contrib tree to teach makefiles standard semantics. ... and in completely unrelated news: * Make postmaster.opts arbitrary options-aware. I still think we need to save the environment as well.
2000-07-09 15:14:19 +02:00
return true;
Create postmaster.pid and postmaster.opts under $PGDATA
1999-12-03 07:26:34 +01:00
}
Mark many strings in backend not covered by elog for translation. Also, make strings in xlog.c look more like English and less like binary noise.
2001-06-03 16:53:56 +02:00
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
/*
* MaxLivePostmasterChildren
*
* This reports the number of entries needed in per-child-process arrays
* (the PMChildFlags array, and if EXEC_BACKEND the ShmemBackendArray).
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
* These arrays include regular backends, autovac workers, walsenders
* and background workers, but not special children nor dead_end children.
* This allows the arrays to have a fixed maximum size, to wit the same
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* too-many-children limit enforced by canAcceptConnections(). The exact value
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
* isn't too critical as long as it's more than MaxBackends.
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
*/
int
MaxLivePostmasterChildren(void)
{
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
return 2 * (MaxConnections + autovacuum_max_workers + 1 +
Move max_wal_senders out of max_connections for connection slot handling Since its introduction, max_wal_senders is counted as part of max_connections when it comes to define how many connection slots can be used for replication connections with a WAL sender context. This can lead to confusion for some users, as it could be possible to block a base backup or replication from happening because other backend sessions are already taken for other purposes by an application, and superuser-only connection slots are not a correct solution to handle that case. This commit makes max_wal_senders independent of max_connections for its handling of PGPROC entries in ProcGlobal, meaning that connection slots for WAL senders are handled using their own free queue, like autovacuum workers and bgworkers. One compatibility issue that this change creates is that a standby now requires to have a value of max_wal_senders at least equal to its primary. So, if a standby created enforces the value of max_wal_senders to be lower than that, then this could break failovers. Normally this should not be an issue though, as any settings of a standby are inherited from its primary as postgresql.conf gets normally copied as part of a base backup, so parameters would be consistent. Author: Alexander Kukushkin Reviewed-by: Kyotaro Horiguchi, Petr Jelínek, Masahiko Sawada, Oleksii Kliukin Discussion: https://postgr.es/m/CAFh8B=nBzHQeYAu0b8fjK-AF1X4+_p6GRtwG+cCgs6Vci2uRuQ@mail.gmail.com
2019-02-12 02:07:56 +01:00
max_wal_senders + max_worker_processes);
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
* Connect background worker to a database.
*/
void
Allow background workers to bypass datallowconn THis adds a "flags" field to the BackgroundWorkerInitializeConnection() and BackgroundWorkerInitializeConnectionByOid(). For now only one flag, BGWORKER_BYPASS_ALLOWCONN, is defined, which allows the worker to ignore datallowconn.
2018-04-05 18:59:32 +02:00
BackgroundWorkerInitializeConnection(const char *dbname, const char *username, uint32 flags)
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
{
BackgroundWorker *worker = MyBgworkerEntry;
/* XXX is this the right errcode? */
if (!(worker->bgw_flags & BGWORKER_BACKEND_DATABASE_CONNECTION))
ereport(FATAL,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("database connection requirement not indicated during registration")));
Allow background workers to bypass datallowconn THis adds a "flags" field to the BackgroundWorkerInitializeConnection() and BackgroundWorkerInitializeConnectionByOid(). For now only one flag, BGWORKER_BYPASS_ALLOWCONN, is defined, which allows the worker to ignore datallowconn.
2018-04-05 18:59:32 +02:00
InitPostgres(dbname, InvalidOid, username, InvalidOid, NULL, (flags & BGWORKER_BYPASS_ALLOWCONN) != 0);
Add new function BackgroundWorkerInitializeConnectionByOid. Sometimes it's useful for a background worker to be able to initialize its database connection by OID rather than by name, so provide a way to do that.
2015-02-02 22:23:59 +01:00
/* it had better not gotten out of "init" mode yet */
if (!IsInitProcessingMode())
ereport(ERROR,
(errmsg("invalid processing mode in background worker")));
SetProcessingMode(NormalProcessing);
}
/*
* Connect background worker to a database using OIDs.
*/
void
Allow background workers to bypass datallowconn THis adds a "flags" field to the BackgroundWorkerInitializeConnection() and BackgroundWorkerInitializeConnectionByOid(). For now only one flag, BGWORKER_BYPASS_ALLOWCONN, is defined, which allows the worker to ignore datallowconn.
2018-04-05 18:59:32 +02:00
BackgroundWorkerInitializeConnectionByOid(Oid dboid, Oid useroid, uint32 flags)
Add new function BackgroundWorkerInitializeConnectionByOid. Sometimes it's useful for a background worker to be able to initialize its database connection by OID rather than by name, so provide a way to do that.
2015-02-02 22:23:59 +01:00
{
BackgroundWorker *worker = MyBgworkerEntry;
/* XXX is this the right errcode? */
if (!(worker->bgw_flags & BGWORKER_BACKEND_DATABASE_CONNECTION))
ereport(FATAL,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("database connection requirement not indicated during registration")));
Allow background workers to bypass datallowconn THis adds a "flags" field to the BackgroundWorkerInitializeConnection() and BackgroundWorkerInitializeConnectionByOid(). For now only one flag, BGWORKER_BYPASS_ALLOWCONN, is defined, which allows the worker to ignore datallowconn.
2018-04-05 18:59:32 +02:00
InitPostgres(NULL, dboid, NULL, useroid, NULL, (flags & BGWORKER_BYPASS_ALLOWCONN) != 0);
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/* it had better not gotten out of "init" mode yet */
if (!IsInitProcessingMode())
ereport(ERROR,
Message style improvements
2013-08-08 04:48:40 +02:00
(errmsg("invalid processing mode in background worker")));
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
SetProcessingMode(NormalProcessing);
}
/*
* Block/unblock signals in a background worker
*/
void
BackgroundWorkerBlockSignals(void)
{
PG_SETMASK(&BlockSig);
}
void
BackgroundWorkerUnblockSignals(void)
{
PG_SETMASK(&UnBlockSig);
}
#ifdef EXEC_BACKEND
static pid_t
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
bgworker_forkexec(int shmem_slot)
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
{
char *av[10];
int ac = 0;
char forkav[MAXPGPATH];
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
av[ac++] = "postgres";
av[ac++] = forkav;
av[ac++] = NULL; /* filled in by postmaster_forkexec */
av[ac] = NULL;
Assert(ac < lengthof(av));
return postmaster_forkexec(ac, av);
}
#endif
/*
* Start a new bgworker.
* Starting time conditions must have been checked already.
*
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
* Returns true on success, false on failure.
* In either case, update the RegisteredBgWorker's state appropriately.
*
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
* This code is heavily based on autovacuum.c, q.v.
*/
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
static bool
Move more bgworker code to bgworker.c; also, some renaming. Per discussion on pgsql-hackers. Michael Paquier, slightly modified by me. Original suggestion from Amit Kapila.
2013-08-16 21:14:54 +02:00
do_start_bgworker(RegisteredBgWorker *rw)
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
{
pid_t worker_pid;
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
Assert(rw->rw_pid == 0);
/*
* Allocate and assign the Backend element. Note we must do this before
Check for too many postmaster children before spawning a bgworker. The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-07 18:39:09 +02:00
* forking, so that we can handle failures (out of memory or child-process
* slots) cleanly.
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
*
* Treat failure as though the worker had crashed. That way, the
Check for too many postmaster children before spawning a bgworker. The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-07 18:39:09 +02:00
* postmaster will wait a bit before attempting to start it again; if we
* tried again right away, most likely we'd find ourselves hitting the
* same resource-exhaustion condition.
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
*/
if (!assign_backendlist_entry(rw))
{
rw->rw_crashed_at = GetCurrentTimestamp();
return false;
}
Reduce log level for background worker events from LOG to DEBUG1. Per discussion, LOG is just too chatty for something that will happen as routinely as this. Pavel Stehule
2015-06-26 17:23:32 +02:00
ereport(DEBUG1,
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
(errmsg("starting background worker process \"%s\"",
rw->rw_worker.bgw_name)));
#ifdef EXEC_BACKEND
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
#else
switch ((worker_pid = fork_process()))
#endif
{
case -1:
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
/* in postmaster, fork failed ... */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
ereport(LOG,
(errmsg("could not fork worker process: %m")));
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
/* undo what assign_backendlist_entry did */
ReleasePostmasterChildSlot(rw->rw_child_slot);
rw->rw_child_slot = 0;
free(rw->rw_backend);
rw->rw_backend = NULL;
/* mark entry as crashed, so we'll try again later */
rw->rw_crashed_at = GetCurrentTimestamp();
break;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
#ifndef EXEC_BACKEND
case 0:
/* in postmaster child ... */
Commonalize process startup code. Move common code, that was duplicated in every postmaster child/every standalone process, into two functions in miscinit.c. Not only does that already result in a fair amount of net code reduction but it also makes it much easier to remove more duplication in the future. The prime motivation wasn't code deduplication though, but easier addition of new common code.
2015-01-13 13:12:37 +01:00
InitPostmasterChild();
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/* Close the postmaster's sockets */
ClosePostmasterPorts(false);
Do not let PostmasterContext survive into background workers. We don't want postmaster child processes to contain a copy of the postmaster's PostmasterContext. That would be a waste of memory at least, and at worst a security issue, since there are copies of the semi-sensitive pg_hba and pg_ident data in there. All other child process types delete the PostmasterContext after forking, but the original coding of the background worker patch (commit da07a1e85) did not do so. It appears that the only reason for that was to avoid copying the bgworker's MyBgworkerEntry out of that context; but the couple of additional statements needed to do so are hardly good justification for it. Hence, copy that data and then clear the context as other child processes do. Because this patch changes the memory context in which a bgworker function gains control, back-patching it would be a bit risky, so we won't fix this in back branches. The "security" complaint is pretty thin anyway for generic bgworkers; only with the introduction of parallel query is there any question of running untrusted code in a bgworker process. Discussion: <14111.1470082717@sss.pgh.pa.us>
2016-08-03 20:48:05 +02:00
/*
* Before blowing away PostmasterContext, save this bgworker's
* data where it can find it.
*/
MyBgworkerEntry = (BackgroundWorker *)
MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
/* Release postmaster's working memory context */
MemoryContextSwitchTo(TopMemoryContext);
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
Move more bgworker code to bgworker.c; also, some renaming. Per discussion on pgsql-hackers. Michael Paquier, slightly modified by me. Original suggestion from Amit Kapila.
2013-08-16 21:14:54 +02:00
StartBackgroundWorker();
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
exit(1); /* should not get here */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
break;
#endif
default:
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
/* in postmaster, fork successful ... */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
rw->rw_pid = worker_pid;
Allow notifications to bgworkers without database connections. Previously, if one background worker registered another background worker and set bgw_notify_pid while for the second background worker, it would not receive notifications from the postmaster unless, at the time the "parent" was registered, BGWORKER_BACKEND_DATABASE_CONNECTION was set. To fix, instead instead of including only those background workers that requested database connections in the postmater's BackendList, include them all. There doesn't seem to be any reason not do this, and indeed it removes a significant amount of duplicated code. The other option is to make PostmasterMarkPIDForWorkerNotify look at BackgroundWorkerList in addition to BackendList, but that adds more code duplication instead of getting rid of it. Patch by me. Review and testing by Ashutosh Bapat.
2015-09-01 21:30:19 +02:00
rw->rw_backend->pid = rw->rw_pid;
Allow discovery of whether a dynamic background worker is running. Using the infrastructure provided by this patch, it's possible either to wait for the startup of a dynamically-registered background worker, or to poll the status of such a worker without waiting. In either case, the current PID of the worker process can also be obtained. As usual, worker_spi is updated to demonstrate the new functionality. Patch by me. Review by Andres Freund.
2013-08-28 20:08:13 +02:00
ReportBackgroundWorkerPID(rw);
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
/* add new worker to lists of backends */
dlist_push_head(&BackendList, &rw->rw_backend->elem);
#ifdef EXEC_BACKEND
ShmemBackendArrayAdd(rw->rw_backend);
#endif
return true;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
}
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
return false;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
}
/*
* Does the current postmaster state require starting a worker with the
* specified start_time?
*/
static bool
bgworker_should_start_now(BgWorkerStartTime start_time)
{
switch (pmState)
{
case PM_NO_CHILDREN:
case PM_WAIT_DEAD_END:
case PM_SHUTDOWN_2:
case PM_SHUTDOWN:
case PM_WAIT_BACKENDS:
case PM_WAIT_READONLY:
case PM_WAIT_BACKUP:
break;
case PM_RUN:
if (start_time == BgWorkerStart_RecoveryFinished)
return true;
/* fall through */
case PM_HOT_STANDBY:
if (start_time == BgWorkerStart_ConsistentState)
return true;
/* fall through */
case PM_RECOVERY:
case PM_STARTUP:
case PM_INIT:
if (start_time == BgWorkerStart_PostmasterStart)
return true;
/* fall through */
}
return false;
}
/*
* Allocate the Backend struct for a connected background worker, but don't
* add it to the list of backends just yet.
*
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
* On failure, return false without changing any worker state.
*
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
* Some info from the Backend is copied into the passed rw.
*/
static bool
assign_backendlist_entry(RegisteredBgWorker *rw)
{
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
Backend *bn;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
Check for too many postmaster children before spawning a bgworker. The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-07 18:39:09 +02:00
/*
* Check that database state allows another connection. Currently the
* only possible failure is CAC_TOOMANY, so we just log an error message
* based on that rather than checking the error code precisely.
*/
if (canAcceptConnections(BACKEND_TYPE_BGWORKER) != CAC_OK)
{
ereport(LOG,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
errmsg("no slot available for new worker process")));
return false;
}
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
/*
* Compute the cancel key that will be assigned to this session. We
* probably don't need cancel keys for background workers, but we'd better
* have something random in the field to prevent unfriendly people from
* sending cancels to them.
*/
if (!RandomCancelKey(&MyCancelKey))
{
ereport(LOG,
(errcode(ERRCODE_INTERNAL_ERROR),
Further cleanup from the strong-random patch. Also use the new facility for generating RADIUS authenticator requests, and salt in chkpass extension. Reword the error messages to be nicer. Fix bogus error code used in the message in BackendStartup.
2016-12-12 10:55:32 +01:00
errmsg("could not generate random cancel key")));
Replace PostmasterRandom() with a stronger source, second attempt. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom Unlike the current pgcrypto function, the source is chosen by configure. That makes it easier to test different implementations, and ensures that we don't accidentally fall back to a less secure implementation, if the primary source fails. All of those methods are quite reliable, it would be pretty surprising for them to fail, so we'd rather find out by failing hard. If no strong random source is available, we fall back to using erand48(), seeded from current timestamp, like PostmasterRandom() was. That isn't cryptographically secure, but allows us to still work on platforms that don't have any of the above stronger sources. Because it's not very secure, the built-in implementation is only used if explicitly requested with --disable-strong-random. This replaces the more complicated Fortuna algorithm we used to have in pgcrypto, which is unfortunate, but all modern platforms have /dev/urandom, so it doesn't seem worth the maintenance effort to keep that. pgcrypto functions that require strong random numbers will be disabled with --disable-strong-random. Original patch by Magnus Hagander, tons of further work by Michael Paquier and me. Discussion: https://www.postgresql.org/message-id/CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com Discussion: https://www.postgresql.org/message-id/CAB7nPqRWkNYRRPJA7-cF+LfroYV10pvjdz6GNvxk-Eee9FypKA@mail.gmail.com
2016-12-05 12:42:59 +01:00
return false;
}
bn = malloc(sizeof(Backend));
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
if (bn == NULL)
{
ereport(LOG,
(errcode(ERRCODE_OUT_OF_MEMORY),
errmsg("out of memory")));
return false;
}
bn->cancel_key = MyCancelKey;
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bkend_type = BACKEND_TYPE_BGWORKER;
bn->dead_end = false;
Allow discovery of whether a dynamic background worker is running. Using the infrastructure provided by this patch, it's possible either to wait for the startup of a dynamically-registered background worker, or to poll the status of such a worker without waiting. In either case, the current PID of the worker process can also be obtained. As usual, worker_spi is updated to demonstrate the new functionality. Patch by me. Review by Andres Freund.
2013-08-28 20:08:13 +02:00
bn->bgworker_notify = false;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
rw->rw_backend = bn;
rw->rw_child_slot = bn->child_slot;
return true;
}
/*
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
* If the time is right, start background worker(s).
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
*
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
* As a side effect, the bgworker control variables are set or reset
* depending on whether more workers may need to be started.
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
*
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
* We limit the number of workers started per call, to avoid consuming the
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
* postmaster's attention for too long when many such requests are pending.
* As long as StartWorkerNeeded is true, ServerLoop will not block and will
* call this function again after dealing with any other issues.
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
*/
static void
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
maybe_start_bgworkers(void)
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
{
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
#define MAX_BGWORKERS_TO_LAUNCH 100
int num_launched = 0;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
TimestampTz now = 0;
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
slist_mutable_iter iter;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
/*
* During crash recovery, we have no need to be called until the state
* transition out of recovery.
*/
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
if (FatalError)
{
StartWorkerNeeded = false;
HaveCrashedWorker = false;
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
return;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
}
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
/* Don't need to be called again unless we find a reason for it below */
StartWorkerNeeded = false;
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
HaveCrashedWorker = false;
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
slist_foreach_modify(iter, &BackgroundWorkerList)
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
{
RegisteredBgWorker *rw;
rw = slist_container(RegisteredBgWorker, rw_lnode, iter.cur);
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
/* ignore if already running */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
if (rw->rw_pid != 0)
continue;
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
/* if marked for death, clean up and remove from list */
Provide a reliable mechanism for terminating a background worker. Although previously-introduced APIs allow the process that registers a background worker to obtain the worker's PID, there's no way to prevent a worker that is not currently running from being restarted. This patch introduces a new API TerminateBackgroundWorker() that prevents the background worker from being restarted, terminates it if it is currently running, and causes it to be unregistered if or when it is not running. Patch by me. Review by Michael Paquier and KaiGai Kohei.
2013-10-18 16:21:25 +02:00
if (rw->rw_terminate)
{
ForgetBackgroundWorker(&iter);
continue;
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
* If this worker has crashed previously, maybe it needs to be
* restarted (unless on registration it specified it doesn't want to
* be restarted at all). Check how long ago did a crash last happen.
* If the last crash is too recent, don't start it right away; let it
* be restarted once enough time has passed.
*/
if (rw->rw_crashed_at != 0)
{
if (rw->rw_worker.bgw_restart_time == BGW_NEVER_RESTART)
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
{
Report failure to start a background worker. When a worker is flagged as BGW_NEVER_RESTART and we fail to start it, or if it is not marked BGW_NEVER_RESTART but is terminated before startup succeeds, what BgwHandleStatus should be reported? The previous code really hadn't considered this possibility (as indicated by the comments which ignore it completely) and would typically return BGWH_NOT_YET_STARTED, but that's not a good answer, because then there's no way for code using GetBackgroundWorkerPid() to tell the difference between a worker that has not started but will start later and a worker that has not started and will never be started. So, when this case happens, return BGWH_STOPPED instead. Update the comments to reflect this. The preceding fix by itself is insufficient to fix the problem, because the old code also didn't send a notification to the process identified in bgw_notify_pid when startup failed. That might've been technically correct under the theory that the status of the worker was BGWH_NOT_YET_STARTED, because the status would indeed not change when the worker failed to start, but now that we're more usefully reporting BGWH_STOPPED, a notification is needed. Without these fixes, code which starts background workers and then uses the recommended APIs to wait for those background workers to start would hang indefinitely if the postmaster failed to fork a worker. Amit Kapila and Robert Haas Discussion: http://postgr.es/m/CAA4eK1KDfKkvrjxsKJi3WPyceVi3dH1VCkbTJji2fuwKuB=3uw@mail.gmail.com
2017-12-06 14:49:30 +01:00
int notify_pid;
notify_pid = rw->rw_worker.bgw_notify_pid;
Improve ilist.h's support for deletion of slist elements during iteration. Previously one had to use slist_delete(), implying an additional scan of the list, making this infrastructure considerably less efficient than traditional Lists when deletion of element(s) in a long list is needed. Modify the slist_foreach_modify() macro to support deleting the current element in O(1) time, by keeping a "prev" pointer in addition to "cur" and "next". Although this makes iteration with this macro a bit slower, no real harm is done, since in any scenario where you're not going to delete the current list element you might as well just use slist_foreach instead. Improve the comments about when to use each macro. Back-patch to 9.3 so that we'll have consistent semantics in all branches that provide ilist.h. Note this is an ABI break for callers of slist_foreach_modify(). Andres Freund and Tom Lane
2013-07-24 23:41:55 +02:00
ForgetBackgroundWorker(&iter);
Report failure to start a background worker. When a worker is flagged as BGW_NEVER_RESTART and we fail to start it, or if it is not marked BGW_NEVER_RESTART but is terminated before startup succeeds, what BgwHandleStatus should be reported? The previous code really hadn't considered this possibility (as indicated by the comments which ignore it completely) and would typically return BGWH_NOT_YET_STARTED, but that's not a good answer, because then there's no way for code using GetBackgroundWorkerPid() to tell the difference between a worker that has not started but will start later and a worker that has not started and will never be started. So, when this case happens, return BGWH_STOPPED instead. Update the comments to reflect this. The preceding fix by itself is insufficient to fix the problem, because the old code also didn't send a notification to the process identified in bgw_notify_pid when startup failed. That might've been technically correct under the theory that the status of the worker was BGWH_NOT_YET_STARTED, because the status would indeed not change when the worker failed to start, but now that we're more usefully reporting BGWH_STOPPED, a notification is needed. Without these fixes, code which starts background workers and then uses the recommended APIs to wait for those background workers to start would hang indefinitely if the postmaster failed to fork a worker. Amit Kapila and Robert Haas Discussion: http://postgr.es/m/CAA4eK1KDfKkvrjxsKJi3WPyceVi3dH1VCkbTJji2fuwKuB=3uw@mail.gmail.com
2017-12-06 14:49:30 +01:00
/* Report worker is gone now. */
if (notify_pid != 0)
kill(notify_pid, SIGUSR1);
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
continue;
Allow background workers to be started dynamically. There is a new API, RegisterDynamicBackgroundWorker, which allows an ordinary user backend to register a new background writer during normal running. This means that it's no longer necessary for all background workers to be registered during processing of shared_preload_libraries, although the option of registering workers at that time remains available. When a background worker exits and will not be restarted, the slot previously used by that background worker is automatically released and becomes available for reuse. Slots used by background workers that are configured for automatic restart can't (yet) be released without shutting down the system. This commit adds a new source file, bgworker.c, and moves some of the existing control logic for background workers there. Previously, there was little enough logic that it made sense to keep everything in postmaster.c, but not any more. This commit also makes the worker_spi contrib module into an extension and adds a new function, worker_spi_launch, which can be used to demonstrate the new facility.
2013-07-16 19:02:15 +02:00
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
/* read system time only when needed */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
if (now == 0)
now = GetCurrentTimestamp();
if (!TimestampDifferenceExceeds(rw->rw_crashed_at, now,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
rw->rw_worker.bgw_restart_time * 1000))
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
{
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
/* Set flag to remember that we have workers to start later */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
HaveCrashedWorker = true;
continue;
}
}
if (bgworker_should_start_now(rw->rw_worker.bgw_start_time))
{
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
/* reset crash time before trying to start worker */
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
rw->rw_crashed_at = 0;
/*
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
* Try to start the worker.
*
* On failure, give up processing workers for now, but set
* StartWorkerNeeded so we'll come back here on the next iteration
* of ServerLoop to try again. (We don't want to wait, because
* there might be additional ready-to-run workers.) We could set
* HaveCrashedWorker as well, since this worker is now marked
* crashed, but there's no need because the next run of this
* function will do that.
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
*/
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
if (!do_start_bgworker(rw))
{
StartWorkerNeeded = true;
Allow notifications to bgworkers without database connections. Previously, if one background worker registered another background worker and set bgw_notify_pid while for the second background worker, it would not receive notifications from the postmaster unless, at the time the "parent" was registered, BGWORKER_BACKEND_DATABASE_CONNECTION was set. To fix, instead instead of including only those background workers that requested database connections in the postmater's BackendList, include them all. There doesn't seem to be any reason not do this, and indeed it removes a significant amount of duplicated code. The other option is to make PostmasterMarkPIDForWorkerNotify look at BackgroundWorkerList in addition to BackendList, but that adds more code duplication instead of getting rid of it. Patch by me. Review and testing by Ashutosh Bapat.
2015-09-01 21:30:19 +02:00
return;
Fix postmaster's handling of fork failure for a bgworker process. This corner case didn't behave nicely at all: the postmaster would (partially) update its state as though the process had started successfully, and be quite confused thereafter. Fix it to act like the worker had crashed, instead. In passing, refactor so that do_start_bgworker contains all the state-change logic for bgworker launch, rather than just some of it. Back-patch as far as 9.4. 9.3 contains similar logic, but it's just enough different that I don't feel comfortable applying the patch without more study; and the use of bgworkers in 9.3 was so small that it doesn't seem worth the extra work. transam/parallel.c is still entirely unprepared for the possibility of bgworker startup failure, but that seems like material for a separate patch. Discussion: https://postgr.es/m/4905.1492813727@sss.pgh.pa.us
2017-04-24 18:16:58 +02:00
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
/*
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
* If we've launched as many workers as allowed, quit, but have
* ServerLoop call us again to look for additional ready-to-run
* workers. There might not be any, but we'll find out the next
* time we run.
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
*/
Allow multiple bgworkers to be launched per postmaster iteration. Previously, maybe_start_bgworker() would launch at most one bgworker process per call, on the grounds that the postmaster might otherwise neglect its other duties for too long. However, that seems overly conservative, especially since bad effects only become obvious when many hundreds of bgworkers need to be launched at once. On the other side of the coin is that the existing logic could result in substantial delay of bgworker launches, because ServerLoop isn't guaranteed to iterate immediately after a signal arrives. (My attempt to fix that by using pselect(2) encountered too many portability question marks, and in any case could not help on platforms without pselect().) One could also question the wisdom of using an O(N^2) processing method if the system is intended to support so many bgworkers. As a compromise, allow that function to launch up to 100 bgworkers per call (and in consequence, rename it to maybe_start_bgworkers). This will allow any normal parallel-query request for workers to be satisfied immediately during sigusr1_handler, avoiding the question of whether ServerLoop will be able to launch more promptly. There is talk of rewriting the postmaster to use a WaitEventSet to avoid the signal-response-delay problem, but I'd argue that this change should be kept even after that happens (if it ever does). Backpatch to 9.6 where parallel query was added. The issue exists before that, but previous uses of bgworkers typically aren't as sensitive to how quickly they get launched. Discussion: https://postgr.es/m/4707.1493221358@sss.pgh.pa.us
2017-04-26 22:17:29 +02:00
if (++num_launched >= MAX_BGWORKERS_TO_LAUNCH)
{
StartWorkerNeeded = true;
return;
}
Background worker processes Background workers are postmaster subprocesses that run arbitrary user-specified code. They can request shared memory access as well as backend database connections; or they can just use plain libpq frontend database connections. Modules listed in shared_preload_libraries can register background workers in their _PG_init() function; this is early enough that it's not necessary to provide an extra GUC option, because the necessary extra resources can be allocated early on. Modules can install more than one bgworker, if necessary. Care is taken that these extra processes do not interfere with other postmaster tasks: only one such process is started on each ServerLoop iteration. This means a large number of them could be waiting to be started up and postmaster is still able to quickly service external connection requests. Also, shutdown sequence should not be impacted by a worker process that's reasonably well behaved (i.e. promptly responds to termination signals.) The current implementation lets worker processes specify their start time, i.e. at what point in the server startup process they are to be started: right after postmaster start (in which case they mustn't ask for shared memory access), when consistent state has been reached (useful during recovery in a HOT standby server), or when recovery has terminated (i.e. when normal backends are allowed). In case of a bgworker crash, actions to take depend on registration data: if shared memory was requested, then all other connections are taken down (as well as other bgworkers), just like it were a regular backend crashing. The bgworker itself is restarted, too, within a configurable timeframe (which can be configured to be never). More features to add to this framework can be imagined without much effort, and have been discussed, but this seems good enough as a useful unit already. An elementary sample module is supplied. Author: Álvaro Herrera This patch is loosely based on prior patches submitted by KaiGai Kohei, and unsubmitted code by Simon Riggs. Reviewed by: KaiGai Kohei, Markus Wanner, Andres Freund, Heikki Linnakangas, Simon Riggs, Amit Kapila
2012-12-06 18:57:52 +01:00
}
}
}
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
Allow discovery of whether a dynamic background worker is running. Using the infrastructure provided by this patch, it's possible either to wait for the startup of a dynamically-registered background worker, or to poll the status of such a worker without waiting. In either case, the current PID of the worker process can also be obtained. As usual, worker_spi is updated to demonstrate the new functionality. Patch by me. Review by Andres Freund.
2013-08-28 20:08:13 +02:00
/*
* When a backend asks to be notified about worker state changes, we
* set a flag in its backend entry. The background worker machinery needs
* to know when such backends exit.
*/
bool
PostmasterMarkPIDForWorkerNotify(int pid)
{
dlist_iter iter;
Backend *bp;
dlist_foreach(iter, &BackendList)
{
bp = dlist_container(Backend, elem, iter.cur);
if (bp->pid == pid)
{
bp->bgworker_notify = true;
return true;
}
}
return false;
}
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
#ifdef EXEC_BACKEND
/*
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
* The following need to be available to the save/restore_backend_variables
Fix syslogger so that log_truncate_on_rotation works in the first rotation. In the original coding of the log rotation stuff, we did not bother to make the truncation logic work for the very first rotation after postmaster start (or after a syslogger crash and restart). It just always appended in that case. It did not seem terribly important at the time, but we've recently had two separate complaints from people who expected it to work unsurprisingly. (Both users tend to restart the postmaster about as often as a log rotation is configured to happen, which is maybe not typical use, but still...) Since the initial log file is opened in the postmaster, fixing this requires passing down some more state to the syslogger child process. It's always been like this, so back-patch to all supported branches.
2012-07-31 20:36:54 +02:00
* functions. They are marked NON_EXEC_STATIC in their home modules.
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
*/
extern slock_t *ShmemLock;
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
extern slock_t *ProcStructLock;
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
extern PGPROC *AuxiliaryProcs;
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
extern PMSignalData *PMSignalState;
Create typedef pgsocket for storing socket descriptors. This silences some warnings on Win64. Not using the proper SOCKET datatype was actually wrong on Win32 as well, but didn't cause any warnings there. Also create define PGINVALID_SOCKET to indicate an invalid/non-existing socket, instead of using a hardcoded -1 value.
2010-01-10 15:16:08 +01:00
extern pgsocket pgStatSock;
Fix syslogger so that log_truncate_on_rotation works in the first rotation. In the original coding of the log rotation stuff, we did not bother to make the truncation logic work for the very first rotation after postmaster start (or after a syslogger crash and restart). It just always appended in that case. It did not seem terribly important at the time, but we've recently had two separate complaints from people who expected it to work unsurprisingly. (Both users tend to restart the postmaster about as often as a log rotation is configured to happen, which is maybe not typical use, but still...) Since the initial log file is opened in the postmaster, fixing this requires passing down some more state to the syslogger child process. It's always been like this, so back-patch to all supported branches.
2012-07-31 20:36:54 +02:00
extern pg_time_t first_syslogger_file_time;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#ifndef WIN32
Non-Windows EXEC_BACKEND path was broken by recent write_inheritable_socket change ... it's got to return true.
2009-08-28 19:42:54 +02:00
#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#define read_inheritable_socket(dest, src) (*(dest) = *(src))
#else
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
pid_t childPid);
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#endif
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/* Save critical backend variables into the BackendParameters struct */
#ifndef WIN32
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
static bool
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
save_backend_variables(BackendParameters *param, Port *port)
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#else
static bool
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
save_backend_variables(BackendParameters *param, Port *port,
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
HANDLE childProcess, pid_t childPid)
#endif
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
{
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
memcpy(&param->port, port, sizeof(Port));
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
if (!write_inheritable_socket(&param->portsocket, port->sock, childPid))
return false;
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
StrNCpy -> strlcpy (not complete)
2007-02-10 15:58:55 +01:00
strlcpy(param->DataDir, DataDir, MAXPGPATH);
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
memcpy(&param->ListenSocket, &ListenSocket, sizeof(ListenSocket));
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
param->MyCancelKey = MyCancelKey;
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
param->MyPMChildSlot = MyPMChildSlot;
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Avoid "could not reattach" by providing space for concurrent allocation. We've long had reports of intermittent "could not reattach to shared memory" errors on Windows. Buildfarm member dory fails that way when PGSharedMemoryReAttach() execution overlaps with creation of a thread for the process's "default thread pool". Fix that by providing a second region to receive asynchronous allocations that would otherwise intrude into UsedShmemSegAddr. In pgwin32_ReserveSharedMemoryRegion(), stop trying to free reservations landing at incorrect addresses; the caller's next step has been to terminate the affected process. Back-patch to 9.4 (all supported versions). Reviewed by Tom Lane. He also did much of the prerequisite research; see commit bcbf2346d69f6006f126044864dd9383d50d87b4. Discussion: https://postgr.es/m/20190402135442.GA1173872@rfd.leadboat.com
2019-04-09 06:39:00 +02:00
#ifdef WIN32
param->ShmemProtectiveRegion = ShmemProtectiveRegion;
#endif
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
param->UsedShmemSegID = UsedShmemSegID;
param->UsedShmemSegAddr = UsedShmemSegAddr;
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
param->ShmemLock = ShmemLock;
param->ShmemVariableCache = ShmemVariableCache;
param->ShmemBackendArray = ShmemBackendArray;
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Reduce the number of semaphores used under --disable-spinlocks. Instead of allocating a semaphore from the operating system for every spinlock, allocate a fixed number of semaphores (by default, 1024) from the operating system and multiplex all the spinlocks that get created onto them. This could self-deadlock if a process attempted to acquire more than one spinlock at a time, but since processes aren't supposed to execute anything other than short stretches of straight-line code while holding a spinlock, that shouldn't happen. One motivation for this change is that, with the introduction of dynamic shared memory, it may be desirable to create spinlocks that last for less than the lifetime of the server. Without this change, attempting to use such facilities under --disable-spinlocks would quickly exhaust any supply of available semaphores. Quite apart from that, it's desirable to contain the quantity of semaphores needed to run the server simply on convenience grounds, since using too many may make it harder to get PostgreSQL running on a new platform, which is mostly the point of --disable-spinlocks in the first place. Patch by me; review by Tom Lane.
2014-01-09 00:49:14 +01:00
#ifndef HAVE_SPINLOCKS
param->SpinlockSemaArray = SpinlockSemaArray;
#endif
Change the way that LWLocks for extensions are allocated. The previous RequestAddinLWLocks() method had several disadvantages. First, the locks would be in the main tranche; we've recently decided that it's useful for LWLocks used for separate purposes to have separate tranche IDs. Second, there wasn't any correlation between what code called RequestAddinLWLocks() and what code called LWLockAssign(); when multiple modules are in use, it could become quite difficult to troubleshoot problems where LWLockAssign() ran out of locks. To fix, create a concept of named LWLock tranches which can be used either by extension or by core code. Amit Kapila and Robert Haas
2016-02-04 22:43:04 +01:00
param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
Relax the requirement that all lwlocks be stored in a single array. This makes it possible to store lwlocks as part of some other data structure in the main shared memory segment, or in a dynamic shared memory segment. There is still a main LWLock array and this patch does not move anything out of it, but it provides necessary infrastructure for doing that in the future. This change is likely to increase the size of LWLockPadded on some platforms, especially 32-bit platforms where it was previously only 16 bytes. Patch by me. Review by Andres Freund and KaiGai Kohei.
2014-01-27 17:07:44 +01:00
param->MainLWLockArray = MainLWLockArray;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
param->ProcStructLock = ProcStructLock;
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
param->ProcGlobal = ProcGlobal;
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
param->AuxiliaryProcs = AuxiliaryProcs;
Move "hot" members of PGPROC into a separate PGXACT array. This speeds up snapshot-taking and reduces ProcArrayLock contention. Also, the PGPROC (and PGXACT) structures used by two-phase commit are now allocated as part of the main array, rather than in a separate array, and we keep ProcArray sorted in pointer order. These changes are intended to minimize the number of cache lines that must be pulled in to take a snapshot, and testing shows a substantial increase in performance on both read and write workloads at high concurrencies. Pavan Deolasee, Heikki Linnakangas, Robert Haas
2011-11-25 14:02:10 +01:00
param->PreparedXactProcs = PreparedXactProcs;
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
param->PMSignalState = PMSignalState;
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
if (!write_inheritable_socket(&param->pgStatSock, pgStatSock, childPid))
return false;
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
param->PostmasterPid = PostmasterPid;
Clean up the rather historically encumbered interface to now() and current time: provide a GetCurrentTimestamp() function that returns current time in the form of a TimestampTz, instead of separate time_t and microseconds fields. This is what all the callers really want anyway, and it eliminates low-level dependencies on AbsoluteTime, which is a deprecated datatype that will have to disappear eventually.
2005-06-30 00:51:57 +02:00
param->PgStartTime = PgStartTime;
Add pg_conf_load_time() function to report when the Postgres configuration files were last loaded. George Gensure
2008-05-04 23:13:36 +02:00
param->PgReloadTime = PgReloadTime;
Fix syslogger so that log_truncate_on_rotation works in the first rotation. In the original coding of the log rotation stuff, we did not bother to make the truncation logic work for the very first rotation after postmaster start (or after a syslogger crash and restart). It just always appended in that case. It did not seem terribly important at the time, but we've recently had two separate complaints from people who expected it to work unsurprisingly. (Both users tend to restart the postmaster about as often as a log rotation is configured to happen, which is maybe not typical use, but still...) Since the initial log file is opened in the postmaster, fixing this requires passing down some more state to the syslogger child process. It's always been like this, so back-patch to all supported branches.
2012-07-31 20:36:54 +02:00
param->first_syslogger_file_time = first_syslogger_file_time;
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
Only use the pipe chunking protocol if we know the syslogger should be catching stderr output, and we are not ourselves the syslogger. Otherwise, go directly to stderr. Bug noticed by Tom Lane. Backpatch as far as 8.0.
2007-07-19 21:13:43 +02:00
param->redirection_done = redirection_done;
Fix pg_upgrade for EXEC_BACKEND builds (e.g. Windows) by properly passing the -b/binary-upgrade flag. Backpatch to 9.1.X.
2011-09-29 23:20:53 +02:00
param->IsBinaryUpgrade = IsBinaryUpgrade;
Inherit max_safe_fds to child processes in EXEC_BACKEND mode. Postmaster sets max_safe_fds by testing how many open file descriptors it can open, and that is normally inherited by all child processes at fork(). Not so on EXEC_BACKEND, ie. Windows, however. Because of that, we effectively ignored max_files_per_process on Windows, and always assumed a conservative default of 32 simultaneous open files. That could have an impact on performance, if you need to access a lot of different files in a query. After this patch, the value is passed to child processes by save/restore_backend_variables() among many other global variables. It has been like this forever, but given the lack of complaints about it, I'm not backpatching this.
2012-03-29 07:19:11 +02:00
param->max_safe_fds = max_safe_fds;
Only use the pipe chunking protocol if we know the syslogger should be catching stderr output, and we are not ourselves the syslogger. Otherwise, go directly to stderr. Bug noticed by Tom Lane. Backpatch as far as 8.0.
2007-07-19 21:13:43 +02:00
Fix background workers for EXEC_BACKEND Commit da07a1e8 was broken for EXEC_BACKEND because I failed to realize that the MaxBackends recomputation needed to be duplicated by subprocesses in SubPostmasterMain. However, instead of having the value be recomputed at all, it's better to assign the correct value at postmaster initialization time, and have it be propagated to exec'ed backends via BackendParameters. MaxBackends stays as zero until after modules in shared_preload_libraries have had a chance to register bgworkers, since the value is going to be untrustworthy till that's finished. Heikki Linnakangas and Álvaro Herrera
2013-01-02 16:01:14 +01:00
param->MaxBackends = MaxBackends;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
if (!write_duplicated_handle(&param->initial_signal_pipe,
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
pgwin32_create_signal_listener(childPid),
childProcess))
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
return false;
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
#else
memcpy(&param->postmaster_alive_fds, &postmaster_alive_fds,
sizeof(postmaster_alive_fds));
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#endif
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
memcpy(&param->syslogPipe, &syslogPipe, sizeof(syslogPipe));
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
StrNCpy -> strlcpy (not complete)
2007-02-10 15:58:55 +01:00
strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
StrNCpy -> strlcpy (not complete)
2007-02-10 15:58:55 +01:00
strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
Autovacuum loose end mop-up. Provide autovacuum-specific vacuum cost delay and limit, both as global GUCs and as table-specific entries in pg_autovacuum. stats_reset_on_server_start is now OFF by default, but a reset is forced if we did WAL replay. XID-wrap vacuums do not ANALYZE, but do FREEZE if it's a template database. Alvaro Herrera
2005-08-11 23:11:50 +02:00
StrNCpy -> strlcpy (not complete)
2007-02-10 15:58:55 +01:00
strlcpy(param->ExtraOptions, ExtraOptions, MAXPGPATH);
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
return true;
}
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Create a built-in log rotation program, so that we no longer have to recommend that people go get Apache's rotatelogs program. Additional benefits are that configuration is done through GUC, rather than externally, and that the postmaster can monitor the log rotator and restart it after failure (though we certainly hope that won't happen often). Andreas Pflug, some rework by Tom Lane.
2004-08-06 01:32:13 +02:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#ifdef WIN32
/*
* Duplicate a handle for usage in a child process, and write the child
* process instance of the handle to the parameter file.
*/
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
static bool
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
HANDLE hChild = INVALID_HANDLE_VALUE;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
if (!DuplicateHandle(GetCurrentProcess(),
src,
childProcess,
&hChild,
0,
TRUE,
DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
{
ereport(LOG,
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
(errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
GetLastError())));
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
return false;
}
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
*dest = hChild;
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
return true;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
}
Integrate src/timezone library for all platforms. There is more we can and should do now that we control our own destiny for timezone handling, but this commit gets the bulk of the picayune diffs in place. Magnus Hagander and Tom Lane.
2004-05-21 07:08:06 +02:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/*
* Duplicate a socket for usage in a child process, and write the resulting
* structure to the parameter file.
* This is required because a number of LSPs (Layered Service Providers) very
* common on Windows (antivirus, firewalls, download managers etc) break
* straight socket inheritance.
*/
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
static bool
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
{
dest->origsocket = src;
Create typedef pgsocket for storing socket descriptors. This silences some warnings on Win64. Not using the proper SOCKET datatype was actually wrong on Win32 as well, but didn't cause any warnings there. Also create define PGINVALID_SOCKET to indicate an invalid/non-existing socket, instead of using a hardcoded -1 value.
2010-01-10 15:16:08 +01:00
if (src != 0 && src != PGINVALID_SOCKET)
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
{
/* Actual socket */
if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
{
ereport(LOG,
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
(errmsg("could not duplicate socket %d for use in backend: error code %d",
Add some casts to try to silence most of the remaining format warnings on MinGW-W64.
2011-04-28 21:05:58 +02:00
(int) src, WSAGetLastError())));
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
return false;
}
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
}
Avoid terminating the postmaster on a number of "can't happen" cases during backend startup on Win32. Instead, log the error and just forget about the potentially dangling process, since we can't do anything about it anyway.
2009-08-06 11:50:22 +02:00
return true;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
}
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/*
* Read a duplicate socket structure back, and get the socket descriptor.
*/
static void
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
SOCKET s;
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Create typedef pgsocket for storing socket descriptors. This silences some warnings on Win64. Not using the proper SOCKET datatype was actually wrong on Win32 as well, but didn't cause any warnings there. Also create define PGINVALID_SOCKET to indicate an invalid/non-existing socket, instead of using a hardcoded -1 value.
2010-01-10 15:16:08 +01:00
if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
Ensure that close() and fclose() are checked for errors, at least in cases involving writes. Per recent discussion about the possibility of close-time failures on some filesystems. There is a TODO item for this, too.
2004-01-26 23:35:32 +01:00
{
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/* Not a real socket! */
*dest = src->origsocket;
Ensure that close() and fclose() are checked for errors, at least in cases involving writes. Per recent discussion about the possibility of close-time failures on some filesystems. There is a TODO item for this, too.
2004-01-26 23:35:32 +01:00
}
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
else
{
/* Actual socket, so create from structure */
s = WSASocket(FROM_PROTOCOL_INFO,
FROM_PROTOCOL_INFO,
FROM_PROTOCOL_INFO,
&src->wsainfo,
0,
0);
if (s == INVALID_SOCKET)
{
write_stderr("could not create inherited socket: error code %d\n",
WSAGetLastError());
exit(1);
}
*dest = s;
Ensure that close() and fclose() are checked for errors, at least in cases involving writes. Per recent discussion about the possibility of close-time failures on some filesystems. There is a TODO item for this, too.
2004-01-26 23:35:32 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* To make sure we don't get two references to the same socket, close
* the original one. (This would happen when inheritance actually
* works..
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
*/
closesocket(src->origsocket);
}
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
}
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#endif
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
static void
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
read_backend_variables(char *id, Port *port)
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
{
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
BackendParameters param;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
#ifndef WIN32
/* Non-win32 implementation reads from file */
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
FILE *fp;
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
/* Open file */
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
fp = AllocateFile(id, PG_BINARY_R);
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
if (!fp)
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
{
Make logging_collector=on work with non-windows EXEC_BACKEND again. Commit b94ce6e80 reordered postmaster's startup sequence so that the tempfile directory is only cleaned up after all the necessary state for pg_ctl is collected. Unfortunately the chosen location is after the syslogger has been started; which normally is fine, except for !WIN32 EXEC_BACKEND builds, which pass information to children via files in the temp directory. Move the call to RemovePgTempFiles() to just before the syslogger has started. That's the first child we fork. Luckily EXEC_BACKEND is pretty much only used by endusers on windows, which has a separate method to pass information to children. That means the real world impact of this bug is very small. Discussion: 20150113182344.GF12272@alap3.anarazel.de Backpatch to 9.1, just as the previous commit was.
2015-01-13 21:02:47 +01:00
write_stderr("could not open backend variables file \"%s\": %s\n",
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
id, strerror(errno));
exit(1);
}
if (fread(&param, sizeof(param), 1, fp) != 1)
{
write_stderr("could not read from backend variables file \"%s\": %s\n",
id, strerror(errno));
exit(1);
}
/* Release file */
FreeFile(fp);
if (unlink(id) != 0)
{
write_stderr("could not remove file \"%s\": %s\n",
id, strerror(errno));
exit(1);
}
#else
/* Win32 version uses mapped file */
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
HANDLE paramHandle;
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
BackendParameters *paramp;
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Send paramHandle to subprocesses as 64-bit on Win64 The handle to the shared memory segment containing startup parameters was sent as 32-bit even on 64-bit systems. Since HANDLEs appear to be allocated sequentially this shouldn't be a problem until we reach 2^32 open handles in the postmaster, but a 64-bit value should be sent across as 64-bit, and not zero out the top 32 bits. Noted by Tom Lane.
2010-11-16 12:40:56 +01:00
#ifdef _WIN64
paramHandle = (HANDLE) _atoi64(id);
#else
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
paramHandle = (HANDLE) atol(id);
Send paramHandle to subprocesses as 64-bit on Win64 The handle to the shared memory segment containing startup parameters was sent as 32-bit even on 64-bit systems. Since HANDLEs appear to be allocated sequentially this shouldn't be a problem until we reach 2^32 open handles in the postmaster, but a 64-bit value should be sent across as 64-bit, and not zero out the top 32 bits. Noted by Tom Lane.
2010-11-16 12:40:56 +01:00
#endif
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
paramp = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
if (!paramp)
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
{
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
write_stderr("could not map view of backend variables: error code %lu\n",
GetLastError());
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
exit(1);
}
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
memcpy(&param, paramp, sizeof(BackendParameters));
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
if (!UnmapViewOfFile(paramp))
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
{
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
write_stderr("could not unmap view of backend variables: error code %lu\n",
GetLastError());
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
exit(1);
}
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
if (!CloseHandle(paramHandle))
{
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
write_stderr("could not close handle to backend parameter variables: error code %lu\n",
GetLastError());
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
exit(1);
}
#endif
Refactor EXEC_BACKEND code so that postmaster child processes reattach to shared memory as soon as possible, ie, right after read_backend_variables. The effective difference from the original code is that this happens before instead of after read_nondefault_variables(), which loads GUC information and is apparently capable of expanding the backend's memory allocation more than you'd think it should. This should fix the failure-to-attach-to-shared-memory reports we've been seeing on Windows. Also clean up a few bits of unnecessarily grotty EXEC_BACKEND code.
2004-12-29 22:36:09 +01:00
restore_backend_variables(&param, port);
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
}
/* Restore critical backend variables from the BackendParameters struct */
static void
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
restore_backend_variables(BackendParameters *param, Port *port)
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
{
memcpy(port, &param->port, sizeof(Port));
read_inheritable_socket(&port->sock, &param->portsocket);
SetDataDir(param->DataDir);
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
memcpy(&ListenSocket, &param->ListenSocket, sizeof(ListenSocket));
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
MyCancelKey = param->MyCancelKey;
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
MyPMChildSlot = param->MyPMChildSlot;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
Avoid "could not reattach" by providing space for concurrent allocation. We've long had reports of intermittent "could not reattach to shared memory" errors on Windows. Buildfarm member dory fails that way when PGSharedMemoryReAttach() execution overlaps with creation of a thread for the process's "default thread pool". Fix that by providing a second region to receive asynchronous allocations that would otherwise intrude into UsedShmemSegAddr. In pgwin32_ReserveSharedMemoryRegion(), stop trying to free reservations landing at incorrect addresses; the caller's next step has been to terminate the affected process. Back-patch to 9.4 (all supported versions). Reviewed by Tom Lane. He also did much of the prerequisite research; see commit bcbf2346d69f6006f126044864dd9383d50d87b4. Discussion: https://postgr.es/m/20190402135442.GA1173872@rfd.leadboat.com
2019-04-09 06:39:00 +02:00
#ifdef WIN32
ShmemProtectiveRegion = param->ShmemProtectiveRegion;
#endif
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
UsedShmemSegID = param->UsedShmemSegID;
UsedShmemSegAddr = param->UsedShmemSegAddr;
ShmemLock = param->ShmemLock;
ShmemVariableCache = param->ShmemVariableCache;
ShmemBackendArray = param->ShmemBackendArray;
Reduce the number of semaphores used under --disable-spinlocks. Instead of allocating a semaphore from the operating system for every spinlock, allocate a fixed number of semaphores (by default, 1024) from the operating system and multiplex all the spinlocks that get created onto them. This could self-deadlock if a process attempted to acquire more than one spinlock at a time, but since processes aren't supposed to execute anything other than short stretches of straight-line code while holding a spinlock, that shouldn't happen. One motivation for this change is that, with the introduction of dynamic shared memory, it may be desirable to create spinlocks that last for less than the lifetime of the server. Without this change, attempting to use such facilities under --disable-spinlocks would quickly exhaust any supply of available semaphores. Quite apart from that, it's desirable to contain the quantity of semaphores needed to run the server simply on convenience grounds, since using too many may make it harder to get PostgreSQL running on a new platform, which is mostly the point of --disable-spinlocks in the first place. Patch by me; review by Tom Lane.
2014-01-09 00:49:14 +01:00
#ifndef HAVE_SPINLOCKS
SpinlockSemaArray = param->SpinlockSemaArray;
#endif
Change the way that LWLocks for extensions are allocated. The previous RequestAddinLWLocks() method had several disadvantages. First, the locks would be in the main tranche; we've recently decided that it's useful for LWLocks used for separate purposes to have separate tranche IDs. Second, there wasn't any correlation between what code called RequestAddinLWLocks() and what code called LWLockAssign(); when multiple modules are in use, it could become quite difficult to troubleshoot problems where LWLockAssign() ran out of locks. To fix, create a concept of named LWLock tranches which can be used either by extension or by core code. Amit Kapila and Robert Haas
2016-02-04 22:43:04 +01:00
NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
Relax the requirement that all lwlocks be stored in a single array. This makes it possible to store lwlocks as part of some other data structure in the main shared memory segment, or in a dynamic shared memory segment. There is still a main LWLock array and this patch does not move anything out of it, but it provides necessary infrastructure for doing that in the future. This change is likely to increase the size of LWLockPadded on some platforms, especially 32-bit platforms where it was previously only 16 bytes. Patch by me. Review by Andres Freund and KaiGai Kohei.
2014-01-27 17:07:44 +01:00
MainLWLockArray = param->MainLWLockArray;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
ProcStructLock = param->ProcStructLock;
Rearrange backend startup sequence so that ShmemIndexLock can become an LWLock instead of a spinlock. This hardly matters on Unix machines but should improve startup performance on Windows (or any port using EXEC_BACKEND). Per previous discussion.
2006-01-04 22:06:32 +01:00
ProcGlobal = param->ProcGlobal;
Cleanup the bootstrap code a little, and rename "dummy procs" in the code comments and variables to "auxiliary proc", per Heikki's request.
2007-03-07 14:35:03 +01:00
AuxiliaryProcs = param->AuxiliaryProcs;
Move "hot" members of PGPROC into a separate PGXACT array. This speeds up snapshot-taking and reduces ProcArrayLock contention. Also, the PGPROC (and PGXACT) structures used by two-phase commit are now allocated as part of the main array, rather than in a separate array, and we keep ProcArray sorted in pointer order. These changes are intended to minimize the number of cache lines that must be pulled in to take a snapshot, and testing shows a substantial increase in performance on both read and write workloads at high concurrencies. Pavan Deolasee, Heikki Linnakangas, Robert Haas
2011-11-25 14:02:10 +01:00
PreparedXactProcs = param->PreparedXactProcs;
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
PMSignalState = param->PMSignalState;
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
read_inheritable_socket(&pgStatSock, &param->pgStatSock);
PostmasterPid = param->PostmasterPid;
Clean up the rather historically encumbered interface to now() and current time: provide a GetCurrentTimestamp() function that returns current time in the form of a TimestampTz, instead of separate time_t and microseconds fields. This is what all the callers really want anyway, and it eliminates low-level dependencies on AbsoluteTime, which is a deprecated datatype that will have to disappear eventually.
2005-06-30 00:51:57 +02:00
PgStartTime = param->PgStartTime;
Add pg_conf_load_time() function to report when the Postgres configuration files were last loaded. George Gensure
2008-05-04 23:13:36 +02:00
PgReloadTime = param->PgReloadTime;
Fix syslogger so that log_truncate_on_rotation works in the first rotation. In the original coding of the log rotation stuff, we did not bother to make the truncation logic work for the very first rotation after postmaster start (or after a syslogger crash and restart). It just always appended in that case. It did not seem terribly important at the time, but we've recently had two separate complaints from people who expected it to work unsurprisingly. (Both users tend to restart the postmaster about as often as a log rotation is configured to happen, which is maybe not typical use, but still...) Since the initial log file is opened in the postmaster, fixing this requires passing down some more state to the syslogger child process. It's always been like this, so back-patch to all supported branches.
2012-07-31 20:36:54 +02:00
first_syslogger_file_time = param->first_syslogger_file_time;
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
Only use the pipe chunking protocol if we know the syslogger should be catching stderr output, and we are not ourselves the syslogger. Otherwise, go directly to stderr. Bug noticed by Tom Lane. Backpatch as far as 8.0.
2007-07-19 21:13:43 +02:00
redirection_done = param->redirection_done;
Fix pg_upgrade for EXEC_BACKEND builds (e.g. Windows) by properly passing the -b/binary-upgrade flag. Backpatch to 9.1.X.
2011-09-29 23:20:53 +02:00
IsBinaryUpgrade = param->IsBinaryUpgrade;
Inherit max_safe_fds to child processes in EXEC_BACKEND mode. Postmaster sets max_safe_fds by testing how many open file descriptors it can open, and that is normally inherited by all child processes at fork(). Not so on EXEC_BACKEND, ie. Windows, however. Because of that, we effectively ignored max_files_per_process on Windows, and always assumed a conservative default of 32 simultaneous open files. That could have an impact on performance, if you need to access a lot of different files in a query. After this patch, the value is passed to child processes by save/restore_backend_variables() among many other global variables. It has been like this forever, but given the lack of complaints about it, I'm not backpatching this.
2012-03-29 07:19:11 +02:00
max_safe_fds = param->max_safe_fds;
Only use the pipe chunking protocol if we know the syslogger should be catching stderr output, and we are not ourselves the syslogger. Otherwise, go directly to stderr. Bug noticed by Tom Lane. Backpatch as far as 8.0.
2007-07-19 21:13:43 +02:00
Fix background workers for EXEC_BACKEND Commit da07a1e8 was broken for EXEC_BACKEND because I failed to realize that the MaxBackends recomputation needed to be duplicated by subprocesses in SubPostmasterMain. However, instead of having the value be recomputed at all, it's better to assign the correct value at postmaster initialization time, and have it be propagated to exec'ed backends via BackendParameters. MaxBackends stays as zero until after modules in shared_preload_libraries have had a chance to register bgworkers, since the value is going to be untrustworthy till that's finished. Heikki Linnakangas and Álvaro Herrera
2013-01-02 16:01:14 +01:00
MaxBackends = param->MaxBackends;
Implement new PostmasterIsAlive() check for WIN32, per Claudio Natoli. In passing, align a few error messages with the style guide.
2004-05-30 05:50:15 +02:00
#ifdef WIN32
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
PostmasterHandle = param->PostmasterHandle;
pgwin32_initial_signal_pipe = param->initial_signal_pipe;
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
#else
memcpy(&postmaster_alive_fds, &param->postmaster_alive_fds,
sizeof(postmaster_alive_fds));
Implement new PostmasterIsAlive() check for WIN32, per Claudio Natoli. In passing, align a few error messages with the style guide.
2004-05-30 05:50:15 +02:00
#endif
Final rearrangement of main postgresql child process (ie. BackendFork/SSDataBase/pgstat) startup, to allow fork/exec calls to closely mimic (the soon to be provided) Win32 CreateProcess equivalent calls. Claudio Natoli
2004-01-07 00:15:22 +01:00
Fix Win32 problems with signals and sockets, by making the forkexec code even uglier than it was already :-(. Also, on Windows only, use temporary shared memory segments instead of ordinary files to pass over critical variable values from postmaster to child processes. Magnus Hagander
2004-11-17 01:14:14 +01:00
memcpy(&syslogPipe, &param->syslogPipe, sizeof(syslogPipe));
Integrate src/timezone library for all platforms. There is more we can and should do now that we control our own destiny for timezone handling, but this commit gets the bulk of the picayune diffs in place. Magnus Hagander and Tom Lane.
2004-05-21 07:08:06 +02:00
StrNCpy -> strlcpy (not complete)
2007-02-10 15:58:55 +01:00
strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
StrNCpy -> strlcpy (not complete)
2007-02-10 15:58:55 +01:00
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
Autovacuum loose end mop-up. Provide autovacuum-specific vacuum cost delay and limit, both as global GUCs and as table-specific entries in pg_autovacuum. stats_reset_on_server_start is now OFF by default, but a reset is forced if we did WAL replay. XID-wrap vacuums do not ANALYZE, but do FREEZE if it's a template database. Alvaro Herrera
2005-08-11 23:11:50 +02:00
StrNCpy -> strlcpy (not complete)
2007-02-10 15:58:55 +01:00
strlcpy(ExtraOptions, param->ExtraOptions, MAXPGPATH);
This patch is the next step towards (re)allowing fork/exec. Claudio Natoli
2003-12-20 18:31:21 +01:00
}
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
Convert the arithmetic for shared memory size calculation from 'int' to 'Size' (that is, size_t), and install overflow detection checks in it. This allows us to remove the former arbitrary restrictions on NBuffers etc. It won't make any difference in a 32-bit machine, but in a 64-bit machine you could theoretically have terabytes of shared buffers. (How efficiently we could manage 'em remains to be seen.) Similarly, num_temp_buffers, work_mem, and maintenance_work_mem can be set above 2Gb on a 64-bit machine. Original patch from Koichi Suzuki, additional work by moi.
2005-08-21 01:26:37 +02:00
Size
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
ShmemBackendArraySize(void)
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
{
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
return mul_size(MaxLivePostmasterChildren(), sizeof(Backend));
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
}
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
void
ShmemBackendArrayAllocation(void)
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
{
Convert the arithmetic for shared memory size calculation from 'int' to 'Size' (that is, size_t), and install overflow detection checks in it. This allows us to remove the former arbitrary restrictions on NBuffers etc. It won't make any difference in a 32-bit machine, but in a 64-bit machine you could theoretically have terabytes of shared buffers. (How efficiently we could manage 'em remains to be seen.) Similarly, num_temp_buffers, work_mem, and maintenance_work_mem can be set above 2Gb on a 64-bit machine. Original patch from Koichi Suzuki, additional work by moi.
2005-08-21 01:26:37 +02:00
Size size = ShmemBackendArraySize();
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
ShmemBackendArray = (Backend *) ShmemAlloc(size);
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
/* Mark all slots as empty */
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
memset(ShmemBackendArray, 0, size);
}
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
static void
ShmemBackendArrayAdd(Backend *bn)
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
{
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
/* The array slot corresponding to my PMChildSlot should be free */
int i = bn->child_slot - 1;
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
Assert(ShmemBackendArray[i].pid == 0);
ShmemBackendArray[i] = *bn;
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
}
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
static void
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
ShmemBackendArrayRemove(Backend *bn)
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
{
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
int i = bn->child_slot - 1;
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
Install a "dead man switch" to allow the postmaster to detect cases where a backend has done exit(0) or exit(1) without having disengaged itself from shared memory. We are at risk for this whenever third-party code is loaded into a backend, since such code might not know it's supposed to go through proc_exit() instead. Also, it is reported that under Windows there are ways to externally kill a process that cause the status code returned to the postmaster to be indistinguishable from a voluntary exit (thank you, Microsoft). If this does happen then the system is probably hosed --- for instance, the dead session might still be holding locks. So the best recovery method is to treat this like a backend crash. The dead man switch is armed for a particular child process when it acquires a regular PGPROC, and disarmed when the PGPROC is released; these should be the first and last touches of shared memory resources in a backend, or close enough anyway. This choice means there is no coverage for auxiliary processes, but I doubt we need that, since they shouldn't be executing any user-provided code anyway. This patch also improves the management of the EXEC_BACKEND ShmemBackendArray array a bit, by reducing search costs. Although this problem is of long standing, the lack of field complaints seems to mean it's not critical enough to risk back-patching; at least not till we get some more testing of this mechanism.
2009-05-05 21:59:00 +02:00
Assert(ShmemBackendArray[i].pid == bn->pid);
/* Mark the slot as empty */
ShmemBackendArray[i].pid = 0;
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
}
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
#endif /* EXEC_BACKEND */
Code review for EXEC_BACKEND changes. Reduce the number of #ifdefs by about a third, make it work on non-Windows platforms again. (But perhaps I broke the WIN32 code, since I have no way to test that.) Fold all the paths that fork postmaster child processes to go through the single routine SubPostmasterMain, which takes care of resurrecting the state that would normally be inherited from the postmaster (including GUC variables). Clean up some places where there's no particularly good reason for the EXEC and non-EXEC cases to work differently. Take care of one or two FIXMEs that remained in the code.
2004-05-28 07:13:32 +02:00
Drops in the CreateProcess calls for Win32 (essentially wrapping up the fork/exec portion of the port), and fixes a handful of whitespace issues Claudio Natoli
2004-01-11 04:49:31 +01:00
#ifdef WIN32
Remove support for using wait3() in place of waitpid(). All Unix-oid platforms that we currently support should have waitpid(), since it's in V2 of the Single Unix Spec. Our git history shows that the wait3 code was added to support NextStep, which we officially dropped support for as of 9.2. So get rid of the configure test, and simplify the macro spaghetti in reaper(). Per suggestion from Fujii Masao.
2012-07-05 20:00:40 +02:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Subset implementation of waitpid() for Windows. We assume pid is -1
Remove support for using wait3() in place of waitpid(). All Unix-oid platforms that we currently support should have waitpid(), since it's in V2 of the Single Unix Spec. Our git history shows that the wait3 code was added to support NextStep, which we officially dropped support for as of 9.2. So get rid of the configure test, and simplify the macro spaghetti in reaper(). Per suggestion from Fujii Masao.
2012-07-05 20:00:40 +02:00
* (that is, check all child processes) and options is WNOHANG (don't wait).
*/
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
static pid_t
Remove support for using wait3() in place of waitpid(). All Unix-oid platforms that we currently support should have waitpid(), since it's in V2 of the Single Unix Spec. Our git history shows that the wait3 code was added to support NextStep, which we officially dropped support for as of 9.2. So get rid of the configure test, and simplify the macro spaghetti in reaper(). Per suggestion from Fujii Masao.
2012-07-05 20:00:40 +02:00
waitpid(pid_t pid, int *exitstatus, int options)
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
DWORD dwd;
ULONG_PTR key;
OVERLAPPED *ovl;
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* Check if there are any dead children. If there are, return the pid of
* the first one that died.
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
*/
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
if (GetQueuedCompletionStatus(win32ChildQueue, &dwd, &key, &ovl, 0))
Fix for postmaster.c function win32_waitpid(int *exitstatus) call to Win32 WaitForMultipleObjects: ret = WaitForMultipleObjects(win32_numChildren, win32_childHNDArray, FALSE, 0); Problem is 'win32_numChildren' could be more then 64 ( function supports ), problem basically arise ( kills postgres ) when you create more then 64 connections and terminate some of them sill leaving more then 64. Claudio Natoli
2004-08-29 05:16:30 +02:00
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
*exitstatus = (int) key;
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
return dwd;
[all] Removed call to getppid in SendPostmasterSignal, replacing with a PostmasterPid variable, which gets set (early) in PostmasterMain getppid would not be the postmaster? [fork/exec] Implements processCancelRequest by keeping an array of pid/cancel_key structs in shared mem [fork/exec] Moves AttachSharedMemoryAndSemaphores call for backends into SubPostmasterMain [win32] Implements reaper/waitpid by keeping an arrays of children pids,handles in postmaster local mem - this item is largely untested, for reasons which should be obvious, but appears sound [win32/all] Added extern for pgpipe in Win32 case, and changed the second pipe call (which seems to have been missed earlier) to pgpipe [win32] #define'd ftruncate to chsize in the Win32 case [win32] PG_USLEEP for Win32 has a misplaced paren. Fixed. [win32] DLLIMPORT handling for MingW case Claudio Natoli
2004-01-26 23:59:54 +01:00
}
return -1;
}
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
/*
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
* Note! Code below executes on a thread pool! All operations must
* be thread safe! Note that elog() and friends must *not* be used.
Separate out bgwriter code into a logically separate module, rather than being random pieces of other files. Give bgwriter responsibility for all checkpoint activity (other than a post-recovery checkpoint); so this child process absorbs the functionality of the former transient checkpoint and shutdown subprocesses. While at it, create an actual include file for postmaster.c, which for some reason never had its own file before.
2004-05-30 00:48:23 +02:00
*/
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
static void WINAPI
pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired)
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
win32_deadchild_waitinfo *childinfo = (win32_deadchild_waitinfo *) lpParameter;
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
DWORD exitcode;
pgindent files for Tom.
2004-05-27 17:07:41 +02:00
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
if (TimerOrWaitFired)
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
return; /* timeout. Should never happen, since we use
* INFINITE as timeout value. */
Win32 signals cleanup. Patch by Magnus Hagander, with input from Claudio Natoli and Bruce Momjian (and some cosmetic fixes from Neil Conway). Changes: - remove duplicate signal definitions from pqsignal.h - replace pqkill() with kill() and redefine kill() in Win32 - use ereport() in place of fprintf() in some error handling in pqsignal.c - export pg_queue_signal() and make use of it where necessary - add a console control handler for Ctrl-C and similar handling on Win32 - do WaitForSingleObjectEx() in CHECK_FOR_INTERRUPTS() on Win32; query cancelling should now work on Win32 - various other fixes and cleanups
2004-02-08 23:28:57 +01:00
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
/*
* Remove handle from wait - required even though it's set to wait only
* once
*/
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
UnregisterWaitEx(childinfo->waitHandle, NULL);
if (!GetExitCodeProcess(childinfo->procHandle, &exitcode))
{
/*
* Should never happen. Inform user and set a fixed exitcode.
*/
Message improvements
2007-11-15 21:04:38 +01:00
write_stderr("could not read exit code for process\n");
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
exitcode = 255;
}
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
if (!PostQueuedCompletionStatus(win32ChildQueue, childinfo->procId, (ULONG_PTR) exitcode, NULL))
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
write_stderr("could not post child completion status\n");
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
/*
* Handle is per-process, so we close it here instead of in the
* originating thread
*/
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
CloseHandle(childinfo->procHandle);
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
/*
* Free struct that was allocated before the call to
* RegisterWaitForSingleObject()
*/
Change win32 child-death tracking code to use a threadpool to wait for childprocess deaths instead of using one thread per child. This drastastically reduces the address space usage and should allow for more backends running. Also change the win32_waitpid functionality to use an IO Completion Port for queueing child death notices instead of using a fixed-size array.
2007-10-26 23:50:10 +02:00
free(childinfo);
/* Queue SIGCHLD signal */
pg_queue_signal(SIGCHLD);
Win32 signals cleanup. Patch by Magnus Hagander, with input from Claudio Natoli and Bruce Momjian (and some cosmetic fixes from Neil Conway). Changes: - remove duplicate signal definitions from pqsignal.h - replace pqkill() with kill() and redefine kill() in Win32 - use ereport() in place of fprintf() in some error handling in pqsignal.c - export pg_queue_signal() and make use of it where necessary - add a console control handler for Ctrl-C and similar handling on Win32 - do WaitForSingleObjectEx() in CHECK_FOR_INTERRUPTS() on Win32; query cancelling should now work on Win32 - various other fixes and cleanups
2004-02-08 23:28:57 +01:00
}
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
#endif /* WIN32 */
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
/*
* Initialize one and only handle for monitoring postmaster death.
*
* Called once in the postmaster, so that child processes can subsequently
* monitor if their parent is dead.
*/
static void
InitPostmasterDeathWatchHandle(void)
{
#ifndef WIN32
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
/*
* Create a pipe. Postmaster holds the write end of the pipe open
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* (POSTMASTER_FD_OWN), and children hold the read end. Children can pass
* the read file descriptor to select() to wake up in case postmaster
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
* dies, or check for postmaster death with a (read() == 0). Children must
* close the write end as soon as possible after forking, because EOF
* won't be signaled in the read end until all processes have closed the
* write fd. That is taken care of in ClosePostmasterPorts().
*/
Assert(MyProcPid == PostmasterPid);
Avoid depending on non-POSIX behavior of fcntl(2). The POSIX standard does not say that the success return value for fcntl(F_SETFD) and fcntl(F_SETFL) is zero; it says only that it's not -1. We had several calls that were making the stronger assumption. Adjust them to test specifically for -1 for strict spec compliance. The standard further leaves open the possibility that the O_NONBLOCK flag bit is not the only active one in F_SETFL's argument. Formally, therefore, one ought to get the current flags with F_GETFL and store them back with only the O_NONBLOCK bit changed when trying to change the nonblock state. In port/noblock.c, we were doing the full pushup in pg_set_block but not in pg_set_noblock, which is just weird. Make both of them do it properly, since they have little business making any assumptions about the socket they're handed. The other places where we're issuing F_SETFL are working with FDs we just got from pipe(2), so it's reasonable to assume the FDs' properties are all default, so I didn't bother adding F_GETFL steps there. Also, while pg_set_block deserves some points for trying to do things right, somebody had decided that it'd be even better to cast fcntl's third argument to "long". Which is completely loony, because POSIX clearly says the third argument for an F_SETFL call is "int". Given the lack of field complaints, these missteps apparently are not of significance on any common platforms. But they're still wrong, so back-patch to all supported branches. Discussion: https://postgr.es/m/30882.1492800880@sss.pgh.pa.us
2017-04-21 21:55:56 +02:00
if (pipe(postmaster_alive_fds) < 0)
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
ereport(FATAL,
(errcode_for_file_access(),
errmsg_internal("could not create pipe to monitor postmaster death: %m")));
/*
* Set O_NONBLOCK to allow testing for the fd's presence with a read()
* call.
*/
Avoid depending on non-POSIX behavior of fcntl(2). The POSIX standard does not say that the success return value for fcntl(F_SETFD) and fcntl(F_SETFL) is zero; it says only that it's not -1. We had several calls that were making the stronger assumption. Adjust them to test specifically for -1 for strict spec compliance. The standard further leaves open the possibility that the O_NONBLOCK flag bit is not the only active one in F_SETFL's argument. Formally, therefore, one ought to get the current flags with F_GETFL and store them back with only the O_NONBLOCK bit changed when trying to change the nonblock state. In port/noblock.c, we were doing the full pushup in pg_set_block but not in pg_set_noblock, which is just weird. Make both of them do it properly, since they have little business making any assumptions about the socket they're handed. The other places where we're issuing F_SETFL are working with FDs we just got from pipe(2), so it's reasonable to assume the FDs' properties are all default, so I didn't bother adding F_GETFL steps there. Also, while pg_set_block deserves some points for trying to do things right, somebody had decided that it'd be even better to cast fcntl's third argument to "long". Which is completely loony, because POSIX clearly says the third argument for an F_SETFL call is "int". Given the lack of field complaints, these missteps apparently are not of significance on any common platforms. But they're still wrong, so back-patch to all supported branches. Discussion: https://postgr.es/m/30882.1492800880@sss.pgh.pa.us
2017-04-21 21:55:56 +02:00
if (fcntl(postmaster_alive_fds[POSTMASTER_FD_WATCH], F_SETFL, O_NONBLOCK) == -1)
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
ereport(FATAL,
(errcode_for_socket_access(),
Standardize spelling of "nonblocking" Only adjusted the user-exposed messages and documentation, not all source code comments.
2013-04-19 05:35:19 +02:00
errmsg_internal("could not set postmaster death monitoring pipe to nonblocking mode: %m")));
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
#else
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
/*
* On Windows, we use a process handle for the same purpose.
*/
if (DuplicateHandle(GetCurrentProcess(),
GetCurrentProcess(),
GetCurrentProcess(),
&PostmasterHandle,
0,
TRUE,
DUPLICATE_SAME_ACCESS) == 0)
ereport(FATAL,
Use consistent format for reporting GetLastError() Use something like "error code %lu" for reporting GetLastError() values on Windows. Previously, a mix of different wordings and formats were in use.
2011-08-23 21:00:52 +02:00
(errmsg_internal("could not duplicate postmaster handle: error code %lu",
GetLastError())));
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
#endif /* WIN32 */
Introduce a pipe between postmaster and each backend, which can be used to detect postmaster death. Postmaster keeps the write-end of the pipe open, so when it dies, children get EOF in the read-end. That can conveniently be waited for in select(), which allows eliminating some of the polling loops that check for postmaster death. This patch doesn't yet change all the loops to use the new mechanism, expect a follow-on patch to do that. This changes the interface to WaitLatch, so that it takes as argument a bitmask of events that it waits for. Possible events are latch set, timeout, postmaster death, and socket becoming readable or writeable. The pipe method behaves slightly differently from the kill() method previously used in PostmasterIsAlive() in the case that postmaster has died, but its parent has not yet read its exit code with waitpid(). The pipe returns EOF as soon as the process dies, but kill() continues to return true until waitpid() has been called (IOW while the process is a zombie). Because of that, change PostmasterIsAlive() to use the pipe too, otherwise WaitLatch() would return immediately with WL_POSTMASTER_DEATH, while PostmasterIsAlive() would claim it's still alive. That could easily lead to busy-waiting while postmaster is in zombie state. Peter Geoghegan with further changes by me, reviewed by Fujii Masao and Florian Pflug.
2011-07-08 17:27:49 +02:00
}