opensourcemirror
/
postgresql
mirror of git://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Releases Wiki Activity
postgresql/src/backend/access/common/printtup.c

718 lines
19 KiB
C
Raw Normal View History

Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
1999-02-14 00:22:53 +01:00
* printtup.c
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
* Routines to print out tuples to the destination (both frontend
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
* clients and standalone backends are supported here).
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
*
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Update copyright for 2019 Backpatch-through: certain files through 9.4
2019-01-02 18:44:25 +01:00
* Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
Add: * Portions Copyright (c) 1996-2000, PostgreSQL, Inc to all files copyright Regents of Berkeley. Man, that's a lot of files.
2000-01-26 06:58:53 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
* IDENTIFICATION
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/backend/access/common/printtup.c
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*-------------------------------------------------------------------------
*/
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
#include "postgres.h"
#include "access/printtup.h"
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
#include "libpq/libpq.h"
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
#include "libpq/pqformat.h"
Fix the mechanism for reporting the original table OID and column number of columns of a query result so that it can "see through" cursors and prepared statements. Per gripe a couple months back from John DeSoi.
2005-06-22 19:45:46 +02:00
#include "tcop/pquery.h"
Add a bunch of pseudo-types to replace the behavior formerly associated with OPAQUE, as per recent pghackers discussion. I still want to do some more work on the 'cstring' pseudo-type, but I'm going to commit the bulk of the changes now before the tree starts shifting under me ...
2002-08-22 02:01:51 +02:00
#include "utils/lsyscache.h"
Cooperate with the Valgrind instrumentation framework. Valgrind "client requests" in aset.c and mcxt.c teach Valgrind and its Memcheck tool about the PostgreSQL allocator. This makes Valgrind roughly as sensitive to memory errors involving palloc chunks as it is to memory errors involving malloc chunks. Further client requests in PageAddItem() and printtup() verify that all bits being added to a buffer page or furnished to an output function are predictably-defined. Those tests catch failures of C-language functions to fully initialize the bits of a Datum, which in turn stymie optimizations that rely on _equalConst(). Define the USE_VALGRIND symbol in pg_config_manual.h to enable these additions. An included "suppression file" silences nominal errors we don't plan to fix. Reviewed in earlier versions by Peter Geoghegan and Korry Douglas.
2013-06-27 02:00:08 +02:00
#include "utils/memdebug.h"
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
#include "utils/memutils.h"
Add a bunch of pseudo-types to replace the behavior formerly associated with OPAQUE, as per recent pghackers discussion. I still want to do some more work on the 'cstring' pseudo-type, but I'm going to commit the bulk of the changes now before the tree starts shifting under me ...
2002-08-22 02:01:51 +02:00
Other then: indextuple.c:159: warning: `bp' might be used uninitialized in this function this directory passes -Wall -Werror under FreeBSD
1996-10-21 00:04:49 +02:00
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
static void printtup_startup(DestReceiver *self, int operation,
pgindent run.
2003-08-04 02:43:34 +02:00
TupleDesc typeinfo);
Stop the executor if no more tuples can be sent from worker to leader. If a Gather node has read as many tuples as it needs (for example, due to Limit) it may detach the queue connecting it to the worker before reading all of the worker's tuples. Rather than let the worker continue to generate and send all of the results, have it stop after sending the next tuple. More could be done here to stop the worker even quicker, but this is about as well as we can hope to do for 9.6. This is in response to a problem report from Andreas Seltenreich. Commit 44339b892a04e94bbb472235882dc6f7023bdc65 should be actually be sufficient to fix that example even without this change, but it seems better to do this, too, since we might otherwise waste quite a large amount of effort in one or more workers. Discussion: CAA4eK1KOKGqmz9bGu+Z42qhRwMbm4R5rfnqsLCNqFs9j14jzEA@mail.gmail.com Amit Kapila
2016-06-06 20:52:58 +02:00
static bool printtup(TupleTableSlot *slot, DestReceiver *self);
static bool printtup_20(TupleTableSlot *slot, DestReceiver *self);
static bool printtup_internal_20(TupleTableSlot *slot, DestReceiver *self);
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
static void printtup_shutdown(DestReceiver *self);
static void printtup_destroy(DestReceiver *self);
Improve performance of SendRowDescriptionMessage. There's three categories of changes leading to better performance: - Splitting the per-attribute part of SendRowDescriptionMessage into a v2 and a v3 version allows avoiding branches for every attribute. - Preallocating the size of the buffer to be big enough for all attributes and then using pq_write* avoids unnecessary buffer size checks & resizing. - Reusing a persistently allocated StringInfo for all SendRowDescriptionMessage() invocations avoids repeated allocations & reallocations. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:49:31 +02:00
static void SendRowDescriptionCols_2(StringInfo buf, TupleDesc typeinfo,
List *targetlist, int16 *formats);
static void SendRowDescriptionCols_3(StringInfo buf, TupleDesc typeinfo,
List *targetlist, int16 *formats);
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/* ----------------------------------------------------------------
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* printtup / debugtup support
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* ----------------------------------------------------------------
*/
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
/* ----------------
* Private state for a printtup destination object
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
*
* NOTE: finfo is the lookup info for either typoutput or typsend, whichever
* we are using for this column.
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
* ----------------
*/
pgindent run over code.
1999-05-25 18:15:34 +02:00
typedef struct
{ /* Per-attribute information */
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
Oid typoutput; /* Oid for the type's text output fn */
Oid typsend; /* Oid for the type's binary output fn */
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
bool typisvarlena; /* is it varlena (ie possibly toastable)? */
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
int16 format; /* format code for this column */
FmgrInfo finfo; /* Precomputed call info for output fn */
Another pgindent run. Sorry folks.
1999-05-26 00:43:53 +02:00
} PrinttupAttrInfo;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
pgindent run over code.
1999-05-25 18:15:34 +02:00
typedef struct
{
DestReceiver pub; /* publicly-known function pointers */
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
StringInfoData buf; /* output buffer */
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
Portal portal; /* the Portal we are printing from */
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
bool sendDescrip; /* send RowDescription at startup? */
pgindent run over code.
1999-05-25 18:15:34 +02:00
TupleDesc attrinfo; /* The attr info we are set up for */
int nattrs;
PrinttupAttrInfo *myinfo; /* Cached info about each attr */
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
MemoryContext tmpcontext; /* Memory context for per-row workspace */
Another pgindent run. Sorry folks.
1999-05-26 00:43:53 +02:00
} DR_printtup;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
/* ----------------
* Initialize: create a DestReceiver for printtup
* ----------------
*/
pgindent run over code.
1999-05-25 18:15:34 +02:00
DestReceiver *
Clean up the API for DestReceiver objects by eliminating the assumption that a Portal is a useful and sufficient additional argument for CreateDestReceiver --- it just isn't, in most cases. Instead formalize the approach of passing any needed parameters to the receiver separately. One unexpected benefit of this change is that we can declare typedef Portal in a less surprising location. This patch is just code rearrangement and doesn't change any functionality. I'll tackle the HOLD-cursor-vs-toast problem in a follow-on patch.
2008-11-30 21:51:25 +01:00
printtup_create_DR(CommandDest dest)
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
{
Clean up the API for DestReceiver objects by eliminating the assumption that a Portal is a useful and sufficient additional argument for CreateDestReceiver --- it just isn't, in most cases. Instead formalize the approach of passing any needed parameters to the receiver separately. One unexpected benefit of this change is that we can declare typedef Portal in a less surprising location. This patch is just code rearrangement and doesn't change any functionality. I'll tackle the HOLD-cursor-vs-toast problem in a follow-on patch.
2008-11-30 21:51:25 +01:00
DR_printtup *self = (DR_printtup *) palloc0(sizeof(DR_printtup));
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
self->pub.receiveSlot = printtup; /* might get changed later */
Rename fields of DestReceiver to avoid collisions with (ill-considered) macros in some platforms' sys/socket.h.
2003-08-06 19:46:46 +02:00
self->pub.rStartup = printtup_startup;
self->pub.rShutdown = printtup_shutdown;
self->pub.rDestroy = printtup_destroy;
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
self->pub.mydest = dest;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
Rename the members of CommandDest enum so they don't collide with other uses of those names. (Debug and None were pretty bad names anyway.) I hope I catched all uses of the names in comments too.
2005-11-03 18:11:40 +01:00
/*
* Send T message automatically if DestRemote, but not if
* DestRemoteExecute
*/
self->sendDescrip = (dest == DestRemote);
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
self->attrinfo = NULL;
self->nattrs = 0;
self->myinfo = NULL;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
self->tmpcontext = NULL;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
pgindent run over code.
1999-05-25 18:15:34 +02:00
return (DestReceiver *) self;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
}
Clean up the API for DestReceiver objects by eliminating the assumption that a Portal is a useful and sufficient additional argument for CreateDestReceiver --- it just isn't, in most cases. Instead formalize the approach of passing any needed parameters to the receiver separately. One unexpected benefit of this change is that we can declare typedef Portal in a less surprising location. This patch is just code rearrangement and doesn't change any functionality. I'll tackle the HOLD-cursor-vs-toast problem in a follow-on patch.
2008-11-30 21:51:25 +01:00
/*
* Set parameters for a DestRemote (or DestRemoteExecute) receiver
*/
void
SetRemoteDestReceiverParams(DestReceiver *self, Portal portal)
{
DR_printtup *myState = (DR_printtup *) self;
Assert(myState->pub.mydest == DestRemote ||
myState->pub.mydest == DestRemoteExecute);
myState->portal = portal;
if (PG_PROTOCOL_MAJOR(FrontendProtocol) < 3)
{
/*
* In protocol 2.0 the Bind message does not exist, so there is no way
* for the columns to have different print formats; it's sufficient to
* look at the first one.
*/
if (portal->formats && portal->formats[0] != 0)
myState->pub.receiveSlot = printtup_internal_20;
else
myState->pub.receiveSlot = printtup_20;
}
}
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
static void
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
printtup_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
{
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
DR_printtup *myState = (DR_printtup *) self;
pgindent run.
2003-08-04 02:43:34 +02:00
Portal portal = myState->portal;
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
/* create buffer to be used for all messages */
initStringInfo(&myState->buf);
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/*
* Create a temporary memory context that we can reset once per row to
* recover palloc'd memory. This avoids any problems with leaks inside
* datatype output routines, and should be faster than retail pfree's
* anyway.
*/
myState->tmpcontext = AllocSetContextCreate(CurrentMemoryContext,
"printtup",
Add macros to make AllocSetContextCreate() calls simpler and safer. I found that half a dozen (nearly 5%) of our AllocSetContextCreate calls had typos in the context-sizing parameters. While none of these led to especially significant problems, they did create minor inefficiencies, and it's now clear that expecting people to copy-and-paste those calls accurately is not a great idea. Let's reduce the risk of future errors by introducing single macros that encapsulate the common use-cases. Three such macros are enough to cover all but two special-purpose contexts; those two calls can be left as-is, I think. While this patch doesn't in itself improve matters for third-party extensions, it doesn't break anything for them either, and they can gradually adopt the simplified notation over time. In passing, change TopMemoryContext to use the default allocation parameters. Formerly it could only be extended 8K at a time. That was probably reasonable when this code was written; but nowadays we create many more contexts than we did then, so that it's not unusual to have a couple hundred K in TopMemoryContext, even without considering various dubious code that sticks other things there. There seems no good reason not to let it use growing blocks like most other contexts. Back-patch to 9.6, mostly because that's still close enough to HEAD that it's easy to do so, and keeping the branches in sync can be expected to avoid some future back-patching pain. The bugs fixed by these changes don't seem to be significant enough to justify fixing them further back. Discussion: <21072.1472321324@sss.pgh.pa.us>
2016-08-27 23:50:38 +02:00
ALLOCSET_DEFAULT_SIZES);
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
if (PG_PROTOCOL_MAJOR(FrontendProtocol) < 3)
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Send portal name to frontend (obsolete cruft, gone in proto 3.0)
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
*
* If portal name not specified, use "blank" portal.
*/
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
const char *portalName = portal->name;
if (portalName == NULL || portalName[0] == '\0')
Another round of protocol changes. Backend-to-frontend messages now all have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-22 02:08:07 +02:00
portalName = "blank";
pq_puttextmessage('P', portalName);
}
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* If we are supposed to emit row descriptions, then send the tuple
* descriptor of the tuples.
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
*/
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
2006-08-12 04:52:06 +02:00
if (myState->sendDescrip)
Improve performance of SendRowDescriptionMessage. There's three categories of changes leading to better performance: - Splitting the per-attribute part of SendRowDescriptionMessage into a v2 and a v3 version allows avoiding branches for every attribute. - Preallocating the size of the buffer to be big enough for all attributes and then using pq_write* avoids unnecessary buffer size checks & resizing. - Reusing a persistently allocated StringInfo for all SendRowDescriptionMessage() invocations avoids repeated allocations & reallocations. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:49:31 +02:00
SendRowDescriptionMessage(&myState->buf,
typeinfo,
Fix the mechanism for reporting the original table OID and column number of columns of a query result so that it can "see through" cursors and prepared statements. Per gripe a couple months back from John DeSoi.
2005-06-22 19:45:46 +02:00
FetchPortalTargetList(portal),
portal->formats);
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
/* ----------------
* We could set up the derived attr info at this time, but we postpone it
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
* until the first call of printtup, for 2 reasons:
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
* 1. We don't waste time (compared to the old way) if there are no
pgindent run over code.
1999-05-25 18:15:34 +02:00
* tuples at all to output.
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
* 2. Checking in printtup allows us to handle the case that the tuples
pgindent run over code.
1999-05-25 18:15:34 +02:00
* change type midway through (although this probably can't happen in
* the current executor).
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
* ----------------
*/
}
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
/*
* SendRowDescriptionMessage --- send a RowDescription message to the frontend
Implement feature of new FE/BE protocol whereby RowDescription identifies the column by table OID and column number, if it's a simple column reference. Along the way, get rid of reskey/reskeyop fields in Resdoms. Turns out that representation was not convenient for either the planner or the executor; we can make the planner deliver exactly what the executor wants with no more effort. initdb forced due to change in stored rule representation.
2003-05-06 02:20:33 +02:00
*
* Notes: the TupleDesc has typically been manufactured by ExecTypeFromTL()
* or some similar function; it does not contain a full set of fields.
* The targetlist will be NIL when executing a utility function that does
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
* not have a plan. If the targetlist isn't NIL then it is a Query node's
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* targetlist; it is up to us to ignore resjunk columns in it. The formats[]
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
* array pointer might be NULL (if we are doing Describe on a prepared stmt);
* send zeroes for the format codes in that case.
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
*/
void
Improve performance of SendRowDescriptionMessage. There's three categories of changes leading to better performance: - Splitting the per-attribute part of SendRowDescriptionMessage into a v2 and a v3 version allows avoiding branches for every attribute. - Preallocating the size of the buffer to be big enough for all attributes and then using pq_write* avoids unnecessary buffer size checks & resizing. - Reusing a persistently allocated StringInfo for all SendRowDescriptionMessage() invocations avoids repeated allocations & reallocations. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:49:31 +02:00
SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo,
List *targetlist, int16 *formats)
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
{
int natts = typeinfo->natts;
int proto = PG_PROTOCOL_MAJOR(FrontendProtocol);
Improve performance of SendRowDescriptionMessage. There's three categories of changes leading to better performance: - Splitting the per-attribute part of SendRowDescriptionMessage into a v2 and a v3 version allows avoiding branches for every attribute. - Preallocating the size of the buffer to be big enough for all attributes and then using pq_write* avoids unnecessary buffer size checks & resizing. - Reusing a persistently allocated StringInfo for all SendRowDescriptionMessage() invocations avoids repeated allocations & reallocations. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:49:31 +02:00
/* tuple descriptor message type */
pq_beginmessage_reuse(buf, 'T');
/* # of attrs in tuples */
pq_sendint16(buf, natts);
if (proto >= 3)
SendRowDescriptionCols_3(buf, typeinfo, targetlist, formats);
else
SendRowDescriptionCols_2(buf, typeinfo, targetlist, formats);
pq_endmessage_reuse(buf);
}
/*
* Send description for each column when using v3+ protocol
*/
static void
SendRowDescriptionCols_3(StringInfo buf, TupleDesc typeinfo, List *targetlist, int16 *formats)
{
int natts = typeinfo->natts;
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
int i;
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
ListCell *tlist_item = list_head(targetlist);
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
Improve performance of SendRowDescriptionMessage. There's three categories of changes leading to better performance: - Splitting the per-attribute part of SendRowDescriptionMessage into a v2 and a v3 version allows avoiding branches for every attribute. - Preallocating the size of the buffer to be big enough for all attributes and then using pq_write* avoids unnecessary buffer size checks & resizing. - Reusing a persistently allocated StringInfo for all SendRowDescriptionMessage() invocations avoids repeated allocations & reallocations. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:49:31 +02:00
/*
* Preallocate memory for the entire message to be sent. That allows to
* use the significantly faster inline pqformat.h functions and to avoid
* reallocations.
*
* Have to overestimate the size of the column-names, to account for
* character set overhead.
*/
enlargeStringInfo(buf, (NAMEDATALEN * MAX_CONVERSION_GROWTH /* attname */
+ sizeof(Oid) /* resorigtbl */
+ sizeof(AttrNumber) /* resorigcol */
+ sizeof(Oid) /* atttypid */
+ sizeof(int16) /* attlen */
+ sizeof(int32) /* attypmod */
+ sizeof(int16) /* format */
) * natts);
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
for (i = 0; i < natts; ++i)
{
Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n). This is a mechanical change in preparation for a later commit that will change the layout of TupleDesc. Introducing a macro to abstract the details of where attributes are stored will allow us to change that in separate step and revise it in future. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 20:19:07 +02:00
Form_pg_attribute att = TupleDescAttr(typeinfo, i);
Oid atttypid = att->atttypid;
int32 atttypmod = att->atttypmod;
Improve performance of SendRowDescriptionMessage. There's three categories of changes leading to better performance: - Splitting the per-attribute part of SendRowDescriptionMessage into a v2 and a v3 version allows avoiding branches for every attribute. - Preallocating the size of the buffer to be big enough for all attributes and then using pq_write* avoids unnecessary buffer size checks & resizing. - Reusing a persistently allocated StringInfo for all SendRowDescriptionMessage() invocations avoids repeated allocations & reallocations. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:49:31 +02:00
Oid resorigtbl;
AttrNumber resorigcol;
int16 format;
/*
* If column is a domain, send the base type and typmod instead.
* Lookup before sending any ints, for efficiency.
*/
atttypid = getBaseTypeAndTypmod(atttypid, &atttypmod);
In RowDescription messages, report columns of domain datatypes as having the type OID and typmod of the underlying base type. Per discussions a few weeks ago with Andreas Pflug and others. Note that this behavioral change affects both old- and new-protocol clients.
2003-05-13 20:39:50 +02:00
Improve performance of SendRowDescriptionMessage. There's three categories of changes leading to better performance: - Splitting the per-attribute part of SendRowDescriptionMessage into a v2 and a v3 version allows avoiding branches for every attribute. - Preallocating the size of the buffer to be big enough for all attributes and then using pq_write* avoids unnecessary buffer size checks & resizing. - Reusing a persistently allocated StringInfo for all SendRowDescriptionMessage() invocations avoids repeated allocations & reallocations. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:49:31 +02:00
/* Do we have a non-resjunk tlist item? */
while (tlist_item &&
((TargetEntry *) lfirst(tlist_item))->resjunk)
tlist_item = lnext(tlist_item);
if (tlist_item)
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
{
Improve performance of SendRowDescriptionMessage. There's three categories of changes leading to better performance: - Splitting the per-attribute part of SendRowDescriptionMessage into a v2 and a v3 version allows avoiding branches for every attribute. - Preallocating the size of the buffer to be big enough for all attributes and then using pq_write* avoids unnecessary buffer size checks & resizing. - Reusing a persistently allocated StringInfo for all SendRowDescriptionMessage() invocations avoids repeated allocations & reallocations. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:49:31 +02:00
TargetEntry *tle = (TargetEntry *) lfirst(tlist_item);
resorigtbl = tle->resorigtbl;
resorigcol = tle->resorigcol;
tlist_item = lnext(tlist_item);
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
}
Improve performance of SendRowDescriptionMessage. There's three categories of changes leading to better performance: - Splitting the per-attribute part of SendRowDescriptionMessage into a v2 and a v3 version allows avoiding branches for every attribute. - Preallocating the size of the buffer to be big enough for all attributes and then using pq_write* avoids unnecessary buffer size checks & resizing. - Reusing a persistently allocated StringInfo for all SendRowDescriptionMessage() invocations avoids repeated allocations & reallocations. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:49:31 +02:00
else
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
{
Improve performance of SendRowDescriptionMessage. There's three categories of changes leading to better performance: - Splitting the per-attribute part of SendRowDescriptionMessage into a v2 and a v3 version allows avoiding branches for every attribute. - Preallocating the size of the buffer to be big enough for all attributes and then using pq_write* avoids unnecessary buffer size checks & resizing. - Reusing a persistently allocated StringInfo for all SendRowDescriptionMessage() invocations avoids repeated allocations & reallocations. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:49:31 +02:00
/* No info available, so send zeroes */
resorigtbl = 0;
resorigcol = 0;
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
}
Improve performance of SendRowDescriptionMessage. There's three categories of changes leading to better performance: - Splitting the per-attribute part of SendRowDescriptionMessage into a v2 and a v3 version allows avoiding branches for every attribute. - Preallocating the size of the buffer to be big enough for all attributes and then using pq_write* avoids unnecessary buffer size checks & resizing. - Reusing a persistently allocated StringInfo for all SendRowDescriptionMessage() invocations avoids repeated allocations & reallocations. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:49:31 +02:00
if (formats)
format = formats[i];
else
format = 0;
pq_writestring(buf, NameStr(att->attname));
pq_writeint32(buf, resorigtbl);
pq_writeint16(buf, resorigcol);
pq_writeint32(buf, atttypid);
pq_writeint16(buf, att->attlen);
pq_writeint32(buf, atttypmod);
pq_writeint16(buf, format);
}
}
/*
* Send description for each column when using v2 protocol
*/
static void
SendRowDescriptionCols_2(StringInfo buf, TupleDesc typeinfo, List *targetlist, int16 *formats)
{
int natts = typeinfo->natts;
int i;
for (i = 0; i < natts; ++i)
{
Form_pg_attribute att = TupleDescAttr(typeinfo, i);
Oid atttypid = att->atttypid;
int32 atttypmod = att->atttypmod;
/* If column is a domain, send the base type and typmod instead */
atttypid = getBaseTypeAndTypmod(atttypid, &atttypmod);
pq_sendstring(buf, NameStr(att->attname));
/* column ID only info appears in protocol 3.0 and up */
pq_sendint32(buf, atttypid);
pq_sendint16(buf, att->attlen);
pq_sendint32(buf, atttypmod);
/* format info only appears in protocol 3.0 and up */
Extended query protocol: parse, bind, execute, describe FE/BE messages. Only lightly tested as yet, since libpq doesn't know anything about 'em.
2003-05-05 02:44:56 +02:00
}
}
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
/*
* Get the lookup info that printtup() needs
*/
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
static void
Another pgindent run. Sorry folks.
1999-05-26 00:43:53 +02:00
printtup_prepare_info(DR_printtup *myState, TupleDesc typeinfo, int numAttrs)
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
{
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
int16 *formats = myState->portal->formats;
pgindent run over code.
1999-05-25 18:15:34 +02:00
int i;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
/* get rid of any old data */
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
if (myState->myinfo)
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
pfree(myState->myinfo);
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
myState->myinfo = NULL;
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
myState->attrinfo = typeinfo;
myState->nattrs = numAttrs;
if (numAttrs <= 0)
return;
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
pgindent run over code.
1999-05-25 18:15:34 +02:00
myState->myinfo = (PrinttupAttrInfo *)
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
palloc0(numAttrs * sizeof(PrinttupAttrInfo));
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
for (i = 0; i < numAttrs; i++)
{
pgindent run over code.
1999-05-25 18:15:34 +02:00
PrinttupAttrInfo *thisState = myState->myinfo + i;
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
int16 format = (formats ? formats[i] : 0);
Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n). This is a mechanical change in preparation for a later commit that will change the layout of TupleDesc. Introducing a macro to abstract the details of where attributes are stored will allow us to change that in separate step and revise it in future. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 20:19:07 +02:00
Form_pg_attribute attr = TupleDescAttr(typeinfo, i);
pgindent run over code.
1999-05-25 18:15:34 +02:00
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
thisState->format = format;
if (format == 0)
{
Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n). This is a mechanical change in preparation for a later commit that will change the layout of TupleDesc. Introducing a macro to abstract the details of where attributes are stored will allow us to change that in separate step and revise it in future. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 20:19:07 +02:00
getTypeOutputInfo(attr->atttypid,
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
&thisState->typoutput,
&thisState->typisvarlena);
Turns out it's easy to cache the fmgr function lookup result, too ... another little bit of speed for SELECT.
1999-01-27 02:11:43 +01:00
fmgr_info(thisState->typoutput, &thisState->finfo);
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
}
else if (format == 1)
{
Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n). This is a mechanical change in preparation for a later commit that will change the layout of TupleDesc. Introducing a macro to abstract the details of where attributes are stored will allow us to change that in separate step and revise it in future. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 20:19:07 +02:00
getTypeBinaryOutputInfo(attr->atttypid,
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
&thisState->typsend,
&thisState->typisvarlena);
fmgr_info(thisState->typsend, &thisState->finfo);
}
else
Error message editing in backend/access.
2003-07-21 22:29:40 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("unsupported format code: %d", format)));
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
}
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/* ----------------
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
* printtup --- print a tuple in protocol 3.0
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* ----------------
*/
Stop the executor if no more tuples can be sent from worker to leader. If a Gather node has read as many tuples as it needs (for example, due to Limit) it may detach the queue connecting it to the worker before reading all of the worker's tuples. Rather than let the worker continue to generate and send all of the results, have it stop after sending the next tuple. More could be done here to stop the worker even quicker, but this is about as well as we can hope to do for 9.6. This is in response to a problem report from Andreas Seltenreich. Commit 44339b892a04e94bbb472235882dc6f7023bdc65 should be actually be sufficient to fix that example even without this change, but it seems better to do this, too, since we might otherwise waste quite a large amount of effort in one or more workers. Discussion: CAA4eK1KOKGqmz9bGu+Z42qhRwMbm4R5rfnqsLCNqFs9j14jzEA@mail.gmail.com Amit Kapila
2016-06-06 20:52:58 +02:00
static bool
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
printtup(TupleTableSlot *slot, DestReceiver *self)
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
TupleDesc typeinfo = slot->tts_tupleDescriptor;
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
DR_printtup *myState = (DR_printtup *) self;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
MemoryContext oldcontext;
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
StringInfo buf = &myState->buf;
Make sure printtup() always sends the number of columns previously advertised in RowDescription message. Depending on the physical tuple's column count is not really correct, since according to heap_getattr() conventions the tuple may be short some columns, which will automatically get read as nulls. Problem has been latent since forever, but was only exposed by recent change to skip a projection step in SELECT * FROM...
2003-05-26 19:51:38 +02:00
int natts = typeinfo->natts;
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
int i;
/* Set or update my derived attribute info, if needed */
if (myState->attrinfo != typeinfo || myState->nattrs != natts)
printtup_prepare_info(myState, typeinfo, natts);
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
/* Make sure the tuple is fully deconstructed */
slot_getallattrs(slot);
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/* Switch into per-row context so we can recover memory below */
oldcontext = MemoryContextSwitchTo(myState->tmpcontext);
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
/*
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
* Prepare a DataRow message (note buffer is in per-row context)
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
*/
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
pq_beginmessage_reuse(buf, 'D');
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 06:00:46 +02:00
pq_sendint16(buf, natts);
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
/*
* send the attributes of this tuple
*/
for (i = 0; i < natts; ++i)
{
PrinttupAttrInfo *thisState = myState->myinfo + i;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
Datum attr = slot->tts_values[i];
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
if (slot->tts_isnull[i])
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
{
Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 06:00:46 +02:00
pq_sendint32(buf, -1);
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
continue;
}
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
/*
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
* Here we catch undefined bytes in datums that are returned to the
Cooperate with the Valgrind instrumentation framework. Valgrind "client requests" in aset.c and mcxt.c teach Valgrind and its Memcheck tool about the PostgreSQL allocator. This makes Valgrind roughly as sensitive to memory errors involving palloc chunks as it is to memory errors involving malloc chunks. Further client requests in PageAddItem() and printtup() verify that all bits being added to a buffer page or furnished to an output function are predictably-defined. Those tests catch failures of C-language functions to fully initialize the bits of a Datum, which in turn stymie optimizations that rely on _equalConst(). Define the USE_VALGRIND symbol in pg_config_manual.h to enable these additions. An included "suppression file" silences nominal errors we don't plan to fix. Reviewed in earlier versions by Peter Geoghegan and Korry Douglas.
2013-06-27 02:00:08 +02:00
* client without hitting disk; see comments at the related check in
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
* PageAddItem(). This test is most useful for uncompressed,
* non-external datums, but we're quite likely to see such here when
* testing new C functions.
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
*/
if (thisState->typisvarlena)
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
VALGRIND_CHECK_MEM_IS_DEFINED(DatumGetPointer(attr),
VARSIZE_ANY(attr));
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
if (thisState->format == 0)
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
{
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
/* Text output */
char *outputstr;
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
outputstr = OutputFunctionCall(&thisState->finfo, attr);
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
pq_sendcountedtext(buf, outputstr, strlen(outputstr), false);
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
}
else
{
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
/* Binary output */
bytea *outputbytes;
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
outputbytes = SendFunctionCall(&thisState->finfo, attr);
Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 06:00:46 +02:00
pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ);
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
pq_sendbytes(buf, VARDATA(outputbytes),
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
VARSIZE(outputbytes) - VARHDRSZ);
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
}
}
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
pq_endmessage_reuse(buf);
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/* Return to caller's context, and flush row's temporary memory */
MemoryContextSwitchTo(oldcontext);
MemoryContextReset(myState->tmpcontext);
Stop the executor if no more tuples can be sent from worker to leader. If a Gather node has read as many tuples as it needs (for example, due to Limit) it may detach the queue connecting it to the worker before reading all of the worker's tuples. Rather than let the worker continue to generate and send all of the results, have it stop after sending the next tuple. More could be done here to stop the worker even quicker, but this is about as well as we can hope to do for 9.6. This is in response to a problem report from Andreas Seltenreich. Commit 44339b892a04e94bbb472235882dc6f7023bdc65 should be actually be sufficient to fix that example even without this change, but it seems better to do this, too, since we might otherwise waste quite a large amount of effort in one or more workers. Discussion: CAA4eK1KOKGqmz9bGu+Z42qhRwMbm4R5rfnqsLCNqFs9j14jzEA@mail.gmail.com Amit Kapila
2016-06-06 20:52:58 +02:00
return true;
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
}
/* ----------------
* printtup_20 --- print a tuple in protocol 2.0
* ----------------
*/
Stop the executor if no more tuples can be sent from worker to leader. If a Gather node has read as many tuples as it needs (for example, due to Limit) it may detach the queue connecting it to the worker before reading all of the worker's tuples. Rather than let the worker continue to generate and send all of the results, have it stop after sending the next tuple. More could be done here to stop the worker even quicker, but this is about as well as we can hope to do for 9.6. This is in response to a problem report from Andreas Seltenreich. Commit 44339b892a04e94bbb472235882dc6f7023bdc65 should be actually be sufficient to fix that example even without this change, but it seems better to do this, too, since we might otherwise waste quite a large amount of effort in one or more workers. Discussion: CAA4eK1KOKGqmz9bGu+Z42qhRwMbm4R5rfnqsLCNqFs9j14jzEA@mail.gmail.com Amit Kapila
2016-06-06 20:52:58 +02:00
static bool
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
printtup_20(TupleTableSlot *slot, DestReceiver *self)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
TupleDesc typeinfo = slot->tts_tupleDescriptor;
pgindent run over code.
1999-05-25 18:15:34 +02:00
DR_printtup *myState = (DR_printtup *) self;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
MemoryContext oldcontext;
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
StringInfo buf = &myState->buf;
Make sure printtup() always sends the number of columns previously advertised in RowDescription message. Depending on the physical tuple's column count is not really correct, since according to heap_getattr() conventions the tuple may be short some columns, which will automatically get read as nulls. Problem has been latent since forever, but was only exposed by recent change to skip a projection step in SELECT * FROM...
2003-05-26 19:51:38 +02:00
int natts = typeinfo->natts;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i,
j,
My first cut at libpq revision didn't handle MULTIBYTE correctly, but I think it's OK now...
1999-04-25 21:27:47 +02:00
k;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
/* Set or update my derived attribute info, if needed */
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
if (myState->attrinfo != typeinfo || myState->nattrs != natts)
printtup_prepare_info(myState, typeinfo, natts);
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
/* Make sure the tuple is fully deconstructed */
slot_getallattrs(slot);
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/* Switch into per-row context so we can recover memory below */
oldcontext = MemoryContextSwitchTo(myState->tmpcontext);
Remove dashes in comments that don't need them, rewrap with pgindent.
2001-03-22 07:16:21 +01:00
/*
* tell the frontend to expect new tuple data (in ASCII style)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
pq_beginmessage_reuse(buf, 'D');
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Remove dashes in comments that don't need them, rewrap with pgindent.
2001-03-22 07:16:21 +01:00
/*
* send a bitmap of which attributes are not null
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
j = 0;
k = 1 << 7;
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
for (i = 0; i < natts; ++i)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Recent changes got the sense of the notnull bit backwards in the 2.0 protocol output routines. Mea culpa :-(. Per report from Kris Jurka.
2005-04-23 19:45:35 +02:00
if (!slot->tts_isnull[i])
Use heap_attisnull, rather than heap_getattr, for a small but useful speedup.
1999-01-24 23:50:58 +01:00
j |= k; /* set bit if not null */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
k >>= 1;
Use heap_attisnull, rather than heap_getattr, for a small but useful speedup.
1999-01-24 23:50:58 +01:00
if (k == 0) /* end of byte? */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 06:00:46 +02:00
pq_sendint8(buf, j);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
j = 0;
k = 1 << 7;
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Use heap_attisnull, rather than heap_getattr, for a small but useful speedup.
1999-01-24 23:50:58 +01:00
if (k != (1 << 7)) /* flush last partial byte */
Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 06:00:46 +02:00
pq_sendint8(buf, j);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Remove dashes in comments that don't need them, rewrap with pgindent.
2001-03-22 07:16:21 +01:00
/*
* send the attributes of this tuple
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
for (i = 0; i < natts; ++i)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
pgindent run over code.
1999-05-25 18:15:34 +02:00
PrinttupAttrInfo *thisState = myState->myinfo + i;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
Datum attr = slot->tts_values[i];
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
char *outputstr;
pgindent run over code.
1999-05-25 18:15:34 +02:00
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
if (slot->tts_isnull[i])
From: Massimo Dal Zotto <dz@cs.unitn.it> After some playing with gdb I found that in printtup() there is a non null attribute with typeinfo->attrs[i]->atttypid = 0 (invalid oid). Unfortunately attibutes with invalid type are neither printed nor marked as null, and this explains why psql doesn't get all the expected data. So I made this patch to printtup():
1998-08-30 21:30:38 +02:00
continue;
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
Assert(thisState->format == 0);
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
outputstr = OutputFunctionCall(&thisState->finfo, attr);
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
pq_sendcountedtext(buf, outputstr, strlen(outputstr), true);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
pq_endmessage_reuse(buf);
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/* Return to caller's context, and flush row's temporary memory */
MemoryContextSwitchTo(oldcontext);
MemoryContextReset(myState->tmpcontext);
Stop the executor if no more tuples can be sent from worker to leader. If a Gather node has read as many tuples as it needs (for example, due to Limit) it may detach the queue connecting it to the worker before reading all of the worker's tuples. Rather than let the worker continue to generate and send all of the results, have it stop after sending the next tuple. More could be done here to stop the worker even quicker, but this is about as well as we can hope to do for 9.6. This is in response to a problem report from Andreas Seltenreich. Commit 44339b892a04e94bbb472235882dc6f7023bdc65 should be actually be sufficient to fix that example even without this change, but it seems better to do this, too, since we might otherwise waste quite a large amount of effort in one or more workers. Discussion: CAA4eK1KOKGqmz9bGu+Z42qhRwMbm4R5rfnqsLCNqFs9j14jzEA@mail.gmail.com Amit Kapila
2016-06-06 20:52:58 +02:00
return true;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
/* ----------------
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
* printtup_shutdown
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
* ----------------
*/
static void
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
printtup_shutdown(DestReceiver *self)
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
{
pgindent run over code.
1999-05-25 18:15:34 +02:00
DR_printtup *myState = (DR_printtup *) self;
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
if (myState->myinfo)
pfree(myState->myinfo);
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
myState->myinfo = NULL;
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
myState->attrinfo = NULL;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
if (myState->tmpcontext)
MemoryContextDelete(myState->tmpcontext);
myState->tmpcontext = NULL;
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
}
/* ----------------
* printtup_destroy
* ----------------
*/
static void
printtup_destroy(DestReceiver *self)
{
pfree(self);
Another SELECT speedup: extract OIDs of column print functions only once per SELECT, not once per tuple. 10% here, 10% there, pretty soon you're talking about real speedups ...
1999-01-27 01:36:28 +01:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/* ----------------
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* printatt
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* ----------------
*/
static void
printatt(unsigned attributeId,
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
Form_pg_attribute attributeP,
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
char *value)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Pass atttypmod to the frontend.
1998-05-14 19:18:14 +02:00
printf("\t%2d: %s%s%s%s\t(typeid = %u, len = %d, typmod = %d, byval = %c)\n",
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
attributeId,
New NameStr macro to convert Name to Str. No need for var.data anymore. Fewer calls to nameout. Better use of RelationGetRelationName.
1999-11-08 00:08:36 +01:00
NameStr(attributeP->attname),
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
value != NULL ? " = \"" : "",
value != NULL ? value : "",
value != NULL ? "\"" : "",
(unsigned int) (attributeP->atttypid),
attributeP->attlen,
Pass atttypmod to the frontend.
1998-05-14 19:18:14 +02:00
attributeP->atttypmod,
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
attributeP->attbyval ? 't' : 'f');
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/* ----------------
Restructure command destination handling so that we pass around DestReceiver pointers instead of just CommandDest values. The DestReceiver is made at the point where the destination is selected, rather than deep inside the executor. This cleans up the original kluge implementation of tstoreReceiver.c, and makes it easy to support retrieving results from utility statements inside portals. Thus, you can now do fun things like Bind and Execute a FETCH or EXPLAIN command, and it'll all work as expected (e.g., you can Describe the portal, or use Execute's count parameter to suspend the output partway through). Implementation involves stuffing the utility command's output into a Tuplestore, which would be kind of annoying for huge output sets, but should be quite acceptable for typical uses of utility commands.
2003-05-06 22:26:28 +02:00
* debugStartup - prepare to print tuples for an interactive backend
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
* ----------------
*/
void
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
debugStartup(DestReceiver *self, int operation, TupleDesc typeinfo)
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
{
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
int natts = typeinfo->natts;
int i;
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
/*
* show the return type of the tuples
*/
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
for (i = 0; i < natts; ++i)
Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n). This is a mechanical change in preparation for a later commit that will change the layout of TupleDesc. Introducing a macro to abstract the details of where attributes are stored will allow us to change that in separate step and revise it in future. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 20:19:07 +02:00
printatt((unsigned) i + 1, TupleDescAttr(typeinfo, i), NULL);
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
printf("\t----\n");
Clean up BeginCommand and related routines. BeginCommand and EndCommand are now both invoked once per received SQL command (raw parsetree) from pg_exec_query_string. BeginCommand is actually just an empty routine at the moment --- all its former operations have been pushed into tuple receiver setup routines in printtup.c. This makes for a clean distinction between BeginCommand/EndCommand (once per command) and the tuple receiver setup/teardown routines (once per ExecutorRun call), whereas the old code was quite ad hoc. Along the way, clean up the calling conventions for ExecutorRun a little bit.
2002-02-27 20:36:13 +01:00
}
/* ----------------
* debugtup - print one tuple for an interactive backend
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* ----------------
*/
Stop the executor if no more tuples can be sent from worker to leader. If a Gather node has read as many tuples as it needs (for example, due to Limit) it may detach the queue connecting it to the worker before reading all of the worker's tuples. Rather than let the worker continue to generate and send all of the results, have it stop after sending the next tuple. More could be done here to stop the worker even quicker, but this is about as well as we can hope to do for 9.6. This is in response to a problem report from Andreas Seltenreich. Commit 44339b892a04e94bbb472235882dc6f7023bdc65 should be actually be sufficient to fix that example even without this change, but it seems better to do this, too, since we might otherwise waste quite a large amount of effort in one or more workers. Discussion: CAA4eK1KOKGqmz9bGu+Z42qhRwMbm4R5rfnqsLCNqFs9j14jzEA@mail.gmail.com Amit Kapila
2016-06-06 20:52:58 +02:00
bool
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
debugtup(TupleTableSlot *slot, DestReceiver *self)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
TupleDesc typeinfo = slot->tts_tupleDescriptor;
Make sure printtup() always sends the number of columns previously advertised in RowDescription message. Depending on the physical tuple's column count is not really correct, since according to heap_getattr() conventions the tuple may be short some columns, which will automatically get read as nulls. Problem has been latent since forever, but was only exposed by recent change to skip a projection step in SELECT * FROM...
2003-05-26 19:51:38 +02:00
int natts = typeinfo->natts;
pgindent run before 6.3 release, with Thomas' requested changes.
1998-02-26 05:46:47 +01:00
int i;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
Datum attr;
heapattr functions now return a Datum, not char *.
1997-09-12 06:09:08 +02:00
char *value;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
bool isnull;
Change CREATE TYPE to require datatype output and send functions to have only one argument. (Per recent discussion, the option to accept multiple arguments is pretty useless for user-defined types, and would be a likely source of security holes if it was used.) Simplify call sites of output/send functions to not bother passing more than one argument.
2005-05-01 20:56:19 +02:00
Oid typoutput;
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
bool typisvarlena;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
for (i = 0; i < natts; ++i)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
attr = slot_getattr(slot, i + 1, &isnull);
Replace typtoout() and gettypelem() with a single routine, so that fetching an attribute value needs only one SearchSysCacheTuple call instead of two redundant searches. This speeds up a large SELECT by about ten percent, and probably will help GROUP BY and SELECT DISTINCT too.
1999-01-24 06:40:49 +01:00
if (isnull)
continue;
Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n). This is a mechanical change in preparation for a later commit that will change the layout of TupleDesc. Introducing a macro to abstract the details of where attributes are stored will allow us to change that in separate step and revise it in future. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 20:19:07 +02:00
getTypeOutputInfo(TupleDescAttr(typeinfo, i)->atttypid,
Change CREATE TYPE to require datatype output and send functions to have only one argument. (Per recent discussion, the option to accept multiple arguments is pretty useless for user-defined types, and would be a likely source of security holes if it was used.) Simplify call sites of output/send functions to not bother passing more than one argument.
2005-05-01 20:56:19 +02:00
&typoutput, &typisvarlena);
pgindent run.
2003-08-04 02:43:34 +02:00
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
value = OidOutputFunctionCall(typoutput, attr);
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n). This is a mechanical change in preparation for a later commit that will change the layout of TupleDesc. Introducing a macro to abstract the details of where attributes are stored will allow us to change that in separate step and revise it in future. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 20:19:07 +02:00
printatt((unsigned) i + 1, TupleDescAttr(typeinfo, i), value);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
printf("\t----\n");
Stop the executor if no more tuples can be sent from worker to leader. If a Gather node has read as many tuples as it needs (for example, due to Limit) it may detach the queue connecting it to the worker before reading all of the worker's tuples. Rather than let the worker continue to generate and send all of the results, have it stop after sending the next tuple. More could be done here to stop the worker even quicker, but this is about as well as we can hope to do for 9.6. This is in response to a problem report from Andreas Seltenreich. Commit 44339b892a04e94bbb472235882dc6f7023bdc65 should be actually be sufficient to fix that example even without this change, but it seems better to do this, too, since we might otherwise waste quite a large amount of effort in one or more workers. Discussion: CAA4eK1KOKGqmz9bGu+Z42qhRwMbm4R5rfnqsLCNqFs9j14jzEA@mail.gmail.com Amit Kapila
2016-06-06 20:52:58 +02:00
return true;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/* ----------------
Update 3.0 protocol support to match recent agreements about how to handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-05-08 20:16:37 +02:00
* printtup_internal_20 --- print a binary tuple in protocol 2.0
*
* We use a different message type, i.e. 'B' instead of 'D' to
* indicate a tuple in internal (binary) form.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
* This is largely same as printtup_20, except we use binary formatting.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* ----------------
*/
Stop the executor if no more tuples can be sent from worker to leader. If a Gather node has read as many tuples as it needs (for example, due to Limit) it may detach the queue connecting it to the worker before reading all of the worker's tuples. Rather than let the worker continue to generate and send all of the results, have it stop after sending the next tuple. More could be done here to stop the worker even quicker, but this is about as well as we can hope to do for 9.6. This is in response to a problem report from Andreas Seltenreich. Commit 44339b892a04e94bbb472235882dc6f7023bdc65 should be actually be sufficient to fix that example even without this change, but it seems better to do this, too, since we might otherwise waste quite a large amount of effort in one or more workers. Discussion: CAA4eK1KOKGqmz9bGu+Z42qhRwMbm4R5rfnqsLCNqFs9j14jzEA@mail.gmail.com Amit Kapila
2016-06-06 20:52:58 +02:00
static bool
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
printtup_internal_20(TupleTableSlot *slot, DestReceiver *self)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
TupleDesc typeinfo = slot->tts_tupleDescriptor;
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
DR_printtup *myState = (DR_printtup *) self;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
MemoryContext oldcontext;
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
StringInfo buf = &myState->buf;
Make sure printtup() always sends the number of columns previously advertised in RowDescription message. Depending on the physical tuple's column count is not really correct, since according to heap_getattr() conventions the tuple may be short some columns, which will automatically get read as nulls. Problem has been latent since forever, but was only exposed by recent change to skip a projection step in SELECT * FROM...
2003-05-26 19:51:38 +02:00
int natts = typeinfo->natts;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i,
j,
k;
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
/* Set or update my derived attribute info, if needed */
if (myState->attrinfo != typeinfo || myState->nattrs != natts)
printtup_prepare_info(myState, typeinfo, natts);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
/* Make sure the tuple is fully deconstructed */
slot_getallattrs(slot);
Resurrect heap_deformtuple(), this time implemented as a singly nested loop over the fields instead of a loop around heap_getattr. This is considerably faster (O(N) instead of O(N^2)) when there are nulls or varlena fields, since those prevent use of attcacheoff. Replace loops over heap_getattr with heap_deformtuple in situations where all or most of the fields have to be fetched, such as printtup and tuptoaster. Profiling done more than a year ago shows that this should be a nice win for situations involving many-column tables.
2004-06-04 22:35:21 +02:00
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/* Switch into per-row context so we can recover memory below */
oldcontext = MemoryContextSwitchTo(myState->tmpcontext);
Remove dashes in comments that don't need them, rewrap with pgindent.
2001-03-22 07:16:21 +01:00
/*
* tell the frontend to expect new tuple data (in binary style)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
pq_beginmessage_reuse(buf, 'B');
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Remove dashes in comments that don't need them, rewrap with pgindent.
2001-03-22 07:16:21 +01:00
/*
* send a bitmap of which attributes are not null
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
j = 0;
k = 1 << 7;
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
for (i = 0; i < natts; ++i)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Recent changes got the sense of the notnull bit backwards in the 2.0 protocol output routines. Mea culpa :-(. Per report from Kris Jurka.
2005-04-23 19:45:35 +02:00
if (!slot->tts_isnull[i])
Use heap_attisnull, rather than heap_getattr, for a small but useful speedup.
1999-01-24 23:50:58 +01:00
j |= k; /* set bit if not null */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
k >>= 1;
Use heap_attisnull, rather than heap_getattr, for a small but useful speedup.
1999-01-24 23:50:58 +01:00
if (k == 0) /* end of byte? */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 06:00:46 +02:00
pq_sendint8(buf, j);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
j = 0;
k = 1 << 7;
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Use heap_attisnull, rather than heap_getattr, for a small but useful speedup.
1999-01-24 23:50:58 +01:00
if (k != (1 << 7)) /* flush last partial byte */
Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 06:00:46 +02:00
pq_sendint8(buf, j);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Remove dashes in comments that don't need them, rewrap with pgindent.
2001-03-22 07:16:21 +01:00
/*
* send the attributes of this tuple
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
for (i = 0; i < natts; ++i)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
PrinttupAttrInfo *thisState = myState->myinfo + i;
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
Datum attr = slot->tts_values[i];
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
bytea *outputbytes;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Revise TupleTableSlot code to avoid unnecessary construction and disassembly of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-16 22:38:10 +01:00
if (slot->tts_isnull[i])
Make tuple receive/print routines TOAST-aware. Formerly, printtup would leak memory when printing a toasted attribute, and printtup_internal didn't work at all...
2000-12-01 23:10:31 +01:00
continue;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
Assert(thisState->format == 1);
Included patches should fix following problems in the muti-byte enabled PostgreSQL 6.4. o binary cursor does not work o pg_dumpall produces incorrect create database statemnt Tatsuo Ishii t-ishii@sra.co.jp
1998-12-12 23:04:09 +01:00
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
outputbytes = SendFunctionCall(&thisState->finfo, attr);
Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 06:00:46 +02:00
pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ);
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
pq_sendbytes(buf, VARDATA(outputbytes),
Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCall messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-09 20:08:48 +02:00
VARSIZE(outputbytes) - VARHDRSZ);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Revise backend libpq interfaces so that messages to the frontend can be generated in a buffer and then sent to the frontend in a single libpq call. This solves problems with NOTICE and ERROR messages generated in the middle of a data message or COPY OUT operation.
1999-04-25 05:19:27 +02:00
Use one stringbuffer for all rows printed in printtup.c. This avoids newly allocating, and then possibly growing, the stringbuffer for every row. For wide rows this can substantially reduce memory allocator overhead, at the price of not immediately reducing memory usage after outputting an especially wide row. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 01:26:35 +02:00
pq_endmessage_reuse(buf);
Prevent memory leaks from accumulating across printtup() calls. Historically, printtup() has assumed that it could prevent memory leakage by pfree'ing the string result of each output function and manually managing detoasting of toasted values. This amounts to assuming that datatype output functions never leak any memory internally; an assumption we've already decided to be bogus elsewhere, for example in COPY OUT. range_out in particular is known to leak multiple kilobytes per call, as noted in bug #8573 from Godfried Vanluffelen. While we could go in and fix that leak, it wouldn't be very notationally convenient, and in any case there have been and undoubtedly will again be other leaks in other output functions. So what seems like the best solution is to run the output functions in a temporary memory context that can be reset after each row, as we're doing in COPY OUT. Some quick experimentation suggests this is actually a tad faster than the retail pfree's anyway. This patch fixes all the variants of printtup, except for debugtup() which is used in standalone mode. It doesn't seem worth worrying about query-lifespan leaks in standalone mode, and fixing that case would be a bit tedious since debugtup() doesn't currently have any startup or shutdown functions. While at it, remove manual detoast management from several other output-function call sites that had copied it from printtup(). This doesn't make a lot of difference right now, but in view of recent discussions about supporting "non-flattened" Datums, we're going to want that code gone eventually anyway. Back-patch to 9.2 where range_out was introduced. We might eventually decide to back-patch this further, but in the absence of known major leaks in older output functions, I'll refrain for now.
2013-11-03 17:33:05 +01:00
/* Return to caller's context, and flush row's temporary memory */
MemoryContextSwitchTo(oldcontext);
MemoryContextReset(myState->tmpcontext);
Stop the executor if no more tuples can be sent from worker to leader. If a Gather node has read as many tuples as it needs (for example, due to Limit) it may detach the queue connecting it to the worker before reading all of the worker's tuples. Rather than let the worker continue to generate and send all of the results, have it stop after sending the next tuple. More could be done here to stop the worker even quicker, but this is about as well as we can hope to do for 9.6. This is in response to a problem report from Andreas Seltenreich. Commit 44339b892a04e94bbb472235882dc6f7023bdc65 should be actually be sufficient to fix that example even without this change, but it seems better to do this, too, since we might otherwise waste quite a large amount of effort in one or more workers. Discussion: CAA4eK1KOKGqmz9bGu+Z42qhRwMbm4R5rfnqsLCNqFs9j14jzEA@mail.gmail.com Amit Kapila
2016-06-06 20:52:58 +02:00
return true;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}