Commit Graph

38917 Commits

Author SHA1 Message Date
Tom Lane 6cd1549235 Stamp 9.4.25. 2019-11-11 17:13:41 -05:00
Tom Lane 84aad60b9c Doc: fix ancient mistake, or at least obsolete info, in rules example.
The example of expansion of multiple views claimed that the resulting
subquery nest would not get fully flattened because of an aggregate
function.  There's no aggregate in the example, though, only a user
defined function confusingly named MIN().  In a modern server, the
reason for the non-flattening is that MIN() is volatile, but I'm
unsure whether that was true back when this text was written.

Let's reduce the confusion level by using LEAST() instead (which
we didn't have at the time this example was created).  And then
we can just say that the planner will flatten the sub-queries, so
the rewrite system doesn't have to.

Noted by Paul Jungwirth.  This text is old enough to vote, so
back-patch to all supported branches.

Discussion: https://postgr.es/m/CA+renyXZFnmp9PcvX1EVR2dR=XG5e6E-AELr8AHCNZ8RYrpnPw@mail.gmail.com
2019-11-11 14:39:55 -05:00
Peter Eisentraut c86e0dc37f Translation updates
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 0b30fe39819568078943f4fb97f99e8c9421194c
2019-11-11 10:20:03 +01:00
Tom Lane 8a2e037a15 Release notes for 12.1, 11.6, 10.11, 9.6.16, 9.5.20, 9.4.25. 2019-11-10 18:31:13 -05:00
Tom Lane 18622caa37 Remove test case that doesn't work with --disable-integer-timestamps.
Commits a7145f6bc et al. added a test to verify integer overflow
detection in interval_mul.  That only applies with integer timestamps,
of course, so it's problematic in pre-v10 branches where we supported
float timestamps.  The test was only marginally worth the trouble to
begin with, so just remove it in those branches.  Per buildfarm.

Discussion: https://postgr.es/m/6437.1573319193@sss.pgh.pa.us
2019-11-09 15:50:16 -05:00
Etsuro Fujita 1bd67d5358 postgres_fdw: Fix error message for PREPARE TRANSACTION.
Currently, postgres_fdw does not support preparing a remote transaction
for two-phase commit even in the case where the remote transaction is
read-only, but the old error message appeared to imply that that was not
supported only if the remote transaction modified remote tables.  Change
the message so as to include the case where the remote transaction is
read-only.

Also fix a comment above the message.

Also add a note about the lack of supporting PREPARE TRANSACTION to the
postgres_fdw documentation.

Reported-by: Gilles Darold
Author: Gilles Darold and Etsuro Fujita
Reviewed-by: Michael Paquier and Kyotaro Horiguchi
Backpatch-through: 9.4
Discussion: https://postgr.es/m/08600ed3-3084-be70-65ba-279ab19618a5%40darold.net
2019-11-08 17:00:38 +09:00
Bruce Momjian 87477b6771 docs: clarify that only INSERT and UPDATE triggers can mod. NEW
The point is that DELETE triggers cannot modify any values.

Reported-by: Eugen Konkov

Discussion: https://postgr.es/m/919823407.20191029175436@yandex.ru

Backpatch-through: 9.4
2019-11-07 15:49:59 -05:00
Tom Lane b20233aac7 Move declaration of ecpg_gettext() to a saner place.
Declaring this in the client-visible header ecpglib.h was a pretty
poor decision.  It's not meant to be application-callable (and if
it was, putting it outside the extern "C" { ... } wrapper means
that C++ clients would fail to call it).  And the declaration would
not even compile for a client, anyway, since it would not have the
macro pg_attribute_format_arg().  Fortunately, it seems that no
clients have tried to include this header with ENABLE_NLS defined,
or we'd have gotten complaints about that.  But we have no business
putting such a restriction on client code.

Move the declaration to ecpglib_extern.h, since in fact nothing
outside src/interfaces/ecpg/ecpglib/ needs to call it.

The practical effect of this is just that clients can now safely
#include ecpglib.h while having ENABLE_NLS defined, but that seems
like enough of a reason to back-patch it.

Discussion: https://postgr.es/m/20590.1573069709@sss.pgh.pa.us
2019-11-07 14:21:52 -05:00
Tom Lane 8d380864a5 Fix integer-overflow edge case detection in interval_mul and pgbench.
This patch adopts the overflow check logic introduced by commit cbdb8b4c0
into two more places.  interval_mul() failed to notice if it computed a
new microseconds value that was one more than INT64_MAX, and pgbench's
double-to-int64 logic had the same sorts of edge-case problems that
cbdb8b4c0 fixed in the core code.

To make this easier to get right in future, put the guts of the checks
into new macros in c.h, and add commentary about how to use the macros
correctly.

Back-patch to all supported branches, as we did with the previous fix.

Yuya Watari

Discussion: https://postgr.es/m/CAJ2pMkbkkFw2hb9Qb1Zj8d06EhWAQXFLy73St4qWv6aX=vqnjw@mail.gmail.com
2019-11-07 11:23:06 -05:00
Fujii Masao 1accf99748 Fix assertion failure when running pgbench -s.
If there is the WAL page that the continuation WAL record just fits within
(i.e., the continuation record ends just at the end of the page) and
the LSN in such page is specified with -s option, previously pg_waldump
caused an assertion failure. The cause of this assertion failure was that
XLogFindNextRecord() that pg_waldump -s calls mistakenly handled
such special WAL page.

This commit changes XLogFindNextRecord() so that it can handle
such WAL page correctly.

Back-patch to all supported versions.

Author: Andrey Lepikhov
Reviewed-by: Fujii Masao, Michael Paquier
Discussion: https://postgr.es/m/99303554-5dd5-06e6-f943-b3005ccd6edd@postgrespro.ru
2019-11-07 16:33:58 +09:00
Michael Paquier 15d90a02a0 Fix timestamp of sent message for write context in logical decoding
When sending data for logical decoding using the streaming replication
protocol via a WAL sender, the timestamp of the sent write message is
allocated at the beginning of the message when preparing for the write,
and actually computed when the write message is ready to be sent.

The timestamp was getting computed after sending the message.  This
impacts anything using logical decoding, causing for example logical
replication to report mostly NULL for last_msg_send_time in
pg_stat_subscription.

This commit makes sure that the timestamp is computed before sending the
message.  This is wrong since 5a991ef, so backpatch down to 9.4.

Author: Jeff Janes
Discussion: https://postgr.es/m/CAMkU=1z=WMn8jt7iEdC5sYNaPgAgOASb_OW5JYv-vMdYaJSL-w@mail.gmail.com
Backpatch-through: 9.4
2019-11-06 16:12:56 +09:00
Bruce Momjian 00ac1eced6 doc: fix plurality typo on bgwriter doc sentence
Reported-by: matthew.alton@gmail.com

Discussion: https://postgr.es/m/157204060717.1042.8194076510523669244@wrigleys.postgresql.org

Backpatch-through: 9.4
2019-11-05 20:54:04 -05:00
Tom Lane 762b256538 Avoid logging complaints about abandoned connections when using PAM.
For a long time (since commit aed378e8d) we have had a policy to log
nothing about a connection if the client disconnects when challenged
for a password.  This is because libpq-using clients will typically
do that, and then come back for a new connection attempt once they've
collected a password from their user, so that logging the abandoned
connection attempt will just result in log spam.  However, this did
not work well for PAM authentication: the bottom-level function
pam_passwd_conv_proc() was on board with it, but we logged messages
at higher levels anyway, for lack of any reporting mechanism.
Add a flag and tweak the logic so that the case is silent, as it is
for other password-using auth mechanisms.

Per complaint from Yoann La Cancellera.  It's been like this for awhile,
so back-patch to all supported branches.

Discussion: https://postgr.es/m/CACP=ajbrFFYUrLyJBLV8=q+eNCapa1xDEyvXhMoYrNphs-xqPw@mail.gmail.com
2019-11-05 14:27:38 -05:00
Alvaro Herrera 9fb25fda6d Change pg_restore -f- to dump to stdout instead of to ./-
Starting with PostgreSQL 12, pg_restore refuses to run when neither -d
nor -f are specified (c.f. commit 413ccaa74d), and it also makes "-f -"
mean the old implicit behavior of dumping to stdout.  However, older
branches write to a file called ./- when invoked like that, making it
impossible to write pg_restore scripts that work across versions.  This
is a partial backpatch of the aforementioned commit to all older
supported branches, providing an upgrade path.

Discussion: https://postgr.es/m/20191006190839.GE18030@telsasoft.com
2019-11-05 09:57:35 -03:00
Tom Lane da5cd7a68a Stabilize pg_dump output order for similarly-named triggers and policies.
The code only compared two triggers' names and namespaces (the latter
being the owning table's schema).  This could result in falling back
to an OID-based sort of similarly-named triggers on different tables.
We prefer to avoid that, so add a comparison of the table names too.
(The sort order is thus table namespace, trigger name, table name,
which is a bit odd, but it doesn't seem worth contorting the code
to work around that.)

Likewise for policy objects, in 9.5 and up.

Complaint and fix by Benjie Gillam.  Back-patch to all supported
branches.

Discussion: https://postgr.es/m/CAMThMzEEt2mvBbPgCaZ1Ap1N-moGn=Edxmadddjq89WG4NpPtQ@mail.gmail.com
2019-11-04 16:25:05 -05:00
Peter Eisentraut 7ab7c5c685 Catch invalid typlens in a couple of places
Rearrange the logic in record_image_cmp() and record_image_eq() to
error out on unexpected typlens (either not supported there or
completely invalid due to corruption).  Barring corruption, this is
not possible today but it seems more future-proof and robust to fix
this.

Reported-by: Peter Geoghegan <pg@bowt.ie>
2019-11-04 10:03:18 +01:00
Michael Paquier f88f7206ed Fix race condition at backend exit when deleting element in syncrep queue
When a backend exits, it gets deleted from the syncrep queue if present.
The queue was checked without SyncRepLock taken in exclusive mode, so it
would have been possible for a backend to remove itself after a WAL
sender already did the job.  Fix this issue based on a suggestion from
Fujii Masao, by first checking the queue without the lock.  Then, if the
backend is present in the queue, take the lock and perform an additional
lookup check before doing the element deletion.

Author: Dongming Liu
Reviewed-by: Kyotaro Horiguchi, Fujii Masao, Michael Paquier
Discussion: https://postgr.es/m/a0806273-8bbb-43b3-bbe1-c45a58f6ae21.lingce.ldm@alibaba-inc.com
Backpatch-through: 9.4
2019-11-01 22:39:09 +09:00
Tom Lane ddcc582a4f Fix pg_dump's handling of circular dependencies in views.
This is a back-patch of the v10 commit d8c05aff5.  The motivation for
doing this now is that we received a complaint that a view with a
circular dependency is dumped with an extra bogus command "ALTER TABLE
ONLY myview REPLICA IDENTITY NOTHING", because pg_dump forgets that it's
a view not a table, and the relreplident value stored for a view is that.
So you'll get an error message during restore even if not using --clean;
this would break pg_upgrade for example.  While that could be handled
with a one-line patch, it seems better to back-patch d8c05aff5, since that
produces cleaner more future-proof output and fixes an additional bug.

Per gripe from Alex Williams.  Back-patch to 9.4-9.6 (even if 9.3 were
still in support, it hasn't got REPLICA IDENTITY so no bug).

Discussion: https://postgr.es/m/NFqxoEi7-8Rw9OW0f-GwHcjvS2I4YQXov4g9OoWv3i7lVOZdLWkAWl9jQQqwEaUq6WV0vdobromhW82e8y5I0_59yZTXcZnXsrmFuldlmZc=@protonmail.com

Original commit message follows:

pg_dump's traditional solution for breaking a circular dependency involving
a view was to create the view with CREATE TABLE and then later issue CREATE
RULE "_RETURN" ... to convert the table to a view, relying on the backend's
very very ancient code that supports making views that way.  We've wanted
to get rid of that kluge for a long time, but the thing that finally
motivates doing something about it is the recognition that this method
fails with the --clean option, because it leads to issuing DROP RULE
"_RETURN" followed by DROP TABLE --- and the backend won't let you drop a
view's _RETURN rule.

Instead, let's break circular dependencies by initially creating the view
using CREATE VIEW AS SELECT NULL::columntype AS columnname, ... (so that
it has the right column names and types to support external references,
but no dependencies beyond the column data types), and then later dumping
the ON SELECT rule using the spelling CREATE OR REPLACE VIEW.  This method
wasn't available when this code was originally written, but it's been
possible since PG 7.3, so it seems fine to start relying on it now.

To solve the --clean problem, make the dropStmt for an ON SELECT rule
be CREATE OR REPLACE VIEW with the same dummy target list as above.
In this way, during the DROP phase, we first reduce the view to have
no extra dependencies, and then we can drop it entirely when we've
gotten rid of whatever had a circular dependency on it.

(Note: this should work adequately well with the --if-exists option, since
the CREATE OR REPLACE VIEW will go through whether the view exists or not.
It could fail if the view exists with a conflicting column set, but we
don't really support --clean against a non-matching database anyway.)

This allows cleaning up some other kluges inside pg_dump, notably that
we don't need a notion of reloptions attached to a rule anymore.

Although this is a bug fix, commit to HEAD only for now.  The problem's
existed for a long time and we've had relatively few complaints, so it
doesn't really seem worth taking risks to fix it in the back branches.
We might revisit that choice if no problems emerge.

Discussion: <19092.1479325184@sss.pgh.pa.us>
2019-10-26 17:37:20 -04:00
Bruce Momjian 7db3651090 docs: fix wording of REFRESH CONCURRENTLY manual page
Reported-by: Asim / apraveen@pivotal.io

Discussion: https://postgr.es/m/157076828181.26165.15231292023740913543@wrigleys.postgresql.org

Backpatch-through: 9.4
2019-10-23 20:29:02 -04:00
Michael Paquier 02f7f246b5 Clean up properly error_context_stack in autovacuum worker on exception
Any callback set would have no meaning in the context of an exception.
As an autovacuum worker exits quickly in this context, this could be
only an issue within EmitErrorReport(), where the elog hook is for
example called.  That's unlikely to going to be a problem, but let's be
clean and consistent with other code paths handling exceptions.  This is
present since 2909419, which introduced autovacuum.

Author: Ashwin Agrawal
Reviewed-by: Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/CALfoeisM+_+dgmAdAOHAu0k-ZpEHHqSSG=GRf3pKJGm8OqWX0w@mail.gmail.com
Backpatch-through: 9.4
2019-10-23 10:26:28 +09:00
Tom Lane fedcab3523 Deal with yet another issue related to "Norwegian (Bokmål)" locale.
It emerges that recent versions of Windows (at least 2016 Standard)
spell this locale name as "Norwegian Bokmål_Norway.1252", defeating
our mapping code that translates "Norwegian (Bokmål)_Norway" to
something that's all-ASCII (cf commits db29620d4 and aa1d2fc5e).
Add another mapping entry to handle this spelling.

Per bug #16068 from Robert Ford.  Like the previous patches,
back-patch to all supported branches.

Discussion: https://postgr.es/m/16068-4cb6eeaa7eb46d93@postgresql.org
2019-10-21 14:19:03 -04:00
Tom Lane 6d2b18d070 Use CFLAGS_SL while probing linkability of libperl.
On recent Red Hat platforms (at least RHEL 8 and Fedora 30, maybe older),
configure's probe for libperl failed if the user forces CFLAGS to be -O0.
This is because some code in perl's inline.h fails to be optimized away
at -O0, and said code doesn't work if compiled without -fPIC.

To fix, add CFLAGS_SL to the compile flags used during the libperl probe.
This is a better simulation of the way that plperl is built, anyway,
so it might forestall other issues in future.

Per gripe from Kyotaro Horiguchi.  Back-patch to all supported branches,
since people might want to build older branches on these platforms.

Discussion: https://postgr.es/m/20191010.144533.263180400.horikyota.ntt@gmail.com
2019-10-21 13:52:26 -04:00
Tom Lane 727c2ccfef Select CFLAGS_SL at configure time, not in platform-specific Makefiles.
Move the platform-dependent logic that sets CFLAGS_SL from
src/makefiles/Makefile.foo to src/template/foo, so that the value
is determined at configure time and thus is available while running
configure's tests.

On a couple of platforms this might save a few microseconds of build
time by eliminating a test that make otherwise has to do over and over.
Otherwise it's pretty much a wash for build purposes; in particular,
this makes no difference to anyone who might be overriding CFLAGS_SL
via a make option.

This patch in itself does nothing with the value and thus should not
change any behavior, though you'll probably have to re-run configure
to get a correctly updated Makefile.global.  We'll use the new
configure variable in a follow-on patch.

Per gripe from Kyotaro Horiguchi.  Back-patch to all supported branches,
because the follow-on patch is a portability bug fix.

Discussion: https://postgr.es/m/20191010.144533.263180400.horikyota.ntt@gmail.com
2019-10-21 12:32:36 -04:00
Noah Misch 930787c7fa For PowerPC instruction "addi", use constraint "b".
Without "b", a variant of the tas() code miscompiles on macOS 10.4.
This may also fix a compilation failure involving macOS 10.1.  Today's
compilers have been allocating acceptable registers with or without this
change, but this future-proofs the code by precisely conveying the
acceptable registers.  Back-patch to 9.4 (all supported versions).

Reviewed by Tom Lane.

Discussion: https://postgr.es/m/20191009063900.GA4066266@rfd.leadboat.com
2019-10-18 20:20:33 -07:00
Fujii Masao 14c59185b2 Fix failure of archive recovery with recovery_min_apply_delay enabled.
recovery_min_apply_delay parameter is intended for use with streaming
replication deployments. However, the document clearly explains that
the parameter will be honored in all cases if it's specified. So it should
take effect even if in archive recovery. But, previously, archive recovery
with recovery_min_apply_delay enabled always failed, and caused assertion
failure if --enable-caasert is enabled.

The cause of this problem is that; the ownership of recoveryWakeupLatch
that recovery_min_apply_delay uses was taken only when standby mode
is requested. So unowned latch could be used in archive recovery, and
which caused the failure.

This commit changes recovery code so that the ownership of
recoveryWakeupLatch is taken even in archive recovery. Which prevents
archive recovery with recovery_min_apply_delay from failing.

Back-patch to v9.4 where recovery_min_apply_delay was added.

Author: Fujii Masao
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/CAHGQGwEyD6HdZLfdWc+95g=VQFPR4zQL4n+yHxQgGEGjaSVheQ@mail.gmail.com
2019-10-18 22:35:52 +09:00
Alvaro Herrera abd5071d21 Fix minor bug in logical-replication walsender shutdown
Logical walsender should exit when it catches up with sending WAL during
shutdown; but there was a rare corner case when it failed to because of
a race condition that puts it back to wait for more WAL instead -- but
since there wasn't any, it'd not shut down immediately.  It would only
continue the shutdown when wal_sender_timeout terminates the sleep,
which causes annoying waits during shutdown procedure.  Restructure the
code so that we no longer forget to set WalSndCaughtUp in that case.

This was an oversight in commit c6c333436.

Backpatch all the way down to 9.4.

Author: Craig Ringer, Álvaro Herrera
Discussion: https://postgr.es/m/CAMsr+YEuz4XwZX_QmnX_-2530XhyAmnK=zCmicEnq1vLr0aZ-g@mail.gmail.com
2019-10-17 15:06:05 +02:00
Thomas Munro 080cf32d22 Fix bug that could try to freeze running multixacts.
Commits 801c2dc7 and 801c2dc7 made it possible for vacuum to
try to freeze a multixact that is still running.  That was
prevented by a check, but raised an error.  Repair.

Back-patch all the way.

Author: Nathan Bossart, Jeremy Schneider
Reported-by: Jeremy Schneider
Reviewed-by: Jim Nasby, Thomas Munro
Discussion: https://postgr.es/m/DAFB8AFF-2F05-4E33-AD7F-FF8B0F760C17%40amazon.com
2019-10-17 10:14:51 +13:00
Tomas Vondra bc3a94dc00 Add missing include to pg_upgrade/version.c
Commit 8d48e6a724 uses RELKIND_ constants when building the query, but
did not include the header defining them. On 10+ this header is already
included, but on 9.6 and earlier it was missing. It compiles just fine,
but then fails during execution

  ERROR:  column "relkind_relation" does not exist

Fix by adding the necessary header file, and backpatch to 9.4-.

Backpatch-to: 9.4-
Discussion: https://postgr.es/m/16045-673e8fa6b5ace196%40postgresql.org
2019-10-16 16:28:48 +02:00
Tomas Vondra 235a52ca0f Improve the check for pg_catalog.line data type in pg_upgrade
The pg_upgrade check for pg_catalog.line data type when upgrading from
9.3 had a couple of issues with domains and composite types. Firstly, it
triggered false positives for composite types unused in objects with
storage. This was enough to trigger an unnecessary pg_upgrade failure:

  CREATE TYPE line_composite AS (l pg_catalog.line)

On the other hand, this only happened with composite types directly on
the pg_catalog.line data type, but not with a domain. So this was not
detected

  CREATE DOMAIN line_domain AS pg_catalog.line;
  CREATE TYPE line_composite_2 AS (l line_domain);

unlike the first example. These false positives and inconsistencies are
unfortunate, but what's worse we've failed to detected objects using the
pg_catalog.line data type through a domain. So we missed cases like this

  CREATE TABLE t (l line_composite_2);

The consequence is clusters broken after a pg_upgrade.

This fixes these false positives and false negatives by using the same
recursive CTE introduced by eaf900e842 for sql_identifier. 9.3 did not
support domains on composite types, but we can still have multi-level
composite types.

Backpatch all the way to 9.4, where the format for pg_catalog.line data
type changed.

Author: Tomas Vondra
Backpatch-to: 9.4-
Discussion: https://postgr.es/m/16045-673e8fa6b5ace196%40postgresql.org
2019-10-16 13:31:00 +02:00
Michael Paquier 952af2476f Doc: Fix various inconsistencies
This fixes multiple areas of the documentation:
- COPY for its past compatibility section.
- SET ROLE mentioning INHERITS instead of INHERIT
- PREPARE referring to stmt_name, that is not present.
- Extension documentation about format name with upgrade scripts.

Backpatch down to 9.4 for the relevant parts.

Author: Alexander Lakhin
Discussion: https://postgr.es/m/bf95233a-9943-b341-e2ff-a860c28af481@gmail.com
Backpatch-through: 9.4
2019-10-16 13:11:01 +09:00
Noah Misch b705582b48 AIX: Stop adding option -qsrcmsg.
With xlc v16.1.0, it causes internal compiler errors.  With xlc versions
not exhibiting that bug, removing -qsrcmsg merely changes the compiler
error reporting format.  Back-patch to 9.4 (all supported versions).

Discussion: https://postgr.es/m/20191003064105.GA3955242@rfd.leadboat.com
2019-10-12 00:21:51 -07:00
Michael Paquier 59800f7cef Flush logical mapping files with fd opened for read/write at checkpoint
The file descriptor was opened with read-only to fsync a regular file,
which would cause EBADFD errors on some platforms.

This is similar to the recent fix done by a586cc4b (which was broken by
me with 82a5649), except that I noticed this issue while monitoring the
backend code for similar mistakes.  Backpatch to 9.4, as this has been
introduced since logical decoding exists as of b89e151.

Author: Michael Paquier
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/20191006045548.GA14532@paquier.xyz
Backpatch-through: 9.4
2019-10-09 13:31:38 +09:00
Bruce Momjian 7ae8a9959c docs: Improve A?synchronous Multimaster Replication descr.
The docs for sync and async multimaster replication were unclear about
when to use it, and when it has benefits;  this change clarifies that.

Reported-by: juha-pekka.eloranta@reaktor.fi

Discussion: https://postgr.es/m/156856543824.1274.12180817186798859836@wrigleys.postgresql.org

Backpatch-through: 9.4
2019-10-07 18:06:07 -04:00
Bruce Momjian 630a8f5e2d docs: clarify that today/tomorrow/yesterday is at 00:00
This should help people clearly know that these days start at midnight.

Reported-by: David Harper

Discussion: https://postgr.es/m/156258047907.1181.11324468080514061996@wrigleys.postgresql.org

Backpatch-through: 9.4
2019-10-07 17:26:46 -04:00
Bruce Momjian 873cd4f606 doc: move mention of log_min_error_statement in a better spot
Previously it was mentioned in the lock_timeout docs in a confusing
location.

Reported-by: ivaylo.zlatanov@gmail.com

Discussion: https://postgr.es/m/157019615723.25307.15449102262106437404@wrigleys.postgresql.org

Backpatch-through: 9.4
2019-10-07 14:33:31 -04:00
Andrew Dunstan 8f88090914 Handle spaces in OpenSSL install location for MSVC
First, make sure that the .exe name is quoted when trying to get the
version number. Also, don't quote the lib name for using in the project
files if it's already been quoted. This second change applies to all
libraries, not just OpenSSL.

This has clearly been broken forever, so backpatch to all live branches.
2019-10-04 15:38:36 -04:00
Tom Lane b6a6c129fc Fix bitshiftright()'s zero-padding some more.
Commit 5ac0d9360 failed to entirely fix bitshiftright's habit of
leaving one-bits in the pad space that should be all zeroes,
because in a moment of sheer brain fade I'd concluded that only
the code path used for not-a-multiple-of-8 shift distances needed
to be fixed.  Of course, a multiple-of-8 shift distance can also
cause the problem, so we need to forcibly zero the extra bits
in both cases.

Per bug #16037 from Alexander Lakhin.  As before, back-patch to all
supported branches.

Discussion: https://postgr.es/m/16037-1d1ebca564db54f4@postgresql.org
2019-10-04 10:34:21 -04:00
Tom Lane 6899be2890 Avoid unnecessary out-of-memory errors during encoding conversion.
Encoding conversion uses the very simplistic rule that the output
can't be more than 4X longer than the input, and palloc's a buffer
of that size.  This results in failure to convert any string longer
than 1/4 GB, which is becoming an annoying limitation.

As a band-aid to improve matters, allow the allocated output buffer
size to exceed 1GB.  We still insist that the final result fit into
MaxAllocSize (1GB), though.  Perhaps it'd be safe to relax that
restriction, but it'd require close analysis of all callers, which
is daunting (not least because external modules might call these
functions).  For the moment, this should allow a 2X to 4X improvement
in the longest string we can convert, which is a useful gain in
return for quite a simple patch.

Also, once we have successfully converted a long string, repalloc
the output down to the actual string length, returning the excess
to the malloc pool.  This seems worth doing since we can usually
expect to give back several MB if we take this path at all.

This still leaves much to be desired, most notably that the assumption
that MAX_CONVERSION_GROWTH == 4 is very fragile, and yet we have no
guard code verifying that the output buffer isn't overrun.  Fixing
that would require significant changes in the encoding conversion
APIs, so it'll have to wait for some other day.

The present patch seems safely back-patchable, so patch all supported
branches.

Alvaro Herrera and Tom Lane

Discussion: https://postgr.es/m/20190816181418.GA898@alvherre.pgsql
Discussion: https://postgr.es/m/3614.1569359690@sss.pgh.pa.us
2019-10-03 17:34:26 -04:00
Tom Lane 4829576ba4 Allow repalloc() to give back space when a large chunk is downsized.
Up to now, if you resized a large (>8K) palloc chunk down to a smaller
size, aset.c made no attempt to return any space to the malloc pool.
That's unpleasant if a really large allocation is resized to a
significantly smaller size.  I think no such cases existed when this
code was designed, and I'm not sure whether they're common even yet,
but an upcoming fix to encoding conversion will certainly create such
cases.  Therefore, fix AllocSetRealloc so that it gives realloc()
a chance to do something with the block.  This doesn't noticeably
increase complexity, we mostly just have to change the order in which
the cases are considered.

Back-patch to all supported branches.

Discussion: https://postgr.es/m/20190816181418.GA898@alvherre.pgsql
Discussion: https://postgr.es/m/3614.1569359690@sss.pgh.pa.us
2019-10-03 13:56:27 -04:00
Andrew Gierth 3473f81dd2 Selectively include window frames in expression walks/mutates.
query_tree_walker and query_tree_mutator were skipping the
windowClause of the query, without regard for the fact that the
startOffset and endOffset in a WindowClause node are expression trees
that need to be processed. This was an oversight in commit ec4be2ee6
from 2010 which added the expression fields; the main symptom is that
function parameters in window frame clauses don't work in inlined
functions.

Fix (as conservatively as possible since this needs to not break
existing out-of-tree callers) and add tests.

Backpatch all the way, since this has been broken since 9.0.

Per report from Alastair McKinley; fix by me with kibitzing and review
from Tom Lane.

Discussion: https://postgr.es/m/DB6PR0202MB2904E7FDDA9D81504D1E8C68E3800@DB6PR0202MB2904.eurprd02.prod.outlook.com
2019-10-03 11:18:20 +01:00
Amit Kapila 668918ff13 Fix oversight in commit 4429f6a9e3.
The test name and the following test cases suggest the index created
should be hash index, but it forgot to add 'using hash' in the test case.
This in itself won't improve code coverage as there were some other tests
which were covering the corresponding code.  However, it is better if the
added tests serve their actual purpose.

Reported-by: Paul A Jungwirth
Author: Paul A Jungwirth
Reviewed-by: Mahendra Singh
Backpatch-through: 9.4
Discussion: https://postgr.es/m/CA+renyV=Us-5XfMC25bNp-uWSj39XgHHmGE9Rh2cQKMegSj52g@mail.gmail.com
2019-09-27 09:09:35 +05:30
Tom Lane 70c1245064 Doc: clarify handling of duplicate elements in array containment tests.
The array <@ and @> operators do not worry about duplicates: if every
member of array X matches some element of array Y, then X is contained
in Y, even if several members of X get matched to the same Y member.
This was not explicitly stated in the docs though, so improve matters.

Discussion: https://postgr.es/m/156614120484.1310.310161642239149585@wrigleys.postgresql.org
2019-09-23 12:37:04 -04:00
Tom Lane 8a17afe84b Fix failure to zero-pad the result of bitshiftright().
If the bitstring length is not a multiple of 8, we'd shift the
rightmost bits into the pad space, which must be zeroes --- bit_cmp,
for one, depends on that.  This'd lead to the result failing to
compare equal to what it should compare equal to, as reported in
bug #16013 from Daryl Waycott.

This is, if memory serves, not the first such bug in the bitstring
functions.  In hopes of making it the last one, do a bit more work
than minimally necessary to fix the bug:

* Add assertion checks to bit_out() and varbit_out() to complain if
they are given incorrectly-padded input.  This will improve the
odds that manual testing of any new patch finds problems.

* Encapsulate the padding-related logic in macros to make it
easier to use.

Also, remove unnecessary padding logic from bit_or() and bitxor().
Somebody had already noted that we need not re-pad the result of
bit_and() since the inputs are required to be the same length,
but failed to extrapolate that to the other two.

Also, move a comment block that once was near the head of varbit.c
(but people kept putting other stuff in front of it), to put it in
the header block.

Note for the release notes: if anyone has inconsistent data as a
result of saving the output of bitshiftright() in a table, it's
possible to fix it with something like
UPDATE mytab SET bitcol = ~(~bitcol) WHERE bitcol != ~(~bitcol);

This has been broken since day one, so back-patch to all supported
branches.

Discussion: https://postgr.es/m/16013-c2765b6996aacae9@postgresql.org
2019-09-22 17:46:00 -04:00
Tom Lane 29e47f83ca Update time zone data files to tzdata release 2019c.
DST law changes in Fiji and Norfolk Island.  Historical corrections
for Alberta, Austria, Belgium, British Columbia, Cambodia, Hong Kong,
Indiana (Perry County), Kaliningrad, Kentucky, Michigan, Norfolk
Island, South Korea, and Turkey.
2019-09-20 19:54:36 -04:00
Amit Kapila c16eb2d530 Fix typo in commit 82fa3ff867.
Reported-By: Kuntal Ghosh (off-list)
Backpatch-through: 9.4, like 82fa3ff867
2019-09-20 08:34:12 +05:30
Alexander Korotkov 332eda5bd3 Improve handling of NULLs in KNN-GiST and KNN-SP-GiST
This commit improves subject in two ways:

 * It removes ugliness of 02f90879e7, which stores distance values and null
   flags in two separate arrays after GISTSearchItem struct.  Instead we pack
   both distance value and null flag in IndexOrderByDistance struct.  Alignment
   overhead should be negligible, because we typically deal with at most few
   "col op const" expressions in ORDER BY clause.
 * It fixes handling of "col op NULL" expression in KNN-SP-GiST.  Now, these
   expression are not passed to support functions, which can't deal with them.
   Instead, NULL result is implicitly assumed.  It future we may decide to
   teach support functions to deal with NULL arguments, but current solution is
   bugfix suitable for backpatch.

Reported-by: Nikita Glukhov
Discussion: https://postgr.es/m/826f57ee-afc7-8977-c44c-6111d18b02ec%40postgrespro.ru
Author: Nikita Glukhov
Reviewed-by: Alexander Korotkov
Backpatch-through: 9.4
2019-09-19 22:10:46 +03:00
Michael Paquier 27b4121311 Doc: Fix incorrect mention to connection_object in CONNECT command of ECPG
This fixes an inconsistency with this parameter name not listed in the
command synopsis, and connection_name is the parameter name more
commonly used in the docs for ECPG commands.

Reported-by: Yusuke Egashita
Discussion: https://postgr.es/m/156870956796.1259.11456186889345212399@wrigleys.postgresql.org
Backpatch-through: 9.4
2019-09-19 13:19:23 +09:00
Amit Kapila 3698c4480b Doc: document autovacuum interruption.
It's important users be able to know (without looking at the source code)
that running DDL or DDL-like commands can interrupt autovacuum which can
lead to a lot of dead tuples and hence slower database operations.

Reported-by: James Coleman
Author: James Coleman
Reviewed-by: Amit Kapila
Backpatch-through: 9.4
Discussion:https://postgr.es/m/CAAaqYe-XYyNwML1=f=gnd0qWg46PnvD=BDrCZ5-L94B887XVxQ@mail.gmail.com
2019-09-19 09:26:33 +05:30
Alvaro Herrera e8c7f40a1f logical decoding: process ASSIGNMENT during snapshot build
Most WAL records are ignored in early SnapBuild snapshot build phases.
But it's critical to process some of them, so that later messages have
the correct transaction state after the snapshot is completely built; in
particular, XLOG_XACT_ASSIGNMENT messages are critical in order for
sub-transactions to be correctly assigned to their parent transactions,
or at least one assert misbehaves, as reported by Ildar Musin.

Diagnosed-by: Masahiko Sawada
Author: Masahiko Sawada
Discussion: https://postgr.es/m/CAONYFtOv+Er1p3WAuwUsy1zsCFrSYvpHLhapC_fMD-zNaRWxYg@mail.gmail.com
2019-09-13 16:36:28 -03:00
Tom Lane ca08ea52bf Fix usage of whole-row variables in WCO and RLS policy expressions.
Since WITH CHECK OPTION was introduced, ExecInitModifyTable has
initialized WCO expressions with the wrong plan node as parent -- that is,
it passed its input subplan not the ModifyTable node itself.  Up to now
we thought this was harmless, but bug #16006 from Vinay Banakar shows it's
not: if the input node is a SubqueryScan then ExecInitWholeRowVar can get
confused into doing the wrong thing.  (The fact that ExecInitWholeRowVar
contains such logic is certainly a horrid kluge that doesn't deserve to
live, but figuring out another way to do that is a task for some other day.)

Andres had already noticed the wrong-parent mistake and fixed it in commit
148e632c0, but not being aware of any user-visible consequences, he quite
reasonably didn't back-patch.  This patch is simply a back-patch of
148e632c0, plus addition of a test case based on bug #16006.  I also added
the test case to v12/HEAD, even though the bug is already fixed there.

Back-patch to all supported branches.  9.4 lacks RLS policies so the
new test case doesn't work there, but I'm pretty sure a test could be
devised based on using a whole-row Var in a plain WITH CHECK OPTION
condition.  (I lack the cycles to do so myself, though.)

Andres Freund and Tom Lane

Discussion: https://postgr.es/m/16006-99290d2e4642cbd5@postgresql.org
Discussion: https://postgr.es/m/20181205225213.hiwa3kgoxeybqcqv@alap3.anarazel.de
2019-09-12 18:29:18 -04:00