Commit Graph

37114 Commits

Author SHA1 Message Date
Andrew Dunstan 524440579f Keep perl style checker happy
It doesn't like code before "use strict;".

(cherry picked from commit ed2c2d063b)
2023-01-19 11:53:19 -05:00
Andrew Dunstan 96feeab4b3 Fix searchpath for modern Perl for genbki.pl
This was fixed for MSVC tools by commit 1df92eeafe, but per
buildfarm member bowerbird genbki.pl needs the same treatment.

Backpatch to all live branches.

(cherry picked from commit 51884fa16a)
2023-01-19 11:53:19 -05:00
Tom Lane 2e8bedc3d6 Create common infrastructure for cross-version upgrade testing.
To test pg_upgrade across major PG versions, we have to be able to
modify or drop any old objects with no-longer-supported properties,
and we have to be able to deal with cosmetic changes in pg_dump output.
Up to now, the buildfarm and pg_upgrade's own test infrastructure had
separate implementations of the former, and we had nothing but very
ad-hoc rules for the latter (including an arbitrary threshold on how
many lines of unchecked diff were okay!).  This patch creates a Perl
module that can be shared by both those use-cases, and adds logic
that deals with pg_dump output diffs in a much more tightly defined
fashion.

This largely supersedes previous efforts in commits 0df9641d3,
9814ff550, and 62be9e4cd, which developed a SQL-script-based solution
for the task of dropping old objects.  There was nothing fundamentally
wrong with that work in itself, but it had no basis for solving the
output-formatting problem.  The most plausible way to deal with
formatting is to build a Perl module that can perform editing on the
dump files; and once we commit to that, it makes more sense for the
same module to also embed the knowledge of what has to be done for
dropping old objects.

Back-patch versions of the helper module as far as 9.2, to
support buildfarm animals that still test that far back.
It's also necessary to back-patch PostgreSQL/Version.pm,
because the new code depends on that.  I fixed up pg_upgrade's
002_pg_upgrade.pl in v15, but did not look into back-patching
it further than that.

Tom Lane and Andrew Dunstan

Discussion: https://postgr.es/m/891521.1673657296@sss.pgh.pa.us
2023-01-16 20:35:53 -05:00
Michael Paquier a5171b8800 Fix compilation warnings with libselinux 3.1 in contrib/sepgsql/
Upstream SELinux has recently marked security_context_t as officially
deprecated, causing warnings with -Wdeprecated-declarations.  This is
considered as legacy code for some time now by upstream as
security_context_t got removed from most of the code tree during the
development of 2.3 back in 2014.

This removes all the references to security_context_t in sepgsql/ to be
consistent with SELinux, fixing the warnings.  Note that this does not
impact the minimum version of libselinux supported.

This has been applied first as 1f32136 for 14~, but no other branches
got the call.  This is in line with the recent project policy to have no
warnings in branches where builds should still be supported (9.2~ as of
today).  Per discussion with Tom Lane and Álvaro Herrera.

Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/20200813012735.GC11663@paquier.xyz
Discussion: https://postgr.es/m/20221103181028.raqta27jcuypor4l@alvherre.pgsql
Backpatch-through: 9.2
2022-11-09 09:40:06 +09:00
Tom Lane ee4c440144 Rename parser token REF to REF_P to avoid a symbol conflict.
In the latest version of Apple's macOS SDK, <sys/socket.h>
fails to compile if "REF" is #define'd as something.
Apple may or may not agree that this is a bug, and even if
they do accept the bug report I filed, they probably won't
fix it very quickly.  In the meantime, our back branches will all
fail to compile gram.y.  v15 and HEAD currently escape the problem
thanks to the refactoring done in 98e93a1fc, but that's purely
accidental.  Moreover, since that patch removed a widely-visible
inclusion of <netdb.h>, back-patching it seems too likely to break
third-party code.

Instead, change the token's code name to REF_P, following our usual
convention for naming parser tokens that are likely to have symbol
conflicts.  The effects of that should be localized to the grammar
and immediately surrounding files, so it seems like a safer answer.

Per project policy that we want to keep recently-out-of-support
branches buildable on modern systems, back-patch all the way to 9.2.

Discussion: https://postgr.es/m/1803927.1665938411@sss.pgh.pa.us
2022-10-16 15:27:04 -04:00
Tom Lane f6ce9cee62 Fix configure's AC_CHECK_DECLS tests to work correctly with clang.
Back-port commit 16fbac39f into 9.3 and 9.2, so that these
out-of-support branches can be built with clang without wading
through a pile of warnings about strlcpy and related functions.
check_decls.m4 required some adaptation to work with autoconf 2.63,
but nothing too major.

Discussion: https://postgr.es/m/26819.1542515567@sss.pgh.pa.us
Discussion: https://postgr.es/m/1081321.1663775084@sss.pgh.pa.us
2022-09-21 15:42:00 -04:00
Tom Lane bf21612344 Suppress more variable-set-but-not-used warnings from clang 15.
Mop up assorted set-but-not-used warnings in the back branches.
This includes back-patching relevant fixes from commit 152c9f7b8
the rest of the way, but there are also several cases that did not
appear in HEAD.  Some of those we'd fixed in a retail way but not
back-patched, and others I think just got rewritten out of existence
during nearby refactoring.

While here, also back-patch b1980f6d0 (PL/Tcl: Fix compiler warnings
with Tcl 8.6) into 9.2, so that that branch compiles warning-free
with modern Tcl.

Per project policy, this is a candidate for back-patching into
out-of-support branches: it suppresses annoying compiler warnings
but changes no behavior.  Hence, back-patch all the way to 9.2.

Discussion: https://postgr.es/m/514615.1663615243@sss.pgh.pa.us
2022-09-21 13:52:38 -04:00
Tom Lane 8c8ee5c991 Disable -Wdeprecated-non-prototype in the back branches.
There doesn't seem to be any good ABI-preserving way to silence
clang 15's -Wdeprecated-non-prototype warnings about our tree-walk
APIs.  While we've fixed it properly in HEAD, the only way to not
see hundreds of these in the back branches is to disable the
warnings.  We're not going to do anything about them, so we might
as well disable them.

I noticed that we also get some of these warnings about fmgr.c's
support for V0 function call convention, in branches before v10
where we removed that.  That's another area we aren't going to
change, so turning off the warning seems fine for that too.

Per project policy, this is a candidate for back-patching into
out-of-support branches: it suppresses annoying compiler warnings
but changes no behavior.  Hence, back-patch all the way to 9.2.

Discussion: https://postgr.es/m/CA+hUKGKpHPDTv67Y+s6yiC8KH5OXeDg6a-twWo_xznKTcG0kSA@mail.gmail.com
2022-09-20 18:59:54 -04:00
Tom Lane a3b685fdd1 Fix pl/perl test case so it will still work under Perl 5.36.
Perl 5.36 has reclassified the warning condition that this test
case used, so that the expected error fails to appear.  Tweak
the test so it instead exercises a case that's handled the same
way in all Perl versions of interest.

This appears to meet our standards for back-patching into
out-of-support branches: it changes no user-visible behavior
but enables testing of old branches with newer tools.
Hence, back-patch as far as 9.2.

Dagfinn Ilmari Mannsåker, per report from Jitka Plesníková.

Discussion: https://postgr.es/m/564579.1654093326@sss.pgh.pa.us
2022-06-01 16:15:47 -04:00
Tom Lane f70550327b Suppress variable-set-but-not-used warning from clang 13.
In the normal configuration where GEQO_DEBUG isn't defined,
recent clang versions have started to complain that geqo_main.c
accumulates the edge_failures count but never does anything
with it.  As a minimal back-patchable fix, insert a void cast
to silence this warning.  (I'd speculated about ripping out the
GEQO_DEBUG logic altogether, but I don't think we'd wish to
back-patch that.)

Per recently-established project policy, this is a candidate
for back-patching into out-of-support branches: it suppresses
an annoying compiler warning but changes no behavior.  Hence,
back-patch all the way to 9.2.

Discussion: https://postgr.es/m/CA+hUKGLTSZQwES8VNPmWO9AO0wSeLt36OCPDAZTccT1h7Q7kTQ@mail.gmail.com
2022-01-23 11:10:10 -05:00
Andrew Dunstan 020bfc8419
Make DLSUFFIX easily discoverable by build scripts
This will enable things like the buildfarm client to discover more
reliably if certain libraries have been installed.

Discussion: https://postgr.es/m/859e7c91-7ef4-d4b4-2ca2-8046e0cbee09@2ndQuadrant.com

This is a backpatch to 9.3 and 9.2 of commits b40cb99b85 and d9cdb1ba9e
now that we are supporting limited builds of older versions.
2021-12-15 17:57:10 -05:00
Tom Lane fcf1f27fe8 Fix compiler warning
With some newer gcc versions (8 and 9) you get a -Wformat-overflow
warning here.  In PG11 and later this was already fixed.  Since it's
trivial, backport it to get the older branches building without
warnings.

This back-patches commit 333a186dc into out-of-support branches,
pursuant to newly-established project policy.  The point is to
suppress scary-looking warnings so that people building these
branches needn't expend brain cells verifying that it's safe
to ignore the warnings.

Discussion: https://postgr.es/m/d0316012-ece7-7b7e-2d36-9c38cb77cb3b@enterprisedb.com
2021-12-13 00:24:38 -05:00
Tom Lane a1fe82d987 Silence another gcc 11 warning.
Per buildfarm and local experimentation, bleeding-edge gcc isn't
convinced that the MemSet in reorder_function_arguments() is safe.
Shut it up by adding an explicit check that pronargs isn't negative,
and by changing MemSet to memset.  (It appears that either change is
enough to quiet the warning at -O2, but let's do both to be sure.)

This back-patches commit 1046dbedd into out-of-support branches,
pursuant to newly-established project policy.  The point is to
suppress scary-looking warnings so that people building these
branches needn't expend brain cells verifying that it's safe
to ignore the warnings.

Discussion: https://postgr.es/m/d0316012-ece7-7b7e-2d36-9c38cb77cb3b@enterprisedb.com
2021-12-13 00:08:33 -05:00
Tom Lane 6eb7efdf0f Suppress -Warray-parameter warnings in pgcrypto/sha2.c.
This is exactly the same problem as commit 1b242f42b fixed in ecpg,
but in contrib/pgcrypto.  Commit 273c458a2 eliminated the problem
here for v10 and up.  We hadn't noticed for exactly the same reasons
enumerated in bbbf22cf3.

Back-patch down to 9.2, pursuant to newly-established project policy
about keeping out-of-support branches buildable.

Discussion: https://postgr.es/m/d0316012-ece7-7b7e-2d36-9c38cb77cb3b@enterprisedb.com
2021-12-12 20:24:50 -05:00
Tom Lane 9610c71584 Reformat imath.c macro to remove -Wmisleading-indentation warnings.
Recent versions of gcc whine about the admittedly-completely-illegible
formatting of this macro.  We've not noticed for a few reasons:

* In v12 and up, the problem is gone thanks to 48e24ba6b.
(Back-patching that doesn't seem prudent, though, so this patch
just manually improves the macro's formatting.)

* Buildfarm animals that might have complained, such as caiman,
do not because they use --with-openssl and so don't build imath.c.

* In a manual run such as "make all check-world", you won't see the
warning because it gets buried in an install.log file.  You have to
do "make -C contrib all" or the like to see it.

I noticed this because in older branches, the last bit doesn't
happen so "check-world" actually does spew the warnings to stderr.
Maybe we should rethink how that works, because the newer behavior
is not an improvement IMO.

Back-patch down to 9.2, pursuant to newly-established project policy
about keeping out-of-support branches buildable.

Discussion: https://postgr.es/m/d0316012-ece7-7b7e-2d36-9c38cb77cb3b@enterprisedb.com
2021-12-12 19:12:23 -05:00
Tom Lane 580261836c Clean up compilation warnings coming from PL/Perl with clang-12~
clang-12 has introduced -Wcompound-token-split-by-macro, that is causing
a large amount of warnings when building PL/Perl because of its
interactions with upstream Perl.  This commit adds one -Wno to CFLAGS at
./configure time if the flag is supported by the compiler to silence all
those warnings.

This back-patches commit 9ff47ea41 into out-of-support branches,
pursuant to newly-established project policy.  The point is to
suppress scary-looking warnings so that people building these
branches needn't expend brain cells verifying that it's safe
to ignore the warnings.

Discussion: https://postgr.es/m/d0316012-ece7-7b7e-2d36-9c38cb77cb3b@enterprisedb.com
2021-12-12 18:06:11 -05:00
Tom Lane b30e23fe58 Make ecpg's rjulmdy() and rmdyjul() agree with their declarations.
We had "short *mdy" in the extern declarations, but "short mdy[3]"
in the actual function definitions.  Per C99 these are equivalent,
but recent versions of gcc have started to issue warnings about
the inconsistency.  Clean it up before the warnings get any more
widespread.

This back-patches commit 1b242f42b into out-of-support branches,
pursuant to newly-established project policy.  The point is to
suppress scary-looking warnings so that people building these
branches needn't expend brain cells verifying that it's safe
to ignore the warnings.

Discussion: https://postgr.es/m/d0316012-ece7-7b7e-2d36-9c38cb77cb3b@enterprisedb.com
2021-12-12 16:59:17 -05:00
Tom Lane 119b9d060e Make pg_upgrade's test.sh less chatty.
Remove "set -x", and pass "-A trust" to initdb explicitly,
to suppress almost all of the noise this script used to emit
on stderr.

This back-patches commit eb9812f27 into out-of-support branches,
pursuant to newly-established project policy.  The point is to
suppress useless noise on stderr when running check-world.

Discussion: https://postgr.es/m/d0316012-ece7-7b7e-2d36-9c38cb77cb3b@enterprisedb.com
2021-12-12 16:14:22 -05:00
Tom Lane 404d987646 Add checks for valid multibyte character length in UtfToLocal, LocalToUtf.
This back-patches commit d9f37e666 into out-of-support branches,
pursuant to newly-established project policy.  The point is to
suppress "uninitialized variable" warnings so that people building
these branches needn't expend brain cells verifying that it's safe
to ignore the warnings.

Discussion: https://postgr.es/m/d0316012-ece7-7b7e-2d36-9c38cb77cb3b@enterprisedb.com
2021-12-12 15:58:57 -05:00
Tom Lane f24374bce6 Use return instead of exit() in configure
Using exit() requires stdlib.h, which is not included.  Use return
instead.  Also add return type for main().

This back-patches commit 1c0cf52b3 into out-of-support branches,
pursuant to a newly-established project policy that we'll try to keep
out-of-support branches buildable on modern platforms for at least
ten major releases back, ensuring people can test pg_dump and psql
compatibility against servers that far back.  With the current
development branch being v15, that works out to keeping 9.2 and up
buildable as of today.

This fix is needed to get through 'configure' when using recent
macOS (and possibly other clang-based toolchains).  It seems to
be sufficient to get through 'check-world', although there are
annoyances such as compiler warnings, which will be dealt with
separately.

Original patch by Peter Eisentraut

Discussion: https://postgr.es/m/d0316012-ece7-7b7e-2d36-9c38cb77cb3b@enterprisedb.com
2021-12-12 14:54:32 -05:00
Tom Lane c10bb239d2 Stamp 9.3.25. 2018-11-05 16:53:28 -05:00
Peter Eisentraut 1394a48db9 Translation updates
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 8e48d753b142594b95b0e149ce0cd5b3317a42cc
2018-11-05 15:13:49 +01:00
Tom Lane 89c563f1bb Release notes for 11.1, 10.6, 9.6.11, 9.5.15, 9.4.20, 9.3.25. 2018-11-04 16:57:15 -05:00
Tom Lane 33c697e9d4 Make ts_locale.c's character-type functions cope with UTF-16.
On Windows, in UTF8 database encoding, what char2wchar() produces is
UTF16 not UTF32, ie, characters above U+FFFF will be represented by
surrogate pairs.  t_isdigit() and siblings did not account for this
and failed to provide a large enough result buffer.  That in turn
led to bogus "invalid multibyte character for locale" errors, because
contrary to what you might think from char2wchar()'s documentation,
its Windows code path doesn't cope sanely with buffer overflow.

The solution for t_isdigit() and siblings is pretty clear: provide
a 3-wchar_t result buffer not 2.

char2wchar() also needs some work to provide more consistent, and more
accurately documented, buffer overrun behavior.  But that's a bigger job
and it doesn't actually have any immediate payoff, so leave it for later.

Per bug #15476 from Kenji Uno, who deserves credit for identifying the
cause of the problem.  Back-patch to all active branches.

Discussion: https://postgr.es/m/15476-4314f480acf0f114@postgresql.org
2018-11-03 13:56:10 -04:00
Tom Lane 1aad3a724a Yet further rethinking of build changes for macOS Mojave.
The solution arrived at in commit e74dd00f5 presumes that the compiler
has a suitable default -isysroot setting ... but further experience
shows that in many combinations of macOS version, XCode version, Xcode
command line tools version, and phase of the moon, Apple's compiler
will *not* supply a default -isysroot value.

We could potentially go back to the approach used in commit 68fc227dd,
but I don't have a lot of faith in the reliability or life expectancy of
that either.  Let's just revert to the approach already shipped in 11.0,
namely specifying an -isysroot switch globally.  As a partial response to
the concerns raised by Jakob Egger, adjust the contents of Makefile.global
to look like

CPPFLAGS = -isysroot $(PG_SYSROOT) ...
PG_SYSROOT = /path/to/sysroot

This allows overriding the sysroot path at build time in a relatively
painless way.

Add documentation to installation.sgml about how to use the PG_SYSROOT
option.  I also took the opportunity to document how to work around
macOS's "System Integrity Protection" feature.

As before, back-patch to all supported versions.

Discussion: https://postgr.es/m/20840.1537850987@sss.pgh.pa.us
2018-11-02 18:54:00 -04:00
Bruce Momjian 471ae731c1 docs: adjust simpler language for NULL return from ANY/ALL
Adjustment to commit 8610c973dd.

Reported-by: Tom Lane

Discussion: https://postgr.es/m/17406.1541168421@sss.pgh.pa.us

Backpatch-through: 9.3
2018-11-02 13:05:30 -04:00
Bruce Momjian 7fc0c4c9ec GUC: adjust effective_cache_size docs and SQL description
Clarify that effective_cache_size is both kernel buffers and shared
buffers.

Reported-by: nat@makarevitch.org

Discussion: https://postgr.es/m/153685164808.22334.15432535018443165207@wrigleys.postgresql.org

Backpatch-through: 9.3
2018-11-02 09:10:59 -04:00
Bruce Momjian f499dabbd5 doc: use simpler language for NULL return from ANY/ALL
Previously the combination of "does not return" and "any row" caused
ambiguity.

Reported-by: KES <kes-kes@yandex.ru>

Discussion: https://postgr.es/m/153701242703.22334.1476830122267077397@wrigleys.postgresql.org

Reviewed-by: David G. Johnston

Backpatch-through: 9.3
2018-11-02 08:54:33 -04:00
Tom Lane 82dd1c2714 Fix memory leak in repeated SPGIST index scans.
spgendscan neglected to pfree all the memory allocated by spgbeginscan.
It's possible to get away with that in most normal queries, since the
memory is allocated in the executor's per-query context which is about
to get deleted anyway; but it causes severe memory leakage during
creation or filling of large exclusion-constraint indexes.

Also, document that amendscan is supposed to free what ambeginscan
allocates.  The docs' lack of clarity on that point probably caused this
bug to begin with.  (There is discussion of changing that API spec going
forward, but I don't think it'd be appropriate for the back branches.)

Per report from Bruno Wolff.  It's been like this since the beginning,
so back-patch to all active branches.

In HEAD, also fix an independent leak caused by commit 2a6368343
(allocating memory during spgrescan instead of spgbeginscan, which
might be all right if it got cleaned up, but it didn't).  And do a bit
of code beautification on that commit, too.

Discussion: https://postgr.es/m/20181024012314.GA27428@wolff.to
2018-10-31 17:04:43 -04:00
Tom Lane 7dd8b3ca82 Sync our copy of the timezone library with IANA release tzcode2018g.
This patch absorbs an upstream fix to "zic" for a recently-introduced
bug that made it output data that some 32-bit clients couldn't read.
Given the current source data, the bug only manifests in zones with
leap seconds, which we don't generate, so that there's no actual
change in our installed timezone data files from this.  Still, in
case somebody uses our copy of "zic" to do something else, it seems
best to apply the fix promptly.

Also, update the README's notes about converting upstream code to
our conventions.
2018-10-31 09:48:30 -04:00
Tom Lane 3bf4edace7 Update time zone data files to tzdata release 2018g.
DST law changes in Morocco (with, effectively, zero notice).
Historical corrections for Hawaii.
2018-10-31 08:36:41 -04:00
Magnus Hagander bd3fc725b2 Fix missing whitespace in pg_dump ref page
Author: Daniel Gustafsson <daniel@yesql.se>
2018-10-29 12:36:49 +01:00
Andrew Dunstan 075641fd00 Fix perl searchpath for modern perl for MSVC tools
Modern versions of perl no longer include the current directory in the
perl searchpath, as it's insecure. Instead of adding the current
directory, we get around the problem by adding the directory where the
script lives.

Problem noted by Victor Wagner.

Solution adapted from buildfarm client code.

Backpatch to all live versions.
2018-10-28 12:27:58 -04:00
Tom Lane cdf7ff3152 Sync our copy of the timezone library with IANA release tzcode2018f.
About half of this is purely cosmetic changes to reduce the diff between
our code and theirs, like inserting "const" markers where they have them.

The other half is tracking actual code changes in zic.c and localtime.c.
I don't think any of these represent near-term compatibility hazards, but
it seems best to stay up to date.

I also fixed longstanding bugs in our code for producing the
known_abbrevs.txt list, which by chance hadn't been exposed before,
but which resulted in some garbage output after applying the upstream
changes in zic.c.  Notably, because upstream removed their old phony
transitions at the Big Bang, it's now necessary to cope with TZif files
containing no DST transition times at all.
2018-10-19 19:36:34 -04:00
Tom Lane 84261eb10b Update time zone data files to tzdata release 2018f.
DST law changes in Chile, Fiji, and Russia (Volgograd).
Historical corrections for China, Japan, Macau, and North Korea.

Note: like the previous tzdata update, this involves a depressingly
large amount of semantically-meaningless churn in tzdata.zi.  That
is a consequence of upstream's data compression method assigning
unstable abbreviations to DST rulesets.  I complained about that
to them last time, and this version now uses an assignment method
that pays some heed to not changing abbreviations unnecessarily.
So hopefully, that'll be better going forward.
2018-10-19 17:02:26 -04:00
Tom Lane 015fd381fe Still further rethinking of build changes for macOS Mojave.
To avoid the sorts of problems complained of by Jakob Egger, it'd be
best if configure didn't emit any references to the sysroot path at all.
In the case of PL/Tcl, we can do that just by keeping our hands off the
TCL_INCLUDE_SPEC string altogether.  In the case of PL/Perl, we need to
substitute -iwithsysroot for -I in the compile commands, which is easily
handled if we change to using a configure output variable that includes
the switch not only the directory name.  Since PL/Tcl and PL/Python
already do it like that, this seems like good consistency cleanup anyway.

Hence, this replaces the advice given to Perl-related extensions in commit
5e2217131; instead of writing "-I$(perl_archlibexp)/CORE", they should
just write "$(perl_includespec)".  (The old way continues to work, but not
on recent macOS.)

It's still the case that configure needs to be aware of the sysroot
path internally, but that's cleaner than what we had before.

As before, back-patch to all supported versions.

Discussion: https://postgr.es/m/20840.1537850987@sss.pgh.pa.us
2018-10-18 14:55:23 -04:00
Tom Lane e2501d8013 Fix minor bug in isolationtester.
If the lock wait query failed, isolationtester would report the
PQerrorMessage from some other connection, meaning there would be
no message or an unrelated one.  This seems like a pretty unlikely
occurrence, but if it did happen, this bug could make it really
difficult/confusing to figure out what happened.  That seems to
justify patching all the way back.

In passing, clean up another place where the "wrong" conn was used
for an error report.  That one's not actually buggy because it's
a different alias for the same connection, but it's still confusing
to the reader.
2018-10-17 15:06:38 -04:00
Tom Lane c776cd4724 Improve tzparse's handling of TZDEFRULES ("posixrules") zone data.
In the IANA timezone code, tzparse() always tries to load the zone
file named by TZDEFRULES ("posixrules").  Previously, we'd hacked
that logic to skip the load in the "lastditch" code path, which we use
only to initialize the default "GMT" zone during GUC initialization.
That's critical for a couple of reasons: since we do not support leap
seconds, we *must not* allow "GMT" to have leap seconds, and since this
case runs before the GUC subsystem is fully alive, we'd really rather
not take the risk of pg_open_tzfile throwing any errors.

However, that still left the code reading TZDEFRULES on every other
call, something we'd noticed to the extent of having added code to cache
the result so it was only done once per process not a lot of times.
Andres Freund complained about the static data space used up for the
cache; but as long as the logic was like this, there was no point in
trying to get rid of that space.

We can improve matters by looking a bit more closely at what the IANA
code actually needs the TZDEFRULES data for.  One thing it does is
that if "posixrules" is a leap-second-aware zone, the leap-second
behavior will be absorbed into every POSIX-style zone specification.
However, that's a behavior we'd really prefer to do without, since
for our purposes the end effect is to render every POSIX-style zone
name unsupported.  Otherwise, the TZDEFRULES data is used only if
the POSIX zone name specifies DST but doesn't include a transition
date rule (e.g., "EST5EDT" rather than "EST5EDT,M3.2.0,M11.1.0").
That is a minority case for our purposes --- in particular, it
never happens when tzload() invokes tzparse() to interpret a
transition date rule string found in a tzdata zone file.

Hence, if we legislate that we're going to ignore leap-second data
from "posixrules", we can postpone the TZDEFRULES load into the path
where we actually need to substitute for a missing date rule string.
That means it will never happen at all in common scenarios, making it
reasonable to dynamically allocate the cache space when it does happen.
Even when the data is already loaded, this saves some cycles in the
common code path since we avoid a memcpy of 23KB or so.  And, IMO at
least, this is a less ugly hack on the IANA logic than what we had
before, since it's not messing with the lastditch-vs-regular code paths.

Back-patch to all supported branches, not so much because this is a
critical change as that I want to keep all our copies of the IANA
timezone code in sync.

Discussion: https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
2018-10-17 12:26:48 -04:00
Tom Lane 19ac2cb2ae Back off using -isysroot on Darwin.
Rethink the solution applied in commit 5e2217131 to get PL/Tcl to
build on macOS Mojave.  I feared that adding -isysroot globally might
have undesirable consequences, and sure enough Jakob Egger reported
one: it complicates building extensions with a different Xcode version
than was used for the core server.  (I find that a risky proposition
in general, but apparently it works most of the time, so we shouldn't
break it if we don't have to.)

We'd already adopted the solution for PL/Perl of inserting the sysroot
path directly into the -I switches used to find Perl's headers, and we
can do the same thing for PL/Tcl by changing the -iwithsysroot switch
that Apple's tclConfig.sh reports.  This restricts the risks to PL/Perl
and PL/Tcl themselves and directly-dependent extensions, which is a lot
more pleasing in general than a global -isysroot switch.

Along the way, tighten the test to see if we need to inject the sysroot
path into $perl_includedir, as I'd speculated about upthread but not
gotten round to doing.

As before, back-patch to all supported versions.

Discussion: https://postgr.es/m/20840.1537850987@sss.pgh.pa.us
2018-10-16 16:27:15 -04:00
Tom Lane 9d974fa39d Avoid rare race condition in privileges.sql regression test.
We created a temp table, then switched to a new session, leaving
the old session to clean up its temp objects in background.  If that
took long enough, the eventual attempt to drop the user that owns
the temp table could fail, as exhibited today by sidewinder.
Fix by dropping the temp table explicitly when we're done with it.

It's been like this for quite some time, so back-patch to all
supported branches.

Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=sidewinder&dt=2018-10-16%2014%3A45%3A00
2018-10-16 13:56:58 -04:00
Tom Lane baa94ebe2c Avoid statically allocating gmtsub()'s timezone workspace.
localtime.c's "struct state" is a rather large object, ~23KB.  We were
statically allocating one for gmtsub() to use to represent the GMT
timezone, even though that function is not at all heavily used and is
never reached in most backends.  Let's malloc it on-demand, instead.

This does pose the question of how to handle a malloc failure, but
there's already a well-defined error report convention here, ie
set errno and return NULL.

We have but one caller of pg_gmtime in HEAD, and two in back branches,
neither of which were troubling to check for error.  Make them do so.
The possible errors are sufficiently unlikely (out-of-range timestamp,
and now malloc failure) that I think elog() is adequate.

Back-patch to all supported branches to keep our copies of the IANA
timezone code in sync.  This particular change is in a stanza that
already differs from upstream, so it's a wash for maintenance purposes
--- but only as long as we keep the branches the same.

Discussion: https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
2018-10-16 11:50:19 -04:00
Tom Lane 3a60c8bb16 Check for stack overrun in standard_ProcessUtility().
ProcessUtility can recurse, and indeed can be driven to infinite
recursion, so it ought to have a check_stack_depth() call.  This
covers the reported bug (portal trying to execute itself) and a bunch
of other cases that could perhaps arise somewhere.

Per bug #15428 from Malthe Borch.  Back-patch to all supported branches.

Discussion: https://postgr.es/m/15428-b3c2915ec470b033@postgresql.org
2018-10-15 14:01:38 -04:00
Michael Paquier 0c99e71965 Avoid duplicate XIDs at recovery when building initial snapshot
On a primary, sets of XLOG_RUNNING_XACTS records are generated on a
periodic basis to allow recovery to build the initial state of
transactions for a hot standby.  The set of transaction IDs is created
by scanning all the entries in ProcArray.  However it happens that its
logic never counted on the fact that two-phase transactions finishing to
prepare can put ProcArray in a state where there are two entries with
the same transaction ID, one for the initial transaction which gets
cleared when prepare finishes, and a second, dummy, entry to track that
the transaction is still running after prepare finishes.  This way
ensures a continuous presence of the transaction so as callers of for
example TransactionIdIsInProgress() are always able to see it as alive.

So, if a XLOG_RUNNING_XACTS takes a standby snapshot while a two-phase
transaction finishes to prepare, the record can finish with duplicated
XIDs, which is a state expected by design.  If this record gets applied
on a standby to initial its recovery state, then it would simply fail,
so the odds of facing this failure are very low in practice.  It would
be tempting to change the generation of XLOG_RUNNING_XACTS so as
duplicates are removed on the source, but this requires to hold on
ProcArrayLock for longer and this would impact all workloads,
particularly those using heavily two-phase transactions.

XLOG_RUNNING_XACTS is also actually used only to initialize the standby
state at recovery, so instead the solution is taken to discard
duplicates when applying the initial snapshot.

Diagnosed-by: Konstantin Knizhnik
Author: Michael Paquier
Discussion: https://postgr.es/m/0c96b653-4696-d4b4-6b5d-78143175d113@postgrespro.ru
Backpatch-through: 9.3
2018-10-14 22:24:01 +09:00
Tom Lane fb583c80d2 Remove abstime, reltime, tinterval tables from old regression databases.
In the back branches, drop these tables after the regression tests are
done with them.  This fixes failures of cross-branch pg_upgrade testing
caused by these types having been removed in v12.  We do lose the ability
to test dump/restore behavior with these types in the back branches, but
the actual loss of code coverage seems to be nil given that there's nothing
very special about these types.

Discussion: https://postgr.es/m/20181009192237.34wjp3nmw7oynmmr@alap3.anarazel.de
2018-10-12 19:33:57 -04:00
Tom Lane 01187f32cd Back-patch addition of the ALLOCSET_FOO_SIZES macros.
These macros were originally added in commit ea268cdc9, and back-patched
into 9.6 before 9.6.0.  However, some extensions would like to use them
in older branches, and there seems no harm in providing them.  So add
them to all supported branches.  Per suggestions from Christoph Berg and
Andres Freund.

Discussion: https://postgr.es/m/20181012170355.bhxi273skjt6sag4@alap3.anarazel.de
2018-10-12 14:49:33 -04:00
Tom Lane 6e63e06975 Allow btree comparison functions to return INT_MIN.
Historically we forbade datatype-specific comparison functions from
returning INT_MIN, so that it would be safe to invert the sort order
just by negating the comparison result.  However, this was never
really safe for comparison functions that directly return the result
of memcmp(), strcmp(), etc, as POSIX doesn't place any such restriction
on those library functions.  Buildfarm results show that at least on
recent Linux on s390x, memcmp() actually does return INT_MIN sometimes,
causing sort failures.

The agreed-on answer is to remove this restriction and fix relevant
call sites to not make such an assumption; code such as "res = -res"
should be replaced by "INVERT_COMPARE_RESULT(res)".  The same is needed
in a few places that just directly negated the result of memcmp or
strcmp.

To help find places having this problem, I've also added a compile option
to nbtcompare.c that causes some of the commonly used comparators to
return INT_MIN/INT_MAX instead of their usual -1/+1.  It'd likely be
a good idea to have at least one buildfarm member running with
"-DSTRESS_SORT_INT_MIN".  That's far from a complete test of course,
but it should help to prevent fresh introductions of such bugs.

This is a longstanding portability hazard, so back-patch to all supported
branches.

Discussion: https://postgr.es/m/20180928185215.ffoq2xrq5d3pafna@alap3.anarazel.de
2018-10-05 16:01:30 -04:00
Tom Lane 4ed6dc7d4b Set snprintf.c's maximum number of NL arguments to be 31.
Previously, we used the platform's NL_ARGMAX if any, otherwise 16.
The trouble with this is that the platform value is hugely variable,
ranging from the POSIX-minimum 9 to as much as 64K on recent FreeBSD.
Values of more than a dozen or two have no practical use and slow down
the initialization of the argtypes array.  Worse, they cause snprintf.c
to consume far more stack space than was the design intention, possibly
resulting in stack-overflow crashes.

Standardize on 31, which is comfortably more than we need (it looks like
no existing translatable message has more than about 10 parameters).
I chose that, not 32, to make the array sizes powers of 2, for some
possible small gain in speed of the memset.

The lack of reported crashes suggests that the set of platforms we
use snprintf.c on (in released branches) may have no overlap with
the set where NL_ARGMAX has unreasonably large values.  But that's
not entirely clear, so back-patch to all supported branches.

Per report from Mateusz Guzik (via Thomas Munro).

Discussion: https://postgr.es/m/CAEepm=3VF=PUp2f8gU8fgZB22yPE_KBS0+e1AHAtQ=09schTHg@mail.gmail.com
2018-10-02 12:41:28 -04:00
Tom Lane 01c7a87df9 Fix corner-case failures in has_foo_privilege() family of functions.
The variants of these functions that take numeric inputs (OIDs or
column numbers) are supposed to return NULL rather than failing
on bad input; this rule reduces problems with snapshot skew when
queries apply the functions to all rows of a catalog.

has_column_privilege() had careless handling of the case where the
table OID didn't exist.  You might get something like this:
	select has_column_privilege(9999,'nosuchcol','select');
	ERROR:  column "nosuchcol" of relation "(null)" does not exist
or you might get a crash, depending on the platform's printf's response
to a null string pointer.

In addition, while applying the column-number variant to a dropped
column returned NULL as desired, applying the column-name variant
did not:
	select has_column_privilege('mytable','........pg.dropped.2........','select');
	ERROR:  column "........pg.dropped.2........" of relation "mytable" does not exist
It seems better to make this case return NULL as well.

Also, the OID-accepting variants of has_foreign_data_wrapper_privilege,
has_server_privilege, and has_tablespace_privilege didn't follow the
principle of returning NULL for nonexistent OIDs.  Superusers got TRUE,
everybody else got an error.

Per investigation of Jaime Casanova's report of a new crash in HEAD.
These behaviors have been like this for a long time, so back-patch to
all supported branches.

Patch by me; thanks to Stephen Frost for discussion and review

Discussion: https://postgr.es/m/CAJGNTeP=-6Gyqq5TN9OvYEydi7Fv1oGyYj650LGTnW44oAzYCg@mail.gmail.com
2018-10-02 11:54:13 -04:00
Michael Paquier 4b2fb124d1 Fix documentation of pgrowlocks using "lock_type" instead of "modes"
The example used in the documentation is outdated as well.  This is an
oversight from 0ac5ad5, which bumped up pgrowlocks but forgot some bits
of the documentation.

Reported-by: Chris Wilson
Discussion: https://postgr.es/m/153838692816.2950.12001142346234155699@wrigleys.postgresql.org
Backpatch-through: 9.3
2018-10-02 16:37:16 +09:00
Tom Lane de0bea8d4d Lock relation used to generate fresh data for RMV.
Back-patch the 9.4-era commit 2636ecf78 into 9.3, as that fixes a case
where we open a relation while not holding any lock on it.  It's
probably mostly safe anyway, since no other session could touch the
newly-created table; but I think CheckTableNotInUse could be fooled
if one tried.

Per testing with a patch that complains if we open a relation without
holding any lock on it.  I don't plan to back-patch that patch, but we
should close the holes it identifies in all supported branches.

Discussion: https://postgr.es/m/2038.1538335244@sss.pgh.pa.us
2018-10-01 11:42:12 -04:00