postgresql/src/backend/catalog/README

$PostgreSQL: pgsql/src/backend/catalog/README,v 1.11 2007/05/11 17:57:11 tgl Exp $

This directory contains .c files that manipulate the system catalogs;
src/include/catalog contains the .h files that define the structure
of the system catalogs.

When the compile-time scripts (such as Gen_fmgrtab.sh and genbki.sh)
execute, they grep the DATA statements out of the .h files and munge
these in order to generate the postgres.bki file.  The .bki file is then
used as input to initdb (which is just a wrapper around postgres
running single-user in bootstrapping mode) in order to generate the
initial (template) system catalog relation files.

-----------------------------------------------------------------

People who are going to hose around with the .h files should be aware
of the following facts:

- It is very important that the DATA statements be properly formatted
(e.g., no broken lines, proper use of white-space and _null_).  The
scripts are line-oriented and break easily.  In addition, the only
documentation on the proper format for them is the code in the
bootstrap/ directory.  Just be careful when adding new DATA
statements.

- Some catalogs require that OIDs be preallocated to tuples because
of cross-references from other pre-loaded tuples.  For example, pg_type
contains pointers into pg_proc (e.g., pg_type.typinput), and pg_proc
contains back-pointers into pg_type (pg_proc.proargtypes).  For such
cases, the OID assigned to a tuple may be explicitly set by use of the
"OID = n" clause of the .bki insert statement.  If no such pointers are
required to a given tuple, then the OID = n clause may be omitted
(then the system generates an OID in the usual way, or leaves it 0 in a
catalog that has no OIDs).  In practice we usually preassign OIDs
for all or none of the pre-loaded tuples in a given catalog, even if only
some of them are actually cross-referenced.

- We also sometimes preallocate OIDs for catalog tuples whose OIDs must
be known directly in the C code.  In such cases, put a #define in the
catalog's .h file, and use the #define symbol in the C code.  Writing
the actual numeric value of any OID in C code is considered very bad form.
Direct references to pg_proc OIDs are common enough that there's a special
mechanism to create the necessary #define's automatically: see
backend/utils/Gen_fmgrtab.sh.  We also have standard conventions for setting
up #define's for the pg_class OIDs of system catalogs and indexes.  For all
the other system catalogs, you have to manually create any #define's you
need.

- If you need to find a valid OID for a new predefined tuple,
use the unused_oids script.  It generates inclusive ranges of
*unused* OIDs (e.g., the line "45-900" means OIDs 45 through 900 have
not been allocated yet).  Currently, OIDs 1-9999 are reserved for manual
assignment; the unused_oids script simply looks through the include/catalog
headers to see which ones do not appear in "OID =" clauses in DATA lines.
(As of Postgres 8.1, it also looks at CATALOG and DECLARE_INDEX lines.)
You can also use the duplicate_oids script to check for mistakes.

- The OID counter starts at 10000 at bootstrap.  If a catalog row is in a
table that requires OIDs, but no OID was preassigned by an "OID =" clause,
then it will receive an OID of 10000 or above.

- To create a "BOOTSTRAP" table you have to do a lot of extra work: these
tables are not created through a normal CREATE TABLE operation, but spring
into existence when first written to during initdb.  Therefore, you must
manually create appropriate entries for them in the pre-loaded contents of
pg_class, pg_attribute, and pg_type.  Avoid making new catalogs be bootstrap
catalogs if at all possible; generally, only tables that must be written to
in order to create a table should be bootstrapped.

- Certain BOOTSTRAP tables must be at the start of the Makefile
POSTGRES_BKI_SRCS variable, as these cannot be created through the standard
heap_create_with_catalog process, because it needs these tables to exist
already.  The list of files this currently includes is:
	pg_proc.h pg_type.h pg_attribute.h pg_class.h
Also, indexing.h must be last, since the indexes can't be created until all
the tables are in place, and toasting.h should probably be next-to-last
(or at least after all the tables that need toast tables).  There are
reputedly some other order dependencies in the .bki list, too.

-----------------------------------------------------------------

When munging the .c files, you should be aware of certain conventions:

- The system catalog cache code (and most catalog-munging code in
general) assumes that the fixed-length portions of all system catalog
tuples are in fact present, because it maps C struct declarations onto
them.  Thus, the variable-length fields must all be at the end, and
only the variable-length fields of a catalog tuple are permitted to be
NULL.  For example, if you set pg_type.typrelid to be NULL, a
piece of code will likely perform "typetup->typrelid" (or, worse,
"typetyp->typelem", which follows typrelid).  This will result in
random errors or even segmentation violations.  Hence, do NOT insert
catalog tuples that contain NULL attributes except in their
variable-length portions!  (The bootstrapping code is fairly good about
marking NOT NULL each of the columns that can legally be referenced via
C struct declarations ... but those markings won't be enforced against
DATA commands, so you must get it right in a DATA line.)

- Modification of the catalogs must be performed with the proper
updating of catalog indexes!  That is, most catalogs have indexes
on them; when you munge them using the executor, the executor will
take care of doing the index updates, but if you make direct access
method calls to insert new or modified tuples into a heap, you must
also make the calls to insert the tuple into ALL of its indexes!  If
not, the new tuple will generally be "invisible" to the system because
most of the accesses to the catalogs in question will be through the
associated indexes.