7f380c59f8
Previously, the core scanner's yy_transition[] array had 37045 elements. Since that number is larger than INT16_MAX, Flex generated the array to contain 32-bit integers. By reimplementing some of the bulkier scanner rules, this patch reduces the array to 20495 elements. The much smaller total length, combined with the consequent use of 16-bit integers for the array elements reduces the binary size by over 200kB. This was accomplished in two ways: 1. Consolidate handling of quote continuations into a new start condition, rather than duplicating that logic for five different string types. 2. Treat Unicode strings and identifiers followed by a UESCAPE sequence as three separate tokens, rather than one. The logic to de-escape Unicode strings is moved to the filter code in parser.c, which already had the ability to provide special processing for token sequences. While we could have implemented the conversion in the grammar, that approach was rejected for performance and maintainability reasons. Performance in microbenchmarks of raw parsing seems equal or slightly faster in most cases, and it's reasonable to expect that in real-world usage (with more competition for the CPU cache) there will be a larger win. The exception is UESCAPE sequences; lexing those is about 10% slower, primarily because the scanner now has to be called three times rather than one. This seems acceptable since that feature is very rarely used. The psql and epcg lexers are likewise modified, primarily because we want to keep them all in sync. Since those lexers don't use the space-hogging -CF option, the space savings is much less, but it's still good for perhaps 10kB apiece. While at it, merge the ecpg lexer's handling of C-style comments used in SQL and in C. Those have different rules regarding nested comments, but since we already have the ability to keep track of the previous start condition, we can use that to handle both cases within a single start condition. This matches the core scanner more closely. John Naylor Discussion: https://postgr.es/m/CACPNZCvaoa3EgVWm5yZhcSTX6RAtaLgniCPcBVOCwm8h3xpWkw@mail.gmail.com |
||
|---|---|---|
| .. | ||
| .gitignore | ||
| Makefile | ||
| README | ||
| analyze.c | ||
| check_keywords.pl | ||
| gram.y | ||
| parse_agg.c | ||
| parse_clause.c | ||
| parse_coerce.c | ||
| parse_collate.c | ||
| parse_cte.c | ||
| parse_enr.c | ||
| parse_expr.c | ||
| parse_func.c | ||
| parse_node.c | ||
| parse_oper.c | ||
| parse_param.c | ||
| parse_relation.c | ||
| parse_target.c | ||
| parse_type.c | ||
| parse_utilcmd.c | ||
| parser.c | ||
| scan.l | ||
| scansup.c | ||
README
src/backend/parser/README Parser ====== This directory does more than tokenize and parse SQL queries. It also creates Query structures for the various complex queries that are passed to the optimizer and then executor. parser.c things start here scan.l break query into tokens scansup.c handle escapes in input strings gram.y parse the tokens and produce a "raw" parse tree analyze.c top level of parse analysis for optimizable queries parse_agg.c handle aggregates, like SUM(col1), AVG(col2), ... parse_clause.c handle clauses like WHERE, ORDER BY, GROUP BY, ... parse_coerce.c handle coercing expressions to different data types parse_collate.c assign collation information in completed expressions parse_cte.c handle Common Table Expressions (WITH clauses) parse_expr.c handle expressions like col, col + 3, x = 3 or x = 4 parse_func.c handle functions, table.column and column identifiers parse_node.c create nodes for various structures parse_oper.c handle operators in expressions parse_param.c handle Params (for the cases used in the core backend) parse_relation.c support routines for tables and column handling parse_target.c handle the result list of the query parse_type.c support routines for data type handling parse_utilcmd.c parse analysis for utility commands (done at execution time) See also src/common/keywords.c, which contains the table of standard keywords and the keyword lookup function. We separated that out because various frontend code wants to use it too.