Software update: SQLite 3.35.2

SQLite is a database system developed in C that can be used as a database for websites and embedded applications, among other things. According to the developers, SQLite requires no installation and administration, supports databases up to two terabytes in size and stores an entire database in one file. Furthermore, it supports almost the full SQL92 specification and it is easy via, among others Tcl/Tk to guide. The developer has released version 3.35.2 with the following list of changes:

Version 3.35.2

• Fix a problem in the appendvfs.c extension that was introduced into version 3.35.0.
• Ensure that date/time functions with no arguments (which generate responses that depend on the current time) are treated as non-deterministic functions. Ticket 2c6c8689fb5f3d2f
• Fix a problem in the sqldiff utility program having to do with unusual whitespace characters in a virtual table definition.
• Limit the new UNION ALL optimization described by item 8c in the 3.35.0 release so that it does not try to make too many new subqueries. See forum thread 140a67d3d2 for details.

Version 3.35.1

• Fix a bug in the new DROP COLUMN feature when used on columns that are indexed and that are quoted in the index definition.
• Improve the built-in documentation for the .dump command in the CLI.

Version 3.35.0

• Added built-in SQL math functions(). (Requires the -DSQLITE_ENABLE_MATH_FUNCTIONS compile-time option.)
• Added support for ALTER TABLE DROP COLUMN.
• Generalize UPSERT:
  • Allow multiple ON CONFLICT clauses that are evaluated in order,
  • The final ON CONFLICT clause may omit the conflict target and yet still use DO UPDATE.
• Add support for the RETURNING clause on DELETE, INSERT, and UPDATE statements.
• Use less memory when running VACUUM on databases containing very large TEXT or BLOB values. It is no longer necessary to hold the entire TEXT or BLOB in memory all at once.
• Add support for the MATERIALIZED and NOT MATERIALIZED hints when specifying common table expressions. The default behavior was formerly NOT MATERIALIZED, but is now changed to MATERIALIZED for CTEs that are used more than once.
• The SQLITE_DBCONFIG_ENABLE_TRIGGER and SQLITE_DBCONFIG_ENABLE_VIEW settings are modified so that they only control triggers and views in the main database schema or in attached database schemas and not in the TEMP schema. TEMP triggers and views are always allowed.
• Query scheduler/optimizer improvements:
  • Enhancements to the min/max optimization so that it works better with the IN operator and the OP_SeekScan optimization of the previous release.
  • Attempt to process EXISTS operators in the WHERE clause as if they were IN operators, in cases where this is a valid transformation and seems likely to improve performance.
  • Allow UNION ALL sub-queries to be flattened even if the parent query is a join.
  • Use an index, if appropriate, on IS NOT NULL expressions in the WHERE clause, even if STAT4 is disabled.
  • Expressions of the form “x IS NULL” or “x IS NOT NULL” might be converted to simply FALSE or TRUE, if “x” is a column that has a “NOT NULL” constraint and is not involved in an outer join.
  • Avoid checking foreign key constraints on an UPDATE statement if the UPDATE does not modify any columns associated with the foreign key.
  • Allow WHERE terms to be pushed down into sub-queries that contain window functions, as long as the WHERE term is made up of entirely of constants and copies of expressions found in the PARTITION BY clauses of all window functions in the sub-query.
• CLI enhancements:
  • Enhance the “.stats” command to accept new arguments “stmt” and “vmstep”, causing prepare statement statistics and only the virtual-machine step count to be shown, respectively.
  • Add the “.filectrl data_version” command.
  • Enhance the “.once” and “.output” commands so that if the destination argument begins with “|” (indicating that output is redirected into a pipe) then the argument does not need to be quoted.
• Bug fixes:
  • Fix a potential NULL pointer dereference when processing a syntactically incorrect SELECT statement with a correlated WHERE clause and a “HAVING 0” clause. (Also fixed in the 3.34.1 patch release.)
  • Fix a bug in the IN-operator optimization of version 3.33.0 that can cause an incorrect answer.
  • Fix incorrect answers from the LIKE operator if the pattern ends with “%” and there is an “ESCAPE ‘_'” clause.

Version 3.34.1

• Fix a potential use-after-free bug when processing aa subquery with both a correlated WHERE clause and a “HAVING 0” clause and where the parent query is an aggregate.
• Fix documentation typos
• Fix minor problems in extensions.

Version 3.34.0

• Added the sqlite3_txn_state() interface for reporting on the current transaction state of the database connection.
• Enhance recursive common table expressions to support two or more recursive terms as is done by SQL Server, since this helps make queries against graphs easier to write and faster to execute.
• Improved error messages on CHECK constraint failures.
• CLI enhancements:
  • The .read dot-command now accepts a pipeline in addition to a filename.
  • Added options –data-only and –nosys to the .dump dot-command.
  • Added the –nosys option to the .schema dot-command.
  • Table name quoting works correctly for the .import dot-command.
  • The generate_series(START,END,STEP) table-valued function extension is now built into the CLI.
  • The .databases dot-command now shows the status of each database file as determined by sqlite3_db_readonly() and sqlite3_txn_state().
  • Added the –tabs command-line option that sets .mode tabs.
  • The –init option reports an error if the file named as its argument cannot be opened. The –init option also now honors the –bail option.
• Query scheduler improvements:
  • Improved estimates for the cost of running a DISTINCT operator.
  • When doing an UPDATE or DELETE using a multi-column index where only a few of the earlier columns of the index are useful for the index lookup, postpone doing the main table seek until after all WHERE clause constraints have been evaluated, in case those constraints can be covered by unused later terms of the index, thus avoiding unnecessary main table seeks.
  • The new OP_SeekScan opcode is used to improve performance of multi-column index look-ups when later columns are constrained by an IN operator.
• The BEGIN IMMEDIATE and BEGIN EXCLUSIVE commands now work even if one or more attached database files are read-only.
• Enhanced FTS5 to support trigram indexes.
• Improved performance of WAL mode locking primitives in cases where there are hundreds of connections all accessing the same database file at once.
• Enhanced the carray() table-valued function to include a single-argument form that is bound using the auxiliary sqlite3_carray_bind() interface.
• The substr() SQL function can now also be called “substring()” for compatibility with SQL Server.
• The syntax diagrams are now implemented as Pikchr scripts and rendered as SVG for improved legibility and ease of maintenance.

Version 3.33.0

• Support for UPDATE FROM following the PostgreSQL syntax.
• Increase the maximum size of database files to 281 TB.
• Extended the PRAGMA integrity_check statement so that it can optionally be limited to verifying just a single table and its indexes, rather than the entire database file.
• Added the decimal extension for doing arbitrary-precision decimal arithmetic.
• Enhancements to the ieee754 extension for working with IEEE 754 binary64 numbers.
• CLI enhancements:
  • Added four new output modes: “box”, “json”, “markdown”, and “table”.
  • The “column” output mode automatically expands columns to contain the longest output row and automatically turns “.header” on if it has not been previously set.
  • The “quote” output mode honors “.separator”
  • The decimal extension and the ieee754 extension are built-in to the CLI
• Query scheduler improvements:
  • Add the ability to find a full-index-scan query plan for queries using INDEXED BY which previously would fail with “no query solution”.
  • Do a better job of detecting missing, incomplete, and/or dodgy sqlite_stat1 data and generates good query plans in spite of the misinformation.
  • Improved performance of queries like “SELECT min(x) FROM t WHERE y IN (?,?,?)” assuming an index on t(x,y).
• In WAL mode, if a writer crashes and leaves the shm file in an inconsistent state, subsequent transactions are now able to recover the shm file even if there are active read transactions. Before this enhancement, shm file recovery that scenario would result in an SQLITE_PROTOCOL error.