Yellowbrick Data Warehouse Version 5.2 Change History
5.2.26 Yellowbrick Release Notes
Release Date: 12/14/2024
Upgrade Requirements
Read the following information before beginning an upgrade to Version 5.2.
Important
Customers should always contact Customer Support ahead of the planned upgrade date to review the process and schedule the upgrade.
Upgrades from Version 5.x
If you are upgrading from a version prior to 5.2.10, additional steps may be required to upgrade the cluster. Customer Support will walk you through the process.
For upgrades from 5.2.9 or earlier: Before starting the upgrade, on the version you are upgrading from, run:
ALTER SYSTEM SET autovacuum TO 'off';
SELECT pg_reload_conf();
After a successful upgrade, when the database is up, run:
ALTER SYSTEM SET autovacuum TO 'on';
SELECT pg_reload_conf();
For upgrades from 5.2.0, 5.2.1, and 5.2.2: An additional step at the end of the upgrade process is required to update view definitions. Technical Support will walk you through the process.
See also Replication Compatibility and Upgrade Process.
Upgrades from Version 4.x
Be sure to read Changes in Behavior and Compatibility.
For upgrades from 4.1.15 or earlier: Before starting the upgrade, on the version you are upgrading from, run:
ALTER SYSTEM SET autovacuum TO 'off';
SELECT pg_reload_conf();
After a successful upgrade, when the database is up, run:
ALTER SYSTEM SET autovacuum TO 'on';
SELECT pg_reload_conf();
Upgrades from Version 3.x
- Version 3.3.x or higher is a prerequisite for upgrading to 5.2.
- A Version 5.2 upgrade from Version 3.3.x has required one-time procedures. These steps are specific to 3.x upgrades to 5.2; they are not required for upgrades from 4.x or 5.1.x.
- A critical pre-upgrade script must be scheduled and run weeks in advance of the upgrade itself.
- A separate post-upgrade Yellowbrick network software upgrade step is required.
- A separate post-upgrade Yellowbrick background shard rewrite process must be manually initiated. This step can be scheduled to minimize the impact on your databases.
Changes in Behavior and Compatibility
Summary of Changes in Behavior
ybtools Compatibility
Yellowbrick recommends that you always upgrade ybtools to match the Yellowbrick server version you are running (for example, Version 5.2). Using Version 5.2 of ybtools with an older server version (such as Version 4.1 or 5.1) may result in error messages for some commands. Using previous versions of ybtools with a Version 5.2 database is likely to work but is deprecated and not supported. See also the following section: Backward Compatibility for Backup and Restore.
Important: On CentOS and Red Hat client platforms, you must first remove any existing 3.x version of ybtools. Then you can proceed with the installation of the Version 5.2 ybtools. You cannot upgrade directly from an earlier 3.x version of ybtools to Version 5.2.
Replication Compatibility and Upgrade Process
Important: Both systems used for database replication (source and target) must be running the same major version of the database software. For example, you cannot replicate from a 4.x source to a 5.x target or vice versa.
In Version 5.2.19, you cannot replicate to a target system on any earlier version; both systems must be running Version 5.2.19 of the database software. For example, you cannot replicate from a 5.2.19 source to a 5.2.18 target. This behavior is enforced because of a change in the system catalog for Version 5.2.18.
You can replicate a Version 5.x database to a later 5.x database (such as 5.1.x to 5.2.19). For example, if you are upgrading from Version 5.1.x, and you want to reverse replication in Version 5.2 (using PROMOTE), follow these steps:
- Pause the replica on the Version 5.1.x source system.
- Upgrade the target system to Version 5.2.19.
- Verify that replication works from 5.1.x to 5.2.19.
- Attempt a promote on the 5.2.19 target. This operation will fail with an expected error because you cannot replicate to a system that is running an earlier version of the software. The replica will remain paused on the 5.1.x source system.
- Upgrade the source system to 5.2.19.
- Attempt the promote again on the 5.2.19 target system. This operation should succeed.
- Attempt a new replication cycle from the reversed source to the new target (both 5.2.18 systems). The replication cycle should succeed on the new target.
Backward Compatibility for Backup and Restore
A backup taken in Version 3.2.x, 3.3.x, 4.x, or 5.x prior to a 5.2 upgrade can be restored in Version 5.2.
After a 5.2 upgrade or installation, the previous version of the backup and restore tools is compatible with Version 5.2. The Version 5.2 ybtools package installs two sets of backup and restore tools: legacy tools and new 5.2 tools.
The legacy tools are deprecated and will be removed in a future release, along with the ability to restore backups taken in previous releases. There are significant differences in the functionality and client tool options between the legacy tools and the new tools. The tool sets are distinguished by their names:
ybtools Version | Backup | Restore | List/Manage Backups |
---|---|---|---|
New in Version 4.x and later | ybbackup | ybrestore | ybbackupctl |
Legacy in Version 4.x and later | ybbackup-old | ybrestore-old | ybbackup-list-old |
Version 3.x | ybbackup | ybrestore | ybbackup-list |
To use the legacy tools in the 5.2 package, be sure to use the correct tool name in the command. For example, to return help for the legacy version of ybbackup:
$ ybbackup-old --help
Yellowbrick Backup and Restore version 5.2.0-33445
...
Important:
To use backup and restore in Version 5.2, you must follow these steps:
- Use the legacy version of ybtools (-old) to restore any 3.x backups into 5.2 databases. Use the new 4.x or 5.x version of ybtools to restore any 4.x or 5.x backups into 5.2 databases.
- Take new full backups of your 5.2 databases with the new 5.2 ybtools.
- Stop using the legacy ybtools for backup and restore purposes. Use only the new tools.
What's New in Version 5.2.26
Version 5.2.26 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
DLE_DST and LE_DST Functions
These functions return the difference between two character strings in terms of the number of changes required to convert one to the other. The DLE_DST function uses the Damerau-Levenshtein edit distance algorithm and the LE_DST function uses the Levenshtein edit distance algorithm to compute the "distance" between the strings. See the updated Version 5.2.x documentation for details.
Query Memory Usage Change
In Version 5.2.26, complex queries now return an error if their memory usage during planning exceeds 40GiB instead of triggering the OS Out Of Memory.
What's New in Version 5.2.25
Kerberos SSO Support on Windows
The Yellowbrick client tools now support Kerberos SSO on Windows. To enable and configure Kerberos on a Yellowbrick appliance, contact Yellowbrick Customer Support.
What's New in Version 5.2.24
Version 5.2.24 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.23
Version 5.2.23 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
Behavior of the CHAR Data Type
The CHAR data type implementation in Yellowbrick 5.x has resulted in some behavioral inconsistencies. For details about changes introduced in Version 5.2.23 to address these problems, see Upcoming Behavioral Changes to the CHAR Data Type.
Default Behavior of WLM Restart Rules
By default, a subset of recoverable error codes triggers an attempted restart, and an attempted restart occurs only once per query. For details, see "Restarting Queries" in the updated Version 5.2.23 documentation.
JSON Functions
Two new SQL functions for JSON manipulation, JSON_INGEST and JSON_LOOKUP, are generally available in Version 5.2.23. See the updated Version 5.2.23 documentation for details and examples.
What's New in Version 5.2.22
Version 5.2.22 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
GSSAPI Support in ybtools
The Yellowbrick client tools (ybtools) now have GSSAPI support, which enables support for Kerberos authentication. To enable and configure Kerberos on a Yellowbrick appliance, contact Yellowbrick Customer Support.
Type-Safe Casting Functions
Three new SQL functions are supported for compatibility with applications developed in other databases. These functions implement type-safe casting from VARCHAR strings to DATE, TIMESTAMP, and DOUBLE data types:
- SAFE_TO_DATE
- SAFE_TO_TIMESTAMP
- SAFE_TO_DOUBLE
For more details, see the SQL Functions section in the Version 5.2.x documentation.
What's New in Version 5.2.21
Version 5.2.21 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
Kerberos Authentication
Version 5.2.21 supports Kerberos Version 5 network authentication. To enable and configure Kerberos on a Yellowbrick appliance, contact Yellowbrick Customer Support.
What's New in Version 5.2.20
Version 5.2.20 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.19
Version 5.2.19 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.18
Version 5.2.18 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
Query Plans for GROUP BY in Decorrelated Subqueries
In previous versions, the GROUP BY clause in a decorrelated subquery of the following form:
SELECT ... FROM t1 WHERE x IN (SELECT y FROM t2 WHERE t2.x = t1.x GROUP BY y)
was always executed as part of the query plan. In Version 5.2.18, a GROUP BY clause in this kind of query is only executed if statistics suggest that its inclusion will improve performance. Therefore, you may see a change in the plan for this type of query.
What's New in Version 5.2.17
Version 5.2.17 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.16
Version 5.2.16 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.15
Version 5.2.15 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
Yellowbrick Client Upgrades for log4j
To ensure that the Yellowbrick client tools are not subject to known log4j vulnerabilities, the log4j jar used by ybtools has been upgraded to version 2.17.1.
What's New in Version 5.2.14
Version 5.2.14 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.13
Version 5.2.13 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
Disabling ANALYZE Operations for ybload
Two configuration parameters are available to control the use of ANALYZE operations that occur after data is written to the database:
ybd_analyze_after_loads: Defaults to ON and triggers an automatic ANALYZE if 10% or more of the rows have changed for the target table when a ybload operation completes. To disable the automatic ANALYZE, set this parameter to OFF. This parameter is new in Version 5.2.13 and applies to workloads that require a large number of continuous trickle-feed ybload operations.
Note that setting this parameter to OFF does not prevent an immediate ANALYZE from being run after intermediate load operations, such as after an INSERT into a target table from a staging table. However, setting ybd_analyze_after_writes to OFF will prevent an immediate ANALYZE in these cases.
ybd_analyze_after_writes: Defaults to ON and continues to work as described in the Version 5.2.x documentation.
Both parameters and their current settings are visible in the output for SHOW ALL.
What's New in Version 5.2.12
Version 5.2.12 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.11
Version 5.2.11 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
Increased Memory Estimation
Queries may experience slightly higher memory estimation, up to a maximum of approximately 500MB more than was used in Versions 5.2.8 and earlier. The actual increase depends on how much disk space is available on each worker. This change in estimation corrects an issue in Version 5.2.9 where memory estimation was set too high and out-of-memory (OOM) conditions could occur when a query started spilling.
Protection from log4j Vulnerabilities
To ensure that the Yellowbrick server and client tools are not subject to known log4j vulnerabilities, unused packages and their dependencies were removed from the log4j distribution used by Yellowbrick. Specifically:
- The RHEL log4j package and RPMs have been removed from the manager node.
- The log4j jar used by Yellowbrick on the manager node has been replaced with yb-log-4.1.16.jar, which does not contain the vulnerable classes (JMSAppender, JMSSink, SimpleSocketServer, and SocketServer).
- The log4j jar used by ybtools has been replaced with yb-log-4.1.16.jar, which does not contain the vulnerable classes (JMSAppender, JMSSink, SimpleSocketServer, and SocketServer).
Note that a logger is still required when integrating with Spark, and it is the responsibility of the Spark runtime to provide it.
What's New in Version 5.2.10
Version 5.2.10 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.9
Version 5.2.9 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
Deprecated Feature: Restartable Replication
Warning
The restartable replication feature that was introduced as a tech preview in Version 5.2.0 is deprecated and disabled in Version 5.2.9. This feature is not supported under any circumstances for use on production systems; if you would like to use it purely for testing purposes, contact Customer Support.
TO_NUMBER Function
You can convert a number string to an actual numeric value, using a specified format.
Note that the addition of this function introduces a change to the system catalog version.
Support for 2,000 Columns in a Table
The maximum number of user-defined columns in a table has been increased to 2,000.
Disabling Join Elimination for LEFT Joins
Join elimination is a query optimization that can occur during planning when primary and foreign keys exist on the tables. The query plan may eliminate redundant joins when these key relationships have been declared. However, because Yellowbrick does not enforce primary key and foreign key constraints, your tables may have key relationships that are not valid. If this is true, the elimination of joins between those tables may cause queries to return wrong results.
To prevent this problem from occurring specifically for LEFT joins, you can set the enable_join_elimination configuration parameter to OFF in Version 5.2.9. For more details and examples, see the CREATE TABLE section of the Version 5.2.9 documentation.
Query Planning Changes
Default constants for costing have been changed to optimize memory consumption for queries.
Explicit Casts for Numeric Calculations
In previous versions, an explicit cast for a numeric expression was ignored, and the return type was adjusted to fit the precision and scale of the actual value. In Version 5.2.9, such explicit casts are respected, and the return type of the expression is based on the requested precision and scale.
For example, in Version 5.2.9 the following CTAS statement returns DECIMAL(21,10):
yellowbrick_test=# create table t as
select 3.14159::NUMERIC(20,10) + 1::NUMERIC(1,0) from sys.const;
SELECT 1
yellowbrick_test=# \d t
Table "public.t"
Column | Type | Modifiers
----------+----------------+-----------
?column? | numeric(21,10) |
The same statement in earlier 5.2.x versions returns DECIMAL(11,10):
yellowbrick_test=# create table t as
select 3.14159::NUMERIC(20,10) + 1::NUMERIC(1,0) from sys.const;
SELECT 1
yellowbrick_test=# \d t
Table "public.t"
Column | Type | Modifiers
---------+----------------+-----------
?column? | numeric(11,10) |
What's New in Version 5.2.8
Version 5.2.8 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.7
Version 5.2.7 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.6
Version 5.2.6 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.5
Version 5.2.5 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.4
Version 5.2.4 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x. The following sections cover new features and operational changes in this release.
Tech Preview: Setting Configuration Parameters Within Queries
A new SETTING clause may be specified before a query so that it runs with specific values applied for one or more configuration parameters. Contact Technical Support for details.
Operational Changes in 5.2.4
Note the following significant changes to behavior in this release.
Increased Memory Requirements
In Version 5.2.4, all queries that involve table scans will see an increase in the amount of memory required for execution. The amount of increase depends on the shape of the query plan (number of tables, columns, column types, and so on). A rough estimate of the range is from 300 MiB to a few GiB. You may need to tune your WLM settings to factor in the new memory sizes. Whatever changes you make, you are strongly advised to dry-run these changes on a development system.
Cursor Behavior
In Version 5.2.4, a number of fixes result in changes to the behavior of cursors. For details about supported cursor moves and fetches, see the descriptions of the DECLARE, FETCH, and MOVE commands in the Version 5.2.4 documentation.
Note: Cursors execute their underlying queries when they are declared, and these queries are tagged as DECLARE operations, not as SELECT operations. Therefore, these queries cannot be restarted by WLM, and they are not subject to the WLM global row limit.
Updated System Catalog
5.2.4 introduces a new version of the system catalog. If you are using database replication, you need to upgrade the source and target of the replicated system at the same time. See Replication Compatibility and Upgrade Process.
Management of Lime Cache During Installation
During installation, it is no longer necessary to manually move the lime cache and delete the files later. This operation is now managed automatically by the installer.
What's New in Version 5.2.3
Version 5.2.3 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.2
Version 5.2.2 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
Reduction in Size of Worker Log Files
Based on customer feedback, a number of changes have been made to reduce the growth and size of the worker log files.
What's New in Version 5.2.1
Version 5.2.1 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.2.x.
What's New in Version 5.2.0
Version 5.2.0 contains the following new features and enhancements. There are also some significant changes in behavior and fixes for critical issues reported by customers.
New Features in Version 5.2.0
The following new features are introduced in Version 5.2.0.
Query Planning Enhancements
A number of enhancements have been made in query planning for this release:
- General improvements in join ordering. This includes the flipping of some LEFT OUTER JOINS to RIGHT OUTER JOINS
- Better use of declared primary and foreign keys in query planning
- Improved estimates of join cost and cardinality of matching key values
- Various improvements and adjustments to the EXPLAIN command output
Minimum Configuration for Andromeda Platform
The Andromeda platform supports a new three-node minimum configuration.
Operational Changes in 5.2.0
The following section describes significant changes in behavior between Version 5.2 and previous releases.
System View Updates
The following system views are changed in Version 5.2:
- sys.query and sys.log_query have significant column changes from prior versions.
- In sys.* views and tables the column "user_name" has been changed to "username".
- sys.replica has a new is_table_level_restart column.
- sys.replica_status and sys.log_replica_status both have a new retries column.
Encryption Keystore Alert
A new user-defined system alert is triggered when the encryption keystore is locked.
ybcli Changes
- The SSD section of the blade status output has been reformatted and now includes the SSD firmware version. Note that on the Andromeda platform the blade status output includes an additional line with BMC and FPGA information.
ybtools Changes
- In previous releases, RPMs for ybtools on RHEL and AIX systems had to be installed in the default location (/opt/ybtools). In Version 5.2, you can specify an alternative path.
- The ybdumpschema and ybdumproles help text has been updated for consistency. In previous releases, there were some unnecessary differences in the output for the two tools.
SQL Changes
- In CREATE TABLE, the ON keyword is now supported for DISTRIBUTE ON RANDOM and DISTRIBUTE ON REPLICATE. Previously, the ON keyword was only supported for hash distribution (DISTRIBUTE ON column).
- The default transaction isolation level can be changed only by internal system users. The only supported transaction isolation level that superusers can set is
read ``committed
. - The DEALLOCATE command now has an IF EXISTS option.
- The ADD_MONTHS function always returns the last day of the result month when the input expression is the last day of the month. For example, 2000 was a leap year. Therefore, 2000-02-28 + 1 month returns 2000-03-28, and 2000-02-29 + 1 month returns 2000-03-31.
Known Issues in Version 5.2.x
You may encounter the following issues in Version 5.2.x. Use the workarounds provided in the description and contact Customer Support for additional information.
Issue # | Description |
---|---|
27992 | Database restore operations sometimes fail and report the following error:YCATALOGRESTORE: Cannot find mapping for foreign key If you run into this problem, contact Customer Support for a workaround. |
24660 | In Versions 5.2.18 and earlier, a rare race condition may occur when using joins that involve spilling, resulting in the query failing and ultimately causing a worker crash. In Version 5.2.19 and later versions, this type of query no longer causes a worker crash when it fails, but it does return an exception. |
20786 | Running EXPLAIN ANALYZE on any write statement (INSERT, UPDATE, DELETE) modifies the underlying table, without warning. To work around this problem, run the EXPLAIN ANALYZE statement inside a transaction so you can roll it back if needed. |
20361 | The sys.log_query_pre5 view contains query execution summaries for queries run prior to the 5.x upgrade. Only superusers are allowed to access this table. |
20182 | You must create encryption keys in the same database as the tables that will reference them. Do not use cross-database references to encryption keys in your CREATE TABLE statements. If you do this, you will lose those references when the database is backed up and restored (or replicated). |
19980 | After upgrading from 4.1.x to 5.2.x, you may encounter the following error during database replication: `YCATALOGRESTORE: duplicate key value violates unique constraint "pg_type_typname_nsp_index" If you run into this problem, contact Customer Support for a workaround. |
19886 | After changing the spill space percentage on your system (Temporary Space Reservation setting in the SMC), you must stop and restart the database, using ybcli. Otherwise the change will not take effect. |
19578 | Selecting from the sys.query view returns incorrect results when a WHERE clause condition contains a NOT LIKE operator. For example:SELECT query_id, type, username<br>FROM sys.query<br>WHERE username NOT LIKE 'sys_ybd%'; This problem does not occur for queries against other system views or user tables. |
15784, 16097 | When the row store becomes full (which triggers a MAJOR alert), DELETE operations may fail and some statistics about running queries may be lost. |
16101 | The SMC does not display row counts in the row store (for example, in the Table Details screen). |
Issues Fixed in Version 5.2.x
The following issues were fixed in Version 5.2.x.
Release | Issue # | Description |
---|---|---|
5.2.26 | 32559 | Using the JSON_LOOKUP function with the jpointer option when JSON documents contained empty strings caused database instability. In addition, using the JSON_INGEST function followed by the JSON_LOOKUP function containing the ingested JSON column and jpath option returned the following error: Error: malformed input |
5.2.26 | 32530 | In some instances, the datepart boundary feature in DATEDIFF functions was incorrectly applied for hour, minute, second, epoch, and millisecond dateparts, causing an extra second to be added to the results. |
5.2.26 | 32420 | An internal error in Postgres prevented "Configure > Users" in the SMC from running. |
5.2.26 | 32092 | The GETBIT function returned wrong results when used with BIGINT columns. |
5.2.26 | 31747 | The description of the contents of a backup bundle was improved in the Version 5.2.26 documentation. |
5.2.26 | 30805 | If a temporary table had rows in the rowstore, selecting from the sys.table_info view would return the following error:ERROR: OID out of range |
5.2.26 | 29053 | When replicating from 4 chassis to 8 chassis, incremental replication failed on the source system with a timeout error. |
5.2.26 | 28137 | Under certain circumstances, parity rebuild after a drive replacement would take longer than expected. |
5.2.26 | 27777 | When the ybrelay client requested a commit, the ybrelay server did not wait to receive all of the data to be written to the database, especially when decompression of the data was time-consuming. In Version 5.2.26, the shutdown process has been improved so that all the data is committed |
5.2.26 | 26078 | Under certain circumstances after a lime restart, some lime queries would fail with the following error:KE032 Attempt to use spill space above limit causing the SMC "Worker CPU Utilization" and "Worker Memory Usage" dashboards to not display current statistics and the sys.log_query view to not be populated. |
5.2.24 | 28973 | In stored procedures performing small inserts to the rowstore, all INSERT statements within BEGIN...EXCEPTION...END blocks were ignored until a new block was started or the end of the transaction was reached, causing data to be missing from the results of the full transaction. |
5.2.23 | 29595 | Under certain conditions, a query with multiple IN lists would sometimes return wrong results. |
5.2.23 | 29495 | Use of the --log-level and --logfilelog-level ybrelay options were not reflected in the ybrelay logs. |
5.2.23 | 28679 | Some DDL files were missing from the Windows and Java-based ybtools downloads, causing problems with Kerberos configuration. |
5.2.23 | 28655 | When output CHAR columns were not referenced consistently in the GROUP BY clause (with or without table qualifiers), the query returned the following error:ERROR: variable not found in subplan target list |
5.2.23 | 27793 | Prepared statements with typemod arguments that were executed with a generic plan produced wrong results. |
5.2.23 | 27628 | In some instances, a cursor fetch query in "client wait" state was not canceled during quiesce. |
5.2.23 | 27470 | Default WLM restart rules were applied three times and for a large number of error codes. In Version 5.2.23, a smaller subset of error codes triggers a restart, and an attempted restart occurs only once per query. |
5.2.23 | 26268 | A hung query in idle state was not present in the sys.query view and would not respond to yb_terminate_session or pg_termination_backend. In Version 5.2.23, query cancellation has been improved for certain code paths in the planner. |
5.2.22 | 27998 | In a query with GROUP BY and LIMIT clauses, a rare race condition between spilling and cleaning code would sometimes result in a cluster restart. |
5.2.22 | 27956 | Joining on strings where one string column is small (<=30) and the second string column is larger (>30) would return wrong results. |
5.2.22 | 27272 | Some queries with COUNT(DISTINCT) functions would return incorrect results because of problems with the join order. |
5.2.22 | 27155 | A problem with the rewrite order involving CHAR data types would sometimes result in an internal error. |
5.2.22 | 27147 | A query with multiple subqueries containing the CONCAT function caused a system crash during the planning phase. |
5.2.22 | 26770 | Joins on CHAR columns performed poorly because the planner added a redundant TRIM function for variable CHAR strings, which prevented bloom filters from being applied. |
5.2.22 | 26700 | The third-party syslog-ng package was upgraded to fix a slow file descriptor leak. This leak occurred when the internal log files kept on each CMP were rotated, and triggered a DCS alert. |
5.2.22 | 26487 | The documentation was updated to clarify that SUBSTRING and SUBSTR are not equivalent functions. SUBSTRING returns a VARCHAR data type, but SUBSTR returns the data type of its input string. |
5.2.22 | 26365 | Running a stored procedure containing loops and an explicit exception statement would cause Postgres to crash if exception handling was involved. |
5.2.22 | 26302 | The Yellowbrick installer proceeded as far as the firmware upgrade step, then stopped progressing and failed to time out. |
5.2.22 | 26248 | A distribution logic issue for SEMI LEFT joins caused redundant redistribution. |
5.2.22 | 26082 | In some instances, a NOT restriction of correlated subqueries was incorrectly applied, causing wrong results. |
5.2.22 | 25960 | After setting a configuration parameter, using SET SESSION ROLE and SET LOCAL ROLE commands in succession within the same transaction caused a Postgres crash. |
5.2.22 | 25834 | The IN-list rewrite optimization now works with all comparison operators. In previous releases, only the ‘=‘ and ‘<>’ operators were supported for this optimization. |
5.2.22 | 23982 | When a regular user executed a long-running SQL statement, the query was not visible in the SMC. |
5.2.22 | 23274 | Rolling back a transaction with an active backup snapshot would sometimes cause a system crash. |
5.2.22 | 22873 | Cluster reconfiguration during a system restart could, in some rare cases, result in a deadlock because the self-healing mechanism could block indefinitely. In Version 5.2.22, the self-healing mechanism is more deterministic. |
5.2.21 | 26228 | Some equality conditions on CHAR values with trailing spaces returned wrong results. |
5.2.21 | 25417 | On the Andromeda platform, logins to the SMC were slow. In Version 5.2.21, to mitigate this problem, the ybd-telegraf service is disabled by default. |
5.2.21 | 25229 | For security reasons, the google-gson dependency for backup and restore was upgraded to 2.9.1. |
5.2.21 | 25019 | A query with multiple rewritten IN lists that constrained the same constants and referenced these constants in the GROUP BY clause returned the following error:Variable not found in subplan target list |
5.2.21 | 20376 | Database users granted "Manager" privileges in the SMC were unable to view queries executed by other users. |
5.2.21 | 15709 | A query plan with a multi-column join and no filtering predicates did not apply a range restriction, resulting in a full scan on a large fact table. |
5.2.21 | 13312 | The documentation was updated to clarify the description of the last_value column in the sys.sequence view. |
5.2.20 | 24898 | CREATE EXTERNAL TABLE failed with the following error after a large number of table creations:OutOfMemoryError: Direct buffer memory |
5.2.20 | 24784 | A common table expression (CTE) with an ORDER BY clause returned wrong results. |
5.2.20 | 24779 | ACLs defined with ALTER DEFAULT PRIVILEGES on the replication source system were not replicated to the target appliance. To change this behavior, add the following option to the WITH clause of the ALTER DATABASE ADD REPLICA statement:user_resolution_mode 'PLACEHOLDER' |
5.2.19 | 24157 | A worker crash caused by a race condition could occur while using partitioned tables when join queries involved spilling. |
5.2.18 | 29039 | The Yellowbrick client tools (ybtools) now support AIX 7.2. |
5.2.18 | 24255 | Views created in Version 5.2.15 and earlier that include CASE expressions would sometimes return errors. |
5.2.18 | 24234 | The returned rows counter was not always reset for some restarted queries, causing the maximum rows limit to be reached before the restarted query actually returned the maximum number of rows. |
5.2.18 | 23977 | During an upgrade, some tables appeared to be vacuumed when they were not, sometimes causing SQL statements to return the following error:ERROR: could not access status of transaction <br>DETAIL: Could not open file "pg_clog/0000" |
5.2.18 | 23912 | An IN list rewrite would sometimes return wrong results because data was not cast to the correct data type. |
5.2.18 | 23871 | During an upgrade, a table containing duplicate keys that returned a duplicate key value error would cause the upgrade to fail. |
5.2.18 | 23869, 23826 | A memory allocation issue would cause a crash in the manager node stack. |
5.2.18 | 23741 | In some instances, a hung monitoring service was not terminated, causing a manager node failover to be unsuccessful and resulting in both manager nodes becoming secondary. |
5.2.18 | 23694 | Insufficient system pool memory would cause some GC operations to stall. |
5.2.18 | 23307 | A hung query was not present in the sys.query view and would not respond to yb_terminate_session, preventing a GC operation. |
5.2.18 | 23305 | When a blade was being booted, the ybcli blade status command incorrectly reported "Cluster Status: ERROR." |
5.2.18 | 23094 | Using ybcli maintenance mode for DNS changes could cause issues with the port listener. |
5.2.18 | 22715 | Version 5.2.18 improves the ability to set multiple domains on the network manager. |
5.2.18 | 22613 | When executed in an aborted transaction, a ROLLBACK TO SAVEPOINT query would not be removed from the sys.query view. |
5.2.18 | 21722 | When implicit casting was used for subqueries that included ORDER BY, GROUP BY, or DISTINCT clauses, the clauses were applied to the incorrect data type, sometimes resulting in a Postgres crash. |
5.2.18 | 20015 | An upgrade issue would prevent autovacuum from running on some tables, sometimes causing Postgres to emit the following warning:WARNING: some databases have not been vacuumed in over 2 billion transactions |
5.2.17 | 23064 | After an upgrade to 5.2.13 or 5.2.14, replication source databases unable to reach the target system were put in READONLY mode. In Version 5.2.17, the system retries connections to the target system for 10 minutes, bringing the source database out of READONLY mode if the target is available. |
5.2.17 | 22800 | Improvements were made to the memory trimming behavior on the manager nodes to reduce memory consumption. |
5.2.16 | 23023 | Using DISTINCT inside the MEDIAN function would cause the system to crash. |
5.2.16 | 22886 | After an upgrade, some front-end queries would return the following warning:WARNING: out of shared memory. |
5.2.16 | 22755 | Incorrect implicit casting of CHAR values containing only digits in INSERT queries returned an error. |
5.2.16 | 22750 | Parsing a query with a very deeply nested case expression could cause out of memory errors and a system crash. |
5.2.16 | 22633 | Toast tables and indexes related to schema-excluded objects were not excluded during the replication cycle, resulting in orphaned relations and ultimately causing a database restart. |
5.2.16 | 22462 | A CTAS statement that included a common table expression caused a Postgres crash. |
5.2.16 | 21746 | Out-of-memory (OOM) conditions would sometimes occur within the cluster manager when a large number of blades were in use. |
5.2.16 | 20674 | A CREATE VIEW statement with a CASE expression now returns VARCHAR columns with the correct width. |
5.2.16 | 20377 | Entries from yb_deletes tables were included in sys.aalog (AutoAnalyzeLog), resulting in a reduction of disk space and slower auto-analyze performance. |
5.2.16 | 19908 | Selecting all from sys.view took longer to return results than selecting all from other system views. |
5.2.15 | 22308 | Character strings inside a NULLIF expression returned wrong results because auto-trimming was not applied to them. |
5.2.15 | 22168 | Misleading histogram statistics were being created for certain table distributions, leading to inefficient query plans and slower performance. |
5.2.15 | 22140 | The log4j components for ybtools were upgraded to address security concerns. See What's New in Version 5.2.15. |
5.2.15 | 22113 | After an upgrade to 5.2, replication failed on the source system with backup errors because of a problem with duplicate "relfilenodes." |
5.2.15 | 15303, 22454 | After an upgrade, starting replication sometimes returned the following error:Replication can't proceed due to system upgrade in process. and "ybcli system status" command incorrectly reported shard rewrite as incomplete. |
5.2.14 | 22117 | When a replicated view contained a UDF that was used by a subquery, selecting from the view on the destination database returned a "cache lookup failed" error. |
5.2.14 | 22106 | A query with a very large IN list returned a "KE002: Out of Memory" error. |
5.2.14 | 21439 | The presence of unicode characters in short character strings could cause incorrect compression and wrong results when the strings were compared. |
5.2.14 | 21281 | On the Andromeda platform, spilling scalability was improved for aggregations. |
5.2.14 | 20458 | Join queries run under low-memory conditions (in a slot only a little larger than the estimated memory requirement) triggered repeated spill cycles and failed with an assertion error. |
5.2.14 | 17916 | The system could be left in a hung state when queries with very complex regular expressions were submitted. In Version 5.2.14, when a regex expression is too complex, the query returns an error that specifies the regex pattern that caused the problem:Regex '<pattern>' is too complex. Exceeded maximum number of state transitions In addition, "yielding" inside the regex engine has been introduced, such that before a regex pattern is declared as "too complex," system stability is improved for complex patterns that do not reach the maximum number of state transitions. |
5.2.13 | 21828 | The immediate ANALYZE operation that runs after a bulk load could not be disabled. See Disabling ANALYZE Operations for ybload. |
5.2.13 | 20896 | When a join key was also the target of a constant filter, the selectivity of the join was incorrectly estimated. |
5.2.12 | 21737 | Failure of a GC operation resulted in a "Metadata storage full error." |
5.2.12 | 21734 | Performance of scans against the sys.fsinfo virtual table was improved. |
5.2.12 | 21717 | INSERT statements that required implicit casts on numeric constants inserted garbage values into the table. |
5.2.12 | 21711 | An internal setting for worker spill space could not be retrieved and caused a Postgres crash. |
5.2.12 | 21631 | In Version 5.2.11, verbose mode was turned off because it caused a problem during installer rollback. In Version 5.2.12, the installer issue has been fixed and verbose mode is turned on by default. |
5.2.12 | 21519, 21522 | When a backup contained only a large number of table deletes, a ybrestore operation exceeded a configurable timeout in the restore service, causing the restore to error out although it was complete. |
5.2.12 | 21293 | On 3-chassis and 4-chassis systems, the server crashed while running queries against the sys.fsinfo virtual table. |
5.2.11 | 21608 | In Version 5.2.10, verbose mode was set as the default for upgrades. In Version 5.2.11, verbose mode has been turned off because it causes a problem during installer rollback. |
5.2.11 | 21389 | Certain join queries returned a Generic UnexpectedException error when they contained a LIMIT clause but no ORDER BY clause. |
5.2.11 | 21380 | Memory for all joins was over-estimated as a result of a previous fix for skew handling. See Increased Memory Estimation. |
5.2.11 | 21069 | A rare, intermittent worker crash caused by a race condition could occur on the Andromeda platform when queries involved spilling. |
5.2.11 | 21262, 20958, 20946, 20470 | Global parameters set in PostgreSQL configuration files and SMART data for the manager node drives did not appear in the results of the system diagnostics command. In Version 5.2.11, all of this information is included. The diagnostics package now also includes the contents of sys.vt_cluster_manager_events and the check_xfs.py script. |
5.2.11 | 21249, 21233, 21387 | To ensure that the Yellowbrick server and client tools are not subject to known log4j vulnerabilities, unused packages and their dependencies were removed from the log4j distribution used by Yellowbrick. See Protection from log4j Vulnerabilities. |
5.2.10 | 21297 | To optimize query planning for nested-loop joins, cost calibration was improved. |
5.2.10 | 21166 | Postgres crashed because of a problem with processing ACL settings when roles were set with the ALTER ROLE command. |
5.2.10 | 21079 | When a stored procedure that created a table with a CLUSTER ON clause was executed, Postgres crashed. |
5.2.10 | 20640 | The ybloadcat and ybdumpcat logs were missing during an upgrade. In Version 5.2.10, verbose mode is the default for upgrades so that pg.log files are redirected to the ybinstaller.log. |
5.2.10 | 19375 | A restore failure was caused by missing deletes in the backup bundle. |
5.2.9 | 21182 | For loop joins, the planner sometimes did not choose the smaller table to hold in memory. |
5.2.9 | 20958 | System diagnostics results now include the settings for Postgres configuration parameters, including the output of SHOW ALL and global Postgres configuration files. |
5.2.9 | 20957 | The ASCII function is now included in the Version 5.2 documentation. |
5.2.9 | 20941 | A transaction-consistency check in the FSCK code resulted in an assertion error. |
5.2.9 | 20940 | Join elimination could cause wrong results for LEFT joins when primary and foreign keys did not have referential integrity. In Version 5.2.9, a workaround is available; see Disabling Join Elimination for LEFT Joins. |
5.2.9 | 20886 | Backups taken over AWS Private Link failed with a timeout error. |
5.2.9 | 20849 | The description of the w.resourcePool property for WLM rules was improved in the Version 5.2 documentation. |
5.2.9 | 20835 | The description of the ybbackup --exclude option was improved in the Version 5.2 documentation. |
5.2.9 | 20821 | A section was added to the Version 5.2 backup and restore documentation to explain how to give users access to encrypted table columns in a restored database. |
5.2.9 | 20786 | The EXPLAIN ANALYZE documentation was updated to clarify that write statements (INSERT, UPDATE, DELETE) used with EXPLAIN ANALYZE commit permanent changes to the target table if they are not run within a transaction that can be rolled back. |
5.2.9 | 20770 | If an explicit numeric cast was used in an expression, the resulting data type was adjusted to fit the value of the result. In Version 5.2.9, the return type is based on the requested explicit cast. See Explicit Casts for Numeric Calculations. |
5.2.9 | 20760 | Postgres crashed because of a problem with processing NOT IN expressions within subqueries. |
5.2.9 | 20750 | Restore failures could occur when tables were not restored in the optimal order. |
5.2.9 | 20722 | At some point in the cycle, a database restore would hang. All activities dropped to 0, as indicated by the read/write network statistics in the log. |
5.2.9 | 20712 | The used_bytes column in the sys.storage view is described incorrectly in the Version 5.3 documentation. The used_bytes value is not a precise sum of other values in the view. You cannot derive the exact amount of space used per worker; used_bytes is an approximation, roughly equal to:distributed_bytes + replicated_bytes + random_bytes + scratch_bytes + some overhead |
5.2.9 | 20619 | Detection of composite primary-key/foreign-key relationships did not take the error margin for distinct values into account, resulting in the wrong join order being chosen. |
5.2.9 | 20517 | The Infiniband route-determination algorithm was changed from Min Hop to LASH to prevent issues in large multi-chassis systems. |
5.2.9 | 20466 | Because of a memory-estimation problem, certain queries intermittently returned the following error:ERROR: query ran out of memory in pool "Small" |
5.2.9 | 20407 | The NVL/COALESCE function returned wrong results because trailing space characters were not trimmed correctly. |
5.2.9 | 20289 | The macOS version of ybsql Version 5.2.4 crashed on startup. |
5.2.9 | 20172 | DECIMAL values lost precision when implicit casting was used for subquery results in set operations, such as UNION and UNION ALL. |
5.2.9 | 17371 | A CTAS query that returned an error during parsing or planning logged a type of "create table as" in sys.log_query instead of the expected "ctas" type. |
5.2.8 | 20701 | Under certain conditions, an INSERT query that was restarted in a different WLM resource pool could cause data loss. For more details, request the Support Bulletin on this topic from Customer Support. |
5.2.8 | 20639 | Timeouts for Postgres shutdown during installation were increased to prevent upgrade failures. |
5.2.8 | 20633 | The documentation on cross-database queries has been improved to clarify that writes to remote tables are not supported. |
5.2.7 | 20653 | A security issue made it possible for users to drop database objects that they did not own. For more details, request the Security Bulletin on this topic from Customer Support. |
5.2.7 | 20624 | The output of ybdumpcat now includes the number of rows exported from each table. |
5.2.7 | 20622 | Improvements were made to the preupgrade script to log deletes for system catalog tables. |
5.2.7 | 20621 | Log messages were added to the pg.log when table delete records were inserted for pg_class or pg_type. When a DROP TABLE is executed, check the pg.log for a line of the form:inserted td record oid <oid of dropped table> in relid 1259... |
5.2.7 | 20448 | A query against the sys.query view with an IN list condition returned wrong results. |
5.2.7 | 20356 | Under memory pressure, a rare issue occurred with compute blades writing compiled queries to disk. |
5.2.7 | 18058 | When queries failed with out of memory errors, a race condition resulted in a worker crash. |
5.2.6 | 20492 | Database restore operations could hang when network environment settings caused the termination of connections that were idle for a long time. In Version 5.2.6, the length of time that a network connection can remain idle for large data transfers has been reduced. |
5.2.6 | 20450 | The compiler was overly aggressive in its memory consumption for large queries. |
5.2.5 | 20425 | Declaring a cursor on a query containing a single count(*) in the projection list resulted in an error. |
5.2.5 | 20381,20394, 20396 | In rare instances, the cancellation of a stored procedure executing multi-table scans failed and resulted in a worker crash. |
5.2.5 | 20357 | Using a WLM throttle rule incorrectly returned error code WL012. |
5.2.5 | 20311 | A race condition prevented the server from starting up. |
5.2.5 | 20183 | The cancellation of a holdable cursor query in "client wait" state was prevented until initial transaction was completed. |
5.2.5 | 19658, 20149 | An aggregate query ran successfully on the Tinman platform but failed on the Andromeda platform with a "Failed to schedule task" error. |
5.2.5 | 18556 | When restoring large backup bundles, restore operations sometimes failed and reported an "invalid key" error. |
5.2.4 | 20139 | Customers requested a way to set configuration parameters for a specific query. See Setting Configuration Parameters Within Single Queries. |
5.2.4 | 20127 | A workaround was implemented for a problem with missing table deletes during replication. Contact Customer Support for details. |
5.2.4 | 20059 | Role-specific settings defined with ALTER USER/ROLE were not applied for users running Java ybtools such as ybload and ybunload. |
5.2.4 | 20057, 19954, 19939, 19728 | Multiple issues with cursors have been addressed, including one causing server crashes, and updates to the cursor implementation have been made. See the Version 5.2.4 documentation for details about cursor fetch and move support. |
5.2.4 | 19992 | Version 5.2.4 increases memory requirements for all queries with table scans. See Increased Memory Requirements. |
5.2.4 | 19983 | Slow query processing of the regexp_replace function could cause a system crash. In Version 5.2.4, performance of regexp_replace processing has been improved. |
5.2.4 | 19981 | A stored procedure had the wrong return type in pg_proc, causing a problem when the ybdumpschema tool was run. |
5.2.4 | 19961 | Installation on systems with large lime and query caches has been optimized in Version 5.2.4. |
5.2.4 | 19956 | After a successful replication cycle, the statistics on the target system were not correct. |
5.2.4 | 19926 | In the SMC, the Monitor Load Processes screen listed job names as "n/a." |
5.2.4 | 19887 | An aggregate query that spills to disk ran successfully on the Tinman platform but failed on the Andromeda platform with an out-of-memory error. In Version 5.2.4, significantly less spill space is requested for this type of query. |
5.2.4 | 19877 | When there was data in the row store, dropping a NOT NULL constraint on a column with an ALTER TABLE command resulted in an internal error, and the table data became inaccessible. |
5.2.4 | 19580 | The cluster became unresponsive for a query with a regex expression that used greedy quantifiers. |
5.2.4 | 18452 | The Execution Timeline in the SMC displayed only the query ID for a running query; no other information was visible. |
5.2.3 | 19974 | A function was added to repair Version 5.2.x views with incorrect joins (see issue 19973). Contact Customer Support for assistance. |
5.2.3 | 19973 | Changes to join ordering introduced in Version 5.2.0 caused unintended changes to joins used in views. For example, LEFT joins became FULL joins. In Version 5.2.3, the original join ordering is restored for views. |
5.2.2 | 19801 | A Postgres crash occurred when the result of a correlated subquery was compared with the result of a CURRENT_DATE function. |
5.2.2 | 19782 | Running the pre-upgrade script resulted in a deadlock when orphan dependencies were found in pg_depend. |
5.2.2 | 19755 | A Postgres crash occurred with a SIGSEGV error because the pg_get_ruledef() function did not correctly resolve late-bound views. |
5.2.2 | 19723 | During planning for ANTI-joins, statistics were assigned incorrectly to the two sides of the join, causing an internal error. |
5.2.2 | 19714 | Excessive amounts of data were being captured for remote diagnostics. A number of changes were made to reduce the growth and size of the worker log files, including shortening references to UUIDs for hostnames and logging only the first 512 characters of SQL text. |
5.2.2 | 19699 | An INSERT INTO SELECT or CTAS with an ORDER BY clause resulted in a hang caused by a race condition. |
5.2.1 | 19697 | A restore operation failed to automatically create the restored database if the username that initiated the restore contained characters that require escaping (such as periods and hyphens). After a full system upgrade, backups taken with previous releases can be restored successfully in 5.2.1. |
5.2.1 | 19635 | Creating or dropping database in the SMC returned an exception:Connection Error: database "null" does not exist |
5.2.1 | 16474 | When the result of an IN list subquery required a cast, the query failed with the following error:Missing constant for test expression In Version 5.2.1, the cast that is applied to the result of an IN list subquery is pushed down into the subquery. |
5.2.0 | 19349, 18348 | When a user attempted a password change, the SMC locked out other users and returned a "Service is unavailable" message. |
5.2.0 | 19072 | The Postgres acl_explode() function returned the following error when the ACL abbreviation "Z" was part of the input to the function:SQL: ERROR: unrecognized aclright:8388608 |
5.2.0 | 18922 | The MONTHS_BETWEEN function did not work fully as documented. In Version 5.2, all months are considered to have 31 days, and the result is an integral value if both month inputs either specify the same day or specify the last day of the month. |
5.2.0 | 18671 | False PSU alerts were being issued from the CMPs. |
5.2.0 | 18529 | A problem with the decorrelation of expressions in a select-list subquery resulted in the following error:SQL Error [XX000]: ERROR: variable not found in subplan target list. |
5.2.0 | 18046 | After an upgrade from Version 3.x, the word BATCH could not be used as an alias unless the AS keyword preceded it. |
5.2.0 | 17502 | The ybdumpschema tool returned unexpected "dependency loop" warnings. In Version 5.2, these warnings are no longer emitted. |
5.2.0 | 17206, 17020 | In Version 5.2, a new user-defined system alert is triggered when the encryption keystore is locked. |
5.2.0 | 16589 | Expressions contained in join conditions are now pushed down and evaluated prior to the join operation in the case of cardinality-increasing joins. |
5.2.0 | 15365 | The ADD_MONTHS function returned the wrong result when the input expression was a February date in a leap year. |
5.2.0 | 14560 | When a non-existent fully qualified column was referenced in a query, the following error was returned: "Whole row references are not allowed." The correct error, returned in Version 5.2, is "column column_name does not exist." |
CVE Fixes in Version 5.2.x
The following CVE is fixed in Version 5.2.26:
Release | CVE Fix # | Issue # |
---|---|---|
5.2.26 | CVE-2021-42392 | 29538 |
Prior Releases
For information on releases prior to those listed above, please contact Yellowbrick Support
Parent topic:Yellowbrick Documentation