Skip to content

5.2.27 Yellowbrick Release Notes

Release Date: 11/25/2024

Upgrade Requirements 

Read the following information before beginning an upgrade to Version 5.2.

Important

Customers should always contact Customer Support ahead of the planned upgrade date to review the process and schedule the upgrade.

Upgrades from Version 5.x

If you are upgrading from a version prior to 5.2.10, additional steps may be required to upgrade the cluster. Customer Support will walk you through the process.

For upgrades from 5.2.9 or earlier: Before starting the upgrade, on the version you are upgrading from, run:

ALTER SYSTEM SET autovacuum TO 'off';
SELECT pg_reload_conf();

After a successful upgrade, when the database is up, run:

ALTER SYSTEM SET autovacuum TO 'on';
SELECT pg_reload_conf();

For upgrades from 5.2.0, 5.2.1, and 5.2.2: An additional step at the end of the upgrade process is required to update view definitions. Technical Support will walk you through the process.

See also Replication Compatibility and Upgrade Process.

Upgrades from Version 4.x

Be sure to read Changes in Behavior and Compatibility.

For upgrades from 4.1.15 or earlier: Before starting the upgrade, on the version you are upgrading from, run:

ALTER SYSTEM SET autovacuum TO 'off';
SELECT pg_reload_conf();

After a successful upgrade, when the database is up, run:

ALTER SYSTEM SET autovacuum TO 'on';
SELECT pg_reload_conf();

Changes in Behavior in This Version

  • 5.2.23

    Behavior of the CHAR Data Type

    The CHAR data type implementation in Yellowbrick 5.x has resulted in some behavioral inconsistencies.

  • 5.2.23

    Default Behavior of WLM Restart Rules

    By default, a subset of recoverable error codes triggers an attempted restart, and an attempted restart occurs only once per query.

    For details, see "Restarting Queries" in the updated Version 5.2.23 documentation.

    You may need to update your existing WLM query restart rules as a result.

  • 5.2.18

    Query Plans for GROUP BY in Decorrelated Subqueries

    In previous versions, the GROUP BY clause in a decorrelated subquery of the following form:

    SELECT ...
      FROM t1
      WHERE x IN (
        SELECT y
        FROM t2
        WHERE t2.x = t1.x
        GROUP BY y
      )

    was always executed as part of the query plan. In Version 5.2.18, a GROUP BY clause in this kind of query is only executed if statistics suggest that its inclusion will improve performance. Therefore, you may see a change in the plan for this type of query.

  • 5.2.11

    Increased Memory Estimation for Some Statements

    Queries may experience slightly higher memory estimation, up to a maximum of approximately 500MB more than was used in Versions 5.2.8 and earlier. The actual increase depends on how much disk space is available on each worker. This change in estimation corrects an issue in Version 5.2.9 where memory estimation was set too high and out-of-memory (OOM) conditions could occur when a query started spilling.

    This may affect the WLM pool that your existing WLM rules assign statements to.

  • 5.2.9

    Deprecated Feature: Restartable Replication

    The restartable replication feature that was introduced as a tech preview in Version 5.2.0 is deprecated and disabled in Version 5.2.9. This feature is not supported under any circumstances for use on production systems; if you would like to use it purely for testing purposes, contact Customer Support.

  • 5.2.9

    Query Planning Costing Changes

    Default constants for costing have been changed to optimize memory consumption for queries. This may affect the WLM pools statements are assigned to under your existing WLM rules.

  • 5.2.9

    Explicit Casts for Numeric Calculations are not Ignored

    In previous versions, an explicit cast for a numeric expression was ignored, and the return type was adjusted to fit the precision and scale of the actual value. In Version 5.2.9, such explicit casts are respected, and the return type of the expression is based on the requested precision and scale.

    For example, in Version 5.2.9 the following CTAS statement returns DECIMAL(21,10):

    yellowbrick_test=# create table t as
    select 3.14159::NUMERIC(20,10) + 1::NUMERIC(1,0) from sys.const;
    SELECT 1
    yellowbrick_test=# \d t
    Table "public.t"
      Column   | Type           | Modifiers
    ----------+----------------+-----------
      ?column? | numeric(21,10) |

    The same statement in earlier 5.2.x versions returns DECIMAL(11,10):

    yellowbrick_test=# create table t as
    select 3.14159::NUMERIC(20,10) + 1::NUMERIC(1,0) from sys.const;
    SELECT 1
    yellowbrick_test=# \d t
    Table "public.t"
      Column  | Type           | Modifiers
    ----------+----------------+-----------
    ?column? | numeric(11,10) |
  • 5.2.4

    Increased Memory Requirements for table scans

    In Version 5.2.4, all queries that involve table scans will see an increase in the amount of memory required for execution. The amount of increase depends on the shape of the query plan (number of tables, columns, column types, and so on). A rough estimate of the range is from 300 MiB to a few GiB for quereis with a large number of columns or very wide string columns..

    You may need to tune your WLM rules, particularly if pool assignments are not based on the planner memory estimated sizes. If your WLM rules are based on planner memory estimates, you may see statements now running in larger pools. Whatever changes you make, we advise a dry-run of these changes on a development system.

  • 5.2.4

    Cursor Behavior Changes

    In Version 5.2.4, a number of fixes result in changes to the behavior of cursors including thier type for WLM rule and sys.log_query classifcation. For details about supported cursor moves and fetches, see the descriptions of the DECLARE, FETCH, and MOVE commands in the Version 5.2.4 documentation.

    Note: Holdable cursors execute their underlying queries when they are declared, and these queries are tagged as DECLARE operations, not as SELECT operations. Therefore, these queries cannot be restarted by WLM, and they are not subject to the WLM global row limit.

  • 5.2.4

    Updated System Catalog

    5.2.4 introduces a new version of the system catalog. If you are using database replication, you need to upgrade the source and target of the replicated system at the same time. See Replication Compatibility and Upgrade Process.

  • 5.2.0

    System View Updates

    The following system views are changed in Version 5.2:

    • sys.query and sys.log_query have significant column changes from prior versions.
    • In sys.* views and tables the column "user_name" has been changed to "username".
    • sys.replica has a new is_table_level_restart column.
    • sys.replica_status and sys.log_replica_status both have a new retries column.
  • 5.2.0

    ybcli Changes

    • The SSD section of the blade status output has been reformatted and now includes the SSD firmware version. Note that on the Andromeda platform the blade status output includes an additional line with BMC and FPGA information.
  • 5.2.0

    SQL Changes

    • In CREATE TABLE, the ON keyword is now supported for DISTRIBUTE ON RANDOM and DISTRIBUTE ON REPLICATE. Previously, the ON keyword was only supported for hash distribution (DISTRIBUTE ON column).
    • The default transaction isolation level can be changed only by internal system users. The only supported transaction isolation level that superusers can set is read committed.
    • The DEALLOCATE command now has an IF EXISTS option.
    • The ADD_MONTHS function always returns the last day of the result month when the input expression is the last day of the month. For example, 2000 was a leap year. Therefore, 2000-02-28 + 1 month returns 2000-03-28, and 2000-02-29 + 1 month returns 2000-03-31.

Changes in Compatibility in This Version

  • 5.2.0

    ybtools Compatibility

    Yellowbrick recommends that you always upgrade ybtools to match the Yellowbrick server version you are running (for example, Version 5.2). Using Version 5.2 of ybtools with an older server version (such as Version 4.1 or 5.1) may result in error messages for some commands. Using previous versions of ybtools with a Version 5.2 database is likely to work but is deprecated and not supported. See also the following section: Backward Compatibility for Backup and Restore.

    Important: On CentOS and Red Hat client platforms, you must first remove any existing 3.x version of ybtools. Then you can proceed with the installation of the Version 5.2 ybtools. You cannot upgrade directly from an earlier 3.x version of ybtools to Version 5.2.

  • 5.2.0

    Replication Compatibility and Upgrade Process

    Important: Both systems used for database replication (source and target) must be running the same major version of the database software. For example, you cannot replicate from a 4.x source to a 5.x target or vice versa.

    In Version 5.2.19, you cannot replicate to a target system on any earlier version; both systems must be running Version 5.2.19 of the database software. For example, you cannot replicate from a 5.2.19 source to a 5.2.18 target. This behavior is enforced because of a change in the system catalog for Version 5.2.18.

    You can replicate a Version 5.x database to a later 5.x database (such as 5.1.x to 5.2.19). For example, if you are upgrading from Version 5.1.x, and you want to reverse replication in Version 5.2 (using PROMOTE), follow these steps:

    1. Pause the replica on the Version 5.1.x source system.
    2. Upgrade the target system to Version 5.2.19.
    3. Verify that replication works from 5.1.x to 5.2.19.
    4. Attempt a promote on the 5.2.19 target. This operation will fail with an expected error because you cannot replicate to a system that is running an earlier version of the software. The replica will remain paused on the 5.1.x source system.
    5. Upgrade the source system to 5.2.19.
    6. Attempt the promote again on the 5.2.19 target system. This operation should succeed.
    7. Attempt a new replication cycle from the reversed source to the new target (both 5.2.18 systems). The replication cycle should succeed on the new target.
  • 5.2.0

    Backward Compatibility for Backup and Restore

    A backup taken in Version 3.2.x, 3.3.x, 4.x, or 5.x prior to a 5.2 upgrade can be restored in Version 5.2.

    After a 5.2 upgrade or installation, the previous version of the backup and restore tools is compatible with Version 5.2. The Version 5.2 ybtools package installs two sets of backup and restore tools: legacy tools and new 5.2 tools.

    The legacy tools are deprecated and will be removed in a future release, along with the ability to restore backups taken in previous releases. There are significant differences in the functionality and client tool options between the legacy tools and the new tools. The tool sets are distinguished by their names:

    ybtools Version Backup Restore List/Manage Backups
    New in Version 4.x and later ybbackup ybrestore ybbackupctl
    Legacy in Version 4.x and later ybbackup-old ybrestore-old ybbackup-list-old
    Version 3.x ybbackup ybrestore ybbackup-list

    To use the legacy tools in the 5.2 package, be sure to use the correct tool name in the command. For example, to return help for the legacy version of ybbackup:

    $ ybbackup-old --help
    Yellowbrick Backup and Restore version 5.2.0-33445
    ...

    Important: To use backup and restore in Version 5.2, you must follow these steps:

    1. Use the legacy version of ybtools (-old) to restore any 3.x backups into 5.2 databases. Use the new 4.x or 5.x version of ybtools to restore any 4.x or 5.x backups into 5.2 databases.
    2. Take new full backups of your 5.2 databases with the new 5.2 ybtools.
    3. Stop using the legacy ybtools for backup and restore purposes. Use only the new tools.

Known Issues in This Version

You may encounter the following issues in Version 5.2.27. Use the workarounds provided in the description and contact Customer Support for additional information.

Issue #Description
27992

Database restore operations sometimes fail and report the following error:

YCATALOGRESTORE: Cannot find mapping for foreign key

If you run into this problem, contact Customer Support for a workaround.

24660

In Versions 5.2.18 and earlier, a rare race condition may occur when using joins that involve spilling, resulting in the query failing and ultimately causing a worker crash. In Version 5.2.19 and later versions, this type of query no longer causes a worker crash when it fails, but it does return an exception.

20786

Running EXPLAIN ANALYZE on any write statement (INSERT, UPDATE, DELETE) modifies the underlying table, without warning. To work around this problem, run the EXPLAIN ANALYZE statement inside a transaction so you can roll it back if needed.

20361

The sys.log_query_pre5 view contains query execution summaries for queries run prior to the 5.x upgrade. Only superusers are allowed to access this table.

20182

You must create encryption keys in the same database as the tables that will reference them. Do not use cross-database references to encryption keys in your CREATE TABLE statements. If you do this, you will lose those references when the database is backed up and restored (or replicated).

19980

After upgrading from 4.1.x to 5.2.x, you may encounter the following error during database replication:

YCATALOGRESTORE: duplicate key value violates unique constraint "pg_type_typname_nsp_index"

If you run into this problem, contact Customer Support for a workaround.

19886

After changing the spill space percentage on your system (Temporary Space Reservation setting in the SMC), you must stop and restart the database, using ybcli. Otherwise the change will not take effect.

19578

Selecting from the sys.query view returns incorrect results when a WHERE clause condition contains a NOT LIKE operator. For example:

SELECT query_id, type, username
FROM sys.query
WHERE username NOT LIKE 'sys_ybd%';

This problem does not occur for queries against other system views or user tables.

15784, 16097

When the row store becomes full (which triggers a MAJOR alert), DELETE operations may fail and some statistics about running queries may be lost.

16101 

The SMC does not display row counts in the row store (for example, in the Table Details screen).

What's New in This Version

5.2.27

Improved cluster reconfiguration time

Cluster reconfiguration time on restart, particularly for appliances with a large number of chassis has been improved.

5.2.26

DLE_DST and LE_DST Functions

These two functions have been added. They return the difference between two character strings in terms of the number of changes required to convert one to the other. The DLE_DST function uses the Damerau-Levenshtein edit distance algorithm and the LE_DST function uses the Levenshtein edit distance algorithm to compute the "distance" between the strings.

5.2.25

Kerberos SSO Support in ybools on Windows

The Yellowbrick client tools now support Kerberos SSO on Windows. To enable and configure Kerberos on a Yellowbrick appliance, contact Yellowbrick Customer Support.

5.2.23

JSON_INGEST and JSON_LOOKUP Functions

Two new SQL functions for JSON manipulation, JSON_INGEST and JSON_LOOKUP, are available in Version 5.2.23. See the updated Version 5.2.23 documentation for details and examples.

5.2.22

Kerberos Support in ybtools

The Yellowbrick client tools (ybtools) now have GSSAPI support, which enables support for Kerberos authentication. To enable and configure Kerberos on a Yellowbrick appliance, contact Yellowbrick Customer Support.

5.2.22

Type-Safe Casting Functions

Three new SQL functions are supported for compatibility with applications developed in other databases. These functions implement type-safe casting from VARCHAR strings to DATE, TIMESTAMP, and DOUBLE data types:

  • SAFE_TO_DATE
  • SAFE_TO_TIMESTAMP
  • SAFE_TO_DOUBLE

For more details, see the SQL Functions section in the Version 5.2.x documentation.

5..2.21

Kerberos Authentication

Version 5.2.21 supports Kerberos Version 5 network authentication. To enable and configure Kerberos on a Yellowbrick appliance, contact Yellowbrick Customer Support.

5.2.15

Yellowbrick Client Upgrades for log4j

To ensure that the Yellowbrick client tools are not subject to known log4j vulnerabilities, the log4j jar used by ybtools has been upgraded to version 2.17.1.

5.2.13

Disabling ANALYZE Operations for ybload

Two configuration parameters are available to control the use of ANALYZE operations that occur after data is written to the database:

ybd_analyze_after_loads

Defaults to ON and triggers an automatic ANALYZE if 10% or more of the rows have changed for the target table when a ybload operation completes. To disable the automatic ANALYZE, set this parameter to OFF. This parameter is new in Version 5.2.13 and applies to workloads that require a large number of continuous trickle-feed ybload operations.

Note that setting this parameter to OFF does not prevent an immediate ANALYZE from being run after intermediate load operations, such as after an INSERT into a target table from a staging table. However, setting ybd_analyze_after_writes to OFF will prevent an immediate ANALYZE in these cases.

ybd_analyze_after_writes

Defaults to ON and continues to work as described in the Version 5.2.x documentation.

Both parameters and their current settings are visible in the output for SHOW ALL.

5.2.11

Protection from log4j Vulnerabilities

To ensure that the Yellowbrick server and client tools are not subject to known log4j vulnerabilities, unused packages and their dependencies were removed from the log4j distribution used by Yellowbrick. Specifically:

  1. The RHEL log4j package and RPMs have been removed from the manager node.
  2. The log4j jar used by Yellowbrick on the manager node has been replaced with yb-log-4.1.16.jar, which does not contain the vulnerable classes (JMSAppender, JMSSink, SimpleSocketServer, and SocketServer).
  3. The log4j jar used by ybtools has been replaced with yb-log-4.1.16.jar, which does not contain the vulnerable classes (JMSAppender, JMSSink, SimpleSocketServer, and SocketServer).

Note that a logger is still required when integrating with Spark, and it is the responsibility of the Spark runtime to provide it.

5.2.9

TO_NUMBER Function

You can convert a number string to an actual numeric value, using a specified format.

Note that the addition of this function introduces a change to the system catalog version.

5.2.9

Support for 2,000 Columns in a Table

The maximum number of user-defined columns in a table has been increased to 2,000.

5.2.9

Disabling Join Elimination for LEFT Joins

Join elimination is a query optimization that can occur during planning when primary and foreign keys exist on the tables. The query plan may eliminate redundant joins when these key relationships have been declared. However, because Yellowbrick does not enforce primary key and foreign key constraints, your tables may have key relationships that are not valid. If this is true, the elimination of joins between those tables may cause queries to return wrong results.  |

To prevent this problem from occurring specifically for LEFT joins, you can set the enable_join_elimination configuration parameter to OFF in Version 5.2.9. For more details and examples, see the CREATE TABLE section of the Version 5.2.9 documentation.

5.2.4

Tech Preview: Setting Configuration Parameters Within Queries

A new SETTING clause may be specified before a query so that it runs with specific values applied for one or more configuration parameters. Contact Technical Support for details.

5.2.4

Management of Lime Cache During Installation

During installation, it is no longer necessary to manually move the lime cache and delete the files later. This operation is now managed automatically by the installer.

5.2.0

Query Planning Enhancements

A number of enhancements have been made in query planning for this release:

  • General improvements in join ordering. This includes the flipping of some LEFT OUTER JOINS to RIGHT OUTER JOINS
  • Better use of declared primary and foreign keys in query planning
  • Improved estimates of join cost and cardinality of matching key values
  • Various improvements and adjustments to the EXPLAIN command output
5.2.0

Encryption Keystore Alert

A new user-defined system alert is triggered when the encryption keystore is locked.

5.2.0

ybtools Improvements

  • In previous releases, RPMs for ybtools on RHEL and AIX systems had to be installed in the default location (/opt/ybtools). In Version 5.2, you can specify an alternative path.
  • The ybdumpschema and ybdumproles help text has been updated for consistency. In previous releases, there were some unnecessary differences in the output for the two tools.
5.2.0

Three-node Configuration for the Andromeda Platform

The Andromeda platform supports a new three-node minimum configuration.

Issues Fixed in This Version

The following issues are addressed in this version:

ReleaseIssue #Description
5.2.2734145

Numeric types used in inputs to a UNION ALL operator, under very specific circumstances (the difference in scale between both inputs to the UNION ALL is greater than the difference in precision, and the numeric scale on the second input is greater than the first) may cause the wrong scale to be returned.

5.2.2733577

Use of LEFT(f() , N) or RIGHT(f(), N), N < 30 in a CTAS can cause system instability and possible outage.

5.2.2733374

Duplicated targetList entries result in incorrect values for distribution.

5.2.2733234

Fix for CVE-2023-38408

5.2.2632559

Using the JSON_LOOKUP function with the jpointer option when JSON documents contained empty strings caused database instability. In addition, using the JSON_INGEST function followed by the JSON_LOOKUP function containing the ingested JSON column and jpath option returned the following error:

Error: malformed input
5.2.2632533

The JSON_LOOKUP function returned a compile error when a NOT NULL constraint was defined on the column used for the first argument.

5.2.2632530

In some instances, the datepart boundary feature in DATEDIFF functions was incorrectly applied for hour, minute, second, epoch, and millisecond dateparts, causing an extra second to be added to the results.

5.2.2632420

An internal error in Postgres prevented "Configure > Users" in the SMC from running.

5.2.2632092

The GETBIT function returned wrong results when used with BIGINT columns.

5.2.2631747

The description of the contents of a backup bundle was improved in the Version 5.2.26 documentation.

5.2.2630805

If a temporary table had rows in the rowstore, selecting from the sys.table_info view would return the following error:

ERROR: OID out of range
5.2.2629699

OS and process interactions would interrupt LDAP, causing LDAP login to fail with the following error:

Can't contact LDAP server
5.2.2629683

In Version 5.2.26, the database row store alert message has been improved to indicate that it is the catalog row store that is becoming full.

5.2.2629053

When replicating from 4 chassis to 8 chassis, incremental replication failed on the source system with a timeout error.

5.2.2628137

Under certain circumstances, parity rebuild after a drive replacement would take longer than expected.

5.2.2627777

When the ybrelay client requested a commit, the ybrelay server did not wait to receive all of the data to be written to the database, especially when decompression of the data was time-consuming. In Version 5.2.26, the shutdown process has been improved so that all the data is committed

5.2.2626078

Under certain circumstances after a lime restart, some lime queries would fail with the following error:

KE032 Attempt to use spill space above limit

causing the SMC "Worker CPU Utilization" and "Worker Memory Usage" dashboards to not display current statistics and the sys.log_query view to not be populated.

5.2.2429270

Joining on strings where both string columns are larger (>30) but of differing lengths would return wrong results.

5.2.2428973

In stored procedures performing small inserts to the rowstore, all INSERT statements within BEGIN...EXCEPTION...END blocks were ignored until a new block was started or the end of the transaction was reached, causing data to be missing from the results of the full transaction.

5.2.2329595

Under certain conditions, a query with multiple IN lists would sometimes return wrong results.

5.2.2329495

Use of the --log-level and --logfilelog-level ybrelay options were not reflected in the ybrelay logs.

5.2.2328679

Some DDL files were missing from the Windows and Java-based ybtools downloads, causing problems with Kerberos configuration.

5.2.2328655

When output CHAR columns were not referenced consistently in the GROUP BY clause (with or without table qualifiers), the query returned the following error:

ERROR: variable not found in subplan target list
5.2.2327810

Running a query with multiple COALESCE columns would sometimes cause out of memory errors and a Postgres crash.

5.2.2327793

Prepared statements with typemod arguments that were executed with a generic plan produced wrong results.

5.2.2327628

In some instances, a cursor fetch query in "client wait" state was not canceled during quiesce.

5.2.2327470

Default WLM restart rules were applied three times and for a large number of error codes. In Version 5.2.23, a smaller subset of error codes triggers a restart, and an attempted restart occurs only once per query.

5.2.2326268

A hung query in idle state was not present in the sys.query view and would not respond to yb_terminate_session or pg_termination_backend. In Version 5.2.23, query cancellation has been improved for certain code paths in the planner.

5.2.2326155

Under certain conditions, running ybcli commands from the secondary manager node on the Andromeda platform would return the following error:

Can not retrieve this platform's hardware generation.
5.2.2320811

For CASE WHEN expressions with CHAR and VARCHAR types, the implicit casting hierarchy was not followed properly. In Version 5.2.23, a VARCHAR type is returned if there is at least one VARCHAR type in the branches of the CASE expression.

5.2.2227998

In a query with GROUP BY and LIMIT clauses, a rare race condition between spilling and cleaning code would sometimes result in a cluster restart.

5.2.2227992

Database restore operations sometimes fail and report the following error:

YCATALOGRESTORE: Cannot find mapping for foreign key

If you run into this problem, contact Customer Support for a workaround.

5.2.2227956

Joining on strings where one string column is small (<=30) and the second string column is larger (>30) would return wrong results.

5.2.2227272

Some queries with COUNT(DISTINCT) functions would return incorrect results because of problems with the join order.

5.2.2227155

A problem with the rewrite order involving CHAR data types would sometimes result in an internal error.

5.2.2227147

A query with multiple subqueries containing the CONCAT function caused a system crash during the planning phase.

5.2.2226770

Joins on CHAR columns performed poorly because the planner added a redundant TRIM function for variable CHAR strings, which prevented bloom filters from being applied.

5.2.2226700

The third-party syslog-ng package was upgraded to fix a slow file descriptor leak. This leak occurred when the internal log files kept on each CMP were rotated, and triggered a DCS alert.

5.2.2226487

The documentation was updated to clarify that SUBSTRING and SUBSTR are not equivalent functions. SUBSTRING returns a VARCHAR data type, but SUBSTR returns the data type of its input string.

5.2.2226365

Running a stored procedure containing loops and an explicit exception statement would cause Postgres to crash if exception handling was involved.

5.2.2226302

The Yellowbrick installer proceeded as far as the firmware upgrade step, then stopped progressing and failed to time out.

5.2.2226248

A distribution logic issue for SEMI LEFT joins caused redundant redistribution.

5.2.2226082

In some instances, a NOT restriction of correlated subqueries was incorrectly applied, causing wrong results.

5.2.2225960

After setting a configuration parameter, using SET SESSION ROLE and SET LOCAL ROLE commands in succession within the same transaction caused a Postgres crash.

5.2.2225834

The IN-list rewrite optimization now works with all comparison operators. In previous releases, only the ‘=‘ and ‘<>’ operators were supported for this optimization.

5.2.2223982

When a regular user executed a long-running SQL statement, the query was not visible in the SMC.

5.2.2223274

Rolling back a transaction with an active backup snapshot would sometimes cause a system crash.

5.2.2222873

Cluster reconfiguration during a system restart could, in some rare cases, result in a deadlock because the self-healing mechanism could block indefinitely. In Version 5.2.22, the self-healing mechanism is more deterministic.

5.2.2126228

Some equality conditions on CHAR values with trailing spaces returned wrong results.

5.2.2125417

On the Andromeda platform, logins to the SMC were slow. In Version 5.2.21, to mitigate this problem, the ybd-telegraf service is disabled by default.

5.2.2125229

For security reasons, the google-gson dependency for backup and restore was upgraded to 2.9.1.

5.2.2125019

A query with multiple rewritten IN lists that constrained the same constants and referenced these constants in the GROUP BY clause returned the following error:

Variable not found in subplan target list
5.2.2124674

When a regular user ran a cross-database query that referenced a LANGUAGE SQL UDF in a remote database, the query failed with an error:

ERROR: function with OID nnnnnnn does not exist
5.2.2123999

Incorrect implicit casting with string functions resulted in a full table scan.

5.2.2120376

Database users granted "Manager" privileges in the SMC were unable to view queries executed by other users.

5.2.2115709

A query plan with a multi-column join and no filtering predicates did not apply a range restriction, resulting in a full scan on a large fact table.

5.2.2113312

The documentation was updated to clarify the description of the last_value column in the sys.sequence view.

5.2.2024898

CREATE EXTERNAL TABLE failed with the following error after a large number of table creations:

OutOfMemoryError: Direct buffer memory
5.2.2024888

Queries against sys.view returned a database lookup error. In Version 5.2.20, when a specific view definition is not found, a warning is returned and other views are displayed.

5.2.2024784

A common table expression (CTE) with an ORDER BY clause returned wrong results.

5.2.2024779

ACLs defined with ALTER DEFAULT PRIVILEGES on the replication source system were not replicated to the target appliance. To change this behavior, add the following option to the WITH clause of the ALTER DATABASE ADD REPLICA statement:

user_resolution_mode 'PLACEHOLDER'
5.2.2024765, 24660

A rare race condition between spilling and join cleaning code resulted in a worker crash. This crash was more likely to occur for join queries that spill to disk under heavy memory pressure.

5.2.1924157

A worker crash caused by a race condition could occur while using partitioned tables when join queries involved spilling.

5.2.1829039

The Yellowbrick client tools (ybtools) now support AIX 7.2.

5.2.1824255

Views created in Version 5.2.15 and earlier that include CASE expressions would sometimes return errors.

5.2.1824234

The returned rows counter was not always reset for some restarted queries, causing the maximum rows limit to be reached before the restarted query actually returned the maximum number of rows.

5.2.1823977

During an upgrade, some tables appeared to be vacuumed when they were not, sometimes causing SQL statements to return the following error:

ERROR: could not access status of transaction
DETAIL: Could not open file "pg_clog/0000"
5.2.1823954, 23953, 23952, 23951, 23950

Version 5.2.18 addresses some vulnerabilities in the Yellowbrick ssh configuration.

5.2.1823912

An IN list rewrite would sometimes return wrong results because data was not cast to the correct data type.

5.2.1823871

During an upgrade, a table containing duplicate keys that returned a duplicate key value error would cause the upgrade to fail.

5.2.1823869, 23826

A memory allocation issue would cause a crash in the manager node stack.

5.2.1823741

In some instances, a hung monitoring service was not terminated, causing a manager node failover to be unsuccessful and resulting in both manager nodes becoming secondary.

5.2.1823694

Insufficient system pool memory would cause some GC operations to stall.

5.2.1823307

A hung query was not present in the sys.query view and would not respond to yb_terminate_session, preventing a GC operation.

5.2.1823305

When a blade was being booted, the ybcli blade status command incorrectly reported "Cluster Status: ERROR."

5.2.1823094

Using ybcli maintenance mode for DNS changes could cause issues with the port listener.

5.2.1822715

Version 5.2.18 improves the ability to set multiple domains on the network manager.

5.2.1822613

When executed in an aborted transaction, a ROLLBACK TO SAVEPOINT query would not be removed from the sys.query view.

5.2.1821722

When implicit casting was used for subqueries that included ORDER BY, GROUP BY, or DISTINCT clauses, the clauses were applied to the incorrect data type, sometimes resulting in a Postgres crash.

5.2.1820015

An upgrade issue would prevent autovacuum from running on some tables, sometimes causing Postgres to emit the following warning:

WARNING: some databases have not been vacuumed in over 2 billion transactions
5.2.1723255

When estimating range predicates using histograms, the max row estimations could sometimes be significantly underestimated.

5.2.1723064

After an upgrade to 5.2.13 or 5.2.14, replication source databases unable to reach the target system were put in READONLY mode. In Version 5.2.17, the system retries connections to the target system for 10 minutes, bringing the source database out of READONLY mode if the target is available.

5.2.1722800

Improvements were made to the memory trimming behavior on the manager nodes to reduce memory consumption.

5.2.1623023

Using DISTINCT inside the MEDIAN function would cause the system to crash.

5.2.1622886

After an upgrade, some front-end queries would return the following warning:

WARNING: out of shared memory.
5.2.1622806

The installation preupgrader could terminate early when validating tables with only system attributes.

5.2.1622755

Incorrect implicit casting of CHAR values containing only digits in INSERT queries returned an error.

5.2.1622750

Parsing a query with a very deeply nested case expression could cause out of memory errors and a system crash.

5.2.1622633

Toast tables and indexes related to schema-excluded objects were not excluded during the replication cycle, resulting in orphaned relations and ultimately causing a database restart.

5.2.1622462

A CTAS statement that included a common table expression caused a Postgres crash.

5.2.1621746

Out-of-memory (OOM) conditions would sometimes occur within the cluster manager when a large number of blades were in use.

5.2.1620674

A CREATE VIEW statement with a CASE expression now returns VARCHAR columns with the correct width.

5.2.1620377 

Entries from yb_deletes tables were included in sys.aalog (AutoAnalyzeLog), resulting in a reduction of disk space and slower auto-analyze performance.

5.2.1619908

Selecting all from sys.view took longer to return results than selecting all from other system views.

5.2.1522308

Character strings inside a NULLIF expression returned wrong results because auto-trimming was not applied to them.

5.2.1522168

Misleading histogram statistics were being created for certain table distributions, leading to inefficient query plans and slower performance.

5.2.1522140

The log4j components for ybtools were upgraded to address security concerns. See What's New in Version 5.2.x.

5.2.1522113

After an upgrade to 5.2, replication failed on the source system with backup errors because of a problem with duplicate "relfilenodes."

5.2.1515303, 22454

After an upgrade, starting replication sometimes returned the following error:

Replication can't proceed due to system upgrade in process.

and "ybcli system status" command incorrectly reported shard rewrite as incomplete.

5.2.1422287

Selecting from the sys.query view with a NOT LIKE predicate returned wrong results.

5.2.1422117

When a replicated view contained a UDF that was used by a subquery, selecting from the view on the destination database returned a "cache lookup failed" error.

5.2.1422106

A query with a very large IN list returned a "KE002: Out of Memory" error.

5.2.1421439

The presence of unicode characters in short character strings could cause incorrect compression and wrong results when the strings were compared.

5.2.1421281

On the Andromeda platform, spilling scalability was improved for aggregations.

5.2.1420458

Join queries run under low-memory conditions (in a slot only a little larger than the estimated memory requirement) triggered repeated spill cycles and failed with an assertion error.

5.2.1417916

The system could be left in a hung state when queries with very complex regular expressions were submitted. In Version 5.2.14, when a regex expression is too complex, the query returns an error that specifies the regex pattern that caused the problem:

Regex '<pattern>' is too complex. Exceeded maximum number of state transitions

In addition, "yielding" inside the regex engine has been introduced, such that before a regex pattern is declared as "too complex," system stability is improved for complex patterns that do not reach the maximum number of state transitions.

5.2.1321892

The pre-upgrade script returned a Python error when the --only flag was used. In Version 5.3.6, you can use this option to run a single check on all databases. You can use the --database option to run all pre-upgrade checks on a single database. You can run a single check on a single database by using both options together. (These command-line options are not supported in ybcli.)

5.2.1321828

The immediate ANALYZE operation that runs after a bulk load could not be disabled. See Disabling ANALYZE Operations for ybload.

5.2.1320896

When a join key was also the target of a constant filter, the selectivity of the join was incorrectly estimated.

5.2.1221737

Failure of a GC operation resulted in a "Metadata storage full error."

5.2.1221734

Performance of scans against the sys.fsinfo virtual table was improved.

5.2.1221717

INSERT statements that required implicit casts on numeric constants inserted garbage values into the table.

5.2.1221711

An internal setting for worker spill space could not be retrieved and caused a Postgres crash.

5.2.1221631

In Version 5.2.11, verbose mode was turned off because it caused a problem during installer rollback. In Version 5.2.12, the installer issue has been fixed and verbose mode is turned on by default.

5.2.1221519, 21522

When a backup contained only a large number of table deletes, a ybrestore operation exceeded a configurable timeout in the restore service, causing the restore to error out although it was complete.

5.2.1221293

On 3-chassis and 4-chassis systems, the server crashed while running queries against the sys.fsinfo virtual table.

5.2.1121608

In Version 5.2.10, verbose mode was set as the default for upgrades. In Version 5.2.11, verbose mode has been turned off because it causes a problem during installer rollback.

5.2.1121389

Certain join queries returned a Generic UnexpectedException error when they contained a LIMIT clause but no ORDER BY clause.

5.2.1121380

Memory for all joins was over-estimated as a result of a previous fix for skew handling. See Increased Memory Estimation.

5.2.1121069

A rare, intermittent worker crash caused by a race condition could occur on the Andromeda platform when queries involved spilling.

5.2.1121262, 20958, 20946, 20470

Global parameters set in PostgreSQL configuration files and SMART data for the manager node drives did not appear in the results of the system diagnostics command. In Version 5.2.11, all of this information is included. The diagnostics package now also includes the contents of sys.vt_cluster_manager_events and the check_xfs.py script.

5.2.1121249, 21233, 21387

To ensure that the Yellowbrick server and client tools are not subject to known log4j vulnerabilities, unused packages and their dependencies were removed from the log4j distribution used by Yellowbrick. See Protection from log4j Vulnerabilities.

5.2.1021297

To optimize query planning for nested-loop joins, cost calibration was improved.

5.2.1021166

Postgres crashed because of a problem with processing ACL settings when roles were set with the ALTER ROLE command.

5.2.1021079

When a stored procedure that created a table with a CLUSTER ON clause was executed, Postgres crashed.

5.2.1020640

The ybloadcat and ybdumpcat logs were missing during an upgrade. In Version 5.2.10, verbose mode is the default for upgrades so that pg.log files are redirected to the ybinstaller.log.

5.2.1019375

A restore failure was caused by missing deletes in the backup bundle.

5.2.921182

For loop joins, the planner sometimes did not choose the smaller table to hold in memory.

5.2.920958

System diagnostics results now include the settings for Postgres configuration parameters, including the output of SHOW ALL and global Postgres configuration files.

5.2.920957

The ASCII function is now included in the Version 5.2 documentation.

5.2.920941

A transaction-consistency check in the FSCK code resulted in an assertion error.

5.2.920940

Join elimination could cause wrong results for LEFT joins when primary and foreign keys did not have referential integrity. In Version 5.2.9, a workaround is available; see Disabling Join Elimination for LEFT Joins.

5.2.920886

Backups taken over AWS Private Link failed with a timeout error.

5.2.920849

The description of the w.resourcePool property for WLM rules was improved in the Version 5.2 documentation.

5.2.920835

The description of the ybbackup --exclude option was improved in the Version 5.2 documentation.

5.2.920821

A section was added to the Version 5.2 backup and restore documentation to explain how to give users access to encrypted table columns in a restored database.

5.2.920786

The EXPLAIN ANALYZE documentation was updated to clarify that write statements (INSERT, UPDATE, DELETE) used with EXPLAIN ANALYZE commit permanent changes to the target table if they are not run within a transaction that can be rolled back.

5.2.920770

If an explicit numeric cast was used in an expression, the resulting data type was adjusted to fit the value of the result. In Version 5.2.9, the return type is based on the requested explicit cast. See Explicit Casts for Numeric Calculations.

5.2.920760

Postgres crashed because of a problem with processing NOT IN expressions within subqueries.

5.2.920750

Restore failures could occur when tables were not restored in the optimal order.

5.2.920722

At some point in the cycle, a database restore would hang. All activities dropped to 0, as indicated by the read/write network statistics in the log.

5.2.920712

The used_bytes column in the sys.storage view is described incorrectly in the Version 5.3 documentation. The used_bytes value is not a precise sum of other values in the view. You cannot derive the exact amount of space used per worker; used_bytes is an approximation, roughly equal to:

distributed_bytes + replicated_bytes + random_bytes + scratch_bytes + some overhead
5.2.920710

The SMC showed a value of 0 for temporary space when the limit was set to 20%.

5.2.920619

Detection of composite primary-key/foreign-key relationships did not take the error margin for distinct values into account, resulting in the wrong join order being chosen.

5.2.920517

The Infiniband route-determination algorithm was changed from Min Hop to LASH to prevent issues in large multi-chassis systems.

5.2.920466

Because of a memory-estimation problem, certain queries intermittently returned the following error:

ERROR: query ran out of memory in pool "Small"
5.2.920465

The sys.cluster view logged duplicate rows for the same worker with multiple roles. Because of this problem, the cluster failed to start.

5.2.920407

The NVL/COALESCE function returned wrong results because trailing space characters were not trimmed correctly.

5.2.920289

The macOS version of ybsql Version 5.2.4 crashed on startup.

5.2.920172

DECIMAL values lost precision when implicit casting was used for subquery results in set operations, such as UNION and UNION ALL.

5.2.917371

A CTAS query that returned an error during parsing or planning logged a type of "create table as" in sys.log_query instead of the expected "ctas" type.

5.2.820701

Under certain conditions, an INSERT query that was restarted in a different WLM resource pool could cause data loss. For more details, request the Support Bulletin on this topic from Customer Support.

5.2.820639

Timeouts for Postgres shutdown during installation were increased to prevent upgrade failures.

5.2.820633

The documentation on cross-database queries has been improved to clarify that writes to remote tables are not supported.

5.2.720653

A security issue made it possible for users to drop database objects that they did not own. For more details, request the Security Bulletin on this topic from Customer Support.

5.2.720624

The output of ybdumpcat now includes the number of rows exported from each table.

5.2.720622

Improvements were made to the preupgrade script to log deletes for system catalog tables.

5.2.720621

Log messages were added to the pg.log when table delete records were inserted for pg_class or pg_type. When a DROP TABLE is executed, check the pg.log for a line of the form:

inserted td record oid <oid of dropped table> in relid 1259...
5.2.720538

A rare race condition could cause a restore or replication failure.

5.2.720448

A query against the sys.query view with an IN list condition returned wrong results.

5.2.720356

Under memory pressure, a rare issue occurred with compute blades writing compiled queries to disk.

5.2.718058

When queries failed with out of memory errors, a race condition resulted in a worker crash.

5.2.620492

Database restore operations could hang when network environment settings caused the termination of connections that were idle for a long time. In Version 5.2.6, the length of time that a network connection can remain idle for large data transfers has been reduced.

5.2.620450

The compiler was overly aggressive in its memory consumption for large queries.

5.2.520425

Declaring a cursor on a query containing a single count(*) in the projection list resulted in an error.

5.2.520381,  20394, 20396

In rare instances, the cancellation of a stored procedure executing multi-table scans failed and resulted in a worker crash.

5.2.520361

The sys.log_query_pre5 view contains query execution summaries for queries run prior to the 5.x upgrade. Only superusers are allowed to access this table.

5.2.520357

Using a WLM throttle rule incorrectly returned error code WL012.

5.2.520311

A race condition prevented the server from starting up.

5.2.520183

The cancellation of a holdable cursor query in "client wait" state was prevented until initial transaction was completed.

5.2.519658, 20149

An aggregate query ran successfully on the Tinman platform but failed on the Andromeda platform with a "Failed to schedule task" error.

5.2.518556

When restoring large backup bundles, restore operations sometimes failed and reported an "invalid key" error.

5.320182

You must create encryption keys in the same database as the tables that will reference them. Do not use cross-database references to encryption keys in your CREATE TABLE statements. If you do this, you will lose those references when the database is backed up and restored (or replicated).

5.319886

After changing the spill space percentage on your system (Temporary Space Reservation setting in the SMC), you must stop and restart the database, using ybcli. Otherwise the change will not take effect.

5.319578

Selecting from the sys.query view returns incorrect results when a WHERE clause condition contains a NOT LIKE operator. For example:

SELECT query_id, type, username
FROM sys.query
WHERE username NOT LIKE 'sys_ybd%';

This problem does not occur for queries against other system views or user tables.

5.2.420139

Customers requested a way to set configuration parameters for a specific query. See Setting Configuration Parameters Within Single Queries.

5.2.420127

A workaround was implemented for a problem with missing table deletes during replication. Contact Customer Support for details.

5.2.420059

Role-specific settings defined with ALTER USER/ROLE were not applied for users running Java ybtools such as ybload and ybunload.

5.2.420057, 19954, 19939, 19728

Multiple issues with cursors have been addressed, including one causing server crashes, and updates to the cursor implementation have been made. See the Version 5.2.4 documentation for details about cursor fetch and move support.

5.2.419992

Version 5.2.4 increases memory requirements for all queries with table scans. See Increased Memory Requirements.

5.2.419983

Slow query processing of the regexp_replace function could cause a system crash. In Version 5.2.4, performance of regexp_replace processing has been improved.

5.2.419981

A stored procedure had the wrong return type in pg_proc, causing a problem when the ybdumpschema tool was run.

5.2.419980

After upgrading from 4.1.x to 5.2.x, you may encounter the following error during database replication: YCATALOGRESTORE: duplicate key value violates unique constraint "pg_type_typname_nsp_index"  If you run into this problem, contact Customer Support for a workaround.

5.2.419961

Installation on systems with large lime and query caches has been optimized in Version 5.2.4.

5.2.419956

After a successful replication cycle, the statistics on the target system were not correct.

5.2.419926

In the SMC, the Monitor Load Processes screen listed job names as "n/a."

5.2.419887

An aggregate query that spills to disk ran successfully on the Tinman platform but failed on the Andromeda platform with an out-of-memory error. In Version 5.2.4, significantly less spill space is requested for this type of query.

5.2.419877

When there was data in the row store, dropping a NOT NULL constraint on a column with an ALTER TABLE command resulted in an internal error, and the table data became inaccessible.

5.2.419580

The cluster became unresponsive for a query with a regex expression that used greedy quantifiers.

5.2.418452

The Execution Timeline in the SMC displayed only the query ID for a running query; no other information was visible.

5.2.319974

A function was added to repair Version 5.2.x views with incorrect joins (see issue 19973). Contact Customer Support for assistance.

5.2.319973

Changes to join ordering introduced in Version 5.2.0 caused unintended changes to joins used in views. For example, LEFT joins became FULL joins. In Version 5.2.3, the original join ordering is restored for views.

5.2.219801

A Postgres crash occurred when the result of a correlated subquery was compared with the result of a CURRENT_DATE function.

5.2.219782

Running the pre-upgrade script resulted in a deadlock when orphan dependencies were found in pg_depend.

5.2.219755

A Postgres crash occurred with a SIGSEGV error because the pg_get_ruledef() function did not correctly resolve late-bound views.

5.2.219723

During planning for ANTI-joins, statistics were assigned incorrectly to the two sides of the join, causing an internal error.

5.2.219714

Excessive amounts of data were being captured for remote diagnostics. A number of changes were made to reduce the growth and size of the worker log files, including shortening references to UUIDs for hostnames and logging only the first 512 characters of SQL text.

5.2.219699

An INSERT INTO SELECT or CTAS with an ORDER BY clause resulted in a hang caused by a race condition.

5.2.119697

A restore operation failed to automatically create the restored database if the username that initiated the restore contained characters that require escaping (such as periods and hyphens). After a full system upgrade, backups taken with previous releases can be restored successfully in 5.2.1.

5.2.119635

Creating or dropping database in the SMC returned an exception:

Connection Error: database "null" does not exist
5.2.119577

A cluster could fail to restart under the following conditions:

  • A DELETE was run in the same transaction as a subsequent INSERT INTO SELECT.
  • The DELETE and INSERT both targeted the same table.
  • A WLM rule caused the INSERT to restart.

Although the transaction completed, the DELETE was not committed. If a database restart was attempted at this point, an FSCK failure occurred.

5.2.119280

The "ybcli health blade" command still reported that a blade was degraded after a successful drive replacement.

5.2.116474

When the result of an IN list subquery required a cast, the query failed with the following error:

Missing constant for test expression

In Version 5.2.1, the cast that is applied to the result of an IN list subquery is pushed down into the subquery.

5.2.019434

During the activation of a WLM profile with a long timeout interval, Postgres did not receive the signal for a canceled query.

5.2.019349, 18348

When a user attempted a password change, the SMC locked out other users and returned a "Service is unavailable" message.

5.2.019072

The Postgres acl_explode() function returned the following error when the ACL abbreviation "Z" was part of the input to the function:

SQL: ERROR: unrecognized aclright:8388608
5.2.018965

RPMs for ybtools on RHEL and AIX systems had to be installed in the default location (/opt/ybtools). In Version 5.2, you can specify an alternative path.

5.2.018922

The MONTHS_BETWEEN function did not work fully as documented. In Version 5.2, all months are considered to have 31 days, and the result is an integral value if both month inputs either specify the same day or specify the last day of the month.

5.2.018671

False PSU alerts were being issued from the CMPs.

5.2.018529

A problem with the decorrelation of expressions in a select-list subquery resulted in the following error:

SQL Error [XX000]: ERROR: variable not found in subplan target list.
5.2.018225

User-defined functions were not being preserved across upgrades, causing ERROR cache lookup failed errors.

5.2.018046

After an upgrade from Version 3.x, the word BATCH could not be used as an alias unless the AS keyword preceded it.

5.2.017502

The ybdumpschema tool returned unexpected "dependency loop" warnings. In Version 5.2, these warnings are no longer emitted.

5.2.017206, 17020

In Version 5.2, a new user-defined system alert is triggered when the encryption keystore is locked.

5.2.016589

Expressions contained in join conditions are now pushed down and evaluated prior to the join operation in the case of cardinality-increasing joins.

5.2.015365

The ADD_MONTHS function returned the wrong result when the input expression was a February date in a leap year.

5.2.014560

When a non-existent fully qualified column was referenced in a query, the following error was returned: "Whole row references are not allowed." The correct error, returned in Version 5.2, is "column column_name does not exist."

CVE Fixes in This Version

The following CVEs are addressed in this version:

ReleaseCVE FixDescription
5.2.27CVE-2023-38408
5.2.26CVE-2021-42392

Parent topic:Yellowbrick Documentation