Product DocumentationAppearance
Yellowbrick Data Warehouse Version 5.4 Change History
5.4.10 Yellowbrick Release Notes
Release Date: 3/20/2025
Upgrade Requirements
Read the following information before beginning an upgrade to Version 5.4. Contact Customer Support to review the process and schedule the upgrade.
Upgrades from Version 5.x
- Upgrades from 5.3.x versions to Version 5.4 are supported; no special steps are required.
- Upgrades from 5.2.0, 5.2.1, and 5.2.2 to Version 5.4 require an additional step to update view definitions. Yellowbrick Customer Support will walk you through the process.
Upgrades from Version 4.x Not Supported
- Version 5.4.3 does not support upgrades from Version 4.x.
Changes in Behavior and Compatibility
Behavior of the CHAR Data Type
The CHAR data type implementation in Yellowbrick 5.x has resulted in some behavioral inconsistencies. For details about changes introduced in Version 5.4.9 to address these problems, see Upcoming Behavioral Changes to the CHAR Data Type.
Default Behavior of WLM Restart Rules
By default, a subset of recoverable error codes triggers an attempted restart, and an attempted restart occurs only once per query. For details, see "Restarting Queries" in the updated Version 5.4 documentation.
Deprecation of Legacy ybtools
The legacy version of ybbackup and ybrestore tools ("BAR1") will no longer be supported in major releases greater than Version 5.x. Only the new version of these tools ("BAR2"), which became available with Version 4.x and 5.x, will continue to be supported. The legacy tools continue to be shipped with Version 5.4.x.
ybtools Compatibility
Yellowbrick recommends that you always upgrade ybtools to match the Yellowbrick server version you are running (for example, Version 5.4). Using Version 5.4 of ybtools with an older server version (such as Version 5.2) may result in error messages for some commands. Using previous versions of ybtools with a Version 5.4 database is likely to work but is deprecated and not supported. See also the following section: Backward Compatibility for Backup and Restore.
Important
On CentOS and Red Hat client platforms, you must first remove any existing 3.x version of ybtools. Then you can proceed with the installation of the Version 5.4 ybtools. You cannot upgrade directly from an earlier 3.x version of ybtools to Version 5.4.
Replication Compatibility and Upgrade Process
Important
Both systems used for database replication (source and target) must be running the same major version of the database software. For example, you cannot replicate from a **4.x **source to a **5.x **target or vice versa.
You can replicate a Version 5.x database to a later 5.x database (such as 5.2 to 5.4). You cannot replicate from a later 5.x source to an earlier 5.x target (such as 5.4 to 5.2).
If you are upgrading from Version 5.2.x, for example, and you want to reverse replication in 5.4 (using PROMOTE), follow these steps:
- Pause the replica on the Version 5.2.x source system.
- Upgrade the target system to Version 5.4.
- Verify that replication works from 5.2.x to 5.4.
- Attempt to promote on the 5.4 target. This operation will fail with an expected error because you cannot replicate to a system that is running an earlier version of the software. The replica will remain paused on the 5.2.x source system.
- Upgrade the source system to 5.4.
- Attempt to promote again on the 5.4 target system. This operation should succeed.
- Attempt a new replication cycle from the reversed source to the new target (both 5.4 systems). The replication cycle should succeed on the new target.
Backward Compatibility for Backup and Restore
A backup taken in Version 3.2.x, 3.3.x, 4.x, or 5.x prior to a 5.4 upgrade can be restored in Version 5.4.
After a 5.4 upgrade or installation, the previous version of the backup and restore tools is compatible with Version 5.4. The Version 5.4 ybtools package installs two sets of backup and restore tools: legacy tools and new 5.4 tools.
*The legacy tools are deprecated and will be removed in a future release, along with the ability to restore backups taken in previous releases. *There are significant differences in the functionality and client tool options between the legacy tools and the new tools. The tool sets are distinguished by their names:
| ybtools Version | Backup | Restore | List/Manage Backups |
|---|---|---|---|
| New in Version 4.x and later | ybbackup | ybrestore | ybbackupctl |
| Legacy in Version 4.x and later | ybbackup-old | ybrestore-old | ybbackup-list-old |
| Version 3.x | ybbackup | ybrestore | ybbackup-list |
To use the legacy tools in the 5.4 package, be sure to use the correct tool name in the command. For example, to return help for the legacy version of ybbackup:
$ ybbackup-old --help
Yellowbrick Backup and Restore version 5.4.0-33445
...
Important:
To use backup and restore in Version 5.4, you must follow these steps:
- Use the legacy version of ybtools (-old) to restore any 3.x backups into 5.4 databases. Use the new 4.x or 5.x version of ybtools to restore any 4.x or 5.x backups into 5.4 databases.
- Take new full backups of your 5.4 databases with the new 5.4 ybtools.
- Stop using the legacy ybtools for backup and restore purposes. Use only the new tools.
Deprecated Feature: Restartable Replication
The restartable replication feature that was introduced as a tech preview in Version 5.2.0 is deprecated and disabled in Version 5.4. This feature is not supported under any circumstances for use on production systems; if you would like to use it purely for testing purposes, contact Customer Support.
Removing Trailing Whitespace Characters with ybload
In Version 5.4, the following new ybload options are supported:
--trim-trailing-white--no-trim-trailing-white
For example, if --trim-trailing-white is used, the string 'abc ' (which contains trailing spaces), is trimmed to 'abc'.
If you do not specify either of these options, the default behavior is to preserve trailing whitespace characters.
Note: When you use --format CSV, whitespace characters inside quoted fields are not trimmed.
Operational Change for ybtools on Windows
In Version 5.4, ybtools on Windows platforms no longer includes .bat files, such as ybload.bat, ybbackup.bat, ybunload.bat, and so on.
What's New in Version 5.4.10
Version 5.4.10 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.4.x.
JSON Functions
Two new SQL functions for JSON manipulation, JSON_INGEST and JSON_LOOKUP, are generally available in Version 5.4.10. See the updated Version 5.4.10 documentation for details and examples.
What's New in Version 5.4.9
Version 5.4.9 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.4.x.
Behavior of the CHAR Data Type
The CHAR data type implementation in Yellowbrick 5.x has resulted in some behavioral inconsistencies. For details about changes introduced in Version 5.4.9 to address these problems, see Upcoming Behavioral Changes to the CHAR Data Type.
What's New in Version 5.4.8
Version 5.4.8 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.4.x.
What's New in Version 5.4.7
Version 5.4.7 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.4.x.
GSSAPI Support in ybtools
The Yellowbrick client tools (ybtools) now have GSSAPI support, which enables support for Kerberos authentication. To enable and configure Kerberos on a Yellowbrick appliance, contact Yellowbrick Customer Support.
What's New in Version 5.4.6
Version 5.4.6 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.4.x.
What's New in Version 5.4.5
Version 5.4.5 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.4.x.
What's New in Version 5.4.4
Version 5.4.4 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.4.x.
What's New in Version 5.4.3
Version 5.4.3 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.4.x.
Maximum Number of Tables Increased to 100,000
Yellowbrick appliances now support a maximum of 100,000 persistent tables per appliance. The previous limit was 70,000.
What's New in Version 5.4.2
Version 5.4.2 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.4.x.
What's New in Version 5.4.1
Version 5.4.1 provides critical bug fixes for issues reported by customers. See Issues Fixed in Version 5.4.x.
What's New in Version 5.4.0
The following new features are available in Version 5.4.0:
Parquet Format Loads and Unloads
The ybload and ybunload client tools support loading and unloading files in Apache Parquet format.
GROUP BY Extensions
The GROUP BY clause now supports the GROUPING SETS, CUBE, and ROLLUP extensions. These extensions make it possible to return aggregate results for multiple GROUP BY expressions within one query.
The GROUPING() function is also supported. You can use this function to identify and filter rows returned by queries that use grouping sets.
Single-Table Restore Operations
You can now restore a single database table from a backup, as opposed to restoring all the objects in a database. You can restore a table to its source database or a separate target database or system.
Table Aliases for SET Columns in UPDATE Statements
When the target table for an UPDATE is given an alias, that alias can be used in the SET clause as well.
Number of Table Columns Increased to 2000
The maximum number of user-defined columns in a table has been increased to 2000.
New SQL Functions
INV_NORM: returns an inverse normal distribution x-value, given a known probability (or percentile) and optional mean and standard deviation values.
ROUND_VAR: a version of the ROUND function that accepts a column name or an expression for its second argument.
TO_NUMBER: Convert a number string to an actual numeric value, using a specified format.
yb_get_view_table_usage(): return information about views and the tables they depend on.
System View Changes
The sys.restore and sys.log_restore views now have a "table_name" column to log the name of a table that was restored in a single-table operation.
New ybcli Commands
The following ybcli commands are new in this release:
- config cert get and config cert create-selfsigned are new commands that return the status of the current Yellowbrick Data on the host system. If the certificate is self-signed, you can regenerate it.
- log clang is a new command that returns the log for compiler-related events.
TLS Support for ybrelay and Spark
TLS support for ybrelay and Spark is now generally available; this feature is no longer in Tech Preview mode. Note: ybrelay is not supported on Windows clients.
Setting Configuration Parameters Within Queries
A new SETTING clause may be specified before a query so that it runs with specific values applied for one or more configuration parameters.
Known Issues in Version 5.4.x
You may encounter the following issues in Version 5.4.x:
| Issue # | Description |
|---|---|
| 27992 | Database restore operations sometimes fail and report the following error: YCATALOGRESTORE: Cannot find mapping for foreign key If you run into this problem, contact Customer Support for a workaround. |
| 24660 | In Versions 5.2.18 and earlier, a rare race condition may occur when using joins that involve spilling, resulting in the query failing and ultimately causing a worker crash. In Version 5.4.x, such a query no longer causes a worker crash when it fails, but it does return an exception. |
| 21423 | Databases with names containing UTF-8 characters cannot be dropped via the SMC. If you run into this problem, contact Customer Support for a workaround. |
| 20786 | Running EXPLAIN ANALYZE on any write statement (INSERT, UPDATE, DELETE) modifies the underlying table, without warning. To work around this problem, run the EXPLAIN ANALYZE statement inside a transaction so you can roll it back if needed |
| 19886 | After changing the spill space percentage on your system (Temporary Space Reservation setting in the SMC), you must stop and restart the database, using ybcli. Otherwise the change will not take effect. |
| 16101 | The SMC does not display row counts in the row store (for example, in the Table Details screen). |
| 14447 | If you use a full restore to seed a replica, you must start replication by specifying the chain name in the ALTER DATABASE command. For example: yellowbrick=# alter database sourcedb alter replica sourcedb_replica start with chain 'February2020';START REPLICAIf you do not specify the chain, the first replication cycle will duplicate the seeded table rows. For details about seeding a replica, see the "Database Replication" section in the product documentation. |
Issues Fixed in Version 5.4.x
The following issues are fixed in Version 5.4.x:
| Release | Issue # | Description |
|---|---|---|
| 5.4.10 | 31151 | A regression in Versions 5.4.8 and 5.4.9 caused some queries with Protegrity functions to return the following error: ERROR: Assemble failed |
| 5.4.10 | 30371 | A system restart caused by a race condition could occur for nested subqueries. |
| 5.4.10 | 29495 | Use of the --log-level and --logfilelog-level ybrelay options were not reflected in the ybrelay logs. |
| 5.4.10 | 29270 | Joining on strings where both string columns are larger (>30) but of differing lengths would return wrong results. |
| 5.4.10 | 28973 | In stored procedures performing small inserts to the rowstore, all INSERT statements within BEGIN...EXCEPTION...END blocks were ignored until a new block was started or the end of the transaction was reached, causing data to be missing from the results of the full transaction. |
| 5.4.10 | 28137 | Under certain circumstances, parity rebuild after a drive replacement would take longer than expected. |
| 5.4.10 | 27810 | Running a query with multiple COALESCE columns would sometimes cause out of memory errors and a Postgres crash. |
| 5.4.9 | 28655 | When output CHAR columns were not referenced consistently in the GROUP BY clause (with or without table qualifiers), the query returned the following error: ERROR: variable not found in subplan target list |
| 5.4.9 | 27628 | In some instances, a cursor fetch query in “client wait” state was not canceled during quiesce. |
| 5.4.8 | 28053 | The number of distinct values was poorly estimated for scans with filters of the form: s<column> = <constant>These estimates have been improved in Version 5.4.8. |
| 5.4.8 | 27793 | Prepared statements with typemod arguments that were executed with a generic plan produced wrong results. |
| 5.4.8 | 26268 | A hung query in idle state was not present in the sys.query view and would not respond to yb_terminate_session or pg_termination_backend. In Version 5.4.8, query cancellation has been improved for certain code paths in the planner. |
| 5.4.8 | 26155 | Under certain conditions, running ybcli commands from the secondary manager node on the Andromeda platform would return the following error:Can not retrieve this platform's hardware generation |
| 5.4.7 | 27956 | Joining on strings where one string column is small (<=30) and the second string column is larger (>30) would return wrong results. |
| 5.4.7 | 27470 | Default WLM restart rules were applied three times and for a large number of error codes. In Version 5.4.7, a smaller subset of error codes triggers a restart, and an attempted restart occurs only once per query. |
| 5.4.7 | 27155 | A problem with the rewrite order involving CHAR data types would sometimes result in an internal error. |
| 5.4.7 | 27147 | A query with multiple subqueries containing the concat(*) function caused a system crash when the query was run during planning. |
| 5.4.7 | 26365 | Executing a stored procedure containing loops and an explicit exception statement would cause Postgres to crash if exception handling was involved. |
| 5.4.7 | 26113 | A large number of SHOW statements referencing ybd_analyze_after_writes and ybd_analyze_after_loads were logged in sys.log_query, causing OOM and spill space errors. In Version 5.4.7, these statements are not logged. |
| 5.4.7 | 26082 | In some instances, a NOT restriction of correlated subqueries was incorrectly applied, causing wrong results. |
| 5.4.7 | 22873 | Cluster reconfiguration during a system restart could, in some rare cases, result in a deadlock because the self-healing mechanism could block indefinitely. In Version 5.4.7, the self-healing mechanism is more deterministic. |
| 5.4.7 | 9485 | System view queries could return query text strings in GZIP-compressed format. In Version 5.4.7, these compressed strings are correctly decoded. |
| 5.4.6 | 26770 | Joins on CHAR columns performed poorly because the planner added a redundant TRIM function for variable CHAR strings, which prevented bloom filters from being applied. |
| 5.4.6 | 26700 | The third-party syslog-ng package was upgraded to fix a slow file descriptor leak. This leak occurred when the internal log files kept on each CMP were rotated, and triggered a DCS alert. |
| 5.4.6 | 26680 | After an upgrade to Version 5.4.5, some queries required a lot more memory to run (or required more spill space and still ran out of memory). |
| 5.4.6 | 26594 | A UNION ALL query that selected and constrained sequence values failed with the following error: Generic UnexpectedException - Can't dlopen |
| 5.4.6 | 26487 | The documentation was updated to clarify that SUBSTRING and SUBSTR are not equivalent functions. SUBSTRING returns a VARCHAR data type, but SUBSTR returns the data type of its input string. |
| 5.4.6 | 26486, 26477 | After an upgrade to Version 5.4.5, the rewrite of LEFT joins with a NULL clause into an ANTI join caused a Postgres crash when the planner tried to access an uninitialized field from an unplanned subquery. |
| 5.4.6 | 26302 | The Yellowbrick installer proceeded as far as the firmware upgrade step, then stopped progressing and failed to time out. |
| 5.4.6 | 26248 | A distribution logic issue for SEMI LEFT joins caused redundant redistribution. |
| 5.4.6 | 25960 | After setting a configuration parameter, using SET SESSION ROLE and SET LOCAL ROLE commands in succession within the same transaction caused a Postgres crash. |
| 5.4.6 | 25834 | The IN-list rewrite optimization now works with all comparison operators. In previous releases, only the '=' and '<>' operators were supported for this optimization. |
| 5.4.6 | 25802 | The --spark-sql application option did not accept a path that started with "file:". For example, calling spark-submit with the following syntax resulted in a "ParseException" error: --spark-sql file:parquet.spark.sql |
| 5.4.6 | 23274 | Rolling back a transaction with an active backup snapshot would sometimes cause a system crash. |
| 5.4.5 | 26228 | A regression in Version 5.4.x caused some equality conditions on CHAR values with trailing spaces to return wrong results. |
| 5.4.4 | 26106 | In some cases, the planner subquery estimation logic ignored casts. This behavior caused the wrong operators to be called and the database to shut down. |
| 5.4.4 | 25417 | On the Andromeda platform, logins to the SMC were slow. In Version 5.2.21, to mitigate this problem, the ybd-telegraf service is disabled by default. |
| 5.4.4 | 24899 | Instead of being garbage-collected, table deletes were included in full backups, making the backup bundle grow very large. |
| 5.4.3 | 25534 | Unsupported ybunload operations timed out instead of failing immediately. |
| 5.4.3 | 24674 | When a regular user ran a cross-database query that referenced a LANGUAGE SQL UDF in a remote database, the query failed with an error: ERROR: function with OID nnnnnnn does not exist |
| 5.4.3 | 23999 | Incorrect implicit casting with string functions resulted in a full table scan. |
| 5.4.2 | 25229 | For security reasons, the google-gson dependency for backup and restore was upgraded to 2.9.1. |
| 5.4.2 | 25071 | On the Andromeda platform, spill queries assigned to the large resource pool ran out of memory. |
| 5.4.2 | 25019 | A query with multiple rewritten IN lists that constrained the same constants and referenced these constants in the GROUP BY clause returned the following error: Variable not found in subplan target list |
| 5.4.2 | 24898 | CREATE EXTERNAL TABLE failed with the following error after a large number of table creations: OutOfMemoryError: Direct buffer memory |
| 5.4.2 | 24784 | A common table expression (CTE) with an ORDER BY clause returned wrong results. |
| 5.4.2 | 24779 | ACLs defined with ALTER DEFAULT PRIVILEGES on the replication source system were not replicated to the target appliance. To change this behavior, add the following option to the WITH clause of the ALTER DATABASE ADD REPLICA statement: user_resolution_mode 'PLACEHOLDER' |
| 5.4.2 | 24765, 24660 | A rare race condition between spilling and join cleaning code resulted in a worker crash. This crash was more likely to occur for join queries that spill to disk under heavy memory pressure. |
| 5.4.2 | 24157 | A worker crash caused by a race condition could occur while using partitioned tables when join queries involved spilling. |
| 5.4.2 | 17888 | The ybbackupctl --fix-bundle-paths command always returned an exit code of 0. In Version 5.4.2, if any backup paths cannot be fixed, this command correctly returns an error. |
| 5.4.2 | 13312 | The documentation was updated to clarify the description of the last_value column in the sys.sequence view. |
| 5.4.1 | 24789 | When an IN list subquery containing LIMIT and ORDER BY clauses was decorrelated, the IN list rewrite was incorrectly applied, causing wrong results. In Version 5.4.1, decorrelation of IN list subqueries containing LIMIT clauses results in the following error: This form of correlated subquery is not supported. |
| 5.4.1 | 24777 | ORDER BY clauses were removed from IN list subqueries that included a LIMIT clause, causing queries to return wrong results. |
| 5.4.0 | 24516 | When temporary tables were being created, a rare race condition involving the allocation and vacuuming of associated ROWUNIQUE values could cause a system crash. |
| 5.4.0 | 24453 | During an upgrade to 5.2.18 or earlier, some query_id and session_id numbers were reduced, resulting in duplicated entries in the sys.log_query and sys.log_session views. |
| 5.4.0 | 24365, 23953 | Version 5.4.0 addresses some vulnerabilities to the SMC, Lime, and Postgres ports by disabling all weak CBC ciphers. |
| 5.4.0 | 24255 | Views created in Version 5.2.15 and earlier that include CASE expressions would sometimes return errors. |
| 5.4.0 | 23977 | During an upgrade, some tables appeared to be vacuumed when they were not, sometimes causing SQL statements to return the following error:ERROR: could not access status of transactionDETAIL: Could not open file "pg_clog/0000" |
| 5.4.0 | 23954, 23952, 23951, 23950 | Version 5.4.0 addresses some vulnerabilities in the Yellowbrick ssh configuration. |
| 5.4.0 | 23912 | An IN list rewrite would sometimes return wrong results because data was not cast to the correct data type. |
| 5.4.0 | 23871 | During an upgrade, a table containing duplicate keys that returned a duplicate key value error would cause the upgrade to fail. |
| 5.4.0 | 23869, 23826 | A memory allocation issue would cause a crash in the manager node stack. |
| 5.4.0 | 23741 | In some instances, a hung monitoring service was not terminated, causing a manager node failover to be unsuccessful and resulting in both manager nodes becoming secondary. |
| 5.4.0 | 23694 | Insufficient system pool memory would cause some GC operations to stall. |
| 5.4.0 | 23307 | A hung query was not present in the sys.query view and would not respond to yb_terminate_session, preventing a GC operation. |
| 5.4.0 | 23305 | When a blade was being booted, the ybcli blade status command incorrectly reported "Cluster Status: ERROR." |
| 5.4.0 | 23094 | Using ybcli maintenance mode for DNS changes could cause issues with the port listener. |
| 5.4.0 | 23064, 22630 | After an upgrade to 5.2.13 or 5.2.14, replication source databases unable to reach the target system were put in READONLY mode with no additional attempts to connect to the target. In Version 5.4.0, the system retries connections to the target system for 10 minutes, bringing the source database out of READONLY mode if the target is available. |
| 5.4.0 | 23023 | Using DISTINCT inside the MEDIAN function would cause the system to crash. |
| 5.4.0 | 22986, 22750 | Parsing a query with a complex nested case expression would sometimes cause out-of-memory errors and a system crash. |
| 5.4.0 | 22886 | After an upgrade, some front-end queries would return the following warning: WARNING: out of shared memory. |
| 5.4.0 | 22806 | The installation preupgrader could terminate early when validating tables with only system attributes. |
| 5.4.0 | 22715 | Version 5.4.0 improves the ability to set multiple domains on the network manager. |
| 5.4.0 | 22633 | Toast tables and indexes related to schema-excluded objects were not excluded during the replication cycle, resulting in orphaned relations and ultimately causing a database restart. |
| 5.4.0 | 22613 | When executed in an aborted transaction, a ROLLBACK TO SAVEPOINT query would not be removed from the sys.query view. |
| 5.4.0 | 22462 | A CTAS statement that included a common table expression caused a Postgres crash. |
| 5.4.0 | 22308 | Character strings inside a NULLIF expression returned wrong results because auto-trimming was not applied to them. |
| 5.4.0 | 22218 | Queries including conversions of CHAR to DATE data type would sometimes return an out-of-range error. |
| 5.4.0 | 22203 | Issues during query execution caused simple correlated subquery usage in the predicate to result in a database crash. |
| 5.4.0 | 22192 | The description of ALTER DATABASE DROP REPLICA was improved in the 5.4 documentation. |
| 5.4.0 | 22168 | Misleading histogram statistics were being created for certain table distributions, leading to inefficient query plans and slower performance. |
| 5.4.0 | 22142, 21881, 20518, 20514, 17683, 14502, 12218, 11324 | In Version 5.4.0, performance of query compilation and code generation was improved. |
| 5.4.0 | 22117 | When a replicated view contained a UDF that was used by a subquery, selecting from the view on the destination database returned a "cache lookup failed" error. |
| 5.4.0 | 22113 | After an upgrade to 5.2, replication failed on the source system with backup errors because of a problem with duplicate "relfilenodes." |
| 5.4.0 | 22106 | A query with a very large IN list returned a "KE002: Out of Memory" error. |
| 5.4.0 | 22103 | A query with an IN list condition containing a GROUP BY, DISTINCT, or ORDER BY operation applied after the cast would return the following error: ERROR: variable not found in subplan target list |
| 5.4.0 | 21933 | Using ALTER statements to rename database objects would allow new names to include line breaks, which resulted in errors when such objects were used in a CREATE TABLE AS statement. In Version 5.4.0, newlines are not permitted in object names. |
| 5.4.0 | 21892 | The pre-upgrade script returned a Python error when the --only flag was used. In Version 5.4.0, you can use this option to run a single check on all databases. You can use the --database option to run all pre-upgrade checks on a single database. You can run a single check on a single database by using both options together. (These command-line options are not supported in ybcli.) |
| 5.4.0 | 21737 | Failure of a GC operation resulted in a "Metadata storage full error." |
| 5.4.0 | 21734 | Performance of scans against the sys.fsinfo virtual table was improved. |
| 5.4.0 | 21733, 21293 | On 3-chassis and 4-chassis systems, the server crashed while running queries against the sys.fsinfo virtual table. |
| 5.4.0 | 21722 | When implicit casting was used for subqueries that included ORDER BY, GROUP BY, or DISTINCT clauses, the clauses were applied to the incorrect data type, sometimes resulting in a Postgres crash. |
| 5.4.0 | 21717 | INSERT statements that required implicit casts on numeric constants inserted garbage values into the table. |
| 5.4.0 | 21711 | An internal setting for worker spill space could not be retrieved and caused a Postgres crash. |
| 5.4.0 | 21531, 21510 | Improvements were made to the check_xfs.py script to prevent hanging. |
| 5.4.0 | 21519 | When a backup contained only a large number of table deletes, a ybrestore operation exceeded a configurable timeout in the restore service, causing the restore to error out although it was complete. |
| 5.4.0 | 21439 | The presence of unicode characters in short character strings could cause incorrect compression and wrong results when the strings were compared. |
| 5.4.0 | 21389 | Certain join queries returned a Generic UnexpectedException error when they contained a LIMIT clause but no ORDER BY clause. |
| 5.4.0 | 21380 | Memory for all joins was over-estimated as a result of a previous fix for skew handling. |
| 5.4.0 | 21351 | ybload failed to round up input data with a fractional second value of more than six digits of precision, resulting in the row being skipped. |
| 5.4.0 | 21315 | When a bulk load failed, a race condition could cause the SMC to report incorrect status and health for the cluster. |
| 5.4.0 | 21290 | Under certain conditions, running the ybdumpschema tool took significantly longer than expected. |
| 5.4.0 | 21281 | On the Andromeda platform, spilling scalability was improved for aggregations. |
| 5.4.0 | 21249 | To ensure that the Yellowbrick server and client tools are not subject to known log4j vulnerabilities, unused packages and their dependencies were removed from the log4j distribution used by Yellowbrick. |
| 5.4.0 | 21214 | On the Andromeda platform, a prioritization issue involving heartbeat signals would sometimes cause a heartbeat failure on large clusters. |
| 5.4.0 | 21182 | For loop joins, the planner sometimes did not choose the smaller table to hold in memory. |
| 5.4.0 | 21166 | Postgres crashed because of a problem with processing ACL settings when roles were set with the ALTER ROLE command. |
| 5.4.0 | 21122 | SEMI RIGHT joins could return wrong results because of a race condition. Any query of the following form could trigger this behavior, especially larger, partitioned queries: SELECT a from foo where a IN (select b from bar); |
| 5.4.0 | 21079 | When a stored procedure that created a table with a CLUSTER ON clause was executed, Postgres crashed. |
| 5.4.0 | 21069 | A rare, intermittent worker crash caused by a race condition could occur on the Andromeda platform when queries involved spilling. |
| 5.4.0 | 21052, 21008 | When the geqo and geqo_threshold configuration parameters were set explicitly for a cursor-related query, the system crashed. |
| 5.4.0 | 20972, 20967 | A planner optimization prevented the deduplication of init plans, causing excessive memory consumption during query compilation. |
| 5.4.0 | 20957 | The ASCII function is now included in the Version 5.4 documentation. |
| 5.4.0 | 20941 | A transaction-consistency check in the FSCK code resulted in an assertion error. |
| 5.4.0 | 20896 | When a join key was also the target of a constant filter, the selectivity of the join was incorrectly estimated. |
| 5.4.0 | 20886 | Backups taken over AWS Private Link failed with a timeout error. |
| 5.4.0 | 20859 | In some cases, ybload could fail with an index out of bounds exception while generating a bad row file. |
| 5.4.0 | 20849 | The description of the w.resourcePool property for WLM rules was improved in the Version 5.4 documentation. |
| 5.4.0 | 20835 | The description of the ybbackup --exclude option was improved in the Version 5.4.0 documentation. |
| 5.4.0 | 20833 | A configuration parameter caused certain queries to return an "unrecognized node type" error. |
| 5.4.0 | 20821 | In Version 5.4.0, a section was added to the backup and restore documentation to explain how to give users access to encrypted table columns in a restored database. |
| 5.4.0 | 20811 | For CASE WHEN expressions with CHAR and VARCHAR types, the implicit casting hierarchy was not followed properly. In Version 5.4.0, a VARCHAR type is returned if there is at least one VARCHAR type in the branches of the CASE expression. |
| 5.4.0 | 20810 | A new configuration parameter was implemented to allow string functions such as TRIM to be ignored when selectivity of a clause that contains these functions is estimated. |
| 5.4.0 | 20794 | A misconfiguration of the code generation service during system startup would sometimes cause the service to be unavailable. |
| 5.4.0 | 20760 | A Postgres crash occurred because of problems with processing nested NOT IN subqueries. |
| 5.4.0 | 20722 | At some point in the cycle, a database restore would hang. All activities dropped to 0, as indicated by the read/write network statistics in the log. |
| 5.4.0 | 20712 | The used_bytes column in the sys.storage view is described incorrectly in the documentation of earlier versions. The used_bytes value is not a precise sum of other values in the view. You cannot derive the exact amount of space used per worker; used_bytes is an approximation, roughly equal to: distributed_bytes + replicated_bytes + random_bytes + scratch_bytes + some overhead |
| 5.4.0 | 20710 | The SMC showed a value of 0 for temporary space when the limit was set to 20%. |
| 5.4.0 | 20709 | When a column name was used in the flags argument, the REGEXP_LIKE function returned the error "RPC channel broken" causing workers to crash. Version 5.4.0 returns the following expected error for this case: ERROR: Flags found in REGEXP must be a string literal |
| 5.4.0 | 20701 | Under certain conditions, an INSERT query that was restarted in a different WLM resource pool could cause data loss. For more details, request the Support Bulletin on this topic from Customer Support. |
| 5.4.0 | 20674 | A CREATE VIEW statement with a CASE expression now returns VARCHAR columns with the correct width. |
| 5.4.0 | 20653 | A security issue made it possible for users to drop database objects that they did not own. |
| 5.4.0 | 20640 | The ybloadcat and ybdumpcat logs were missing during an upgrade. In Version 5.4.0, verbose mode is the default for upgrades so that pg.log files are redirected to the ybinstaller.log. |
| 5.4.0 | 20638 | The SMC displayed the creator of the WLM profile as the user who activated the profile, rather than the actual user who activated it. |
| 5.4.0 | 20624 | The output of ybdumpcat now includes the number of rows exported from each table. |
| 5.4.0 | 20622 | Improvements were made to the preupgrade script to log deletes for system catalog tables. |
| 5.4.0 | 20621 | Log messages were added to the pg.log when table delete records were inserted for pg_class or pg_type. When a DROP TABLE is executed, check the pg.log for a line of the form: inserted td record oid <oid of dropped table> in relid 1259... |
| 5.4.0 | 20619 | Detection of composite primary-key/foreign-key relationships did not take the error margin for distinct values into account, resulting in the wrong join order being chosen. |
| 5.4.0 | 20593 | The number of distinct values was poorly estimated for expressions of the form: func(var) = <const> These estimates have been improved in Version 5.4.0. |
| 5.4.0 | 20589 | Using the ybd_analyze_after_writes configuration parameter returned an unexpected warning, which could interrupt automation scripts. |
| 5.4.0 | 20586 | For primary-key/foreign-key join estimation on composite keys, the cardinality was estimated as the size of the primary key, not the foreign key. |
| 5.4.0 | 20577 | The TO_NUMBER function returned a compile error when a NOT NULL constraint was defined on the column used for the first argument. |
| 5.4.0 | 20538 | A rare race condition could cause a restore or replication failure. |
| 5.4.0 | 20517 | The Infiniband route-determination algorithm was changed from Min Hop to LASH to prevent issues in large multi-chassis systems. |
| 5.4.0 | 20492 | Database restore operations could hang when network environment settings caused the termination of connections that were idle for a long time. In Version 5.4.0, the length of time that a network connection can remain idle for large data transfers has been reduced. |
| 5.4.0 | 20465 | The sys.cluster view logged duplicate rows for the same worker with multiple roles. Because of this problem, the cluster failed to start. |
| 5.4.0 | 20458 | Join queries run under low-memory conditions (in a slot only a little larger than the estimated memory requirement) triggered repeated spill cycles and failed with an assertion error. |
| 5.4.0 | 20450 | The compiler was overly aggressive in its memory consumption for large queries. |
| 5.4.0 | 20448 | A query against the sys.query view with an IN list condition returned wrong results. |
| 5.4.0 | 20407 | The NVL/COALESCE function returned wrong results because trailing space characters were not trimmed correctly. |
| 5.4.0 | 20377 | Entries from yb_deletes tables were included in sys.aalog (AutoAnalyzeLog), resulting in a reduction of disk space and slower auto-analyze performance. |
| 5.4.0 | 20376 | Database users granted "Manager" privileges in the SMC were unable to view queries executed by other users. |
| 5.4.0 | 20356 | Under memory pressure, a rare issue occurred with compute blades writing compiled queries to disk. |
| 5.4.0 | 20310 | In some instances, queries that should have been restartable were unable to restart. |
| 5.4.0 | 20253 | Several system views with privilege checks incorrectly omitted tables in other databases for non-superusers. |
| 5.4.0 | 20172 | DECIMAL values lost precision when implicit casting was used for subquery results in set operations, such as UNION and UNION ALL. |
| 5.4.0 | 20015 | An upgrader issue would prevent autovacuum from running on some tables, sometimes causing Postgres to emit the following warning: WARNING: some databases have not been vacuumed in over 2 billion transactions |
| 5.4.0 | 20014 | Spilling a join that is skewed would sometimes result in an out-of-memory (OOM) condition if the skew value exceeded the memory. |
| 5.4.0 | 19992 | Version 5.4.0 increases memory requirements for all queries with table scans. |
| 5.4.0 | 19973 | Changes to join ordering introduced in Version 5.2.0 caused unintended changes to joins used in views. For example, LEFT joins became FULL joins. In Version 5.4.0, the original join ordering is restored for views. |
| 5.4.0 | 19889 | Version 5.4.0 improves cardinality estimations of composite keys. |
| 5.4.0 | 19072 | The Postgres acl_explode() function returned the following error when the ACL abbreviation "Z" was part of the input to the function: SQL: ERROR: unrecognized aclright:8388608 |
| 5.4.0 | 19011 | When parquet files with unsigned INT64 data were loaded via ybrelay and Spark, the load failed with the following error: <br> Parquet types not supported: INT64 (UINT_64) <br> To load unsigned INT64 data in Version 5.4.0, you can use the new ybload feature for loading parquet files, or you can upgrade to Spark 3.2. |
| 5.4.0 | 18345 | Under certain conditions, running the ybcli system status command on the Andromeda platform would incorrectly report the platform as Tinman. |
| 5.4.0 | 18058 | When queries failed with out-of-memory errors, a race condition resulted in a worker crash. |
| 5.4.0 | 17916 | The system could be left in a hung state when queries with very complex regular expressions were submitted. In Version 5.4.0, when a regex expression is too complex, the query returns an error that specifies the regex pattern that caused the problem: Regex '<pattern>' is too complex. Exceeded maximum number of state transitions In addition, "yielding" inside the regex engine has been introduced, such that before a regex pattern is declared as "too complex," system stability is improved for complex patterns that do not reach the maximum number of state transitions. |
| 5.4.0 | 17695 | Under certain conditions, compiling queries with large IN lists would return an out-of-memory error. |
| 5.4.0 | 17371 | A CTAS query that returned an error during parsing or planning logged a type of "create table as" in sys.log_query instead of the expected "ctas" type. |
| 5.4.0 | 16831 | Casting a two-digit year CHAR string stored in a table to a DATE data type failed with the following error:ERROR: datetime field overflow |
| 5.4.0 | 16474 | When the result of an IN list subquery required a cast, the query failed with the following error:Missing constant for test expressionIn Version 5.4.0, the cast that is applied to the result of an IN list subquery is pushed down into the subquery. |
| 5.4.0 | 15970 | Queries with CASE WHEN expressions would sometimes become stuck in lime in a `prepare` state. |
| 5.4.0 | 14178 | Running the DESCRIBE command on tables with VARCHAR set as the default displayed an unnecessary cast. |
| 5.4.0 | 11635 | A query with a large number of OR clauses failed after fifteen minutes with the following error:ERROR: Query too complex. |
| 5.4.0 | 8719 | Version 5.4.0 improves the compile time for queries containing nested subqueries. |
Postgres CVE Fixes in Version 5.4.x
The following CVEs were fixed by Postgres and have been ported into Yellowbrick Version 5.4:
| Release | CVE Fix # | Issue # |
|---|---|---|
| 5.4.0 | CVE-2021-32027 | 20995 |
| 5.4.0 | CVE-2021-23222 | 20995 |
| 5.4.0 | CVE-2021-23214 | 20995 |
| 5.4.0 | CVE-2019-10130 | 19536 |
| 5.4.0 | CVE-2017-7548 | 19457, 19419 |
| 5.4.0 | CVE-2017-7484 | 19554 |
| 5.4.0 | CVE-2016-5424 | 19659 |
| 5.4.0 | CVE-2016-5423 | 19637 |
| 5.4.0 | CVE-2016-3065 | N/A |
| 5.4.0 | CVE-2016-2193 | N/A |
| 5.4.0 | CVE-2016-0773 | N/A |
Prior Releases
For information on releases prior to those listed above, please contact Yellowbrick Support
Parent topic:Yellowbrick Documentation