- Upgrading Greenplum
- Release 6.23
- Release 6.22
- Release 6.21
- Release 6.20
- Release 6.19
- Release 6.18
- Release 6.17
- Release 6.16
- Release 6.15
- Release 6.14
- Release 6.13
- Release 6.12
- Release 6.11
- Release 6.10
- Release 6.9
- Release 6.8
- Release 6.7
- Release 6.6
- Release 6.5
- Release 6.4
- Release 6.3
- Release 6.2
- Release 6.1
- Release 6.0
- Release 6.0.1
- Release 6.0.0
- New Features
- PostgreSQL Core Features
- Zstandard Compression Algorithm
- Relaxed Rules for Specifying Table Distribution Columns
- Resource Groups Features
- PL/pgSQL Procedural Language Enhancements
- Replicated Table Data
- Concurrency Improvements in Greenplum 6
- Additional Contrib Modules
- PXF Version 5.8.1
- Additional Greenplum Database Features
- Greenplum 6.0 Beta Features
- Changed Features
- Removed Features
- Deprecated Features
- Known Issues and Limitations
This document contains release information about VMware Tanzu Greenplum Database 6.x releases. For previous versions of the release notes for Greenplum Database, go to VMware Tanzu Greenplum Database Documentation. For information about Greenplum Database end of life, see VMware Tanzu Greenplum Database end of life policy.
VMware Tanzu Greenplum 6 software is available for download from the VMware Tanzu Greenplum page on VMware Tanzu Network.
VMware Tanzu Greenplum 6 is based on the open source Greenplum Database project code.
Upgrading Greenplum
See Upgrading to Greenplum 6 to upgrade your existing Greenplum software.
Release 6.23
Release 6.23.0
Release Date: 2022-12-21
VMware Tanzu Greenplum 6.23.0 is a minor release that includes new and changed features and resolves several issues.
Features
Greenplum Database 6.23.0 includes these new and changed features:
The
CREATE TABLE
command includes a new storage parameter –analyze_hll_non_part_table
– which allows you to force collection of HLL statistics even if the table is not part of a partitioned table.The
gpmt gp_log_collector
command now takes a new option –-with-gpcc
– which collects Greenplum Command Center logs from a variety of locations, including:$GPCC_HOME/logs/*
$GPCC_HOME/conf/app.conf
$HOME/gpmetrics/*
(Greenplum Command Center 6.8.0+ on Greenplum Database 6.x or Greenplum Command Center 4.16.0+ on Greenplum Database 5.x)$MASTER_DATA_DIRECTORY/gpmetrics
(Greenplum Command Center 6.7.0- on Greenplum Database 6.x or Greenplum Command Center 4.15.0- on Greenplum Database 5.x)- The output of the
gppkg -q --all
command
The
gpmt gp_log_collector
command now takes a new option –with-gpss
– which collects all logs relevant to Greenplum Streaming Server.The
CPUSET
syntax for reserving CPU cores for a resource group has changed to require separate separate core specifications for the master host and segments hosts. When usingCPUSET
withCREATE RESOURCE GROUP
orALTER RESOURCE GROUP
you must configure cores for the master host first and then segment hosts, using a semicolon to separate the two. For example,CPUSET='1;1-3'
configures core 1 for the master host, and cores 1 through 3 for segment hosts.The Greenplum Database cluster management utilities now use version 5.7.0 of the
psutil
Python library.The embedded Python distribution is updated to version 2.7.18.
Greenplum Database 6.23 limits the number of supported
CUBE
grouping columns to a maximum of 12.Improves
analyzedb
performance when it operates on a large number of tables.The
gpsd
andminirepro
diagnostic tools now include the number of segments in their command output.The performance of the
minirepro
diagnostic tool is improved by using the Postgres Planner instead of the Pivotal Query Optimizer duringminirepro
restore.Backports upstream PostgreSQL
pg_dump
performance enhancements.Includes
pgcrypto
module changes to facilitate testing.The license file for the Windows Client and Loader Tools Package is updated with new terms.
Greenplum Database 6.23.0 introduces three new modules:
- ltree, which provides data types for representing labels of data stored in a hierarchical tree-like structure.
- timestamp9, which provides an efficient nanosecond-precision timestamp data type.
- uuid-ossp, which provides functions to generate universally unique identifiers (UUIDs).
Version 6.23.0 introduces the following new server configuration parameters:
- gp_print_create_gang_time - directs Greenplum Database to log additional information about worker process (gang) creation during query execution.
- gp_max_parallel_cursors - limits the number of active parallel retrieve cursors allowed on a Greenplum Database cluster.
- optimizer_skew_factor - provides a method to configure GPORCA for skew-aware join costing.
The
diskquota
module was updated to version 2.1, which includes these changes:- The 10 database limit is removed,
diskquota
can now monitor up to 50 databases. - The default value of
diskquota.max_active_tables
is changed to300 * 1024
to reduce default memory usage. diskquota
adds a new configuration parameter,diskquota.max_workers
, that you can set to specify the maximum number of diskquota worker processes that may be running at one time.diskquota
also adds thediskquota.max_table_segments
configuration parameter. You can set this parameter to specify the maximum number of table shards supported bydiskquota
in the Greenplum Database cluster, which gates the number of tables that it can support.
Refer to the diskquota module documentation for installation and upgrade instructions.
- The 10 database limit is removed,
Resolved Issues
Server
32596
Resolved an issue where Greenplum Database dereferenced a null pointer and PANIC
ed during execution of a query that included record-type parameters.
32567
Resolved an issue where a query could not be terminated with pg_terminate_backend()
because Greenplum Database polled an invalid ( -1
) file descriptor. Greenplum now removes invalid file descriptors from the polling list.
32557
Resolved an issue where Greenplum Database returned the error correlated subquery with skip-level correlations is not supported
when it executed a query (that ran successfully in a previous version of Greenplum) that included a single-level correlated subquery.
32502
Added additional logging code for resource queue events.
Resolved an issue where Greenplum Database returned the error no pre-assigned OID for relation ...
when an index was created with a WHERE
clause that invoked a function.
32573
Resolved an issue where Greenplum Database was throwing a “PANIC”,“XX000”,“Unexpected internal error Master process received signal SIGSEGV” error due to incorrect handling of errors in the explain analyze phase of execution.
Resolved a problem where tables created using the WITH (appendonly, orientation=column)
had no entries in pg_attribute_encoding
, resulting in a table that could not be queried.
Resolved a problem where GRANT
and REVOKE
statements that involved both partitioned and non-partitioned tables applied their changes only to partitioned tables, with no error or indication that non-partitioned tables were ignored. The code was modified to ensure that such statements are applied to both partitioned and non-partitioned tables.
122
Resolved problem where pg_dumpall
, used by the VMware Greenplum upgrade utility, did not correctly dump tablespaces in VMware Greenplum version 5.x databases.
121
Resolved a problem where upgrade could fail with a pg_upgrade
error similar to: could not open Append-Only segment file ""base/16410/20863.1665"": No such file or directory"
n/a
Resolved a problem that could cause the error, ERROR: external table <table> command ended with SHELL TERMINATED by signal SIGPIPE (13)
, with external table queries executed on the master segment when all tuples were not retrieved from a table scan. The problem was fixed by ensuring that squelching mechanics are applied after a SIGPIPE for queries that excecute on the master node, as they are applied for queries executed on segment nodes.
Query Processing
32481
Resolved an issue where Greenplum Database cancelled a query due to high virtual memory usage when object identifier duplication caused GPORCA to fall back to the Postgres Planner. GPORCA can now handle object identifiers that are not unique across the system.
32453
Resolves an issue where GPORCA failed to produce a plan when the query involved an update on a partitioned table by removing an unused DML column.
32291
Resolves an issue where GPORCA generated a poor plan because it did not identify operation skew. GPORCA introduces the new optimizer_skew_factor server configuration parameter that you can use to enable skew-aware join costing.
Resolves an issue where a query failed with the error data is corrupt
because GPORCA used an incorrect hashing function when it translated a constant value for a text-type domain.
N/A
Resolves an issue where a query that included a DynamicSeqScan with a subplan failed with the error Unexpected internal error
.
Cluster Management
32190
Resolved an issue where gprecoverseg
failed to detect than an instance of pg_basebackup
was already running, which led to the corruption of the data directory and a PANIC
error.
Extensions
N/A
Resolved an issue where diskquota
failed to launch new background workers before a slots used by a previous worker was freed. Also changed the logging level for background worker events to less verbose (from INFO
to DEBUG1
).
N/A
Resolved an issue where, when running pg_basebackup
, the last WAL segment file could have a checksum mismatch across the primary and mirror segments.
Other
Resolved an issue with the open source distribution of Greenplum Database where the installation package was missing required Python libraries.
Release 6.22
Release 6.22.2
Release Date: 2022-11-18
VMware Tanzu Greenplum 6.22.2 is a maintenance release that resolves several issues.
Resolved Issues
Server
N/A
Resolved an issue where treating non-plan nodes as plan nodes was causing memory corruption and PANIC
errors.
N/A
Resolved an issue where a CREATE INDEX
command on an AO/CO table would hang if there was a pre-existing read-only transaction accessing the same table.
N/A
Resolved an issue where a key internal process was crashing when theree were too many simultaneous connections to the database.
N/A
Resolved an issue where memory corruption occurred after a VACUUM
command due to an internal data structure not being correctly initialized.
N/A
Resolved an issue where DELETE FROM
queries on AO/CO auxiliary tables were erroring out.
N/A
Resolved an issue with pg_dump
where, during gpcopy
operations, external leaf tables were copied as heap tables instead of external tables.
32523, 32314
Resolved an issue where Greenplum Database might drop packets and generate interconnect errors due to incorrect cache eviction logic on remote peer shutdown.
32367
Resolved an issue where Greenplum Database was consuming high amounts of memory when processing column oriented tables that had high numbers of partitions.
32367, 14122
Resolved memory leaks related to dynamic bitmap scan on column-oriented tables.
32495
Resolved a performance issue that occured when Greenplum Database planned a query that included a subselect containing a GROUP BY
clause (as distinct).
32491
Resolved an issue where pg_upgrade
failed when it restored an external table that had a dropped column(s).
32455
Resolved an issue where a PANIC
error occurred after an attempt to split a partition into a new partition and a default partition on a table that did not have a default partition.
32458
Resolved an issue where running an internal function on a partition table caused the master Greenplum Database instance to crash.
32500
Resolved an issue introduced in Greenplum Database 6.21.0 where the error "PANIC","XX000","Unexpected internal error: Master process received signal SIGSEGV
occurred after executing a query that included an unsupported feature.
Resolved an issue where a snapshot could not be imported because the timestamp was missing, affecting snapshot visibility.
Resolved an issue where Greenplum Database returned the error operator 37 is not a valid ordering operator
because it incorrectly generated a sort operator and equality operator when the types of left and right operand differed.
Resolved a compile warning by replacing a strncpy()
call with memcpy()
.
Query Processing
32525
Resolved a problem where GPORCA would fall back to the planner for queries having an intersection of bit values if one of the bit vectors returned a size of 0. GPORCA now correctly removes the empty vector and produces a correct query plan.
32505
Resolved an issue where GPORCA did not generate a performant plan when the query included an N-ary join with only inner joins, and the predicate was false
.
32480
Resolved a crash that could occur during query planning for certain UNION
queries if the output column for the first child of the UNION
was dropped in the projection, making statistics for the column unavailable. The problem was resolved by ensuring that none of the project eleents are dropped.
32450
Resolved a problem where GPORCA failed to do direct dispatch when filtering gp_segment_id
on conjunction constraints.
32291
GPORCA now produces a more performant query plan for queries where it can propagate a predicate from scalar ANY
subquery to the outer relation.
Fixed a regression in PostGIS compatibility that caused an exception if the TVF constant value was NULL. GPORCA now correctly falls back to the planner in this case.
Resolved an issue where GPORCA could crash when a query involved dynamic partition elimination with a dynamic partial scan.
Resolved a dependency issue that occurred when minirepo
was run on a query that accessed a materialized view.
Data Flow
32536
Resolved a pgbouncer
problem where admin users logging into the pgbouncer database were logged in as the pgbouncer
user instead of as the configured admin_user
.
Release 6.22.1
Release Date: 2022-10-07
VMware Tanzu Greenplum 6.22.1 is a maintenance release that resolves several issues.
Resolved Issues
Server
32314-1
Resolved a problem where queries could hang while using an interconnect proxy if network issues occurred during common TCP interconnect teardown operations. A timeout mechanism was introduced to ensure that sender(s) have closed the peer end of the receiver’s connections.
32314-2
Resolved a problem where logs from ic_proxy did not consider the current setting of the gp_log_interconnect
configuration parameter. The parameter value is now taken into consideration in the ic_proxy
logging mechanism.
Resolves an issue where a mirror failed to start archiving WAL after it was promoted when the mirror was configured with archive_mode=on
.
Resolves an issue where Greenplum Database wasted resources when it examined plan nodes that belong to other slices and initialized unnecessary structures. Greenplum now eliminates alien nodes before execution for entry db.
The psql
command \d+
now shows correct index information for append-only auxiliary tables.
Resolves an issue where Greenplum Database returned a wrong result or silently errored out when a subplan contained a SharedScan and the subplan’s parent Plan included motion, by fixing cross-slice interaction detection for subplans.
11948
Resolved a crash that could occur because Greenplum incorrectly allowed REINDEX TABLE
on a partitioned table from within a PL/SQL function. Because REINDEX TABLE
on partitioned table expands the table and starts a new transaction to reindex each table, the operation cannot be rolled back and may crash if called within PL/SQL. The code was modified to prevent this operation from running inside of a function.
Query Processing
32467
Resolved a query failure that could occur if a predicate with a set-returning function was pushed down in cases where the context could not accept a set. The existing logic was updated to ensure that such predicates are not pushed down.
32465
Resolves an issue where Greenplum Database consumed a large amount of memory when GPORCA planned certain queries that generated a dynamic bitmap scan.
32307
Resolved a problem where GPORCA generated poor query plans because it did not permit aggregate functions from running on replicated slices. This meant that results were always broadcasted or gathered on a single segment in order to ensure consistency and correct results. The code was modified to ensure that commonly-used aggregate functions which are not sensitive to the order of data (such as sum, avg, count, min, and max) are safely executed. For cases where aggregate functions are sensitive to the order of data (such as array_agg and custom user-created functions) this optimization is avoided.
32233
Addressed a performance problem where outer join operations performed unnecessary data redistribution to create a hash distribution specification from the inner relation. The code was modified to derive a combined hash specification for outer join operations from both relations.
Cluster Management
32289
Resolves an issue where gpcheckcat
incorrectly flagged default privilege objects as having missing dependencies.
Resolved a problem where gpactivatestandby
promotion could fail because the process did not wait long enough for the walreceiver process to shut down. The problem was fixed by synchronizing the standby promotion timeout value with the mirror promotion timeout value.
Release 6.22.0
Release Date: 2022-09-09
VMware Tanzu Greenplum 6.22.0 is a minor release that includes new and changed features and resolves several issues.
Features
Greenplum Database 6.22.0 includes these new and changed features:
Greenplum Database now includes a plugin that allows you to monitor Greenplum Text clusters via Greenplum Command Center. For more information, see Monitoring Greenplum Text.
A new extension,
plpython3u
, is provided to support developing Python 3.9 functions with PL/Python. See PL/Python Language. An additional Data Science Package for Python is provided to install common Python 3.9 modules for developingplypython3u
functions. See Data Science Package for Python.The Data Science Package for Python 2.7 was updated to add the
regex
package. The package versions forlxml
,nltk
,tenserflow
,tenserflow-gpu
, andkeras
were updated to address CVEs.pg_dump
performance was improved to reduce the time required to dump database schemas during the upgrade process. This especially benefits clusters that have a large number of append-optimized tables.Greenplum Database now includes a view –
gp_suboverflowed_backend
– created by the new extensiongp_subtransaction_overflow
– to help users query for suboverflowed backends. Users can use segment id and process id information reported in the view to terminate the offending backends, thereby preventing degradation of performance. For more information, see Monitoring a Greenplum System.Greenplum Database now includes an extension –
gp_pitr
– to support implementing Point-in-Time Recovery for Greenplum Database 6X.gp_pitr
creates a new view –gp_stat_archiver
– and two new internally used user-defined functions. For more information, see Additional Supplied Modules.The
pgcrypto
extension was updated to support FIPS encryption via thepgcrypto.fips
configuration parameter. See Configuring FIPS Encryption in thepgcrypto
documentation.The
diskquota
module was updated to version 2.0.1, and introduces a change to logging behavior. Now, if thediskquota
module is enabled in a database that has existing data but you do not run thediskquota.init_table_size_table()
function, a single error message is logged when the Greenplum cluster starts up. In earlier versions ofdiskquota
, the error message was logged repeatedly (approximately every 2 seconds) and could flood the log monitor system with the same error.Greenplum Database introduces a new server configuration parameter, optimizer_enable_replicated_table, that you can use to control GPORCA’s behavior when it encounters DML operations on replicated tables. By default, GPORCA attempts to plan and execute operations on replicated tables.
Greenplum Streaming Server (GPSS) version 1.8.0 is included, which includes new features, changes, and bug fixes. Refer to the Greenplum Streaming Server Documentation for more information about this release and for upgrade instructions.
The
s3
protocol now supports obtaining S3 authentication credentials from the default AWS credential file,~/.aws/credentials
.gpfdist
now supports reading and writing files compressed with the Zstandard compression algorithm.The
file
protocol now supports reading files compressed with the Zstandard compression algorithm.Greenplum Database 6.22.0 introduces a new module, ip4r, that provides data types for operations on IPv4 and IPv6 IP addresses.
Greenplum Database 6.22.0 introduces a new module, postgresql-hll, that provides HyperLogLog data types for PostgreSQL and Greenplum Database.
The gp_percentile_agg module has been updated fix a bug and to generate more performant plans for ordered-set aggregates with skewed data. If you are currently using this module, you must upgrade the module after you upgrade Greenplum Database.
Greenplum Database 6.22.0 adds the
f_smgr
public interface for storage operations.The performance of
gpexpand
computation of table size has been improved.The PgBouncer distributed with Greenplum Database 6.22.0 adds support for native LDAP authentication between the
psql
client and thepgbouncer
process. Refer to Configuring LDAP-based Authentication for PgBouncer for configuration information.Greenplum Database supports configuring the
ldapsearchfilter
attribute inpg_hba.conf
. You can optionally use this attribute to provide a search filter to use when doing search+bind authentication. See LDAP Authentication.Greenplum Database improves
pg_dump
performance by gathering AO metadata in a single query.The VMware Tanzu Greenplum Connector for Apache NiFi version 1.1.0 is available, which includes a change. Refer to the Greenplum Connector for Apache NiFi Documentation for more information about this release.
The
gpmt gp_log_collector
utility now includes the--with-gpbackup
option to collect logs related to backup and restore operations. See gpmt gp_log_collector.The DataDirect ODBC drivers were updated to version 7.1.6+7.16.445 to include the latest fixes and to no longer require acceptance of a separate licensing agreement. If you have installed a DataDirect ODBC driver version 7.1.6+7.16.389 or earlier, see Upgrading DataDirect ODBC Drivers. The new version resolves a compatibility issue with the Greenplum Direct Query Custom .pqx connector and Microsoft’s PowerBI Gateway. Both the ODBC driver and .pqx connector must be upgraded on Desktop and Gateway systems.
Resolved Issues
Server
32398
Fixes double freeing of a buffer during stream teardown that resulted in a PANIC.
32384
Resolves an issue where a CTE query that involved several columns caused Greenplum Database to PANIC.
32381
Resolves Postgres CVE-2022-2625, Extension scripts replace objects not belonging to the extension.
32271
Resolves an issue where SELECT
or other read-only operations resulted in setting hint bits on pages, which generated Full Page Images (FPI) WAL records, and introduced a large replication lag. Greenplum Database now extends the use of the wait_for_replication_threshold
server configuration parameter for this scenario.
32063
Resolves an issue where gpupgrade execute
ran slow due to inefficient and repetitive pg_dump
metadata queries.
31879, 32167
Subtransaction overflows that are causing performance problems can now be easily identified and terminated.
31298
Resolves an issue where a big lag between the primary and mirror during incremental recovery halted transactions on the primary.
Resolves an issue where creating a bitmap index on a column, and using the bitmap index to scan with a NULL array condition on this column, failed.
Resolves an occasional overflow state by adding a GUC to control the output of a suboverflow transaction SQL statement.
Resolved an issue where the current client_encoding is applied as the default value, causing a problem when client encoding is different from server encoding. The problem is fixed by using GPDB server encoding by default, whether it is QD or QE, making the default encoding of external tables the same as the database encoding.
Resolves an issue where a query on a replicated table that was created from INSERT INTO
SELECT FROM
another replicated table returned incorrect results when sequence generation was involved.
Fixed potential memory leak by using delete to destroy objects allocated by new[].
Fixes query time failures caused by escaped characters in S3 URLs.
Improves performance by adding a GUC to control the output of suboverflow SQL statement transactions.
Resolves an issue where the final query plan included unnecessary sort nodes when the query included grouping sets with distinct qualified aggregates.
Query Processing
32273
Adds server configuration parameter optimizer_enable_replicated_table
, which is on
by default, to control GPORCA planning and execution of operations on a replicated table. GPORCAfalls back to the Postgres Planner when this parameter is set to off
.
Data Flow
Removes unnecessary null pointer checks in gpcloud.
Replaces escape characters with spaces in s3
protocol URLs to prevent query time failures.
31797
Resolves an issue with s3
protocol S3 authentication by adding support for obtaining credentials from the default AWS credential file.
32029
Resolves a pg_exttable
catalog table encoding mismatch by defaulting to the server encoding.
32365
Resolves a gpfdist
performance issue by adding support for Zstandard compression.
Cluster Management
181803382
Resolves an issue where gpinitsystem
could throw an error when running with the debug ( -D
) option.
32157
Resolves an issue where gpexpand
would repeat the error Port <port-number> used multiple times on <host-name>
if a segments were not in their preferred roles. gpexpand
now correctly assigns port numbers in this case, but generates a warning message to indicate that segments are not operating in their preferred roles.
DataDirect ODBC
32225
The DataDirect ODBC driver version 7.1.6+7.16.445 resolves a compatibility issue with the Greenplum Direct Query Custom .pqx connector and Microsoft’s PowerBI Gateway. Both the ODBC driver and .pqx connector must be upgraded on Desktop and Gateway systems. See Upgrading DataDirect ODBC Drivers.
Release 6.21
Release 6.21.3
Release Date: 2022-10-28
Server
32495
Resolved a problem where the planner produced an inefficient query plan and poor performance if a subquery contained a GROUP BY
clause (as DISTINCT
), because the sublink could not be converted to a join.
Release 6.21.2
Release Date: 2022-08-25
Server
32358
Index creation is now allowed on partitioned tables containing external partitions. Creating a non-unique index on a partitioned table will ignore external partitions, but fail if any of the indexes is a constraint index.
32293
Updates the index VACUUM
strategy for Append-Optimized tables to correctly determine when an index entry can be removed. An index entry is now removed only after ensuring that the entry points to a droppable segment, and is therefore already invisible.
31941
LDAP bind credentials that were leaked to the log after failed login attempts are no longer a security risk.
The EPQ routine for AO/AOCS relations and DML node has been disallowed.
Duplicate gp_fastsequence
values ( ctid
) no longer appear in AO tables.
Query Processing
32348
Non-optimal plans that caused queries to hang were fixed by setting the number of rebinds (rescans) correctly when statistics are copied.
Release 6.21.1
Release Date: 2022-08-03
Features
Greenplum Database 6.21.1 includes a new metrics collector extension that is compatible with Greenplum Command Center 6.8.
Resolved Issues
Server
13881
Resolves an issue in transaction processing where transactions were erroneously rolled back because gang processes were being deleted.
12690
Resolves an issue where, when a memory limit was hit, a query was throwing errors without calling the correct cleanup function.
13874
Resolves an error handling issue that caused PANIC errors during query execution.
32237
Fixes for PostgreSQL CVE-2022-1552 have been backported to secure Autovacuum
, REINDEX
, and other commands that activate relevant protections too late or not at all.
32211
A pg_upgrade
check for parent partitions with seg entries was added to prevent upgrade failure, and logicalEof was added to errdetail in OpenAOSegmentFile
.
32210
Primary now writes a special WAL at the place where the mirror is supposed to continue streaming the rest of the broken WAL.
31937
pg_resetxlog
infrastructure is used to set the oldest xid instead of freezing the catalog, preventing CLOG truncation during a segment upgrade.
Resolves an error, FATAL: Unexpected internal error (list.c:395)
, that could occur when creating a plan for a correlated subquery against a partitioned table. The problem occurred in some queries when an InitPlan could not be created, and the planner generated copies of nested subplans in the wrong order. The fix ensures that first level subplans and nested suplans are generated correctly in this situation.
Improves planning performance for DML queries by avoiding unnecessary planning for join tree relations, especially with complex subqueries.
A fix for the logic in pg_lock_status()
that keeps it from missing some rows and counting others twice has been backported.
Resolves a problem where unnecessary checks were performed when runaway_detector_activation_percent
was deactivated (set to 100).
An error is thrown when COPY FROM/INSERT will result in inserted data being invisible to user queries issued from master.
Resolved an assertion failure, "FATAL" error "Unexpected internal error (vmem_tracker.c:436)","Process 399817 will wait for gp_debug_linger=120 seconds before termination
, that could occur during initialization when resource groups were enabled.
Appendoptimized materialized views now have invalid relfrozenxid.
Resolves a problem where CREATE EXTENSION WITH SCHEMA
failed because Tanzu Greenplum did not set up the search_path
on both the Query Dispatcher and the Query Executor before executing the extension script.
Query Processing
32302
Prevents optimizer from crashing during query optimization due to lack of statistics.
32273
Adds a GUC that keeps queries on distributed replicated tables from hanging when using ORCA by controlling fallback for replicated tables.
32238
ORCA now requires an index type to properly cost bitmap scans before performing full table scans.
32233
Eliminates query performance difference between GPORCA and Postgres.
32125
The master logger process that was creating high load on master, causing segments to go down, was fixed by processing the string with two kinds of modification.
32061
ORCA now prevents creation of spill files by pushing cast operations below the join when safe.
Fallback to Postgres optimizer when a target list in CTE producer is empty.
Fixes incorrect addrefs that caused memory leaks.
Data Flow
32199
Resolves a problem in the fixed width formatting code that could cause the error The line delimiter specified in the Formatter arguments: < 3> is not located in the data file
even though the text was formatted correctly. The problem occurred because the code did not handle the situation where the end of a buffer coincided with the end of the data (and therefore does not include an expected newline character).
32151
Resolves a gpfdist
crash that could occur during session clean up if a session timed due to a disk full condition. Instead of crashing, gpfdist
now logs an error for this condition.
Tanzu Greenplum on vSphere
1102
Resolved an issue where users were seeing WARNINGS in gpinitsystem logs when there was a mismatch between what the hostname
utility reported and the actual segment host name.
1042
Resolved an issue where “command not found” warnings were seen during deployment of Greenplum.
1034
Resolved an issue where deployment would fail if the segment count was equal to 1 or odd numbers for Mirrored Greenplum deployments.
844
Resolved an issue where a manual shutdown of the Guest OS would show a ‘failed’ state for the Greenplum Virtual Service.
Release 6.21.0
Release Date: 2022-06-10
VMware Tanzu Greenplum 6.21.0 is a minor release that includes new and changed features and resolves several issues.
Features
Greenplum Database 6.21.0 includes these new and changed features:
The
LOCK TABLE
SQL command now includes aMASTER ONLY
option; when enabled, Greenplum Database locks tables on the master only, rather than on the master and all of the segments. This option is particularly useful for metadata-only operations.The gp_parallel_retrieve_cursor contrib module is no longer Beta, it is promoted to a supported feature.
The greenplum_fdw contrib module is no longer Beta, it is promoted to a supported feature.
Greenplum Database now supports the
SET TRANSACTION SNAPSHOT
command.Greenplum Database 6.21.0 adds support for both the
SCRAM-SHA-256
password hashing algorithm and thescram-sha-256
client authentication method. Refer to Protecting Passwords in Greenplum Database for more information.Greenplum Database now includes a configuration parameter — gp_interconnect_address_type — to specify the type of address binding strategy Greenplum Database uses for communication between segment host sockets. This allows users, if their configuration permits, to specify that Greenplum use a unicast address — rather than a wildcard address — for interconnect sockets. This reduces port usage on segment hosts and prevents interconnect traffic from being routed through unintended (and possibly slower) network interfaces.
Log messages that are generated for connection checks between the Query Executor and Query Dispatcher now include the error number that was generated by the receiver.
Greenplum Database introduces a new server configuration parameter, gp_log_endpoints, that you can use to enable logging of parallel retrieve cursor endpoint details.
Greenplum Database introduces the new resource group server configuration parameter gp_resource_group_enable_recalculate_query_mem. The default value of this parameter is
false
; Greenplum Database calculates the maximum per-query memory allotment for each segment host based on the memory configuration and the number of primary segments on the master host. If the memory configuration of your Greenplum Database master and segment hosts differ, set the parameter totrue
so that Greenplum calculates the maximum per-transaction memory allotment for a segment host based on the configuration of that host.Greenplum Database 6.21.0 introduces a new module, gp_percentile_agg, that you can use to improve GPORCA plan generation performance for queries that include ordered-set aggregate functions such as
percentile_cont()
,percentile_disc()
, andmedian()
.Greenplum Database 6.21.0 includes version 2.0 of the
diskquota
module. This version introduces support for schema and role tablespace disk quotas as well as per-segment tablespace quotas.diskquota
can also now enforce quota hard limits; if a query exceeds a disk quota during execution,diskquota
terminates the query. Refer to the module documentation for more information and for upgrade instructions.The VMware Tanzu Greenplum Connector for Apache NiFi version 1.0.1 is available, which includes a change. See the Connector 1.0 Release Notes for more information.
The Greenplum Magic Tool (gpmt) family of tools now includes the
gp_storage_rca_collector
tool, which collects storage-related artifacts. See the storage_rca_collector topic for details.The gpmt
gp_log_collector
tool now collects PXF logs andgpupgrade
logs. In addition, this tool has been enhanced to collect system initialization timestamps,/var/log/dmesg
content, and output fromtop
andsar
commands.The PgBouncer distributed with Greenplum Database has been updated to version 1.16.1.
Greenplum Streaming Server (GPSS) version 1.7.2 is included, which includes changes and bug fixes. Refer to the Greenplum Streaming Server Documentation for more information about this release and for upgrade instructions.
For customers deploying Greenplum on vSphere, the following new features and changes have been introduced in this release:
The Greenplum Virtual Machine now provides a service running on the master VM that monitors the postmaster process and restarts the process if it fails. This improves the reliability and availability of the postmaster process. It is enabled by default.
The Greenplum Virtual Machine now provides a service running on each segment VM that monitors the segment process and restarts the process if it fails. This improves the reliability and availability of the segment process. It is enabled by default.
Greenplum on vSphere is now supported on vSphere 6.7 and higher.
Greenplum on vSphere now supports mirrorless Greenplum deployments.
The VxRail reference architecture has been updated to recommend 100GbE networking instead of 25GbE. Increasing the available network throughput significantly improves storage system performance and provides a more balanced system across resources.
The PostGIS spatial database extension version
2.5.4+pivotal.7.build.1
is included, which contains updatedGEOS
andEXPAT
dependencies. Because theGEOS
geometry library is updated to version 3.10.2; PostGIS now includes support for the ST_Subdivide, ST_ClipByBox2D, ST_VoronoiLines, and ST_VoronoiPolygons functions.
Resolved Issues
Server
32247, 13532
Prevents panic in Planner by not bringing a SegmentGeneral path that refs outer Params to SingleQE.
32229
Resolves a problem with TCP/UDP connections communicating across different subnets by adding a new GUC, gp_interconnect_address_type
, which allows users to switch from the wildcard default to a unicast IP address to support motion communication. (Resolved by PR-13344.)
32220
Moves incrementing txns_using_rel
to follow ERROR reports to fix leaks in AppendOnlyHash entries. (Resolved by PR-13570.)
32213, 13549
Resolves wrong date behavior by rejecting ambiguous 5 digit date strings in non-standard formats.
32173
Resolves slow intermittent query response by using hostname instead of an IP address to compute the host segments.
32067, 32141
Resolves an issue where OOM errors have occurred during queries by adding a new GUC, gp_resource_group_enable_recalculate_query_mem
.
32066
Prevents gp_tablespace_segment_location()
from executing on entrydb
QE. (Resolved by PR-13287.)
31957
Enhances OLAP performance by ensuring that both sides of the NOT-IN sublink testexpr
are non-nullable. (Resolved by PR-13420.)
Fixes incorrect scan position during bitmap index words scans. (Resolved by PR-13522.)
32205
Falls back to Planner from ORCA if a target list is empty.
Resolves a cache lookup failure for attribute 0 of relation xxx from child inheritance/partition table to parent.
182378648
Enables recursion for utility mode connections to propagate ALTER OWNER commands to child tables during pg_restore, which must run in utility mode.
n/a
Reverts a problematic commit that enclosed a check for interrupts (CFI) in a try-catch statement. (Resolved by PR-13568.)
n/a
Adds additional GUC defaults for ICW mirrorless tests. (Resolved by PR-13563.)
n/a
Provides checks on fatal exit status and outputs file location for users of pg_upgrade
. (Resolved by PR-13475.)
n/a
Fixes double freeing during a debug build that caused Assertion failure. (Resolved by PR-13472.)
n/a
Adds distributed snapshot support to pg_export_snapshot
. (Resolved by PR-13470.)
n/a
Allows LOCK TABLE to be conducted on MASTER ONLY to improve performance. (Resolved by PR-13463.)
n/a
Preserves append-optimized options for tables without indexes when creating internal tables from query results. (Resolved by PR-13461.)
n/a
Backports part of recovery TAP tests that can be supported on Greenplum 6 and related TAP framework changes. (Resolved by PR-13375.)
n/a
Restores visibility of some GUCs that were hidden from users’ pg_settings
. (Resolved by PR-13353.)
n/a
Resolves suboptimal plans by enabling a GatherMerge alternative for non-EstMaster singleton distribution. (Resolved by PR-13294.)
n/a
Improves the authentication framework by backporting scram-sha256 to Greenplum 6. (Resolved by PR-13286.)
n/a
Resolves an issue where debug trace output was incorrectly reported at DEBUG3 level by adding an external boolean server configuration parameter, gp_log_endpoints
, to print endpoints information to server log. (Resolved by PR-13254.)
n/a
Upgrades pgbouncer
to support scram-sha256 to provide a strong password hash for Greenplum. (Resolved by PR-13146.)
n/a
Fixes queries with mode() aggregate crashes when QueryFinishPending
is set to true. (Resolved by PR-13096.)
Query Processing
31988
Resolves performance differences between ORCA and Planner by generating an alternative that pushes aggregates below Materialize.
31995
Adds support for ordered aggregates by preventing ORCA from falling back to Planner, which resulted in suboptimal performance for Madlib functions.
13242
Enables a check to redistribute tuples before updating, preventing wrong results for ORCA plans that use split updates.
181905507
Resolves an issue where Assertion failed in a debug build by accounting for the memory for the object.
32205
Resolves cases where improper handling of empty target lists results in a crash.
181707276
Resolves a DQA issue that did not include the aggregation stage in hash calculation.
n/a
Resolves an issue where ORCA pushed UDFs down to the segments, but made suboptimal cost comparisons between plans. (Resolved by PR-13384.)
n/a
Correctly represents Table Value Function in ORCA as a constant value, preventing it from returning incorrect results. (Resolved by PR-13535.)
n/a
Resolves an issue with incorrect results from replicated tables. (Resolved by PR-13058.)
n/a
Resolves a performance issue with ORCA producing suboptimal plans. (Resolved by PR-13327.)
Cluster Management
180823003
Resolves an issue where gpstate
was incorrectly reporting mirror segments as not initialized when they had been initialized using pg_ctl
. This was due to the mirror segment port number listed in postgresql.conf
being out of sync with the number listed in the gp_segment_configuration
table.
Data Flow
32209
Creates gpload6
to update the data loading utility for Greenplum 6.x. (Resolved by PR-13465.)
32179, 31372
Fixes a reported gpfdist
error, unknown meta type 108
. (Resolved by PR-13457, PR-13458.)
32169, 13448
Fixes a crash that occurs when an error is thrown during cdbdisp_destroyDispatcherState
.
Extensions
n/a
Fixes a memory leak in SetupUDPIFCInterconnect_Internal() with read-only transactions. (Resolved by PR-10314.)
Release 6.20
Release 6.20.5
Release Date: 2022-06-02
VMware Tanzu Greenplum 6.20.5 is a maintenance release that resolves two issues.
Resolved Issues
Query Optimizer
32142
Resolves an issue where a query on a partitioned table took longer to plan and execute with GPORCA than with the Postgres Planner. GPORCA planning time performance is improved by disabling partition constraint calculations for such queries.
Server
32119
Resolves an issue where a user-defined function was unable to access a replicated temporary table because the identifier of the session’s temporary namespace was not dispatched to all gangs.
Release 6.20.4
Release Date: 2022-05-18
VMware Tanzu Greenplum 6.20.4 is a maintenance release that resolves a single issue.
Resolved Issues
Server
32229, 32204
The fix for 32140, added in Greenplum 6.20.3, introduced a regression. Interconnect failures could occur if multiple subnets where configured on the same host, but routing between the subnets was not enabled. This problem was resolved in version 6.20.4 by reverting the original fix.
Release 6.20.3
Release Date: 2022-04-08
VMware Tanzu Greenplum 6.20.3 is a maintenance release that resolves several issues.
Resolved Issues
Server
32133
Resolves an issue where an attempted failover to mirror segments resulted in a PANIC
error on all primary segments because FTS was failing the primary over prematurely.
32140
Resolves an issue where queries were failing with the message “bind: Address already in use”. This was due to Greenplum Database running out of UDP ephemeral ports when creating listener and sender sockets for UDP interconnect, due to high concurrency (exacerbated by primary segment density – the number of acting primaries on a given segment host).
32111, 13245
Greenplum Database crashed during the redistribution phase of cluster expansion when processing an AO table that had an index because Greenplum ignored the storage options on the original table. Greenplum now reconstructs the storage options of the original table during this operation rather than use the current default storage options.
Removes platform-specific limitation from socket message type recognition logic by using int
.
Resolves an issue where the Postgres Planner returned the error unexpected join qual in JOIN_LASJ_NOTIN join
while planning a query that included a multi-column NOT IN
with a NULL
constant value.
Updates the Postgres Planner to allow direct dispatch of gp_segment_id
and include it as a distributed column constraint.
Cluster Management
32160
Resolves an issue where rebalancing segments with gprecoverseg -r
was failing with timeout errors. This has been resolved by increasing the timeout value for promoting mirror segments to primary.
Query Optimizer
32130
Resolves an issue where an ANALYZE
operation on a table that included an xml
-type column took a long time to complete because there is no equality operator for the type.
Resolves an issue where GPORCA might crash when a strict function with zero arguments was used as a filter for NULL rejection.
181646913
Resolves a C++ compiler and nullptr
usage mismatch by using NULL
instead.
Data Flow
32076
Resolves an issue where Greenplum Database ignored the ON MASTER
clause provided in a writable external table definition that specified the s3
protocol. Greenplum now fully supports ON MASTER
for s3
writable external tables.
31730
Improves gpload
by choosing a better default distribution key for the staging table. gpload
now uses the distribution key(s) of the target table as the distribution key(s) for the staging table. If the target table was created DISTRIBUTED RANDOMLY
, gpload
uses the MATCH_COLUMNS
as the staging table’s distribution key(s).
181229489
Resolves an issue where gpload
did not recognize special characters like capital letters in a staging table name when the name was enclosed in double quotes.
Adds SSL connection regression test cases.
Release 6.20.1
Release Date: 2022-03-25
VMware Tanzu Greenplum 6.20.1 is a maintenance release that resolves several issues.
Resolved Issues
VMware Tanzu Greenplum 6.20.1 resolves these issues:
Server
32136, 13292
Resolves an issue where queries using recursive CTE with a window function or an ordered set aggregate function (special aggs) would cause a PANIC.
32123, 13265
Resolves an issue where the result of an UPDATE
query that included a NOT IN
operator identified that one row was updated, when in fact the row was not updated.
32086, 13195
Resolves an issue where minirepro
returned the error cache lookup failed for relation N
due to Greenplum’s special handling of a type conversion from unknown
to text
.
32050
Resolves issues with error check with pg_upgrade check mode. Upgrade checks are now allowed to progess even if there are failures, and can also check the target cluster.
Removes a memory leak in the dtx recovery process.
Removes memory leaks.
181428106
Adds a check to error out when a DECLARE ... PARALLEL RETRIEVE ...
command is invoked, but the gp_parallel_retrieve_cursor
extension is not registered in the database.
Cluster Management
32018
Resolves an issue where running gpstate -e
on the master generated the message FATAL","57P03","connections to primary segments are not allowed
.
Query Optimizer
32126
Resolves an issue where GPORCA returned the error could not determine which collation to use
when a query used a Nested Loop Index join, and the join condition included a function like upper()
; GPORCA now propogates collation information in this case.
32072
Resolves an edge case where GPORCA returned unexpected results on a query that included an immutable function returning a custom data type; Greenplum Database now falls back to the Postgres Planner.
Resolves a performance issue that occurred when GPORCA generated an inefficient query plan for a query that included a common table expression (CTE) and where the result set was used more than once.
181647276
Resolves an issue where Greenplum Database crashed when it encountered a query using an ALL
operator that had an implicit cast on the left-hand side of the expression.
181576677
Fixes an issue where a previous bug fix broke the printing of expression properties used to debug.
Release 6.20.0
VMware Tanzu Greenplum 6.20.0 is a minor release that includes feature changes and resolves several issues.
Features
Greenplum Database 6.20.0 includes these new and changed features:
- Greenplum Database 6.20 has been tested on and is supported for deployment to Red Hat Enterprise Linux 64-bit 8.7 or later (RHEL8).
Important
A kernel issue in Red Hat Enterprise Linux 8.5 and 8.6 can cause I/O freezes and synchronization problems with XFS filesystems. This issue is fixed in RHEL 8.7. See RHEL8: xfs_buf deadlock between inode deletion and block allocation.
Note For existing Tanzu Greenplum 6.20 or later deployments, VMware does not support upgrading the underlying operating system from RHEL7 or earlier to RHEL8 without also rewriting the database data. If you upgrade the operating system to RHEL8, you must also rewrite the Greenplum data by restoring a full backup that was created using
gpbackup
, or by using the VMware Tanzu Greenplum Data Copy Utility (gpcopy
).
PostGIS 2.5.4+pivotal.6.build.1 is included to provide support for deployment on Red Hat Enterprise Linux 64-bit 8 (RHEL8).
MADlib version 1.19.0 is included to provide support for deployment on Red Hat Enterprise Linux 64-bit 8 (RHEL8). This release includes additional fixes and features as described in the MADlib Release Notes.
Mirrors can now be added, recovered, or moved independently of each other. This has two consequences:
Previously, Greenplum Database required that when adding, recovering, or moving mirrors, all mirrors had to successfully complete the
pg_basebackup
phase before any mirrors could be marked asup
. Now, mirrors that successfully completepg_basebackup
can be marked asup
independently of other mirrors.Previously, if adding, recovering, or moving mirrors failed during
pg_basebackup
then none of those mirrors would be marked asup
and the user would have to re-run these operations for all mirrors. Now, the user only need re-run these operations for those mirrors that failed duringpg_basebackup
.
The Console now provides more detailed information on failures during moving, recovering or adding mirrors. Specifically, it reports the hostname and port number of the mirrors that failed to be moved/recovered/added. It also includes the location of log files for failed
pg_basebackup
andpg_rewind
operations, and the location of the data directory forpg_ctl
start failures.The
gpstate -e
command now includes an additional section in its output: “Segments in recovery.” This section reports on the progress of each segment’s recovery. Specifically, for each segment it reports:- the recovery type (incremental or full)
- the number of completed bytes
- the number of total bytes
- the percentage of recovery the segment has completed
gp_log_collector
now collects data frompg_db_role_setting
instead ofpg_settings
and now collects settings related to resource groups.The Query Optimizer (GPORCA) improves query performance by supporting query plan generation for ordered aggregates. You can configure GPORCA to generate query plans for ordered aggregates by setting the new optimizer_enable_orderedagg server configuration parameter to
true
.The Query Optimizer (GPORCA) now supports bitmap scans on partitioned tables that contain both heap and AO partitions.
Greenplum 6.20.0 now bundles the btree_gin PostgreSQL contrib module. This module provides sample generalized inverted index (GIN) operator classes that implement B-tree equivalent behavior for certain data types.
Greenplum 6.20.0 now bundles the pg_trgm PostgreSQL contrib module. This module provides functions and operators for determining the similarity of alphanumeric text based on trigram matching. The module also provides index operator classes that support fast searching for similar strings.
The pageinspect module now includes support for bitmap indexes as described in the module documentation. If you are currently using the module in a database, you must upgrade the module to access the new functionality.
Greenplum 6.20.0 introduces the new contrib module gp_array_agg. This module implements a parallel
array_agg()
aggregate function for Greenplum Database.Greenplum 6.20.0 introduces support for retrieving data, in parallel and on demand, directly from Greenplum Database segments via the new gp_parallel_retrieve_cursor (Beta) contrib module.
Greenplum 6.20.0 introduces support for accessing data stored in one or more external Greenplum Database clusters via the new greenplum_fdw (Beta) contrib module, which implements a foreign-data wrapper.
The gp_sparse_vector contrib module includes a bug fix. If you are currently using the module in a database, you must upgrade the module to obtain the fix.
The
CREATE SERVER
command exposes a newOPTIONS
namednum_segments
. Thegreenplum_fdw
foreign-data wrapper supports this option; you use it to identify the number of segments in the remote Greenplum Database cluster.Greenplum Streaming Server (GPSS) version 1.7.0 is included, which includes new and changed features and bug fixes. Refer to the Greenplum Streaming Server Documentation for more information about this release and for upgrade instructions.
The PXF version 6.3.0 distribution is available with this release; you can download it from the Release Download directory named Greenplum Platform Extension Framework on VMware Tanzu Network. Refer to the PXF documentation for information about this release and for installation and upgrade instructions.
For customers deploying Greenplum on Dell EMC VxRail, the following enhancements have been introduced to the guidelines, and reference templates:
- Greenplum Command Center (GPCC) is now included by default in the Greenplum Virtual Machine. After deploying Greeplum, GPCC will be accessible and ready to use. For details on the GPCC user account, see Setting Up the GPCC Login User.
- The Storage Policy has been updated from Stripe 1 to Stripe 4, to improve vSAN performance.
- The Greenplum deployment via your own template now allows you to enable vApp options that will be used by Terraform during deployment. For details see Installing the Greenplum Database Software
- This release adds documentation for changing the Data Disk size during the Greenplum Virtual Machine deployment. For details see Modifying Resources.
Resolved Issues
VMware Tanzu Greenplum 6.20.0 resolves these issues:
Cluster Management
174741420
Resolves an issue where gpstop
was throwing an error and failed to stop the cluster if the standby master host was unreachable.
Query Optimizer
31891
Resolves a query performance issue on a large table where GPORCA’s default behaviour was to use a group aggregate. GPORCA now introduces the gp_array_agg
hash aggregate that can be used in this circumstance.
Server
32038
Resolves an issue where SELECT * from <view>
was intermittently failing with the message “FATAL: Internal error: Using fd > 65535 in MPP_FD_SET”.
32089
Resolves an issue where the server was failing to acquire resources on one or more segments due to shared snapshot collisions. The code now handles cleanup of worker processes differently during idle cleanup session timeouts.
31606
Resolves an issue where the identical query was running significantly more slowly in 6.x than in 5.28 because an incorrect amount of memory was allocated to the query.
32009, 12986
Resolves an issue with the gp_sparse_vector
module where using the ==
operator against a NULL
operand generated a PANIC.
Resolves an issue where index corruption could occur when an invalid snapshot was used with an AO table.
Resolves an issue where calling the
gpmt
utility with theprimarymirror_lengths
option was failing due to unavailability of thepg_filespace_entry
table. Queries have been updated to use thegp_segment_configuration
table instead.Resolves an issue where calling the
gpmt
utility with thetablecollect
option was failing because thepg_class
table no longer has thereltoastidxid
column.
Data Flow
31996, 7321
Resolves an issue where gpload
occasionally failed and reported the error GC object already tracked
due to a double-free of an object.
Release 6.19
Release 6.19.4
Release Date: 2022-03-11
VMware Tanzu Greenplum 6.19.4 is a maintenance release that resolves several issues.
Resolved Issues
VMware Tanzu Greenplum 6.19.4 resolves these issues:
Data Flow
32102, 8012
Resolved an issue where gpfdist
would fail to load some data in a .gz
data file if the file contained multiple end-of-file (EOF) flags. gpfdist
code was modified so that it continues reading from a .gz
data file until it reaches the actual EOF flag.
Server
32098, 32079
If the checkpointer process failed to open segment 1, the segment number was incorrectly reset to 0. This could cause an infinite loop of retries and failures and lead to the pg_xlog
file growing in size with the checkpointer process hanging. The problem was resolved by ensuring that the segment number is not reset to 0 in this situation.
32095
Resolved an issue where gpexpand
was failing with error “table has already been expanded partition prepare”. The gpexpand
code has been modified so that the expand script – a two-stage process – can be run again in the second stage if it fails in the first stage.
Release 6.19.3
Release Date: 2022-02-25
VMware Tanzu Greenplum 6.19.3 is a maintenance release that resolves several issues.
Resolved Issues
VMware Tanzu Greenplum 6.19.3 resolves these issues:
Server
32056
Resolves an issue where querying the gp_toolkit.gp_resgroup_status_per_segment
view returned the error value ... is out of range for type integer
when the OID of a resource group was greater than 2^31.
32026
Resolves an issue where a pg_upgrade
check for heterogenous partition tables failed during a major version gpupgrade
when a Greenplum Database 5.x partitioned table included a dropped column(s).
31977
Resolves an issue where ALTER TABLE ... SET WITH (REORGANIZE=false) DISTRIBUTED RANDOMLY
incorrectly triggered a data redistribution.
Resolves an issue where disk space for a dropped relation was not freed on transaction commit; Greenplum Database now truncates the files.
Resolves a file descriptor leak on AO segment files.
Resolves an issue where the access of an incorrect variable could cause a crash on an out-of-page read.
Resolves an issue where a CREATE TABLE .. AS (...) WITH NO DATA
could crash the server when pg_attribute_encoding
was omitted.
Resolves an issue where the execution of a multi-level correlated query with a high level of nesting could cause a segfault or provide incorrect results.
Query Optimizer
32074
Resolves a performance issue that was observed when GPORCA planned a query on a partitioned table with a large number of columns and indexes.
179048211
In some cases, the Query Optimizer generated incorrect results when the query included an =ALL
operator. This issue is resolved.
Data Flow
32080
Resolves an issue where gpload
, when REUSE_TABLES: true
, could not find a staging table to reuse, and failed to create the staging table in the public
schema. This issue is resolved; gpload
now creates staging tables in the same schema as specified by EXTERNAL: SCHEMA:
.
PostGIS
178647728
The latest PostGIS release fixes the RPATH
values for PostGIS and dependent library ( .so
) files. With this fix, you can use gpupgrade
on systems that have data of PostGIS-defined types including geometry
, geography
, etc., provided that the latest build of PostGIS 2.1.5+pivotal.3 is installed on both the Greenplum Database 5 and Greenplum Database 6 clusters.
Release 6.19.2
Release Date: 2022-02-11
VMware Tanzu Greenplum 6.19.2 is a maintenance release that resolves several issues.
Changed Features
Greenplum Database 6.19.2 includes these changes:
- Greenplum 6.19.2 introduces a change to WAL record replay for AO tables to enable future support of archive recovery.
Resolved Issues
VMware Tanzu Greenplum 6.19.2 resolves these issues:
Server
32068
Resolves an issue where Greenplum Database generated a PANIC and the error ERRORDATA_STACK_SIZE exceeded
during the execution of certain queries after an upgrade to version 6.19.0 or 6.19.1.
32002
Resolves an issue where incremental recovery failed because pg_rewind
was erroneously trying to delete the gpbackup
default directory.
Resolves a memory leak that was introduced with the fix for issue 31740.
Clarifies a misleading error message that was displayed when resource group-based resource managment was active and Greenplum Database detected an out of memory condition. Greenplum now outputs different error messages based on the current states of Vmem, system, query, and resource group memory allocation.
Query Optimizer
Resolves an issue where the Query Optimizer returned incorrect query results when the query plan included a materialize over a bitmap scan by removing the unnecessary projection of recheck/scalar filter columns.
Cluster Management
32012, 31894
Resolves an issue where gpcheckcat
returned the error invalid input syntax for integer
during the orphaned_toast_tables
test because the command did not correctly handle a temporary toast table.
Resolves an issue where invoking gpconfig
or gpinitsystem -p
to set the value of a server configuration parameter(s) comments out any parameter setting in the postgresql.conf
file whose name starts with the same text string.
32042
Resolves an issue introduced in Greenplum Database 6.19.0 where gpinitisystem
was failing during the add mirrors portion of initializing, due to entries missing from the pg_hba.conf
file.
Data Flow
31936, 31913
In certain cases, an INSERT
operation performed on a gpfdist
external writable table both did not write the complete data set and reported no error. This issue is resolved; gpfdist
now correctly reports and completes error processing when it encounters a retry failure due to poor network conditions.
Release 6.19.1
Release Date: 2022-01-21
VMware Tanzu Greenplum 6.19.1 is a maintenance release that resolves several issues.
Changed Features
Greenplum Database 6.19.1 includes these changes:
- Greenplum Database 6.19.1 introduces the server configuration parameter gp_log_resqueue_priority_sleep_time. You can use this debug parameter when resource queue-based resource management is active to enable statement sleep duration logging for post-query concurrency analysis.
- When
wal_level=minimal
, Greenplum Database now suppresses insert and truncate WAL records for mirrorless clusters to decrease disk space usage and to improve performance of AO/AOCO operations. - The
acl
andowner
tests are no longer run by default when you invoke thegpcheckcat
utility. You can choose to run one or both of these test suites by specifying the-R
option to the command. - The
gpexpand.status_detail
table includes a changed column (root_partition_name
is now namedroot_partition_oid
) and a new column (rel_storage
).
Resolved Issues
VMware Tanzu Greenplum 6.19.1 resolves these issues:
Server
31957, 12930
Resolves a runtime performance issue that was observed when a query that included a NOT IN / IN
construct was more performant when run on Greenplum 5.x compared to Greenplum 6.x.
31850, 12703
Resolves an issue where Greenplum Database might PANIC if a primary segment goes down during a transaction that operates on a cursor.
31019
Resolves an issue where the gpexpand.status_detail
table did not allow for tracking partitions during expansion by expanding each leaf partition separately, in parallel.
Resolves an issue where the Query Dispatcher failed when it encountered an error processing nextval()
and subsequently cancelled all unfinished Query Executors.
Resolves an issue where Greenplum Database incorrectly restored a null-valued server configuration parameter.
Resolves an issue where Greenplum Database logged an INFO
-level message with the text provided user name (<username1>) and authenticated user name (<username2>) do not match
because it performed an unnecessary authentication check.
Synchronizes the Greenplum Database amgetbitmap
definition with upstream PostgreSQL.
Resolves an issue where an intermittent LWLock
panic error occurred when canceling a query that is waiting on a resource queue.
Resolves an issue where PANIC errors were occurring due to the server erroneously collecting execution statistics when the query failed.
Query Optimizer
31884
Resolves an issue where the Query Optimizer generated a plan for a query that included a subquery ALL
construct that could generate incorrect results or crash during execution.
31986
Resolves an issue where a query crashed during optimization when the Query Optimizer relied on statistics that were not yet derived; Greenplum Database now falls back to the Postgres Planner.
31953
Resolves an issue where a query returned the error could not determine which collation to use for string comparison
when the Query Optimizer could not determine the collation string to use for a string comparison when the column was generated by an expression.
31956
Resolves an issue where users could not cancel analyzedb
because process interrupts were not being handled correctly.
180241036
In some cases, the Query Optimizer generated an incorrect cardinality estimate when a predicate contained an ALL
operator. This issue is resolved.
Cluster Management
31311
Resolves an issue where gpcheckcat
ran slowly due to long-running acl
and owner
tests by removing these test suites from the default set of tests that are run.
Release 6.19.0
Release Date: 2021-11-24
VMware Tanzu Greenplum 6.19.0 is a minor release that includes feature changes and resolves several issues.
Features
Greenplum Database 6.19.0 includes these new and changed features:
- Greenplum 6.19.0 includes the utilities gpmemwatcher and gpmemreport for tracking and analyzing memory usage.
- The Greenplum Platform Extension Framework (PXF) software is removed from the Greenplum Database Server distribution. If you are running PXF from your Greenplum installation (
$GPHOME/pxf
), you must download and install a PXFrpm
ordeb
package to continue using the service in your Greenplum Database 6.19.0+ cluster. See Upgrading Greenplum in these release notes and Upgrading PXF When you Upgrade Greenplum 6 for upgrade information. - Greenplum 6.19.0 introduces three new connection server configuration parameters: gp_dispatch_keepalives_count, gp_dispatch_keepalives_idle, and gp_dispatch_keepalives_interval. You can use these parameters to tune TCP keepalives between a Query Dispatcher and its Query Executors.
- Greenplum 6.19.0 includes commits from PostgreSQL releases 9.4.24 to 9.4.26.
- The s3 external table protocol now supports a
config_server
option. You can use this option in theCREATE EXTERNAL TABLE
LOCATION
clause to specify thehttp/https
server URL from which Greenplum Database obtains thes3
protocol configuration file. - gpload adds a
NEWLINE
property to the load control file. You can use this property to specify the type of line break present in text and CSV files that you load with the tool. - Greenplum 6.19.0 introduces the enable_implicit_timeformat_YYYYMMDDHH24MISS server configuration parameter. You can set this parameter to
on
to instruct Greenplum Database 6.19+ to enable the deprecated implicit conversion of a string with the timestamp format YYYYMMDDHH24MISS into a valid date/time type. - The Query Optimizer (GPORCA) improves optimization time performance by not generating nested loop join alternatives when a hash join is possible. You can configure GPORCA to restore its prior behavior by setting the new optimizer_force_comprehensive_join_implementation server configuration parameter to
true
.
Resolved Issues
VMware Tanzu Greenplum 6.19.0 resolves these issues:
Cluster Management
31849
Resolved an issue where the memreport
utility was failing with the error Unable to find postmaster process
due to incorrect parsing options.
179158687
Resolved a gpinitsystem
failure that could occur if .bashrc
was configured to display banner text. A marker has been added to the output so that gpinitsystem
can ignore banner text when interpreting output messages.
180235316
Resolved an issue where the log file did not show that Greenplum was started in restricted mode if master only mode was used ( gpstart -m -R
).
180291768, 179715424
Resolved a packcore/gdb test failure that occurred when environment variables were set on RHEL8 systems.
Data Flow
31768
Resolved an issue where gpload
was reporting the error No such file or directory",,,,,,"CREATE EXTENSION IF NOT EXISTS dataflow;
when connecting to a Greenplum Database 5.x. This fix checks the Greenplum version to determine whether it should create a new extension.
Greenplum Installation
572
Resolved an issue with the rpm
installers, so installing Greenplum does not overwrite the symlink if the install file is for a different major version.
Query Planner
31759
Resolves in issue where an EXPLAIN ANALYZE
query never completed by improving optimization time performance. GPORCA now deactivates the generation of nested loop join alternatives when a hash join is available.
31712
Fixed a bug where a window aggregate query crashed during execution by improving MemoryContextContains()
.
31391
Improved the default optimization cost model for GPORCA since the exhaustive2
algorithm was causing performance regression.
180418263
Removes a stale assertion that caused GPORCA to generate different plans when running Greenplum Database with and without asserts.
180467007
Resolves an issue where GPORCA generated an incorrect plan for a CTAS operation because it created a hash distribution spec with the wrong operator families.
Server
Postgres CVE fixes
This release backports the following Postgres CVE fixes:
- CVE-2021-23214 Server processes unencrypted bytes from man-in-the-middle.
- CVE-2021-23222 libpq processes unencrypted bytes from man-in-the-middle.
31952
Resolved an issue where gpload
reported the error date/time field value out of range: <date>
when loading data originating in Greenplum 5 by introducing the new server configuration parameter enable_implicit_timeformat_YYYYMMDDHH24MISS
.
31940
When global_deadlock_detector
was set to on
, parallel update jobs could sometimes fail and cause a segment crash (SIGSEGV). When global_deadlock_detector
is enabled, the Query Executor can become blocked by another transaction’s UPDATE operation. However, when the Query Executor later became unblocked it would attempt to re-initialize all sub-plans even if they were not needed for the plan being re-evaluated. Greenplum now considers subplan motions when re-initializing, and correctly throws an error instead of causing a PANIC condition.
31887
Resolves a resource queue issue where a session with multiple active portals did not decrement the active statement count following a deadlock report or statement cancellation.
31808
When running a COPY
command using SELECT
, WHERE
and ON SEGMENT
, Greenplum was not saving the segment data correctly. This issue is now resolved.
31654
Resolved an issue where not all spill files were using the GUC temp_tablespaces
. Now all spill files will use temp_tablespaces
if set.
Fixes Query Catalog Table errors when using targetlists
with generate_series
functions, and Index Only Scan plans.
Resolves generation of faulty plans that could result in segfault on indexed tables. The issue occurred when path nodes with bottleneck locus types (Entry and SingleQE), and equal numsegments
, were moved.
Resolves a PL/pgSQL function infinite recursion issue by always writing a snapshot file to the default tablespace, even when temp_tablespaces
is set.
12860 - pg_upgrade
Deactivates the pg_upgrade
check for line datatypes.
Resolves an issue where Greenplum Database may incorrectly drop ORDER BY
and DISTINCT ON
clauses in a subquery when it pulls up sublinks.
<!–
12212 - gpexpand
Resolves an issue where Greenplum Database copied the postgresql.auto.conf
file that resided on the master host to the new host(s) during an expand operation. Because the segment hosts may have different server configuration parameter settings, Greenplum now copies a postgresql.auto.conf
file that resides on an existing segment host to the new host(s). –>
Fixed a bug that caused data corruption during scan on bitmap index when there was an INSERT
operation running in the backend.
Improved the logging and adjusted log level from INFO to DEBUG for the message We thought we were done (...) but libpq says we are still busy
.
179498672
When using mirrorless clusters, queries did not fail during virtual machine restart. This issue has been fixed by adding new GUCs to tune TCP keepalives for connections between the Query Dispatcher and Query Executors: gp_dispatch_keepalives_count, gp_dispatch_keepalives_idle, and gp_dispatch_keepalives_interval.
Release 6.18
Release 6.18.1
Release Date: 2021-10-29
VMware Tanzu Greenplum 6.18.1 is a maintenance release that resolves several issues.
Resolved Issues
VMware Tanzu Greenplum 6.18.1 resolves these issues:
Server
31756
In some cases, an INSERT
query that was run on a table containing a sequence column could not be cancelled. This issue is resolved; Greenplum Database now adds an interrupt check when it processes the next value of a sequence.
31740
Resolved an issue where incremental gprecoverseg
failed to bring a mirror segment back online. This was due to the checkpointer process on a newly recovered mirror failing because the recovery process was searching for a file that had been rightfully deleted.
Introduces a contentId
( %c
) parameter to the WAL archive_command
and restore_command
; this parameter identifies a WAL stream from a given segment.
Removes the redundant calls to Gpmon_Incr_Rows_Out()
that made the rowsout
field of gpperfmon qexec
packets incorrect for Sort
nodes.
Resolves an AOCO table inconsistency that occurred when an ALTER TABLE ... ADD COLUMN
operation was rolled back.
179726460
The pg_xlogdump
utility was not providing sufficient information to assist in diagnosing bitmap index issues. This has been resolved. pg_xlogdump
now includes bitmap-related information.
179719406
The pg_log
utility was not providing sufficient information to assist in diagnosing bitmap index issues. This has been resolved. The bitmap index name is now included in Greenplum Database logs.
Query Optimizer
31784
If ANALYZE
was performed against a leaf partition as part of a transaction, statistics for the leaf were never merged. ANALYZE
would first update the leaf statistics, but when checking to determine if all leaves have been analyzed, it would use a cache that did not include the updated number of rows. This would then trigger a sampling of the root table, which can be expensive. The problem was resolved by advancing the command counter to make the page and tuple updates visible when merging the leaf statistics.
31774
Resolves an issue where the Query Optimizer generated an incorrect plan and returned wrong results on a select count(*)
query when it incorrectly handled a nested set-returning function.
31767
In some cases, the Query Optimizer ran out of memory when planning a query because it did not correctly propagate the OOM error that would ultimately cancel the query. This issue is resolved in the Query Optimizer runaway cleaner exception handler.
Release 6.18.0
Release Date: 2021-10-8
VMware Tanzu Greenplum 6.18.0 is a minor release that includes feature changes and resolves several issues.
Features
Greenplum Database 6.18.0 includes these new and changed features:
- The
gpstate -e
andgpstate -s
commands now provide more detailed output about the status of primary-mirror segment WAL synchronization. Moreover, as part of these output changes:- The
gpstate -s
output fieldsChange tracking data size
,Estimated total data to synchronize
, andData synchronized
output fields are now gone. - The
Mirror status
field ingpstate -s
output for primary segments now has just two valid values: Synchronized or Not in Sync.
- The
- Greenplum 6.18.0 introduces a new Query Optimizer server configuration parameter, optimizer_xform_bind_threshold. You can use this parameter to reduce the optimization time and overall memory usage of queries that include deeply nested expressions by specifying the maximum number of bindings per transform that GPORCA produces per group expression.
- The new gp_autostats_allow_nonowner server configuration parameter enables you to configure Greenplum Database to trigger automatic statistics collection on a table when the table is updated by a non-owner. This parameter is off by default.
- Greenplum 6.18.0 introduces the new contrib module gp_legacy_string_agg. This module implements the single-argument
string_agg( text )
function that is available in Greenplum 5; you may choose to use the module to aid migration to Greenplum 6. - The license file for the Windows Client and Loader Tools Package was updated to the latest version.
- Greenplum 6.18.0 removes the
~/.gphostcache
file; the management utilities now use an alternate mechanism to map hostnames to interfaces. - To enhance the supportability of the product and to aid debugging efforts, Greenplum Database now reports both the reserved and maximum virtual memory allocation when it encounters an out of memory (OOM) condition.
Resolved Issues
VMware Tanzu Greenplum 6.18.0 resolves these issues:
Server
Postgres CVE fixes
This release backports the following Postgres CVE fixes:
- CVE-2020-25696 psql’s \gset allows overwriting specially treated variables.
- CVE-2020-25695/ Multiple features escape “security restricted operation” sandbox
- CVE-2021-32027/Buffer overrun from integer overflow in array subscripting calculations
31736, 31727
When the log_lock_waits
GUC was enabled it resulted in spurious deadlock reports and orphaned wait queue states which, in turn, could lead to memory corruption of certain internal tables. This issue is resolved by disabling the log_lock_waits
GUC.The log_lock_waits
GUC is not supported by Greenplum Database.
31736 - Resource Queues
Due to improper error handling, Greenplum Database raised a duplicate portal identifier warning that was in some cases immediately followed by an out of shared memory error. This issue is resolved; Greenplum Database now raises distinct errors for duplicate portal identifier and out of shared memory.
31734
Greenplum was generating the error ERROR: interconnect error: A HTAB entry for motion node 77 already exists
after creating the extension for the ltree
module. This issue has been resolved.
31725 - Execution
In some cases, Greenplum Database generated a PANIC when the user cancelled a query on an AO table due to a double free of a visimap object. This issue is resolved.
31708 - Catalog and Metadata
Resolves an issue where a non-superuser who ran VACUUM FULL
on a table that they had no permission to access could block further access to the table by currently running transactions. Greenplum Database now performs the permission check before it acquires a lock on the table.
31704 - Planner
In some cases, Greenplum Database returned an incorrect plan when an UPDATE
statement included a subquery. This issue is resolved.
31679 - Functions and Languages
Resolves an issue where a function invocation failed with the error cannot create a unique ID for path type: 116
by disallowing the unique row id path when Greenplum Database encounters an index-only scan.
31654 - Storage
Resolves an issue where Greenplum Database did not honor the temp_tablespaces
setting when it generated temporary files during a sort operation on a large data set. Greenplum now places these temporary files in a tablespace specified in temp_tablespaces
.
31617 - Resource Groups
A database instance was failing to start up, with the message Command pg_ctl reports Master gdm instance active
because a mirror segment couldn’t be recovered. This was due to the incorrect type of lock being used when creating or altering a resource group. This issue is resolved.
31615 - Analyze
Resolves an issue where Greenplum Database did not collect statistics on a table when the table was updated by a non-owner. Greenplum Database 6.18 introduces the new server configuration parameter gp_autostats_allow_nonowner (default is off) to enable automatic statistics collection when a non-owner updates a table.
31517
Fixed a bug with function fixup_unknown_vars_in_setop
that would cause Greenplum Database to PANIC
.
31385 - Planner
Resolves an issue where Greenplum Database returned wrong results for a query because it chose an incorrect motion type during plan tree iteration.
Reinstates the Greenplum Database-specific --wrapper
and --wrapper-args
options to the pg_ctl
command that were available in Greenplum Database 5.
Fixes an environmental variable generation issue caused when symlinks were pointing to alternative locations than the ones expected.
Resolves an issue where Greenplum Database, during node recovery, generated an error when the field standby_mode=on
was set in the recovery.conf
file. The error was similar to:
"FATAL","22023","recovery command file ""recovery.conf"" request for standby mode not specified",,,,,,,0,,"xlog.c",5465
Greenplum Database now supports server recovery in a non-continuous mode using standby_mode=on
.
pg_relation_filenode()
function did not provide proper output for Append Optimized (AO) auxiliary tables. This issue has been resolved.
Resolves an issue where Greenplum Database threw an assertion failure when the size of an AO table exceeded 1GB.
Resolves an issue where Greenplum Database failed to complete a query when the target relation of an UPDATE
or DELETE
operation was a partition table and Greenplum generated a unique row id plan.
12299 - gpexpand
Resolves an issue where WAL-replication was using excessive external bandwith due to misuse of the hostname and address in gp_segment_configuration
. Greenplum Database now uses the primary’s address
for WAL-replication in gpexpand
.
Greenplum Database would initialize pg_aocsseg
table entries with frozen tuples to ensure these entries were implicitly visible even after a rollback. This strategy created issues with the roll back of Append Optimized Columnar (AOC) tables. This issue has now been resolved.
In some cases, the value of a server configuration parameter could be inconsistent among the query dispatcher (QD) and/or query executors (QEs) when the parameter value was updated and then reset in the same session. This issue is resolved.
Query Optimizer
31733
Queries were crashing due to GPORCA prematurely terminating the motion node before interconnect was torn down. This issue is now resolved.
31640
Running queries with GPORCA enabled generated the error ERROR: could not open existing temporary file "base/pgsql_tmp/pgsql_tmp_SIRW_145011_97_0": No such file or directory
, caused by temporary file name changes during cross-slice communication. This issue has been resolved.
179345712
Resolved an issue with the minirepro
utility where it generated incorrect input when inserting columns containing arrays.
179244161
Resolved an issue where Greenplum processes crashed when trying to write an error message to the log due to a null pointer dereference.
167242211
Resolved an issue that caused wrong results when running queries with GPORCA when having the IS DISTINCT FROM FALSE
predicate inside a NOT EXIST
subquery.
Cluster Management
31746
Resolves an issue where Greenplum Database threw cosmetic errors during gpstate
execution when a database of the same name as the $USER
running the command did not exist.
31581
gpconfig
would fail with error similar to ValueError: filedescriptor out of range in select()
due to liveness checks performed on all hostnames associated with each database id. This issue has been resolved and liveness checks are now performend only on unique hostnames.
31558 - gpinitsystem
If a gpinisystem
operation failed before creating the necessary segment backout scripts, manual steps were required to clean up the directories after the failure. To resolve this problem gpinisystem
now creates a single backout script earlier in the process; the script can be used to cleanup all segment directories even if a failure occurs later in the gpinisystem
operation.
179336062
The Greenplum utilities gprecoverseg
, gpinitstandby
, gpstart
, and gpstop
fail when there is a message banner in .bashrc
. This issue has been resolved, and now banner messages are not parsed.
178936675 - gpstate
Beginning in Greenplum 6.0, the gpstate
utility no longer provided mirror synchronization status. This problem was resolved by adding the Mirror status
field in gpstate -s
output for primary segments.
178831426 - gpcheckperf
When executing gpcheckperf
via gpssh
, the gpcheckperf
utility could issue a pkill -f
command that killed the gpssh
process and disconnected the established SSH connections. This problem was resolved by removing the use of pkill -f
in gpcheckperf
.
178761563
Removes unrelated output message regarding netperf
test, that was shown when running gpcheckperf -r N.
178637128
Fixes the issue of the missing --help
option for gpcheckcat
.
Data Loading
N/A
Reverts a previously-committed gpload
improvement that removed a left-join merge because it introduced a performance regression in certain situations.
31613, 12454
Resolves an issue where a COPY
command failed to read a file that included special characters at the end of a line because Greenplum Database did not recognize the EOL character.
Release 6.17
Release 6.17.7
Release Date: 2021-10-2
VMware Tanzu Greenplum 6.17.7 is a maintenance release that resolves a single issue.
Resolved Issue
Server
31818
The fix for issue 11308, added in Greenplum 6.16, introduced a regression that could cause incorrect results for queries having a bitmap index condition. This problem was resolved in 6.17.7 by reverting the original fix.
Release 6.17.6
Release Date: 2021-9-25
VMware Tanzu Greenplum 6.17.6 is a maintenance release that resolves a single issue.
Resolved Issue
Server
31807
Incorrect bitmap index page data was written to WAL logs, resulting in bitmap index corruption after failover to mirror segments or after crash recovery on primaries. This issue has been resolved.
Release 6.17.5
Release Date: 2021-9-22
VMware Tanzu Greenplum 6.17.5 is a maintenance release that resolves a single issue.
Resolved Issue
Postgres Query Planner
A previous fix to resolve duplicate sequence values (178253995) resulted in a performance regression. An incomplete hash key in the fix caused most hash table lookups to fail, which increased the time required to insert data having columns with sequence values. The problem was resolved by ensuring that the hash key contains complete information.
Release 6.17.4
Release Date: 2021-9-21
VMware Tanzu Greenplum 6.17.4 is a maintenance release that resolves several issues.
Resolved Issues
VMware Tanzu Greenplum 6.17.4 resolves these issues:
Query Optimizer
31715
A memory allocation function failed to handle a null pointer exception, which could lead to a system panic when creating the memory pools on Greenplum segments. This issue was resolved, by raising an exception if the memory allocation fails.
31732
A query plan with a nestled loop join and a dynamic table/index scan on the inner side of the join could cause an out of memory exception during a high concurrency workload. This was caused by an inefficient use of executor memory in the dynamic table/index scan operators. This issue has been resolved.
Server
31776
Segment failover caused bitmap index corruption and returned the error Invalid page in block 0 of relation
. This was caused by the metapage not being present in the shared buffers. This issue has been resolved.
Release 6.17.3
Release Date: 2021-8-30
VMware Tanzu Greenplum 6.17.3 is a maintenance release that resolves several issues.
Resolved Issues
VMware Tanzu Greenplum 6.17.3 resolves these issues:
Query Optimizer
31714
In some cases, when processing queries that contained inner joins, the optimizer failed to process statistics, which resulted in a PANIC. This has been resolved.
Server
31726, 31696
A lock table corruption issue caused the Greenplum Database master segment to PANIC if a query waiting for a resource queue lock was cancelled, or if the query errored with a deadlock report. This issue is resolved.
Release 6.17.2
Release Date: 2021-8-6
VMware Tanzu Greenplum 6.17.2 is a maintenance release that resolves several issues.
Resolved Issues
VMware Tanzu Greenplum 6.17.2 resolves these issues:
Cluster Management
31558
This fix creates a single backout script when gpinitsystem
runs so it is possible to clean up all segment directories in case of an error.
178761563
The gpcheckperf
output displayed a NOTICE
about deprecated flags for gpnetbench
, however these flags are still used in netperf
. This fix removes the NOTICE
message.
178637128
The command gpcheckcat
now has a --help
option.
31581
For large clusters gpconfig
returned the error filedescriptor out of range in select()
due to the liveness check being performed on all hostnames associated with each Database ID. This fix addresses this issue by performing liveness check only on unique hostnames.
Extensions
31610
This version introduces a new type extension pljavat
which only installs a language handler for the trusted language pljava
and not the untrusted language pljavau
.
Query Optimizer
178747560
For queries that insert from a replicated table into a distributed table, the query optimizer could generate a plan that produced incorrect results. Specifically, for operations involving a SEQUENCE
function on the replicated table, the result was not guaranteed to be identical across different segments. This issue was resolved by identifying these scenarios and redistributing tuples from a single segment, rather than exploiting the advantages of replicated table.
31609
The query optimizer generated a better plan with optimizer_join_order
set to “greedy” in comparison to the default value of “exhaustive.” This occurred because the default join order algorithm was partial to finding a plan that exploited dynamic partition elimination (DPE) and sometimes ignoring cases where both static and dynamic partition elimination would improve plan performance. This was done to prevent increasing the optimization time significantly at the expense of a sub-optimal plan. To avoid generating sub-optimal plans, the query optimizer now generates both partition elimination options but only for the greedy algorithm, regardless of the join order chosen. This helps to generate some lower-cost alternatives while incurring only a moderate increase in optimization time (15%).
31556
Creating a temp table with DISTRIBUTED RANDOMLY
from a DISTRIBUTED REPLICATED
table put all records into one instance when the query optimizer was enabled. This issue was resolved by adding the necessary logic to insure that, when inserting from a replicated table into a randomly-distributed table, the query optimizer randomizes the tuples to data skew.
31515
GPORCA could generate wrong results if a query used both a WITH
clause and a join condition across different output columns of the same derived tables. The problem occurred because the distribution policy created for the WITH
clause subquery was incorrectly propagated. GPORCA has resolved this problem by pushing properties into the WITH
clause only when all columns come from the same derived table.
31448
Incorrectly estimating cardinality for casted columns resulted in a bad join order, which could lead to a query that did not finish. This issue was resolved by updating the cardinality estimation model when the input histogram is not well formed, as is the case when columns are casted.
Server
31564
Resolved an issue where a database server process was not properly holding the partitionLock
and ResQueueLock
after releasing a resource queue lock following a deadlock error, which resulted in a segmentation fault.
gpload
31466
Resolved an issue where gpload
would fail with column names that used uppercase or mixed-case characters. gpload
now automatically adds double quotes to column names that are not already quoted in the YAML control file.
Release 6.17.1
Release Date: 2021-7-23
VMware Tanzu Greenplum 6.17.1 is a maintenance release that resolves several issues and changes one feature.
Changed Feature
Deactivated unsafe TLS 1.0 and TLS 1.1 protocols for gpfdist
.
Resolved Issues
VMware Tanzu Greenplum 6.17.1 resolves these issues:
Postgres Planner
31630
Fixed a problem where the Postgres Planner failed to consider aggregates on the entry flow. This could cause queries to fail with: ERROR: MIN/MAX subplan has unexpected flowtype
.
Query Optimizer
31522
During a left outer join query, with an index on the join column, the Query Optimizer explored a plan search space that hit an exception, was unable to generate a plan, and reverted to the Postgres Planner. This issue has been fixed by changing the requested collocation property, and eliminating the invalid search space.
174732670
The query optimizer added an unnecessary Explicit Redistribute Motion when generating a plan for queries using DELETE
on a randomly distributed table. This issue has been resolved.
Server
Resolves a data corruption issue that could occur with concurrent DML queries on AO/AOCO tables when Greenplum Database was not aware of committed transactions that started after snapshot initiation.
Release 6.17.0
Release Date: 2021-7-9
VMware Tanzu Greenplum 6.17.0 is a minor release that includes feature changes and resolves several issues.
Greenplum Database 6.17.0 includes these new and changed features:
Features
- The PXF version 6.1.0 distribution is available with this release; you can download it from the Release Download directory named Greenplum Platform Extension Framework on VMware Tanzu Network. Refer to the PXF documentation for information about this release and for installation and upgrade instructions.
- The
gprecoverseg
,gpaddmirrors
, andgpmovemirrors
utilities now include a-b
option to specify the maximum number of segments per host to operate on in parallel. - The
gpecheckcat
utility now permits users to skip one or more tests, with the new-s
option. In addition, the-R
option now accepts a comma-separated list of of multiple tests to run. - The Progress DataDirect JDBC Driver v6.0.0+181 is included in this release. See the Readme file for a list of new features available in v6.0.0
Resolved Issues
VMware Tanzu Greenplum 6.17.0 resolves these issues:
Server
178150185
Resolved an issue where the message, An exception was encountered during the execution of a statement
was unable to be suppressed. Users can now control the verbosity of the exception message with the log_min_messages
GUC.
31335
Resolved an out of memory condition that could occur because the server held certain data contexts longer than necessary.
31185
When altering a role from superuser to non-superuser, the role resource group was not changing from admin_group
to default_group
. This fix addresses this issue.
31320
Resolved an issue where issuing a UPDATE/DELETE
statement while a VACUUM operation was running returned the error tuple concurrently updated
, due to an out of date snapshot.
31435
Fixed the error unrecognized parameter "appendoptimized"
that was reported when using the appendoptimized
clause for a table partition definition.
31584
Resolved a bug that caused the database to crash when running queries against tables that had not been redistributed after an expansion.
31588
Resolved an issue where a CTAS
statement referencing a table created with the DISTRIBUTED REPLICATED
clause was generating incorrect results with the Legacy Planner.
Query Optimizer
31481
For partitioned tables with indexes, ORCA would cause a PANIC
while trying to create an index plan, when root and leaf partitions did not have the same underlying dropped column structure. This issue has been resolved.
31463
Running queries with the GPORCA query optimizer that used an INNER JOIN
with an EXCEPT
clause was causing a fallback to Legacy Planner and reporting the error: No plan has been computed for required properties
. This issue has been resolved.
Resolved an issue where running queries with a LEFT OUTER JOIN
were producing incorrect results as it was being replaced with an INNER JOIN
in the subquery plan.
31440
Resolved an issue where running a query using UNION ALL
and an external table was returning incorrect results with GPORCA query optimizer, due to not all information from the external table being gathered.
177874343
The ORCA Query Optimizer was unnecessarily opting out of certain optimizations when it encountered a NO SQL
function. It no longer does that.
178477120
Introduced a fix to verify that materialize does not project in ORCA Query Optimizer.
Resolved an issue with array overflow for function argument types which was causing memory corruption and a database crash with ORCA Query Optimizer.
31527
Resolved a bug in the implementation of Left Outer Joins that was causing a database crash when using ORCA Query Optimizer.
Cluster Management
178173416
Resolved an issue where gprecoverseg -r
– when it called gprescoverseg
internally a second time – did not correctly pass on the -B
and -b
options.
31417
Resolved an issue where gprecoverseg
was updating the pg_hba.conf
file on all primaries regardless of whether those primaries needed to be updated.
31350
Resolved an issue where gpmovemirrors -B
was calling gprecoverseg
with an excessively large number of parallel processes.
31338, 31351
Resolved an issue where old mirror directories were not being removed from old tablespace directories after a mirror was moved using gpmovemirrors
.
31419
Resolved an issue where gprecoverseg
was printing out empty pg_rewind
progress lines for mirrors.
31519
gprecoverseg
no longer adds pg_hba.conf
entries if they are already present in the file.
31579
For all GUCs with a vartype of string
, you may no longer enclose the value you pass to gpconfig -c
in single quotes.
178254153
Resolved an issue where gprecoverseg -p
was reporting an error despite the operation being successful.
177862886
Resolved an issue where gprecoverseg -p
would error out when master could not connect to an individual segment host.
Postgres Query Planner
178238691
Resolved an issue where an error occurred when the planner was executing a query that performed a UNION ALL
between a replicated table and a subquery with an explicit sequence nextval
.
178253995
Resolved an issue where the sequence executor node was generating duplicate sequence values.
Release 6.16
Release 6.16.3
Release Date: 2021-07-02
VMware Tanzu Greenplum 6.16.3 is a maintenance release that resolves several issues.
Resolved Issues
VMware Tanzu Greenplum 6.16.3 resolves these issues:
Server
31549, 31577
Resolved an issue that made Greenplum unable to cast unknown-type literals to cstring
in certain circumstances. This problem could surface in several ways including query panics, out of memory conditions, and the following compression error: "ERROR","XX000","compressed data is corrupt (pg_lzcompress.c:737)"
.
Analyze
31589
When running ANALYZE
on a leaf table, Greenplum would compute statistics for all columns when merging statistics for the parent table, even if one or more columns had been configured with STATISTICS
set to 0 (to deactivate statistics collection for that column). Greenplum no longer computes statistics for columns that have STATISTICS
set to 0 when merging statistics for the parent table.
Release 6.16.2
Release Date: 2021-06-04
VMware Tanzu Greenplum 6.16.2 is a maintenance release that resolves several issues.
Resolved Issues
VMware Tanzu Greenplum 6.16.2 resolves these issues:
Cluster Management
178140424
Resolved an issue where, when gprecoverseg -v
was invoked, pg_rewind
’s log was generated but was not preserved in case of an incremental recovery failure.
Server
178280660
Resolved an issue with the Postgres planner partition selection when the types of the partitioning key and the search values are different.
31436
Resolved an issue in which, when running \df+
on a function that has exec location INITPLAN
, the Execute on
column did not properly display “initplan”.
31335
Resolved an out of memory condition that could occur because the server held certain data contexts longer than necessary.
Fixed an inconsistency between master and segments regarding the value of collname
when creating a DOMAIN
.
Resolved an issue in which CREATE MATERIALIZED VIEW
was failing with ERROR: division by zero
when WITH NO DATA
was being specified.
Query Executor
31439
Resolved an issue which caused the database to PANIC
due to a double free of the memory context TupleSort
.
Release 6.16.1
Release Date: 2021-5-21
VMware Tanzu Greenplum 6.16.1 is a maintenance release that resolves several issues.
Resolved Issues
VMware Tanzu Greenplum 6.16.1 resolves these issues:
n/a - gprecoverseg
Resolved an issue where using gprecoverseg
to perform an incremental recovery removed log files from the pg_log
directory. gprecoverseg
now retains files under pg_log
so that they can be used for troubleshooting after an incremental recovery.
177336745 - Cluster Management
Resolved an issue with gpconfig
where the process hung when cancelling it due to hosts being unreachable.
31453, 177970816 - Query Optimizer
Resolved an issue in which the optimizer was failing with a segfault while querying a view, if the view had a join between tables and any of the table had a column that was dropped after view creation.
31415 - Server
Resolved an issue in which REFRESH MATERIALIZED QUERY
was failing if the WHERE
clause included an embedded query.
31405 - Server
Resolved an issue in which certain queries resulted in a master panic because of the way temporary files were being tracked.
31383 - Server
Resolved an issue that was causing slow query performance for view creation when using the NOT in
clause.
31380 - Cluster Management
The new functionality for gprecoverseg
to ignore segments on unreachable hosts had introduced an issue for the gprecoverseg -p newhost
option as it skipped the case of a new host replacing an unreachable host. This issue is now resolved.
31368 - Cluster Management
Resolved an issue with gpmovemirror
which caused the wrong error message to be printed when there was an issue with the provided configuration file.
31361 - Query Executor
Resolved an issue in which multiple segments failed with PANIC errors due to how newly allocated memory was being handled.
31337 - Query Optimizer
Resolved an issue in which the optimizer was crashing – instead of simply throwing an error – when it was running out of memory.
31297 - Server
Resolved an issue in which VACUUM FULL ANALYZE
was failing to clear some tables of bloat.
31279 - Server
Resolved an issue with incremental recovery when a failed segment that was acting as primary had to be recovered. The issue was caused by WAL files required by the recovery being removed.
31110 - Server
Resolved an issue in which a View with an unknown field could not be restored because columns were not being correctly cast from type cstring
to type date
.
30762 - Server
Resolved an issue with gpload
in which a global transaction was not aborted when the session was reset.
29998 - Query Optimizer
Resolved an issue in which the optimizer was returning incorrect results when the query involved a CTE with an EXCEPT
clause. The issue occurred because the query optimizer did not add scalar casts for any input columns whose types did not match the output types of the SetOp
.
Release 6.16.0
Release Date: 2021-4-22
VMware Tanzu Greenplum 6.16.0 is a minor release that includes feature changes and resolves several issues.
Features
Greenplum Database 6.16.0 includes these new and changed features:
- Greenplum Resource Groups include now a new mode for assigning CPU resources by percentage, Ceiling enforcement mode, apart from the existing Elastic mode.
- The PXF version 6.0.0 distribution is available with this release; you can download it from the Release Download directory named Greenplum Platform Extension Framework on VMware Tanzu Network. Refer to the PXF documentation for information about this release and for installation and upgrade instructions.
- Greenplum Streaming Server (GPSS) version 1.5.3 is included, which includes changes and bug fixes. Refer to the GPSS Documentation for more information about this release and for upgrade instructions.
- Greenplum Database 6.16.0 includes MADlib version 1.18.0, which introduces new Deep Learning features, improvements, and bug fixes. See the MADlib 1.18.0 Release Notes for a complete list of changes.
The
gp_sparse_vector
module now installs its functions and objects into a schema namedsparse_vector
. See Resolved Issue 31360.Note: If you are using
gp_sparse_vector
in your current Greenplum Database distribution, review Upgrading the gp_sparse_vector Module for upgrade implications and instructions.The default value of the optimizer_join_order server configuration parameter is changed from
exhaustive2
toexhaustive
, which was the default used in Greenplum versions prior to version 6.14.0. The Greenplum Database Query Optimizer (GPORCA) uses this configuration parameter to identify the join enumeration algorithm for a query. See Resolved Issue 31391.
Resolved Issues
VMware Tanzu Greenplum 6.16.0 resolves these issues:
31391 - Query Optimizer
Resolves an issue where a query using the GPORCA query optimizer took longer to complete on Greenplum Database version 6.15.0 with the optimizer_join_order
server configuration parameter default value of exhaustive2
. The default value of this configuration parameter is changed (back to) exhaustive
.
31360 - gp_sparse_vector
Resolves an issue where the gp_sparse_vector
array_agg()
function overrode the system pg_catalog.array_agg()
function and returned a different type of array. The gp_sparse_vector
module now installs its functions and objects into a separate schema named sparse_vector
.
31333 - Optimizer
GPORCA would generate invalid plans for certain queries with scalar subquery in the projection list, and a predicate that could use an underlying index. This could cause Greenplum Database to PANIC
when running certain functions with the Optimizer enabled. This issue has now been resolved.
31308 - Server
Fixed an issue of queries hanging when logging is configured to write NOTICE
level messages.
31296 - Server
Fixed an issue with gpcheckcat
reporting an error when a VIEW
is defined without a target and does not have an entry in pg_attribute
.
31272 - Data Flow
In dual stack IPv4 and IPv6 hosts, gpfdist
would bind to an IPv4 port but fail to bind to an IPv6 port, if there was another process listening on the same IPv6 port. This issue has been resolved.
31259 - Query Optimizer
When querying partitioned tables that had been defined with open boundaries, the Query Optimizer was performing unnecessary scans. This issue has been resolved.
31247 - Cluster Management
gpstate
would log ERROR/FATAL
messages relating to gpexapnd.status
when no expansion was in progress, and when gpstate -s
output showed a healthy cluster state. This issue has been resolved.
31346 - Query Optimizer
Resolved an issue with incorrect group pathkey that was causing the Greenplum database to go into recovery mode when using GROUP BY GROUPING SETS
.
11655 - Query Optimizer
Fixed the error Lookup of object 0.0.0.0 in cache failed
when querying a partitioned table.
11308 - Server
Fixes an issue where a bitmap index scan running concurrently with an index INSERT
on a full bitmap page, occasionally failed to read the correct tid
.
Release 6.15
Release 6.15.0
Release Date: 2021-3-11
VMware Tanzu Greenplum 6.15.0 is a minor release that includes changed features and resolves several issues.
Features
Greenplum Database 6.15.0 includes these new and changed features:
gprecoverseg
now rebalances segments whose hosts are reachable even if there are other segments whose hosts are not; for segments on unreachable hosts,gprecoverseg
now emits a warning message.Greenplum Streaming Server (GPSS) version 1.5.2 is included, which introduces bug fixes. Refer to the GPSS Release Notes for more information on release content and to access the GPSS documentation.
Note: If you have previously used GPSS in your Greenplum 6.x installation, you are required to perform upgrade actions as described in Upgrading the Streaming Server.
The
analyzedb
utility has a new--skip_orca_root_stats
option. When this option is specified,analyzedb
will not update root partition statistics. This option should be used only if GPORCA is deactivated.
Resolved Issues
VMware Tanzu Greenplum 6.15.0 resolves these issues:
31267 - Server
Fixed error resource group wait queue is corrupted (resgroup.c:3502)
that caused a PANIC
on Greenplum Master.
31262 - Server
Resolved an issue with inconsistencies in returned results when using the grouping
function.
31260/30895 - External Tables
Fixed issue with Web External Tables when escape='OFF'
.
31249 - Query Optimizer
Certain queries that selected random rows in a partitioned table, using functions like random()
or timeofday()
, would cause host PANIC
. This issue has been resolved.
31244 - analyzedb
Added --skip_orca_root_stats
option, which prevents analyzedb
from updating root partition statistics.
31228 - Data Flow
dblink
includes function dblink_connect_no_auth
that skips authentication checks.
31223 - Query Optimizer
Fixed optimizer issue with cardinality estimation for point queries on CDOUBLE
types, such as timestamp
.
31220 - Data Flow
Resolved an issue in which gpfdist
reads from external tables were resulting in “No route to host” errors.
31210 - Interconnect
Resolves a high memory usage issue with proxy background worker processes when gp_interconnect_type = proxy
, and the pause/resume flow control process causes a busy receiver to cache or duplicate packets while waiting for the backend to consume them. Greenplum Database now uses a more active flow control mechanism to control packet buffering and transmission.
31191 - Server
Fixed planner issue of queries crashing with FailedAssertion
error due to planner issue.
31055 - Server
Prohibits the execution of queries with set returning functions in WHERE
clause, which caused segment panic. Greenplum now generates an error similar to: set-returning functions are not allowed in WHERE clause
.
176693327 - Cluster Management
gpinitsystem
now treats equivalent locales as equivalent (for example, en_US.UTF-8
is now treated the same as en_US.utf8
).
176521407 - Cluster Management
Because gprecoverseg
now reports progress on both incremental and full segment recovery, there is no longer any need to call gpstate -m
to determine recovery progress.
Release 6.14
Release 6.14.1
Release Date: 2021-2-22
VMware Tanzu Greenplum 6.14.1 is a maintenance release that resolves several issues and includes related changes
Changed Features
Greenplum Database 6.14.1 includes this change:
- PostGIS version 2.5.4+pivotal.4 is included, which resolves the segfault described in PostGIS ticket #4691.
Resolved Issues
VMware Tanzu Greenplum 6.14.1 resolves these issues:
31258 - Server
Resolves an issue where the array
typecasts of operands in view definitions with operators were erroneously transformed into anyarray
typecasts. This caused errors in the backup and restore of the view definitions.
31249 - Query Optimizer
Resolves an issue where Greenplum Database generated a PANIC during query execution when the Query Optimizer attempted to access the argument of a function (such as random()
or timeofday()
), but the query did not invoke the function with an argument.
31242 - Server
Optimized locking to resolve an issue with certain SELECT
queries on the pg_partitions
system view, which were waiting on locks taken by other operations.
31232 - Server
Resolves an issue where, after an upgrade from version 5.28 to 6.12, a query execution involving external tables resulted in a query PANIC and segment failover. This issue has been resolved by optimizing the query subplans.
31211 - gpfdist
When an external table was configured with a transform, gpfdist
would sporadically return the error 404 Multiple reader to a pipe is forbidden
. This issue is resolved.
176684985 - Query Optimizer
This release improves Greenplum Database’s performance for joins with multiple join predicates.
Release 6.14.0
Release Date: 2021-2-5
VMware Tanzu Greenplum 6.14.0 is a minor release that includes changed features and resolves several issues.
Features
Greenplum Database 6.14.0 includes these new and changed features:
- CentOS/RHEL 8 and SUSE Linux Enterprise Server x86_64 12 (SLES 12) Clients packages are available with this Greenplum Database release; you can download them from the Release Download directory named Greenplum Clients on VMware Tanzu Network.
- The PXF version 5.16.1 distribution is available with this release; you can download it from the Release Download directory named Greenplum Platform Extension Framework on VMware Tanzu Network.
- The default value of the optimizer_join_order server configuration parameter is changed from
exhaustive
toexhaustive2
. The Greenplum Database Query Optimizer (GPORCA) uses this configuration parameter to identify the join enumeration algorithm for a query. With this new default, GPORCA operates with an emphasis on generating join orders that are suitable for dynamic partition elimination. This often results in faster optimization times and/or better execution plans, especially when GPORCA evaluates large joins. The Faster Optimization of Join Queries in ORCA blog provides additional information about this feature. - The default cost model for the optimizer_cost_model server configuration parameter,
calibrated
, has been enhanced; GPORCA is now more likely to choose a faster bitmap index with nested loop joins rather than hash joins. - GPORCA boosts query execution performance by improving its partition selection algorithm to more often eliminate the default partition.
- GPORCA now generates a plan alternative for a right outer join transform from a left outer join when equivalent. GPORCA’s cost model determines if/when to pick this alternative; using such a plan can greatly improve query execution performance by introducing partition selectors that reduce the number of partitions scanned.
- The output of the
gprecoverseg -a -s
command has been updated to show more verbose progress information. Users can now monitor the progress of the recovering segments in incremental mode. - The
gpcheckperf
command has been updated to support Internet Protocol version 6 (IPv6).
Resolved Issues
VMware Tanzu Greenplum 6.14.0 resolves these issues:
31195 - Server: Execution
Resolves an issue where Greenplum Database generated a PANIC when the pg_get_viewdef_name_ext()
function was invoked with a non-view relation.
31094 - Server: Execution
Resolves an issue where a query terminated abormally with the error Context should be init first
when gp_workfile_compression=on
because Greenplum Database ignored a failing return value from a ZSTD
initialization function.
31067 - Query Optimizer
Resolves a performance issue where GPORCA did not consistently eliminate the default partition when the filter condition in a query matched more than a single partition. GPORCA has improved its partition selection algorithm for predicates that contain only disjunctions of equal comparisons where one side is the partition key by categorizing these comparisions as equal filters.
31062 - Cluster Management
Resolves a documentation and --help
output issue for the gprecoverseg
, gpaddmirrors
, gpmovemirrors
, gpinitstandby
utilities, where the --hba-hostnames
command line flag details were missing.
31044 - Query Optimizer
Fixes a plan optimizer issue where the query would fail due to the planning time being dominated by the sort process of irrelevant indexes.
30974 - Server: Execution
Greenplum Database generated a PANIC when a query run in a utility mode connection invoked the gp_toolkit.gp_param_setting()
function. This issue is resolved; Greenplum now ignores a function’s EXECUTE ON
options when in utility mode, and executes the function only on the local node.
30950 - Query Optimizer
Resolves an issue where GPORCA did not use dynamic partition elimination and spent a long time planning a query that included a mix of unions, outer joins, and subqueries. GPORCA now caches certain object pointers to avoid repeated metadata lookups, substantially decreasing planning time for such queries when optimizer_join_order
is set to query
or exhaustive2
.
30947 - Query Optimizer
Resolves an issue where Greenplum Database returned the error no hash_seq_search scan for hash table "Dynamic Table Scan Pid Index"
because GPORCA generated a query plan that incorrectly rescanned a partition selector during dynamic partition elimination. GPORCA now generates a plan that does not demand such a rescan.
11211 - Server
During the parallel recovery and rebalance of segment nodes after a failure, if an error occurred during segment resynchronization, the main recovery process would halt and wait indefinitely. This issue has been fixed.
11058 - Query Optimizer
Resolves an optimizer issue where CTE queries with a RETURNING
clause would fail with the error INSERT/UPDATE/DELETE must be executed by a writer segworker group
.
174873438 - Planner
Resolves an issue where an index scan generated for a query involving a system table and a replicated table could return incorrect results. Greenplum no longer generates the index scan in this situation.
Release 6.13
Release 6.13.0
Release Date: 2020-12-18
VMware Tanzu Greenplum 6.13.0 is a minor release that includes changed features and resolves several issues.
Features
Greenplum Database 6.13.0 includes these new and changed features:
- This release introduces the new VMware Tanzu Greenplum Connector for Apache NiFi 1.0.0. The Connector provides a fast and simple, UI-based way to build data ingestion pipelines for Greenplum Database, code-free. The Connector is available as a separate download from VMware Tanzu Network. Refer to the VMware Tanzu Greenplum Connector for Apache NiFi documentation for installation, configuration, and usage information for the Connector.
Greenplum Streaming Server (GPSS) version 1.5.0 is included, which introduces many new and changed features and bug fixes. Refer to the GPSS Release Notes for more information on release content and to access the GPSS documentation.
Note: If you have previously used GPSS in your Greenplum 6.x installation, you are required to perform upgrade actions as described in Upgrading the Streaming Server.
The distribution includes the
advanced_password_check
contrib
module; you can use this module to specify password string quality policies for Greenplum Database. Refer to the advanced_password_check module documentation for more information.The Greenplum Database Query Optimizer exposes a new server configuration parameter to enable index-only scans. These scans answer queries from an index alone without requiring any heap access. This configuration parameter is named optimizer_enable_indexonlyscan, and is enabled by default.
Resolved Issues
VMware Tanzu Greenplum 6.13.0 resolves these issues:
30994 - Cluster Management
Resolved an issue where gpstart
could fail with an non-descript error when a segment host was unreachable during Greenplum Database initialization. gpstart
now continues the startup process even if some cluster hosts are unreachable.
31028 - Cluster Management
When using gpconfig -s client_min_messages
to set the client messaging level (for example “notice” or “warning”), the output showed “error” instead of the level configured by the user. This issue has been resolved.
10720 - Planner
Resolved a problem that could cause the Postgres Planner to crash or return incorrect results if a query used a grouping expression (rather than a column name) along with other ungrouped targets that referenced the group key.
10961 - Locking
Improved the locking behavior to avoid deadlocks that can occur when creating multiple indexes on an Append-Only table.
30951 - Query Optimizer
Resolves an issue where a query performed on a serial
-type column of a replicated Greenplum table (created DISTRIBUTED REPLICATED
) failed to return a consistent value on all segments.
30976 - Server: Execution
In some cases, Greenplum Database returned incorrect results when it ran certain system functions in an EntryDb (a special query executor (QE) that runs on the master instance). This issue is resolved; Greenplum now returns the error “ This query is not currently supported by GPDB.
” when it encounters a function invocation that it does not support.
31011 - Server: Execution
Resolved a problem that could cause a segfault when spilling hash tables to disk. In Greenplum 6 this problem was reported with a materialized view query that produced a HashAggregate plan.
31073 - Segment Mirroring
Resolves an issue where the kernel Recv-Q buffer filled up on a Greenplum mirror segment instance when the mirror sent statistics to the statistics collector, but the collector was not running because the mirror was not in hot standby mode.
31082 - Server: Execution
Resolved an issue where the planner generated an incorrect plan for some queries that included a merge join and dynamic partition elimination. The storage needed for the merge join key was incorrectly allocated on the inner side of the plan tree. This led to a SIGSEGV when executing the sort node of the inner side of the merge join. The problem was resolved by ensuring that only the nodes that need to use the join key are provisioned with the right amount of storage, leaving the rest of the plan tree intact.
31090 - Storage: Segment Mirroring
pg_rewind
was updated to reduce the number of lstat
operations it performs, in order to improve the performance of incremental recovery with a large number of data files.
173157638 - Cluster Management
When the user specified a valid port range using the gpaddmirrors -p
option, gpaddmirrors
would generate an error similar to error: Value of port offset supplied via -p option produces ports outside of the valid range. Mirror port base range must be between 6432 and 61000
. This issue has been resolved.
Release 6.12
Release 6.12.1
Release Date: 2020-11-20
VMware Tanzu Greenplum 6.12.1 is a maintenance release that resolves several issues.
Resolved Issues
VMware Tanzu Greenplum 6.12.1 resolves these issues:
9207 - Server Standby
Fixed a memory overflow condition that could cause a standby node to shut down with the error, "FATAL","XX000","the limit of 500 distributed transactions has been reached (cdbdtxrecovery.c:571)"
.
11003 - Postgres Planner
Resolved an issue where a query that selected a constant and that specified one or more empty GROUPING SETS
returned incorrect results.
30923 - Server Execution, Planner
Resolved a problem where a query could return incorrect results if segments held a NULL value in an empty set.
30946 - Query Optimizer
A query on a table with a btree
index ran longer than expected because the Query Optimizer did not perform partition elimination when the query included an index join with both a local and a join predicate. This issue is resolved; the Query Optimizer improves dynamic and static partition elimination when indexes are present.
30993 - Optimizer
Fixed an issue where certain IN queries performed slowly because full table scans were used instead of indexes.
30999 - Cluster Management
Fixes an issue where moving a mirror segment to an alternate host, using gprecoverseg -F
, was failing.
31007 - Cluster Management
Fixes an issue where no error was logged or reported when the incremental recovery of a segment using gprecoverseg
failed. The logs now include a message similar to: [WARNING]:-Incremental recovery failed for my-segment-name. You must use gprecoverseg -F to recover the segment.
31014 - analyzedb
Fixed a problem where analyzedb
could fail if a table was dropped and recreated during the analyzedb
operation.
31018 - gpexpand
The redistribute phase of a cluster expansion returned the error ... failed to expand: error ERROR: ... is not a table
when it tried to expand a materialized view. This issue is resolved; Greenplum Databae now supports expanding materialized views.
31026 - Metrics Collector
Resolved an issue where Greenplum Command Center version 6.3.0 and 6.3.1 could not show a visual query plan in the query monitor in certain cases.
31057 - Optimizer
When simplifying constraints during preprocessing, GPORCA did not consider the case where a constraint compared a column to an empty array (for example, EXPLAIN SELECT 1 FROM mytable WHERE mytable.mycolumn=ANY('{}');
). These types of queries would cause GPORCA to crash. This problem was resolved by ensuring that GPORCA now skips merging a constraint with an empty array.
174906043 - Metrics Collector
Resolved an issue where the Greenplum Command Center could not show metrics for CREATE TABLE AS SELECT … FROM
and COPY (SELECT ... FROM ...) TO ...
statements.
175050196 - autovacuum
Resolved a fatal error that could occur when the autovacuum daemon performed a VACUUM
operation on the template0
database.
175372920 - Resource Groups
Resolved an issue that could cause queries to fail if an earlier DROP RESOURCE GROUP
command failed to drop the resource group.
175471857 - Query Dispatcher
Resolved a problem that could cause a transaction to incorrectly use single-phase commit instead of two-phase commit.
275569338 - Query Dispatcher
Resolved a problem that could cause the error, ERROR: unrecognized node type: 2139062143 (copyfuncs.c:6059)
, when the query dispatcher needed to refresh a materialized view.
Release 6.12.0
Release Date: 2020-10-30
VMware Tanzu Greenplum 6.12.0 is a minor release that includes changed features and resolves several issues.
Features
Greenplum Database 6.12.0 includes these new and changed features:
- Greenplum now supports using segment hostnames when defining proxy ports with the
gp_interconnect_proxy_addresses
parameter (previously, IP addresses were required). Note that if a segment instance hostname is bound to a different IP address at runtime, you must executegpstop -U
to re-load thegp_interconnect_proxy_addresses
value. See Configuring Proxies for the Greenplum Interconnect. - Because Greenplum Database does not enforce referential integrity syntax (foreign key constraints), the
TRUNCATE
command was updated so that it truncates a table referenced in a foreign key constraint, even if theCASCADE
option is omitted. - The Greenplum Database 6.12.0 distribution includes the Greenplum Magic Tool (
gpmt
), a diagnostics and data collection tool. - The Greenplum Database 6.12.0 distribution includes the
postgres_fdw
PostgreSQLcontrib
module. Refer to the postgres_fdw module documentation for more information.
Resolved Issues
VMware Tanzu Greenplum 6.12.0 resolves these issues:
174311661 - Query Execution
When executing a long query that contained multi-byte characters, Greenplum could incorrectly truncate the query string (removing multi-byte characters) and, if log_min_duration_statement
was set to 0, could subsequent write an invalid symbol to segment logs. This behavior could cause errors in gp_toolkit
and Command Center. This problem has been resolved.
173190958 - Transactions
In some cases, a Greenplum Database master reset generated one or more orphaned prepared transactions. This issue is resolved; Greenplum now periodically checks for, and aborts, these transactions.
171249005 - Transactions
Resolves an issue where Greenplum Database generated a PANIC when it exhausted retry attempts to abort prepared transactions.
30992 - Transactions
In some cases, Greenplum Database generated a PANIC after reboot due to a race condition between a checkpoint and xlog COMMIT PREPARE
recording. When Greenplum encountered an orphaned prepared transaction that was committed after the xlog was recorded, it returned the error message: cannot abort transaction transaction_number, it was already committed
. This issue is resolved.
30970 - Query Optimizer
In some cases, Greenplum Database crashed when the Query Optimizer attempted to generate an index scan from a predicate that contained a subquery. This issue is resolved; the Query Optimizer now disallows such plans.
30962 - Query Optimizer
The second SELECT
on an external table within a transaction returned zero records because the Query Optimizer did not generate a unique scan number to differentiate the two queries to the external table. This issue is resolved.
30960 - Query Optimizer
Resolves an issue where the Query Optimizer entered an infinite loop when it merged statistics buckets for double
values in UNION
and UNION ALL
queries due to an incorrect comparison of bucket boundary values with a small Epsilon.
30980 - diskquota Module
Resolved an issue that caused master and mirror segments to display the warning, Share memory is not enough for active tables
, when TRUNCATE
and CREATE TABLE
statements were executed.
30942 - Postgres Planner
In some cases, the Postgres Planner crashed or produced incorrect results when the HAVING
clause of a query included a subquery, and one or more columns referenced in the subquery were not also specified in the GROUP BY
column set. This issue is resolved.
10813 - Postgres Planner
Resolved an issue that could cause the Query Dispatcher to crash when creating a query plan for a subquery that has GROUPING SETS
.
10794 - Resource Groups
Resolves an issue where the result of a query on pg_resgroup_get_status(NULL:oid)
could not be saved to a table.
10376 - Query Execution
Resolved an issue where executing CREATE UNIQUE INDEX
on a table partition would implicitly change the partition’s distribution key.
425 - gpbackup
When inserting into a table that is distributed by a bpchar
and using the legacy bpchar
hash operator, rows always used jump consistent hashing instead of legacy (modulo) hashing. This mismatch would cause gprestore
operations to fail for when restoring Greenplum 4.x/5.x data into Greenplum 6.x. The problem occurred because a required hashing function ID was missing from a check function that determined if an attribute used legacy hashing. Greenplum 6.12 resolves this issue by adding the required hashing function ID in the check function.
Release 6.11
Release 6.11.2
Release Date: 2020-10-2
Pivotal Greenplum 6.11.2 is a maintenance release that includes changes and resolves several issues.
Changed Features
Greenplum Database 6.11.2 includes these changes:
- Pivotal GPText version 3.4.5 is included, which includes bug fixes. See the GPText 3.4.5 Release Notes for more information.
- Pivotal Greenplum-Spark connector version 2.0.0 is included, which includes feature changes and bug fixes. See the Greenplum-Spark Connector 2.0.0 Release Notes for more information.
Resolved Issues
Pivotal Greenplum 6.11.2 resolves these issues:
30549 - Management and Monitoring
Greenplum excluded externally-routable loopback addresses from replication entries, which caused utilities such as gpinitstandby
and gpaddmirrors
to fail. This problem has been resolved.
30795 - GPORCA
Fixed a problem where GPORCA did not utilize an index scan for certain subqueries, which could lead to poor performance for affected queries.
30878 - GPORCA
If a CREATE TABLE .. AS
statement was used to create a table with non-legacy (jump consistent) hash algorithm distribution from a source table that used the legacy (modulo) hash algorithm, GPORCA would distribute the data according to the value of gp_use_legacy_hashops
; however, it would set the table’s distribution policy hash algorithm to the value of the original table. This could cause queries to give incorrect results if the distribution policy did not match the data distribution. This problem has been resolved.
30903 / 30966 - Metrics Collector
Workfile entries were sometimes freed prematurely, which could lead to the postmaster
process being reset on segments and failures in query execution, or segment PANIC. This problem has been resolved.
30928 - GPORCA
If gp_use_legacy_hashops
was enabled, GPORCA could crash when generating the query plan for certain queries that included an aggregate. This problem has been resolved.
174812955 - Query Execution
When executing a long query that contained multi-byte characters, Greenplum could incorrectly truncate the query string (removing multi-byte characters) and, if log_min_duration_statement
was set to 0, could subsequent write an invalid symbol to segment logs. This behavior could cause errors in gp_toolkit
and Command Center. This problem has been resolved.
Release 6.11.1
Release Date: 2020-09-17
Pivotal Greenplum 6.11.1 is a maintenance release that includes changes and resolves several issues.
Changed Features
Greenplum Database 6.11.1 includes this change:
- Greenplum Platform Extension Framework (PXF) version 5.15.1 is included, which includes changes and bug fixes. Refer to the PXF Release Notes for more information on release content and to access the PXF documentation.
Resolved Issues
Pivotal Greenplum 6.11.1 resolves these issues:
30751, 173714727 - Query Optimizer
Resolves an issue where a correlated subquery that contained at least one left or right outer join caused the Greenplum Database master to crash when the server configuration parameter optimizer_join_order
was set to exhaustive2
.
30880 - gpload
Fixed a problem where gpload
operations would fail if a table column name included capital letters or special characters.
30901 - GPORCA
For queries that included an outer ref in a subquery, such as select * from foo where foo.a = (select foo.b from bar)
, GPORCA always used the results of the subquery after unnesting the outer reference. This could cause a crash or incorrect results if the subquery returned no rows, or if the subquery contained a projection with multiple values below the outer reference. To address this problem, all such queries now fall back to using the Postgres planner instead of GPORCA. Note that this behavior occurs for cases where GPORCA would have returned correct results, as well as for cases that could cause crashes or return incorrect results.
30913, 170824967 - gpfdists
A command that accessed an external table using the gpfdists
protocol failed if the external table did not use an IP address when specifying a host system in the LOCATION
clause of the external table definition. This issue is resolved in Greenplum 6.11.1.
174609237 - gpstart
gpstart
was updated so that it does not attempt to start a standby master segment when that segment is unreachable, preventing an associated stack trace during startup.
Release 6.11.0
Release Date: 2020-09-11
Pivotal Greenplum 6.11.0 is a minor release that includes changed features and resolves several issues.
Features
Greenplum Database 6.11.0 includes these new and changed features:
- GPORCA partition elimination has been enhanced to support a subset of lossy assignment casts that are order-preserving (increasing) functions, including
timestamp::date
andfloat::int
. For example, GPORCA supports partition elimination when a partition column is defined with thetimestamp
datatype and the query contains a predicate such asWHERE ts::date == '2020-05-10'
that performs a cast on the partitioned column (ts
) to compare column data (atimestamp
) to adate
. - PXF version 5.15.0 is included, which includes new and changed features and bug fixes. Refer to the PXF Release Notes for more information on release content and supported platforms, and to access the PXF documentation.
- Greenplum Command Center 6.3.0 and 4.11.0 are included, which include new workload management and other features, as well as bug fixes. See the Command Center Release Notes for more information.
- The query dispatcher now supports the PostgreSQL
LISTEN
,UNLISTEN
, andNOTIFY
commands. The DataDirect ODBC Drivers for Pivotal Greenplum were updated to version 07.16.0389 (B0562, U0408). This version introduces support for the following datatypes:
Greenplum Datatype ODBC Datatype citext SQL_LONGVARCHAR float SQL_REAL tinyint SQL_SMALLINT wchar SQL_CHAR wvarchar SQL_VARCHAR
Resolved Issues
Pivotal Greenplum 6.11.0 resolves these issues:
30899 - Resource Groups
In some cases when running queries are managed by resource groups, Greenplum Database generated a PANIC when managing runaway queries (queries that use an excessive amount of memory) because of locking issues. This issue is resolved.
30877 - VACUUM
In some cases, running VACUUM
returns ERROR: found xmin <xid> from before relfrozenxid <frozen_xid>
. The error was caused when a previously run VACUUM FULL
was interrupted and aborted on a query executor (QE) and corrupted catalog frozen XID information. This issue is resolved.
30870 - Segment Mirroring
In some cases, performing an incremental recovery of a Greenplum Database segment instance failed with the message requested WAL segment has already been removed
because the recovery checkpoint was not created properly. This issue is resolved.
30858 - analyzedb
analyzedb
failed if analyzedb
attempted to update statistics for a set of tables and one of the tables was dropped and then recreated while analyzedb
was running. analyzedb
has been enhanced better handle the specified situation.
30845 - Query Execution
Under heavy load when running multiple queries, some queries randomly failed with the error Error on receive from seg<ID>
. The error was caused when Greenplum Database encountered a divide by 0 error while managing the backend processes that are used to run queries on the segment instances. This issue is resolved.
30761 - Postgres Planner
In some cases, Greenplum Database generated a PANIC when a DROP VIEW
command was cancelled from the Greenplum Command Center. The PANIC was generated when Greenplum Database did not correctly handle the visibility of the relation.
30721 - gpcheckcat
Resolved a problem where gpcheckcat
would fail with Missing or extraneous entries check
errors if the gp_sparse_vector
extension was installed.
30637 - Query Optimizer
For some queries against partitioned tables, GPORCA did not perform partition elimination when a predicate that includes the partition column also performs an explicit cast. For example, GPORCA would not perform partition elimination when a partition column is defined with the timestamp
datatype and the query contains a predicate such as WHERE ts::date == '2020-05-10'
that performs a cast on the partitioned column ( ts
) to compare column data (a timestamp
) to a date
. GPORCA partition elimination has been improved to support the specified type of query. See Features.
10491 - Postgres Planner
For some queries that contain nested subqueries that do not specify a relation and also contain a nested GROUP BY
clauses, Greenplum Database generated a PANIC. The PANIC was generated when Greenplum Database did not correctly manage the subquery correctly. This is an example of the specified type or query.
SELECT * FROM (SELECT * FROM (SELECT c1, SUM(c2) c2 FROM mytbl GROUP BY c1 ) t2 ) t3
GROUP BY c2, ROLLUP((c1))
ORDER BY 1, 2;
This issue is resolved.
10561 - Server
Greenplum Database does not support altering the datatype of a column defined as a distribution key or with a constraint. When attempting to change the datatype, the error message did not clearly indicate the cause. The error message has been altered to provide more information.
174505130 - Resource Groups
In some cases for a query managed by resource group, the resource group cancelled the query with the message Canceling query because of high VMEM usage
because the resource group calculated the incorrect memory used by the query. This issue is resolved.
174353156 - Interconnect
In some cases when Greenplum Database uses proxies for interconnect communication (the server configuration parameter gp_interconnect_type
is set to proxy
), a Greenplum background worker process became an orphaned process after the postmaster process was terminated. This issue is resolved.
174205590 - Interconnect
When Greenplum Database uses proxies for interconnect communication (the server configuration parameter gp_interconnect_type
is set to proxy
), a query might have hung if the query contains multiple concurrent subplans running on the segment instances. The query hung when the Greenplum interconnect did not properly handle the communication among the concurrent subplans. This issue is resolved.
174483149 - Cluster Management - gpinitsystem
gpinitsystem
now exports the MASTER_DATA_DIRECTORY
environment variable before calling gpconfig
, to avoid throwing warning messages when configuring system parameters on Greenplum Database appliances (DCA).
Release 6.10
Relelase 6.10.1
Release Date: 2020-08-13
Pivotal Greenplum 6.10.1 is a maintenance release that resolves known issues.
Resolved Issues
Pivotal Greenplum 6.10.1 resolves these issues:
n/a
Code changes and testing for the Greenplum interconnect proxy feature were introduced in version 6.10.0, but the feature was not enabled in the final release build. Version 6.10.1 resolves this problem and enables the feature.
10009 External table DELIMITER OFF BUG
Fixed a problem where additional data could be included after the last column of an external table if the DELIMITER 'OFF'
formatting option was used when creating the table.
Release 6.10.0
Release Date: 2020-08-07
Pivotal Greenplum 6.10.0 is a minor release that includes changed features and resolves several issues.
Features
Greenplum Database 6.10.0 includes these new and changed features:
- Greenplum Database 6.10 introduces the server configuration parameter
max_slot_wal_keep_size
that sets the maximum size in megabytes of replication WAL log files on disk per segment instance. The default is -1, Greenplum can retain an unlimited amount of WAL files on disk. - Greenplum Database 6.10 introduces the server configuration parameter
gp_add_column_inherits_table_setting
for append-optimized, column-oriented tables. When adding a column to a table with theALTER TABLE
command, the parameter controls whether the table’s data compression parameters for a column (compresstype
,compresslevel
, andblocksize
) can be inherited fromWITH
clause values when the table was created. The default isoff
, the table’s data compression settings are not considered when adding a column to the table. If the value ison
, the table’sWITH
clause values are considered. - When reading data, the
gpload
utility now supports the control file parameterFILL_MISSING_FIELDS
that can addNULL
values to a data row if the row has trailing field values that are missing. To enable this feature, set the parameter totrue
. Greenplum Database supports using proxies for Greenplum Database interconnect communication to reduce the use of connections and ports during query processing. The interconnect proxies consumes fewer connections and ports than
TCP
mode, and has better performance thanUDPIFC
mode in a high-latency network.To enable interconnect proxies for the Greenplum system, set these system configuration parameters.
- List the proxy ports with the new parameter
gp_interconnect_proxy_addresses
. You must specify a proxy port for the master, standby master, and all segment instances. Set the parameter
gp_interconnect_type
toproxy
. Theproxy
value is new in Greenplum Database 6.10.Note: Code changes and testing for the Greenplum interconnect proxy feature were introduced in version 6.10.0, but the feature was not enabled in the final release build. Use version 6.10.1 to enable the feature.
- List the proxy ports with the new parameter
The PgBouncer connection pooler was updated from version 1.8.1 to version 1.13. To support this change on RHEL/CentOS 6 platforms, the Greenplum Database package now requires
libevent2
instead oflibevent
. RHEL/CentOS 7 requirements are unchanged. See the PgBouncer 1.13.x Release Notes for a summary of changes.The
gpcheckcat
catalog verification utility adds a new test,aoseg_table
, that you can use to check that the vertical partition information on append-optimized, column storage tables is consistent withpg_attribute
.- Greenplum Database 6.10 introduces a new server configuration parameter,
gp_fts_replication_attempt_count
, that you can use to configure the maximum number of times that the Greenplum fault tolerance service (FTS) attempts to establish a primary-mirror replication connection. - PXF version 5.14.0 is included, which includes new and changed features and bug fixes. Refer to the PXF Release Notes for more information on release content and supported platforms, and to access the PXF documentation.
Greenplum Streaming Server (GPSS) version 1.4.1 is included, which includes changes and bug fixes. Refer to the GPSS Release Notes for more information on release content and to access the GPSS documentation.
Note: If you have previously used GPSS in your Greenplum 6.x installation, you are required to perform upgrade actions as described in Upgrading the Streaming Server.
Resolved Issues
Pivotal Greenplum 6.10.0 resolves these issues:
30554 - ANALYZE, gprestore
In some cases, when performing concurrent ANALYZE
operations on large partition tables generated the error Canceling query because of high VMEM usage
. This issue occurred during some restore operations with gprestore
. This issue is resolved.
30583 - Transaction Management
In some cases, a segment mirror went offline when the replication WAL log files on the mirror system caused disk full issues. Greenplum Database introduces the server configuration parameter max_slot_wal_keep_size
to limit the amount of WAL logs stored on disk. See See Features.
30792 - Logging
Resolves a disk space issue encountered when a query that generated a large number of spill files also generated an excessive number of HashJoin: Too many batches computed
log messages by decreasing the severity level of the message.
173680224 - gprecoverseg
In some cases, gprecoverseg
hangs when performing an incremental recovery while trying to perform a clean shutdown of the segment instance due to a locking issue. This issue is resolved.
30781, 9427 - Postgres Planner
For some queries that perform joins between partitioned tables, Greenplum Database returned ERROR: unrecognized path type 106
. This issue is resolved.
10419 - Postgres Planner
In some cases when generating query plans, the Postgres Planner did not handle volatile functions correctly and allowed multiple executions of the function during query execution. This could cause incorrect results. This issue is resolved.
30692 - System Catalog Functions
pg_get_viewdef()
returned an incorrect view definition when the view was created with the Greenplum-specific CASE WHEN (arg1) IS NOT DISTINCT FROM (arg2)
clause. This issue is resolved.
30558 - Query Optimizer
Resolves an issue where execution time and spill file size increased for a query on a larger width table because Greenplum Database overestimated row cardinality when the query specified multiple predicates that included distribution keys.
30512 - MPP: Dispatch
Resolves an issue where Greenplum Database hung while continuously retrying a primary-mirror replication connection. Greenplum 6.10 introduces a new server configuration parameter, gp_fts_replication_attempt_count
, with which you can configure the maximum number of retry attempts.
10141 - ALTER TABLE … INHERIT
Greenplum Database restricted you from creating both a replicated table that inherits from another table, and a table that inherits from a replicated table, but Greenplum did allow you to use the ALTER TABLE ... INHERIT
command to assign inheritance to/from a replicated table after it was created. Update commands on such a table returned the misleading error ModifyTable mixes distributed and entry-only tables
. This issue is resolved; Greenplum 6.10 enforces replicated table inheritance restrictions uniformly, including on ALTER TABLE ... INHERIT
statements.
10057 - Server: Transactions
Greenplum Database returned the error Too many distributed transactions for snapshot
when processing many distributed transactions and the max_prepared_transactions
server configuration parameter was set to a low value. This issue is resolved; Greenplum now uses a more robust method to determine the maximum number of distributed transactions.
10030 - Server: Query Execution
Greenplum Database incorrectly used the value of the gp_enable_global_deadlock_detector
server configuration parameter on query executors to determine the lock mode. This issue is resolved, the parameter is now master only.
9896 - gpexpand
gpexpand
failed on a partitioned Greenplum Database table that included an external table leaf child partition. This issue is resolved; Greenplum Database now skips external table partitions during expansion.
9207 - Server: Transactions
In certain situations, the Greenplum Database standby master host shutdown down and exited abnormally with the FATAL error the limit of *N* distributed transactions has been reached
when the global transaction limit was exceeded. This issue is resolved; Greenplum now uses a more robust method to determine the maximum number of distributed transactions.
173219210 - Resource Groups
The Greenplum Database gp_toolkit.gp_resgroup_status_per_host
view incorrectly reported CPU usage as the sum of resource group CPU usage on all segments on the same host. This issue is resolved; Greenplum Database now correctly reports the average CPU usage of resource groups on all segments on the host.
171849582 - Tablespace
Greenplum Database generated a PANIC when it did not check for the existence of a tablespace directory before attempting to delete the directory. This issue is resolved.
Release 6.9
Release 6.9.1
Release Date: 2020-07-24
Pivotal Greenplum 6.9.1 is a maintenance release that resolves several issues.
Note: Greenplum 6.9.1 also includes the Greenplum Database R Client (GreenplumR) version 1.1.0. This version of GreenplumR adds the input.signature
argument to db.gpapply()
and db.gptapply()
, to match function argument names to table column names when the function input argument is not a single data frame. See Greenplum Database R Client.
Resolved Issues
Pivotal Greenplum 6.9.1 resolves these issues:
n/a
The DataDirect ODBC Drivers were updated to version 7.16.359 to incorporate hot fix changes and certify compatibility with SUSE Linux Enterprise Server 15 clients. See the DataDirect Release Notes for additional information.
10230, 31582 - Server: Execution
Greenplum Database generated a segmentation fault during execution of a hash aggregate query when it incorrectly initialized a tuple memory context for a partitioned table that contained a btree index. This issue is resolved.
30750
The ANALYZE
code did not exclude external tables (which cannot be analyzed) if an external table was part of a table partition hierarchy. This would result in the failure ERROR: unsupported table type
when performing partitioned table operations that use ANALYZE
, such as ALTER TABLE … EXCHANGE PARTITION
. The problem was resolved by updating the ANALYZE
code to correctly exclude these external tables.
Release 6.9.0
Release Date: 2020-06-26
Pivotal Greenplum 6.9.0 is a minor release that includes changed features and resolves several issues.
Features
Greenplum Database 6.9.0 includes these new and changed features:
Greenplum Streaming Server (GPSS) version 1.4.0 is included, which introduces many new and changed features and bug fixes. Refer to the GPSS Release Notes for more information on release content and to access the GPSS documentation.
Note: If you have previously used GPSS in your Greenplum 6.x installation, you are required to perform upgrade actions as described in Upgrading the Streaming Server.
PXF version 5.13.0 is included, the first PXF release to also provide a separate download package that enables you to install PXF in a file system location outside of the Greenplum install directory. Refer to the PXF Release Notes for more information on release content and supported platforms, and to access the PXF documentation.
Resolved Issues
Pivotal Greenplum 6.9.0 resolves these issues:
30630 - Segment Mirroring
In some cases during a failover of a segment instance, Greenplum Database returned the FATAL error requested WAL segment WAL_seg_ID has already been removed
. Greenplum Database WAL replication incorrectly removed segment files before they were processed during failover. This issue is resolved.
10216 - ALTER TABLE, ALTER DOMAIN
In some cases, heap table data is lost when performing concurrent ALTER TABLE
or ALTER DOMAIN
commands where one command alters a table column and the other rewrites or redistributes the table data. For example, performing concurrent ALTER TABLE
commands where one command changes a column data type from int
to text
might cause data loss. This issue might also occur when altering a table column during the data distribution phase of a Greenplum system expansion. Greenplum Database did not correctly capture the current state of the table during command execution. This issue is resolved.
10224 - ALTER TABLE
For a leaf partition of a partitioned table, ALTER TABLE
allowed the distribution policy to be changed to REPLICATED
. This is resolved. ALTER TABLE
no longer allows the change.
30647 - Postgres Planner
Some queries that performed multistage aggregation returned results in the incorrect order. For example, some queries that perform a COUNT
in the select list and also contain a GROUP BY
clause returned results in the incorrect order. This issue is resolved.
10013 - Postgres Planner
In some cases, Greenplum Database generated a PANIC when it encountered a lateral subquery that included a LIMIT 1
or GROUP BY
clause. Greenplum 6.9 resolves this issue by forcing the gathering and materialization of any relation containing a GROUP BY
or LIMIT
clause.
10315 Postgres Planner
For some queries that perform a FULL JOIN
using a subselect that contains a COALESCE
function, Greenplum Database returned "ERROR: could not find hash distribution key expressions in target list"
. This issue is resolved.
8919 - MPP, Query Execution
Greenplum Database did not properly handle concurrent updating operations to a table when one of the operations moved a table distribution key to another segment instance. Now when a table distribution key is moved to another segment instance, a concurrent updating operation returns an error.
173243811 - Resource Groups
When resource groups are enabled and a user attempted to move a running query from one resource group to a resource group configured using the memory_limit=0
with the pg_resgroup_move_query()
function, Greenplum Database returned the error ERROR: group <group_ID> doesn't have enough memory on master
. This issue is resolved.
172931886 - Transaction System
Greenplum 6.9 resolves an issue where restarting a primary could lead to a segment process hang when there were prepared, but not yet committed or aborted, transactions in progress at the time of shutdown.
Release 6.8
Release 6.8.1
Release Date: 2020-06-11
Pivotal Greenplum 6.8.1 is a maintenance release that contains a changed feature and resolves several issues.
Changed Feature
The Greenplum PostGIS extension package has been updated to postgis-2.5.4+pivotal.2
. The release contains these changes:
- Adds support for the PostGIS TIGER geocoder extension and the PostGIS address standardizer and address rules files extensions.
- Removes PostGIS Raster function limitations.
Uses the
CREATE EXTENSION
andDROP EXTENSION
commands to activate and deactivate support for the PostGIS extension and supported, optional PostGIS extensions.Note: The
postgis_manager.sh
script is deprecated and will be removed in a future release of Greenplum PostGIS. To enable or deactivate PostGIS support, use theCREATE EXTENSION
orDROP EXTENSION
command. See Enabling and Removing PostGIS Support.
Resolved Issues
Pivotal Greenplum 6.8.1 resolves these issues:
30664 - Query Optimizer
For a complex CTAS query that has implicit casts in the project list, GPORCA may generate a plan with duplicate eliminating motions, to ensure correctness. However, if a duplicate eliminating motion is performed under the hash operation of a Hash Join, an implicit cast operation creates an additional column that causes memtuple binding issues in the executor. To address this problem, GPORCA now generates a modified plan that prunes the output of any duplicate eliminating motions before sending the output to the hash operation.
30684 - Query Optimizer
GPORCA returned incorrect results for some queries when the query’s select list contains a window function and the window function contains a correlated subquery or an outer reference. Now the query falls back to the Postgres planner.
30615 - Query Optimizer
GPORCA query performance degraded when compared with Greenplum 5 for some queries that perform joins using an equality predicate and the equality predicate contains a function, for example coalesce(tbl1.a, '999999') = coalesce(tbl2.a, '999999')
. The performance issue was caused by inaccurate cardinality estimates. GPORCA cardinality estimation has been improved for the specified type of query.
172732495, 9953 - query execution
Greenplum Database generated a PANIC when executing a query that executes multiple user-defined functions and more than one of the functions is defined with the EXECUTE ON INITPLAN
attribute. This issue is resolved.
172098556 - psql
Resolved a problem where the psql
client \dm
command did not display materialized views.
172094194, 9837 - gprecoverseg
In some cases when recovering segment instances using the gprecoverseg
utility with the -i <recover_config_file>
option to specify details about failed segments to recover, the utility changed some segment instance dbid
values in the Greenplum system configuration. This issue is resolved.
Release 6.8.0
Release Date: 2020-06-05
Pivotal Greenplum 6.8.0 is a minor release that includes changed features and resolves several issues.
Features
Greenplum Database 6.8.0 includes these new and changed features:
Greenplum Streaming Server (GPSS) version 1.3.6 is included, which introduces many new and changed features and bug fixes since the last GPSS version installed in Greenplum 6.x (1.3.1). Refer to the GPSS Release Notes for more information on release content and to access the GPSS documentation.
Note: If you have previously used GPSS in your Greenplum 6.x installation, you are required to perform upgrade actions as described in Upgrading the Streaming Server.
The
gpinitsystem
input configuration file specified with the-I
option supports an additional format to specify hosts. TheQD_PRIMARY_ARRAY
,PRIMARY_ARRAY
, andMIRROR_ARRAY
host parameters may now be specified using either of the following formats:host~port~data_directory/seg_prefix<segment_id>~dbid~content_id
hostname~address~port~data_directory/seg_prefix<segment_id>~dbid~content_id
The first format, which is the pre-existing format, sets both the
hostname
andaddress
columns of thegp_segment_configuration
catalog table to the value in thehost
field. The second format sets thehostname
andaddress
columns of thegp_segment_configuration
catalog table to the values in the respectivehostname
andaddress
fields.PXF version 5.12.0 is included, which introduces new and changed features and bug fixes. See PXF Version 5.12.0 below.
PL/Container version 2.1.2 is included, which introduces the following new features:
- Support for R version 3.6.3.
- A new
--use_local_copy
option to theplcontainer add-image
command that you can use to install the specified image only on the local host.
- Greenplum Database 6.8 adds support for Moving a Query to a Different Resource Group.
- Greenplum Database 6.8 includes a new metrics collector extension that is compatible with Greenplum Command Center 6.2 and above. If you are using Command Center 6.0 or 6.1 you must download and install Command Center 6.2 after you install Greenplum Database 6.8.
PXF Version 5.12.0
PXF includes the following new and changed features:
- PXF trims right-padded white space added by Greenplum before it writes Parquet data.
- PXF bundles newer
hive
,jackson-databind
, and supporting internal libraries. - A PXF server running on Java 11 can now read from Hive using an external table that specifies a
Hive*
profile. - PXF introduces the new custom option
IGNORE_MISSING_PATH
for external tables that you use to read file-based data. Setting this option may be useful when a PXF external table is a child partition of a partitioned Greenplum table. Refer to About PXF External Table Child Partitions for more information. - PXF bundles the
jodd-core
library to satisfy a missing transitive dependency that is required when PXF reads Parquet files that contain data in timestamp format. PXF adds column projection support for the
Hive
andHiveRC
profiles by changing the implementation to use column name-based, rather than column index-based, mapping.Note
If you have existing PXF external tables that specify a
Hive*
profile, you may be required to perform upgrade actions as described in Upgrading PXF.
Resolved Issues
Pivotal Greenplum 6.8.0 resolves these issues:
329, 30602 - PXF
PXF did not correctly read a partitioned Hive table when the external table specified a Hive*
profile and the external table and Hive table had a differing number of columns. This issue is resolved. PXF now supports column projection for the Hive*
profiles and correctly handles this situation.
30611 - Query Optimizer
When falling back to the Postgres planner, GPORCA incorrectly logged messages that were internal messages. This made the log file difficult to read and caused bloat in the file. This issue is resolved. GPORCA message logging has been improved and the internal messages are no longer sent to the log files.
30585 - Locking
Resolved a problem that could corrupt resource queue locks, and potentially other types of locks, in shared memory. This problem could cause errors such as lock lock_name on object object_identifier is already held
.
30557 - DDL
When performing a data reorganization with the ALTER TABLE
command on a leaf partition of a partitioned table that did not change the distribution policy, Greenplum Database returned the error ERROR: can't set the distribution policy
. This type of redistribution is allowed in Greenplum 5. Now Greenplum allows data reorganization on a leaf partition if the distribution policy is not changed.
30289 - Query Optimizer
When GPORCA performed dynamic partition elimination for some queries against partitioned tables that perform joins, GPORCA was not using the correct statistics. This caused a performance degradation when compared with Greenplum 5. GPORCA has improved how statistics are computed tor the specified type of query.
172854840 - Interconnect
In some cases, a query that executes a stable function that contains an SQL statement might hang because the query dispatcher (QD) did not correctly manage the execution of the function and the dispatching of the query plan. This issue is resolved.
172832212 - Interconnect
In some cases, communication between a query dispatcher (QD) and a query executor (QE) on different segments was slow when the Greenplum interconnect type is set to the TCP networking protocol for Greenplum Database interconnect traffic. Now the communication between a QD and a QE is more efficient.
172615233 - Query Optimizer
For text data types, the GPORCA the cardinality estimation algorithm has been improved for equality comparisons. For example, when a query contains an IN clause that contains text elements.
172576000 - COPY
If data format errors occurred while copying data into a partitioned table with a COPY FROM
command in single row error isolation mode, Greenplum Database might crash when a query executor (QE) did not handle the data format error correctly. This issue is resolved.
30487 - Utility Commands
On a Greenplum Database 6 system with FIPS enabled, Greenplum utility commands such as gpinitsystem
returned the error "ERROR:root:code for hash md5 was not found."
This issue is resolved.
30484 - Utility Commands
When initializing a Greenplum Database system with gpinitsystem
, the primary segments were erroneously named using DNS resolvable external hostnames instead of the internal interconnect interface hostnames. At the same time, the segment mirrors were correctly named. This issue is now resolved.
Release 6.7
Release 6.7.1
Release Date: 2020-04-30
Pivotal Greenplum 6.7.1 is a maintenance release that resolves several issues. In addition to these resolved issues:
- Version 6.7.1 updates PostGIS to version 2.5.4, which removes several previous limitations. See Geospatial Analytics for more information.
- The Greenplum R client is no longer considered a Beta feature.
Resolved Issues
Pivotal Greenplum 6.7.1 resolves these issues:
n/a - MADlib
In Greenplum 6.7.0 the MADlib download files that were originally provided, madlib-1.17.0+2-gp6-rhel7-x86_64.tar.gz
and madlib-1.17.0+2-gp6-rhel6-x86_64.tar.gz
, contained MADlib version 1.16 instead of version 1.17. This is resolved in Greenplum 6.7.1, and in Greenplum 6.7.0 with the newly-provided files madlib-1.17.0+3-gp6-rhel7-x86_64.tar.gz
and madlib-1.17.0+3-gp6-rhel6-x86_64.tar.gz
.
9790 - Server
A crash could occur when performing a SELECT
query against a column-oriented table, when the table was created using the WITH NO DATA
clause. The problem occurred because the WITH
clause options were not correctly added to the pg_attribute_encoding
table. This problem has been resolved.
30499 - Server: Execution
Fixed a memory leak that occurred when executing CHECKPOINT
commands.
30559 - Query Optimizer
Queries that contain an IN
clause with a large number of constants took a long time to generate a query plan. Most of the time was spent estimating the cardinality of the IN
clause predicate. The cardinality estimation algorithm has been enhanced and significantly reduces the cardinality estimation time for the specified type of query.
30579 - Interconnect
In some cases during query execution, the query hung with the query dispatcher (QD) waiting for the query executor (QE) on a few segment instances to complete. This issue is resolved.
30844 - gpreload
gpreload
returned the error more than one row returned
when attempting to reload a table and a view with the same name exists in a different schema in the database. This issue is resolved.
172163076 - Server
A subtransaction would incorrectly use 1-phase commit instead of 2-phase commit if \set ON_ERROR_ROLLBACK interactive
was enabled in a client’s .psqlrc
file. This problem has been resolved.
172324858, 9891 - MPP: Locking, Signals, Processes
In some cases, Greenplum Database did not manage snapshots correctly when processing concurrent distributed transactions. This caused a concurrent transaction to access a distributed log file that was no longer available and generated the error message Could not open file ""pg_distributedlog/<file-name>"": No such file or directory
. This issue is resolved.
172348849 - Postgres Planner
Some queries that contain a UNION ALL
that combines the results from SELECT
command that uses a replicated table with another SELECT
command returns the error ERROR: could not build Motion path
. This issue is resolved.
172284550 9823 - ALTER DATABASE
The ALTER DATABASE...FROM CURRENT
command did not set a server configuration parameter for a database. This issue is resolved.
Release 6.7.0
Release Date: 2020-04-17
Pivotal Greenplum 6.7.0 is a minor release that includes changed features and resolves several issues.
Features
Greenplum Database 6.7.0 includes these new and changed features:
- Greenplum Database 6.7 introduces the new gp_resource_group_queuing_timeout server configuration parameter. When the resource group-based resource management scheme is active,
gp_resource_group_queuing_timeout
specifies the maximum amount of time a transaction waits for execution in a queue on a resource group before Greenplum Database cancels the transaction. By default, queued transactions in a resource group can wait indefinitely. Greenplum Database 6.7 includes MADlib version 1.17, which introduces new Deep Learning features, k-Means clustering, and other improvements and bug fixes. See the Apache MADlib page for additional information and Release Notes.
Note: In Greenplum 6.7.0 the MADlib download files that were originally provided,
madlib-1.17.0+2-gp6-rhel7-x86_64.tar.gz
andmadlib-1.17.0+2-gp6-rhel6-x86_64.tar.gz
, contained MADlib version 1.16 instead of version 1.17. This is resolved in Greenplum 6.7.1, and in Greenplum 6.7.0 with the newly-provided filesmadlib-1.17.0+3-gp6-rhel7-x86_64.tar.gz
andmadlib-1.17.0+3-gp6-rhel6-x86_64.tar.gz
.
Resolved Issues
Pivotal Greenplum 6.7.0 resolves these issues:
8539 - Server
Using NOWAIT
in a SELECT FOR UPDATE
statement could result in the error, ERROR: relation "<name>" does not exist
, because locking was not correctly handled for the NOWAIT
clause. This problem has been resolved. Note, however, that NOWAIT
only affects how the SELECT
statement obtains row-level locks. A SELECT FOR UPDATE NOWAIT
statement will always wait for the required table-level lock; it behaves as if NOWAIT
was omitted.
9089 - Server
Fixed a problem where Greenplum Database failed to truncate an append-only, column-oriented table if the CREATE TABLE
and TRUNCATE
statements were executed in the same transaction.
30305 - Resource Groups
A transaction may be queued for execution on a resource group for an extended period of time, particularly when the resource group reached its concurrent transaction limit. This could prevent queries initiated by Greenplum Database superusers from executing. Greenplum Database 6.7 resolves this issue by introducing the gp_resource_group_queuing_timeout server configuration parameter, which specifies the maximum amount of time a queued transaction waits for execution in a resource group before Greenplum cancels the transaction.
30531 - Query Optimizer
An out of memory error occurred when running some queries that contain joins that perform a comparison operation on citext
data. The error occurred because the query falls back to the Postgres Planner. This issue is resolved. Now the query does not fall back to the Postgres planner, the query is executed using GPORCA.
30536 - PL/pgSQL
In a PL/pgSQL procedure, output text from a RAISE NOTICE
statement was not displayed correctly if the text contained a newline (line feed) character. Only the text before the newline character was displayed. This issue is resolved.
Release 6.6
Release 6.6.0
Release Date: 2020-04-06
Pivotal Greenplum 6.6.0 is a minor release that includes changed features and resolves several issues.
Features
Greenplum Database 6.6.0 includes these new and changed features:
For the
CREATE EXTERNAL TABLE
command, theLOG ERRORS
clause now supports thePERSISTENTLY
keyword. TheLOG ERRORS
clause logs information about external table data rows with formatting errors. The error log data is stored internally. When you specifyLOG ERRORS PERSISTENTLY
, the log data persists after the external table is dropped.If you use the
PERSISTENTLY
keyword, you must install the functions that manage the persistent error log information.For information about the error log information and built-in functions for viewing and managing error log information, see CREATE EXTERNAL TABLE.
PXF version 5.11.2 is included, which introduces these changes:
- PXF no longer validates the JDBC
BATCH_SIZE
write option during a read operation. - PXF bundles a newer
jackson-databind
library. - PXF removes references to the unused
pxf-public.classpath
file. This in turn removes spuriousWARNING: Failed to read classpath file ...
log messages. - PXF now bundles Tomcat version 7.0.100.
- PXF no longer validates the JDBC
- Greenplum Database 6.6 includes MADlib version 1.17, which introduces new Deep Learning features, k-Means clustering, and other improvements and bug fixes. See the MADlib 1.17 Release Notes for a complete list of changes.
Resolved Issues
Pivotal Greenplum 6.6.0 resolves these issues:
30483 - Query Optimizer
A query that specified multiple constants in an IN
clause generated a large number of spill files and returned the error workfile per query size limit exceeded
when GPORCA incorrectly normalized a histogram that was not well-defined. This issue is resolved.
30488 - DLL
For some append-optimized partitioned tables, performance was poor when adding a column to the table with the ALTER TABLE... ADD COLUMN
command because the command performed a full table rewrite. Now only data corresponding to the new column is rewritten.
30518 - Query Optimizer
A query that specified an aggregate function such as min()
or count()
that was invoked on a citext
-type column failed with the error cache lookup failed for function 0
because GPORCA incorrectly generated a multi-stage aggregate for the query. This issue is resolved.
30525 - Logging
In some cases, Greenplum Database encountered a segmentation fault and rotated the log file early when the logging level was set to WARNING
or less severe and Greenplum attempted to write to the alert log file after it failed to open the file. This issue is resolved.
171506474 - COPY
When COPY FROM SEGMENT
command copied data into an append-only table, the command did not update the append-only table metadata tupcount
(the number of tuples on a segment, including invisible tuples) and modcount
(the number of data modification operations performed). This issue is resolved.
n/a - gpperfmon
The Ubuntu build of Greenplum Database 6.5.0 did not include the gpperfmon
database, which is required for using Greenplum Command Center. This issue is resolved in version 6.6.0.
Release 6.5
Release 6.5.0
Release Date: 2020-03-20
Pivotal Greenplum 6.5.0 is a minor release that includes changed features and resolves several issues.
Warning: The Ubuntu build of Greenplum Database 6.5.0 does not include the gpperfmon
database, which is required for using Greenplum Command Center. Customers deploying to Ubuntu should not install or upgrade to Greenplum Database 6.5 until a maintenance release is provided to resolve this issue.
Features
Greenplum Database 6.5.0 includes these new and changed features:
When creating a user-defined function, you can specify the attribute
EXECUTE ON INITPLAN
to indicate that the function contains an SQL command that dispatches queries to the segment instances and requires special processing on the master instance by Greenplum Database. When possible, Greenplum Database handles the function on the master instance in the following manner:- First, Greenplum Database executes the function as part of an InitPlan node on the master instance and holds the function output temporarily.
- Then, in the MainPlan of the query plan, the function is called in an EntryDB (a special query executor (QE) that runs on the master instance) and Greenplum Database returns the data that was captured when the function was executed as part of the InitPlan node. The function is not executed in the MainPlan. For more information about the attribute and limitations when using the attribute, see CREATE FUNCTION.
GPORCA introduces a new costing model for bitmap indexes. The new model is designed to choose faster, bitmap nested loop joins instead of hash joins. The new costing model is implemented as a Beta feature, and it is used as a default only if you enable it by setting the configuration parameter:
set optimizer_cost_model = experimental
The
optimizer_cost_model
parameter is required only during the Beta test period for this cost model. After further testing and validation, the new cost model will be enabled by default.Greenplum Database includes the server configuration parameter plan_cache_mode that controls whether a prepared statement (either explicitly prepared or implicitly generated, for example by PL/pgSQL) can be executed using a custom plan or a generic plan plan.
Custom plans are created for each execution using its specific set of parameter values, while generic plans do not rely on the parameter values and can be re-used across executions. By default, choice between these options is made automatically, but it can be overridden by setting this parameter. If the prepared statement has no parameters, a generic plan is always used. The allowed values are
auto
(the default),force_custom_plan
andforce_generic_plan
. This setting is considered when a cached plan is to be executed, not when it is prepared.PXF version 5.11.1 is included, which introduces new and changed features and bug fixes. See PXF Version 5.11.1 below.
The s3 external table protocol automatically recognizes and uncompresses as deflate format any file that it reads that has a
.deflate
extension.Greenplum Database introduces the Greenplum R Client Beta, an interactive in-database data analytics tool. Refer to the Greenplum Database R Client (Beta) documentation for installation and usage information for this tool.
The Greenplum R Client (GreenplumR) is currently a Beta feature, and is not supported for production environments.
gpload
adds the--max_retries
option to specify the number of times the utility attempts to connect to Greenplum Database after a connection timeout. The default value, 0, does not attempt a connection after a timeout.Greenplum Database introduces PL/Container version 3.0 Beta, which:
- Provides support for the new GreenplumR interface.
- Reduces the number of processes created by PL/Container, in order to save system resources.
- Supports more containers running concurrently.
- Includes improved log messages to help diagnose problems. PL/Container 3 is currently a Beta feature, and is not supported for production environments. It provides only an R Docker image for executing functions; Python images are not yet available. See PL/Container Language for installation changes related to PL/Container 3.
PXF Version 5.11.1
PXF includes the following new and changed features:
- PXF provides a
restart
command to stop, and then restart, all PXF server instances in the cluster. See Restarting PXF. - The
pxf [cluster] sync
command now recognizes a[-d | --delete]
option. When specified, PXF deletes files on the remote host(s) that are not present in the PXF user configuration on the Greenplum Database master host. Refer to pxf and pxf cluster. - PXF supports filter predicate pushdown for Parquet data that you access with the Hadoop and Object Store Connectors. Parquet Data Type Mapping describes filter pushdown support for Parquet data types in PXF.
- PXF includes improvements to error handling and error surfacing.
- PXF bundles newer
guava
and Google Cloud Storagehadoop2
libraries. - The PXF
pxf-log4j.properties
template file updates a log filter and changes the level fromINFO
toWARN
. - PXF removes unused and default Tomcat applications and files, hardening its default Tomcat security.
- PXF no longer requires a
$JAVA_HOME
setting ingpadmin
’s.bashrc
file on the master, standby master, and segment hosts. You can now specifyJAVA_HOME
before or during PXF initialization. Refer to the Initialization Overview in the PXF initialization documentation.
Resolved Issues
Pivotal Greenplum 6.5.0 resolves these issues:
307 - PXF
PXF did not correctly handle an external table that was created with the ESCAPE 'OFF'
or DELIMITER 'OFF'
formatting options. This issue is resolved. PXF now correctly neither escapes nor adds delimiters when reading external data with an external table created with these options.
30155 - gpstart
On systems that use a custom tablespace or filespace, gpstart
could fail to start a cluster if the standby master host was down (for example, if the standby was taken offline for maintenance), showing the error:
Error occured while stopping the standby master: ExecutionError: 'non-zero rc: 255' occured.
This problem occurred because gpstart
was attempting to check and sync the filespace or tablespace on the unavailable standby master host. gpstart
was modified to skip filespace and tablespace checks when the standby server is not available.
30255 - Query Optimizer
The GPORCA cost model for bitmap indexes could would sometimes cost bitmap nested loop joins higher than hash joins, resulting in poor query performance. Greenplum Database 6.5 introduces a revised cost model for bitmap indexes to address this issue. See See Features.
30287 - Server: Execution
When GPORCA was enabled, queries against an append-only, column-oriented table could cause a PANIC due to shared memory corruption. The code was modified to guard against out-of-bound writes that caused the memory corruption.
30367 - Query Optimizer
For GPORCA, query performance was poor for some queries against tables with columns that are defined with the citext
datatype. The poor performance was because GPORCA did not gather statistics and calculate cardinalities for those columns. Now GPORCA gathers statistics and calculates cardinalities for columns defined with the citext
datatype.
30369 - Query Execution
Greenplum Database generated a PANIC when executing a query that contains a JOIN LATERAL
and the LATERAL
subquery contains a LIMIT
clause. Now the specified type of query completes.
30379 - ANALYZE
In some cases, performing an ANALYZE
operation on a table with a column that is defined with the citext
datatype returns the error permission denied for schema <name>
. The error was generated when the user performing the operation did not have USAGE
privilege in the schema where the citext
datatype was defined with the CREATE EXTENSION citext
command. Greenplum Database has been modified to not require USAGE
privilege in the citext
datatype schema for ANALYZE
operations.
30382 - VACUUM,TRUNCATE
In some cases, performing a VACUUM FULL
operation on the pg_class
catalog table and concurrently performing a TRUNCATE
operation on a user created heap table returned the error updated tuple is already HEAP_MOVED_OFF
and caused the database to become unavailable. The TRUNCATE
command did not properly manage the heap table entry in pg_class
during the TRUNCATE
operation. This issue is resolved.
30390 - gprecoverseg
In some cases, performance was poor when performing an incremental recovery with the gprecoverseg
utility on a system with a large number of segment instances. Performance is improved, now the utility performs some recovery operations in parallel.
30405 - gpcheckcat
The gpcheckcat
utility failed when the dbid
of Greenplum Database master was not 1
. Now the master dbid
is not required to be 1
.
30426 - Query Execution
Some queries that use the window function cume_dist()
return the error Backward scanning of tuplestores are not supported
if the query generates spill files. This issue is resolved and backward scanning of tuplestore spill files is allowed during query execution.
30437 - Query Optimizer
Queries using dynamic partition elimination (DPE) with range predicates were running slow. This issue has been fixed by allowing only equality comparisons with DPE.
30438 - Catalog and Metadata
If the server configuration parameter gp_use_legacy_hashops
was set to on
, Greenplum Database incorrectly used the non-legacy opclasses when redistributing a table with an ALTER TABLE...(REORGANIZE = TRUE)
command if the command contained DISTRIBUTED BY
clause the did not specify an opclass. This caused SELECT
commands against a table with redistributed data to return incorrect results.
30441 - analyzedb
The analyzedb
utility could fail with an error similar to ERROR: relation "pg_aoseg.pg_aocsseg_xxxxxx" does not exist
if a table was dropped during the analyzedb
operation. This problem was resolved by ensuring that analyzedb
skips any dropped tables when determining the list of tables to analyze.
30450 - PXF
PXF initialization and reset failed when the default system Java version differed from that specified in PXF’s $JAVA_HOME
. This issue is resolved; PXF has added flexibility to the specification of the $JAVA_HOME
setting.
30452 - Dispatch
If the server configuration parameter check_function_bodies
was set in a session on the master, the parameter setting did not persist when a related segment instance session was reset. This caused some functions to fail. Now the parameter setting persists when a segment instance session is reset.
30464 - Query Optimizer
GPORCA incorrectly determined that the plan for a query with a filter on a window function over the distribution key column was direct dispatchable. Now direct dispatch requires the filter to be on a table scan.
30471, 8987 - Postgres Planner
When the Postgres Planner executed some queries that contain a subquery that contain both a distinct qualified aggregate expression and a GROUP BY
clause, the Postgres Planner returned the error could not find pathkey item to sort
. The error was returned when the Postgres Planner supply did not properly manage information used for sorting.
30474 - Query Execution
In some specific situations, some specific types of queries generated a Greenplum Database PANIC. The PANIC occurred when Greenplum Database did not properly handle skew optimization for multi-batch hash joins. Criteria for a query that caused the PANIC include the query contains a join, the join key has segment-local statistics (such as a catalog table), and the join key is one of the most common values, and the query plan is multi-batch hash join and the hash join is rescannable.
30477 - Query Analyze
While gathering statistics for a partitioned table, the pg_class
columns relpages and reltuples were not populated for the root partition, only for leaf partitions. This issue has been fixed by changing the method to calculate if a partition table is empty or not.
30485 - gpinitsystem
When initializing a Greenplum Database system, the gpinitsystem
utility failed to set the password for a user name when the name is numeric.
30493 - analyzedb
When used with the --config-file
option, analyzedb
did not enumerate the leaf partitions of a partitioned table and processed the root partition as a non-partitioned table. For heap tables this produced an error. For append-optimized tables, no error was raised, but DML changes to leaf partitions were not tracked properly. This issue has been resolved. Using the --config-file
option correctly analyzes partitioned tables.
168199193 - COPY
In some cases, performance of the COPY
command in Greenplum Database 6 was poor when compared to Greenplum Database 5. The performance of the COPY
command is improved.
168828451, 8677 - Planner
Some queries returned incorrect results when the queries contain subqueries that perform a join and also contain one or more equality predicates, and optionally contain an IS NULL
predicate. Incorrect results were returned when either a merge join or a nested loop join did not correctly process the predicates. This issue is resolved.
169030090 - Server
Superusers were limited to 3 connections by default, causing “too many clients” errors when users run maintenance scripts. The maximum number of superuser connections is set with the superuser_default_connections
server configuration parameter. This issue is resolved. The default value for this parameter has changed from 3 to 10.
170745356, 9407 - Query Execution
With gp_enable_global_deadlock_detector
set to on
, concurrent updates to the same table could produce an incorrect query result. This issue is resolved. Segments report waited-for transaction IDs to the master so that the master has the same transaction order as the segments.
170861600 - Server
Using ALTER TABLE tablename SPLIT PARTITION
could cause rows to be assigned to the wrong partition, or could cause a crash, if one or more columns before the partition key were dropped. This issue is resolved.
171481916 - gpinitstandby
In some cases, utilities that checked for host IP address such as gpinitstandby
failed. A Python utility ( ifaddrs
) that is used those Greenplum Database utilities caused the failure. ifaddrs
has been updated.
171596248 9679 Query Execution
A Greenplum Database segment instance might generate a PANIC when a query that joins tables with a compound data type generates a query plan that performs a data motion and contains Nested Loop joins. The PANIC occurs due to an error in the prefetch logic for the motion. The prefetch logic issue has been corrected.
Release 6.4
Release 6.4.0
Release Date: 2020-02-11
Pivotal Greenplum 6.4.0 is a minor release that includes changed features and resolves several issues.
Features
Greenplum Database 6.4.0 includes these changed features:
DISCARD ALL
is not supported. The command returns a message that states that the command is not supported and to consider alternatives such asDEALLOCATE ALL
orDISCARD TEMP
. See DISCARD.Greenplum Database resource groups support automatic query termination when resource group global shared memory is enabled. For resource groups that use global shared memory, Greenplum Database gracefully cancels running queries that are managed by those resource groups when the queries consume a large portion of the global shared memory. At this point, the queries would have already consumed available resource group slot memory and group shared memory. Greenplum Database cancels queries in order of memory used, from highest to lowest until the percentage of utilized global shared memory is below the percentage specified by the server configuration parameter runaway_detector_activation_percent.
Global shared memory is enabled for resource groups that are configured to use the
vmtracker
memory auditor, such asadmin_group
anddefault_group
when the sum of theMEMORY_LIMIT
attribute values configured for all resource groups is less than 100. If resource groups are not configured to use global shared memory pool, the resource group automatic query termination feature is not enabled.For information about resource groups, see Using Resource Groups.
Greenplum Database supports creating the standby master or mirror segment instances on hosts that are in a different subnet from the master and primary segment instance hosts. Mirror segment instances can also be moved to hosts that are in a different subnet.
Resolved Issues
Pivotal Greenplum 6.4.0 resolves these issues:
30209, 170762049 - gpaddmirrors
The gpaddmirrors
utility failed to add mirror segments to a Greenplum Database system, when the mirror segments are in a different subnet from the primary segments. This issue is resolved. Now Greenplum Database supports mirror segments in a different subnet from the primary segments.
30266 - Vacuum
The vacuum workflow changed in Greenplum Database 6 to dispatch once per auxiliary table in addition to the dispatch for the main table. This could cause performance problems and permission verification failures on auxiliary tables. This is fixed. The VACUUM
command again dispatches once per segment.
30282 - Resource Management
For some queries managed by Greenplum Database resource groups, the query failed due to an out of memory condition. This caused a segment failure and other issues. This issue is resolved. Resource groups have been enhanced to handle queries that consume a large amount of memory. See Features.
30299 - Server: Execution
For some queries against a partitioned table with multiple column indexes that perform a dynamic index scan, Greenplum Database generated a SIGSEGV. The error occurred when Greenplum Database did not correctly manage dynamic index scan pointer memory. This issue is resolved.
30325 - Query Optimizer
For some queries that reference a view that is defined with a CTE (common table expression) query, and the main query also contains a predicate that is a subquery that references the view, GPORCA returns an error that states could not open temporary file
. The error was caused by incorrect predicate pushdown during preprocessing. This issue is resolved.
30359 - Server: Segment Mirroring
A panic occurred during crash recovery for a mirror when replaying WAL records for a transaction that created an append-optimized table, truncated the table, and then aborted the transaction. This issue is resolved.
30354 - DISCARD
DISCARD TEMP
might not drop temporary tables on segment instances. This issue is resolved.
30360 - Server: Security
Greenplum Database incorrectly logged the message time constraints added on superuser role
every time a superuser role was checked. Now the message is logged only when time constraints are added to a superuser role.
30366 - Server: Execution
Using GRANT commands on partitioned tables across segments would cause the system to PANIC due to alterations in the cached plan. This issue has been resolved.
30371 - Server: Execution
The to_timestamp()
function did not return an error for an out of range value. For example, select to_timestamp('20200340123456','YYYYMMDDHH24MISS');
returned a valid date and time 2020-04-09 12:34:56-07
. This issue is resolved. Now the function returns an error.
30387 - Server: DML
After completing the execution of a user-defined function that changed the value of a server configuration parameter, Greenplum Database restored the original parameter value on the query dispatcher, but did not synchronize and restore the value to the query executors. This issue is resolved.
170787232 - Server: Query Dispatcher
A Query Dispatcher (QD) would run into local deadlock for some prepared statements that execute an UPDATE or DELETE command. This issue has been fixed by taking into consideration the server configuration parameter gp_enable_global_deadlock_detector
.
Release 6.3
Release 6.3.0
Release Date: 2020-1-12
Pivotal Greenplum 6.3.0 is a minor release that includes new features and resolves several issues.
Features
Greenplum Database 6.3.0 includes these new features:
- The sever configuration parameter
wait_for_replication_threshold
is introduced to improve performance for Greenplum Database systems with segment mirroring enabled. The parameter specifies the maximum amount of Write-Ahead Logging (WAL)-based records (in KB) written by a transaction on the primary segment instance before the records are written to the mirror segment instance for replication. As the default, Greenplum Database writes the records to the mirror segment instance when a checkpoint occurs or thewait_for_replication_threshold
value is reached. See wait_for_replication_threshold. The PL/Container version has been updated to 2.1.0. This version supports Docker images with Python 3 installed. These new PL/Container features enable support for Python 3:
A Docker image that is installed with Python 3 -
plcontainer-python3-images-2.1.0.tar.gz
The Docker image can be downloaded from Pivotal Network.
The new value
python3
for the--language
option ofplcontainer runtime-add
command.You specify this value when you add a Docker image that has Python 3 installed on to the Greenplum Database hosts with the
plcontainer runtime-add
command.The GluonTS module has been added to Python Data Science Module package. The Python Data Science modules are installed with Python 3 in the Docker image on Pivotal Network.
Note: PL/Container 2.0.x and earlier do not support Python 3.
For information about PL/Container, see PL/Container Language.
PXF version 5.10.1 is included, which introduces bug fixes.
The metrics collector extension included with Greenplum Database 6.3.0 adds a new
gpcc.enable_query_profiling
server configuration parameter that can be enabled to help with performance troubleshooting. Whenoff
, the default, the metrics collector does not collect queries executed by thegpmon
user in the gpperfmon database or plan node history for queries that run in less than ten seconds. If you enablegpcc.enable_query_profiling
in a session the metrics collector collects those queries in that session. See Metrics Collector Server Configuration Parameters in the Greenplum Command Center documentation for more information.
Resolved Issues
Pivotal Greenplum 6.3.0 is a minor release that resolves these issues:
30214 - Query Optimizer
The GPORCA algebrizer might generate a PANIC when optimizing a query involving a window function where one or more of the columns selected was a subquery. This issue is resolved.
30252 - Storage: Filespace / Tablespace
Data distribution errors could lead to data corruption if UPDATE
or DELETE
statements resulted in data movement between segments (for example, if the UPDATE
of an affected tuple was distributed to another segment). The code was modified to check for distribution problems during UPDATE
and DELETE
operations, and to error out and cancel the operation in order to prevent data corruption. Problems detected in this manner are reported with the error: distribution key of the tuple doesn't belong to current segment (actually from segment_id)
30264 - Segment Mirroring
In some cases when Greenplum Database segment mirroring is enabled, loading a large amount of data caused mirror segment instances to fail due to a timeout issue. This issue has been resolved.
30300 - gpbackup/gprestore
When a view was restored from a backup and the view definition contained the function gp_dist_random()
, the definition of the restored view did not contain the function. Greenplum Database has been updated to resolve this issue. Now the restored view contains the correct definition.
30301 - analyzedb
The analyzedb
command could take a long time to complete when Greenplum Database incorrectly determined that the statistics for a child partition were not up to date and subsequently resampled the statistics for all partitions. This issue is resolved.
30320 - COPY
In some cases, Greenplum Database returned a segment reject limit reached
error when a COPY
operation specified a SEGMENT REJECT LIMIT
and Greenplum encountered a data formatting error. Because Greenplum rescanned the offending line, it returned the error even before the error limit had been reached. This issue is resolved.
30321 - Query Optimizer
When computing the join order for certain queries, ORCA tries to create a set of all the tables that have a predicate in common with the current join tree, and then pick one of the tables from this set. In certain cases involving left joins, ORCA would error out if it was unable to pick a table to join from this set. This has now been fixed and ORCA no longer falls back for such queries.
30334 - Storage: DDL
A system panic could occur if ALTER TABLE
was used to add a new partition, and WITH (OIDS=FALSE)
was specified as the only storage parameter for the new partition. The problem was caused by code that failed to handle the possibility of a null reloption
value, which is generated when the single default storage parameter is specified. The partitioning code was modified to correctly handle the possibility of such null values.
30348 - gpload
Attempting to run gpload
installed with Greenplum Clients 6.2.1 returned this error: No module named gppylib.gpversion
. This issue has been resolved.
169749131 - Segment Mirroring
When Greenplum Database segment mirroring is enabled, loading a large amount of data into append-optimized tables caused database performance issues and might have caused mirror instance failures with a walsender replication timeout error. The issues occurred because Greenplum Database was not efficiently writing transaction log records from primary to mirror segment instances. This issue has been resolved. Writing transaction log records from primary to mirror segment instances has been improved.
170021921 - Workload Manager
Transaction performance issues occurred when the gp_wlm
extension was loaded and the Greenplum Command Center workload management feature was not enabled. This is fixed in the gp_wlm
extension included with Greenplum Database 6.3.0.
170346082 - PXF
The PXF JDBC Connector did not support dynamic session authorization in a remote SQL database. This issue is resolved; PXF now supports session authorization, and introduces the ${pxf.session.user}
value and the jdbc.pool.qualifier
property as described in About Session Authorization in the JDBC Connector configuration documentation.
170476535 - PXF
PXF incorrectly wrote a Parquet decimal value that was specified with precision and scale settings, and was unable to read the decimal value back. This issue is resolved.
Release 6.2
Release 6.2.1
Release Date: 2019-12-12
Pivotal Greenplum 6.2.1 is a minor release that includes new features and resolves several issues.
New Features
Greenplum Database 6.2.1 includes these new features:
Greenplum Database supports materialized views. Materialized views are similar to views. A materialized view enables you to save a frequently used or complex query, then access the query results in a
SELECT
statement as if they were a table. Materialized views persist the query results in a table-like form. Materialized view data cannot be directly updated. To refresh the materialized view data, use theREFRESH MATERIALIZED VIEW
command. See Creating and Managing Materialized Views.Note: Known Issues and Limitations describes a limitation of materialized view support in Greenplum 6.2.1.
The
gpinitsystem
utility supports the--ignore-warnings
option. The option controls the value returned bygpinitsystem
when warnings or an error occurs. If you specify this option,gpinitsystem
returns 0 if warnings occurred during system initialization, and returns a non-zero value if a fatal error occurs. If this option is not specified,gpinitsystem
returns 1 if initialization completes with warnings, and returns value of 2 or greater if a fatal error occurs.PXF version 5.10.0 is included, which introduces several new and changed features and bug fixes. See PXF Version 5.10.0 below.
PXF Version 5.10.0
PXF 5.10.0 includes the following new and changed features:
- PXF has improved its performance when reading a large number of files from HDFS or an object store.
- PXF bundles newer
tomcat
andjackson
libraries. - The PXF JDBC Connector now supports pushdown of
OR
andNOT
logical filter operators when specified in a JDBC named query or in an external table query filter condition. - PXF supports writing Avro-format data to Hadoop and object stores. Refer to Reading and Writing HDFS Avro Data for more information about this feature.
- PXF is now certified with Hadoop 2.x and 3.1.x and Hive Server 2.x and 3.1, and bundles new and upgraded Hadoop libraries to support these versions.
- PXF supports Kerberos authentication to Hive Server 2.x and 3.1.x.
- PXF supports per-server user impersonation configuration.
- PXF supports concurrent access to multiple Kerberized Hadoop clusters. In previous releases of Greenplum Database, PXF supported accessing a single Hadoop cluster secured with Kerberos, and this Hadoop cluster must have been configured as the
default
PXF server. - PXF introduces a new template file,
pxf-site.xml
, to specify the Kerberos and impersonation property settings for a Hadoop or JDBC server configuration. Refer to About Kerberos and User Impersonation Configuration (pxf-site.xml) for more information about this file. - PXF now supports connecting to Hadoop with a configurable Hadoop user identity. PXF previously supported only proxy access to Hadoop via the
gpadmin
Greenplum user. PXF version 5.10.0 deprecates the following configuration properties.
Note: These property settings continue to work.
- The
PXF_USER_IMPERSONATION
,PXF_PRINCIPAL
, andPXF_KEYTAB
settings in thepxf-env.sh
file. You can use thepxf-site.xml
file to configure Kerberos and impersonation settings for your new Hadoop server configurations. - The
pxf.impersonation.jdbc
property setting in thejdbc-site.xml
file. You can use thepxf.service.user.impersonation
property to configure user impersonation for a new JDBC server configuration.
- The
Note: If you have previously configured a PXF JDBC server to access Kerberos-secured Hive, you must upgrade the server definition. See Upgrading PXF in Greenplum 6.x) for more information.
Changed Features
Greenplum Database 6.2.1 includes these changed features:
- Greenplum Stream Server version 1.3.1 is included in the Greenplum distribution.
Resolved Issues
Pivotal Greenplum 6.2.1 is a minor release that resolves these issues:
29454 - gpstart
During Greenplum Database start up, the gpstart
utility did not report when a segment instance failed to start. The utility always displayed 0 skipped segment starts. This issue has been resolved. gpstart
output was also enhanced to provide additional warnings and summary information about the number of skipped segments. For example:
[WARNING]:-****************************************************************************
[WARNING]:-There are 1 segment(s) marked down in the database
[WARNING]:-To recover from this current state, review usage of the gprecoverseg
[WARNING]:-management utility which will recover failed segment instance databases.
[WARNING]:-****************************************************************************
30248, 9022 - DLL
Greenplum Database might generate a PANIC when an index is created on a column of an append-optimized, column-oriented table if the index definition contains a WHERE
clause that references multiple columns. This issue has been resolved.
7545 - Postgres Planner
The Postgres Planner might return incorrect results for queries that contain a subquery in an EXISTS
clause if the subquery includes a LIMIT [ 0| ALL| NULL]
clause or an OFFSET NULL
clause. This issue has been resolved.
8590 - Postgres Planner
A query that used the Postgres planner could return incorrect results if it specified a volatile function in a LIMIT
clause (for example, LIMIT (random() * 10)
). This occurred because Greenplum evaluated the LIMIT
clause separately on each segment instance to obtain a preliminary limit, before evaluating it once again as the query was dispatched. The problem was fixed by ensuring that a volatile functions in a LIMIT
clause functions are not pushed to segment instances for evaluation.
30083 - Postgres Planner
Fixed a problem in the Postgres planner that could result in the error variable not found in subplan target list
. The issue applied to join queries where a table column had a user prescribed CAST applied to it while being both in the select list and in a join condition. At the same time, the column was also part of a motion operator in the query plan.
30200 - Metrics Collector
Greenplum Database 6 stores tablespaces with non-default names as symlinks in the $MASTER_DATA_DIRECTORY/pg_tblspc
directory and the metrics collector did not detect these tablespaces. The metrics collector now follows the symlinks to find the names of the tablespace directories and the data directories located in those tablespaces. After enabling the new metrics collector the tablespaces may not be visible in Greenplum Command Center for up to four hours.
30203 - Query Optimizer
When updating a table’s distribution column, Greenplum Database returned an error that states an UPDATE
statement cannot update distribution columns if a btree index is defined on the distribution column and the UPDATE
command contains an IN
clause. The error was returned when Greenplum Database fell back to the Postgres planner to attempt the UPDATE
operation. This issue has been resolved. Now GPORCA supports the specified type of updates.
30206 - gpinitsystem
An example in the gpinitsystem
help output used an invalid option for specifying the placement of mirror segment instances in a spread configuration. The correct option is --mirror_mode=spread
. This issue has been resolved.
30227 - Server
Greenplum Database with resource groups enabled might generate a PANIC when using an extension with improper debug_query_string
settings. The cause was a message context issue and it has been resolved.
30256 - analyzedb
When executing some queries against partitioned tables, GPROCA would fail because of missing root partition statistics. This was caused by the analyzedb
utility not updating the root partition statistics when generating the partitioned table statistics. This issue has now been resolved.
30292 - External Table
When Greenplum Database attempted to access data from an external table, a PANIC was generated when Greenplum Database could not resolve the host name that is specified in the external table definition. This issue has been resolved. Now Greenplum Database returns an error in the specified situation.
168881383 - PXF
PXF fixed a regression in file and directory name pattern matching that affected the *:text:multi
profiles and S3 Select. This issue has been resolved. PXF now correctly handles wildcards specified in the LOCATION
data path.
8918 - Postgres Planner
The Postgres Planner generated an incorrect result on a JOIN
query when different data types were used in a table column or the query constraints included a constant, and the query required motion. This issue is resolved.
169694492 - Query Optimizer
For a table that has a column that is defined with a btree index, GPORCA fell back to the Postgres planner for queries that use IN
clause against the column or an OR
of simple comparisons on the column such as col = 5 OR col = 7
. Now GPORCA attempts to generate a query plan that uses the index.
169806983 - Greenplum Stream Server
In some cases, reading from Kafka using the default MINIMAL_INTERVAL
(0 seconds) caused GPSS to consume a large amount of CPU resources, even when no new messages existed in the Kafka topic. This issue is resolved in GPSS 1.3.1.
169807372, 169831558 - Greenplum Stream Server
GPSS 1.3.0 did not recognize internal history tables that were created with GPSS 1.2.6 and earlier. In some cases, this caused GPSS to load duplicate messages into Greenplum Database. This issue is resolved in GPSS 1.3.1.
170041280 - PXF
PXF was unable to read data from an encrypted HDFS zone and returned an org.apache.hadoop.crypto.CryptoInputStream cannot be cast to org.apache.hadoop.hdfs.DFSInputStream
error in this situation. This issue is resolved.
Release 6.1
Release 6.1.0
Release Date: 2019-11-1
Pivotal Greenplum 6.1.0 is a minor release that includes new features and resolves several issues.
Features
Greenplum Database 6.1.0 includes these new features:
Greenplum Stream Server 1.3 is included, which introduces new features and bug fixes.
Note: Greenplum Stream Server (GPSS) and Greenplum-Kafka Integration users: Do not upgrade to Greenplum Database 6.1 if you plan to re-submit Kafka load jobs that you initiated with GPSS in Greenplum 6.0.x. Due to a regression, GPSS may load duplicate Kafka messages into Greenplum. Refer to Known Issues and Limitations for more information.
New GPSS features include:
- GPSS now supports log rotation, utilizing a mechanism that you can easily integrate with the Linux
logrotate
system. See Managing GPSS Log Files for more information. - GPSS has added the new
INPUT:FILTER
load configuration property. This property enables you to specify a filter that GPSS applies to Kafka input data before loading it into Greenplum Database. - GPSS displays job progress by partition when you provide the
--partition
flag to thegpsscli progress
command. - GPSS enables you to load Kafka data that was emitted since a specific timestamp into Greenplum Database. To use this feature, you provide the
--force-reset-timestamp
flag when you rungpsscli load
,gpsscli start
, orgpkafka load
. - GPSS now supports update and merge operations on data stored in a Greenplum Database table. The load configuration file accepts
MODE
,MATCH_COLUMNS
,UPDATE_COLUMNS
, andUPDATE_CONDITION
property values to direct these operations. Example: Merging Data from Kafka into Greenplum Using the Greenplum Stream Server provides an example merge scenario. - GPSS supports Kerberos authentication to both Kafka and Greenplum Database.
- GPSS supports SSL encryption between GPSS and Kafka.
- GPSS supports SSL encryption on the data channel between GPSS and Greenplum Database.
- GPSS now supports log rotation, utilizing a mechanism that you can easily integrate with the Linux
The DataDirect JDBC and ODBC drivers were updated to versions 5.1.4.000270 (F000450.U000214) and 07.16.0334 (B0510, U0363), respectively.
The DataDirect JDBC driver introduces support for the
prepareThreshold
connection parameter, which specifies the number of prepared statement executions that can be performed before the driver switches to using server-side prepared statements. This parameter defaults to 0, which preserves the earlier driver behavior of always using server-side prepare for prepared statements. Set a number greater than 1 to set a threshold after which server-side prepare is used.Note:
ExecuteBatch()
always uses server-side prepare for prepared statements. This matches the behavior of the Postgres open source driver.When the
prepareThreshold
value is greater than 1, parameterized operations do not send any SQL prepare calls withconnection.prepareStatement()
. The driver instead sends the query all at once, at execution time. Because of this limitation, the driver must determine the type of every column using the JDBC API before sending the query to the server. This determination works for many data types, but does not work for the following types that could be mapped to multiple Greenplum data types: - BIT VARYING - BOOLEAN - JSON - TIME WITH TIME ZONE - UUIDCOLYou must set
prepareThreshold
to 0 before using parameterized operations with any of the above types. Examine theResultSetMetaData
object in advance to determine if any of the above types are used in a query. Also keep in mind that GPORCA does not support prepared statements that have parameterized values, and will fall back to using the Postgres Planner.See PrepareThreshold in the DataDirect documentation.
Resolved Issues
Pivotal Greenplum 6.1.0 is a minor release that resolves these issues:
8804 - Server
In some cases, running the EXPLAIN ANALYZE
command on a sorted query in utility mode would cause the segment to crash. This issue is fixed. Greenplum Database no longer crashes in this situation.
8636 - Server
Some users encountered Error: unrecognized parameter "appendoptimized"
while creating a partitioned table that specified the appendoptimized=true
storage parameter. This issue is fixed; the Greenplum Database server now properly recognizes the appendoptimized
parameter when it is specified on partition table creation.
26225 - gpcheckcat
The gpcheckcat
utility failed to generate a summary report if there was an orphan TOAST table entry in one of the segments. This is fixed. The string “N/A” is reported when there is no relation OID to report.
29580 - Management and Monitoring
During Greenplum Database startup, an extra empty log file was produced ahead of the current date while performing time-based rotation of log files. For example, if Greenplum started at midnight September 2nd, two log files were generated, gpdb-2019-09-02_000000.csv
and gpdb-2019-09-03_000000.csv
. This issue has now been fixed.
29984 - Server
During startup, idle query executor (QE) processes can commit up to 16MB of memory each, but they are not tracked by the Linux virtual memory tracker. In a worst-case scenario, these idle processes could trigger OOM errors that were difficult to diagnose. To prevent these situations, Greenplum now hard-codes a startup memory cost to account for untracked QE processes.
30112 - Query Optimizer
For some queries against partitioned tables that contain a large amount of data, GPORCA generated a sub-optimal query plan because of inaccurate cardinality estimation. This issue has been resolved. GPORCA cardinality estimation has been improved.
30183, 30184 - analyzedb
When running the analyzedb
command with the --skip_root_stats
option, the command could take a long time to finish when analyzing a partitioned table with many partitions due to how the root partition statistics were handled when the partitions were analyzed. This issue has been resolved. Now, only partition statistics are updated.
Note: GPORCA uses root partition statistics. If you use –skip_root_stats
option, you should ensure that root partition statistics are up to date so that GPORCA does not produce inferior query plans due to stale root partition statistics.
30149 - Query Execution
A query might fail and return an error with the message invalid seek in sequential BufFile
when the server configuration file gp_workfile_compression
is on
and the query spills to temporary workfiles. The error was caused due to an issue working with workfiles that contain compressed data. The issue has been resolved by correctly handling the compressed workfile data.
30150 - Query Execution
A query might fail and return with the message AssignTransactionId() called by Segment Reader process
when the server configuration parameter temp_tablespaces
is set. The error was cause by an internal locking and transaction ID issue. This issue has been resolved by removing the requirement to acquire the lock.
30160 - Query Optimizer
GPORCA might return incorrect results when a the query contains a join predicate where one side is distributed on a citext
column, and the other is not. GPORCA did not use the correct hash when generating a plan that redistributes the citext
column. Now Greenplum Database falls back to the Postgres Planner for the specified type of query.
30183 - analyzedb
The analyzedb
command could take a long time to finish when analyzing a table with many partitions. The command’s performance has been greatly improved by waiting to update the root partition statistics until all leaf partitions of a table have been analyzed.
164823612 - gpss
GPSS incorrectly treated Kafka jobs that specified the same Kafka topic and Greenplum output schema name and output table name, but different database names, as the same job. This issue has been resolved. GPSS now includes the Greenplum database name when constructing a job definition.
167997441 - gpss
GPSS did not save error data to the external table error log when it encountered an incorrectly-formatted JSON or Avro message. This issue has been fixed; invoking gp_read_error_log()
on the external table now displays the offending data.
168130147 - gpss
In some situations, specifying the --force-reset-earliest
flag when loading data failed to read from the correct offset. This problem has been fixed. (Using the --force-reset-xxx
flags outside of an offset mismatch scenario is discouraged.)
168393571 - Query Optimizer
Certain queries with btree indexes on Append Optimized (AO) tables were unnecessarily slow due to GPORCA selecting a scan with high transformation and cost impact. This issue has been fixed by improving GPORCA handling of btree type indexes.
168393645 - Query Optimizer
In some situations, a query ran slow because GPORCA did not produce an optimal plan when it encountered a null-rejecting predicate where an operand could be false or null, but not true. This issue is fixed; GPORCA now produces a more optimal plan when evaluating null-rejecting predicates for AND
and OR
operands.
168705484 - Query Optimizer
For certain queries with a UNION operator over a large number of children, GPORCA query optimization required a long time. This issue has been addressed by adding the ability to derive scalar properties on demand.
168707515 - Query Optimizer
Some queries in GPORCA were consuming more memory than necessary due to suboptimal memory tracking. This has been fixed by optimizing memory accounting inside GPORCA.
169081574 - Interconnect
Greenplum Database might generate a PANIC when the server configuration parameter gp_interconnect_type
is TCP
due to an issue with memory management during interconnect setup. The issue has been resolved by properly managing the internal interconnect object memory.
169117536 - Execution
Greenplum Database might generate a PANIC when the server configuration parameter log_min_messages
is set to debug5
. Greenplum Database did not properly handle a debug5 message correctly. The issue is resolved.
169198230 - Plan Cache
A prepared statement might run slow because a cost model issue prevented Greenplum Database from generating a direct dispatch plan for the statement. This issue is fixed. Greenplum Database now introduces non-direct dispatch cost into the cost model only for cached plans, and tries to use direct dispatch for prepared statements when possible.
Release 6.0
Release 6.0.1
Release Date: 2019-10-11
Pivotal Greenplum 6.0.1 is a maintenance release that includes changed features and resolves several issues.
Changed Features
Greenplum Database 6.0.1 includes these changed features:
- The default value for the server configuration parameter
optimizer_use_gpdb_allocators
has been changed totrue
. Now, as the default, GPORCA uses Greenplum Database memory management when executing queries instead of GPORCA-specific memory management. Greenplum Database memory management has several enhancements when compared to GPORCA-specific memory management. See optimizer_use_gpdb_allocators. - Writing parquet data using the PXF Hadoop and object store connectors is no longer considered a Beta feature in this release.
Resolved Issues
Pivotal Greenplum 6.0.1 is a maintenance release that resolves these issues:
29712 - Query Execution
Greenplum Database writes unnecessary could not unlink file
log messages for spill files when gp_enable_query_metrics
is on
and log_min_messages
is set to INFO
. This issue has been resolved. Logging has been improved to not write the log message.
30058 - Query Execution
An internal EXPLAIN
function, cdbexplain_localExecStats
, operated under the assumption that it was always executed by a query dispatcher (QD) process. However, certain queries could generate plans where the function was executed by a query executor (QE) process. Running such queries with EXPLAIN ANALYZE
would cause all segments to crash with segment faults, and error messages referencing cdbexplain_localExecStats
. This problem has been resolved.
30094 - Resource Management
In some cases Greenplum Database generated a PANIC when a query was terminated and the query involved catalog tables. The PANIC was caused when a backend process did not properly clean up shared memory before exiting. This issue has been resolved. Now backend process memory management has been improved for the specified situation.
30098 - COPY
Greenplum Database generated a PANIC when a COPY
command attempted to write to a catalog table. This issue has been resolved, now Greenplum Database returns an error.
30120 - GRANT
The command GRANT ALL ON ALL TABLES IN SCHEMA <schema> TO <role>
caused a PANIC when tables are partitioned or inherited by child tables. This issue has been fixed.
30130 - gpexpand
The gpexpand
utility might have failed with a Cannot allocate memory
error when system memory or swap space is low. The error was due to an issue with a python process library. This has been resolved by updating the python library.
165660593 - Resource Groups
When resource groups are enabled, Greenplum Database might return an out of memory error when executing a SET
or SHOW
command, or when executing a query when the Greenplum Database server configuration parameter gp_resource_group_bypass
is set to true
. The error is due to an issue with resource group memory accounting. This issue has been resolved, resource group memory accounting has been improved for individual statements.
167847839 - gpconfig
The code dispatched from the master to segments to set a configuration parameter enclosed the value in single quotes. This did not handle values containing embedded single quotes, and parameters with the GUC_LIST_QUOTE
flag, such as search_path
, ended up having different values on the segments than on the master. For example,
SELECT set_config('search_path', 'my_schema,public', false);
was dispatched as
SET search_path TO 'my_schema,public';
instead of
SET search_path TO my_schema,public;
This is fixed. The set_config()
call is now dispatched to the segments as well, passing the (quoted) arguments directly so that the same code runs on the master and segments.
Known Issue 167851039 - PXF
pxf cluster reset
did not reset PXF configuration on the standby master. This issue is fixed; the command now resets PXF configuration on all hosts in the Greenplum cluster, including the standby master.
Known Issue 167851065 - PXF
PXF allowed you to initialize PXF without setting PXF_CONF
. This issue has been resolved. PXF now correctly checks for this setting before continuing with initialization.
Known Issue 167948506 - PXF
In some cases, accessing an S3 object store with PXF failed when PXF was configured for Kerberized Hadoop. This issue is fixed. PXF now handles token renewal appropriately on concurrent access to S3 and a Kerberized Hadoop cluster.
While not required, if you have previously instituted any of the workarounds identified for this issue that are described in Known Issues and Limitations, you may consider reverting the s3-site.xml
modifications or removing the yarn-site.xml
file from your S3 server directory.
168167337- gpinitsystem
When running gpinitsystem
to initialize a Greenplum Database system with mirroring enabled, the utility configured the pg_hba.conf
file in a manner that did not permit incremental recovery of the primary segment instances. This issue is fixed in release 6.0.1. The gpexpand
, gprecoverseg
, and gpaddmirrors
utilities were also updated to ensure that primary and mirror segments always have compatible pg_hba.conf
entries in place after performing their respective operations.
Known Issue 168271005 - PXF
The PXF JDBC Connector failed to access any database when PXF was configured to use the MapR Hadoop distribution, and MapR libraries were present in $PXF_CONF/lib
. This issue has been resolved. A PXF MapR server configuration no longer affects JDBC access using PXF.
168759361 - psql
Greenplum Database was previously built and dynamically linked against libedit
as a required dependency. However, the version of libedit
that is available on Redhat 7 is not compatible with features such as tab-completion in psql
. The readline
library provides the same functionality as libedit
, and all versions available across all supported platforms are compatible with the desired features. Therefore, instead of dynamically linking to libedit
, Greenplum Database is now being built and dynamically linked to readline
. This changes the required dependencies for the installers from libedit
to readline
.
Release 6.0.0
Release Date: 2019-09-03
Pivotal Greenplum 6.0.0 is a major new release of Greenplum that includes new and changed features.
New Features
Pivotal Greenplum 6 includes these new features.
- PostgreSQL Core Features
- Zstandard Compression Algorithm
- Relaxed Rules for Specifying Table Distribution Columns
- Replicated Table Data
- Resource Groups Features
- PL/pgSQL Procedural Language Enhancements
- Concurrency Improvements in Greenplum 6
- Additional Contrib Modules
- PXF Version 5.8.1
- Additional Greenplum Database Features
PostgreSQL Core Features
Pivotal Greenplum 6 incorporates several new features from PostgreSQL versions 8.4 through version 9.4.
INTERVAL Data Type Handling
PostgreSQL 8.4 improves the parsing of INTERVAL
literals to align with SQL standards. This changes the output for queries that use INTERVAL
labels between versions 5.x and 6.x. For example:
$ psql
psql (8.3.23)
Type "help" for help.
gpadmin=# select INTERVAL '1' YEAR;
interval
----------
00:00:00
(1 row)
$ psql
psql (9.2beta2)
Type "help" for help.
gpadmin=# select INTERVAL '1' YEAR;
interval
----------
1 year
(1 row)
See Date/Time Types for more information.
Additional PostgreSQL Features
Greenplum Database 6.0 also includes these features and changes from PostgreSQL:
- Support for user-defined I/O conversion casts. (PostgreSQL 8.4).
- Support for column-level privileges (PostgreSQL 8.4).
- The
pg_db_role_setting
catalog table, which provides support for setting server configuration parameters for a specific database and role combination (PostgreSQL 9.0). - Values in the
relkind
column of thepg_class
catalog table were changed to match entries in PostgreSQL 9.3. - Support for GIN index method (PostgreSQL 8.3).
- Postgres Planner support for the SP-GiST index access method (PostgreSQL 9.2). (GPORCA ignores SP-GiST indexes.)
- Postgres Planner support for ordered-set aggregates and moving-aggregates (PostgreSQL 9.4).
- Support for
jsonb
data type (PostgreSQL 9.4). DELETE
,INSERT
, andUPDATE
supports theWITH
clause, CTE (common table expression) (PostgreSQL 9.1).Collation support to specify sort order and character classification behavior for data at the column level (PostgreSQL 9.1).
Note: GPORCA supports collation only when all columns in the query use the same collation. If columns in the query use different collations, then Greenplum uses the Postgres Planner.
Zstandard Compression Algorithm
Greenplum Database 6.0 adds support for zstd (Zstandard) compression for some database operations. See Enabling Compression.
Relaxed Rules for Specifying Table Distribution Columns
In previous releases, if you specified both a UNIQUE
constraint and a DISTRIBUTED BY
clause in a CREATE TABLE
statement, then the DISTRIBUTED BY
clause was required to be equal to or a left-subset of the UNIQUE
columns. Greenplum 6.x relaxes this rule so that any subset of the UNIQUE
columns is accepted.
This change also affects the rules for how Greenplum 6.x selects a default distribution key. If gp_create_table_random_default_distribution
is off (the default) and you do not include a DISTRIBUTED BY
clause, then Greenplum chooses the table distribution key based on the command:
- If a
LIKE
orINHERITS
clause is specified, then Greenplum copies the distribution key from the source or parent table. - If a
PRIMARY KEY
orUNIQUE
constraints are specified, then Greenplum chooses the largest subset of all the key columns as the distribution key. - If neither constraints nor a
LIKE
orINHERITS
clause is specified, then Greenplum chooses the first suitable column as the distribution key. (Columns with geometric or user-defined data types are not eligible as Greenplum distribution key columns.)
Resource Groups Features
Greenplum Database includes these new resource group features:
- You no longer are required to specify a
MEMORY_LIMIT
when you configure a Greenplum Database resource group. When you specifyMEMORY_LIMIT=0
, Greenplum Database will use the resource group global shared memory pool to service queries running in the group. When you specify
MEMORY_SPILL_RATIO=0
, Greenplum Database will now use thestatement_mem
server configuration parameter setting to identify the initial amount of query operator memory.When used together to configure a resource group (
MEMORY_LIMIT=0
andMEMORY_SPILL_RATIO=0
), these new capabilities provide a memory management scheme similar to that provided by Greenplum Database resource queues.The default values of the
MEMORY_SHARED_QUOTA
,MEMORY_SPILL_RATIO
, andMEMORY_LIMIT
attributes for theadmin_group
anddefault_group
resource groups have been set to use this resource queue-like memory management scheme so that when you initially enable resource groups, your queries will run in a memory environment similar to before.Resource Group admin_group default_group MEMORY_LIMIT 10 0 MEMORY_SHARED_QUOTA 80 80 MEMORY_SPILL_RATIO 0 0
PL/pgSQL Procedural Language Enhancements
PL/pgSQL in Greenplum Database 6.0 includes support for the following new features:
- Attaching
DETAIL
andHINT
text to user-thrown error messages. You can also specify theSQLSTATE
andSQLERRMSG
codes to return on a user-thrown error (PostgreSQL 8.4). - The
RETURN QUERY EXECUTE
statement, which specifies a query to execute dynamically (PostgreSQL 8.4). - Conditional execution using the
CASE
statement (PostgreSQL 8.4). See Conditionals in the PostgreSQL documentation.
Replicated Table Data
The CREATE TABLE
command supports DISTRIBUTED REPLICATED
as a distribution policy. If this distribution policy is specified, Greenplum Database distributes all rows of the table to all segment instances in the Greenplum Database system.
Note: The hidden system columns ( ctid
, cmin
, cmax
, xmin
, xmax
, and gp_segment_id
) cannot be referenced in user queries on replicated tables because they have no single, unambiguous value. Greenplum Database returns a column does not exist
error for the query.
Concurrency Improvements in Greenplum 6
Greenplum Database 6 includes the following concurrency improvements:
- Global Deadlock Detector - Previous versions of Greenplum Database prevented global deadlock by holding exclusive table locks for
UPDATE
andDELETE
operations. While this strategy did prevent deadlocks, it came at the cost of poor performance on concurrent updates. Greenplum Database 6 includes a global deadlock detector. This backend process collects and analyzes lock waiting data in the Greenplum cluster. If the Global Deadlock Detector determines that deadlock exists, it breaks the deadlock by cancelling one or more backend processes. By default, the global deadlock detector is deactivated and table-level exclusive locks are held for table updates. When the global deadlock detector is enabled, Greenplum Database holds row-level exclusive locks and concurrent updates are allowed. See Global Deadlock Detector. - Transaction Lock Optimization - Greenplum Database 6 optimizes transaction lock usage both when you
BEGIN
andCOMMIT
a transaction. This benefits highly concurrent mixed workloads. - Upstream PostgreSQL Features - Greenplum 6 includes upstream PostgreSQL features, including those for fastpath lock, which reduce lock contention. This benefits concurrent short queries and mixed workloads.
VACUUM
can more easily skip pages it cannot lock. This reduces the frequency of a vacuum appearing to be “stuck,” which occurs whenVACUUM
waits to lock a block for cleanup and another session has held a lock on the block for a long time. NowVACUUM
skips a block it cannot lock and retries the block later.VACUUM
rechecks block visibility after it has removed dead tuples. If all remaining tuples in the block are visible to current and future transactions, the block is marked as all-visible.- The tables that are part of a partitioned table hierarchy, but that do not contain data, are age-frozen so that they do not have to be vacuumed separately and do not affect calculation of the number of remaining transaction IDs before wraparound occurs. These tables include the root and intermediate tables in the partition heirarchy and, if they are append-optimized, their associated meta-data tables. This makes it unnecessary to vacuum the root partition to reduce the table’s age, and eliminates the possibly needless vacuuming of all of the child tables.
Additional Contrib Modules
Greenplum Database 6 is distributed with these additional PostgreSQL and Greenplum contrib
modules:
PXF Version 5.8.1
Greenplum Database 6.0 includes PXF 5.8.1, which introduces the following new and changed features:
- The PXF S3 Connector now supports accessing CSV and Parquet data on S3 using the Amazon S3 Select service. Refer to Reading CSV and Parquet Data on S3 Using S3 Select.
- PXF bundles new and upgraded libraries to provide Java 11 support.
- PXF has added support for the
timestamptz
type when writing Parquet data to an external data source. - PXF now provides a
reset
command to reset your local PXF server instance, or all PXF server instances in the cluster, to an uninitialized state. See Resetting PXF. - PXF no longer supports specifying a
DELIMITER
in theCREATE EXTERNAL TABLE
commandLOCATION
URI.
Additional Greenplum Database Features
Greenplum Database 6.0 also includes these features and changes from version 5.x:
- Recursive
WITH
Queries (Common Table Expressions) are no longer considered a Beta feature, and are now enabled by default. See WITH Queries (Common Table Expressions). VACUUM
was updated to more easily skip pages that cannot be locked. This change should greatly reduce the incidence ofVACUUM
getting “stuck” while waiting for other sessions.appendoptimized
alias for theappendonly
table storage option.- New
gp_resgroup_status_per_host
andgp_resgroup_status_per_segment
gp_toolkit
views to display resource group CPU and memory usage on a per-host and/or per-segment basis. - The new gp_stat_replication view contains replication statistics when master or segment mirroring is enabled. The
pg_stat_replication
view contains only master replication statistics. - The
gpfdists
andpsql
programs in the Greenplum Client and Loader Tools package for Windows support OpenSSL encryption. - Greenplum 6 includes some PostgreSQL 9.6 aggregate-related performance improvements.
- The
gpload
utility program provided in the Greenplum Client and Loader Tools package for Windows is compatible with Greenplum Database 5.
Greenplum 6.0 Beta Features
Because Pivotal Greenplum Database is based on the open source Greenplum Database project code, it includes several Beta features to allow interested developers to experiment with their use on development systems. Feedback will help drive development of these features, and they may become supported in future versions of the product.
Warning: Beta features are not recommended or supported for production deployments.
Key experimental features in Greenplum Database 6.0 include:
Storage plugin API for
gpbackup
andgprestore
. Partners, customers, and OSS developers can develop plugins to use in conjunction withgpbackup
andgprestore
.For information about the storage plugin API, see Backup/Restore Storage Plugin API.
Using the Greenplum Platform Extension (PXF) connectors to write Parquet data is a Beta feature.
Changed Features
Greenplum Database 6.0 includes these feature changes:
- The performance characteristics of Greenplum Database under heavy loads have changed in version 6 as compared to previous versions. In particular, you may notice increased I/O operations on primary segments for changes related to GPSS, WAL replication, and other features. All customers are encouraged to perform load testing with real-world data to ensure that the new Greenplum 6 cluster configuration meets their performance needs.
gpbackup
andgprestore
are no longer installed with Greenplum Database 6, but are available separately on Pivotal Network and can be upgraded separately from the core database installation.- Greenplum 6 uses a new jump consistent hash algorithm to map hashed data values to Greenplum segments. The new algorithm ensures that, after new segments are added to the Greenplum 6 cluster, only those rows that hash to the new segment need to be moved. Greenplum 6 hashing has performance characteristics similar to earlier Greenplum releases, but should enable faster database expansion. Note that the new algorithm is more CPU intensive than the previous algorithm, so
COPY
performance may degrade somewhat on CPU-bound systems. The older, legacy hash functions are represented as non-default hash operator classes, named
cdbhash_*_ops
. The non-default operator classes are used when upgrading from Greenplum Database earlier than 6.0. The legacy operator classes are compatible with each other, but if you mix the legacy operator classes with the new ones, queries will require Redistribute Motions.The server configuration parameter gp_use_legacy_hashops controls whether the legacy or default hash functions are used when creating tables that are defined with a distribution column.
The
gp_distribution_policy
system table now contains more information about Greenplum Database tables and the policy for distributing table data across the segments including the operator class of the distribution hash functions.The
gpcheck
utility is no longer included in Greenplum Database 6.The input file format for the
gpmovemirrors
,gpaddmirrors
,gprecoverseg
andgpexpand
utilities has changed. Instead of using a colon character:
as a separator, the new file format uses a pipe character|
. For example, in previous releases a line in agpexpand
input file would resemble:sdw5:sdw5-1:50011:/gpdata/primary/gp9:11:9:p
The updated file format is:
sdw5|sdw5-1|50011|/gpdata/primary/gp9|11|9|p
In addition,
gpaddmirrors
removes themirror
prefix from lines in its input file. Whereas a line from the previous release might resemble:mirror0=0:sdw1:sdw1-1:52001:53001:54001:/gpdata/mir1/gp0
The revised format is:
0=0|sdw1|sdw1-1|52001|53001|54001|/gpdata/mir1/gp0
Greenplum uses direct dispatch to target queries that use
IS NULL
, similar to queries that filter on the table distribution key column(s).The
gpinitsystem
option to specify the standby master data directory changed from-F
to-S
. The-S
option no longer specifies spread mirroring. A newgpinitsystem
option is introduced to specify the mirroring configuration:--mirror-mode={group|spread}
.- The default value of the server configuration parameter
log_rotation_size
has changed from 0 to 1GB. This changes the default log rotation behavior so that a new log file is opened when more than 1GB has been written to the current log file or when the current log file has been open for 24 hours. - The default value of the server configuration parameter
effective_cache_size
has changed from 512MB to 16GB. - The
gpssh-exkeys
utility now requires that you have already set up passwordless SSH from the master host to every other host in the cluster. Runninggpssh-exkeys
then sets up passwordless SSH from every host to every other host. - The
gpstop
smart shutdown behavior has changed. Previously, if you rangpstop -M smart
(or justgpstop
), the utility exited with a message if there were any active client connections. Now,gpstop
waits for current connections to finish before completing the shutdown. If any connections remain open after the timeout period, or if you interrupt with CTRL-C,gpstop
lists the open connections and prompts whether to continue waiting for connections to finish, or to perform a fast or immediate shutdown. The default timeout period is 120 seconds and can be changed with the-t timeout_seconds
option. - In the
pg_stat_activity
andpg_stat_replication
system views, theprocpid
column was renamed topid
to match the associated change in PostgreSQL 9.2. - In the
pg_proc
system table, theproiswin
column was renamed toproiswindow
and relocated in the table to match thepg_proc
system table in PostgreSQL 8.4. - Queries that use
SELECT DISTINCT
andUNION/INTERSECT/EXCEPT
no longer necessarily return sorted output. Previously these queries always removed duplicate rows by using Sort/Unique processing. They now implement hashing to conform to behavior introduced in PostgreSQL 8.4; this method does not produce sorted output. If your application requires sorted output for these queries, alter the queries to use an explicitORDER BY
clause. Note thatSELECT DISTINCT ON
never uses hashing, so its behavior is unchanged from previous versions. - In the
gp_toolkit
schema, thegp_resgroup_config
view no longer contains the columnsproposed_concurrency
,proposed_memory_limit
,proposed_memory_shared_quota
andproposed_memory_spill_ratio
. - In the
pg_resgroupcapability
system table, theproposed
column has been removed. - The
pg_database
system tabledatconfig
column was removed. Greenplum Database now uses thepg_db_role_setting
system table to keep track of per-database and per-role server configuration settings (PostgreSQL 9.0). - The
pg_aggregate
system tableaggordered
column was removed, and several new columns were added to the table to support ordered-set aggregates and moving-aggregates with the Postgres Planner (PostgreSQL 9.4). TheALTER/CREATE/DROP AGGREGATE
SQL command signatures have also been updated to reflect thepg_aggregate
catalog changes. - The
pg_authid
system tablerolconfig
column was removed. Greenplum Database now uses thepg_db_role_setting
system table to keep track of per-database and per-role server configuration settings (PostgreSQL 9.0). - When creating and altering a table that has a distribution column, you can now specify the hash function used to distribute data across segment instances.
- Pivotal Greenplum Database 6 removes the
RECHECK
option fromALTER OPERATOR FAMILY
andCREATE OPERATOR CLASS
DDL (PostgreSQL 8.4). Greenplum now determines whether an index operator is “lossy” on-the-fly at runtime. - Operator-related system catalog tables are modified to support operator families, compatibility, and types (ordering or search).
- System catalog table entries for HyperLogLog (HLL) functions, aggregates, and types are modified to prefix names with
gp_
. Renaming the HLL functions prevents name collisions with external Greenplum Database extensions that use HLL. Any user code written to use the built-in Greenplum Database HLL functions must be updated to use the newgp_
names. - The data-type-specific
lag()
andlead()
functions are removed and replaced with more generic function signatures. Additionally, theoffset
argument to the functions is changed frombigint
to theinteger
data type. Refer to pg_upgrade Checks - Views with lead/lag functions using bigint in thegpupgrade
documentation or About Migrating Views Created with lag()/lead() Functions in the Greenplum Database documentation for information about these changes and the required upgrade or migration actions. - The “legacy optimizer” from previous releases of Greenplum is now referred to as the Postgres planner in both the code and documentation.
The transaction isolation levels in Greenplum Database 6.0 are changed to align with PostgreSQL transaction isolation levels since the introduction of the serializable snapshot isolation (SSI) mode in PostgreSQL 9.1. The new SSI mode, which is not implemented in Greenplum Database, provides true serializability by monitoring concurrent transactions and rolling back transactions that could introduce a serialization anomaly. The existing snapshot isolation (SI) mode guarantees that transactions operate on a single, consistent snapshot of the database, but does not guarantee a consistent result when a set of concurrent transactions is executed in any given sequence.
Greenplum Database 6.0 now allows the
REPEATABLE READ
keywords with SQL statements such asBEGIN
andSET TRANSACTION
. ASERIALIZABLE
transaction in PostgreSQL 9.1 or later uses the new SSI mode. ASERIALIZABLE
transaction in Greenplum Database 6.0 falls back toREPEATABLE READ
, using the SI mode. The following table shows the SQL standard compliance for each transaction isolation level in Greenplum Database 6.0 and PostgreSQL 9.1.Requested Transaction Isolation Level Greenplum Database 6.0 Compliance PostgreSQL 9.1 Compliance READ UNCOMMITTED READ COMMITTED READ COMMITTED READ COMMITTED READ COMMITTED READ COMMITTED REPEATABLE READ REPEATABLE READ (SI) REPEATABLE READ (SI) SERIALIZABLE Falls back to REPEATABLE READ (SI) SERIALIZABLE (SSI) The
CREATE TABLESPACE
command has changed.- The command no longer requires a filespace created with the
gpfilespace
utility. - The
FILESPACE
clause has been removed. - The
WITH
clause has been added to allow specifying a tablespace location for a specific segment instance. - A primary-mirror pair sharing the same content ID must use the same tablespace location.
- The command no longer requires a filespace created with the
- The
ALTER SEQUENCE
SQL command has new clausesSTART [WITH] start
andOWNER TO new_owner
(PostgreSQL 8.4). TheSTART
clause sets the start value that will be used by futureALTER SEQUENCE RESTART
commands, but does not change the current value of the sequence. TheOWNER TO
clause changes the sequence’s owner. - The
ALTER TABLE
SQL command has aSET WITH OIDS
clause to add anoid
system column to a table (PostgreSQL 8.4). Note that using oids with Greenplum Database tables is strongly discouraged. - The
CREATE DATABASE
SQL command has new parametersLC_COLLATE
andLC_CTYPE
to specify the collation order and character classification for the new database. - The
CREATE FUNCTION
SQL command has a new keywordWINDOW
, which indicates that the function is a window function rather than a plain function (PostgreSQL 8.4). - Specifying the index name in the
CREATE INDEX
SQL command is now optional. Greenplum Database constructs a default index name from the table name and indexed columns. - In the
CREATE TABLE
command, the Greenplum Database parser allows commas to be placed between aSUBPARTITION TEMPLATE
clause and its coorespondingSUBPARTITION BY
clause, and between consecutiveSUBPARTITION BY
clauses. Using this undocumented syntax will generate a deprecation warning message. - Superuser privileges are now required to create a protocol. See CREATE PROTOCOL.
- The
CREATE TYPE
SQL command has a newLIKE=type
clause that copies the new type’s representation (INTERNALLENGTH
,PASSEDBYVALUE
,ALIGNMENT
, andSTORAGE
) from an existing type (PostgreSQL 8.4). - The
GRANT
SQL command has new syntax to grant privileges on truncate, foreign data wrappers, and foreign data servers (PostgreSQL 8.4). - The
LOCK
SQL command has an optionalONLY
keyword (PostgreSQL 8.4). When specified, the table is locked without locking any tables that inherit from it. - Using the
LOCK table
statement outside of a transaction raises an error in Greenplum Database 6.0. In earlier releases, the statement executed, although it is only useful when executed inside of a transaction. - The
SELECT
andVALUES
SQL commands support the SQL 2008OFFSET
andFETCH
syntax (PostgreSQL 8.4). These clauses provide an alternative syntax for limiting the results returned by a query. - The
FROM
clause can be omitted from aSELECT
command, but Greenplum Database no longer allows queries that omit theFROM
clause and also reference database tables. - The
ROWS
andRANGE
SQL keywords have changed from reserved to unreserved, and may be used as table or column names without quoting. - In Greenplum 6, a query on an external table with descendants will by default recurse into the descendant tables. This is a change from previous Greenplum Database versions, which never recursed into descendants. To get the previous behavior in Greenplum 6, you must include the
ONLY
keyword in the query to restrict the query to the parent table. - The default value for the
optimizer_force_multistage_agg
server configuration parameter has changed fromtrue
tofalse
. GPORCA will now by default choose between a one-stage or two-stage aggregate plan for a scalar distinct qualified aggregate based on cost. - The
TRUNCATE
SQL command has an optionalONLY
keyword (PostgreSQL 8.4). When specified, the table is truncated without truncating any tables that inherit from it. - The
createdb
command-line utility has new options-l
(--locale
),--lc-collate
, and--lc-ctype
to specify the locale and character classification for the database (PostgreSQL 8.4). - The
pg_dump
,pg_dumpall
, andpg_restore
utilities have a new--role=rolename
option that instructs the utility to executeSET ROLE rolename
after connecting to the database and before starting the dump or restore operation (PostgreSQL 8.4). - The
pg_dump
andpg_dumpall
command-line utilities have a new option--lock-wait-timeout=timeout
(PostgreSQL 8.4). When specified, instead of waiting indefinitely the dump fails if the utility cannot acquire shared table locks within the specified number of milliseconds. - The
-d
and-D
command-line options are removed from thepg_dump
andpg_dumpall
utilities. The corresponding long versions,--inserts
and--column-inserts
are still supported. A new--binary-upgrade
option is added, for use by in-place upgrade utilities. - The
-w
(--no-password
) option was added to thepg_dump
,pg_dumpall
, andpg_restore
utilities. - The
-D
option is removed from thegpexpand
utility. The expansion schema will be created in thepostgres
database. - The
gpstate
utility has a new-x
option, which displays details of an in-progress system expansion.gpstate -s
andgpstate
with no options specified also report if a system expansion is in progress. - The
pg_restore
utility has a new option-j
(--number-of-jobs
) parameter. This option can reduce time to restore a large database by running tasks such as loading data, creating indexes, and creating constraints concurrently. - The
vacuumdb
utility has a new-F
(--freeze
) option to freeze row transaction information. ALTER DATABASE
includes theSET TABLESPACE
clause to change the default tablespace.CREATE DATABASE
includes theCOLLATE
andCTYPE
options for setting the collation order and character classification of the new database.- In the
gp_toolkit
schema, thegp_workfile_*
views have changed due to Greenplum Database 6 workfile enhancements. See Checking Query Disk Spill Space Usage for information aboutgp_workfile_*
views. - The server configuration parameter
gp_workfile_compress_algorithm
has been changed togp_workfile_compression
. When workfile compression is enabled, Greenplum Database uses Zstandard compression. - The Oracle Compatibility Functions are now available in Greenplum Database as an extension, based on the PostgreSQL orafce project at https://github.com/orafce/orafce. Instead of executing a SQL script to install the compatibility functions in a database, you now execute the SQL command
CREATE EXTENSION orafce
. The Greenplum Database 6.0 orafce extension is based on the orafce 3.7 release. See Oracle Compatibility Functions for information about differences between the Greenplum Database compatibility functions and the PostgreSQL orafce extension. - Greenplum Database 6 supports specifying a table column of the
citext
data type as a distribution key. - Greenplum Database 6 provides a single client and loader tool package that you can download and install on a client system. Previous Greenplum releases provided separate client and loader packages. For more information about the Greenplum 6 Clients package, refer to Client Tools in the platform requirements documentation.
- Greenplum Database 6 includes both PostgreSQL-sourced and Greenplum- sourced
contrib
modules. Most of these modules are now packaged as extensions, and you register an extension in Greenplum with theCREATE EXTENSION name
command. Refer to Installing Additional Supplied Modules for more information about registeringcontrib
modules in Greenplum Database 6. When Greenplum Database High Availability is enabled, a primary segment instance is kept up to date with the mirror segment instance using Write-Ahead Logging (WAL)-based streaming replication. See Overview of Segment Mirroring.
The gp_stat_replication view contains replication statistics when master or segment mirroring is enabled.
In previous releases, segment mirroring employed a physical file replication scheme.
In the
gp_segment_configuration
table, thereplication_port
has been removed. Thedatadir
column has been added to display the segment instance data directory. Themode
column values are now s (synchronized) or n (not synchronized). Use thegp_stat_replication
view to determine the synchronization state.The Greenplum Database 6 Client and Loader Tools package for Windows does not support running the
gpfdist
program as a native Windows service.
Removed Features
Pivotal Greenplum Database 6.0 removes these features:
- The
gpseginstall
utility is no longer included. You must install the Greenplum software RPM on each segment host, as described in Installing the Greenplum Database Software. - The
gptransfer
utility is no longer included; usegpcopy
for all functionality that was provided withgptransfer
. - The
gp_fault_strategy
system table is no longer used. Greenplum Database now uses thegp_segment_configuration
system table to determine if mirroring is enabled. - Pivotal Greenplum Database 6 removes the
gpcrondump
,gpdbrestore
, andgpmfr
management utilities. Usegpbackup
andgprestore
to back up and restore Greenplum Database. - Pivotal Greenplum Database 6 no longer supports Veritas NetBackup.
- Pivotal Greenplum Database 6 no longer supports the use of direct I/O to bypass the buffering of memory within the file system cache for backup.
- Pivotal Greenplum Database 6 no longer supports the
gphdfs
external table protocol to access a Hadoop system. Use the Greenplum Platform Extension Framework (PXF) to access Hadoop in version 6. Refer to pxf:// Protocol for information about using thepxf
external table protocol. - Pivotal Greenplum Database 6 no longer supports SSLv3.
- Pivotal Greenplum Database 6 removes the following server configuration parameters:
gp_analyze_relative_error
gp_backup_directIO
gp_backup_directIO_read_chunk_mb
gp_connections_per_thread
gp_enable_sequential_window_plans
gp_idf_deduplicate
gp_snmp_community
gp_snmp_monitor_address
gp_snmp_use_inform_or_trap
gp_workfile_checksumming
- The undocumented
gp_cancel_query()
function, and the configuration parametersgp_cancel_query_print_log
andgp_cancel_query_delay_time
, are removed in Greenplum Database 6. - The
string_agg(expression)
function is removed from Greenplum 6. The function concatenates text values into a string. Thestring_agg(expression, delimiter)
function is still supported. - Pivotal Greenplum Database 6 no longer supports the ability to configure a Greenplum Database system to trigger SNMP (Simple Network Management Protocol) alerts or send email notifications to system administrators if certain database events occur. Use Pivotal Greenplum Command Center alerts to detect and respond to events that occur in a Greenplum system.
- Pivotal Greenplum Database 6 removes the
gpfilespace
utility. TheCREATE TABLESPACE
command no longer requires a filespace created with the utility. Pivotal Greenplum Database 6 no longer automatically casts text from the deprecated timestamp format YYYYMMDDHH24MISS. The format could not be parsed unambiguously in previous Greenplum Database releases. The format is not supported in PostgreSQL 9.4.
For example, this command returns an error in Greenplum Database 6. In previous releases, a timestamp is returned.
# select to_timestamp('20190905140000');
In Greenplum Database 6, this command returns a timestamp.
# select to_timestamp('20190905140000','YYYYMMDDHH24MISS');
Pivotal Greenplum Database 6 removes the
--ignore-version
option from thepg_dump
,pg_dumpall
, andpg_restore
utilities.The
gpcheck
utility is no longer included.
Deprecated Features
Deprecated features will be removed in a future major release of Greenplum Database. VMware Tanzu Greenplum 6.x deprecates:
- The
gpsys1
utility. The
analzyedb
option--skip_root_stats
(deprecated since 6.2).If the option is specified, a warning is issued stating that the option will be ignored.
The server configuration parameter
gp_statistics_use_fkeys
(deprecated since 6.2).The server configuration parameter
gp_ignore_error_table
(deprecated since 6.0).To avoid a Greenplum Database syntax error, set the value of this parameter to
true
when you run applications that executeCREATE EXTERNAL TABLE
orCOPY
commands that include the now removed Greenplum Database 4.3.xINTO ERROR TABLE
clause.Specifying
=>
as an operator name in theCREATE OPERATOR
command (deprecated since 6.0).The Greenplum external table C API (deprecated since 6.0).
Any developers using this API are encouraged to use the new Foreign Data Wrapper API in its place.
Commas placed between a
SUBPARTITION TEMPLATE
clause and its correspondingSUBPARTITION BY
clause, and between consecutiveSUBPARTITION BY
clauses in aCREATE TABLE
command (deprecated since 6.0).Using this undocumented syntax will generate a deprecation warning message.
The timestamp format YYYYMMDDHH24MISS (deprecated since 6.0).
This format could not be parsed unambiguously in previous Greenplum Database releases, and is not supported in PostgreSQL 9.4.
The
createlang
anddroplang
utilities (deprecated since 6.0).The
pg_resqueue_status
system view (deprecated since 6.0).Use the
gp_toolkit.gp_resqueue_status
view instead.The
GLOBAL
andLOCAL
modifiers when creating a temporary table with theCREATE TABLE
andCREATE TABLE AS
commands (deprecated since 6.0).These keywords are present for SQL standard compatibility, but have no effect in Greenplum Database.
Using
WITH OIDS
oroids=TRUE
to assign an OID system column when creating or altering a table (deprecated since 6.0).Allowing superusers to specify the
SQL_ASCII
encoding regardless of the locale settings (deprecated since 6.0).This choice may result in misbehavior of character-string functions when data that is not encoding-compatible with the locale is stored in the database.
The
@@@
text search operator (deprecated since 6.0).This operator is currently a synonym for the
@@
operator.The unparenthesized syntax for option lists in the
VACUUM
command (deprecated since 6.0).This syntax requires that the options to the command be specified in a specific order.
The
plain
pgbouncer
authentication type (auth_type
=plain
) (deprecated since 4.x).
Known Issues and Limitations
VMware Tanzu Greenplum 6 has these limitations:
- Upgrading a Greenplum Database 4 release to VMware Tanzu Greenplum 6 is not supported. Upgrading a Greenplum Database 5.x release to VMware Tanzu Greenplum 6 is supported via gpupgrade. For more information, see the gpupgrade documentation.
- MADlib, GPText, and PostGIS are not yet provided for installation on Ubuntu systems.
The following table lists key known issues in VMware Tanzu Greenplum 6.x.
Issue | Category | Description |
---|---|---|
N/A | Platform Support | A kernel issue in Red Hat Enterprise Linux 8.5 and 8.6 can cause I/O freezes and synchronization problems with XFS filesystems. This issue is fixed in RHEL 8.7. See RHEL8: xfs_buf deadlock between inode deletion and block allocation. |
31010 | Query Optimizer | A view created in Greenplum Database 5.28.3 or older that specified an external table in the FROM clause, and that was migrated to Greenplum Database 6.x, always falls back to the Postgres Planner when queried.Workaround: If you migrated a view from Greenplum Database 5.28.3 or earlier, and the view specified an external table in the |
N/A | Backup/Restore | Restoring the Greenplum Database backup for a table fails in Greenplum 6 versions earlier than version 6.10 when a replicated table has an inheritance relationship to/from another table that was assigned via an ALTER TABLE … INHERIT statement after table creation.Workaround: Use the following SQL commands to determine if Greenplum Database includes any replicated tables that inherit from a parent table, or if there are replicated tables that are inherited by a child table:
If these queries return any tables, you may choose to run |
N/A | Spark Connector | This version of Greenplum is not compatible with Greenplum-Spark Connector versions earlier than version 1.7.0, due to a change in how Greenplum handles distributed transaction IDs. |
N/A | Greenplum on vSphere | Due to a corruption issue in vSAN 7.0.3., the most recent supported version of vSAN for Greenplum on vSphere is vSAN 7.0.2. |
N/A | PXF | Starting in 6.x, Greenplum does not bundle cURL and instead loads the system-provided library. PXF requires cURL version 7.29.0 or newer. The officially-supported cURL for the CentOS 6.x and Red Hat Enterprise Linux 6.x operating systems is version 7.19.. Greenplum Database 6 does not support running PXF on CentOS 6.x or RHEL 6.x due to this limitation.Workaround: Upgrade the operating system of your Greenplum Database 6 hosts to CentOS 7+ or RHEL 7+, which provides a |
29703 | Loading Data from External Tables | Due to limitations in the Greenplum Database external table framework, Greenplum Database cannot log the following types of errors that it encounters while loading data:
Workaround: Clean the input data before loading it into Greenplum Database. |
30594 | Resource Management | Resource queue-related statistics may be inaccurate in certain cases. VMware recommends that you use the resource group resource management scheme that is available in Greenplum 6. |
30522 | Logging | Greenplum Database may write a FATAL message to the standby master or mirror log stating that the database system is in recovery mode when the instance is synchronizing with the master and Greenplum attempts to contact it before the operation completes. Ignore these messages and use gpstate -f output to determine if the standby successfully synchronized with the Greenplum master; the command returns Sync state: sync if it is synchronized. |
30537 | Postgres Planner | The Postgres Planner generates a very large query plan that causes out of memory issues for the following type of CTE (common table expression) query: the WITH clause of the CTE contains a partitioned table with a large number partitions, and the WITH reference is used in a subquery that joins another partitioned table.Workaround: If possible, use the GPORCA query optimizer. With the server configuration parameter |
170824967 | gpfdists | For Greenplum Database 6.x, a command that accesses an external table that uses the gpfdists protocol fails if the external table does not use an IP address when specifying a host system in the LOCATION clause of the external table definition. |
n/a | Materialized Views | By default, certain gp_toolkit views do not display data for materialized views. If you want to include this information in gp_toolkit view output, you must redefine a gp_toolkit internal view as described in Including Data for Materialized Views. |
168957894 | PXF | The PXF Hive Connector does not support using the Hive profiles to access Hive transactional tables.Workaround: Use the PXF JDBC Connector to access Hive. |
168548176 | gpbackup | When using gpbackup to back up a Greenplum Database 5.7.1 or earlier 5.x release with resource groups enabled, gpbackup returns a column not found error for t6.value AS memoryauditor . |
164791118 | PL/R | PL/R cannot be installed using the deprecated createlang utility, and displays the error:Workaround: Use CREATE EXTENSION to install PL/R, as described in the documentation. |
N/A | Greenplum Client/Load Tools on Windows | The Greenplum Database client and load tools on Windows have not been tested with Active Directory Kerberos authentication. |
N/A | Greenplum Installation | For Greenplum Database 6.16.1 and 6.16.2, the library libevent is not listed as a dependency for the rpm for RHEL 7. Resolved in 6.16.3. |