Working with XML Data

Greenplum Database supports the xml data type that stores XML data.

The xml data type checks the input values for well-formedness, providing an advantage over simply storing XML data in a text field. Additionally, support functions allow you to perform type-safe operations on this data; refer to XML Function Reference, below.

Use of this data type requires the installation to have been built with configure —with-libxml.

The xml type can store well-formed “documents”, as defined by the XML standard, as well as “content” fragments, which are defined by reference to the more permissive document node of the XQuery and XPath model. Roughly, this means that content fragments can have more than one top-level element or character node. The expression xmlvalue IS DOCUMENT can be used to evaluate whether a particular xml value is a full document or only a content fragment.

This section contains the following topics:

Parent topic: Querying Data

Creating XML Values

To produce a value of type xml from character data, use the function xmlparse:

  1. xmlparse ( { DOCUMENT | CONTENT } value)

For example:

  1. XMLPARSE (DOCUMENT '<?xml version="1.0"?><book><title>Manual</title><chapter>...</chapter></book>')
  2. XMLPARSE (CONTENT 'abc<foo>bar</foo><bar>foo</bar>')

The above method converts character strings into XML values according to the SQL standard, but you can also use Greenplum Database syntax like the following:

  1. xml '<foo>bar</foo>'
  2. '<foo>bar</foo>'::xml

The xml type does not validate input values against a document type declaration (DTD), even when the input value specifies a DTD. There is also currently no built-in support for validating against other XML schema languages such as XML schema.

The inverse operation, producing a character string value from xml, uses the function xmlserialize:

  1. xmlserialize ( { DOCUMENT | CONTENT } value AS type )

type can be character, character varying, or text (or an alias for one of those). Again, according to the SQL standard, this is the only way to convert between type xml and character types, but Greenplum Database also allows you to simply cast the value.

When a character string value is cast to or from type xml without going through XMLPARSE or XMLSERIALIZE, respectively, the choice of DOCUMENT versus CONTENT is determined by the XML OPTION session configuration parameter, which can be set using the standard command:

  1. SET XML OPTION { DOCUMENT | CONTENT };

or simply like Greenplum Database:

  1. SET XML OPTION TO { DOCUMENT | CONTENT };

The default is CONTENT, so all forms of XML data are allowed.

Encoding Handling

Be careful when dealing with multiple character encodings on the client, server, and in the XML data passed through them. When using the text mode to pass queries to the server and query results to the client (which is the normal mode), Greenplum Database converts all character data passed between the client and the server, and vice versa, to the character encoding of the respective endpoint; see Character Set Support. This includes string representations of XML values, such as in the above examples. Ordinarily, this means that encoding declarations contained in XML data can become invalid, as the character data is converted to other encodings while travelling between client and server, because the embedded encoding declaration is not changed. To cope with this behavior, encoding declarations contained in character strings presented for input to the xml type are ignored, and content is assumed to be in the current server encoding. Consequently, for correct processing, character strings of XML data must be sent from the client in the current client encoding. It is the responsibility of the client to either convert documents to the current client encoding before sending them to the server, or to adjust the client encoding appropriately. On output, values of type xml will not have an encoding declaration, and clients should assume all data is in the current client encoding.

When using binary mode to pass query parameters to the server and query results back to the client, no character set conversion is performed, so the situation is different. In this case, an encoding declaration in the XML data will be observed, and if it is absent, the data will be assumed to be in UTF-8 (as required by the XML standard; note that Greenplum Database does not support UTF-16). On output, data will have an encoding declaration specifying the client encoding, unless the client encoding is UTF-8, in which case it will be omitted.

Note:

Processing XML data with Greenplum Database will be less error-prone and more efficient if the XML data encoding, client encoding, and server encoding are the same. Because XML data is internally processed in UTF-8, computations will be most efficient if the server encoding is also UTF-8.

Accessing XML Values

The xml data type is unusual in that it does not provide any comparison operators. This is because there is no well-defined and universally useful comparison algorithm for XML data. One consequence of this is that you cannot retrieve rows by comparing an xml column against a search value. XML values should therefore typically be accompanied by a separate key field such as an ID. An alternative solution for comparing XML values is to convert them to character strings first, but note that character string comparison has little to do with a useful XML comparison method.

Because there are no comparison operators for the xml data type, it is not possible to create an index directly on a column of this type. If speedy searches in XML data are desired, possible workarounds include casting the expression to a character string type and indexing that, or indexing an XPath expression. Of course, the actual query would have to be adjusted to search by the indexed expression.

Processing XML

To process values of data type xml, Greenplum Database offers the functions xpath and xpath_exists, which evaluate XPath 1.0 expressions.

  1. xpath(xpath, xml [, nsarray])

The function xpath evaluates the XPath expression xpath (a text value) against the XML value xml. It returns an array of XML values corresponding to the node set produced by the XPath expression.

The second argument must be a well formed XML document. In particular, it must have a single root node element.

The optional third argument of the function is an array of namespace mappings. This array should be a two-dimensional text array with the length of the second axis being equal to 2 (i.e., it should be an array of arrays, each of which consists of exactly 2 elements). The first element of each array entry is the namespace name (alias), the second the namespace URI. It is not required that aliases provided in this array be the same as those being used in the XML document itself (in other words, both in the XML document and in the xpath function context, aliases are local).

Example:

  1. SELECT xpath('/my:a/text()', '<my:a xmlns:my="http://example.com">test</my:a>',
  2.              ARRAY[ARRAY['my', 'http://example.com']]);
  4.  xpath  
  5. --------
  6.  {test}
  7. (1 row)

To deal with default (anonymous) namespaces, do something like this:

  1. SELECT xpath('//mydefns:b/text()', '<a xmlns="http://example.com"><b>test</b></a>',
  2.              ARRAY[ARRAY['mydefns', 'http://example.com']]);
  4.  xpath
  5. --------
  6.  {test}
  7. (1 row)
  1. xpath_exists(xpath, xml [, nsarray])

The function xpath_exists is a specialized form of the xpath function. Instead of returning the individual XML values that satisfy the XPath, this function returns a Boolean indicating whether the query was satisfied or not. This function is equivalent to the standard XMLEXISTS predicate, except that it also offers support for a namespace mapping argument.

Example:

  1. SELECT xpath_exists('/my:a/text()', '<my:a xmlns:my="http://example.com">test</my:a>',
  2.                      ARRAY[ARRAY['my', 'http://example.com']]);
  4.  xpath_exists  
  5. --------------
  6.  t
  7. (1 row)

Mapping Tables to XML

The following functions map the contents of relational tables to XML values. They can be thought of as XML export functionality:

  1. table_to_xml(tbl regclass, nulls boolean, tableforest boolean, targetns text)
  2. query_to_xml(query text, nulls boolean, tableforest boolean, targetns text)
  3. cursor_to_xml(cursor refcursor, count int, nulls boolean,
  4.               tableforest boolean, targetns text)

The return type of each function is xml.

table_to_xml maps the content of the named table, passed as parameter tbl. The regclass type accepts strings identifying tables using the usual notation, including optional schema qualifications and double quotes. query_to_xml executes the query whose text is passed as parameter query and maps the result set. cursor_to_xml fetches the indicated number of rows from the cursor specified by the parameter cursor. This variant is recommended if large tables have to be mapped, because the result value is built up in memory by each function.

If tableforest is false, then the resulting XML document looks like this:

  1. <tablename>
  2.   <row>
  3.     <columnname1>data</columnname1>
  4.     <columnname2>data</columnname2>
  5.   </row>
  7.   <row>
  8.     ...
  9.   </row>
  11.   ...
  12. </tablename>

If tableforest is true, the result is an XML content fragment that looks like this:

  1. <tablename>
  2.   <columnname1>data</columnname1>
  3.   <columnname2>data</columnname2>
  4. </tablename>
  6. <tablename>
  7.   ...
  8. </tablename>
  10. ...

If no table name is available, that is, when mapping a query or a cursor, the string table is used in the first format, row in the second format.

The choice between these formats is up to the user. The first format is a proper XML document, which will be important in many applications. The second format tends to be more useful in the cursor_to_xml function if the result values are to be later reassembled into one document. The functions for producing XML content discussed above, in particular xmlelement, can be used to alter the results as desired.

The data values are mapped in the same way as described for the function xmlelement, above.

The parameter nulls determines whether null values should be included in the output. If true, null values in columns are represented as:

  1. <columnname xsi:nil="true"/>

where xsi is the XML namespace prefix for XML schema Instance. An appropriate namespace declaration will be added to the result value. If false, columns containing null values are simply omitted from the output.

The parameter targetns specifies the desired XML namespace of the result. If no particular namespace is wanted, an empty string should be passed.

The following functions return XML schema documents describing the mappings performed by the corresponding functions above:

  1. able_to_xmlschema(tbl regclass, nulls boolean, tableforest boolean, targetns text)
  2. query_to_xmlschema(query text, nulls boolean, tableforest boolean, targetns text)
  3. cursor_to_xmlschema(cursor refcursor, nulls boolean, tableforest boolean, targetns text)

It is essential that the same parameters are passed in order to obtain matching XML data mappings and XML schema documents.

The following functions produce XML data mappings and the corresponding XML schema in one document (or forest), linked together. They can be useful where self-contained and self-describing results are desired:

  1. table_to_xml_and_xmlschema(tbl regclass, nulls boolean, tableforest boolean, targetns text)
  2. query_to_xml_and_xmlschema(query text, nulls boolean, tableforest boolean, targetns text)

In addition, the following functions are available to produce analogous mappings of entire schemas or the entire current database:

  1. schema_to_xml(schema name, nulls boolean, tableforest boolean, targetns text)
  2. schema_to_xmlschema(schema name, nulls boolean, tableforest boolean, targetns text)
  3. schema_to_xml_and_xmlschema(schema name, nulls boolean, tableforest boolean, targetns text)
  5. database_to_xml(nulls boolean, tableforest boolean, targetns text)
  6. database_to_xmlschema(nulls boolean, tableforest boolean, targetns text)
  7. database_to_xml_and_xmlschema(nulls boolean, tableforest boolean, targetns text)

Note that these potentially produce large amounts of data, which needs to be built up in memory. When requesting content mappings of large schemas or databases, consider mapping the tables separately instead, possibly even through a cursor.

The result of a schema content mapping looks like this:

  1. <schemaname>
  3. table1-mapping
  5. table2-mapping
  7. ...
  9. </schemaname>

where the format of a table mapping depends on the tableforest parameter, as explained above.

The result of a database content mapping looks like this:

  1. <dbname>
  3. <schema1name>
  4.   ...
  5. </schema1name>
  7. <schema2name>
  8.   ...
  9. </schema2name>
  11. ...
  13. </dbname>

where the schema mapping is as above.

The example below demonstrates using the output produced by these functions, The example shows an XSLT stylesheet that converts the output of table_to_xml_and_xmlschema to an HTML document containing a tabular rendition of the table data. In a similar manner, the results from these functions can be converted into other XML-based formats.

  1. <?xml version="1.0"?>
  2. <xsl:stylesheet version="1.0"
  3.     xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  4.     xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  5.     xmlns="http://www.w3.org/1999/xhtml"
  6. >
  8.   <xsl:output method="xml"
  9.       doctype-system="http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"
  10.       doctype-public="-//W3C/DTD XHTML 1.0 Strict//EN"
  11.       indent="yes"/>
  13.   <xsl:template match="/*">
  14.     <xsl:variable name="schema" select="//xsd:schema"/>
  15.     <xsl:variable name="tabletypename"
  16.                   select="$schema/xsd:element[@name=name(current())]/@type"/>
  17.     <xsl:variable name="rowtypename"
  18.                   select="$schema/xsd:complexType[@name=$tabletypename]/xsd:sequence/xsd:element[@name='row']/@type"/>
  20.     <html>
  21.       <head>
  22.         <title><xsl:value-of select="name(current())"/></title>
  23.       </head>
  24.       <body>
  25.         <table>
  26.           <tr>
  27.             <xsl:for-each select="$schema/xsd:complexType[@name=$rowtypename]/xsd:sequence/xsd:element/@name">
  28.               <th><xsl:value-of select="."/></th>
  29.             </xsl:for-each>
  30.           </tr>
  32.           <xsl:for-each select="row">
  33.             <tr>
  34.               <xsl:for-each select="*">
  35.                 <td><xsl:value-of select="."/></td>
  36.               </xsl:for-each>
  37.             </tr>
  38.           </xsl:for-each>
  39.         </table>
  40.       </body>
  41.     </html>
  42.   </xsl:template>
  44. </xsl:stylesheet>

XML Function Reference

The functions described in this section operate on values of type xml. The section XML Predicatesalso contains information about the xml functions and function-like expressions.

Function:

xmlcomment

Synopsis:

  1. xmlcomment(text)

The function xmlcomment creates an XML value containing an XML comment with the specified text as content. The text cannot contain “—“ or end with a “-“ so that the resulting construct is a valid XML comment. If the argument is null, the result is null.

Example:

  1. SELECT xmlcomment('hello');
  3.   xmlcomment
  4. --------------
  5.  <!--hello-->

Function:

xmlconcat

Synopsis:

  1. xmlconcat(xml[, …])

The function xmlconcat concatenates a list of individual XML values to create a single value containing an XML content fragment. Null values are omitted; the result is only null if there are no nonnull arguments.

Example:

  1. SELECT xmlconcat('<abc/>', '<bar>foo</bar>');
  3.       xmlconcat
  4. ----------------------
  5.  <abc/><bar>foo</bar>

XML declarations, if present, are combined as follows:

  • If all argument values have the same XML version declaration, that version is used in the result, else no version is used.
  • If all argument values have the standalone declaration value “yes”, then that value is used in the result.
  • If all argument values have a standalone declaration value and at least one is “no”, then that is used in the result. Otherwise, the result will have no standalone declaration.
  • If the result is determined to require a standalone declaration but no version declaration, a version declaration with version 1.0 will be used because XML requires an XML declaration to contain a version declaration.

Encoding declarations are ignored and removed in all cases.

Example:

  1. SELECT xmlconcat('<?xml version="1.1"?><foo/>', '<?xml version="1.1" standalone="no"?><bar/>');
  3.              xmlconcat
  4. -----------------------------------
  5.  <?xml version="1.1"?><foo/><bar/>

Function:

xmlelement

Synopsis:

  1. xmlelement(name name [, xmlattributes(value [AS attname] [, ... ])] [, content, ...])

The xmlelement expression produces an XML element with the given name, attributes, and content.

Examples:

  1. SELECT xmlelement(name foo);
  3.  xmlelement
  4. ------------
  5.  <foo/>
  7. SELECT xmlelement(name foo, xmlattributes('xyz' as bar));
  9.     xmlelement
  10. ------------------
  11.  <foo bar="xyz"/>
  13. SELECT xmlelement(name foo, xmlattributes(current_date as bar), 'cont', 'ent');
  15.              xmlelement
  16. -------------------------------------
  17.  <foo bar="2017-01-26">content</foo>

Element and attribute names that are not valid XML names are escaped by replacing the offending characters by the sequence _xHHHH_, where HHHH is the character’s Unicode codepoint in hexadecimal notation. For example:

  1. SELECT xmlelement(name "foo$bar", xmlattributes('xyz' as "a&b"));
  3.             xmlelement
  4. ----------------------------------
  5.  <foo_x0024_bar a_x0026_b="xyz"/>

An explicit attribute name need not be specified if the attribute value is a column reference, in which case the column’s name will be used as the attribute name by default. In other cases, the attribute must be given an explicit name. So this example is valid:

  1. CREATE TABLE test (a xml, b xml);
  2. SELECT xmlelement(name test, xmlattributes(a, b)) FROM test;

But these are not:

  1. SELECT xmlelement(name test, xmlattributes('constant'), a, b) FROM test;
  2. SELECT xmlelement(name test, xmlattributes(func(a, b))) FROM test;

Element content, if specified, will be formatted according to its data type. If the content is itself of type xml, complex XML documents can be constructed. For example:

  1. SELECT xmlelement(name foo, xmlattributes('xyz' as bar),
  2.                             xmlelement(name abc),
  3.                             xmlcomment('test'),
  4.                             xmlelement(name xyz));
  6.                   xmlelement
  7. ----------------------------------------------
  8.  <foo bar="xyz"><abc/><!--test--><xyz/></foo>

Content of other types will be formatted into valid XML character data. This means in particular that the characters <, >, and & will be converted to entities. Binary data (data type bytea) will be represented in base64 or hex encoding, depending on the setting of the configuration parameter xmlbinary. The particular behavior for individual data types is expected to evolve in order to align the SQL and Greenplum Database data types with the XML schema specification, at which point a more precise description will appear.

Function:

xmlforest

Synopsis:

  1. xmlforest(content [AS name] [, ...])

The xmlforest expression produces an XML forest (sequence) of elements using the given names and content.

Examples:

  1. SELECT xmlforest('abc' AS foo, 123 AS bar);
  3.           xmlforest
  4. ------------------------------
  5.  <foo>abc</foo><bar>123</bar>
  8. SELECT xmlforest(table_name, column_name)
  9. FROM information_schema.columns
  10. WHERE table_schema = 'pg_catalog';
  12.                                          xmlforest
  13. -------------------------------------------------------------------------------------------
  14.  <table_name>pg_authid</table_name><column_name>rolname</column_name>
  15.  <table_name>pg_authid</table_name><column_name>rolsuper</column_name>

As seen in the second example, the element name can be omitted if the content value is a column reference, in which case the column name is used by default. Otherwise, a name must be specified.

Element names that are not valid XML names are escaped as shown for xmlelement above. Similarly, content data is escaped to make valid XML content, unless it is already of type xml.

Note that XML forests are not valid XML documents if they consist of more than one element, so it might be useful to wrap xmlforest expressions in xmlelement.

Function:

xmlpi

Synopsis:

  1. xmlpi(name target [, content])

The xmlpi expression creates an XML processing instruction. The content, if present, must not contain the character sequence ?>.

Example:

  1. SELECT xmlpi(name php, 'echo "hello world";');
  3.             xmlpi
  4. -----------------------------
  5.  <?php echo "hello world";?>

Function:

xmlroot

Synopsis:

  1. xmlroot(xml, version text | no value [, standalone yes|no|no value])

The xmlroot expression alters the properties of the root node of an XML value. If a version is specified, it replaces the value in the root node’s version declaration; if a standalone setting is specified, it replaces the value in the root node’s standalone declaration.

  1. SELECT xmlroot(xmlparse(document '<?xml version="1.1"?><content>abc</content>'),
  2.                version '1.0', standalone yes);
  4.                 xmlroot
  5. ----------------------------------------
  6.  <?xml version="1.0" standalone="yes"?>
  7.  <content>abc</content>

Function:

xmlagg

  1. xmlagg (xml)

The function xmlagg is, unlike the other functions described here, an aggregate function. It concatenates the input values to the aggregate function call, much like xmlconcat does, except that concatenation occurs across rows rather than across expressions in a single row. See Using Functions and Operators for additional information about aggregate functions.

Example:

  1. CREATE TABLE test (y int, x xml);
  2. INSERT INTO test VALUES (1, '<foo>abc</foo>');
  3. INSERT INTO test VALUES (2, '<bar/>');
  4. SELECT xmlagg(x) FROM test;
  5.         xmlagg
  6. ----------------------
  7.  <foo>abc</foo><bar/>

To determine the order of the concatenation, an ORDER BY clause may be added to the aggregate call. For example:

  1. SELECT xmlagg(x ORDER BY y DESC) FROM test;
  2.         xmlagg
  3. ----------------------
  4.  <bar/><foo>abc</foo>

The following non-standard approach used to be recommended in previous versions, and may still be useful in specific cases:

  1. SELECT xmlagg(x) FROM (SELECT * FROM test ORDER BY y DESC) AS tab;
  2.         xmlagg
  3. ----------------------
  4.  <bar/><foo>abc</foo>

XML Predicates

The expressions described in this section check properties of xml values.

Expression:

IS DOCUMENT

Synopsis:

  1. xml IS DOCUMENT

The expression IS DOCUMENT returns true if the argument XML value is a proper XML document, false if it is not (that is, it is a content fragment), or null if the argument is null.

Expression:

XMLEXISTS

Synopsis:

  1. XMLEXISTS(text PASSING [BY REF] xml [BY REF])

The function xmlexists returns true if the XPath expression in the first argument returns any nodes, and false otherwise. (If either argument is null, the result is null.)

Example:

  1. SELECT xmlexists('//town[text() = ''Toronto'']' PASSING BY REF '<towns><town>Toronto</town><town>Ottawa</town></towns>');
  3.  xmlexists
  4. ------------
  5.  t
  6. (1 row)

The BY REF clauses have no effect in Greenplum Database, but are allowed for SQL conformance and compatibility with other implementations. Per SQL standard, the first BY REF is required, the second is optional. Also note that the SQL standard specifies the xmlexists construct to take an XQuery expression as first argument, but Greenplum Database currently only supports XPath, which is a subset of XQuery.

Expression:

xml_is_well_formed

Synopsis:

  1. xml_is_well_formed(text)
  2. xml_is_well_formed_document(text)
  3. xml_is_well_formed_content(text)

These functions check whether a text string is well-formed XML, returning a Boolean result. xml_is_well_formed_document checks for a well-formed document, while xml_is_well_formed_content checks for well-formed content. xml_is_well_formed does the former if the xmloption configuration parameter is set to DOCUMENT, or the latter if it is set to CONTENT. This means that xml_is_well_formed is useful for seeing whether a simple cast to type xml will succeed, whereas the other two functions are useful for seeing whether the corresponding variants of XMLPARSE will succeed.

Examples:

  1. SET xmloption TO DOCUMENT;
  2. SELECT xml_is_well_formed('<>');
  3.  xml_is_well_formed 
  4. --------------------
  5.  f
  6. (1 row)
  8. SELECT xml_is_well_formed('<abc/>');
  9.  xml_is_well_formed 
  10. --------------------
  11.  t
  12. (1 row)
  14. SET xmloption TO CONTENT;
  15. SELECT xml_is_well_formed('abc');
  16.  xml_is_well_formed 
  17. --------------------
  18.  t
  19. (1 row)
  21. SELECT xml_is_well_formed_document('<pg:foo xmlns:pg="http://postgresql.org/stuff">bar</pg:foo>');
  22.  xml_is_well_formed_document 
  23. -----------------------------
  24.  t
  25. (1 row)
  27. SELECT xml_is_well_formed_document('<pg:foo xmlns:pg="http://postgresql.org/stuff">bar</my:foo>');
  28.  xml_is_well_formed_document 
  29. -----------------------------
  30.  f
  31. (1 row)

The last example shows that the checks include whether namespaces are correctly matched.

Need help? Visit the Greenplum Database Community