Complete Oracle XSD Guide – Register XSD & Validate XML
In this blog post we cover the most important aspects of working with XML Schema Definitions (XSD) in Oracle.
Here is what you can expect from this article.
- We start with a brief introduction of XML Schema Definition for those new to XML and related concepts.
- While you can work with XML only there are significant benefits of also using an XSD. We will cover those benefits.
- We will walk you through the most important aspects of working with XSD in Oracle. Registering an XSD and validating XML documents against this schema. We will provide step by step examples and sample code to get you started.
- Last but not least we show how you can generate Oracle tables and a relational target schema from an XSD. The Oracle database does not offer a feature to automatically create Oracle tables from an XSD or XML. Instead we use Flexter to generate a relational schema from AIXM XSD, convert some XML to Oracle tables, and query the data with SQL.
Introduction to XML Schema Definition (XSD)
XML Schema Definition (XSD) is an XML schema definition language used to define the structure and data types of XML documents. It is a recommendation by the World Wide Web Consortium (W3C) and is used to describe and validate the structure and content of XML documents.
Other XML schema definition languages
Another schema definition language for XML is DTD. It is considered a legacy language. XSD has replaced DTD.
Relax NG and Schematron are also XML schema defintion languages. Schematron is often used together with XSD for very complex scenarios.
An XSD is similar to a schema in a relational database defined by Data Definition Language (DDL).
Use Flexter to turn XML and JSON into Valuable Insights
- 100% Automation
- 0% Coding
Key Features of XSD
- XSD offers a rich set of built-in data types such as strings, integers, dates, and more, as well as the ability to create custom data types.
- Elements and attributes can be explicitly defined with their types and constraints, ensuring precise structure and content for XML documents.
- XSD supports namespaces, which allows the combination of elements and attributes from different XML vocabularies without conflicts.
- XSD supports deriving new types from existing ones through restriction (narrowing down value sets) and extension (adding new elements or attributes).
- Complex types enable the definition of elements that contain other elements and attributes, allowing for nested XML structures.
- XSD enforces various constraints, such as minimum and maximum occurrences (minOccurs, maxOccurs), value patterns (pattern), and value ranges (minInclusive, maxInclusive), to ensure data validity.
- Elements and types can inherit properties from other elements and types, promoting reuse and reducing redundancy across XML schemas.
Advantages of using XSD in Oracle
Using an XSD inside the Oracle database is optional. It is perfectly fine to just work with XML documents. However, using an XSD together with XML files has significant advantages.
- Validation: It ensures that XML documents adhere to a defined structure and rules, including correct data types, number of elements, and length of data.
- Database Constraints: XML Schema can be applied to XMLType tables or columns in Oracle to restrict data to XML documents that meet specified schema criteria.
- Storage Definition: In Oracle XML DB, XML Schema defines how XML data is stored, whether as binary XML or in an object-relational model, optimizing how XML data types map to SQL types.
- Predictive Use: Without prior details about XML documents, XML Schema helps predict the best storage methods and access protocols, ensuring data integrity and efficient retrieval.
Key aspects of XSD support in Oracle
Oracle has comprehensive support for XSD. It has been a core feature of the Oracle database since Oracle 9i, which was released in 2001. This is a key differentiator to other database engines such as Snowflake, BigQuery or Redshift, which do not have support for XSD. In the case of BigQuery and Redshift we don’t even have support for XML.
In this section we will go through the core concepts of working with XSDs in Oracle:
- Registering an XSD
- Validating an XML document against an XSD
Understanding those two concepts will go a long way of working with XSD and XML in Oracle.
Another set of core skills when working with XML is to understand the various storage, parsing, conversion and query options for XML. We have covered these steps in a separate article Best Way to Load & Convert XML Data to Oracle Tables
In this post we focus on working with XSDs in Oracle.
We will provide a step by step walkthrough of first registering an XSD with Oracle and then validating an XML against the XSD.
We will use the following sample XML and XSD files for the walkthrough.
Sample XML Bookstore
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
<?xml version="1.0" encoding="UTF-8"?> <bookstore> <book> <title>Effective Java</title> <author> <firstName>Joshua</firstName> <lastName>Bloch</lastName> </author> <publisher>Addison-Wesley</publisher> <publicationDate>2018-01-11</publicationDate> <isbn>978-0134685991</isbn> <price>45.00</price> </book> <book> <title>Clean Code</title> <author> <firstName>Robert</firstName> <lastName>Martin</lastName> </author> <publisher>Prentice Hall</publisher> <publicationDate>2008-08-01</publicationDate> <isbn>978-0132350884</isbn> <price>40.00</price> </book> <book> <title>Design Patterns</title> <author> <firstName>Erich</firstName> <lastName>Gamma</lastName> <firstName>Richard</firstName> <lastName>Helm</lastName> <firstName>Ralph</firstName> <lastName>Johnson</lastName> <firstName>John</firstName> <lastName>Vlissides</lastName> </author> <publisher>Addison-Wesley</publisher> <publicationDate>1994-10-31</publicationDate> <isbn>978-0201633610</isbn> <price>54.95</price> </book> </bookstore> |
Sample XSD BookStoreSchema.xsd
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:element name="bookstore"> <xs:complexType> <xs:sequence> <xs:element name="book" maxOccurs="unbounded" minOccurs="0"> <xs:complexType> <xs:sequence> <xs:element type="xs:string" name="title"/> <xs:element name="author"> <xs:complexType> <xs:choice maxOccurs="unbounded" minOccurs="0"> <xs:element type="xs:string" name="firstName"/> <xs:element type="xs:string" name="lastName"/> </xs:choice> </xs:complexType> </xs:element> <xs:element type="xs:string" name="publisher"/> <xs:element type="xs:date" name="publicationDate"/> <xs:element type="xs:string" name="isbn"/> <xs:element type="xs:float" name="price"/> </xs:sequence> </xs:complexType> </xs:element> </xs:sequence> </xs:complexType> </xs:element> </xs:schema> |
There are other things you need to be aware of but these are the two most important steps when working with an XSD in Oracle.
How to register an XSD in Oracle?
When you register an XML Schema Definition (XSD) in Oracle, the database can validate XML data against the defined schema.
The process involves several steps: preparing the XSD file, creating a directory object in Oracle, granting necessary privileges, and using the DBMS_XMLSCHEMA.registerSchema procedure to register the schema. This allows the database to ensure that the XML data adheres to the specified structure and types defined in the XSD, enhancing data integrity.
Step 1: Create Directory object
1 2 |
CREATE DIRECTORY 'XML_DIR' AS '/home/oracle/Downloads/XML_DIR/'; GRANT READ, WRITE ON DIRECTORY XML_DIR TO MyUser; |
Step 2: Register the BookStoreSchema.xsd XSD file
1 2 3 4 5 6 7 8 |
BEGIN DBMS_XMLSCHEMA.registerSchema( SCHEMAURL=> 'http://www.example.com/BookStoreSchema.xsd', SCHEMADOC=> bfilename('XML_DIR','BookStoreSchema.xsd'), LOCAL=> TRUE, GENTYPES=> FALSE, GENTABLES=> FALSE); END; |
SCHEMAURL The schemaURL parameter is a string that uniquely identifies the XML schema being registered. It acts as a logical reference or identifier for the schema within the Oracle database. The schemaURL is not necessarily a URL pointing to an actual location on the web but rather serves as a unique name or identifier for the XML schema within the database.
Key Points about schemaURL
- Unique Identifier: The schemaURL must be unique within the database. It is used by the database to refer to the schema.
- Logical Reference: It is a logical name that can be used to reference the schema in subsequent operations, such as validation, updates, or deletions.
- Naming Convention: While it looks like a URL, it does not need to resolve to an actual resource. It’s a convention to use URL-like strings to ensure global uniqueness, especially in systems where schemas might be exchanged or shared.
GEN_TYPES => TRUE: When this option is set to TRUE, Oracle generates SQL object types for each complex type defined in the XML schema.
GEN_TABLES => TRUE: Setting this option to TRUE instructs Oracle to generate default tables for each root element defined in the XML schema. Create an XMLType Table using the registered XML Schema.
When registering an XML Schema in Oracle with the GEN_TYPES and GEN_TABLES options set to TRUE, Oracle automatically generates SQL object types for each complex type within the schema and creates default XMLType tables for each root element. You can insert XML documents into these XMLType tables and query them to view the XML documents. These tables feature one XMLType column with a system-generated name.
However, you cannot directly query the tables for the complex types like normal tables in Oracle. Instead, these are SYS_NTS (Oracle Native Type Manager) objects, which are internal constructs used by Oracle to manage the storage and retrieval of XML data, including XML schemas and their components. These objects are not intended to be accessed directly through standard SQL queries.
Create a database table using the registered XML schema.
When setting GEN_TABLES to TRUE Oracle creates tables for us (this is the default behavior if not specified)
It creates table with system generated names like ‘bookstore503_TAB’ or ‘Customer531_TAB’.
We set GEN_TABLES to FALSE and create our own table to store the XML. This gives us more control, e.g. we can give the table a name rather than relying on a system generated table.
1 2 3 4 5 |
CREATE TABLE BookStore ( bookstore_data XMLType ) XMLTYPE COLUMN bookstore_data XMLSCHEMA "http://www.example.com/BookStoreSchema.xsd" ELEMENT "bookstore"; |
How to validate an XML against an XSD in Oracle
Verify XML against XSD
XMLSCHEMA.SCHEMAVALIDATE is an Oracle function used for validating XML documents against XML schemas. It’s used to check if an XML document conforms to the rules defined in an XML schema.
For a Valid XML Document :
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
DECLARE l_xml CLOB; l_xmltype XMLTYPE; is_valid boolean; BEGIN l_xml := '<?xml version="1.0" encoding="UTF-8"?> <bookstore> <book> <title>Effective Java</title> <author> <firstName>Joshua</firstName> <lastName>Bloch</lastName> </author> <publisher>Addison-Wesley</publisher> <publicationDate>2018-01-11</publicationDate> <isbn>978-0134685991</isbn> <price>45.00</price> </book> </bookstore>'; l_xmltype := XMLTYPE(l_xml, 'http://www.example.com/BookStoreSchema.xsd'); l_xmltype.schemavalidate; END; |
PL/SQL procedure successfully completed.
For an invalid XML Document that violates the constraint around multiple book titles.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
DECLARE l_xml CLOB; l_xmltype XMLTYPE; is_valid boolean; BEGIN l_xml := '<?xml version="1.0" encoding="UTF-8"?> <bookstore> <book> <!-- Violation: More than one <title> element --> <title>Clean Code</title> <title>Another Title</title> <!-- Extra title --> <author> <firstName>Robert</firstName> <lastName>Martin</lastName> </author> <publisher>Prentice Hall</publisher> <publicationDate>2008-08-01</publicationDate> <isbn>978-0132350884</isbn> <price>40.00</price> </book> </bookstore>'; l_xmltype := XMLTYPE(l_xml, 'http://www.example.com/BookStoreSchema.xsd'); l_xmltype.schemavalidate; END; |
Oracle throws an error
Validate XML against XSD when inserting data
When you insert XML data into the table, Oracle automatically validates it against the registered schema.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
INSERT INTO BookStore VALUES (XMLType('<?xml version="1.0" encoding="UTF-8"?> <bookstore> <book> <title>Effective Java</title> <author> <firstName>Joshua</firstName> <lastName>Bloch</lastName> </author> <publisher>Addison-Wesley</publisher> <publicationDate>2018-01-11</publicationDate> <isbn>978-0134685991</isbn> <price>45.00</price> </book> </bookstore>')); |
Query the ‘BookStore’ table to view the inserted xml data.
When you attempt to insert XML data that does not conform to the constraints defined by the XSD, Oracle will raise an error.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
INSERT INTO BookStore VALUES (XMLType('<?xml version="1.0" encoding="UTF-8"?> <bookstore> <book> <!-- Violation: More than one <title> element --> <title>Clean Code</title> <title>Another Title</title> <!-- Extra title --> <author> <firstName>Robert</firstName> <lastName>Martin</lastName> </author> <publisher>Prentice Hall</publisher> <publicationDate>2008-08-01</publicationDate> <isbn>978-0132350884</isbn> <price>40.00</price> </book> </bookstore>')); |
How to create Oracle Tables from XSD
When registering an XSD inside Oracle, the database creates various system tables and types in the background. These tables are helper objects used internally by Oracle.
Registering an XSD does not create relational tables in Oracle that can be queried using SQL by an end user. If you are looking to create Oracle tables from an XSD you need to take a different approach.
In this section we show how you can convert an XSD to a relational Oracle database schema and create Oracle tables from your XSD. You will require an external tool such as Flexter.
It is a simple two step process
First we create a Data Flow. The Data Flow will generate a relational schema from your XSD and if available sample XML documents.
Next we convert the data.
Step 1: Create a Data Flow
Register your XSD with Flexter to generate a relational target schema of your XSD. Optionally you can also provide Flexter with a sample of XML files. Sometimes this can be useful for very large schemas where you just use a particular section of the schema.
Step 2: Convert XML to Oracle tables
In the second step we can then convert XML on a regular schedule to the Oracle tables generated in the first step. It couldn’t get any easier.
Example of creating tables from AIXM XSD in Oracle
In this example we show how you can use Flexter to convert XML based on a complex XSD to Oracle tables.
We use some fictitious data set based on AIXM. AIXM is an industry data standard in the aviation industry. Industry data standards tend to be very complex as they need to cover many different types of business processes in a flexible way.
You can download the data set we use for this walkthrough from the AIXM Github repository.
You can download the XSD from the AIXM website.
In the first step we create a Flexter Data Flow. We use both the information in the XSD and the XML samples to create a relational target schema on Oracle.
In this example we use Flexter’s command line utility but you can also call a REST API.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
xml2er --mem 16g AIXM_XML.zip # schema origin: 851 job: 12615 xsd2er -a 851 -g3 AIXM_XSD.zip --stop-policy 0+1 # schema schema: 853 logical: 361 job: 12619 |
This results in the following target schema for Oracle
As you can see the schema is massive with almost 20K columns. This is despite the fact that Flexter also applies various optimisation algorithms to simplify the schema.
Flexter also generates a source to target map for data lineage for all of the XPaths in the XML documents. Here is an extract from the source to target map
As you can see for each XML element and XPath we get a corresponding table and column name.
Now that we have generated our target schema it is time for some XML processing.
We take the same sample of XML files to convert our AIXM sample data to Oracle. It’s one line of code on the command line. Flexter also ships with a RestAPI for programmatic access.
1 |
xml2er --mem 16G -x853 AIXM_XML.zip -u oracle_user -p 'pass' -o "jdbc:oracle:thin:@abc.io:1521:YY" |
And that’s it. All that is left is to query the data using our ER diagram and Oracle SQL
1 2 3 4 5 6 7 |
SELECT am.filepath ,hm.airspace_id ,hm.airspace_identifier FROM ADRMessage am JOIN hasMember hm ON hm.FK_ADRMessage=am.PK_ADRMessage WHERE Airspace_id IS NOT NULL |
You can use the free online version of Flexter and try convert XML + XSD to a relational format. Alternatively reach out to us and book a demo of Flexter.
Further reading
Oracle XML & XSD
Oracle documentation on using XSD in Oracle database
Best Way to Load & Convert XML Data to Oracle Tables
XSD conversion to relational database
Converting XSD to database schema: your guide to creating relational models from XML
Optimisation algorithms for converting XML and JSON to a relational format
Flexter
XML conversion guides
XML Converters & Conversion – Ultimate Guide (2024)
Converting XML and JSON to a Data Lake