Visualising XMLs and XSDs of common data standards using Flexter
Extensible Markup Language has become the norm for data transmission and other data related activities in a majority of organizations. XMLs provide developers with a flexible platform to configure and correlate data attributes based upon the product structure and requirement. For different domains, there are various types of XML file formats that are in use. The OTC Derivatives industry uses the FPML XML for its activities. A lot of other industries have custom built XML formats for their usage.
In this walkthrough, we will explain the processing of some of the most frequently used industry standard XML files using Flexter. The industry standard XML files that have been shortlisted for this process are listed below.
Table of Contents
- StratML – Strategic Management XML data
- RIXML – Research Information Markup Language
- OFX – Open Financial Exchange
- NITF – News Industry Text Format
- FpML XML
- FIXML
Processing the industry standard XMLs using Flexter
FpML XML
We have chosen the XML data from FpML for this exercise. FpML (Financial Products Markup Language) is an XML message standard for the OTC Derivatives industry. The standard includes all categories of privately negotiated derivatives. We download the XML data from the FpML website.
We will then use the free version of Flexter to convert the XML data to BigQuery. BigQuery is a distributed database on the Google Cloud that can be used for Business Intelligence use cases. BigQuery does not support XML directly. We use Flexter to first convert our XML data to text (TSV). We can then easily load it into BigQuery.
Processing the FpML XML data using Flexter
In the following section, we describe the processing of the FpML XML files using Flexter
- Access Flexter from https://xmlconverter.sonra.io/
- Click on the ‘Terms and Conditions’ checkbox and click ‘Try Flexter for Free’
- Upload the FpML (Financial products Markup Language) XML file that needs to be transformed and click ‘Continue’.
- On the next page we upload the XSD file (optional) for the XML. In this implementation, we do not use an XSD file. We skip this step and click on ‘Continue’
- On the next page, we provide the e-mail address to which the Flexter output must be sent and click ‘Continue’
- On the next page, click on ‘Finish’ and the output from Flexter will be delivered to the submitted email address.
- An email from Flexter with the link to the transformed data is obtained. The TSV files can be downloaded from this link which redirects to the Flexter page.
- For the XML file from FpML, a total of 14 TSV files were generated and downloaded.
ER Diagram
Download FpML ER Diagram
Flexter generated an entity relationship diagram along with the processed data. The ER diagram establishes the PK-FK relationships between the attributes and defines a schema that can be used in data modelling. The ER diagram that was obtained for the FpML data is shown below.
The Flexter output contains the entity relationship, data lineage and the actual TSV files.
The flexter output for the FpML data is obtained as shown below:
It contains the TSV files and the ER diagram for the XML data.
StratML – Strategic Management XML data
StratML is a standardized XML template and vocabulary for strategic plans, performance plans and performance reports in alignment with policies and standards. This standard will enable the sharing, referencing, indexing, discovery, linking, reuse and analysis of the elements of strategic plans including goal and objective statements. While the initial focus of StratML has been on the US Federal Government, it is believed that private organizations of all sizes will find value in the use of StratML as they draft planning documents.
Processing StartML XML data using Flexter
The Strategic Management XML data was processed using Flexter and the output was obtained as shown below.
ER diagram
Download StratML ER Diagram
RIXML – Research Information Markup Language
RIXML.org is a consortium of buy-side firms, sell-side firms and vendors that have joined together to define an open standard for categorizing, tagging and distributing global investment research. Using the RIXML standard enables firms to improve the process of categorizing, aggregating, comparing, sorting, searching, and distributing global financial research, and of accurately describing interactions to facilitate compliance with MiFID II.
Processing RIXML data using Flexter
The RIXML data was processed using Flexter and the output was obtained as shown below
ER diagram
Download RIXML ER Diagram
OFX – Open Financial Exchange
Open Financial Exchange is an open standard for client-server systems and cloud based APIs for exchanging financial data, and performing financial transactions between financial institutions, and financial applications. OFX has been the dominant direct API for banks to provide data to financial applications since 1997. It is actively deployed at over 7,000 financial institutions, and the remaining institutions have easy access to certified OFX servers via all major technology providers and systems integrators.
Processing OFX XML data using Flexter
The Flexter output for the OFX data was obtained as shown below
ER diagram
Download OFX ER Diagram
NITF – News Industry Text Format
NITF uses the eXtensible Markup Language to define the content and structure of news articles. NITF is an efficient technical standard to improve the management and exchange of information between content providers, intermediaries and consumers.
Processing NITF XML files using Flexter
The NITF XML files were processed using Flexter and the output was obtained as shown below
ER diagram
Download NITF ER Diagram
FIXML
The Financial Information eXchange (FIX) protocol is an electronic communications protocol initiated in 1992 for international real-time exchange of information related to the securities transactions and markets. With trillions of dollars traded annually on the NASDAQ alone, financial service entities are investing heavily in optimizing electronic trading and employing direct market access (DMA) to increase their speed to financial markets. Managing the delivery of trading applications and keeping latency low increasingly requires an understanding of the FIX protocol.
Processing FIXML files using Flexter
The FIXML files were processed using Flexter and the outcome was obtained as shown below
