Flexter completely automates the whole process of converting XML files to a relational format. No custom development is needed. This can save up to 80% of the overall conversion costs. You don’t need to hire external consultants with XML expertise. As one customer put it recently “You did in one day what would have taken us a year”.
Flexter eliminates project risk. We have seen many XML conversion projects fail. The failure rate grows exponentially with the complexity of the XML and data volumes.
We have built scalability into Flexter from the ground up. Flexter can scale up and out across multiple servers. It can handle any volume of data and meet any SLA.
Flexter significantly shortens the duration of XML projects. Developers can focus on data analytics tasks that add value to the business rather than having to wrangle with XML.
The Flexter platform consists of three pluggable modules:
Schema Analyser (xsd2er)
Mapping generator (CalcMap)
Xml Processor (xml2er)
Step 1: The Schema Analyser is a dedicated module that loads, parses out, processes and stores the XML schema information in Flexter's internal metadata DB. This step is only required to be performed once for each schema to be processed. You can either supply an XSD or a representative sample of XML files for this step.
Step 2: Now that we know the exact layout of the source XML it is possible to generate the relational equivalent. Flexter's module, Mapping Generator generates the output schema layout and the mapping to it. Various optimisations of the target schema can be applied during this step.
Step 3: The XML Processor module takes the information generated from the two previous steps, processes the XML, and writes the data to the relational target schema.
The core strength of ETL tools is to transform structured data and work with relational databases. They often struggle with semi-structured data in XML files. While most ETL tools offer functionality to handle simple and flat XML files at low volumes, they have serious limitations:
- They don’t automate the conversion process. ETL developers still need to create data flows (potentially hundreds for complex XMLs) and data pipelines. A significant development effort indeed.
- ETL tools don’t scale beyond a single server for XML processing.
- Most ETL tools can’t handle XML files in batches. They process XML files individually, which has a significant impact on performance
Here are two blog posts where we compare Flexter against two popular ETL tools.
Oracle Data Integrator
Yes. Flexter supports real-time use cases through its streaming engine.
We support both individual XML files and batches of XML files in archives and compressed formats (zip, gzip etc.).
We can pull XML files from network drives, (S)FTP servers, HDFS, S3, XMLTYPE/CLOB in databases etc.
We support most relational databases, e.g. Oracle, MS SQL Server, DB2, PostgreSQL, MySQL, Redshift, Snowflake, BigQuery etc.
We support comma separated and tab separated files as output.
When we generate the target schema we also provide various optional optimisations, e.g. we can influence the level of denormalisation of the target schema and we may optionally eliminate redundant reference data and merge it into one and the same entity.
If you are only using an XSD to generate the target schema all of the possible XPaths will be translated into the target schema. The target schema is more verbose and complex. If the XML files you process conform to your standard then you should not get any warning messages.
However, we often see that XSD designers have been sloppy and do not properly define relationships, cardinality etc. in the XSD. For those scenarios its best to use both the XSD and XML. For gaps and sloppy design in the XSD we override the schema with the stats from the XML sample.
Convert WITSML XML to Excel
Convert FIXML XML to Excel
Convert FHIR XML to Excel
Convert ISO 20022 XML to Text (TSV/CSV) without code
Convert HL7 XML to Text (TSV/CSV) without coding
Convert CRS XML to TSV
Convert ACORD XML to Talend Data Preparation
Convert XML to Trifacta
Convert XML to Excel
Flexter can be used to trickle feed XML files in real time to an analytics engine.
Flexter can be used for data exchange scenarios that require translation of XMLs to a relational format.
Flexter can be used to migrate large volumes of historic XML files to a database.
Flexter can be used to migrate an XML database to a relational database.
Oracle Data Integrator
Informatica Data Services
Free Online Trial
|Max daily data limit||
Cluster of Servers
|File output formats||
Oracle, MS SQL Server, PostgreSQL
Elevate, Re-use, Naming
Elevate, Re-use, Naming
Which data formats apart from XML also give you the heebie jeebies and need to be liberated? Please leave a comment below or reach out to us.