Exporting to Axon : Key Observations
Case Study : Moving metadata from FSA to Informatica Axon
Integrating metadata between different data tools is an exercise that can be quite a challenge. Meta models and approaches can differ significantly...this short article describes the process and key observations for a real life scenario...we welcome comments and feedback if you have similar experiences.
Firstly, this was quite a large, diverse, but fully integral metadata set for a single line of business covering10 Applications, 7 inter-related end to end data flow processes, 15 information domains, 17 technical integrations...close to 4000 attribute mappings. This was by far the biggest interconnected metadata schematic this particular Axon installation has had to cope with...FSA was connecting up most areas of the Axon Meta Model for the first time.
Of course, extracting source metadata from another similar repository has some natural advantages over hand cranked spreadsheets...all the 40 or so bulk import Excel spreadsheets generated from FSA in this case came with a guarantee of integrity and consistency and also a common approach to how those Axon metadata facets interconnect...so most of the hard work was already done in FSA...there were no loose ends and typos to worry about.
Overall, this produced a good match between the two repositories, just a few more slightly more granular FSA facet classifications and types needed to be manually added to the lists of Axon lookups to allow the imports to proceed.
So let's get down to brass tacks, so to speak...
The Axon 6.2 bulk import Process
For this transfer, excel spreadsheets were extracted from FSA into Axon standard template format. The order was key, adding key facet data first, sometimes with both facets and rows in specific order (tree walk for hierarchies)...then the relationships between them were done.
Any new classifications were manually created in Axon in advance to reduce import errors. There didn't seem to be a mechanism to extract and upload this key lookup data through the bulk upload interface...or flick a switch to add them on the fly that we were aware of. The slightly clunky Axon lookup administration UI meant that this was probably the task that took the longest in the overall process.
The import error identification reports were pretty good, identifying line numbers and having verbose and helpful error message, allowing extracts to be rebuild and resubmitted quickly...so the actual import time for all 40 or so import spreadsheets was in the region of 12 development hours...considering that there was some learning as we progressed on this first attempt extract by extract, that's a pretty good timescale for such a large metadata set...re-running the whole process could be done in less than half a working day.
This was a one-off transfer on a completed and stable schematic, so we can't comment on how this would work merging forward changes at scale. We did notice that some Axon Data Sets were repeated after the process was completed, because the System referenced was different.
Meta Model Challenges
The difference in focus between FSA and Axon revealed a few Meta Model differences and also some methodology challenges.
Taxonomy and Classification
FSA defines many hierarchical taxonomies (Application Structure Trees, Feature Trees, Resource Portfolio Trees etc)...these have both user definable levels and node classifications. Axon has a facet based approach, some but not all facets support parents (Data Set is the notable exception) and has no concept of taxonomy level.
Some of the exports, therefore, had to format and provide full path, level and sometimes class in the name and parent name for the facet record. Obviously, it is better to not have to encoded names, so this was unfortunate.
Granularity and Logical Reuse
FSA places much emphasis on disambiguation of business language, concepts and structural reusability. Many Axon Data Set Attributes, in the FSA method are, in fact, deemed as too complex to be atomic and are defined as named reusable groupings of atomics...this means an Axon Data Set Attribute could also be an Axon Data Set.
One observation we made is that the Axon Meta Model does not deal with this well, very possibly because it has an underlying bias to the relational mindset...Data Set maps to Table well, Attribute to Column also maps well...anything outside this rather simplistic view of the data world, even XML or other forms of nested documents...may struggle to define, map and visualise properly in Axon from what we experienced.
FSA Reuse Groups, therefore, also suffer this same problem for the same reason...being logical data subsets used in many logical data sets to promote reuse.
领英推荐
Multiple Languages
FSA can support all its meta model simultaneously across many natural languages...we could not see how Axon could support this or provide the ability for a user to swap to their preferred natural language. This could be a limitation to use for truly global enterprises.
Separation of Concerns
FSA takes great care to separate roles and business/architectural disciplines in what they can add and see. For example, there may be 1000+ applications in a large enterprise, but in FSA each individual user only sees the ones that apply to them through their resource portfolio assignments and application tree linkage.
Axon appears to be weak in this area generally, some of the fuzzy logic that Axon uses to show connections also compounds this problem by showing indirect connections without clearly marking them as such.
Therefore, we found that the user is somewhat bombarded with everything and not always things directly related to the search at hand...as the amount of meta data increases...not being able to "see the wood for the trees" would start to be a problem.
It must be stated though, that during this process, we did not seek or consider ways that Axon could do this...so if there are any Axon experts out there, comment and tell us how...because not everybody out there trying to use Axon starts as an expert in the tool!
Technology Diversity
FSA Data Landscape Management provides the ability to define data sets of many types (UIs, Tables, Payloads etc) across many technology syntaxes (HTML, Java, SQL, XML, JSON, CSV etc) and many platform data type variations (Oracle, SQL*Server, Mainframe etc). Information and technical data lineage is therefore data type aware and governance of physical data structures reflects detailed physical attribute definition.
Axon has a single column for a data format in the attribute facet, FSA constructed a format string from all its underlying syntax configuration, as appropriate for the technology at hand.
So. we are not sure how governance of things like nullability, data length and type would highlight consistent definition across a data flow that that jumps between many technologies...this question was left unanswered.
Lifecycle and Version Control
Many of the Axon bulk import templates had an "Axon Status" column which seemed to reflect a standard Axon governance lifecycle for that type of facet, we saw no evidence of version control and temporality that would enable full tracking of changes over time.
FSA has very granular version control and lifecycle management capabilities, for example, each node in a Feature Tree can be assigned a lifecycle most relevant to its level, classification and type of deliverable.
We saw no way to replicate this in the Axon Meta Model.
FSA to Axon Mappings
Summary
Overall though, the challenges faced were minor compared to the overall benefit of having all this meta data transferred and visible in the chosen data tool for governance and communication, which was Axon.
They just reflect the different focus and coverage of the two different approaches and meta models being integrated.
We got the impression that a business governance user, or possibly even a project development resource may struggle with getting value from Axon once it gets loaded up with many view points and projects, especially if the facets and mappings are inconsistent and every meta data source uses a different import facet and relationship arrangement.
But our stuff exported from FSA...looks great!