Master Data¶

As the authoritative source of information for managing data within your data science project, the master data helps implement robust data pipelines that ensure data scientists work on accurate, high-quality data, thereby reducing the risk of introducing bias into the training data sets.

Master data fields¶

MD fields can be created from the ontology that you create to consolidate domain knowledge and entity semantics. All MD fields do not need to be built from the ontology, and you keep a total freedom in naming and master data semantic conventions. However, we recommend using this unique feature of the TranzAI platform as it provides greater consistency for data lineage and will lead to high productivity gains in feature extraction, model design and model explainability.

Master data management

Field unit¶

Master data management relies on a comprehensive set of standardized units that can be used to secure the consistency of the numerical values consolidated at feature store level.

Unit conversion is managed at the data normalization pipeline level.

Field constraint¶

Field constraints can be used for data quality and to automatically detect incoherent data values at data pipeline runtime. Features created from a core master data field will inherit the constraint of the MD field.

Master data field constraints

Virtualization layer¶

The virtualization layer, designed and maintained through the TranzAI MDM, allows different data sources to be accessed and combined as if stored in a single location without physically consolidating the data into a single repository. This helps overcome challenges associated with heterogeneous data sets, such as data inconsistencies and conflicting data representations.

The virtualization layer provides a unified view of the data while allowing the underlying data sources to be managed and maintained separately.

This layer also provides the ability to decouple storage from compute, a fundamental aspect of the TranzAI platform architecture.

Metadata for all tables in the Feature Store is automatically consolidated through interaction with the TranzAI GUI.