Mapping as the backbone of market data distribution

The structural challenge in multi-vendor market data environments

Most banks operate with multiple market data vendors, driven by coverage requirements, legacy decisions, and the fact that no single market data vendor provides sufficient depth across all domains.

At feed level, each market data vendor delivers internally consistent data that can be consumed by individual systems without significant issues. Complexity arises when data from multiple market data vendors needs to be used together within the same context.

The same underlying entity will then appear multiple times across datasets, with differences in identifiers, naming, and available attributes. While the data from each market data vendor is valid within its own context, these differences introduce ambiguity in how the data is interpreted once combined.

In this article, we examine how mapping defines relationships between datasets and enables consistent interpretation across multiple market data vendors.

The same entity, represented in different ways

Consider a simple example involving company data. Two market data vendors provide data on the same company, each with its own identifiers, naming conventions, and set of attributes.

When this data is loaded into a system, the records are treated as separate entries. There is no inherent relationship between them, even though they represent the same underlying company. As a result, the system has no way of recognising that these records refer to the same entity.

The data from each market data vendor is valid in isolation, but when combined, there is no clear way to determine how records should be interpreted or linked. This ambiguity cannot be resolved by selecting a different market data vendor. It is inherent to any multi-vendor setup.

Mapping as a way to control interpretation

Mapping defines how records from different market data vendors relate to each other. It establishes when records should be treated as representing the same entity, or as being meaningfully connected.

In some cases, this is based on shared identifiers such as LEIs or ISINs. In others, it depends on combinations of attributes, source-specific rules, or explicit matching logic based on multiple fields

Within the BIQH Market Data Platform, these rules are implemented using SQL-based logic, where combinations of fields are evaluated to determine how records should be linked. Importantly, mapping does not replace or overwrite the original data. It defines how data from different market data vendors is linked and interpreted within the model.

Mapping is not limited to identifying equivalent records. It supports a range of use cases, including matching records across market data vendors, grouping related data such as news or events, combining and enriching datasets from multiple sources, prioritising data where overlaps exist, and linking client-specific definitions to external data All of these use cases rely on the same principle: defining relationships between records within a shared model.

Mapping within the BIQH data model

Within the BIQH Market Data Platform, mapping is not handled as a separate reconciliation step after data ingestion. Data from different market data vendors is loaded into a shared data model, where multiple market data vendors populate the same tables.

This applies across core domains such as companies, instruments, listings, and currencies. Records remain identifiable by their source market data vendor, but exist within a single structure that allows relationships to be defined on top.

Mapping is applied at table level and is not limited to a specific domain. Once records are mapped within one table, those relationships can also be used across linked tables within the model.

An important aspect is that mapping is not fixed. It can be defined per client and per application. Different clients may choose to map the same data in different ways, depending on how they intend to use or present that data.

Where standard identifiers are not enough

In domains where standard identifiers are widely adopted, mapping is relatively straightforward. Companies can often be linked using LEIs, and instruments using ISINs.

However, not all datasets provide this level of standardisation. In those cases, identifiers alone are not sufficient to establish reliable relationships between records from different market data vendors.

Mapping then becomes more contextual, relying on combinations of attributes, source-specific logic, or explicit matching rules. How records are linked depends not only on the data itself, but also on how that data is intended to be used.

News as a practical example

News data is typically sourced from multiple market data vendors, each providing its own articles, classifications, and metadata. There is no shared identifier that allows articles to be matched directly across sources.

Without mapping, news data remains fragmented. Articles exist as isolated records, with no consistent link to companies, sectors, or broader themes. The structure of the data is defined by the source market data vendor, rather than by a shared model.

Mapping introduces that structure. Articles can be linked to companies based on the entities they refer to, and to sector classifications sourced from other market data vendors. They can also be grouped into broader categories, such as geopolitical developments or sector-specific topics.

This also enables news to be bundled. Articles from different market data vendors that relate to the same topic can be grouped together, even if they are not identical. Mapping is not used to identify duplicate articles, but to organise related information.

From mapping to combining data

Once records are mapped, data from different market data vendors can be combined or prioritised, for example by defining a leading source where overlaps exist. One market data vendor may provide certain attributes, while another provides complementary data. Through mapping, these can be brought together into a single, consistent output. This assumes that the relevant data and symbology can be used within the applicable licensing constraints.

This is reflected in concepts such as merged tables, where data from multiple market data vendors is consolidated into one view, and linked tables, where relationships between records from different market data vendors are made explicit.

In addition, clients can introduce their own reference data and map it to external data from market data vendors. This allows them to define their own reference layer on top of the data.

Implications for data distribution

Without a mapping approach, each downstream system is required to interpret and connect data from different market data vendors independently. In practice, this leads to different teams or applications applying their own logic, resulting in inconsistencies across systems and use cases.

The same data may then be interpreted in different ways, depending on where and how it is consumed. This makes it difficult to maintain consistency across the organisation.

By defining mapping at client or application level, relationships between records can be reused across systems where needed, while still allowing flexibility for specific applications and use cases.

Conclusion

Working with multiple market data vendors inevitably introduces differences in how data is represented. These differences are not inherently problematic, but they do require a clear approach to how data is connected and interpreted.

Mapping provides that approach. It defines relationships between datasets, enables data to be grouped and structured, and allows data from different market data vendors to be combined in a controlled and consistent way.

For that reason, mapping is not an additional feature. It is a core part of how market data distribution is structured within the BIQH Market Data Platform.

Want to learn more about the BIQH Market Data Platform? Download our processing overview or get in touch.