The new golden copy in market data management

In traditional market data management processes, data from multiple market data vendors is filtered and consolidated into a single version, commonly known as a golden copy. This process ensures that organizations work with a reliable and consistent source of truth. However, traditional approaches often require early decisions on which data is important, which can result in the loss of valuable information later in the process. At BIQH, we do things differently: our golden copy is consolidated and merged, but can include the same data from multiple data vendors. How does it work? We explain it in this article.

What is a silver copy?

In the traditional silver copy raw, unfiltered data is stored, as delivered by various market data vendors. This data is stored in its original form, without any initial filtering or modification. In many systems, this raw data is partially ignored or discarded once a golden copy is created. Important information can be lost early in the process, as decisions about which data to keep are made in this silver copy, and only specific data from one data vendor per each data field will be imported into the golden copy.

At BIQH, however, we retain the entire silver copy, ensuring maximum flexibility and allowing decisions about which data to use to be made later in the process.

What is a golden copy?

A golden copy is the consolidated, processed version of the silver copy. It serves as the single master reference point for all critical data that organizations use for analysis, trading, or reporting. The goal of many firms is to have one reliable, consistent version of the truth: a single golden copy that contains the most current and accurate information. In traditional systems, this often means that important data can be lost once decisions are made about which data to include from the silver copy.

BIQH offers a more flexible approach, where all data is retained even after consolidation, giving organizations more control and certainty.

What is BIQH’s golden copy?

At BIQH, we take a unique approach to the golden copy. Instead of creating a single static version, we manage multiple golden data sets. Where we store vendor data as-is in the silver copy, our golden copy contains a separate golden data set per vendor with validated and mapped data. This ensures that each dataset is cleaned, structured, and ready for use in downstream applications.

In all our tables, we keep data from different data vendors fully separated. This allows us to store, for example, both the last price from data vendor 1 and from data vendor 2 in the same golden copy. This makes it possible to later choose, during data distribution, which market data vendor’s data to use based on business rules and waterfalling strategies.

To learn more about our waterfalling strategies read this blog post

Advantages of BIQH’s golden copy

BIQH’s golden copy offers several key benefits for organizations looking to optimize their market data flows:

1. Waterfalling strategy: Our built-in waterfalling strategy allows clients to prioritize data market data vendors. When data from the primary vendor is unavailable, BIQH automatically switches to a secondary vendor, if authorized. This ensures data continuity without interruptions.
2. Flexibility: By moving the logic from the silver copy to the golden copy, clients can decide at any time which data best fits their needs. This adjustable business logic provides a high degree of flexibility.
3. Retention of data: Unlike traditional systems, where unused data is discarded, BIQH stores all validated data. This means clients can later revisit their market data vendor choices and still access the stored data.

Conclusion

BIQH’s golden copy provides a powerful and flexible solution for managing market data flows. By shifting the business logic from the silver copy to the golden copy, clients gain more control over which data is used and how it is processed. Both the silver and golden copies are fully auditable, with all data being trackable through data lineage insights. This ensures complete transparency in data handling. With the waterfalling strategy and the ability to integrate data from multiple market data vendors, organizations can ensure data continuity without being dependent on a single source. BIQH’s golden copy is suited for modern market data management, offering flexibility, data retention, and complete vendor independence.