Integrating blockchain into existing legacy systems, Learn about Distributed Ledger Technology and Oracles.
In the last decade, blockchain and distributed ledger technologies (DLT) have generated tremendous interest and activity from developers, enterprises, venture capitalists, regulators, and users alike. The innovation of blockchain technology and smart contracts leads to an important question about how legacy digital systems, operated by enterprises, governments, and institutions will be impacted. Presently, the answers to this question have varied from one extreme (“all legacy systems will be replaced”) to the other (“DLT is too slow and unproven to actually replace any working legacy system”). However, the eventual answer may lie somewhere in between, where the utility of select legacy systems is upgraded by DLT integration wherever appropriate, and DLT solutions witness growth in enterprise adoption.
- What is a legacy system
A legacy system is an old or out-dated system, technology or software application that continues to be used by an organization because it still performs the functions it was initially intended to do. Generally, legacy systems no longer have support and maintenance and they are limited in terms of growth.
- What is blockchain and DLTs
Blockchain and distributed ledger technologies (DLTs) are backend infrastructure that store and transfer data among parties within a shared ledger without the need for traditional intermediaries. Since the ledger is redundantly processed and updated by a decentralized network of computers instead of a centrally managed server, users can generally trust the integrity of the data and computations without the need for trusted authorities. The objective is to use a blockchain as a shared ledger for exchanging value between disparate entities in order to achieve greater efficiency, heightened transparency and a reduction in complex reconciliation.
- What are oracles
Oracles serve as an external source of data for DLT systems. They connect the external world to the self‐contained world of DLTs, acting as middleware for data and transaction sharing between environments in a secure and authoritative manner. Information shared by oracles is digitally signed and hence is considered non‐repudiable (assurance that the signature cannot be denied by the party who signed it). Since smart contracts are executed on DLT in a deterministic fashion (i.e. transactions that execute exactly as written because they are verified by all nodes with high fault tolerance), the built-in ability to communicate with the external world becomes difficult to establish without a loss of trust. Oracles become this entity that signs claims about the state of the external world. Since DLT systems were built intentionally to be detached from the external world and its trusted third parties, it is crucial that the links (i.e. oracles) have high integrity. Decentralization, economic incentives, use of trusted execution environments etc. are some of the ways in which oracle integrity is ensured on a varying basis (based on the end use case).
Data flow in a typical DLT‐legacy interoperability framework through oracles.
Oracles are not blockchains themselves but are secure blockchain middleware that operates partly on the blockchain (“on‐chain”) and partly outside of the blockchain (“off‐chain”) asynchronously. The main goal of an oracle is to retrieve external data, validate it and deliver it to the intended entity. Each step of the process requires important considerations to ensure the desired qualities of the blockchain (e.g. being tamper‐resistant, permissionless and immutable) are not lost when expanding to off‐chain data and systems. This requires an oracle to be able to source data from high‐quality application programming interfaces (APIs), show proof of the origin of the data, maintain secure and reliable data delivery, provide economic guarantees to incentivize trusted oracle services and possibly even provide additional cryptographic techniques to keep the data itself private. Some oracles are physical or tangible devices that measure real‐world values, such as temperature or whether a shipment has arrived safely. Other oracles are intangible, and comprise only code. For instance, the recording of external product prices on a blockchain is facilitated by oracles.
In order to ensure communication between off‐chain legacy systems and on‐chain smart contracts and to be able to expand the utility of smart contracts beyond just DLT applications, adopting an approach towards building secure middleware solutions (developed using open‐source technology platforms) that anyone is able to connect to and build on as needed, along with governance models that harmonize inter‐system communication, is required.
Systemic concerns and how integration with DLT can help
Given the scale of the scheme and the multiplicity of stakeholders involved, several concerns are identified and addressed by the implementational agencies.
- Transparency and accountability
As a taxpayer‐funded programme, transparency and accountability become increasingly important for the government and regulators. An advantage blockchain brings over legacy databases is its ability to record the first use of data (in a transaction or publication) on the blockchain, record it immutably (so there is no ability to change it later or, at least, if changes are made, they are publicly viewable) and allow open verification by all participants. The “nodes” of the blockchain can be spread among various government (including ombudsman bodies) and civil‐society stakeholders in a permissioned blockchain network. Generally, assuming the integrity of data coming from the various agencies is to be trusted, this network can serve as a trustworthy, fault‐tolerant, common data store for participants.
- Reducing corrupt behavior using smart contracts
DLT‐based smart contracts can be programmed to automatically transfer welfare benefits to farmers based on whether the data relayed by legacy systems and/or stakeholders satisfy preset conditions (e.g. a certain amount of rainfall occurred). This largely eliminates human discretion in settling claims and improves the claim settlement time and cost via data‐driven automation (as projected in Figure 1).
- Data validation through oracle systems
Since the programme depends on collecting data from multiple independent agencies, concerns about the data’s validity and reliability are always present. Using oracles that draw data from multiple sources to process, validate and relay information from external systems to be stored as immutable records on a blockchain
can help ensure that key datasets are securely transmitted from authentic sources and validated against tampering before the execution of any agreements occurs. However, there still need to be protections for vulnerabilities with regards to data sources being compromised. (This concept is explained in depth later in this white paper.)
Limitations of DLT and challenges of integration
In addition to the above opportunities and benefits, it is important to consider the limitations of blockchain and DLT and the trade‐offs that need to be made.
- Resource consuming and limited scalability
A blockchain spends more resources on consensus protocols (computation and participating nodes) to reduce the risk of double‐spend attack (more prominent in permissionless public blockchains). Also, on large public blockchain networks, transaction verification takes time,; therefore, throughput (the number of transactions being validated and added to the ledger per second) is currently lower than traditional systems. Technology researchers have been working towards scalability solutions for permissionless blockchains, although it is unclear when these will be achieved. Permissioned blockchain networks generally do not have scalability or resource consumption challenges.
- Data dissemination risk
Enterprises are concerned with unintended data dissemination risk on blockchains. Some of these concerns can be partially or fully addressed through private side chains (e.g. Constellation used in Quorum network)4 or advanced cryptography techniques (e.g. zero‐knowledge proofs), but may be costly to implement in the current state. Baseline Protocol also tries to address this concern by using zero‐knowledge proofs to allow enterprises to prove that each counterparty used the same common set of records and functions within a shared business process without internal data leaving their respective databases.
- Unclear regulation
As discussed in later sections, one of the interoperability framework pillars suggests using crypto‐economic guarantees in relation to digital assets. In many jurisdictions, clear regulatory standards are yet to be implemented for the use of such assets, particularly in cross‐border transactions.
Connecting smart contracts with external legacy systems through interoperability standards and legal frameworks
As represented in the figure above, the basic rules for interoperability and legal principles should
be established if middleware solutions are to be developed that will allow data/transaction exchange between legacy and DLT solutions in compliance with existing jurisdictional laws. And as these solutions can be developed by different players on different technologies, the underlying interoperability standards should be flexible and comprehensive to work across legacy systems and DLT solutions, and should ideally have the following characteristics:
- Supported by a large open‐source community building on a secure middleware
- Support most blockchain networks and legacy systems globally
- Embed protocols and technologies to verify data integrity and counterparty performance
- Use a generalized middleware that can satisfy a wide variety of security and performance needs across all IT systems
Further, it is important to mention that an approach for interoperability should emerge from the relevant stakeholders and users after experimenting with various models. Imposing an interoperability standard from the top down may create negative effects on the blockchain trilemma as it may
reduce decentralization by accepting (centralized) data coming from outside. This would also create security loopholes. When standards are used to force interoperability, they may lock all market players into an inferior technology, as Jean Tirole underlined in a ground‐breaking article.8 Accordingly, this paper outlines approaches (strategic pathways) that organizations can adopt to arrive at an ecosystem‐driven interoperability solution rather than prescribing a definitive set of standards.
The approach to building blockchain middleware as the middle layer that can bring both the systems together will help in accelerating the positive effects of blockchain and distributed ledger technology where they are used and bring out the best of both worlds.
Smart contracts and other innovations in blockchain and distributed ledger technologies may significantly alter the way in which business transactions are currently undertaken. Data immutability, fault tolerance, censorship resistance and decentralization are elemental innovations that may lead to a fundamental change in how DLT is perceived by traditional organizations. However, legacy systems for data, communications and computations are the dominant tools for businesses and governments and are likely to be so for the foreseeable future. DLT‐legacy interoperability furthers the potential of DLT and smart contracts by enabling DLT to engage with the real, physical world and apply those elemental innovations in physical‐world use cases — as can be witnessed in a vast number of experiments happening worldwide.