Diving into Dark Pools: Privacy Research on Transparent Blockchains

Author: Muhammad Yusuf, Delphi Digital

Compiled by: Wu Says Blockchain

Translator's Note: This article provides a detailed overview of the workings of different types of dark pools, covering aspects such as privacy architecture, order matching, sources of liquidity, MEV, and anti-censorship. By introducing privacy-enhancing technologies like zero-knowledge proofs, dark pools aim to provide high security while protecting trading privacy. Additionally, the article addresses issues related to regulatory compliance and trade visibility, and explores different business models within dark pools.

Before delving into the inner workings of blockchain-based dark pools, let’s first understand the background of dark pools, how they emerged, why they continue to operate, and to what extent they have changed the game.

Dark Pools of the 1960s

In 1969, when computers were still large enough to occupy one or two rooms, traders shouted orders in trading halls, and institutional investors needed a better way to buy and sell stocks without triggering a chain reaction in the market. Jerome Pustilnik founded Instinet, pioneering electronic trading on Wall Street. Institutional investors could place orders, and Instinet would match and execute buy and sell orders. For a platform like Instinet to function, a significant volume of orders from both buyers and sellers was necessary. So, what made institutional investors eager to participate? Confidentiality. Instinet allowed large companies to operate anonymously by concealing their identities and orders from other participants and the broader market. This not only prevented their trades from impacting the market but also reduced the risk of being front-run by other traders.

As of 2022, more than 60 dark pools have been registered with the SEC. Some are operated by exchanges, such as the New York Stock Exchange (NYSE) and NASDAQ; some are operated by brokers, such as Morgan Stanley's MS Pool and Goldman Sachs' SigmaX; and others operate independently, like Liquidnet and MatchNow.

The issue with dark pools operated by operators is that they may have more incentive to engage in misconduct rather than operate in a compliant manner. The equation is simple: corrupt profits > corrupt costs. Operators can earn more profit than the fines they must pay. In 2018, the SEC fined Citigroup $12 million for misleading investors while operating a dark pool and leaking confidential order information to high-frequency traders, who executed over $9 billion worth of orders targeting Citigroup's clients and profited from it.

Users of traditional dark pools face the risk of being manipulated by operators. This is the heavy price they must pay for participating in a fractured trust system. Since 2011, dark pool operators have paid over $340 million in fines to settle allegations. Compared to the profits they may accumulate, the costs are relatively small. Blockchain-based dark pools attempt to eliminate the need to trust operators with potential malicious behavior, but issues still remain.

Mixers, Pools, and Everything In Between

The design of blockchain is intended to be completely transparent. While it promotes accountability, it is also a double-edged sword. If you receive your salary on-chain, anyone with your wallet address can see your salary over the years and your future payments.

DEXs (decentralized exchanges) and wallets are tracked by wallet tracking platforms and copy trading platforms, the latter even allowing anonymous users to sniper well-performing traders, complicating their trading strategies. Large orders also risk being front-run by seekers in the public mempool.

Before we dive deeper into dark pools, let’s distinguish between mixers and dark pools. Mixers are a subset of dark pools. Mixers like Tornado Cash mix tokens together to sever the link between wallets and assets, making the source of funds difficult to trace. On the other hand, dark pools not only sever the link between wallets and tokens but also help users trade without revealing any information about other parties involved.

Now that we understand the origins and motivations behind traditional dark pools and have differentiated between mixers and dark pools, let’s explore blockchain-based dark pools, which inject privacy into their infrastructure through account/balance models and PETs (privacy-enhancing technologies) such as zero-knowledge proofs, multi-party computation (MPC), and fully homomorphic encryption (FHE).

As of the writing of this article, dark pools like Portal Gate, Renegade, and Tristero are still under development. Meanwhile, privacy protocols like Panther and Railgun are already operational, facilitating privacy-preserving DeFi (decentralized finance) transactions, including swaps, staking, providing liquidity, and lending.

Privacy Architecture

Balance Model

A blockchain is a state machine. The state consists of accounts and transactions. With each transaction of an account, a new set of updates is added to the chain with every state transition. The account balance is updated accordingly. The account balance model is a different way for blockchains to track and manage user account balances. Blockchains like Ethereum, Solana, and Polygon use an account model, while Bitcoin, Zcash, and Monero use a UTXO model. These models differ in how they manage and present the state of the blockchain.

The account model, similar to a bank account, displays the current state as a set of addresses and their balances. Whenever you make a payment, you send an exact amount, with no remainder on either side, and no new balance is created. However, the UTXO (unspent transaction output) model is not so straightforward. When you make a payment, your entire balance is transferred, burning your existing balance. The remainder you have is added to a new balance. The UTXO model displays the current state as a graph of all spent and unspent outputs of transactions.

To illustrate this, consider the following scenario:

Alice's wallet contains two UTXOs, one with 6 coins and another with 5 coins, while Bob's wallet has only one UTXO containing 5 coins. When Alice sends 5 coins to Bob, she merges her two UTXOs into a single transaction, making it difficult for observers to determine the source of the 5 coins she sent to Bob and where the remainder of her UTXOs went. This privacy-protecting mechanism effectively obscures the flow of funds and ensures the privacy of transaction participants.

The newly generated UTXOs have no known owner, requiring more advanced on-chain analysis to associate them with individual users. The account model, on the other hand, has substitutability. It encourages address reuse, making it easy to track an individual user's history.

In the UTXO model, double spending is quite difficult. Each UTXO has a unique "nullifier" generated by itself. When validating a transaction, that is, when the UTXO is spent, its nullifier is also spent, preventing the transaction from being spent again.

Several versions of the UTXO model have been used in privacy-focused L1 and L2s. The Record model used by Aleo is a variant of the UTXO model. Aztec and Polygon Miden mix the UTXO model and the account model.

The UTXO model used by Zcash has largely inspired the UTXO model used in dark pools. Some protocols have modified it to support multiple assets and accounts. Protocols like Portal Gate, Penumbra, Railgun, and Renegade use a "note"-based mechanism, where each UTXO is a note. These notes, like UTXOs, contain information about assets, orders, and values, but are only visible to the owner/user. These notes are stored within a Merkle tree that maintains the state of the chain, called the State Commitment Tree. These notes may be generated from transfers or settlements within the dark pool or may arise from payments to relayers. Renegade uses embedded packaged multi-party computation (MPC) to match UTXOs (orders) and update private states. Multiple notes can also be stored within a single note.

In the Panther protocol, each asset or account is represented as a zAsset (privacy asset) or zNFT (privacy non-fungible token), and the UTXO model operates through a Merkle tree called "Bus," which batches UTXOs, with a maximum of 64 UTXOs per batch, optimizing user cost efficiency. "Bus operators" are responsible for processing the flow, while users are incentivized through rewards priced in $ZKP. When executing a single UTXO, Panther uses a so-called "Taxi" Merkle tree. To support UTXOs across multiple chains, there is also a "Ferry" Merkle tree.

Proof Systems

This is a major component of privacy-enhancing technologies (PETs). Zero-knowledge proofs (ZKPs) are used to prove the ability to possess correct information without disclosing the information itself. In the context of dark pools, ZKPs can be used to prove that users have sufficient funds and permissions to place orders, allowing them to match and settle while keeping these details hidden from dark pool operators, other traders, and external third parties.

Proof systems are cryptographic constructs designed to generate these proofs. They come in various types, each producing proofs of different sizes, requiring different time frames, and consuming different computational resources during generation and verification. Here, we will delve into two important proof systems: Groth16 and UltraPlonk, which have been applied in the dark pools we discussed.

Panther, Penumbra, and Railgun utilize the Groth16 proof system to generate their proofs. Groth16 is known for generating proofs of consistent size, making it a preferred choice, as proof size can impact transaction scale, gas costs, and throughput. On the other hand, Portal Gate and Renegade use the UltraPlonk proof system.

Whether Groth16 or UltraPlonk, both require a trusted setup. In this setup, a group of trusted parties collaborates to establish a set of universal parameters. These parameters are used to verify proofs in a way that does not require the verifier to trust the prover, nor the prover to trust the verifier. It is worth noting that a trusted setup implies a trust assumption about the involved parties. Secure multi-party computation can be used to prevent any party from gaining access to the underlying setup structure. Groth16 relies on multiple parties' inputs to establish its trusted setup, while UltraPlonk requires participation from only one party.

Inner Workings of Dark Pools

Now, let’s understand the inner workings of dark pools, how they protect privacy during transactions, provide liquidity, prevent MEV extraction, resist censorship, and ensure compliance.

Wallet Setup

In most cases, a new dedicated wallet needs to be created to interact with dark pools. Railgun has a third-party wallet provider called Railway Wallet, which allows the creation of an encrypted EOA, enabling you to hold tokens/NFTs without exposing the amounts and types, and also allows private calls to any public smart contract through the Relay Adapt mechanism.

In Renegade, the wallet is equivalent to a UTXO that has a private key and can sign transactions. You need to generate a new UTXO and submit it with a ZK proof. This proof can demonstrate that the new UTXO is indeed new and that you have not used any of the funds within it. Most operations (such as deposits, placing orders, and trading) will invalidate the wallet, so you need to create a new wallet to ensure no double spending occurs. The user experience is similar to other Ethereum wallets.

To use Penumbra, you need to create a new wallet just like using other Cosmos chains. After that, you can fund the wallet through another IBC transfer.

For both Panther and Portal Gate, a KYC process has been established. Ethereum wallets can be used, but the connected wallet users must verify their passports to obtain credentials for interacting with the protocol. Business entities need to undergo a KYB process, sharing their business details. Verification is conducted off-chain by registered compliance providers. These credentials expire periodically and must be re-verified afterward.

Order Discovery and Matching

In terms of order discovery and matching, common DEXs like Uniswap and Curve adopt an automated market maker approach, where orders are matched with liquidity pools. This method is not optimal for privacy, as when orders are matched with public liquidity pools, order details are publicly visible on the blockchain. This is not ideal for traders making large transactions, as it may reveal their intentions to other participants in the market who could exploit this information. Some dark pools ultimately match orders with liquidity pools or use other external public contracts for order matching, employing relays as a privacy-preserving barrier that separates traders from external liquidity sources. Next, we will explore how dark pools acquire liquidity while protecting privacy.

Now, remember that traders have deposited funds into wallets within the dark pool. We will discuss what happens when an order is submitted. Some dark pools take a peer-to-peer order matching approach, while others take a point-to-pool approach, using PETs such as zero-knowledge or multi-party computation, while their internal contracts interact with external public contracts. Adopting a peer-to-peer method or crossing orders between traders can help execute better without slippage. This is because if an order matches with a liquidity pool, the price of the liquidity pool may change between the order submission and execution. When orders cross between traders, they will execute at the price they negotiate between themselves.

For Renegade, which adopts a peer-to-peer method, when traders submit orders, they also choose a relay that can bid on their behalf (matching orders) and pay a fee. Traders generate a ZK proof called "VALID COMMITMENT," proving they own their wallet and its orders. They then send a "handshake" to the network to match with other traders' handshakes. The handshake is essentially an immutable list containing the VALID COMMITMENT ZK proof, hashed order details, nullifier, and key pair. When a trader's handshake matches another trader's handshake, they perform multi-party computation. Renegade uses collaborative ZK-SNARKs to prove that the traders' orders indeed match those of their counterparties. This ensures that order details remain hidden from other parties during order matching. Once the order matches with all required proofs, a new wallet is created to hold the exchanged tokens, followed by verification to ensure that the wallet is indeed new, preventing double spending. The user experience is similar to using any Ethereum wallet.

For Portal Gate, when traders submit orders, an API node receives the order, encrypts it, generates a ZK proof, and then relays the order to the Book. The Book is a set of off-chain nodes that collect orders and run order matching algorithms in an FHE (fully homomorphic encryption) environment. This means that encrypted orders can be matched without needing to decrypt them. Once an order matches successfully, the total results of all order matches are displayed without revealing the details of individual order matches. This also helps protect privacy.

For Railgun, when traders place swap orders through Railgun, a smart contract called the "Adapt Module" executes multiple operations, namely retrieving the private balance specified by the trader in their wallet for swapping and unmasking its privacy. The Adapt Module also verifies whether the order (UTXO) has been invalidated/spent. Then, the 0x API swaps assets among aggregated DEX liquidity, searching for the best rates. The Adapt Module re-masks the swapped assets into the private balance to ensure that their activity or address is not disclosed. This workflow can also be applied to other types of DeFi transactions.

The Panther protocol operates similarly, using a module called Zswap to connect Panther's MASP (multi-asset privacy pool) with other DeFi protocols. Rather than being a "pool," the MASP is more like a Merkle tree of "append-only" properties, where each leaf is a commitment to the UTXOs of assets deposited into the MASP. When traders create orders, Zswap aggregates quotes from other DeFi protocols for traders to choose from. Once an order is placed, Zswap creates an encrypted time-based escrow contract that can privately facilitate exchanges between users without disclosing any details. Once assets are exchanged, users will receive an IOU (I Owe You) for the tokens in the form of ZAsset (ZNFT) from Panther's MASP.

Penumbra's swap module is also called Zswap, but it operates quite differently. When traders create orders, the assets in the trader's private balance are burned, and the encrypted order value is sent to the validators. Processing the order also requires a fee. Users will receive a "swap" NFT to keep a record of the event. The NFT is composed of various parameters, including trading assets, fees, input amounts, and encryption keys. Validators aggregate multiple traders' orders into a batch and then execute these inputs against centralized liquidity positions. Once executed, the outputs are delivered to traders in proportion to each trader's input. In this case, individual data remains confidential, but aggregated data is exposed when validators execute orders against centralized liquidity positions.

Seeking Liquidity

Dark pools like Renegade and Portal Gate operate by anonymously crossing buy and sell orders directly between users, but there is not always enough liquidity to meet order demands. In a typical DEX, you can almost ensure that your order will have sufficient liquidity, but you have to sacrifice privacy and price impact. Privacy—anyone can see what you are selling or buying. Price impact—because guaranteed liquidity comes with a premium, you need to pay along with the order. In dark pools, order information remains hidden before and after the trade.

Renegade follows this design philosophy by providing liquidity through matching order flows with reverse order flows without any intermediaries. In cases of insufficient liquidity, there is a mechanism called "Indication of Interests," where traders can reveal certain details of their orders to the matching order relay. They effectively prove details such as price, size, asset, and order type through zero-knowledge. If traders choose to disclose these details, other traders will be able to view these orders hanging on the order book, potentially leading to faster execution of these orders.

The Panther protocol is not just a dark pool. It also offers a range of other DeFi activities that can be executed privately. Users first deposit funds into the multi-asset privacy pool, representing the deposited assets as ZkAsset (ZkNFT). ZkAsset is essentially an IOU token, similar to the stETH received when staking on Ethereum. Plugins called DeFi Adaptors can privately connect the MASP with existing DeFi protocols. Zswap and Ztrade are adapters connecting the MASP with DEXs like Uniswap, Quickswap, and Curve.

Like Panther or Railgun, Penumbra is also a pathway for various DeFi activities. In Penumbra, when users open LP positions, they effectively create their own mini AMM. This results in thousands of concentrated liquidity positions, all created by users who set the fees they desire. However, this can lead to fragmented liquidity, but the Zswap/DEX engine will consider and aggregate all these individual liquidity positions into a single AMM. This will allow routing of incoming trades across the entire liquidity map.

Portal Gate uses an anonymous AMM DEX called Automaton as a backup for the Book, which is the order book used by the relayer to cross-match users' trades. If the order book cannot facilitate trades due to a lack of liquidity, the orders will be routed to Automaton. Liquidity will be guided within Automaton, just like in any other AMM.

Railgun's Railway DEX uses the 0xAPI created by 0x. This means that orders on the Railway DEX are routed to find the best prices executed in the 0xAPI DEX aggregator. Once a trading route is found, the wallet generates a proof to swap using funds from the trader's balance, masking the incoming tokens from the 0xAPI in the trader's balance.

Business Models

Traditional financial (Tradfi) dark pools are largely seen as an elite club where only those with substantial funds can trade. They pay membership fees to be able to place orders anonymously while also gaining access to relevant reverse order flows from the other side. Trading fees are usually lower than those at traditional financial exchanges. Traditional financial dark pools can also act as brokers, buying securities from sellers at lower prices and then selling them to other buyers at higher prices.

Dark pools in the crypto space have largely abandoned the "operator"-centric design, taking on more of a "facilitator" role. They aim to generate revenue through two main components: protocol fees on successfully matched orders and a portion of the fees traders pay to relayers for processing transactions. A portion of the relay fees can also belong to the relay operators. Traders can also choose to run their own relayers for better privacy and to avoid paying relay fees.

Decentralized dark pools in the crypto space may face "cold start" issues. Unlike DEXs that trade against liquidity pools, most dark pools allow users to trade with each other by matching buy and sell orders. It is crucial for such systems to have traders placing orders on both sides of the market. It is also worth noting that traditional financial dark pools are hosted by banks worth billions like Goldman Sachs and exchanges like the New York Stock Exchange. This makes it relatively easy to provide order flow for their users. There are also independent dark pools, like Liquidnet, which once estimated that at least 100 buy-side firms needed to be operational on their platform from day one to reach critical mass, but ultimately only 38 firms went live. Liquidnet is now among the top three dark pools globally.

Centralized exchanges typically have 10 times the trading volume of decentralized exchanges because they attract a broader user base, offer better user experiences, and ensure compliance, thus increasing supply to meet demand. Liquidity for OTC (over-the-counter) trading is relatively harder to obtain.

Most OTC trading in the crypto space is conducted through OTC desks and custodial smart contracts, and sometimes even through Telegram groups. This is a shocking phenomenon. People are often scammed. It is very convenient for scammers to impersonate others, edit messages, and trading details on Telegram. With dark pools, you may not guarantee instant settlement for illiquid tokens or NFTs, but it ensures a higher level of security and privacy. OTC trades conducted through dark pools allow clients of OTC desks to avoid paying broker fees.

Trading volume is crucial for the effective operation of dark pools. It helps efficiently match orders and protect traders' anonymity. The more anonymous funds in the pool, the more orders in the order book, making it harder to associate depositors and traders with their assets. This is because there are more potential depositors trying to associate assets with them. When protecting assets, it is also important to consider the type of tokens. Protecting common stablecoins like USDC or DAI will provide more anonymity than protecting lesser-known meme tokens, and only a few depositors' tokens can offer higher anonymity.

MEV and Anti-Censorship

Most DEXs operating on public chains, due to their transparent ledgers, allow MEV to exist. Order details on DEXs like Uniswap or Curve are not confidential, enabling seekers and builders to rearrange bundles and transactions in blocks accordingly to extract MEV.

Privacy is important, but it is a subjective issue. Tornado Cash excels at allowing people to conduct financial activities anonymously on-chain. However, because it is open-source and free for anyone to use, it has also been exploited by criminals to launder stolen funds. North Korean hacker group Lazarus has stolen nearly $1 billion ($958 million) to date. The OFAC (Office of Foreign Assets Control) has a list of specially designated nationals and blocked persons. Malicious individuals and entities, like Lazarus, and even the smart contract addresses of Tornado Cash, have been listed by OFAC for non-compliance. Protocols that do not comply with OFAC regulations have a high likelihood of having their transactions excluded from blocks on-chain. Is this a serious anti-censorship issue? Yes. The problem is not limited to block producers. Validators and relayers can also choose to ignore transactions or blocks.

In dark pools, since each order generates ZKPs, order details are largely hidden from all third parties, including validators, eliminating the need to disclose order matching or even settlement details. This architecture can resist MEV and censorship, or at least as long as the transaction workflow remains under the encryption premise of the dark pool. In cases where public contracts are called or shared sequencers are used, opportunities to capture MEV may arise again.

Compliance

Whether you are operating within a protocol or as an individual user, the last thing you want to see is your transaction hanging unprocessed, failing to be included in a block, jeopardizing the success of your state transition, or inviting the SEC for a chat. For the founders and developers of Tornado Cash, these concerns turned into a terrifying reality as they were arrested on money laundering charges. In many countries, the TC front end is also illegal/disabled and is not supported by some centralized exchanges. However, the TC smart contracts themselves can still be used by those with some technical knowledge.

To avoid such situations, dark pools and other privacy-focused protocols have found ways for users to decide for themselves whether to comply.

Renegade allows each trader to choose a set of trading counterparts. Individual traders may only wish to conduct some basic AML/sanctions checks on their counterparts, while institutions may want to conduct KYB/KYC checks on theirs. This counterparty selection logic using ZKPs for compliance checks can be set up before the MPC begins.

Portal Gate and Panther Protocol collaborate with compliance Oracles, which in turn work with traditional compliance service providers to conduct KYC/KYB verification of user accounts off-chain. Once verified, users receive a ZK proof of compliance. This proof allows their wallets to be used with the dark pool protocol, but verification needs to be conducted periodically.

In Penumbra, users can showcase their compliance off-chain using a "transaction perspective," selectively disclosing their activities, including sources of funds. Similarly, in Railgun, users have a "view key" that allows them to see transactions and balances from block X to block Y.

Railgun employs a "Proof of Innocence" developed by Chainway, which is discussed in the Privacy Pools 2.0 paper. Proof of Innocence helps you prove that your transactions are legitimate without revealing any transaction details. A Merkle tree serves as a collection of all previous UTXOs (transactions, balances). With this feature, it can be determined that a specific balance is part of a certain set of transactions. However, to prove that a specific balance is not part of a certain set of transactions, a Sparse Merkle tree that records NULL values is needed. This proof can verify that the balance is not part of a certain set of transactions. In this case, recursive SNARKs are used to prove the accuracy of the proof chain calculation from the initial deposit to the final withdrawal, serving as proof that the user's balance is not part of a certain set of transactions.

Lingering Thoughts

Dark pool users rely on public prices as a benchmark for their trades, but the prices of assets on public exchanges may not reflect the fluctuations occurring within dark pools. Before 2014, price discovery between dark pools and public exchanges was a one-sided relationship. The Financial Industry Regulatory Authority (FINRA) and the Securities and Exchange Commission (SEC) implemented an initiative requiring post-settlement dark pool trading information to be publicly disclosed for each security after a delay of 2-4 weeks. It remains unclear whether similar public disclosure standards will be applied to crypto dark pools in the future to facilitate better price discovery for all market participants.

Pure dark pools must accumulate order flow from both sides of the market to reach a critical mass. We may see many OTC trades shift to dark pools as they will be safer, more private, and more cost-effective than OTC desks and Telegram groups. In cases where private DEXs interact with public liquidity pools or external smart contracts, simulating information leakage may lead to the exposure of certain trades and traders.