The average gas fee once exceeded that of the Ethereum mainnet. What issues did the Odyssey event expose in Arbitrum?

Author: Cookie, Chain Catcher

On the evening of June 29, Beijing time, on the first day of the second phase of the Arbitrum Odyssey event, Arbitrum announced the suspension of the Odyssey event due to high gas fees caused by heavy on-chain load.

According to L2 Fees data, on this day, the average gas fee for each transaction on the Arbitrum network soared to over $9 at its peak, double that of the Ethereum mainnet during the same period.

As one of the most popular Ethereum Layer 2 scaling networks, Arbitrum's primary goal is to significantly reduce gas fees to enhance user experience, but such an occurrence is undoubtedly ironic. So, what exactly happened on the Arbitrum network?

Event Review

The event first revolves around Arbitrum's Odyssey event, which was previously announced as an 8-week exploration activity for Arbitrum ecosystem projects, incentivizing players to engage with ecosystem projects, including cross-chain bridges, DeFi, NFTs, and games. The first week of activities focused on cross-chain bridges, with most transactions initiated from other chains and directly credited, and only a few cross-chain bridges requiring manual withdrawal, so it did not cause any anomalies in the Arbitrum network.

The second week of the Odyssey activities included experience tasks for the fixed-rate lending protocol Yield Protocol and the decentralized trading platform GMX. The operations of these protocols are relatively complex, with GMX's tasks requiring users to complete three interactions. However, the high gas fees made users uncomfortable and prompted them to seek the reasons through various channels.

According to GMX community members, the minimum transaction fee for GMX the previous day was 0.0006 ETH, but it was currently set to 0.005 ETH (setMinExecutionFee = 0.005ETH). This fee is not part of the gas fee but is used for starting and closing GMX interaction contracts. After a large number of users discovered this reason and strongly opposed it, GMX lowered the fee to 0.002 ETH.

However, it should be noted that during this period, the gas fees for the vast majority of applications on Arbitrum did not show a significant increase. For example, the Ethereum mainnet fee was 0.005 ETH, while the gas fee on Arbitrum was half of that—0.0025 ETH ($2.75); in addition, Arbiscan data showed that the transaction fees for the vast majority of on-chain transactions on Arbitrum were between 0.002 ETH and 0.003 ETH (below $3), and did not exceed $6.

In other words, the high gas fees on Arbitrum were primarily averaged out by GMX, and the notion that "Layer 2 network gas fees are significantly higher than Ethereum mainnet" is more of an illusion. At the same time, this data seems to point the finger at GMX, with some even accusing GMX of secretly raising fees to make a profit.

Regarding the reason for the parameter adjustment, GMX responded in the Discord channel that the process of opening and closing positions on GMX involves two parts of transactions: sending requests to open/close positions and keeper executing requests. The costs of these transactions depend on the current gas prices on Arbitrum. In the past few hours, due to a significant increase in on-chain activities related to Odyssey, the gas prices on Arbitrum surged, but the handling fees for sending requests to open/close positions are only used for executing transactions; GMX does not and will not earn any revenue from the handling fees for keeper executing requests.

After GMX's official explanation, the gas fees across the entire Arbitrum network remained high. Around 11 PM, Arbitrum announced that due to the heavy load on-chain leading to gas fees above normal levels, it decided to suspend the Odyssey event and deploy Nitro to increase its capacity and reduce transaction costs, so that all communities and projects within Arbitrum can continue to have the best experience. However, no specific date for the launch of Nitro was announced.

The Real Reason for Arbitrum Network Congestion

The POW mechanism on Ethereum has caused users to suffer from network congestion, especially during periods of large-scale transactions or transfers, where users must pay higher-than-usual gas fees to incentivize miners to process their transactions.

However, Layer 2 also has a similar bidding mechanism. According to GoPlus Security's analysis of the surge in Arbitrum gas fees, Layer 2 fees are divided into two parts: one part is the L1 data submission fee, which theoretically becomes cheaper for each user as the number of users increases; the other part is the operational costs of the Layer 2 Sequencer, which become more expensive as the number of users increases.

Therefore, the real reason for the excessively high gas fees on the Arbitrum network is that the Odyssey event was too popular, exceeding the preset capacity of the Arbitrum network. Currently, the nodes (Sequencer) responsible for processing transactions on the Arbitrum network have a bandwidth limit of 120,000 arbgas per second. Arbgas is a unit of measurement used for computing and storing transaction data. A large influx of users has caused computational gas to surge over 1000 times, resulting in each transaction consuming too much bandwidth for computational gas, significantly reducing the network's ability to process transactions with the number of nodes unchanged.

At the same time, the pricing of arbgas in each transaction is fully defined by the nodes. Theoretically, nodes can lower arbgas to 0 to maintain low fees for Layer 2, but doing so would result in nodes being unable to process a large number of transaction events in time, ultimately leading to network congestion. In this situation, the best emergency solution for Arbitrum is to suspend the Odyssey event, while the fundamental solution is to launch Nitro to expand network bandwidth.

It is understood that Arbitrum Nitro is built on WASM technology and compiles the core of the Ethereum client Geth into Arbitrum, also providing cross-chain communication and a new batching and compression system. Therefore, it is more compatible with EVM and is an order of magnitude faster than the current technology. The official team expects that after Arbitrum Nitro is operational, the execution speed of Layer 2 will increase by 20 to 50 times, and costs will significantly decrease.

GoPlus Security also stated that all Layer 2 networks will face this issue. The solution proposed by GoPlus Security is to optimize the billing model and reduce the actual costs of computation.

How Will the Story of Layer 2 Continue?

The main direction of Layer 2 scaling is Rollup, which involves bundling multiple transactions on the Layer 2 network and then submitting them as a whole to the mainnet for verification and settlement to improve transaction speed. The four major Rollup projects, including Arbitrum, are highly anticipated by the market, but both Optimism and Arbitrum networks have encountered significant issues, reflecting that Layer 2 is still in a very early stage, and with the influx of users, various bugs may become commonplace.

Earlier in June, Optimism opened the airdrop for the OP token, but the high load caused by a large number of users led to severe delays in the mainnet and remote calls (RPC). Optimism deployed 10 engineers to maintain the normal operation of public endpoints while doubling Optimism's capacity to alleviate network delays. In hindsight, Optimism reflected that it greatly underestimated the traffic generated by the airdrop and needed to increase the capacity of public endpoints by 7 times. The lessons learned also included regularly conducting load tests, replacing drops with oversupply, requiring partners to expand capacity in advance, and prioritizing concurrent batch submissions, among others.

There was also the StarkNet v0.9.0 version upgrade, which rendered previous wallet addresses invalid, causing early project participants on the testnet to face issues such as unavailable whitelists, lost activity history, and cumbersome asset transfers.

Although several Layer 2 protocols have proposed solutions to further enhance performance, can their development speed meet the rapidly growing demand of applications? Currently, leading applications have begun to explore other public chains. On June 22, the derivatives leading protocol dYdX announced that its upcoming v4 version would be launched as an independent blockchain based on the Cosmos SDK and Tendermint consensus. The reason is that the development cycle of Stark technology is long, and the complete decentralization of the L2 solution Node Operator network will take a long time.

Layer 2 needs to be well-prepared to welcome users and cannot underestimate user participation and overestimate network performance each time. Adam Cochran, a partner at Cinneamhain Ventures, expressed on his personal social platform that Layer 2 is not a panacea, and for Arbitrum, their Nitro has made significant improvements in batching and compression.