With the current work on the ATLAS release and new HDX Tokenomics, we have been time and resource constrained in communicating the exact background and details of the HDX staking / rebase issue that we were notified of on July 5th 2022. This article explains what happened, how we solved the issue with the help and support of our community, and which steps we have taken to safeguard against similar issues in the future.

We, the Hydranet Team, want to take this opportunity to thank everyone in our community for their continued trust and support, especially in difficult situations like this. None of this would be possible without you, and we are very grateful for the patience, trust, and support we received during this period of uncertainty. You guys are the best!

(Figure 1. “Generous” staking APY resulting from HDX rebase calculation bug)

Executive Summary

• A bug in the HDX rebase calculation logic caused a staking gap of ~6.8mil HDX on July 6th at 03:43 UTC
• All token holders who had HDX staked before that rebase received disproportionately high staking rewards
• Reward distribution was automatically suspended by protocol (OHM feature) until the staking gap is closed, which would have taken ~ 7 months (by extrapolation using average cash inflow rate)
• Bond sales were suspended until the root cause analysis determined that the issue was unrelated to bonds
• A rebase calculation bug present in the underlying OHM contracts was discovered as the root cause
• Several hotfixes were implemented, tested, and evaluated against each other, with the final solution deployed to Arbitrum Mainnet on August 15th at 18:21 UTC
• The staking gap was closed using a one-time mint of ~5.5mil HDX, allowing the continuation of staking rewards to keep incentives for new investors in place after a community governance vote.
• The current OHM-forked HDX contracts have now been re-audited by us and test coverage for critical paths has been significantly improved. We are also applying the lessons learned to the design and implementation of the new HDX Tokenomics and Contracts.

What happened?

1- A bug in the rebase calculation of the OlympusDAO (OHM) smart contracts (which HDX is a fork of) has been continually causing gaps between the HDX balance of the staking contract and the circulating supply of staked HDX.

2- Staking of HDX POL (Protocol-Owned Liquidity) caused a gap of unprecedented size (~6.8mil HDX) in the rewards calculation ahead of the next rebase.

3- The Hydranet Team was notified of unrealistically high staking APY at 22:21 UTC (see Figure 1)

4- The gap manifested itself in the staking contract’s HDX balance on July 6th at 03:43 UTC during the next rebase triggered by a bond sale transaction.

Root Cause

Since HDX is based on the OHM token from OlympusDAO, the token supply is subject to a rebase mechanism to achieve a dynamic target price. Currently, the rebase calculation is executed every 8h (an interval called “epoch”) and adjusts the token supply to account for a 7.5% APY in staking rewards.

The high-level sequence of the rebase function can be described like this:

profits = HDX.balanceOf(StakingContract) - sHDX.circulatingSupply()if profits > 0:  distribute(profits)

with sHDX.circulatingSupply() defined as:

sHDX.totalSupply() - sHDX.balanceOf(StakingContract) gHDX.totalSupply() * index                     StakingContract.supplyInWarmup()

and index representing an accumulator aggregating all rebases (which allows the gHDX token to have a fixed supply). See Figure 3 for the actual code.

Now the issue in the implementation is that the stake function calls the rebase function after transferring HDX to the staking contract but before transferring sHDX to the caller.

(Figure 2. stake() function in staking contract)

This creates the illusion of a balance gap from the point of view of the rebase calculation, inflating the profit by exactly the deposited amount.

(Figure 3. rebase() function in staking contract)

Since the inflated profit then gets stored as the foundation for the rewards distribution at the end of the next epoch, this causes a continuous ping-pong between too low and too high rewards on an epoch-to-epoch basis that has been present from the very first HDX deployment.

The significant difference, in this case, was the unusually high staking volume within a single transaction due to the staking of our POL, creating a gap large enough to show a noticeable (actually, unmistakable) calculation deviation in the staking rewards shown in the web app.

Hotfix

The obvious solution to the issue would be to split the rebase function call-off into a separate variable and adjust the order to:

1- Execute rebase (distributing rewards and starting a new epoch)

2- Transfer HDX to staking contract (from staker)

3- Transfer sHDX to staker (from staking contract)

(Figure 4. stake() function in staking contract with optimal but impossible solution)

However, since an upgrade or redeployment of the staking contract was not part of the original (inherited) design, there was no realistic way to migrate the state without having every user un-stake and re-stake or create a new ERC-20 token and ask users to migrate.

As we realized during simulations that the unstake() function is not affected by the issue, we decided to modify the StakingDistributor contract in a way that a rebase can only be triggered through the unstake() function, and only if it is called from the triggerRebase() method (see Figure 5).

(Figure 5. triggerRebase() function and unlockRebase gate as workaround)

Sounds simple enough, right? Now, however, we have to ensure that triggerRebase() is called exactly every 8h since all staking interactions by users will be blocked by the unlockRebase gate until the rebase is triggered.

Shoutout to Gelato for an easy and (so far) rock-solid solution to that final challenge!

The final challenge in preventing the bug in the future that is…

Restoration of Staking Rewards

Because of the way we found out about this bug (very “generous” staking APY visible on our website), the damage was already done. Exactly 6,847,894.781526 HDX were missing from the balance of the staking contract and all income was being used to fill that gap. Based on our extrapolation of the cash influx at the time it would have taken around 7 months to fill the gap and resume distributing rewards.

As this was not our decision to make, our community voted on the issue of whether to wait 7 months for staking rewards (which would not make the token attractive for new investors) or mint the remaining tokens to fill the gap (which increases inflation and devalues the token).

After the vote ended with 82% in favor of closing the gap, we minted exactly 5,046,246.506007129 tokens (the remaining gap size) and transferred them to the staking contract, closing the gap and returning to our “normal” APY of 9.9% (for now… stay tuned).

Vulnerability of OlympusDAO and other Forks

At the time when we hit the bug, the issue was still present in the main branch of the OlympusDAO smart contract repository¹ — which is what our fork (and others) is based on. However, the bug had already been fixed by the Olympus Team in a different branch and deployed to the mainnet without a public record.

In the time between us encountering the bug and writing this article, the fix has been ported to the main branch² (without an announcement that we are aware of).

If you are maintaining a vulnerable OHM fork and have trouble implementing the fix, feel free to reach out to us.

Future Mitigations and Next Steps

When we noticed the bug after the gap was created, we had to drop our current work on the new HDX Tokenomics (more details soon) to analyze the situation and resolve this pressing issue.

After the issue was fixed, we made the decision to invest more time into securing the current HDX smart contracts and added several unit tests, integration tests, and simulations to reduce the risk of running into similar bugs in the near future.

For the new HDX token design, we plan to migrate to a considerably simpler core design, with all additional features decoupled into separate sets of smart contracts. This will ensure lower complexity of each component (e.g. Staking, Bonds, etc.), create transparent interfaces between them, and prevent anything outside the core logic from directly impacting token supply.

Thank you for taking the time to read about our case and subscribe for more updates from Hydranet, follow us on Twitter, or join our Discord if you enjoyed this article!

PS: Stay tuned for the ATLAS release! Coming up soon…

References: