Lydia Labs Proposed Changes to ICON Monetary Policy

Governance
1

This document describes a path to improve ICON’s economics and core infrastructure in preparation for growing into a cross-chain hub. Without decentralized block production, high availability and an effective penalty system, it unfortunately doesn’t matter how secure the BTP protocol is.

The fact of the matter is that if ICON is not viewed as secure and sufficiently decentralized, a cross-chain protocol running on it will not be viewed as secure and decentralized either, regardless of the underlying technical design. If ICON is compromised, all integrations may also be compromised. This issue with the hub model was raised when speaking with well-researched industry participants in the cross-chain space (funds invested in cross-chain, researchers, other cross-chain solutions, etc.).

Goals

The goals below are all aligned with the messaging that ICON has implemented the most secure, decentralized and scalable cross-chain solution available. Here are 3 goals this proposal aims to achieve:

  1. Effective penalty system
  2. Rewards only for securing the network
  3. Decentralized block production

Specifications

This section specifies the changes proposed by Lydia Labs to achieve the three aforementioned goals.

Effective penalty system

These proposed penalties are modeled off of typical Cosmos SDK slashing rules and penalties. The Cosmos ecosystem has one of the most robust ecosystems of validators supporting many different (but similar) networks.

Part of the Cosmos Hub narrative is offering interchain security (“ICS”), meaning they have put significant thought into making their core network secure and decentralized, otherwise ICS would not be valuable. Therefore, we felt modeling off of an existing and successful ecosystem while leveraging what has already been built in Goloop would be the best path forward.

We propose two types of penalties, the Validation Penalty (which already exists on ICON) and the Double-Sign Penalty. Both result in slashing and “jail”, as defined below.

Jail

Once penalized, regardless of penalty type, the validator will be removed from the validator set and jailed. When in jail, a validator does not earn any rewards. Validators will remain in jail until they successfully validate or propose another block. This means that an offline validator will not earn any rewards. This also means that an offline sub-validator will not earn any rewards until their next opportunity to produce a block, which depends primarily on the validator set size and the amount of rotating sub-validators. The replacement for a removed validator is always based on the highest power of a sub-validator.

Validation penalty

Validation penalty occurs if a node misses 41,040 blocks (95% of blocks) in 1 term. The slashing percentage will be _% of their bond. As mentioned earlier, if a validator suffers a validation penalty it will be Jailed and not earn any rewards until it successfully validates or proposes a new block. Currently, the Validation Penalty on ICON requires multiple offenses before slashing (which is currently set to 0% anyway), making it extremely easy to profitably operate a sub-validator without actually running a node.

Double-sign penalty

When a validator is found to have committed equivocation by signing two blocks at the same height, the validator will be slashed _% of their bond. Currently, there is no double-sign penalty on ICON, so this would need to be developed.

Rewards only for securing the network

Other networks, such as Cosmos and Tezos, feature the concept of validators charging a commission or fee to voters. Block rewards are ascribed to validators, validators take a commission, then validators distribute the remaining tokens to voters. Each validator can determine their own commission rate as a manually-set parameter. Distribution will be automated based on the manually-set parameter.

This system, combined with the effective penalty system mentioned above, rewards voters only if they are securing the network. If the validator is Jailed it does not earn rewards, therefore neither do their voters, as the validator has no rewards to distribute.

Therefore, combined with an effective penalty system, voters must vote for a node that is securing the network in order to receive rewards. If a node is slashed/jailed, the node receives no rewards, which means it has no rewards to distribute, which means the voter receives no rewards. If a node doesn’t have a bond, the node doesn’t receive rewards and therefore neither do their voters.

Decentralized block production

ICON has a two second block time which enables fast finality on the network. Ideally we would like to keep it this way while achieving greater decentralization. A large set of rotating sub-validators achieves this goal.

At most, we can have ⅓ of the validator set rotating. Any more than that would risk a permanent network halt during the rotation, as the network requires ⅔+1 consensus to produce a block.

Therefore, we propose a rotation of 9 sub-validators with 19 fixed main-validators, resulting in 28 total validators per block. This was recommended by the ICONLOOP team to optimize decentralization, safety and speed. This provides decentralized block production and makes it more difficult to coordinate an attack without impacting the speed of the network. Ideally, the rotation-selection process will be randomized, but more R&D is necessary.

Minimum Wage

In order for rotating sub-validators to reliably provide security and decentralization to the network, they must be operating a node. For the network to reliably have a large set of rotating sub-validators, it must be profitable (or at least close to profitable) to operate a sub-validator.

We propose the network allocate some inflation to a minimum wage provided to the top 100 validators. Minimum wage will only be effective when combined with the penalty system outlined above. The rules of Jail apply to minimum wage, meaning offline/malicious validators would not receive minimum wage.

We would also propose a _ICX minimum bond requirement to receive the minimum wage. This further secures the minimum wage by providing a financial barrier to entry. If anybody sees a clear way to profitably abuse this system, please let us know in the comments.

We propose up to 100 nodes be eligible for minimum wage. If there are less than 100 eligible validators, minimum wage is split among the validators that are eligible. To implement this, we will use an inflation bucket similar to iVoter and other buckets.

Our initial proposal is __% of inflation be allocated to minimum wage. This results in a minimum wage of $xyz based on current ICX prices.

10 Likes
2

Sounds good.

Do you consider a potential issue that validators can be concentrated in AWS, making the network less decentralized?

Should minimal wage be activated only when ICX price is below $_? And then decay linearly to $0 as price goes above $__?

2 Likes
3

I believe the above proposal puts us on an improved path towards becoming a decentralized, secure and sustainable blockchain while improving our qualifications as a Hub for interoperability.

We’ve decided to remove parameter specifics from the proposal for things such as $ Minimum Wage and % Slashing Penalty to keep the discussion focused on the proposed changes from a high-level.

Additionally, I would like to add some thoughts regarding these improvements:

  1. As of 11/15/2022, there are 99 Validators w/ delegation ranging from 102-5,082,881 votes but no bond. This means that while these Validators are not receiving rewards, there are 42,624,657 votes receiving rewards for voting for a Validator that isn’t securing the network.

    • With the proposed commission rate feature, these voters would be receiving zero rewards. Hopefully this would result in votes being re-distributed across the active Validator set, helping improve the profitability of all Validators

  2. To build upon point #1, there are also Validators who have provided a bond but are not currently running a node. This means both the Validator and their Voters are receiving ICX rewards without actually securing our network.

    • Not only would these votes be re-distributed to the active Validator set, but there would be less latency in our network as consensus doesn’t need to skip over a proposed block producer who is not running a node

  3. Minimum wage was our best attempt at subsidizing smaller Validators without removing the upside for receiving supplementary delegations.

    • In my opinion, this feature requires the most peer review as there aren’t many examples of this being implemented on other chains
    • One big factor for increasing the # of rotating sub-Validators to 9 and reducing the # of main-Validators to 19 was solving the potential issue of main-Validators setting the minimum wage to $0. Since the sub-Validator voting power is 9/28=32.1%, only 1 “good actor” out of the main-Validator set would be required to veto the $0 minimum wage proposal (10/28=35.7%).
      • This voting power also works in reverse, where it’s unlikely the minimum wage would be set at 100%
      • I think this feature would require much discussion between Validators to determine the optimal subsidy based on node costs and market rates for ICX

  4. Not only would these changes improve the sustainability and size of our sub-Validator set, it would also likely introduce new contributors to the ICON ecosystem

  5. Commission rate can allow sub-Validators to compete for delegation with main-Validators by reducing the commission rate they charge their voters. It could also unlock alternative incentive structures that projects w/ a Validator can implement.

    • Ultimately the market will decide the optimal commission rate for each respective Validator

As stated in the OP, we encourage all to think about the proposed improvements and do additional research as you see fit to try and find potential issues with our proposal. This is meant to be a signaling proposal to better gauge sentiment on these changes, this is not an official Network Proposal.

1 Like
4

Yes this is a potential issue, but making it possible for the ICON Network to identify which SaaS provider (or no SaaS at all) is not possible afaik. At least not with significant R&D and development.

The point of the Minimum Wage is to satisfy the goal of “100 profitable validators”. If validator 100 is earning enough income to cover fixed infrastructure costs (i.e. AWS server costs) without any minimum wage at all, then yes it should be set to 0. I am not planning any sort of elegant solution to this though, just something that’s passively monitored and perhaps a quarterly network proposal to update minimum wage based on current delegations and market price of ICX.

More elegant solutions are possible, such as forcing validators to submit the price of ICX once a day (would be automated of course), then have minimum wage be based on the average submission. But with more elegant solutions come more nuances and issues, such as handling for a validator that’s purposely submitting ICX price of 0.000000001 in order to maximize minimum wage or impact the average submission.

1 Like
5

from a high level perspective I agree with the proposal, Ill be waiting for the details with proposed values to be able to share a more detailed opinion about it.

2 Likes
6

Any initial thoughts on what initial paramers should be?

1 Like
7

Agreed, it would add a lot of complexity at this point.

This should do the trick!

All the rest of the mechanics sound really well thought out.

1 Like
8

Really liking these changes.

I know this is very early on, however, I am just wanting to gauge how soon we could see changes like this come about.

Why?
A few new validators have been building their protocols around the current validator model (pol liquidity etc) and would need to see how they could continue with the above changes.

That said I do know it’s very possible to do this under the new model, would just require a bit more work for us to implement.

I am referencing, Framed, Gangstabet, Eye on ICON, Invictus and others who do this.

1 Like
9

Thats a totally fair question and I wish I had a better answer. If we go with ICONLOOP as the dev team (most expertise, but also tight on resources), then it would probably not go live until second half of 2023 the earliest in my opinion. BTP is their top priority.

If it’s feasible for an outside team with expertise in Golang to contribute these upgrades to the Goloop repo, it could be done in parallel of BTP and hopefully be done sooner. At some point @CyrusVorwald and @errcsool are working to make it easier for outside devs to contribute to Goloop, but I’m not so involved in those efforts and don’t really know the priority/timeline of that.

1 Like
10

You’ll probably be shocked to hear that I actually AGREE with “most” of this proposal (obviously withstanding the actual #s and %s and such).

The only part I don’t think is actually necessary anymore and I think can be removed because of all the other changes needed is the slashing penalty JUST for missing blocks. Since there is jail time which can go on indefinitely and neither the node or voters would get rewards there’s no need to stab the node operator in the gut multiple times while they are down. It is very over the top and could cause big issues if major hosting services go down again for a long period of time. There should be a slashing penalty for malicious behavior yes, but missing blocks is not “malicious” and shouldn’t be punished by missing rewards AND slashing.

2 Likes
11

I believe Solana was trying to address this issue by including incentives via their Foundation Delegation program to promote geographic and hardware diversification of their Validators.

This was also a big talking point at DevCon for the Ethereum Validators, so we should keep an eye on how they promote and incentivize the diversity of their Network’s Validators.

Just my thoughts as I’m not familiar with every POL model from the referenced teams, but the easiest way seems to be setting their commission rate to something higher like 50%. They could cover their node costs w/ the rewards and whatever’s left could be used for their incentive models. With a commission rate that high they would likely deter organic delegation, but for those with a vested interest in the protocol it would probably still make sense to delegate (depending on the protocol’s incentive mechanism).

fwiw Cosmos Hub slashing penalty for downtime is very minuscule (0.01%). It may be worth digging deeper into the reasoning behind that number, as there may be a reason why a small slashing % was chosen opposed to 0%.

(Avalanche has 0% slashing for downtime btw, but Cosmos Hub consensus mechanism more closely resembles ICON’s than Avalanche’s)

2 Likes
12

We proposed rotating validators and maintaining block rewards a year ago, if rotation on a larger scale occurred more regularly (for example 1 term is 1 hour vs 24 hours) with 9 rotating in and out as proposed the reward itself would create incentives to validators to keep node online, if the validator failed to produce blocks their simply jailed and they and voters loose rewards for that term, slashing is not necessary.

The proposal now is to Jail and Slash validators, Jail makes sense, Slashing does not, most slashing offences on Cosmos have been due to operator error trying to achieve higher availability not malicous acitivity.

The minimum wage could be effectively replaced with Block rewards so long as rotation occurs more regularly, so there is upside to validation process. By imposing further requirements on Bond as well as penalities (specifically slashing) you are unlikely to see more validators opt in, it is simply not worth it risk / reward to have so much at stake and potentially loose it because of an honest mistake (not having server online in a term).

Based on the proposal in its current form we are likely to see less validators have a bond because the minimum wage will not be sufficient to offset the downside risks of a slash as well as maintaining a server of adequate resources.

13

Sounds great, really glad to see something like this potentially being implemented.

Jail idea is great but I agree with others, is there really a need for both slashing and jail? surly jail will be a deterrent for malicious behaviour, if slashing is kept I feel it should be reduced to more of a light warning before jail.

I also believe the minimum wage should be evaluated each quarter but should always kick in if ICX falls below a certain price point for more than 96 hours.

In order to receive the minimum wage a bond of some sort should 100% be in place.

Decentralized block production

I have been working on creating a more decentralized system, what I have come up with so far is far more decentralized than what is currently about in the pos model, will write up and send something over via message at a later time but if implemented it would definitely make BTP and the entire system more decentralized and secure, would also work with what you have already suggested.

1 Like
14

Glad to hear you’re mostly onboard @GeoDude. To @theconnectooor 's point, the slashing size on Cosmos for this penalty is 0.01%, quite small. I would be in favor of a small slash to your point about salt in the wound, but not in favor of no slash at all. I feel that Cosmos has it right here and trust their experience. They have one of the most robust validator set in the industry, so the 0.01% slash for missing 95% of blocks in a day has been received well enough.

Either way, it seems that we agree this should be a penalty. Whether it’s 0% or 0.01% or something else can be discussed when we iron out the parameters of the proposal. As opposed to 0%, which might have some unforeseen attack vectors, I’d like to keep it similar to Cosmos, maybe 0.05% since Cosmos slashing entire delegation (which includes voters), while we would slash only the bond because of how ICON is already built (want to minimize additional work).

@WeMove Just to make sure it’s clear to everybody, Jail lasts until you produce/validate a block again. For a top 19 validator, that would be 24 hours until they have their next opportunity to get out of Jail, so only one day of missed rewards. For sub-validators, it looks a bit different because they have less opportunities to get out of jail.

At any given time, there are 72 sub validators (100 - 28), and 9 rotators. Rotations happen once every 24 hours as currently designed, but can be faster as @metanyx mentioned. Assuming we’re at 24 hours, the calculation would be 72 / 9 = 8 days of jail for a sub validator that misses 95% of blocks during their term.

If we go with slashing of 0.05% for example, the penalty for a main validator would be 0.05% of bond slashed and one day of missed rewards, and for sub validators it would be 0.05% of bond slashed + 8 days of missed rewards.

Good to hear - any expectation on when you’d be able to share? Would be interested to take a look and consider implications on our proposal.

We considered this as well, but that always results in a higher incentive to vote for the top 19. Voters will always get a higher APY if they delegate to a node that produces/validates every block, since they earn more block production rewards. A top 19 validator offering 10% commission will always give a higher APY than a rotator offering 10% commission. We also considered that if block production is supposed to give out a minimum wage, top 19 validators will always be earning considerably more than minimum wage in addition to the rewards earned simply from having more votes.

Since the goal we’re trying to achieve is 100 profitable validators, minimum wage felt like a more direct way to achieve that goal than reintroducing a block reward, which has many other knock-on effects, some of which I described above.

1 Like
15

Currently the only number Im very familiar with is the cost of running an ICON 2.0 node which should be the basis to consider the final value for the “minimum wage”

For example the cheapest server on AWS that can run an ICON 2.0 node is a t3a.xlarge (4cpu/16gb RAM) at around $370 but honestly this can barely run a sub node and I didnt tested it during the load of a main validator so I would not recommend using that configuration.

For a node running on AWS my recommendation is to use at the least a t3a.2xlarge which will cost around $470 (taking into consideration the cost for disk size)

to put this numbers into perspective my node is currently in rank 49 and is making around $60 per month at current ICX prices (I have been running the node at a lost since the past year)

There are other options outside of AWS that can give you the benefit of running a good enough node for as low as $200 -$250 per month but they come with some drawbacks

  • arent as easy to setup as AWS
  • require someone with enough knowledge to run and maintain
  • may have latency issues compared with AWS
  • most are bare metal servers there is a risk of them going down at some point.

with all of this taken into consideration I believe that a proper “minimum wage” should be at around $300 per month

3 Likes
16

Do you consider a potential issue that validators can be concentrated in AWS, making the network less decentralized?

Yes this is a potential issue, but making it possible for the ICON Network to identify which SaaS provider (or no SaaS at all) is not possible afaik. At least not with significant R&D and development.

This cloud centralization thing is something I’ve thought/tweeted about before. I don’t think it’s a huge technical issue to have some percentage nodes on the same provider. Though it does get more concerning as that percentage increases. Where that cutoff is, I’m not sure. But @BennyOptions_LL is right – there’s no way for the ICON network to enforce cloud providers. For this sort of thing, it’s best to rely on social consensus.

This is a feature that I’ve been thinking of adding to the RHIZOME Tracker. All major cloud providers have an API to create/monitor resources. What I want to see is validators create an API key that can be used by RHIZOME Tracker to gain some level of insight into the resources used for the node. The API key can be published on-chain as an additional validator parameter perhaps.

This would provide data about the cloud provider, as well as CPU/RAM resources that can be used to calculate minimum wage without having to rely on people to provide the information (let’s face it, if we ask validators to submit their monthly costs for calculation purposes, they’re incentivized to exaggerate the cost). With this method, we could add support all the major providers (AWS, GCP, Azure, DigitalOcean, Vulture, Linode, etc.). With the data, the RHIZOME Tracker governance page would have an additional column for Provider, and maybe the validators who participate in this program would get a verified checkmark or something similar.

Once this system is in place, it would also allow for an infrastructure oracle to be built. Imagine something like Chainlink where validators source FX quotes, come to consensus, and publish the weighted average on-chain. Something like that can be done for validators where each node basically pings the cloud provider’s API with the on-chain API key to determine whether a node is fulfilling its purpose (whatever those rules are). I don’t think we need something as sophisticated as this, and the above off-chain social consensus method would be a good start.

3 Likes
17

I agree with your findings. After testing almost everything on Google Cloud, it seems like the lowest stable configuration is 4 CPU/16 GB RAM. Anything under that (especially on the RAM side) results in frequent restarts. On GCP, a 4 CPU/16 GB RAM/2,000 GB SSD instance is ~$500/month. I think $300/month for a minimum wage is a good start.

@BennyOptions_LL I also wonder if there’s any R&D that could be done to store the block data in a more storage-efficient way. It’s only been 4 years and the node already requires ~1.5TB. I know storage is one of those things that get cheaper over time, but it’s kind of shocking that it’s already at 1.5TB without significant transaction activity.

3 Likes
18

This is an awesome idea that the pessimist in me thinks that it will not be achievable simply because the validators wont go that extra mile to do the required setup to make this happens.

1 Like
20

Oh totally. But I don’t see this as something that everyone has to be involved in. I think the off-chain solution is a good way to test whether this is even possible (in terms of interfacing with the API from these cloud providers to get the right data). If so, perhaps we can come up with an infrastructure working group within the validators (similar to how some validators register for the CPS). That group can routinely audit all validators for uptime, node specs, etc. and sign off on it say every month or even every quarter. We could then use that list to calculate an additional reward boost either for that validator OR for that validator’s voters. If we can assign an economic incentive to it, I think more validators will make the effort. But to start off, we need to come up with a framework to get the data and decide what to do with it and how to judge it. For that, I’d prob want to reach out to validators active in the ecosystem to get started.

2 Likes
21

This sounds great, maybe if validators have the checkmark they could get x% higher minimal wage.

I agree that 5% slashing is too hard, if not removed drastically reducing it could also work.

1 Like