f(x)Core Mainnet Validator Updates

lancelai · April 25, 2023, 2:39am

Hello validators,

The fxCore testnet version is released, the branch is: release/v4.0.x, and the tag is: v4.0.0-rc1

Releases · FunctionX/fx-core

fx-core/CHANGELOG.md at v4.0.0-rc1 · FunctionX/fx-core

The upgrade height is: 8088000, it is expected to be 2-3pm SGT tomorrow

If you use the cosmovisor for the upgrade, you can start installing and deploying today

Regards,
Lance

Edit:
About 22hours+ to go countdown link

lancelai · April 26, 2023, 1:36am

Hey everyone!

The testnet upgrade height is almost here - we’re only about 3k blocks away from hitting 8088000. That means in just 4 hours and 47 minutes approximately.

https://functionx.github.io/fx-core/tools/countdown.html?network=testnet

So, I hope everyone has taken the time to practice and test their systems on the testnet pre-upgrade, to ensure a smooth transition when the mainnet upgrade arrives.

Regards,
Lance

FrenchXCore · April 26, 2023, 6:53am

Hi @lancelai !

2 things which prevented my testnet validator from restarting properly and immediately :
1) double_sign_check_height:
In my '‘config.toml’ file, the environment variable ‘double_sign_check_height’ was set to 10.
Not a big deal, but setting it to 0 made it transition smoothly later on (even though it’s not very safe from my point of view).

2) My SSL/TLS configuration:
My testnet validator also runs a HTTPS RPC interface, that’s why I configured an SSL certificate on it (tls environment variables in config.toml file).
However, on v4 restart, it served me the following errors:

Apr 26 08:30:54 *** cosmovisor[***]: 08:30:54AM ERR Error serving server with TLS err="accept tcp [::]:26657: use of closed network connection" module=node
Apr 26 08:30:54 *** cosmovisor[***]: 08:30:54AM ERR RPC HTTPS server stopped err="accept tcp [::]:26657: use of closed network connection" module=rpc-server
Apr 26 08:30:54 *** cosmovisor[***]: 08:30:54AM ERR Stopped accept routine, as transport is closed module=p2p numPeers=0
Apr 26 08:30:54 *** cosmovisor[***]: 08:30:54AM ERR Error stopping consensus state err="not started" module=consensus

I feel like I have a misconfiguration or that I need to disable HTTPS (it works when I do so).
What do you think ?

Regards,
@FrenchXCore

lancelai · April 26, 2023, 8:23am

hello, for 1 they just added a new commit yesterday to fix the double sign check height
And this is the latest documentation to upgrade: https://functionx.gitbook.io/home/upgrade-instructions/cosmovisor/tutorial-binaries

for 2, with v4 could you run fxcored doctor and post the output here to troubleshoot the working environment?

did the HTTPS RPC previously working fine and caused error after this upgrade or it is a new setup?

FrenchXCore · April 26, 2023, 8:42am

Thanks, didn’t notice yesterday’s latest commit…

The HTTPS configuration did work fine before.
Here is what the doctor commands return in HTTP (-working) and in HTTPS (-notworking) configurations. Not sure that’ll help…

And if I may, you should also include disks/partitions space left in your next doctor command upgrade.

Thanks !

WORKING:

Computer Info:
	OS/Arch: linux/amd64
	CPU:  6
	Memory Total: 16008 MB, Available: 14542 MB, UsedPercent: 7.299702%
fxcored Info:
	Version:  release/v4.0.x-1f04e7a8089ddabc3808d08ec3aa451969d0d2f0
	Git Commit:  1f04e7a8089ddabc3808d08ec3aa451969d0d2f0
	Build Tags:  netgo,ledger
	Go Version:  go1.19.5
	Cosmos SDK Version:  v0.46.12
Genesis:
	File:  /root/.fxcore/config/genesis.json
	Network: Testnet
Upgrade Info:
	File:  /root/.fxcore/data/upgrade-info.json
	Name:  fxv4
	Height:  8088000
Blockchain Data:
	Remote Node:  or tcp://localhost:26657
	Chain ID:  dhobyghaut
	Block Height: 8089299
	Syncing:  false
	Node Info: 
		Version:  release/v4.0.x-1f04e7a8089ddabc3808d08ec3aa451969d0d2f0
		Git Commit:  1f04e7a8089ddabc3808d08ec3aa451969d0d2f0
		Build Tags:  netgo,ledger
		Go Version:  go version go1.19.5 linux/amd64
		Cosmos SDK Version:  v0.46.12
	Warning: The remote node chainId(dhobyghaut) does not match the local genesis chainId(06D0A9659E1EC5B0E57E8E2E5F1B1266094808BC9B4081E1A55011FEF4586ACE)
App Config:
	File:  /root/.fxcore/config/app.toml
Tendermint Config: 
	File: /root/.fxcore/config/config.toml
Cosmovisor:
	Path: /root/.fxcore/cosmovisor
	Current: /root/.fxcore/cosmovisor/current
		fxcored version: release/v4.0.x-1f04e7a8089ddabc3808d08ec3aa451969d0d2f0
	Genesis: /root/.fxcore/cosmovisor/genesis
		fxcored version: release/v3.1.x-49d991f692d6662660c2c2dd9e009238f595c438
	Upgrades: /root/.fxcore/cosmovisor/upgrades
		Version fxv3: /root/.fxcore/cosmovisor/upgrades/fxv3
			fxcored version: release/v3.1.x-49d991f692d6662660c2c2dd9e009238f595c438
			upgrade-info.json: /root/.fxcore/cosmovisor/upgrades/fxv3/upgrade-info.json
			upgrade-info.json content: {"name":"fxv3","time":"0001-01-01T00:00:00Z","height":6578000,"info":"{\"binaries\":{\"darwin/arm64\":\"https://github.com/functionX/fx-core/releases/download/v3.0.0/fx-core_3.0.0_Darwin_arm64.tar.gz\",\"darwin/amd64\":\"https://github.com/functionX/fx-core/releases/download/v3.0.0/fx-core_3.0.0_Darwin_amd64.tar.gz\",\"linux/arm64\":\"https://github.com/functionX/fx-core/releases/download/v3.0.0/fx-core_3.0.0_Linux_arm64.tar.gz\",\"linux/amd64\":\"https://github.com/functionX/fx-core/releases/download/v3.0.0/fx-core_3.0.0_Linux_amd64.tar.gz\",\"windows/x86_64\":\"https://github.com/functionX/fx-core/releases/download/v3.0.0/fx-core_3.0.0_Windows_x86_64.zip\"}}"}
		Version fxv4: /root/.fxcore/cosmovisor/upgrades/fxv4
			fxcored version: release/v4.0.x-1f04e7a8089ddabc3808d08ec3aa451969d0d2f0
			upgrade-info.json: /root/.fxcore/cosmovisor/upgrades/fxv4/upgrade-info.json
			upgrade-info.json content: {"name":"fxv4","time":"0001-01-01T00:00:00Z","height":8088000,"info":"{\"binaries\":{\"darwin/arm64\":\"https://github.com/functionX/fx-core/releases/download/v4.0.0-rc/fx-core_4.0.0-rc_Darwin_arm64.tar.gz\",\"darwin/amd64\":\"https://github.com/functionX/fx-core/releases/download/v4.0.0-rc/fx-core_4.0.0-rc_Darwin_amd64.tar.gz\",\"linux/arm64\":\"https://github.com/functionX/fx-core/releases/download/v4.0.0-rc/fx-core_4.0.0-rc_Linux_arm64.tar.gz\",\"linux/amd64\":\"https://github.com/functionX/fx-core/releases/download/v4.0.0-rc/fx-core_4.0.0-rc_Linux_amd64.tar.gz\",\"windows/x86_64\":\"https://github.com/functionX/fx-core/releases/download/v4.0.0-rc/fx-core_4.0.0-rc_Windows_x86_64.zip\"}}"}
	Current plan: fxv4
	List Path:
		/root/.fxcore/cosmovisor
		├── current -> /root/.fxcore/cosmovisor/upgrades/fxv4
		├── genesis
		│   └── bin
		│       └── fxcored
		└── upgrades
		    ├── fxv3
		    │   ├── bin
		    │   │   └── fxcored
		    │   └── upgrade-info.json
		    └── fxv4
		        ├── bin
		        │   ├── frenchxcore-testnet-doctor-working.txt
		        │   └── fxcored
		        └── upgrade-info.json

NOT WORKING:

Computer Info:
	OS/Arch: linux/amd64
	CPU:  6
	Memory Total: 16008 MB, Available: 14894 MB, UsedPercent: 5.014661%
fxcored Info:
	Version:  release/v4.0.x-1f04e7a8089ddabc3808d08ec3aa451969d0d2f0
	Git Commit:  1f04e7a8089ddabc3808d08ec3aa451969d0d2f0
	Build Tags:  netgo,ledger
	Go Version:  go1.19.5
	Cosmos SDK Version:  v0.46.12
Genesis:
	File:  /root/.fxcore/config/genesis.json
	Network: Testnet
Upgrade Info:
	File:  /root/.fxcore/data/upgrade-info.json
	Name:  fxv4
	Height:  8088000
Blockchain Data:
	Data Dir: /root/.fxcore/data
	Chain ID:  dhobyghaut
	Block Height: 8089342
	Syncing:  true
	Node Type: This node is a validator
	Node Info: 
		Version:  HEAD-5eef07630e89ad0bd786fb08fa0fb937e5c84d67
	Warning: The remote node chainId(dhobyghaut) does not match the local genesis chainId(06D0A9659E1EC5B0E57E8E2E5F1B1266094808BC9B4081E1A55011FEF4586ACE)
App Config:
	File:  /root/.fxcore/config/app.toml
Tendermint Config: 
	File: /root/.fxcore/config/config.toml
Cosmovisor:
	Path: /root/.fxcore/cosmovisor
	Current: /root/.fxcore/cosmovisor/current
		fxcored version: release/v4.0.x-1f04e7a8089ddabc3808d08ec3aa451969d0d2f0
	Genesis: /root/.fxcore/cosmovisor/genesis
		fxcored version: release/v3.1.x-49d991f692d6662660c2c2dd9e009238f595c438
	Upgrades: /root/.fxcore/cosmovisor/upgrades
		Version fxv3: /root/.fxcore/cosmovisor/upgrades/fxv3
			fxcored version: release/v3.1.x-49d991f692d6662660c2c2dd9e009238f595c438
			upgrade-info.json: /root/.fxcore/cosmovisor/upgrades/fxv3/upgrade-info.json
			upgrade-info.json content: {"name":"fxv3","time":"0001-01-01T00:00:00Z","height":6578000,"info":"{\"binaries\":{\"darwin/arm64\":\"https://github.com/functionX/fx-core/releases/download/v3.0.0/fx-core_3.0.0_Darwin_arm64.tar.gz\",\"darwin/amd64\":\"https://github.com/functionX/fx-core/releases/download/v3.0.0/fx-core_3.0.0_Darwin_amd64.tar.gz\",\"linux/arm64\":\"https://github.com/functionX/fx-core/releases/download/v3.0.0/fx-core_3.0.0_Linux_arm64.tar.gz\",\"linux/amd64\":\"https://github.com/functionX/fx-core/releases/download/v3.0.0/fx-core_3.0.0_Linux_amd64.tar.gz\",\"windows/x86_64\":\"https://github.com/functionX/fx-core/releases/download/v3.0.0/fx-core_3.0.0_Windows_x86_64.zip\"}}"}
		Version fxv4: /root/.fxcore/cosmovisor/upgrades/fxv4
			fxcored version: release/v4.0.x-1f04e7a8089ddabc3808d08ec3aa451969d0d2f0
			upgrade-info.json: /root/.fxcore/cosmovisor/upgrades/fxv4/upgrade-info.json
			upgrade-info.json content: {"name":"fxv4","time":"0001-01-01T00:00:00Z","height":8088000,"info":"{\"binaries\":{\"darwin/arm64\":\"https://github.com/functionX/fx-core/releases/download/v4.0.0-rc/fx-core_4.0.0-rc_Darwin_arm64.tar.gz\",\"darwin/amd64\":\"https://github.com/functionX/fx-core/releases/download/v4.0.0-rc/fx-core_4.0.0-rc_Darwin_amd64.tar.gz\",\"linux/arm64\":\"https://github.com/functionX/fx-core/releases/download/v4.0.0-rc/fx-core_4.0.0-rc_Linux_arm64.tar.gz\",\"linux/amd64\":\"https://github.com/functionX/fx-core/releases/download/v4.0.0-rc/fx-core_4.0.0-rc_Linux_amd64.tar.gz\",\"windows/x86_64\":\"https://github.com/functionX/fx-core/releases/download/v4.0.0-rc/fx-core_4.0.0-rc_Windows_x86_64.zip\"}}"}
	Current plan: fxv4
	List Path:
		/root/.fxcore/cosmovisor
		├── current -> /root/.fxcore/cosmovisor/upgrades/fxv4
		├── genesis
		│   └── bin
		│       └── fxcored
		└── upgrades
		    ├── fxv3
		    │   ├── bin
		    │   │   └── fxcored
		    │   └── upgrade-info.json
		    └── fxv4
		        ├── bin
		        │   ├── frenchxcore-testnet-doctor-working.txt
		        │   └── fxcored
		        └── upgrade-info.json

lancelai · April 26, 2023, 8:48am

but you are able to start the node and sync right?
so the issue is just on https?
otherwise, we need more logs

FrenchXCore · April 26, 2023, 8:54am

As soon as I switch to HTTP, it’s working fine…
But if I switch back the following parameters for HTTPS, fxcored wouldn’t sync anymore (with the previously given error messages) :

config.toml

tls_cert_file = "/etc/letsencrypt/live/xxx.xxx.com/fullchain.pem"
tls_key_file = "/etc/letsencrypt/live/xxx.xx.com/privkey.pem"

client.toml

node = "https://xxx.xxx.com:26657"

(this above used to work fine and my SSL cert is up-to-date)

The following is working right now:

config.toml

#tls_cert_file = "/etc/letsencrypt/live/xxx.xxx.com/fullchain.pem"
#tls_key_file = "/etc/letsencrypt/live/xxx.xx.com/privkey.pem"

client.toml

node = "http://127.0.0.1:26657"

lancelai · April 27, 2023, 7:29am

Issues has been fixed and committed to the release/v4.0.x branch: Commits · PundiAI/fx-core · GitHub

Since you has already configured cosmovisor, so just start from step 2 again

The fxcored doctor command has added to display disk space

FrenchXCore · April 27, 2023, 7:33am

Thanks a lot @lancelai !
I’m on it.
Will keep you apprised !

Regards
@FrenchXCore

FrenchXCore · April 27, 2023, 8:17am

Alright, got it working now !

I’m assuming there was some additional checks in Cosmos latest version, and I did have to tweak my config parameters for SSL to work with my LetsEncrypt certificate.

Here’s what needs to be configured to make it work - maybe you could add some chapter to the Gitbook:

1) File “.fxcore/config/config.toml” - [rpc] section
laddr = "tcp://*domain.name*:26657"
tls_cert_file = "/*path*/*to*/*your*/fullchain.pem"
tls_key_file = "/*path*/*to*/*your*/privkey.pem"

2) File “.fxcore/config/app.toml” - [json-rpc] section
address = "*domain.name*:8545"
ws-address = "*domain.name*:8546"

3) File “.fxcore/config/app.toml” - [tls] section
certificate-path = "/*path*/*to*/*your*/fullchain.pem"
key-path = "/*path*/*to*/*your*/privkey.pem"

4) File “.fxcore/config/client.toml”
node = "https://*domain.name*:26657"

And thanks for the disk availability add-on !

Regards,
@FrenchXCore

lancelai · May 12, 2023, 6:45am

Dear validators!

We found a bug on testnet, so it needs to be upgraded again.

We will send another testnet upgrade proposal today.

The upgrade height is expected to be on next Monday afternoon SGT.

So the mainnet will be delayed, and the mainnet upgrade proposal is expected to be next week.

Regards,
Lance

FrenchXCore · May 12, 2023, 6:54am

Got it… Ready !
FrenchXCore testnet validator ready to roll with release/v4.1.x-b1a04da361e67d436fd548758facc6580fa9a907 !

11110 · May 12, 2023, 4:26pm

Validator “FX FriendsX” ready too!)

SCENE · May 13, 2023, 1:43am

Will f(x)Core adopt the new Tendermint Core fork?

CometBFT v0.34 that will replace Tendermint Core v0.34

CosmosSDK is preparing new versions for the 0.45/0.46/0.47 lines where CometBFT v0.34 will replace tendermint v0.34.

lancelai · May 15, 2023, 1:24am

FXCore testnet v4.10 is released:

Release: Release v4.1.0-rc0 · PundiAI/fx-core · GitHub
Upgrade documentation: https://functionx.gitbook.io/home/upgrade-instructions/cosmovisor/tutorial-binaries
Upgrade proposal: PundiScan

lancelai · May 15, 2023, 1:29am

About 7hours+ more reaching the testnet upgrade height !

https://functionx.github.io/fx-core/tools/countdown.html?network=testnet

lancelai · May 15, 2023, 2:18am

Seems yes.

github.com

PundiAI/fx-core/blob/release/v4.1.x/go.mod#L206


      
          	gopkg.in/yaml.v3 v3.0.1 // indirect
          	sigs.k8s.io/yaml v1.3.0 // indirect
          )
          
          replace (
          	// use cosmos fork of keyring
          	github.com/99designs/keyring => github.com/cosmos/keyring v1.2.0
          	// use cosmos style protobufs
          	github.com/gogo/protobuf => github.com/regen-network/protobuf v1.3.3-alpha.regen.1
          	// use informal system fork of tendermint
          	github.com/tendermint/tendermint => github.com/cometbft/cometbft v0.34.27
          )
          
          // use for precompile
          replace (
          	github.com/ethereum/go-ethereum => github.com/evmos/go-ethereum v1.10.26-evmos-rc1
          	github.com/evmos/ethermint => github.com/functionx/ethermint v0.22.0-fxcore-precompile-rc2
          )

FrenchXCore · May 15, 2023, 2:44pm

Looks like the testnet upgrade went smooth…

May 15 10:46:46 *** cosmovisor[2683244]: 10:46:46AM ERR CONSENSUS FAILURE!!! err="UPGRADE \"v4.1.x\" NEEDED at height: 8376000: {\"binaries\":{\"darwin/arm64\":\"https://github.com/functionX/fx-core/releases/down>
May 15 10:46:46 *** cosmovisor[2683244]: 10:46:46AM ERR UPGRADE "v4.1.x" NEEDED at height: 8376000: {"binaries":{"darwin/arm64":"https://github.com/functionX/fx-core/releases/download/v4.1.0-rc0/fx-core_Darwin_ar>
May 15 10:48:32 *** cosmovisor[2799381]: 10:48:32AM INF indexed block exents height=8376001 module=txindex
May 15 10:48:32 *** cosmovisor[2799381]: 10:48:32AM INF committed state app_hash=3005BDF0BEDA3EEC7804E554C8DB9D8FD2646D0BF9D59EE2B14507BF54043F9A height=8376001 module=state num_txs=0

Good job !!! It keeps running !

lancelai · May 22, 2023, 6:57am

#FXCORE Mainnet Upgrade v4.2.x governance proposal #36 is now live

Features include:
Supports #optimism and #arbitrum cross-chain protocol integration
Supports adding multiple cross-chain tokens through a single proposal
and more!

Cast your vote
https://starscan.io/fxcore/proposals/36

malcolmT · May 26, 2023, 6:25am

ATTENTION!

Hey Validators,

After the fxcore v4.2.x upgrade proposal is passed, we will post the v4.2.0 release on github:
Releases · PundiAI/fx-core · GitHub.

This will include 2 bug fixes:

Fix the failure of staking to receive the reward due to not recording the punishment event when the validator is punished. This fix will cause state-machine breaking, so all validators must use the official version of v4.2.0, not the v4.2.0-rc pre-release version.
Fix IBC IBC Security Advisory Huckleberry - Security - Cosmos Hub Forum

If you have previously deployed binaries using v4.2.0-rc, you NEED to update the binaries again.

Thanks!