Ironing out initial incompatibilities with other clients in preparation for the upcoming interoperability meeting.
Summary
Over the last few weeks we have been quietly making strides in client compatibility and optimisations. A Lighthouse node can now successfully run on 50ms slot times (with 8 validators)!
This demonstrates the ability for the Lighthouse beacon node and validator client to perform signing, attesting and block processing at very high speeds. To give context to this, the expected slot time for the mainnet is 6 seconds (however with a very high number of validators).
Below is a summarised list of our specific achievements since the last update:
- Successfully connected and gossiped information with Nimbus nodes!
- Updated the client's networking stack to the latest libp2p networking specification.
- Introduced new CLI sub-commands to allow for greater ease in spinning up testnets.
- Debugged and corrected (with the invaluable help of Raul from Protocol Labs) discrepancies between our rust-libp2p library and the go implementation.
- Performed successful compatibility tests of lower layer libp2p protocols and gossipsub between our networking stack and the go-daemon.
- Updated and optimised our syncing strategy for the new networking specification.
- Optimised the database storage layout. Lighthouse now includes hot/cold db storage for faster read/write access.
- Added compatibility to starting a beacon node from a pre-loaded genesis state.
- Created a new tool to generate beacon states to bootstrap testnets.
Beginnings of a multi-client testnet
We have been working with the other client teams in the time leading up to the planned interoperability meetup in September. In this process we have been resolving compatibility issues in our encodings, consensus layer and networking stack. Notably, Nimbus and Lighthouse have managed to connect and share attestations as Jacek demonstrated on Twitter:
Although there are more incompatibilities to iron out (the goal of the interoperability meeting) it seems we are well on our way to multi-client testnets.
Sub-second slot testnets
Internally we have been building and testing networks with sub-second block times. This process has helped us identify and isolate bottlenecks in our implementation, providing notable improvements in efficiency.
In particular we have isolated issues related to block processing, syncing large volumes of blocks, and small inefficiencies in database read/write access. Although we have made progress in streamlining our implementation there is a lot more work to be done and this will be primary focus of the Lighthouse development team once compatibility issues are resolved and a long-lasting multi-client testnet is operational.
The immediate future
In a few days we will be attending the interoperability meetup with the goal to resolve the remaining compatibility issues between Lighthouse and other clients. Specifically the areas where we expect to see improvements include:
- Genesis state construction compatibility - Ensuring all clients construct a consistent genesis state, forming the basis of a testnet.
- Encoding - Ensuring all objects are encoded as specified and consistent with all other clients.
- Network encoding and protocol conformance - The networking stack has many moving pieces and various encoding specifications. We expect there to be some discrepancies here amongst the clients.
- Discovery v5 - Lighthouse uses a custom-built rust discv5 discovery mechanism which is yet to be tested against any other implementation. We intend to conform to and improve the current go implementation of this protocol.
The not so immediate future
After interoperability has been achieved, the Lighthouse development team plans to dedicate time towards optimisation and security. We will profile the client and streamline all core components in order to make the client as fast as humanly (or even inhumanly) possible. We will also be performing our own internal security review on Lighthouse over the coming months to ensure we have met the high standards we expect of ourselves. Following this, we will submit Lighthouse to be audited by an external security team.
Give us a hand!
As always, we warmly welcome anyone who is interested to come join us in developing Ethereum 2.0. Find us in the Lighthouse Gitter, or on GitHub.