Client interoperability meetup and progress leading up to Devcon-V.

Summary

In September we attended an interoperability meetup where Lighthouse was able to communicate, share blocks, and finalise epochs with every other client that was present. Short-lived testnets were set up with each client individually, and a larger multi-client testnet was built between some clients. We also managed to run a testnet with Nimbus and Lighthouse on two Raspberry Pi computers.

This event demonstrated Lighthouse's ability to run as a fully fledged beacon node, which is interoperable with all other Ethereum 2.0 clients.

Quite a bit of work has been completed since our last update and so we'd like to discuss several significant milestones. As such, we've written about each significant milestone in their own section below, instead of our usual dot-point summary.

Client Interoperability

Team members from Trinity, Prysm, Nimbus, Artemis, Harmony, Lodestar, and Lighthouse came together in Canada with the goal of addressing incompatibilities between our implementations. This was achieved by creating short-lived testnets between each client and ensuring the communication was effective.

We are proud to say that Lighthouse was considered to be the most reliable and conformant client, which resulted in all teams initially trying to form a stable testnet with Lighthouse. Before long, all clients could connect, send and receive blocks, and finalise epochs with Lighthouse. Other clients then began testing amongst themselves, ultimately leading to a short-lived multi-client testnet.

During the meetup, we took the opportunity to discuss the current networking specification and agreed that some modifications were necessary. These modifications (which have now been merged) affect the request/response specification, which deals with how clients synchronise blocks with each other. It was therefore unlikely that clients would have been able to perform long-range syncing, since the specification changed during the gathering.

Even if clients were not able to sync, they could still propagate blocks amongst each other, process them, and finalise epochs. Below is short clip which shows one of the first LAN-based multi-client Ethereum 2.0 networks which solved and finalised blocks between Lighthouse (left) and Nimbus (right).

The interoperability meeting was incredibly productive. In addition to forming testnets, it allowed us to identify and correct a range of small issues in our client while also performing some preliminary stress tests. We created a small testnet with Nimbus, with each client running 2,000 validators on a single machine. Once the stability of Lighthouse improves, we will focus on further optimisations and client profiling with the goal of increasing this number.

Multi-client Discovery

The Discovery v5 specification is currently undergoing a security audit and is yet to be finalised. As such, most clients had not implemented this protocol for the interoperability meeting. To achieve the aforementioned testnets, static peer lists were used.

Since the meeting, we have updated our implementation of discv5 to conform with the latest specification (it was updated after our initial build) and we are testing its compatibility with Felix's go implementation. Prysm have integrated Felix's implementation into their client and it is our aim to achieve discovery via discv5 with Prysm in the near future.

This process has resulted in significant improvements to our discovery related code, in addition to some changes in the Go implementation. Presently, we are waiting for updates to address discrepancies between the specification and the Go implementation before this process can continue. Hopefully we will have multi-client discovery up and running before the next update.

Lighthouse Documentation

Another key takeaway from the interoperability meeting was the degree of difficulty in initially running and connecting to other clients. Each client has a number of CLI parameters and different ways of starting and running a testnet. We found that good documentation around the entire process was a critical factor in the ease of establishing a testnet.

We have therefore allocated some time to improve Lighthouse's documentation in the form of a Rust book. This book lists the different CLI features, parameters, tools, and processes that are required for setting up and running single or multiple Lighthouse nodes, with or without validators. We hope this saves researchers, developers, and advanced users a lot of time when spinning up Lighthouse beacon nodes and/or validator clients.

Ethereum 1.0 Integration

The interoperability testnets were started from a mock genesis state for the sake of speed and efficiency. An Ethereum 1.0 deposit contract was not deployed or used, and no testing was done on Ethereum 1.0 integration.

Prior to this update, Lighthouse did not have any integration with Ethereum 1.0, however that has recently changed. We are in the process of testing and debugging our Ethereum 1.0 integration, which will enable Lighthouse to connect with an Ethereum 1.0 client, read deposits, build a genesis state, and bootstrap in the same manner that it would for a mainnet release.

Lighthouse Security Audit RFP

Our recent successes have brought Lighthouse closer to a point of code stabilisation. As we have mentioned in our previous updates, we plan to perform an extensive security review of Lighthouse prior to the launch of mainnet. In this vein, we have published a Request for Proposal (RFP) for an external security review of Lighthouse. Should the community wish to contribute towards enhancing the security posture of Lighthouse, we have submitted a Moloch DAO request to partially fund this security assessment.

Gitcoin Grant Program

Gitcoin hosted a grant program where the Ethereum foundation and ConsenSys quadratically matched (up to 250x) every DAI contributed to a project. Lighthouse has received donations from 143 different contributors, for which we are extremely grateful. It was incredible to see such community support for our project and we hope to deliver a client that exceeds expectations.

Devcon

The majority of the Lighthouse team will be floating around Devcon in Osaka next week. If you have any questions or would like to chat, feel free to approach any of the friendly people wearing a σ’ shirt. Hope to see you there :).