Ultra Fast Super Slick CI (#4755)

Attempting to improve our CI speeds as its recently been a pain point.

Major changes:

 - Use a github action to pull stable/nightly rust rather than building it each run
 - Shift test suite to `nexttest` https://github.com/nextest-rs/nextest for CI
 
 UPDATE:

So I've iterated on some changes, and although I think its still not optimal I think this is a good base to start from. Some extra things in this PR:
- Shifted where we pull rust from. We're now using this thing: https://github.com/moonrepo/setup-rust . It's got some interesting cache's built in, but was not seeing the gains that Jimmy managed to get. In either case tho, it can pull rust, cargofmt, clippy, cargo nexttest all in < 5s. So I think it's worthwhile. 
- I've grouped a few of the check-like tests into a single test called `code-test`. Although we were using github runners in parallel which may be faster, it just seems wasteful. There were like 4-5 tests, where we would pull lighthouse, compile it, then run an action, like clippy, cargo-audit or fmt. I've grouped these into a single action, so we only compile lighthouse once, then in each step we run the checks. This avoids compiling lighthouse like 5 times.
- Ive made doppelganger tests run on our local machines to avoid pulling foundry, building and making lcli which are all now baked into the images. 
- We have sccache and do not incremental compile lighthouse

Misc bonus things:
- Cargo update
- Fix web3 signer openssl keys which is required after a cargo update
- Use mock_instant in an LRU cache test to avoid non-deterministic test
- Remove race condition in building web3signer tests

There's still some things we could improve on. Such as downloading the EF tests every run and the web3-signer binary, but I've left these to be out of scope of this PR. I think the above are meaningful improvements.



Co-authored-by: Paul Hauner <paul@paulhauner.com>
Co-authored-by: realbigsean <seananderson33@gmail.com>
Co-authored-by: antondlr <anton@delaruelle.net>
This commit is contained in:
Age Manning
2023-10-03 06:33:15 +00:00
parent 1c98806b6f
commit 8a1b77bf89
19 changed files with 616 additions and 475 deletions

View File

@@ -1,52 +1,52 @@
-----BEGIN PRIVATE KEY-----
MIIJQwIBADANBgkqhkiG9w0BAQEFAASCCS0wggkpAgEAAoICAQCci30cib5B/B5Q
Nd8grzi4LxmlyfZFi3VfpukwdwOD1Xk3ODk1OtjAzhK46YhDclvcu98m1Dnib1Z+
eTjRuEEoekIxz2+BbOle7G52LNvuDZpD+HKucqIU3TnEKPPuTYPplZ1n/9EyxXUw
D5uTkn7xXzK8UFXUt73j6I6VFMdHlNcwLcx8KSwBDzvnGT4ew/UL+ThON3j5rIT+
nFHDcC2zoM+6ANdVkL6GHid4/cOcYW6GxB9TRZtEasqze41bC+kXZtPlV5V2nilA
zVj8z9ynwBpHkLH+E6sMUhSEwA++QfI1gGf0FmSBgSIZ3RdPo/dphkLG8fZXKMkM
zKkRm5hcstDP6DnTIYl+CfuVez5gZ0/yelAqXNvTqMKuDhHTTRRYaOXZX4BAiQO2
Q6a6WYLe87E2ka5AF2T2y/BPeXjUwDS/1mFIB3FUGlMLVJt8/RLznXVGoSsYaptt
iiPucQbMPEysCJ4/LZ9zxe3EDWWjpurLHGi/Y/dVziEvg1EoycixdZogKz0QVCz4
++QI0kPDDX7So7CWni2JJuYguF/8CX8QbCT2L8jXf0uQrq76FLKj88A7lS8DzXBt
/pRryiIlDyLenJwHmrv6p+P/FYvgnJHvAEtTynxYm5GA16YWy+Djc5XVgNHjV4Td
X3GueAp+NhBBaHDFvYCbP/oXkRvNRQIDAQABAoICACCSBxxeblblQVtX8g4nVso/
hnsPi61JiEi3/hGG2ZTe4AMEsCZqkXmABrYxZJf/3awN7K5z/n0lxB25VACScQAe
e9JIQf9wLRgCYjM1PycG7n9Q3G9+S0nDA4dUK/h7aUQ6zE68k4aYPbsbrDdmhgHr
WC+FGW6SMjCOjMfo1FOI3MLZ7I8ys8Seqkx5XIrjI4NzvWrMsN9lrSAaXwqmNuQG
Q+ID1cmoPXPDJ1xNlBrfzLK+cHQPafAwte7k+HKmhj9HtjOj5uWQn62ra+Xhy5ud
ZPpZ2Savaem81CcQnNXte5r1Fevbktq9Bt7RuM1ppIrwk8k3w5S72CTRGiYfNPJV
M1RMp46GrXVJdmx3k9LQfKdT6Gv9xTJXYQl7jN0+4uZ7QrVQHpcMpxPsATl+cQQH
wzCTbj2Oqn/30KqkZLyueN2MalRP8mVSe5nD+vvGb/sWLs52kp6QvHdlXER2RBFk
tJ5cGi+vgueoukb+qatiAE2y5MxYCqD02ShGcLos/SUQThRhL+iD8t0h+FoPTD5y
eTNZ85hF1HdypH1If8/YGETg55+fHYUAtYGT6R8lYeFMvBC05suorLBciXShOGuY
4zBbt32fPlsXlLneAtAAFv2BiJMt0TQavWHITLInFW1/aMHDV4/Pq69sRZuHdRaW
XFXD8CjnPUS5tBMQOqYhAoIBAQDLqUo7v3SpIstXmyU7BfUBuTYGS7MzjMhDxFUl
HvmbVZlOXhnPb3p4mW/XHrah9CjFBLJt3CF+PP/njwMw0YtPxCQpQwj0pI8CuveE
4Puq2wEfxVg+JKh1xidNj8230/WINzwfLCVfco7KKmjQX0MgMGaANQ0sGnt/r1eB
MwpY5uID+D5PORXUcHxBWlsVLyzZ9ZqKhAgewr3i7BLX2y7nwqEGlWTt1lxzZGCR
a8NZIAIs3qGzAgtm7O3hMz6XZulVyVSrMmmT8qXT4Lo1nW/9J6slV7Wdp9f++mr9
m2vwrpJtmdPcA+YKPVgoFlKmZpZZbVvd+4uy8ksoxs1/cF7VAoIBAQDExnLQplq2
BPoxEDAnlS+8Jju5en5Pk70HOfpQGUa4/6vY60x/N5sJqc6CcDySkkxRI8jLzMTe
AE9jqM+Z39MvGCH+SF9EPRopbAJIrcChXfvk2Imp7PLFRGrEBju63nQfaHdcefFy
Ia7RA8SCHLCReRdqPjSXbPAYPZK84vVNSfhrkytA4FJnaojvaqJqLQH9vB7CXv18
Fu6w5fnrgARIoBhy2mb0QWzgd9JMsVDgS5XyX/4HBUejjXDdmLosOZ4PJ0GM2+tr
ilO/9KKhV9lqH7DcFgJBNkgVKRD1Ijr21yyOkttB5PULzaTQhzqkorGkWgyTzLWn
ksqOr2cWt0yxAoIBAElIazvAkpvht0WYv/bTF+w81uHBD4R3HgC0fubSlIJ+dKGs
XqEzvd/zZjkEBjeUga8TF5lMYojoLjUGUuGYJQbYzyJBytEs/YDAAhzAUA6Uq3zh
J/WEf1GRscbI/f8tt+YB6hJVckU8FYFNbVW9UYwdnmR3snuyM8ooL9Z/pTOEMMO4
6cLcCazdpPhnKOsghIURSUCabcmTzXv/8m/VoLUoZYTW8PBb9/xVnCH3ot1JFT9M
BOdCzxOEIbytEdKs5z1FKsBHbZIc9+qbrKVqN0fipETVoLZQFPrc5O7IpDiAuJPT
jFZY2MfKdxRFpAvYUjVvkmT4BLapVL4hewRmTNkCggEBAKuJP8/KJSulvSEGNqRa
9kjzn376XKAsb02caixnTHK7Vuh7fq0sIThUUhT9mKBBbswRANtAv6Gz7YE4SPmf
1+6nAAM2ve2zwlm3sWoAJfvF/W+qoJ+EUsJK+TO3J1yozdwPanYwS52t5UKKIUU3
k2jNge75GUmkCs1m58NHqoXc5PWKTtt4cf17LrJfaARdBe5Wjw3sVtdU+nE1mh+E
8rcI8Sc2Yyes3Sf07Fw0+wb8fVPUAJPIM4JNK8XRfQJOnA4jr44GrPyLkqS0sw0p
kvtjcv75JLAKjN39da3sUDCctVf4h7Cy0jee5n1uVV3uAiP+6BX0D6tsWK34FEsG
MZECggEBAIi/sjZNQjplD5zOULEWL8W6b+3CZymR5Qqa0brlx1Lz8h/daIITIFvm
bue/CjIht/oRGLVE8yzw2ojLf424h3h5PjmXMBNHlVkWQXfn6xCI8MjfZ71uA39O
RVCXAYwcghOWZL4Fkz+XQmIOdJ1OPXfU0py943joYZbgXXAYOc/zNylo9j7+bqDK
vLtFd4IIQoRzjsY//FoAuAditf4xDRqLwOh4amboZw1Qmn6bwDnCaKsFmA3o5BYR
4aRUm1dEbZgPtm2tuHQpEKuOPhWHroi3NsEdbhoyy3IUe0c3w4YGgnuvVy616wkV
GlPvUaKC1KX0CX1qT1anVZq9bSMTG+M=
MIIJQgIBADANBgkqhkiG9w0BAQEFAASCCSwwggkoAgEAAoICAQC1R1M9NnRwUsqF
vJzNWPKuY1PW7llwRRWCixiWNvcxukGTa6AMLZDrYO1Y7qlw5m52aHSA2fs2KyeA
61yajG/BsLn1vmTtJMZXgLsG0MIqvhgOoh+ZZbl8biO0gQJSRSDEjf0ogUVM9TCE
t6ydbGnzgs8EESqvyXcreaXfmLI7jiX/BkwCdf+Ru+H3MF96QgAwOz1d8/fxYJvI
pT/DOx4NuMZouSAcUVXgwcVb6JXeTg0xVcL33lluquhYDR0gD5FeV0fPth+e9XMA
H7udim8E5wn2Ep8CAVoeVq6K9mBM3NqP7+2YmU//jLbkd6UvKPaI0vps1zF9Bo8Q
ewiRbM0IRse99ikCVZcjOcZSitw3kwTg59NjZ0Vk9R/2YQt/gGWMVcR//EtbOZGq
zGrLPFKOcWO85Ggz746Saj15N+bqT20hXHyiwYL8DLgJkMR2W9Nr67Vyi9SWSM6r
dRQlezlHq/yNEh+JuY7eoC3VeVw9K1ZXP+OKAwbpcnvd3uLwV91fkpT6kjc6d2h4
bK8fhvF16Em42JypQCl0xMhgg/8MFO+6ZLy5otWAdsSYyO5k9CAa3zLeqd89dS7H
NLdLZ0Y5SFWm6y5Kqu89ErIENafX5DxupHWsruiBV7zhDHNPaGcfTPFe8xuDYsi1
55veOfEiDh4g+X1qjL8x8OEDjgsM3QIDAQABAoICAEP5a1KMPUwzF0Lfr1Jm1JUk
pLb26C2rkf3B56XIFZgddeJwHHMEkQ9Z6JYM5Bd0KJ6Y23rHgiXVN7plRvOiznMs
MAbgblroC8GbAUZ0eCJr5nxyOXQdS1jHufbA21x7FGbvsSqDkrdhR2C0uPLMyMvp
VHP7dey1mEyCkHrP+KFRU5kVxOG1WnBMqdY1Ws/uuMBdLk0xItttdOzfXhH4dHQD
wc5aAJrtusyNDFLC25Og49yIgpPMWe+gAYCm5jFz9PgRtVlDOwcxlX5J5+GSm7+U
XM1bPSmU1TSEH233JbQcqo4HkynB71ftbVUtMhEFhLBYoFO4u5Ncpr+wys0xJY4f
3aJRV5+gtlmAmsKN66GoMA10KNlLp2z7XMlx1EXegOHthcKfgf5D6LKRz8qZhknm
FFgAOg9Bak1mt1DighhPUJ0vLYU6K+u0ZXwysYygOkBJ/yj63ApuPCSTQb7U0JlL
JMgesy1om3rVdN0Oc7hNaxq7VwswkzUTUKS2ZvGozF3MmdPHNm5weJTb3NsWv8Qo
HiK1I88tY9oZ5r91SC82hMErmG4ElXFLxic1B29h3fsIe/l+WjmZRXixD9ugV0gj
CvNa8QD9K3hljlNrR6eSXeO2QOyxAEUr2N1MBlxrnAWZCzXKiTvTx1aKDYhJT0DY
zae/etTLHVjzgdH6GS33AoIBAQDaaWYHa9wkJIJPX4siVCatwWKGTjVfDb5Q9upf
twkxCf58pmbzUOXW3dbaz6S0npR0V6Wqh3S8HW7xaHgDZDMLJ1WxLJrgqDKU3Pqc
k7xnA/krWqoRVSOOGkPnSrnZo6AVc6FR+iwJjfuUu0rFDwiyuqvuXpwNsVwvAOoL
xIbaEbGUHiFsZamm2YkoxrEjXGFkZxQX9+n9f+IAiMxMQc0wezRREc8e61/mTovJ
QJ7ZDd7zLUR7Yeqciy59NOsD57cGtnp1K28I2eKLA4taghgd5bJjPkUaHg9j5Xf6
nsxU2QCp9kpwXvtMxN7pERKWFsnmu8tfJOiUWCpp8SLbIl6nAoIBAQDUefKKjRLa
6quNW0rOGn2kx0K6sG7T45OhwvWXVjnPAjX3/2mAMALT1wc3t0iKDvpIEfMadW2S
O8x2FwyifdJXmkz943EZ/J5Tq1H0wr4NeClX4UlPIAx3CdFlCphqH6QfKtrpQ+Hf
+e8XzjVvdg8Y/RcbWgPgBtOh2oKT5QHDh13/994nH7GhVM7PjLUVvZVmNWaC77zr
bXcvJFF/81PAPWC2JoV6TL/CXvda2tG2clxbSfykfUBPBpeyEijMoxC4UMuCHhbp
NpLfKJQp9XNqbBG2K4jgLQ8Ipk6Vtia/hktLgORf/pbQ4PxEv7OP5e1AOreDg/CW
RnQtBb+/8czbAoIBABfDA8Cm8WpVNoAgKujvMs4QjgGCnLfcrOnuEw2awjs9lRxG
lki+cmLv+6IOmSK1Zf1KU9G7ru2QXjORZA0qZ4s9GkuOSMNMSUR8zh8ey46Bligr
UvlTw+x/2wdcz99nt9DdpZ1flE7tzYMe5UGPIykeufnS/TNYKmlKtivVk75B0ooE
xSof3Vczr4JqK3dnY4ki1cLNy/0yXookV+Wr+wDdRpHTWC9K+EH8JaUdjKqcobbf
I+Ywfu/NDJ++lBr2qKjoTWZV9VyHJ+hr2Etef/Uwujml2qq+vnnlyynPAPfyK+pR
y0NycfCmMoI0w0rk685YfAW75DnPZb3k6B/jG10CggEBAMxf2DoI5EAKRaUcUOHa
fUxIFhl4p8HMPy7zVkORPt2tZLf8xz/z7mRRirG+7FlPetJj4ZBrr09fkZVtKkwJ
9o8o7jGv2hSC9s/IFHb38tMF586N9nPTgenmWbF09ZHuiXEpSZPiJZvIzn/5a1Ch
IHiKyPUYKm4MYvhmM/+J4Z5v0KzrgJXlWHi0GJFu6KfWyaOcbdQ4QWG6009XAcWv
Cbn5z9KlTvKKbFDMA+UyYVG6wrdUfVzC1V6uGq+/49qiZuzDWlz4EFWWlsNsRsft
Pmz5Mjglu+zVqoZJYYGDydWjmT0w53qmae7U2hJOyqr5ILINSIOKH5qMfiboRr6c
GM0CggEAJTQD/jWjHDIZFRO4SmurNLoyY7bSXJsYAhl77j9Cw/G4vcE+erZYAhp3
LYu2nrnA8498T9F3H1oKWnK7u4YXO8ViyQd73ql7iKrMjE98CjfGcTPCXwOcPAts
ZpM8ykgFTsJpXEFvIR5cyZ6XFSw2m/Z7CRDpmwQ8es4LpNnYA7V5Yu/zDE4h2/2T
NmftCiZvkxwgj6VyKumOxXBnGK6lB+b6YMTltRrgD/35zmJoKRdqyLb1szPJtQuh
HjRTa/BVPgA66xYFWhifRUiYKpc0bARTYofHeoDgu6yPzcHMuM70NQQGF+WWJySg
vc3Za4ClKSLmb3ZA9giTswYMev+3BQ==
-----END PRIVATE KEY-----