feat: support time-based and continuous fuzzing and invariant testing

[mds1]

Component

Forge

Describe the feature you would like

Right now fuzz campaigns are defined by their number of runs. It’s also useful to instead define campaigns with a timeout which specifies the duration of the campaign, with a value of 0 meaning to run indefinitely. This is common in other fuzzers such as Echidna.

It would also be helpful to support config via cheat codes as discussed in #744, to allow some fuzz/invariant tests to be defined by number of runs with others defined by a timeout

Additional context

No response

[onbjerg]

Noting here that it does not seem like proptest supports time-based fuzzing out of the box, so we will have to roll our own - that does look like it is somewhat doable, though. We need to wrap the test runner in some time based executor and continually call TestRunner::run_one, but this function takes a case, not a strategy, so it looks like we might need to manage the simplification/complication of values etc. ourselves

[mds1]

Proptest is largely unmaintained at this point. There hasn't been an update for over a year, though there is a fork that's made a few fixes—more info here: proptest-rs/proptest#268

I only mention that because two alternative options for implementation include:

1. Forking and maintaining proptest to add support for features like this
2. Switching to a better maintained fuzzing framework that has this feature implemented (or makes it easy to wrap around). A few alternatives to look into can be found in meta: forge fuzzer improvements #387, but a lot may have changed since that issue was created so there may be newer/better crates not listed

[onbjerg]

I think we should consider switching at some point. I looked around but didn't find anything that immediately struck me but I haven't looked much. I think forking and maintaining a fuzzer on top of Cast + Anvil + Forge + supporting stuff (foundry-hardhat, book etc.) is probably stretching resources a bit thin 😄 Will take a look at the ones linked in #387

[rex-remind101]

Your input on this thread would be appreciated since it sounds like you all have some direction already proptest-rs/proptest#268 (comment)

[0xPhaze]

Any progress on this one?