Setting output behavior programattically / looping over the same assert

[esran]

I have a test that is checking output from an API. The output may take several seconds to become stable. Hence I want to check this within a loop. I've got this working with std::panic::catch_unwind() and specifying the prefix with insta::assert_json_snapshot!("test_case_name", response);. My loop has a timeout, and after my loop I repeat the assert without the catch_unwind to get the actual test result.

The problem I have is that every call in the loop outputs the diff. It doesn't seem to be possible to set the output behaviour programmatically in my test. The Settings struct doesn't have anything for setting the output behaviour.

Is there a different API where I can have it not assert or output and just return whether it is okay or not? Or some way of suppressing the output programmatically? Or even a different approach entirely for doing this sort of test without being massively spammy.

[max-sixty]

Is there a different API where I can have it not assert or output and just return whether it is okay or not? Or some way of suppressing the output programmatically?

I may be misunderstanding — would it be viable to just have a normal assert at first, and then when the output becomes stable run it through assert_snapshot?

[esran]

What would I be asserting? Insta is managing my snapshot. 'AI' did suggest manually loading the snapshot to do exactly this but that feels kludgy and sidestepping insta, relying on insta not changing locations / formats of snapshots.

I did try disabling output during the initial loop (using bag) but learned that insta is building up the errors and dumping them at the end of the test or similar.

For now I've left this particular test alone, but am using insta in other tests that are more .. reliable :)

Maybe a feature request to have something like insta::check_whatever_snapshot that doesn't log anything or output failed messages and just returns a bool or a Result of some sort?

[max-sixty]

Ah, yes, I think this is #426, is that right? (also #31 & #353)

What would I be asserting? Insta is managing my snapshot.

I was thinking in your case, it may be viable to wait until the value is stable — i.e. poll until it hasn't changed in a while — and then pass it to insta. But that's contingent on how the value behaves...

[esran]

I like the polling until consistent approach. That may well work in this case.

I still think there's mileage in having a check_snapshot that doesn't log or panic or fail the test though.

[max-sixty]

check out #841, I added yesterday. I'm unsure if it's worth the additional API surface area. lmk if you try it and find it helpful!