CI: Add unit tests for loss validator

[nickcom007]

I'm not sure whether it's doable through Github Action since we need to test with GPU.

But it's good to at least of some tests that core team can run locally to verify whether it works or not.

If you want to take this issue, please comment below to get assigned.

[vatsalkshah]

Github did recently start supporting GPU hosted runners.