definition of relative error

[j4yan]

Currently the relative error when the reference value is zero is defined using an epsilon to avoid division by zero; see https://github.com/ROCmSoftwarePlatform/composable_kernel/blob/313bbea5886850acab286f45e9d9816cf0b0dca0/library/include/ck/library/host_tensor/host_tensor.hpp#L327

The problem is that this approve results in very large relative error even if the absolute error is small. For example, (v = 1e-8, v_ref = 0, eps = 1e-10), then ref_diff = 100. We can use more reasonable definition, for example:
rel_diff = (v - v_ref) / max(|v|, |v_ref|). See https://en.wikipedia.org/wiki/Relative_change_and_difference for other alternatives.

[j4yan]

The suggested definition doesn't work when v = v_ref = 0 (most likely happens for integers), so we should check it first.

[asroy]

what about:
rel_diff = (v - v_ref) / max(|v|, |v_ref|, |eps|)?

[j4yan]

It covers the corner case I mentioned above, but why do we introduce a user-defined and hard-coded variable?

[aosewski]

In the mentioned wiki article all relative difference definitions are not defined when the f(x,y) is equal to zero. IMHO this is the special case which should be treated separately.

[j4yan]

@asroy Agree with @aosewski.

[asroy]

@aosewski
Your check_err seems sophisticated, it seems to be calculating some kind of max error?

https://github.com/ROCmSoftwarePlatform/composable_kernel/blob/4248ed7ede401859868447462ebf71903bb33e03/library/include/ck/library/utility/check_err.hpp#L20-L62

[aosewski]

It follows the rule that two floating point number shouldn't be directly compared. Instead one should check their proximity. This function is actually doing the same what NumPy's allclose does.

Yes, this is max error which takes into consideration rounding errors. Thus there's not direct comparison.

https://google.github.io/googletest/reference/assertions.html#floating-point
https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
https://www.boost.org/doc/libs/1_63_0/libs/math/doc/html/math_toolkit/float_comparison.html

[aosewski]

Somehow comment from @j4yan has disappeard. The check_err function specialization for half_t is used in RunConvInstances function which is then used in all convnd unit tests.

Regarding the usage of std::abs. Please note that it is always used on value converted to either float or double data type. (https://github.com/ROCmSoftwarePlatform/composable_kernel/blob/develop/test/include/test_util.hpp)