Evaluation protocol IV get the different result from paper #15
Description
Hi
I evaluation the four testing protocols in paper. I don't confirm my method and result is correct.
there is 4 file list txt。[0] gallery_black_glass.txt [1]gallery_no_glass.txt [2]probe_black_glass.txt [3]probe_no_glass.txt
4 test protocols:
I) [1] vs [3]
II) [0] vs [2]
III) [1] vs [2]
IV) [0][1] vs [2][3] (Gallery images contain both eyeglass images and non eyeglass images, so as probe image.)
the result from paper show that III) has the better result than [IV)。
for ResNet-22-A RPR@FAR=10e-4 III): 88.13 IV):78.17 Rank1 III): 95.61 IV):92.31
in my experiment test, I use LightCNN29 model to evaluate, ang get the following result:
for LightCNN29 RPR@FAR=10e-4 III): 87.49 IV):87.03 Rank1 III): 93.39 IV):98.54
as showed, III) has litter better result than IV) and Rank1 smaller than IV）.
I mixture [0][1] as the new Gallery, and mixture [2][3] as the new probe in protocol IV). which should has less degree of difficulty than protocol III), because all of pairs in III) contains non-eyeglass and eyeglass, otherwise，IV)contains some simple pairs of non-eyeglass.
So I don't think IV) could get better than III) from my consider and test
where is wrong for my test? could you tell me