Evaluation unbiased of search #35
Description
Currently, the only method in insert_evals relies on the search method. That means that the synthetic dataset generated is dependent on the search and data processing that was used at the time of generation.
This can be an issue since in many cases you want to benchmark different embedding / data processing methods, and right now the dataset will favour methods that were more in line with whatever method was used to generate the evals, rather than the best ones.
One way we could avoid this is by using a vision model to look at a page and generate some questions pertaining to it.