Annoy Tree Approach

The Approximate Nearest Neighbors based Tree had compelling MRR@10, but, overall, it was not competitive with the baseline or other approaches explored.

It should be noted, the compression methods used on the passages and queries could likely be improved. Our implementation was simplistic. Additionally, there are hyperparameters available for tuning, which were not used in experimentation because of time constraints.

Although the MRR in some cases was ok, when compared to the baseline, the Recall was significantly worse. The time of the Annoy Tree was significantly worse than the baseline, however, our implementation took a greedy approach. With more engineering for optimization, we likely would be able to improve on this time significantly; however, because of the poor MRR and Recall, this approach was not pursued further.