Hedging Your Bets: Optimizing Accuracy-Specificity Trade-offs in Large Scale Visual Recognition

Introduction

As visual recognition scales up to ever larger numbers of categories, maintaining high accuracy is increasingly difficult. In this work, we study the problem of optimizing accuracy-specificity trade-offs in large scale recognition, motivated by the observation that object categories form a semantic hierarchy consisting of many levels of abstraction. A classifier can select the appropriate level, trading off specificity for accuracy in case of uncertainty. By optimizing this trade-off, we obtain classifiers that try to be as specific as possible while guaranteeing an arbitrarily high accuracy. We formulate the problem as maximizing information gain while ensuring a fixed, arbitrarily small error rate with a semantic hierarchy. We propose the Dual Accuracy Reward Trade-off Search (DARTS) algorithm and prove that, under practical conditions, it converges to an optimal solution. Experiments demonstrate the effectiveness of our algorithm on datasets ranging from 65 to over 10,000 categories.

People

Publication

Jia Deng, Jonathan Krause, Alex Berg, Li Fei-Fei. Hedging Your Bets: Optimizing Accuracy-Specificity Trade-offs in Large Scale Visual Recognition. IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2012.
[ Paper ] [ Supplementary materials ] [ Bibtex ]

Code, Data, and Models

  • hedging-1.0.tgz: Source code.
  • hedging-data-minimum.tar: A minimum set of pre-computed features and models for a quick evaluation of the DARTS algorithm using the source code. Untar it and replace the 'features' and 'models' folders. (4.7G)
  • hedging-images.tar: Images of the ILSVRC65 dataset. (4.0GB)
  • hedging-features.tar: Precomputed features of the ILSVRC65 dataset. (9.8GB)
  • hedging-models.tar: Precomputed models on the ILSVRC65 dataset. It includes cross validation models as well as models trained with various C parameters. (15.2GB)

Live Demo (EVA)

www.image-net.org/eva/