Out-of-Distribution Learning

Summary

Out-of-distribution learning is concerned with developing methods where the distribution of the data processed at test time may be different from the distribution of the data used during the training of a model. These techniques go beyond the simple fine-tuning of models. They include techniques such as domain adaptation (Keaton et al., 2023) (Motiian et al., 2017) (Motiian et al., 2017), where models undergo additional training with special losses to adapt to novel training data distributions. OOD techniques aim to go beyong domain generalization (Motiian et al., 2017) where models are made robust to any distortion of the training data distribution, and attempt to produce models that can generalize the original task to something new, sometimes in a compositional fashion (Yao & Doretto, 2010) by leveraging what the model has learned so far. We apply these techniques to several computer vision tasks, including object detection, objects clssification, medical imaging applications, and immersive analytics.

Related Projects

Transfer Learning from Multiple Sources

2 minute read

Transfer learning allows leveraging the knowledge of source domains, available a priori, to help training a classifier for a target domain, where the availab...

Generative Probabilistic Novelty Detection with Adversarial Autoencoders

2 minute read

Stanislav Pidhorskyi, Ranya Almohsen, Donald A Adjeroh, Gianfranco Doretto

References

1. WACV

   CellTranspose: Few-shot Domain Adaptation for Cellular Instance Segmentation Keaton, M. R., Zaveri, R. J., and Doretto, G. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023. abstract bibTeX arXiv pdf

   Automated cellular instance segmentation is a process utilized for accelerating biological research for the past two decades, and recent advancements have produced higher quality results with less effort from the biologist. Most current endeavors focus on completely cutting the researcher out of the picture by generating highly generalized models. However, these models invariably fail when faced with novel data, distributed differently than the ones used for training. Rather than approaching the problem with methods that presume the availability of large amounts of target data and computing power for retraining, in this work we address the even greater challenge of designing an approach that requires minimal amounts of new annotated data as well as training time. We do so by designing specialized contrastive losses that leverage the few annotated samples very efficiently. A large set of results show that 3 to 5 annotations lead to models with accuracy that: 1) significantly mitigate the covariate shift effects; 2) matches or surpasses other adaptation methods; 3) even approaches methods that have been fully retrained on the target distribution. The adaptation training is only a few minutes, paving a path towards a balance between model performance, computing requirements and expert-level annotation needs.

   @inproceedings{keatonZD23wacv,
     abbr = {WACV},
     author = {Keaton, M. R. and Zaveri, R. J. and Doretto, G.},
     booktitle = {{Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision}},
     title = {{CellTranspose: Few-shot Domain Adaptation for Cellular Instance Segmentation}},
     year = {2023},
     month = jan,
     pages = {455--466},
     publisher = {IEEE},
     bib2html_pubtype = {Conferences},
     arxiv = {2212.14121}
   }
   

2. NIPS

   Few-shot adversarial domain adaptation Motiian, S., Jones, Q., Iranmanesh, S. M., and Doretto, G. In Advances in Neural Information Processing Systems, 2017. abstract bibTeX arXiv pdf

   This work provides a framework for addressing the problem of supervised domain adaptation with deep models. The main idea is to exploit adversarial learning to learn an embedded subspace that simultaneously maximizes the confusion between two domains while semantically aligning their embedding. The supervised setting becomes attractive especially when there are only a few target data samples that need to be labeled. In this few-shot learning scenario, alignment and separation of semantic probability distributions is difficult because of the lack of data. We found that by carefully designing a training scheme whereby the typical binary adversarial discriminator is augmented to distinguish between four different classes, it is possible to effectively address the supervised adaptation problem. In addition, the approach has a high “speed” of adaptation, i.e. it requires an extremely low number of labeled target training samples, even one per category can be effective. We then extensively compare this approach to the state of the art in domain adaptation in two experiments: one using datasets for handwritten digit recognition, and one using datasets for visual object recognition.

   @inproceedings{motiianJID17nips,
     abbr = {NIPS},
     author = {Motiian, S. and Jones, Q. and Iranmanesh, S. M. and Doretto, G.},
     title = {Few-shot adversarial domain adaptation},
     booktitle = {Advances in Neural Information Processing Systems},
     year = {2017},
     pages = {6652--6662},
     arxiv = {1711.02536},
     bib2html_pubtype = {Conferences}
   }
   

3. ICCV

   Unified deep supervised domain adaptation andgeneralization Motiian, S., Piccirilli, M., and Adjeroh, G. In Proceedings of IEEE International Conference on Computer Vision, 2017. bibTeX arXiv pdf webpage code

   @inproceedings{motiianPAD17iccv,
     abbr = {ICCV},
     author = {Motiian, S. and Piccirilli, M. and Adjeroh, D. adn Doretto, G.},
     title = {Unified deep supervised domain adaptation andgeneralization},
     booktitle = {Proceedings of IEEE International Conference on Computer Vision},
     year = {2017},
     pages = {5715--5725},
     arxiv = {1709.10190},
     code = {https://github.com/samotiian/CCSA},
     bib2html_pubtype = {Conferences},
     webpage = {{https://vision.csee.wvu.edu/~motiian/Details/CCSA.html}}
   }
   

4. CVPR

   Boosting for transfer learning with multiple sources Yao, Y., and Doretto, G. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010. abstract bibTeX pdf

   Transfer learning allows leveraging the knowledge of source domains, available a priori, to help training a classi- fier for a target domain, where the available data is scarce. The effectiveness of the transfer is affected by the relation- ship between source and target. Rather than improving the learning, brute force leveraging of a source poorly related to the target may decrease the classifier performance. One strategy to reduce this negative transfer is to import knowl- edge from multiple sources to increase the chance of find- ing one source closely related to the target. This work ex- tends the boosting framework for transferring knowledge from multiple sources. Two new algorithms, MultiSource- TrAdaBoost, and TaskTrAdaBoost, are introduced, analyzed, and applied for object category recognition and specific ob- ject detection. The experiments demonstrate their improved performance by greatly reducing the negative transfer as the number of sources increases. TaskTrAdaBoost is a fast algorithm enabling rapid retraining over new targets.

   @inproceedings{yaoD10cvpr,
     abbr = {CVPR},
     author = {Yao, Y. and Doretto, G.},
     title = {Boosting for transfer learning with multiple sources},
     booktitle = {Proceedings of the IEEE Computer Society Conference on Computer Vision
     	and Pattern Recognition},
     year = {2010},
     bib2html_pubtype = {Conferences},
     bib2html_rescat = {Object Classification},
     file = {yaoD10cvpr.pdf:doretto\\conference\\yaoD10cvpr.pdf:PDF;wangDSRT07iccv.pdf:doretto\\conference\\wangDSRT07iccv.pdf:PDF},
     owner = {doretto},
     timestamp = {2008.01.19}
   }