Uncertainties of Latent Representations in Computer Vision

DSpace Repository


Dateien:

URI: http://hdl.handle.net/10900/156771
http://nbn-resolving.de/urn:nbn:de:bsz:21-dspace-1567713
http://dx.doi.org/10.15496/publikation-98103
Dokumentart: PhDThesis
Date: 2024-08-14
Language: English
Faculty: 7 Mathematisch-Naturwissenschaftliche Fakultät
Department: Informatik
Advisor: Kasneci, Enkelejda (Prof. Dr.)
Day of Oral Examination: 2024-06-17
DDC Classifikation: 004 - Data processing and computer science
510 - Mathematics
Keywords: Maschinelles Lernen , Wahrscheinlichkeitstheorie , Maschinelles Sehen
Other Keywords:
Probabilistic Embeddings
Latent Representations
Embeddings
License: http://tobias-lib.uni-tuebingen.de/doku/lic_mit_pod.php?la=de http://tobias-lib.uni-tuebingen.de/doku/lic_mit_pod.php?la=en
Order a printed copy: Print-on-Demand
Show full item record

Abstract:

Uncertainty quantification is a key pillar of trustworthy machine learning. It enables safe reactions under unsafe inputs, like predicting only when the machine learning model detects sufficient evidence, discarding anomalous data, or emitting warnings when an error is likely to be inbound. This is particularly crucial in safety-critical areas like medical image classification or self-driving cars. Despite the plethora of proposed uncertainty quantification methods achieving increasingly higher scores on performance benchmarks, uncertainty estimates are often shied away from in practice. Many machine learning projects start from pretrained latent representations that come without uncertainty estimates. Uncertainties would need to be trained by practitioners on their own, which is notoriously difficult and resource-intense. This thesis makes uncertainty estimates easily accessible by adding them to the latent representation vectors of pretrained computer vision models. Besides proposing approaches rooted in probability and decision theory, such as Monte-Carlo InfoNCE (MCInfoNCE) and loss prediction, we delve into both theoretical and empirical questions. We show that these unobservable uncertainties about unobservable latent representations are indeed provably correct. We also provide an uncertainty-aware representation learning (URL) benchmark to compare these unobservables against observable ground-truths. Finally, we compile our findings to pretrain lightweight representation uncertainties on large-scale computer vision models that transfer to unseen datasets in a zero-shot manner. Our findings do not only advance the current theoretical understanding of uncertainties over latent variables, but also facilitate the access to uncertainty quantification for future researchers inside and outside the field. As downloadable starting points, our pretrained representation uncertainties enable a range of novel practical tasks for straightforward but trustworthy machine learning.

This item appears in the following Collection(s)