Joshua Holstein

The proposed thesis applies the concept of data-centric artificial intelligence (DCAI) in time series classification, focusing on aleatoric and epistemic uncertainty in deep learning algorithm selection.

The challenge:

In the rapidly evolving landscape of deep learning, algorithm selection emerges as a critical step in designing robust and efficient models, especially in the domain of time series classification. Time series classification benchmarks such as the UCR benchmark, applied various AI concepts, including forest-based algorithms (e.g., TS-Chief, Proximity Forest), Convolutional Neural Networks (e.g., Rocket, Darts, InceptionTime), and Transformers (e.g., ConvTran, Autoformer, FEDFormer) on hundreds of time series data sets, achieving varying performance even on individual k-folds on the same data sets.

Preliminary research:

Traditional methods such as neural architecture search, automated machine learning, as well as combined algorithm selection, and hyperparameter optimizations are effective but require considerable computational resources. In contrast to the conventional approach of identifying the model that performs best, averaged over all data sets, this research seeks to adopt a DCAI perspective. We introduce a novel data-centric fingerprint that describes any time series classification dataset and use it to estimate algorithm performance and uncertainty (see Figure above).

This thesis:

We will delve into the impact of data-centric adaptations to the fingerprint on the aleatoric and epistemic uncertainties in estimations for deep learning algorithm selection. By distinguishing between uncertainties arising from inherent data noise (aleatoric) and those attributed to incomplete data knowledge (epistemic) [1], this research seeks to refine algorithm selection more data-centric. Incorporating aleatoric and epistemic uncertainty into algorithm selection allows to account for inherent data variability and knowledge gaps, respectively, to identify which algorithms can reliably predict even under ambiguous data and how to enhance the dataset for improved performance. Data-centric adaptations include but are not limited to the inclusion of additional benchmark datasets and the refinement of dataset characteristics.

We look forward to receiving your application because:

• You are interested in the field of Data-Centric Artificial Intelligence.
• You have expertise in Python (e.g., PyTorch, Tensorflow, SciPy, sktime)
• You have a strong background in Deep Learning and Time Series Analysis (e.g., TS-Chief, Rocket, InceptionTime), or you are willing to rapidly acquire new skills in this domain.
• You are highly motivated to work on recent real-world problems in a self-organized and goal-oriented manner and you bring in your ideas.
• You have proficiency in English, encompassing writing, reading, and speaking.

We offer you a challenging research topic, close supervision, and the opportunity to develop practical and theoretical skills. If you are interested, send your CV, transcript of records, and a brief letter of motivation to Joshua.Holstein@kit.edu.

Recommended introductory literature:

• Jakubik, J., Vössing, M., Kühl, N., Walk, J., & Satzger, G. (2022). Data-centric artificial intelligence. arXiv preprint arXiv:2212.11854.
• Dau, H. A., Bagnall, A., Kamgar, K., Yeh, C. C. M., Zhu, Y., Gharghabi, S. & Keogh, E. (2019). The UCR time series archive. IEEE/CAA Journal of Automatica Sinica, 6(6), 1293-1305.
• Kendall, Alex, and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems 30 (2017).
• Hüllermeier, E., Waegeman, W. Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Mach Learn 110, 457–506 (2021).