Joshua Holstein

Motivation:

In today's information-rich world, organizations across various domains rely heavily on documentation to capture and share knowledge, guide decision-making, and support problem-solving processes. However, as the volume and complexity of this documentation grow, users often struggle to quickly locate and apply the most relevant information for their specific needs. This challenge is particularly amplified for novice users who may lack the domain expertise to effectively navigate and understand the available resources.

Retrieval-Augmented Generation-based AI assistants have emerged as promising tools for bridging these knowledge gaps by leveraging the power of natural language processing and information retrieval. These systems can help users navigate vast amounts of unstructured data and provide contextually relevant answers to their queries. However, the integration of multimodal data, such as images and diagrams commonly found in technical documentation, remains a significant challenge. Incorporating visual information alongside text has the potential to enhance the accessibility and comprehension of complex concepts for users with varying levels of expertise.

Furthermore, the successful deployment and adoption of advanced AI systems in real-world settings is often hindered by factors such as user acceptance, trust, and perceived usefulness. To realize the full potential of RAG-based AI assistants, it is crucial to investigate and address these factors in the context of specific domains and use cases. One such domain where the challenges of knowledge management and problem-solving are particularly evident is manufacturing. Equipment breakdowns can lead to costly downtime and production delays, and effective troubleshooting relies on the availability and accessibility of relevant information, often found in various forms of documentation such as manuals, guides, and standard operating procedures. Operators and maintenance personnel must quickly locate and apply the necessary knowledge to minimize the impact of breakdowns, but the sheer volume and complexity of the available resources can make this a challenging task.

Objectives:

This thesis aims to address these challenges in collaboration with a manufacturing site of Bayer AG, focusing on leveraging the richness and diversity of manuals for troubleshooting manufacturing lines in case of breakdowns. The research will be conducted through two primary streams:

Extend an existing RAG-based AI assistant prototype by incorporating image retrieval capabilities and developing methods for highlighting contextually relevant parts of retrieved images. This enhancement seeks to improve the accessibility and comprehension of troubleshooting information for users with varying levels of domain expertise.

Evaluate the adoption and effectiveness of the enhanced prototype when deployed in real-world manufacturing settings, focusing on user acceptance, usage patterns, and the system's impact on task performance and decision-making processes during equipment breakdown scenarios.

Methodology:

The research will employ a mixed-methods approach, combining qualitative and quantitative techniques:

• Literature review: Conduct a comprehensive review of existing research on RAG-based AI assistants, multimodal information retrieval, and the adoption of AI systems in various domains, with a focus on manufacturing settings.
• System development: Extend the existing RAG-based AI assistant prototype to incorporate image retrieval capabilities and develop methods for highlighting contextually relevant parts of retrieved images.
• User studies: Conduct user studies with operators and maintenance personnel at the Bayer AG manufacturing site to evaluate the enhanced prototype's usability, user acceptance, and perceived usefulness in real-world troubleshooting scenarios.
• Quantitative analysis: Analyze usage patterns, task performance metrics, and decision-making processes to assess the system's impact on troubleshooting effectiveness and efficiency.
• Qualitative analysis: Conduct interviews and focus groups with users to gather insights on their experiences, challenges, and recommendations for further improvement.

We look forward to receiving your application because:

• You are interested in Generative AI and willing to further develop our prototype.
• You are highly motivated to work on recent real-world problems in a self-organized and goal-oriented manner and you bring in your ideas.
• You have proficiency in English, encompassing writing, reading, and speaking.

We offer you a challenging research topic, close supervision, and the opportunity to develop practical and theoretical skills. If you are interested, send your CV, transcript of records, and a brief letter of motivation to Joshua.Holstein@kit.edu