LLM as a Judge for Business Ideas: Evaluate Business Ideas with LLMs

Background

Recent advances have made it possible to distribute Gen-AI systems to a wide range of customers. AI systems as assistants, such as ChatGPT, Claude or Gemini, are being used increasingly in everyday life and at work. While their capabilities are growing rapidly, we are integrating them more and more into our daily lives. One usage for them is to boost creativity of individuals. Recent works have demonstrated that human-AI teams can be more creative, especially in creating new business ideas.

While new AI tools can boost creativity and accelerate the creative process, this advancement creates the need to evaluate the new number of ideas. Recent creative studies focus on human evaluation of human-AI team efforts. Another research stream is the usage of LLMs as a Judge to evaluate, classify or analyze large amounts of text.


Research Goal

The aim of this research is to investigate how LLMs can be used to evaluate business ideas and plans. As a starting point, suitable metrics have to be identified from current literature. Based on the found metrics, different methods to evaluate business ideas with LLMs should be implemented and experimented with. To evaluate the created approaches, the LLM evaluations should be compared to human evaluations, which should be gathered through a survey or interviews.

Your Profile

  • You are interested in the emerging field of gen-AI
  • You are interested in topics around natural language processing
  • You have experience with python and common frameworks for AI
  • You are highly motivated to work on recent real-world problems in a self-organized and goal-oriented working mode, and you bring in your own ideas
  • Very good English skills as the thesis will be written in English

Details

  • Start: Immediately
  • Language: English

We offer you an exciting research topic, close supervision, and the opportunity to develop practical as well as theoretical skills. If you are interested, please send a current transcript of records, a short CV, and a brief motivation (2-3 sentences) to Jonas Liebschner jonas.liebschner∂kit.edu