An Innovative Solution to Design Problems: Applying the Chain-of-Thought Technique to Integrate LLM-Based Agents With Concept Generation Methods

Abstract

To enhance the application capabilities of large language models (LLMs) in conceptual design, this study explores how to achieve deep integration between LLM-based agents and concept generation methods using the chain-of-thought (CoT) technique and evaluates its feasibility. Using GPT-4 as a case study, we designed two agents: IntelliStorm (based on the unstructured brainstorming method) and EvoluTRIZ (based on the structured TRIZ method). Thirty participants were recruited, and through two experimental phases spaced one month apart, a comparative analysis of the effects of collaboration groups (human-agent vs. human-human) and concept generation methods (brainstorming vs. TRIZ) on participants’ physiological activation and creative thinking performance were conducted. The results show that the involvement of LLM-based agents can effectively reduce participants’ electrodermal activity(EDA) response levels, indicating a reduction in cognitive load. Moreover, participants maintained their distinct physiological patterns and performance advantages across different concept generation methods. For example, IntelliStorm, like brainstorming, evokes stronger responses to information stimuli, demonstrating superior thinking fluency; EvoluTRIZ, like the TRIZ, exhibits a higher frequency of information responses, showcasing enhanced thinking elaboration. However, originality tends to favor human-human collaboration. The findings confirm that integrating LLMs with traditional concept generation methods is an effective strategy made possible by combining CoT and retrieval-augmented generation (RAG) technologies. In the future, LLM-based agents are expected to achieve broader application in the design field by incorporating additional concept generation methods.

Keywords

• LLM-based agent
• chain-of-thought fine-tuning
• concept generation method
• EDA
• human-agent collaboration
• human-human collaboration