AI-powered large language models are shaping a new era of learning. Students use AI chatbots for information search and idea inspiration. However, are students’ questioning skills effective enough to interact with an AI Chatbot? This study explores the interaction between students and ChatGPT on idea generation and identifies whether participants can effectively use AI chatbots to simulate creativity for idea generation. The results indicated that, rather than discussing their idea with AI Chatbot for suggestions and recommendations to enhance the existing ideas, many students ask AI to generate more ideas without providing directions. Participants reflected that ChatGPT provided generic ideas and were unsatisfied with its creativity. They are more positive towards using the question guide, developed using SCAMPER questioning technique combined with a narrative approach by the researcher, compared to ChatGPT because the question guide enables perspective-shifting to generate ideas from a new perspective.
1 Introduction
The introduction of the Chat Generative Pre-trained Transformer (ChatGPT) in November 2022 dramatically revolutionised education. This AI-driven conversation platform, developed under the idea of a chatbot, mimics human conversation; therefore, the user does not need any training before interacting with it. ChatGPT has the learning ability that allows its database to grow while conversing with the user, and users can obtain almost everything from its enriched database. Result searching on Google is a one-off, while the interaction with ChatGPT is continuous, so users can fine-tune their criteria without repeating them. Its ability to answer open-ended questions and generate texts has made ChatGPT a popular topic in universities and schools [1, 2].
As the text generated by AI is often indistinguishable from the recent plagiarism tools, academia is concerned with its impact on their ability to evaluate students’ works fairly since students can use ChatGPT as a tool to complete their homework, and it is challenging for teachers to verify the originality of the work. Many educators have raised concerns about academic integrity and plagiarism [3, 4]. However, we lack insights on whether and under what circumstances ChatGPT has the potential to enhance students’ creativity if the attention is merely given to the impact it has on academic integrity.
Anzeige
Powerful questioning is said to help overcome fixation [5], which is a mental blockage caused by past experience, prior knowledge, or prior learning that constrain our creative minds to think of the best alternatives to solve a problem [6, 7]. Yet, the effectiveness of questioning in triggering creative thoughts depends on how questioning as a whole process is conducted to facilitate thinking [5]. In the context of AI Chatbot, particularly large language models (LLM) like ChatGPT, one can get immediate responses to their questions. One can simply ask the AI Chatbot directly to seek ideas. It is uncertain whether questioning in the context of AI Chatbot helps trigger creativity, as suggested by previous studies of questioning for creativity in other contexts [5, 8].
This paper evaluates the perceived usefulness of AI chatbots like ChatGPT as a creativity tool for idea generation and students’ questioning techniques for idea generation. A pre-post experiment was conducted to examine students’ creativity in three different settings (i.e., no tool, with an AI Chatbot as a creativity tool, with a structured question guide as a creativity tool). A creativity challenge was designed and assigned to ten master students studying hospitality management in Taiwan for the study.
To address the research question, “Is an AI Chatbot an effective tool for overcoming fixation in idea generation?” four objectives have been identified:
1.
To assess the perception of using an AI chatbot for idea generation compared to not using any tool and using a structured question guide to overcome fixation.
2.
To analyse the questioning behaviours while utilising the AI chatbot for idea generation.
3.
To evaluate the self-perception of creativity when using the AI chatbot, in contrast to not using any tool and a structured question guide for idea generation.
4.
To compare students’ self-assessed creativity with their perception of using the AI chatbot for idea generation.
This study can comprehensively explore students’ use of ChatGPT, questioning techniques, and the overall impact on creative idea generation across different experiment stages by addressing these research objectives.
Anzeige
2 Theoretical Background
2.1 Idea Generation and Fixation
All innovation starts with an idea. In business settings, an idea can be a partial or complete thought of a solution, method, or simply a concept in response to an identified problem [6, 7]. In idea generation, a major obstacle to overcome is fixation. Fixation is a form of cognitive bias that restricts one’s creative thinking [9]. What contributes to forming such a mental barrier to creativity can often be imperceptible. Even as tedious as how instruction is given for problem-solving can lead to fixation [10]. For example, offering samples or detailed description to students of how the outputs could look can lead to conformity effects, meaning that ideas being generated are already fixated or shaped to a certain type of output, even though one were instructed to develop ideas that are completely different from the samples given [6] One way to help overcome fixation is to ask questions that help challenge the status quo [11].
2.2 Fixation and Questioning
Questioning is argued to be an effective method in triggering creativity as it helps facilitate divergent and convergent thinking [5, 8]. While the former involves a mindset to come up with alternatives to conventional thinking, the latter necessitates the ability to associate remote concepts with creativity. By observing and analysing the idea-generation process of undergraduate students working on a product design project, [5] notices that different types of questions can help guide one to switch thinking modes; in particular, more divergent thinking should be encouraged through generative design questions during idea-generation to allow possibility thinking and more convergent thinking should be encouraged through deep reasoning questions during idea selection to reduce ambiguity [5]. However, formulating questions itself is already a creative act. What makes it even more challenging is that questioning is found to be more effective when it is conducted as a thinking process.
SCAMPER questioning technique has been adopted widely for idea generation, primarily through a perspective-switch of products with ten elements: Substitute, Combine, Adapt, Modify, Magnify, Minimise, Put to other use, Eliminate, Reverse, and Rearrange products [12]. Previous studies suggest that SCAMPER can improve both the quantity and quality of ideas during the divergent thinking process [13]. Yet, the development of questioning as a flow with SCAMPER is still challenging as there is no guideline on how this can be achieved. Its product-oriented nature also limits the perspective-shifting encouraged by SCAMPER. It misses the importance of perspective-shifting in terms of a person’s questioning development.
2.3 Narrative Approach for Fixation
Currently, idea generation is largely dominated by divergent thinking tools, which only rely on generating a large number of alternatives for creative problem-solving. It is said that it is best to avoid logical thinking during divergent thinking. [14] argue that existing creativity tools are created based on the assumption that our divergent thinking and convergent thinking for creativity can be completely separated, just like how a computer works. Without associative thinking, the alternatives’ quality and practicality are in question even though the quantity is there. Hence, they propose using a narrative approach to stimulate associative thinking as a flow during idea generation without having logical thinking stepped in. A narrative approach uses storytelling and creation to promote associative creative thinking. In particular, perspective-switching is effective in helping one see from a different person’s perspective for idea generation. Such a perspective switch does not only focus on viewing the product differently. Rather, it is to see the world of the users differently or to encourage one to play a different role through storytelling.
2.4 Creative Self Concept and Creativity
The role that one’s creative self-concept plays in idea generation [15, 16], their creative behaviours [17] and their creative performance [18] have been essential to creativity research. Previous studies suggest that one’s creative potential can be elevated by one’s creative self-concept, which is formed by self-belief in one’s creative ability and the importance of creativity in shaping self [19]. It is also said to be positively related to creative motivation [20]. However, to the authors’ knowledge, no research has been conducted to examine how one’s self-concept is related to the perceived usefulness of a creativity tool.
3 Methodology
3.1 Experiment and Questionnaire Design
A pre-post experiment with a post-activity survey was conducted to examine the perceived usefulness of AI Chatbot in enhancing student’s creativity when compared to two other scenarios: 1) no tool at all, and 2) with a question guide. ChatGPT was chosen for the study as it has recently become the most popular AI Chatbot among students.
At the beginning of the experiment, students were presented with a scenario sheet for participants to envision themselves in. This sheet is bilingual, containing both English and Chinese and is related to proposing creative ideas for using hotel waste towels to craft gifts to delight hotel guests. The scenario comprises four different market segments, allowing students the flexibility to propose ideas tailored to their target group(s). This experiment consists of three rounds. In the first round, students are required to propose three creative ideas without using any tools within a 10-min time frame. In the second round, they have 10 min to interact with ChatGPT before presenting three distinct creative ideas, which should differ from those in the first round. In the third round, conducted one week later, students are allocated 10 min to utilise a question guide prepared by the researchers, based on the SCAMPER technique [12] combined with a narrative-based approach for enhancing creativity [14]. Subsequently, they are expected to propose another three creative ideas that differ from the ones presented in the first two rounds. Figure 1 illustrates the research framework of this study.
At the end of the experiment, participants were asked to save the whole conversation with ChatGPT to a file and send it to the researcher for analysis. Participants were then required to fill in three questionnaires, including two post-experiment questionnaires and one creative self-concept profile. The key purpose of the post-experiment questionnaire was to obtain their perspectives on the three different creative processes: 1) No tool, 2) With ChatGPT, and 3) With a question guide. To ensure that participants’ motivation for the challenge does not influence their creative outputs in the three rounds of the challenge, post-activity questionnaires included questions regarding outcome-focused and process-focused motivation. These questions were adapted and modified from [21] to improve the validity of the findings. Creative self-concept questions were asked after all three rounds of challenge were taken. The six questions were adopted from [19, 20] to evaluate one’s self-creative concept (see Appendix A).
Fig. 1.
The Research Framework
×
3.2 Question Guide Design
A question guide was devised to assess the effectiveness of questioning in two distinct contexts: questioning with immediate responses (e.g., ChatGPT) and questioning without immediate responses (e.g., self-questioning). This guide was constructed utilising the SCAMPER technique as its foundation. A narrative approach was employed to adapt and enhance the technique to address the potential limitation of the SCAMPER technique, which may not naturally incorporate questioning. Its perspective-shifting that focuses on the product can also be overcome by instilling a guest-focused narrative while using SCAMPER to help develop a question guide. In other words, the question guide was developed based on SCAMPER with a narrative approach to guide students to imagine themselves as if they were the guests and how their “products” interplay with guests’ needs and guest experience as a whole.
During the development of the question guide, several rules were applied, drawing from the work of [5, 10]. First, the questions were designed to encompass both divergent and convergent elements to enhance the originality and relevance of generated ideas. Divergent questions were predominantly used at the outset, followed by an increased emphasis on convergent questions later in the process. Second, the questions were structured to build upon one another, creating a coherent narrative for idea generation. This approach ensured that the questions were not treated as isolated entities but integral parts of a continuous thought process. Third, more abstract categories were employed in the questions to avoid prematurely narrowing down possibilities. For instance, the initial questions encouraged students to contemplate the overall “experience” of the guests rather than focusing narrowly on the old towel at the beginning of the questioning process.
After incorporating the perspective-shifting and guest-focused narrative, the question guide was designed to inspire participants to think from different perspectives and put themselves into different scenarios. The finalised question guide for this experiment consists of eight questions:
1.
Think of a typical guest staying at this hotel. Think of what this person has been through and the environment the person is in. If you were this guest, what frustrates you the most at this moment?
2.
As this guest, if your typical experience at a hotel can be completely rearranged to delight you, what arrangement do you want it to be like?
3.
If someone were to suddenly take away an essential part of the new arrangement, what would that be? How will the whole experience become for you？
4.
If you were to use the old towels to replace the missing part, what would the experience be like for you?
5.
What if the old towels do not look or feel the way a typical towel does, what would they be like? How will their different look or feel change your experience?
6.
If you were to combine the old towels with another type of recycled item to re-create the missing part of the new arrangement, what would it be like? How will you experience these newly transformed wastes as a guest?
7.
Where else can you use these newly transformed wastes as a guest?
8.
Think of what hotels and cities would be like in 30 years. How will this new arrangement made possible with the transformed wastes adapt to your changing needs and the changing environment you are in?
3.3 Data Collection
Invitation emails were sent to all master’s level students in the researchers’ department. Ten students agreed to participate. The experiments were conducted on May 18 and 25, 2023, with four second year and June 7 and 15, 2023, with six first-year Master level students. At stage one, students were asked to propose three ideas without tools. At Stage Two, they were asked to propose three new ideas with the assistance of ChatGPT. After that, they fill out the post-experiment questionnaire. One week later, students were invited to participate in round three. They were given a guide with 15 questions designed to inspire them to think of three more new ideas. After the experiment, participants were asked to complete post-experiment questionnaires and a creative profile survey.
3.4 Data Analysis
Content analysis was carried out to analyse the conversation between participants and ChatGPT. A total of 81 questions were asked. One participant only asked ChatGPT one question. The highest number of questions asked was 13. These questions are further analysed into five categories (Table 1).
Table 1.
Categorisation of Conversation with ChatGPT
Categories
No. of questions
1. Questions purely ask ChatGPT to provide suggestions
20 (24.7%)
2. Assign role to ChatGPT and/or focus on a target user
25 (30.9%)
3. Discuss with ChatGPT for comments or advice
5 (6.2%)
4. Ask ChatGPT for more ideas
27 (33.3%)
5. Others
4 (4.9%)
Table 2.
Demographics of the Participants
ID
Master level
Gender
Self-reflect ChatGPT experience
CS Score
(6 to 30)
Relative
Creativity*
S1
Year 2
M
Beginner
17
Low
S2
Year 2
F
Beginner
26
High
S3
Year 2
F
Just once
19
Low
S4
Year 2
M
Advance
26
High
S5
Year 1
M
Beginner
23
Middle
S6
Year 1
M
Beginner
23
Middle
S7
Year 1
F
Beginner
23
Middle
S8
Year 1
M
Beginner
21
Middle
S9
Year 1
F
Advance
24
High
S10
Year 1
F
Just once
17
Low
* Relative creativity among the participants
Table 2 illustrates the demographics of the participants. All of them have prior experience with ChatGPT, but two mentioned that they only created an account and used it once. Meanwhile, two participants claimed they have extensive experience and use it frequently. The Creative Self-Concept score (CS score) was generated by summing up the responses to six creative self-concept questions from the creative profile. Based on their CS scores, participants were further divided into three groups to indicate their relative creativity among all participants. The relative creativity group were classified by distribute evenly based on the difference between the lowest and the highest score. Therefore, a CS score between 17 and 19 indicated relatively low creativity, while a score between 20 and 23 indicated relatively moderate creativity, and a score between 24 and 26 indicated relatively high creativity.
4 Findings and Discussions
4.1 Questioning ChatGPT
Within the 10 min, participants can interact with ChatGPT in any form and language they prefer until they get satisfactory results. At the beginning of the conversation, half of the students directly asked ChatGPT to propose ideas. “How to reuse waste towels” [S1]; “Help me to find 3 ways to reuse waste towels” [S3]; “Please help me to use waste towels to make different products” [S5]; “…using towel to reuse and transform, give examples” [S6] and “What creative product can be made by waste towels” [S10]. From the results generated by ChatGPT according to these five questions, we found that without specifying “the purpose” (for hotel to delight guests) and target users, ChatGPT provided many ideas that do not match the purpose stated in the scenario sheet. For example, cleaning cloth, wipes, kitchen towels, construction bricks, and gift wrap.
The most frequently asked question was “more ideas.“ One-third of the questions belonged to this category, such as “any newer ideas” [S5], “I need something more creative” [S6], and “Can you provide me with more ideas?” [S8], among others.
Two participants were assigned roles for ChatGPT. S4 instructed the chatbot, saying, “Your role is a fashion designer” [S9]. S4 even assigned six different roles, including recycle product designer, artist, designer, engineer, Nobel Prize winner, and Elon Musk, so that the chatbot could provide a variety of ideas. However, these questions also did not specify the purpose and the target users, resulting in generated results (e.g., biodegradable towels, recycled fibre, and garments) that did not align with the intended purpose.
S7 and S8 began their conversations by providing all the required criteria, including competition, waste towels, four market segments, and delighting the customers to ChatGPT. After the chatbot provided suggestions such as shopping bags, slippers, toiletry bags, facial cotton pads, and yoga mats, both asked the chatbot for more ideas four times before ending the conversation.
Only three participants sought advice from ChatGPT, asking questions like “Do you think… is suitable for hotel guests?” [S3], “What is the meaning of having…” [S4], and “Do you think… is feasible?” [S9]. Moreover, S9 was the only participant who engaged in humanised conversations with the chatbot. However, rather than discussing idea generation, she pressured the chatbot for more useful ideas.
Most participants were primarily focused on the outcome, a creative idea for repurposing waste towels into a product. They often overlooked the importance of considering the target market and its needs. Of the 90 ideas generated, only 13 (14%) mentioned the target users or their specific needs, while the rest focused solely on the product.
In conclusion, throughout their conversations with ChatGPT, most participants did not engage in in-depth discussions to explore their ideas further. None of them sought advice from the chatbot to refine their ideas or overcome creative fixation. Instead, they were eager to obtain creative outputs directly from ChatGPT.
4.2 Perceived Usefulness of Idea Inspiration Tools
In this study, students perceived ChatGPT as a fast and efficient tool [S10] that inspire them to think in different dimensions and discipline [S3, S6], simplify the convergent thinking process among different ideas [S7], and remind them of the discipline that was outlooked [S5]. Interestingly, some participants did not think AI Chatbot is a good tool for idea inspiration. Three of them feel that the ideas provided by ChatGPT were direct [S2], standard, generic [S5] and non-creative [S8]. S9 point out that whenever similar question and request were inputted, ChatGPT provided similar answers. Therefore, the answers were not creative at all.
For the perceived usefulness of the question guide, all Year 2 students indicated it restricted their imagination and exploration of new ideas [S1, S2, S3]. S4 agreed that the question guide “can alter thinking logic but did not strongly impact the final outcome”. On the other hand, Year 1 students have completely different perspectives. They feel that the question guide can guide them to think in different dimensions [S6] and lead to a creative direction [S8]. The question guide offered a story set so S5 could understand the target user’s needs and have a clear direction to think of good ideas. S9 pointed out that the question guide helped her to explore more ideas without barriers and “visualise” the situation [S10].
4.3 The Proposed Creative Ideas
Participants were asked to self-evaluate the creativity of the ideas they proposed in each round using a 5-point Likert scale, where 1 indicated “strongly disagree” and 5 indicated “strongly agree.“ The ANOVA test results showed that individuals with high self-creative concept scores felt significantly more creative (mean = 5.0; STD = 0; F = 13.3; p = 0.004) when suggesting ideas in Round 1 compared to those with medium (mean = 3.75, STD = 0.5) and low creative profile scores (mean = 4.0; STD = 0). However, participants with high self-creative concept scores perceived that the ideas they proposed in Round 3, after using the inspiration tools (ChatGPT [F = 0.7; p = 0.528] and question guide [F = 0.122; p = 0.887]), were not as creative as those in Round 1.
At the end of the second questionnaire, participants were asked to select the three best ideas from the nine they had proposed and express their perceived usefulness of the two inspiration tools. Eleven ideas were from both Round 1 (no tool) and Round 3 (question guide), while six ideas were from Round 2 (ChatGPT).
The results revealed two intriguing phenomena. Firstly, participants favoured the ideas they had generated over those AI recommended. Secondly, despite the expectation that ChatGPT would outperform other tools, participants did not consider the ideas proposed by the chatbot to be creative enough to make them the top three choices.
5 Conclusions
Students admire the capabilities of ChatGPT and find it interesting, motivating, and helpful for study and work [22]. However, when it comes to idea generation, does it have the same effect? The paper starts by asking, “Can AI chatbots help overcome fixation for better idea generation?”. The findings suggest that AI Chatbot has the potential to help expand one’s thinking and knowledge for creative problem-solving. However, its evocativeness, which refers to the capacity of a learning activity or material to give rise to “personal thought” [23], is not fully realised. When one focuses on the creative outputs instead of the creative process, AI Chatbot becomes an ineffective tool for idea generation. When it is seen as a tool to help expand one’s thinking and knowledge as a part of a creative thinking process, it is perceived as fast and effective. Hence, the perceived usefulness of AI Chatbot depends largely on the student’s capability and knowledge in asking questions that help them overcome fixation. Instead of asking divergent and convergent questions in a flow, as suggested by [5], participants of this study focus on gaining creative outputs from ChatGPT by giving orders or requests. When they were not satisfied with the creative outputs offered by ChatGPT, they simply requested “more ideas” instead of asking various questions. Students were found to be fixated on their own questions while using ChatGPT for idea generation.
Asking questions as a form of interactive learning has a long history, dating back to the teachings of Socrates. Interacting with AI chatbots is made easy and engaging through its natural language conversations. Through conversation, questions scaffold learning and promote awareness and thought. AI chatbots’ conversational format enables students to exchange questions and answers, leading to deeper personal reflection. Sadly, students cannot take full advantage of technological advancement as they are rarely taught how to ask questions or even given a chance to ask questions in the classrooms.
This study also finds that the creative self-concept of the students has an impact on the perceived usefulness of ChatGPT in idea generation. Those more confident in their creative ability tend to see ChatGPT as less useful. They prefer generating ideas on their own or with a question guide. Instead of seeing ChatGPT as a tool to help modify or expand their original ideas, they simply prefer not to use it at all. This seems to suggest that those with higher creative self-concept do not see AI chatbots as what helps overcome fixation. Rather, it creates fixation by offering non-creative and similar answers.
The theoretical contribution of this study is twofold. While most studies adopt an outcome-based approach in the evaluation of the effectiveness of an intervention, this study also takes into consideration the user’s perspective as well as their creative concept as an indicator of the effectiveness of a creativity intervention [24]. This study is also among the very few studies to examine the impact of AI Chatbot technology on one’s creativity and questioning technique. In terms of practical contribution, this study sheds light on whether and how AI chatbots and large language models can be adopted to enhance students’ creativity.
Creativity has become one of the most sought-after skills. Nevertheless, schools have not effectively cultivated their students’ creative potential [25]. Students are often expected to apply their creativity skills for idea generation and selection in project-based learning, but they are rarely taught how to do so in the classrooms. Students are often left alone and feel most anxious about generating ideas for creative problem-solving [26]. Now, with the popularisation of AI chatbots, it is unavoidable that students will rely on AI chatbots to help generate ideas to solve problems. Therefore, it is of foremost urgency for educators to explore how students can take full advantage of the technology instead of restricting its usage.
This study has two major limitations. First, the sample size is notably small, with participants from the same university with similar educational backgrounds. Consequently, the findings may not be generalisable to contexts outside this demographic. Second, data collection occurred approximately two months after the launch of ChatGPT in Taiwan with Chinese large language model. Given this relatively short time frame, most participants were still relatively new to ChatGPT, which could have impacted their familiarity and comfort with the tool. Therefore, they may not have been fully adept at utilising the chatbot to its fullest potential during the study.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
