Authentication of SMIs by Culturally Different Audiences: Investigation with Chat GPT-3 Prompt Engineering

Travel restrictions during the Covid-19 pandemic resulted in a substantial increase in “home, daily life scenes, and intimate content” [1], with YouTube (YT) seeing the most significant growth of any social media app among American adults, with usage jumping from 73% in 2019 to 81% in 2021 [2]. The popularity of domestic, culture-related content, even if it does not promote travel directly, indicates that destinations can benefit from the public’s interest in the day-to-day lives of the destination’s people as increased familiarity and positive associations provided by SMIs contribute to a more favorable attitude toward the country’s brand. However, the SMI’s influence is conditioned on followers’ perceptions of SMI authenticity including such factors as sincerity, trustworthiness, originality, spontaneity, and a noncommercial orientation [3, 4]. In this study, we investigate how viewers authenticate SMIs and their content and whether the strategies they employ vary between culturally different audiences such as domestic vs. international. We specifically examine the authentication aspect called existential authenticity [5].

We use online discussions of video content created by Li Ziqi (Ziqi), an SMI from Sichuan province, China, who has millions of followers around the globe. Ziqi synchronously posts her videos on YT and Weibo (WB) focusing on Chinese cultural tradition, rural way of life, handicrafts, and cooking [6]. She has over 128 videos with a total view count exceeding 2.92 billion and an average view count per video of over 29 million [6]. We examine the topics of online discussions and provide quantitative comparisons of domestic (WB) and international (YT) audiences using the interpretive capabilities of artificial intelligence, namely, Large Language Model (LLM) prompt engineering [7, 8].

In this study, we posit that the moments of existential authenticity expressed in online comments signify acceptance of SMIs as authentic. Existential authenticity is an activity-related state [5, 9] that is primarily focused on “emotions, sensations, relationships, and a sense of self” [10:680]. It acts as a “counterdose to the loss of true self” in everyday life [5:358] that is typically associated with the commercialization of mainstream Western culture [11]. Despite the argument that the sense of self is not attainable in modern society as the various roles that individuals play pull them in various directions [12], a positive aspect of the pandemic, if there was one, is that it provided ample opportunities for self-reflection [13]. Existential authenticity includes intra-personal authenticity via sensory organs as well as self-making, and social bonding with SMIs and other followers [5, 14]. Increased cultural awareness and understanding of how similar or different we are from other people and cultures are valuable learning moments and an element of existential authenticity as well. We tentatively propose that authentication via moments of self-reflection, self-discovery, and cultural comparisons will have a greater presence in the discussions on YT as the videos present a stronger “otherizing” element for international audiences, which is one of the ways people authenticate [15].

Four videos that represent the main themes of Ziqi’s videography with the largest number of comments on YT and WB were selected: (1) Using Bamboo to Make a Sofa; (2) The Life of Cotton; (3) Snacks for Spring Festival; and (4) The Scholar’s Four Jewels of China. The data was collected in April 2021. English language users on YT represented international audiences (1,852 messages) while WB users of simplified Chinese language represented the domestic audience of Mainland China (2,980 comments).

LLMs, such as GPT-3 used in this study, can leverage their general knowledge obtained in the pre-training process to infer the comments’ topics, even when the data is incomplete or ambiguous. LLMs can also infer the topic of short comments by transferring knowledge from similar domains and are robust to noise in the data. The effective implementation of LLMs relies on the process of designing and formulating effective prompts or instructions to elicit desired output from LLMs (prompt engineering) [8, 9]. The key concepts of prompt engineering are the precise setting of the context such as providing relevant facts; providing elaborate instructions; conditioning LLM behavior by providing examples; controlling for data biases; iterative refinement of LLM responses; and, finally, result validating. For example, the comments were mapped into the topics using the prompt (only the first 3 points are provided):

“goal = match YouTube review to the best fitting review topic from a list of topics”

steps = “1. Break the list of reviews onto separate reviews using the ‘\n’ symbol as a separator”;

2. For each review find two best matching review topics from the list of review topics separated by the ‘;’ sign;

3. When there are no well-matching topics, assume that the topic is ‘Other’;

We obtained 18 topics of online discussions which were further classified into four larger groups (Table 1): Admiration and a role model (8 categories), Beauty of simple life (6), China’s traditions (3), and Food and cooking (1). Validation was conducted using a stratified random sample of 360 messages; the accuracy was 97.7%. We also tracked the most endorsed comments as they indicated the shared sentiment.

Each original category was examined for the presence of themes indicating existential authenticity experienced by online users, that is, moments of self-reflection, self-discovery, personal and cultural comparisons, and social bonding. For interpretation, we select the comments with the largest number of upvotes as they indicate shared sentiment from the YT and WB audiences and minimize researchers’ bias.

On YT, original categories ##1, 3, 8, 10, and 11 (Table 2) were identified as carrying existential authenticity sentiment (we do not include the names of original categories to save space):

On WB, existential authenticity sentiment was most often expressed in categories ##3, 10, 12, and 13:

Next, we calculated the upvotes per comment ratio for each identified category (Table 2). We adjusted the YT ratio for fair comparisons, as the international audience provided considerably more comments. Following [16], we expressed the difference in sentiment as Ln(ratio_WB/Adj. Ratio_YT). Large negative numbers indicate that YT audiences reflect substantially more on a non-commercialized way of Ziqi’s life and on their own character and abilities in comparison to her skillset. Moderately high positive numbers show that WB users expressed existential authenticity mostly through nostalgia and childhood memories. They are also inspired by the videos to replicate Ziqi’s creations of traditional Chinese objects. We consider this result to support our proposition that authentication via moments of self-reflection, self-discovery, and cultural comparisons has a greater presence on YT. Instances of social bonding were limited and entirely on the WB side.