User-generated content (UGC) created and distributed through social media and tourism-related websites provides potential travelers the opportunity to gain first-hand experiences about destination products and services. UGC is also of great value to tourism service providers. Since UGC represents customers’ opinions and experience outcomes, potential problems, but also drivers behind customer delight can be identified. In this regard, also temporal changes regarding customer requirements can be determined. The aim of this paper is to identify how certain topic areas mentioned in UGC affect customer satisfaction, exemplarily analyzed for the Lake of Constance Region. Furthermore, potential temporal changes regarding customer satisfaction since the outbreak of the COVID-19 pandemic will be examined. A sentiment analysis, topic detection and regression analysis are carried out on two datasets containing UGC before and after the outbreak of the pandemic. Findings show that the pandemic has changed customers’ attitudes towards certain topic areas.
1 Introduction
The Internet has fundamentally changed the way tourism-related information is processed and distributed. Before the introduction of the Internet, experiences about products and services were not readily available. Such content however is now widely accessible in the form of user-generated content (UGC) through social media platforms or dedicated tourism websites [1]. Most importantly, such content enjoys higher credibility among customers due to the lack of commercial self-interest, compared to commercial sources like travel agencies or accommodation providers, and is therefore increasingly used in the planning of upcoming trips [2].
Not only for end users, but also for providers of tourism services, such UGC is of immense value. Since UGC represents the opinions and experiences of customers, problem areas can be uncovered to make tourism planning activities and strategies more customer-oriented and thus effective [3]. More precisely, from 2014 to 2022, the total number of user reviews and opinions on Tripadvisor, one of the most renowned travel review websites, experienced a fivefold increase. Today, Tripadvisor encompasses over 1 billion online customer reviews just for the tourism business domain [4]. The fact that Tripadvisor was one of the most visited travel and tourism websites worldwide in January 2023 [5] undermines tourists’ interest in posting and reading online travel tips and comments. Based on these facts it is of great importance to emphasize the significance of UGC for tourism service providers. In this paper, UGC is used to perform an analysis regarding potential temporal changes in customer satisfaction after the outbreak of the pandemic. Compared to other industries, tourism stands out as a highly volatile business [6]. Furthermore, crises such as SARS or COVID-19 lead to irregular and even dramatic fluctuations in tourism demand, as well as potential temporal changes in customer requirements and needs [7]. This greatly threatens the existence of companies in the tourism sector and challenges e-tourism science as whole [8, 9].
Anzeige
User-generated content (UGC), especially in the form of online reviews on platforms like Tripadvisor, is increasingly used by tourists to provide feedback during or shortly after their trip [10]. Since UGC represents the opinions and experiences of customers, it can be also used to identify specific factors describing how the fulfillment or non-fulfillment of certain topic areas affects customer satisfaction [11]. By analyzing UGC over specific time periods (e.g., before and after the outbreak of the COVID-19 pandemic), differences can be identified that represent changes of the drivers behind tourist satisfaction. Accordingly, the analysis process of this study, first, makes use of a topic detection and sentiment analysis to identify how often a certain topic is mentioned positively or negatively and, second, examines through linear regression which topic has a strong positive or negative influence on overall customer satisfaction. The following research questions are to be answered: (1) can text mining techniques be used to uncover concrete factors affecting customer satisfaction and (2) are these factors showing changes since the outbreak of the COVID-19 pandemic?
Our paper is structured as follows: First, a literature review is presented which shows general procedures for the implementation of topic detection and sentiment analyses for the analysis of UGC making use of text-mining methods in tourism. This is followed by a presentation of the methodology used to identify the positive and negative influences on customer satisfaction for a number of already pre-defined topic areas. Subsequently, these results are presented for both time periods and the differences between the two periods are highlighted. Finally, a discussion of future work and improvements to the methodological approach are presented.
2 Related Work
2.1 Topic Detection
Topic detection, i.e. the automated process of identifying and classifying themes or patterns in text, is widely used in tourism. Ahani et al. [12] evaluated medical travelers’ satisfaction through the analysis of online reviews. By making use of Latent Dirichlet Allocation (LDA), main topic areas have been discovered from such medical tourism reviews. Menner et al. [13] presented an approach to extract topics from UGC. They used methods such as clustering, Latent Semantic Analysis (LSA) and Named-Entity-Recognition (NER) and compared these methods in terms of performance. They found out that NER performed best in identifying topics from UGC. Xiang et al. [14] conducted a comparative analysis of the online review platforms Tripadvisor, Expedia and Yelp. The main topics related to consumers experience have been discovered using LDA. Likewise, Schmunk et al. [15] conducted a topic detection based on online reviews from TripAdvisor, using Support Vector Machines (SVM), the Naive Bayes algorithm, k-NN, and lexicon-based methods. The best results could be achieved using SVM and lexicon-based approaches.
2.2 Sentiment Analysis
Sentiment analysis, i.e., the process of determining and categorizing opinions or emotions expressed in texts, is a well-known method for analyzing tourism reviews [3, 16]. Shi et al. [17] implemented a sentiment analysis for hotel reviews using SVM and achieved an accuracy of 85.2%. Sodanil [18] conducted a sentiment analysis for hotel reviews in various languages. For this purpose, both Thai and English reviews were extracted. Compared with Naïve Bayes and Decision Trees, SVM achieved most accurate sentiment results. In a sentiment analysis for online reviews conducted by Pang et al. [19], the Naive Bayes Classification yielded worst results, while SVM performed best. Garcia et al. [20] presented a lexicon-based approach to implement a sentiment analysis of online reviews for the tourism sector. Their approach uses a self-created annotated lexicon based on positive and negative words appearing within the reviews.
Anzeige
2.3 Comprehensive Analysis of User Generated Content
Finally, approaches that combine the above methods to comprehensively evaluate user-generated content are presented [2]. Ali et al. [21] developed a methodology that combines aspects of topic detection and sentiment analysis to extract valuable insights about the city of Marrakech. This approach ranges from the extraction of UGC to the identification of latent topics using LDA, as well as the application of sentiment analysis for each of these topics. Similarly, Garcia et al. [22] analyzed tweets related to content with the keyword COVID-19. For this purpose, topic detection and sentiment analysis methods were subsequently used in combination. It was found that most posts related to the pandemic are characterized with a negative sentiment. Schmunk et al. [15] presented an approach for extracting decision-relevant information from hotel reviews. These reviews were collected from websites such as TripAdvisor and subjected to the methods of topic detection, subjectivity classification and sentiment analysis. For this aim, a range of different methods were employed and compared in terms of their performance. It was concluded that both SVM and lexicon-based approaches achieved the best results in terms of accuracy.
While the aforementioned works mainly used methods of topic detection and sentiment analysis, this study will combine such methods with a consecutive regression analysis [11]. By doing so, concrete factors can be identified to explain how UGC-extracted topics affect customer satisfaction, both in case of fulfillment and non-fulfillment. Moreover, such compound analyses will be conducted on UGC data extracted for a time period before and after the outbreak of the COVD-19 pandemic, with the aim of highlighting major differences.
3 Method
3.1 Data Collection
This study is based on UGC that was extracted from the website TripAdvisor. By using a web crawler, hotel reviews were extracted during the period 2018–2023 and regarding the five city destinations Konstanz, Lindau, Friedrichshafen, St. Gallen, and Bregenz, all belonging to the Lake of Constance region located at the crossing borders of Germany, Austria, and Switzerland. The selection of these cities results from their importance as prominent tourist centers of the lake of constance region. The dataset consists of 989 reviews from 2018–2019 and 388 reviews from 2020–2023, extracted from 83 hotels in the aforementioned cities. Both datasets include 6 attributes, containing information regarding the hotel, the review title, the review text, the review date and, finally, the user rating on a scale from 1–5. The reduced number of reviews for the 2020–2023 period reflects the post-pandemic decrease of tourism demand.
3.2 Data Preparation
During data extraction, associated HTML-elements were removed and each hotel review has been split into its corresponding sentences since both the methods of topic detection and sentiment analysis are implemented at the sentence level in order to enable a more specific identification of topics. Further steps of data preparation included common tasks, such as tokenization, removal of stopwords and stemming. In numbers, the 989 reviews from 2018–2019 have been split into a total of 7,202 sentences and the 388 reviews from 2020–2023 into 2,712 sentences.
3.3 Topic Detection
After having prepared the extracted UGC data, each sentence is subject to a topic detection analysis. More concretely, this analysis step comprises the automated process of identifying and classifying topics or patterns in UGC-based texts. The topic detection was conducted by a lexicon-based approach. For this regard, wordlists were created that contain words that are typically associated with certain topic areas of the hotel by extending the wordlists used in previous studies [15, 16] and adding new topics such as Booking & Check-In. Using these word lists, each sentence can now be automatically examined to determine if it contains one or more words associated with these pre-defined topic areas. Obviously, the aim is to identify a specific topic area for each entry which best describes its content. There are five predefined topic areas: ‘Location & Property’, ‘Food & Beverages’, ‘Room & Bathroom’, ‘Service & Staff’ and ‘Booking & Check-In’. Each topic area has its own set of words associated with it. If no such words could be identified within a sentence, it is assigned to the residual class N/A (not assignable). An example is the entry “I will definitely return!”, which does not belong to one of the predefined topics and, thus, was assigned to the residual class N/A. Figure 1. Summarizes the sequence of steps performed during this process. By this topic detection process, each entry of both datasets is now classified as either one of the pre-defined topics or as the residual class N/A.
Fig. 1.
Lexicon-based topic detection
×
3.4 Sentiment Analysis
The sentiment analysis is executed by a lexicon-based approach, as well. In this context, the wordlists by Hu and Liu [23] are used to categorize each sentence into the sentiment categories ‘positive’, ‘negative’ or ‘neutral’. These wordlists comprise around 7,000 opinion words. Again, each sentence is automatically examined regarding its content to determine whether it contains one or more positive or negative words contained in the word list. More concrete, in case more positive than negative words could be identified within a sentence, it is assigned to the sentiment positive and vice versa. Instead, if an entry does not contain any of the 7,000 opinion words, it is classified as neutral. This lexicon-based sentiment analysis procedure is summarized in Fig. 2.
Fig. 2.
Lexicon-based sentiment analysis
×
3.5 Regression Analysis
After assigning both a topic and a sentiment to each sentence, this information is now used to feed a linear regression in order to identify and quantify the effect of positive and negative feedback related to certain topic areas on overall customer satisfaction. In contrast to the previous procedures however, this analysis is carried out at the review-level, which is why each individual sentence is traced back to its original review. For each review, it is then counted how many positive or negative sentences for each topic are contained (see Fig. 3). This information is then used to be regressed on overall customer satisfaction.
Fig. 3.
Transformation of dataset for linear regression
×
Based on the above-described datasets, regression analyses were performed separately for both time periods, i.e., 2018–2019 and 2020–2023, respectively. By comparing these two regression models, interesting differences can be detected which describe changes in customers’ preferences for each of these pre-defined topic areas.
4 Findings
In the following section, the findings from the regression models for both time periods are presented. A validation of the conducted topic detection and sentiment analysis was undergone by manually annotating 25% of the sentences in terms of their corresponding topic and sentiment as test dataset. In total, 2,478 sentences have been labeled, allowing to measure the performance of both analytical methods. As outcome, the lexicon-based topic detection could achieve an accuracy of 80.39%, while the lexicon-based sentiment analysis reached an accuracy of 77.28%, both constituting satisfactory results.
4.1 Regression Model for the 2018–2019 Dataset
Below, the regression model based on the dataset before the outbreak of the COVID-19 pandemic (i.e., 2018–2019) is presented (Fig. 4).
Fig. 4.
Regression model pre-COVID
×
Notably, both the topic areas ‘Room & Bathroom’ and ‘Booking & Check-In’ are characterized as having a purely negative impact on customer satisfaction. The non-significant positive influence for both topics has been automatically removed during feature selection. This implies that a non-fulfillment of the quality expectations regarding these topics reduces customer satisfaction significantly. However, at the same time, the respective quality fulfillment would not affect total customer satisfaction positively. Interestingly, the remaining topic areas show both positive and negative impacts on total satisfaction. The topic ‘Food & Beverages’ is the only topic with a dominance of the positive influence on customer satisfaction, hence this topic area shows potential to delight the customer.
4.2 Regression Model for the 2020–2023 Dataset
The regression model based on the dataset after the outbreak of the pandemic is presented in Fig. 5. The topic area ‘Room & Bathroom’ is now characterized by both a positive and negative influence on customer satisfaction. The topic area ‘Booking & Check In’ continues to show a purely negative influence, undermining the negative effect of a bad check-in or check-out procedure on customer satisfaction. The residual topic N/A is now characterized by a purely negative influence, whereas ‘Location & Property’ and ‘Service & Staff’ still show both a positive and negative impact on customer satisfaction. Interestingly, ‘Food & Beverages’ is now characterized by a purely positive influence, thus, culinary offers show the potential to delight the customer after the pandemic. The non-significant negative influence of the topic was automatically removed during feature selection.
Fig. 5.
Regression model post-COVID
×
4.3 Comparison of Results
In this section, we discuss differences between the two regression models, i.e., before and after the Covid-19 pandemic and highlight discrepancies and possible reasons for temporal changes. Figure 6 compares the outcomes of the two regression models.
Fig. 6.
Comparison of pre- and post-COVID regression models
×
Location and Property:
This topic area is characterized by both a positive and negative influence on customer satisfaction. However, after the pandemic, these manifestations have intensified. Hence, fulfillment and non-fulfillment of quality expectations regarding this topic area now have a stronger effect on customer satisfaction. After extended periods of isolation or quarantine, individuals may place heightened emphasis on the value of their vacations or travels. Consequently, there could be an augmented focus on the quality of their surroundings, including the location of the hotel.
Booking and Check-In:
Interestingly, this topic area is characterized by a purely negative influence on customer satisfaction, for both time periods. This most likely corroborates the negative effect a bad check-in or check-out process has on customer satisfaction. However, the negative impact of this topic area in case of non-fulfillment has intensified after the pandemic. Especially in times of increased uncertainty and potential health risks, customers seem to have developed a greater appreciation for processes that are simple, clear and safe. A cumbersome or confusing booking and check-in process could therefore be perceived as particularly disruptive, especially after a longer period of isolation.
Service and Staff:
Interestingly, the topic area ‘Service and Staff’ is the only one that has remained largely unchanged in terms of its characteristic effect on total customer satisfaction. This indicates that the perception of service encounters has remained the same, even after the outbreak of the pandemic. While a good service increases customer satisfaction, a failure to address this important human-to-human interaction deeply worsens it.
N/A (not assignable):
Before the outbreak of the pandemic, entries that could not be assigned to one of the pre-defined topic areas, were characterized by a mixed influence on customer satisfaction. Interestingly, after the outbreak of the pandemic, this topic area has developed into an exclusively negative influence regarding customer satisfaction. This can most likely be attributed to the fact, that the dataset collected after the outbreak of COVID-19 contains a large volume of entries regarding the pandemic itself. Such typically negative entries have been assigned to the residual class, presumably leading to its development into a strictly negative influence factor. In fact, Garcia et al. [22] have shown that such pandemic-related Twitter posts are mostly negative in nature.
Food and Beverages:
This topic area was originally characterized by a mixed influence on customer satisfaction. After the pandemic, however, this has changed into a purely positive influence factor on customer satisfaction. As already highlighted, this suggests that customer expectations decreased after a long period of isolations and lack of restaurant visits. Therefore, customers might have become more willing to highlight positive experiences and eventually overlook negative aspects. Additional efforts made by hotels during the pandemic to improve the quality of food and beverages and regain customers constitutes another potential explanation of the identified changes.
Room and Bathroom:
Before the outbreak of the pandemic, this topic area was perceived as a purely negative impact factor on total customer satisfaction. After the outbreak of the pandemic however, this has changed into both positive and negative influences on customer satisfaction. This could suggest that the Pandemic has heightened customers’ awareness of cleanliness and hygiene, which is why an appropriate and clean room condition is no longer taken for granted, but instead appreciated.
5 Discussion of Results
Today’s era has witnessed a large surge in UGC [4, 5]. This provides rich data for businesses to better understand their customers’ needs and behaviour [2]. Especially after global challenges like the COVID-19 pandemic, it becomes essential for tourism businesses to delve into UGC, allowing them to better understand the evolving needs of their customers [8]. For tourism practitioners, especially hoteliers and destination managers, our proposed methods and findings may offer new insights and managerial implications. Based on the positive and negative impacts identified in the course of the analysis of UGC, businesses can determine which areas demand more attention and additional resources to further reach high levels of overall customer satisfaction. Other than that, it is crucial for hoteliers and destination managers to keep in mind that customer needs and expectations can change, especially during global challenges such as the pandemic. In this context, this study sheds light on how such quality factors may have shifted in the face of the COVID-19 crisis. This offers insights for businesses to adapt their services and strategies, allowing them to respond to such changing demands.
While the concept of analysing UGC to better understand customer needs is not entirely novel [2, 15, 16, 21, 22] this study provides a refined methodology that focuses on identifying possible shifts in customer satisfaction brought about by the pandemic. For this, UGC from TripAdvisor was extracted and analysed. While other research in this area strictly pre-classifies such topic areas as either positive or negative, this work aims to identify concrete factors that describe their positive or negative impact on customer satisfaction in case of fulfilment or non-fulfilment [11] to detect possible differences from before and after the pandemic.
To conclude, this study underlines the significance of UGC as a reflection of changes in societal tastes, needs and sentiments, especially during times of crisis, like the pandemic. In this context, it becomes evident that such shifts can indeed be tracked and better understood through a multi-method analysis of UGC.
6 Conclusion and Outlook
As the landscape of customer satisfaction evolves, especially in challenging times after the COVID-19 pandemic, businesses and researchers alike need robust and reliable methods to analyse online customer feedback. Through the application of text-mining techniques, it is not only possible to better understand the current sentiments of users towards certain topics, but also to keep track of possible changes regarding sentiments and topics. For future research, we recommend applying this proposed methodological procedure to destination regions with predominantly English UGC. UGC on TripAdvisor from the Lake of Constance region is predominantly written in German, which has limited the volume of information extractable, especially since the outbreak of the pandemic. Although the data basis was fairly limited, significant results could be achieved. Nevertheless, choosing a more suitable region with predominantly English UGC constitutes a natural next research activity. Additionally, this study employed a lexicon-based approach to identify pre-defined topics. Employing unsupervised learning techniques, such as clustering or Latent Dirichlet Allocation (LDA), could further facilitate the identification of such topics without prior classification [16]. This may yield more representative and unbiased topic areas suitable for this type of UGC analysis.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
