How Whatsappization of the Chatbot Affects User Satisfaction, Trust, and Acceptance in a Drive-Sharing Task

Authors: Nirit Gavish, Rotem Halutz, Liad Neta

Abstract:

Nowadays, chatbots are gaining more and more attention due to the advent of large language models. One of the important considerations in chatbot design is how to create an interface to achieve high user satisfaction, trust, and acceptance. Since WhatsApp conversations sometimes substitute for face-to-face communication, we studied whether WhatsAppization of the chatbot -making the conversation resemble a WhatsApp conversation more- will improve user satisfaction, trust, and acceptance, or whether the opposite will occur due to the Uncanny Valley (UV) effect. The task was a drive-sharing task, in which participants communicated with a textual chatbot via WhatsApp and could decide whether to participate in a ride to college with a driver suggested by the chatbot. WhatsAppization of the chatbot was done in two ways: By a dialog-style conversation (Dialog versus No Dialog), and by adding WhatsApp indicators – “Last Seen”, “Connected”, “Read Receipts”, and “Typing…” (Indicators versus No Indicators). Our 120 participants were randomly assigned to one of the four 2 by 2 design groups, with 30 participants in each. They interacted with the WhatsApp chatbot and then filled out a questionnaire. The results demonstrated that, as expected from the manipulation, the interaction with the chatbot was longer for the dialog condition compared to the no dialog. This extra interaction, however, did not lead to higher acceptance -quite the opposite, since participants in the dialog condition were less willing to implement the decision made at the end of the conversation with the chatbot and continue the interaction with the driver they chose. The results are even more striking when considering the Indicators condition. Both for the satisfaction measures and the trust measures, participants’ ratings were lower in the Indicators condition compared to the No Indicators. Participants in the Indicators condition felt that the ride search process was harder to operate, and slower (even though the actual interaction time was similar). They were less convinced that the chatbot suggested real trips and they trusted the person offering the ride and referred to them by the chatbot less. These effects were more evident for participants who preferred to share their rides using WhatsApp compared to participants who preferred chatbots for that purpose. Considering our findings, we can say that the WhatsAppization of the chatbot was detrimental. This is true for the both chatbot WhatsAppization methods – by making the conversation more a dialog and adding WhatsApp indicators. For the chosen drive-sharing task, the results were, in addition to lower satisfaction, less trust in the chatbot’s suggestion and even in the driver suggested by the chatbot, and lower willingness to actually undertake the suggested ride. In addition, it seems that the most problematic WhatsAppization method was using WhatsApp’s indicators during the interaction with the chatbot. The current study suggests that a conversation with an artificial agent should also not imitate a WhatsApp conversation very closely. With the proliferation of WhatsApp use, the emotional and social aspect of face-to face commination are moving to WhatsApp communication. Based on the current study’s findings, it is possible that the UV effect also occurs in WhatsAppization, and not only in humanization, of the chatbot, with a similar feeling of eeriness, and is more pronounced for people who prefer to use WhatsApp over chatbots. The current research can serve as a starting point to study the very interesting and important topic of chatbots WhatsAppization. More methods of WhatsAppization and other tasks could be the focus of further studies.

Keywords: chatbot, WhatsApp, humanization, Uncanny Valley, drive sharing