ABSTRACT
The authors develop a multimodal social listening analysis (MSLA) approach as a framework for managers to understand how meaning is constructed in social media posts using both text and other media. The research adds to AI and text analysis approaches by considering the whole meaning of a post rather than an analysis of subsets of information in text and other media. The use of MSLA is validated across the social media platforms of Facebook, Twitter and Instagram. The findings show that MSLA helps (i) reveal structures in what appear to be unstructured multimodal posts; (ii) identify all the sentiment items in a post; (iii) identify implicit meanings, such as irony, humour and sarcasm; and (iv) further identify emotions and judgements in multimodal communication. Importantly, this paper explains how decisions and opinions are made online and how marketing strategies can be tailored towards meanings derived from multimodal communication in social media.
Acknowledgements
The authors would like thank Professors Johnson and Miles for their informative comments on this paper. Profession Rodney Clarke for his guidance and feedback.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Notes
1. Image/text combinations: Exemplifying happiness using multimodal sentiment. ‘“Already for our new workers who are arriving tomorrow”#flowhive’. https://www.facebook.com/TelegraphRetreat/photos/1059544860783551/Flowhive with girl. April 25, 2016.
2. Happy Air Max Day ‘Happy Air Max Day ☺# AirMaxDay’.
https://twitter.com/DeltaHeavyUK/status/1243284455793364992, Accessed April 22, 2022.
3. Depicting appreciation composition positive ’#Tesla#electriccar#Teslamodel3.
P1: Very Cool. Love watching car fans sharing their love for Carlife (thumbs up emoji). Hope to see more’ Source: https://www.instagram.com/p/BZyzG3_Bjsc/?utm_source=ig_web_copy_link, accessed at 21st July, 2022.
Additional information
Notes on contributors
Mehmet Ibrahim Mehmet
Mehmet Ibrahim Mehmet is a Senior Lecturer from the University of Wollongong. He is a social listening expert and has used the technique to publish in a wide range of policy, marketing, environmental and business journals. He has developed several frameworks to better understand how meanings are created, conveyed and interpreted in digital media. In particular, how to tracks communication across multiple platforms concerning multiple stakeholder groups. His passion is improving upstream social marketing strategic approaches to better influence social, tourism and health policy at a state and federal level. UOW profile: https://scholars.uow.edu.au/display/michael_mehmet Twitter: @memzie08 LinkedIn: https://www.linkedin.com/in/dr-michael-mehmet-a1681a17/
Steven D’Alessandro
Steven D’Alessandro is an expert on consumer behaviour and its application to strategies for the profit and not-for-profit sectors. His research also focused on the societal impacts of marketing decisions and how marketing can create a better world. Professor D’Alessandro has published in a wide range of marketing journals and is the Journal of Consumer Behaviour co-editor. Professor D’Alessandro is also well known for applying diverse theoretical and methodological approaches to applied research. UTAS profile: https://www.utas.edu.au/profiles/staff/business-and-economics/steven-dalessandro Twitter: @dal_steve LinkedIn: https://www.linkedin.com/in/profdalessandro/?originalSubdomain=au