639
Views
1
CrossRef citations to date
0
Altmetric
General Articles

More than words can say: a multimodal approach to understanding meaning and sentiment in social media

ORCID Icon & ORCID Icon
Pages 1461-1493 | Received 10 Aug 2020, Accepted 13 Dec 2021, Published online: 23 Aug 2022
 

ABSTRACT

The authors develop a multimodal social listening analysis (MSLA) approach as a framework for managers to understand how meaning is constructed in social media posts using both text and other media. The research adds to AI and text analysis approaches by considering the whole meaning of a post rather than an analysis of subsets of information in text and other media. The use of MSLA is validated across the social media platforms of Facebook, Twitter and Instagram. The findings show that MSLA helps (i) reveal structures in what appear to be unstructured multimodal posts; (ii) identify all the sentiment items in a post; (iii) identify implicit meanings, such as irony, humour and sarcasm; and (iv) further identify emotions and judgements in multimodal communication. Importantly, this paper explains how decisions and opinions are made online and how marketing strategies can be tailored towards meanings derived from multimodal communication in social media.

Acknowledgements

The authors would like thank Professors Johnson and Miles for their informative comments on this paper. Profession Rodney Clarke for his guidance and feedback.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. Image/text combinations: Exemplifying happiness using multimodal sentiment. ‘“Already for our new workers who are arriving tomorrow”#flowhive’. https://www.facebook.com/TelegraphRetreat/photos/1059544860783551/Flowhive with girl. April 25, 2016.

2. Happy Air Max Day ‘Happy Air Max Day ☺# AirMaxDay’.

https://twitter.com/DeltaHeavyUK/status/1243284455793364992, Accessed April 22, 2022.

3. Depicting appreciation composition positive ’#Tesla#electriccar#Teslamodel3.

P1: Very Cool. Love watching car fans sharing their love for Carlife (thumbs up emoji). Hope to see more’ Source: https://www.instagram.com/p/BZyzG3_Bjsc/?utm_source=ig_web_copy_link, accessed at 21st July, 2022.

Additional information

Notes on contributors

Mehmet Ibrahim Mehmet

Mehmet Ibrahim Mehmet is a Senior Lecturer from the University of Wollongong. He is a social listening expert and has used the technique to publish in a wide range of policy, marketing, environmental and business journals. He has developed several frameworks to better understand how meanings are created, conveyed and interpreted in digital media. In particular, how to tracks communication across multiple platforms concerning multiple stakeholder groups. His passion is improving upstream social marketing strategic approaches to better influence social, tourism and health policy at a state and federal level. UOW profile: https://scholars.uow.edu.au/display/michael_mehmet Twitter: @memzie08 LinkedIn: https://www.linkedin.com/in/dr-michael-mehmet-a1681a17/

Steven D’Alessandro

Steven D’Alessandro is an expert on consumer behaviour and its application to strategies for the profit and not-for-profit sectors. His research also focused on the societal impacts of marketing decisions and how marketing can create a better world. Professor D’Alessandro has published in a wide range of marketing journals and is the Journal of Consumer Behaviour co-editor. Professor D’Alessandro is also well known for applying diverse theoretical and methodological approaches to applied research. UTAS profile: https://www.utas.edu.au/profiles/staff/business-and-economics/steven-dalessandro Twitter: @dal_steve LinkedIn: https://www.linkedin.com/in/profdalessandro/?originalSubdomain=au

Log in via your institution

Log in to Taylor & Francis Online

PDF download + Online access

  • 48 hours access to article PDF & online version
  • Article PDF can be downloaded
  • Article PDF can be printed
USD 53.00 Add to cart

Issue Purchase

  • 30 days online access to complete issue
  • Article PDFs can be downloaded
  • Article PDFs can be printed
USD 222.00 Add to cart

* Local tax will be added as applicable

Related Research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.