Sentiment Analysis
The transcribed text was imported into Jupyter in the form of a matrix. The matrix cells contained text for each participant for each painting in each condition. This dataset was analyzed using the SentinemntIntensityAnalyzer library function in Python. After filtering out connecting words in each cell, each remaining word was scored from 0 to 1 for their respective classification of positive, negative, or neutral, with 1 indicating a strong sense for each. Finally, the dataset was separated into two tables, one with the negative scores in each cell, and one with the positive scores in each cell. The positive and negative scores for each condition are summarized in
Table 1. An independent samples t-test showed that participants reported significantly greater positive statements (
M=0.30;
SD=0.03) than negative statements (
M=0.13;
SD=0.02) after viewing the paintings in the experiment,
t(622) = -12.59,
p < 0.001.
Next, repeated measures ANOVA test was conducted on the Positive and Negative scores respectively to analyze the effect of the AAR cues on participants’ feelings about the paintings. Results showed that there was a significant main effect of AAR cue type on Positive scores
F(2,50) = 3.48,
p = 0.039, η
2 = 0.122. A post hoc test with the Bonferroni correction (α=0.05/3 = 0.0167) revealed that there was a significant difference between Basic AAR and Enhanced AAR conditions (
p = 0.011), but no significant differences between the other conditions.
Figure 3 shows mean positive scores for all four paintings; Enhanced AAR had the highest positive scores in all paintings, except for in ‘The Return’.
Results also showed a significant main effect of AAR cue type on Negative scores
F(2,50) = 3.56,
p = 0.035, η
2 = 0.126. Although a post hoc test with Bonferroni correction (α=0.05/3=0.0167) showed results that did not reach the conservative significance level with the adjusted alpha, the tendency was that Basic AAR had higher negative scores than the No AAR (
p = 0.025), and Enhanced AAR conditions (
p = 0.049).
Figure 4 shows mean negative scores for all four paintings, where Basic AAR had the highest negative scores in all paintings, with the exception of ‘The Return’.
Lastly, the feedback from each participant for each painting-audio combination was classified as Supporting or Countering. Statements were classified as ‘Supporting’ if the words explicitly stated participants’ preferences towards the sound-painting combination, or if the words demonstrated a strong or accurate understanding of the painting. Statements were ‘Countering’ if they explicitly stated disapproval towards the sound-painting combination, or if they displayed a weak or inaccurate interpretation of the painting. In total, there were 242 instances Supporting and Countering statements which were grouped by condition. A single participant could have provided both Supporting and Countering arguments for a painting, in which case their statement was considered in both cases. For Basic AAR cues, there were 49 instances of Supporting and 45 instances of Countering statements. For Enhanced AAR cues, there were 80 instances of Supporting and 45 instances of Countering statements. There were also 23 instances of Supporting statements for the No AAR condition, meaning they preferred to view the painting without music, or provided better description than with AAR cues. Based on the differences in the number of Supporting statements, it can be concluded that Enhanced AAR cues led to 39% greater preference over Basic AAR cues, and a 71% greater preference over No AAR cues. Basic AAR cues led to a 53% greater preference over No AAR cues. Examples of statements for each condition includes: Supporting: “I saw the painting inside the painting, and I saw things I didn’t’ see before, like people are actually fighting…I did not notice that before” (P1 for ‘The School of Athens’ in No AAR condition) Supporting: “Yeah, that was a good match, music was very sharp, following the sharp edges of the painting, jagged edges, it matches” (P8 for ‘Guernica’ in Basic AAR condition)
Supporting: “The song, I felt, goes well with the painting, like it does create the scene, like a castle in the background…kind of conveys to you the color that used, puts you in that right in that spot right there” (P25 for ‘The Return’ in Enhanced AAR condition)
Supporting: “The slow, soft, but kind of techno beat… music combined made it seem like kind of relaxing just as if you’re waiting for something” (P5 for ‘The Woman with a Parasol in Enhanced AAR condition)
Countering: “Every time there’s a beat, I feel like I have to change the frame of my eye, and it’s sort of tiring looking at a painting like that.” (P1 for ‘The School of Athens’ in Basic AAR condition)
Countering: “Feels like it’s scary, the sound makes you feel like a scary environment” (P23 for ‘The Return’ in Enhanced AAR condition).
Emergent Themes
Thematic analysis using an inductive approach also revealed the following themes. These themes support the findings from the Sentiment Analysis, and they are discussed below.
Sense of Darkness with Basic AAR cues
Both Supporting and Countering statements in the basic AAR condition made the recurring point that participants tended to focus on forlorn aspects of the painting or take on a more nihilistic outlook on the painting. There were 24 instances of participants reporting feeling negative emotions and focusing on darker imagery when viewing paintings accompanied by the Basic AAR cues. This was evident through the consistent use of phrases such as, “serious and scary”, “the tone of the notes, just make it seem darker”, “noticed the shadows a lot more”, and “definitely had a negative view”. For example, in ‘Woman with a Parasol’, P14 reported, “as soon as I started hearing the music, I noticed the storm clouds”, even though there were only a few strokes of gray color in the sky. In ‘The Return’, P5 commented, “since the music is so spread apart, it makes it feels darker than what I seems”, and “makes me focus more on the darker parts of the image, like I noticed the guy was injured on the stretcher, and didn’t really see that earlier”; these comments show that despite the painting containing bright colors to depict a sunrise, their focus was on the smaller, more negative details. In “The School of Athens”, P11 said, “you want to look at the painting, but it triggers your attention to like the negative symbols in piece such as the sculptures of people screaming”. They are referring to a single face that is part of a sculpture in the background of the painting; due to its low saliency and lack of colors, this detail is not easily noticed. However, in the painting ‘Guernica’, the use of Basic AAR cues caused participants to understand the painting better since the artwork itself is designed to promote negative feelings. Participants reported, “the music just feels very dark, kind of depressing” (P5), “low cords corresponded to the darker tones of the piece” (P22), “with this music, I was mainly thinking of darkness and chaos” (P6), and “felt even more disturbing” (P16).
Sense of Immersion with Enhanced AAR cues
When participants viewed the paintings with Enhanced AAR cues, there were 14 instances of participants reporting feeling as though they were a part of the painting. In ‘Woman with a Parasol’, P25 reported, “it did give the feeling of wind and the rustling of wings”, and P22 used “floating, windy” to describe the painting. This corroborates with the flowing edges of the woman’s dress, and open sky in the painting. When viewing ‘The School of Athens’, P9 reported, “the echo that came in the music. made me feel like I was standing in the cathedral”; P18 said, “felt like I was front row to this scene”. This corresponds to the structure in the painting that is depicted with archways. P13 commented, “felt more real, I could feel it” when viewing ‘The Return’, and P18 also remarked, “I feel like I can just step into the painting”. This painting showed a vast open space with lots of small details. When viewing ‘Guernica’, P26 reported, “feeling a shadow come over me”, and P21 said, “lack of sound in areas in the music, made it sound like they are in some sort of dark void”. This is especially true given the dark nature of the artwork and the use of contrasting black and white colors to portray the horrors of the second world war.