Our findings unfold across three sections corresponding with the research questions introduced in Introduction, and analysed and discussed within the theoretical framework set out in Remembering the Dead In an Algorithmic Present. In Deep Nostalgia’s Remediated Memory, we explore the concept of remediated memory, that is, how the shift from photo to video in Deep Nostalgia reshapes memory and memorialisation practices. In Deep Nostalgia’s Algorithmic Nostalgia, we focus on the affective dimensions and resonances of the creations, exploring how they might be understood as algorithmic nostalgia. Finally, in The Logics of Socio-Technical Infrastructures, we discuss how social network logics shape users’ interactions with Deep Nostalgia, and how they underscored its intense virality at the time of the launch.
Nonetheless, we noted that there were other, more subtle, ways in which we could interpret some of the users’ comments and practices in our sample in relation to ethical concerns – from coding for user’s reaction to the fidelity of animations, to noting what kinds of content were the most popular. In 4.1, there is a subsidiary focus on the authenticity of animations framed within current debates about deepfakes. In 4.2, we analyse the ethics of reviving the dead, and in 4.3, we highlight the datafication of users by Deep Nostalgia compounded by its reliance on social media, in this particular case, on Twitter.
Deep Nostalgia’s remediated memory
Deep Nostalgia is powered by a deep learning algorithm created by D-ID to match each uploaded photo with a compatible ‘driver’ video to support that photo’s animation most convincingly. It does so by analysing key features in the photo such as head orientation, and matching those features to a suitable blueprint video. This means that, like other algorithmic systems, Deep Nostalgia operates best at a degree of abstraction; in the design of such a system, a programmer will exclude more atypical or ‘chaotic’ data inputs in a bid to secure more predictable and persuasive outputs (
Markham, 2020:10). This underpinning logic leads to a degree of uniformity and conformity during Deep Nostalgia’s process of remediation. These are not the subject’s own movements: they have to all intents and purposes been directed by a programmer. El-Hadi captures this well when he calls the videos ‘digital frankensteins’ (2021); creations of people with gestures that they perhaps never intended or meant.
This shift, from photograph to video (now with driver video sedimented within it), is clearly significant in its attempt to maximise immediacy. As the subject in the photograph is set in motion, so too is the user’s line of vision, a potentially more involved and intimate engagement. This is what we conceptualise as a remediated memory of that person. The short, looped video refashions other media (the photograph, the blueprint video), and any memory attached to the photo comes to be represented, echoed or distorted, within a new medium. The process could even be said to put those memories into conversation with other(s’) memories in that it relies on movements and expressions that are borrowed from another; the coming together of two datafied ‘bodies’ (as ‘frankensteins’ to echo El-Hadi above).
Remediation operates in this case in the ways in which the new medium – powered by artificial intelligence – intersects with the ‘old’ medium of analogue or digital photography; not superseding it, but refashioning it for mennonic purposes. This form of algorithmically remediated memory oscillates between two strategies. The first attempts to eliminate the medium, making the user ‘believe that he [sic] is in the presence of the objects of representation’ (
Bolter and Grusin, 1999: 272–73) – in this case, their deceased relative. The second draws attention to the medium (and materiality) given that it is the user who actively triggers the remediation of the photograph – an example of what Bolter and Grusin term ‘hypermediacy’ (
1999; 272). In suggesting the concept of remediated memory, we pick up on van Dijck’s prompt to consider the process of remediation when thinking about ‘mediated memory’ (
2007: 48–49). We do so in order to highlight the ambivalent, iterative and hypermediated quality of memories that have undergone remediation, especially where they are co-constructed through algorithms and/or networked with (in) social media, as is the case here. Of course, remediation has significant implications for memory work both individually and collectively where it unsettles or recasts our connections with our past(s). There are profound ethical considerations related to broader debates about memory modification and enhancement, as well as the externalisation of memory, through new and emerging technologies (
Kourken and Sutton, 2017).
This process of mnenomic remediation was an important theme in the tweets in our sample. We found evidence in comments on the technology, the movement of bodies, or the authenticity of the resulting animations. We present results from our analysis in the following table (Table 1) and discuss them below:
Deep Nostalgia’s remediated memory is intriguing to consider in light of broader scholarship about the ambiguity of digital memory practices, which
Garde-Hansen et al. (2009) propose destabilise the boundaries between life and non-life, and organic and inorganic things. In
Algorithmic Practices and Memory Work we situated Deep Nostalgia within the context of work on ‘algorithmic afterlife (
Lambert et al., 2018) given that the qualities of ‘synthetic resurrection’ (
Ajder, 2019) are evident in its animations. Here, we focus on reactions of users to the embodied qualities of the animations, a recurring theme in 61% of tweets in our sample.
16 For example:
‘Animei a foto do meu avô no #DeepNostalgia e mandei pra minha mãe no whatsapp. A véia ficou tão emocionada que chorou: “Como vc conseguiu isso? Chegou me da nervoso, parecia ele com sorriso mesmo”’. [I animated my grandfather's photo on #DeepNostalgia and sent it to my mom on whatsapp. The old woman was so moved that she cried: “How did you do that? It made me nervous, he looked like he was smiling.”]
We identified two key mnemonic responses to Deep Nostalgia’s practices of remediation in our sample, sometimes in isolation, and in other instances, in combination (in keeping with the ambivalence and the double logic of remediation we record in this article). In the first response, remediation solidifies or even amplifies an image’s memorative significance; the video is understood to restore the person in the image and further solidifies any extant mnemonic attachments. Here, the emphasis is on the ways artificial intelligence underscores a kind of fixity or eternalisation, resurrecting people (however incompletely) through animation, and even supporting the creation of new posthumous memories, of the smile for example. In the second response, the process of remediation seems somehow to undermine an image’s memorative clout. The meaning of the image, and perhaps its associated mnemonic qualities, become(s) less certain or fixed through the hypermediacy of remediation. For example, Bolter and Grusin (1999:28) suggest that in a still photograph it is the light reflecting off a person which provides the invaluable contact point between that person and their representation in the resultant image. In Deep Nostalgia, creations that contact point is obscured by, or perhaps finds itself in antagonism with, the algorithmic process; it begins to bend and flex in a way that is not wholly convincing or comforting.
Many of those in our Twitter sample who commented on the nature of the embodiment on offer through Deep Nostalgia demonstrated a kind of ambivalence in response to these gestures, displaying conflicting responses to it simultaneously: ‘I’m both in awe and creeped out. Still super cool‘. The video outputs amount to forms of ‘virtual creepiness’ (
Anderson, 2019: unpaged) for many, and there is a cumulative sense of the uncanny in our dataset, with users referring to the animated images as ‘creepy’, ‘weird’, ‘freaky’ or in similar terms in 11% of tweets our sample:
‘Ok so some of the #DeepNostalgia stuff is so uncanny it’s terrifying, but I took a photo of my grandad from the archive of family photos I made a while ago and ran it through and honestly - the result is quite neat!’
This notion of the uncanny is interesting, connecting with debates within robotics and artificial intelligence about feelings of unsettlement provoked by representations that are human, but not quite human enough. The uncanny, according to Arnold-de-Simine, can be both enabling and dangerous, allowing people to ‘hold potentially conflicting reactions (disturbing/comforting) in suspension’ (
2019:92), as we saw in the ambivalence expressed by many in our Twitter sample. This notion of ambivalence will be a recurring one in the sections that follow, demonstrating the unsettled quality of our responses to these technologies, and of the technologies themselves.
We might note here how cautious MyHeritage is in its wording about the authenticity of Deep Nostalgia creations, not least as a way of distinguishing them from deepfakery, a variance they are at pains to point out: ‘the end result is not authentic, but rather, a technological simulation of how the person in your photos would have moved and looked if they were captured on video’.
17 Notwithstanding efforts by MyHeritage to present a credible line on how genuine these creations are – no doubt important for users who are genealogists also
18 – concerns about authenticity do circulate around these recreations (see Table 1). These debates are not new however, having been a feature of discussions about deepfakes (
Maras and Alexandrou, 2019), but also in relation to other forms of historical recreation such as theatre or copying (
Jones, 1990;
Parry, 2013). In relation to representations of the past, Lowenthal has argued that ‘all “olden times” are potentially fraudulent’ (
1990:17). Within the context of developments in technology however, we should expect these debates to re-surface, as we see here amongst a small percentage of users in relation to deep learning approaches.
Deep Nostalgia’s algorithmic nostalgia
Here, we explore Deep Nostalgia’s claim that it ‘bring [s] beloved ancestors back to life’ for nostalgic purposes. We offer an overview of our data sample, analysed against the concept of algorithmic nostalgia. We explore how the technology as operationalised by the users in our sample crafts, charges, or contains differing versions of nostalgia, particularly through its affective impacts.
In our study, we were therefore attentive to ‘the emotional landscape’ (
Powell, 2018:13) produced by Deep Nostalgia’s creations. MyHeritage noted proudly in a tweet on Mar 3 that ‘#DeepNostalgia is bringing MyHeritage users to tears!’ and this emotional resonance was in evidence in 60% of our sample. People were willing to overtly perform their (overwhelmingly positive) emotional responses to these moving images within social networks:
19
‘Thank You to @MyHeritage for animating this photo of my Grandfather. He tragically died at the hands of another in 1966, years before I was born. This is the first time I’ve seen him smile. #DeepNostalgia #RootsTechConnect’
As noted in
Remembering the Dead In an Algorithmic Present, interest in our digitally mediated emotional lives has been increasing in recent years, not least for commercial reasons (
McStay, 2018:1). The imperative to make people feel – and to encourage expression or performance of feeling through, for example, the use of emoji – is now strong, whether it be excitement, pride, anger or despair that we are experiencing. Advertisers and brands, as well as political and cultural actors, have paid increased attention to expressed or implied ‘sentiment’ (
Puschmann and Powell, 2018), and emoticons and animated gifs have become part of the day-to-day vernacular of digital communications (
Miltner and Highfield, 2017). This is worthy of note in relation to Deep Nostalgia given the ambitions of the corporate actors here: MyHeritage aims to promote and increase users for its genealogy services through exposure, and social media companies (including Twitter) create economic value through connectivity and the collection of data. The likelihood of both these ambitions being met is significantly increased where users’ emotions can be mobilised. This can be understood as ethically jarring; the calculations made as Deep Nostalgia processes an image are computational and abstract, but the impacts they produce are emotional and psychological.
An important part of the success of the affective dimension of Deep Nostalgia can perhaps be attributed to a kind of ‘biomediation’ (
Garde-Hansen et al., 2009: 12) where technology interfaces with the human and communicates through processes that are more than semiotic and cognitive (
Angel and Gibbs, 2006). We saw in our sample that it is often movement in the images that triggers emotional resonances. This is unsurprising given that most of the animated images feature deceased relatives, many of whom would not (or could not) have been captured through moving imagery in their lifetime.
In fact, in its framing Deep Nostalgia encourages users to animate faces of ancestors, family and historical figures, not ‘photos featuring living people without their permission’. This was predominantly the case in our sample,
20 where the majority of images were of deceased family members or people from history, as is demonstrated in Table 2:
Interestingly, across the top 10 tweets on all attention measures in our sample (likes, retweets, comments and quoted tweets) all but one featured historical figures, and half were paintings or sculptures, like a popular recreation of a Neanderthal model shared by the Natural History Museum, London. This suggests people felt more inclined to like, share or comment on posts where the animated subject was evidently not somebody’s deceased relative. This begins to suggest the ethical considerations that are thrown into sharp relief by D-ID, MyHeritage, Deep Nostalgia, and its creations. Of particular concern is the question of how a deceased person might be considered to have consented to the uploading of their image. Although it may be true that the dead cannot object, and there are no legal obstructions that prevent a relative from animating an image, there are clearly ethical and moral considerations here where a person’s right to be forgotten meets another person’s sense of a duty to remember, and the agency that they might feel comes with that.
Deep Nostalgia recreations are about more than their deceased subjects however, to which they have more than an indexical relationship. They become symbolic mnemonic objects that mediate users’ own identities and not just those of their deceased relatives. The chance Deep Nostalgia gives to users to reanimate photographs from their personal archives can be read as filling in gaps in familial histories (although clearly not straightforwardly), and another aspect of self-construction. The use of imagery more broadly within social media has become an important aspect of identity construction and self-representation, and how a person’s photos intermingle with the photos of others is an important aspect of those processes. Deep Nostalgia might contribute to the co-creation of networked identities as ‘users contribute to the stories of each other’ (
Leaver, 2018), in this case including those already dead.
Susan Sontag understands photographs as ‘incitements to reverie’
(2008:16), and we understand our dataset to be nostalgic in that it constitutes a psychosocial mobilisation of emotion (as we saw above), more often than not, galvanised around past-ness; pasts that are constructed as silent and smiling in the Deep Nostalgia videos. We questioned whether this amounted to an excess of ‘historical musing’ which is ‘myopic, perpetual and ultimately destructive’, or whether it could instead be understood as a ‘reflexive, collective or adaptive view of history’ (
Lizardi, 2016). Our research suggests that the history on offer in Deep Nostalgia is more persistent than adaptive, and quite literally perpetual in that it plays on a loop. These mechanisms are coded by programmers in ways which are no doubt context and culturally dependent, and the nostalgia assembled is (over)determined and situated as a result. Affective remediated memories underscore a pervasive algorithmic nostalgia in our dataset then; outputs which tend toward uniformity and conformity (as we noted in 4.1) feed a generalised nostalgia in our data which is generated and organised through automated and recursive mechanisms.
21 The Deep Nostalgia interface makes it possible for users to easily onboard, upload and animate their photographs, and, crucially, share the resulting video on social media, effectively nostalgia-ficating what is otherwise a marketing tool.
It is too easy however to dismiss Deep Nostalgia creations and the algorithmic nostalgia they elicit as sentiment and myopia. As Sayers notes, ‘nostalgia is not always, or only, a sign of stuck-ness’ (2020:190). In some moments we saw uses of Deep Nostalgia that accorded well with the general sense we can have in social networks, and in society more generally (according to
Lipovetsky 2005), of living in a perpetual present; distanced from the past and feeling insecure about the future. But in other moments, there was interest in the past which might be understood as productive or generative. Significantly, we found uses of the technology aimed at revindicating a collective memory of people that experience racism in ways that seemed more productive than myopic, for example in the frequent animation of the Black abolitionist Frederick Douglass.
22 Again, the ambivalence of the technology is at play here as Deep Nostalgia ‘conflates the desire to honor the past with an impulse to appropriate it’ (
El-Hadi, 2021). The multiplicity of meanings attached to the process of animating, remembering and sharing these images are negotiated by users in ways that collapse and exceed the intended uses of the technology, while ethical considerations remain – for example, the disproportionately harmful effect that these technologies might have on Black people (
El-Hadi, 2021).
There was curiosity evident in our sample also – about what future for remembering these videos might anticipate or set in motion, and what possibilities might flow from that. Deep Nostalgia creations are then about the past, the present and the future, a paradox symptomatic of the ambivalence of this technology, and of responses to it in our dataset. This is unsurprising given that, as Routledge contends, nostalgia is a ‘complex emotional experience’ (2016: 44), that can be both past and future-oriented in its adaptive qualities (
FioRito and Routledge, 2020) and which can itself be felt as ambivalent.
Algorithmic nostalgia is clearly of interest in relation to the technology’s memorative effects and affects, as well as how these are achieved through processes of automation and remediation. It is also intriguing to examine in the context of social networks’ algorithmic and datafication practices, where monetising connectivity and user data emerge as considerations. In the next section, we explore these in more detail.
The logics of socio-technical infrastructures
In this section, we respond to research question three by exploring the ways social media’s connective and attention logics shaped how users interacted with Deep Nostalgia. Our analysis reveals that the outcome of using Deep Nostalgia is not always zero-sum a memory (and by extension a memorialist). It resulted in dialogic practices that were communicative, performative and extractive too.
Sharing via social media is clearly an integral part of the logic of Deep Nostalgia, blurring the boundaries between private and public memory. 70% of tweets in our sample shared animated images, a figure which increased in the members of the public category, where 88% of people had used Deep Nostalgia and shared the results. We found that people often used (and shared) Deep Nostalgia more than once, posting threads of multiple animated videos: ‘I’m still obsessed with #DeepNostalgia’.
Nearly a quarter of tweets featured the default text accompanying animated images when shared directly from the MyHeritage app or website; ‘I love the way MyHeritage brought the people in my photo to life! Try it too and be amazed. #DeepNostalgia’. These users therefore acted as inadvertent marketeers for MyHeritage, given that sharing an animated photograph was inextricably linked with promoting the technology, whether actively encouraging others to use it or not. When looking at the comments, the success of this strategy can be seen as we found many users asking how to use the feature and where to find it. Animated images linked to the MyHeritage website in most cases, where to use Deep Nostalgia, users were asked to sign up and relinquish, at the very least, a name and email address.
For MyHeritage, the sharing of videos performed an invaluable social advertising function where promotional messages were blended with users’ sense of identity and belonging (
van Dijck, 2017). In terms of public endorsement and brand image, such an approach is invaluable for a company like MyHeritage; their product (genealogical services) is turned into a story that can be excitedly shared. According to
van Dijck (2017), such practices do not constitute social or collective memory however so much as mere ‘connectivity’ in service of the social media platforms and their business models. The animated image is merely a(nother) ‘transaction in a data network’ (
Dewdney, 2022:24). Echoing Bory’s observations, elements of the spectacle are in evidence here, employed by an AI company as a way of advocating for its products (
Bory, 2019). This is noted by some users who criticise or ridicule the technology for these very reasons:
‘Deepfake technology is used to bring dead relatives back to life. I’m sure marketers are already strategizing this for marketing plans down the road.#deepnostalgia #deepfake’
We found in the tweets, as one would expect, many of the vernacular elements of Twitter as a platform. The playfulness of social media interactions appeared in 13 % of tweets in our sample. These frequently featured gifs as a framing device. Several tweets also made connections to other popular posthumous animations, for example, in the Harry Potter universe, offering a critical reading of the purportedly innovative nature of the technology. While not prevalent in our sample, such responses are a helpful reminder of the importance of the visual in social media cultures (e.g.
Leaver et al., 2019) and the extent to which the entire Deep Nostalgia campaign was enmeshed within and shaped by the ‘internet’s visual turn’ (
Vaccari and Chadwick, 2020). Here we are reminded that the adoption or ‘domestication’ of these technologies is not always linear; users find alternative ways of employing them, including those that are subversive or whimsical (
Kitchin, 2017: 19).
Social media companies are navigating the emergence of deep learning technologies in real time and are clearly disinclined to restrict activities that are gaining traction and attention, not least given the importance of ‘compulsive connectivity’ (
van Dijck, 2017) to their business models. The response of social media companies to deepfake technologies has thus been slow and, so far, mixed; TikTok has now banned what it calls ‘synthetic or manipulated content’, Facebook has banned deepfakes although with exceptions for parody and satire, and Twitter’s policy is that tweets featuring deepfakes should be labelled as such, only removing them if they are likely to cause harm. Our data suggests that most users were not troubled by these ethical concerns however or were at least able to put them to one side to participate in what had become a shared mnemonic experience within peoples’ social media feeds.
The viral nature of this case study is worth reflecting on here, alongside a broader look at the ways AI is discussed in society. According to
Nguyen et al. (2021) AI tends to be framed within discourses about technological trends, economic potentials, data risks and questions of governance, but Deep Nostalgia also has cultural and historical dimensions which are less common, and a strong personal/familial aspect too. This unusual emphasis may explain Deep Nostalgia’s rapid (albeit fleeting) popularity within social networks; a chance to partake in an activity that was somehow dissonant or not normative, yet also familiar given its characteristics. The easy-to-use capability for sharing creations on Twitter and Facebook,
23 coupled with the perceived novelty of the technology and the combination of emotional resonance and playful elements, no doubt contributed to the viral success of Deep Nostalgia in the weeks following its launch. To some degree, the connective nature of genealogy communities online might also bear on its popularity, given that ‘genealogists use technology to research their family, but they also use technology like social media to connect with other family researchers and to share ideas and information’ (
Kennedy-Eden and Gretzel, 2021). Deep Nostalgia did not trend for long however and was ultimately unable to sustain itself as a phenomenon. Nevertheless, its work for MyHeritage was done.
Following Natale, we might note this as the point when Deep Nostalgia’s deception became ‘banal’ and disappeared into the fabric of our daily lives (
2021); the fact that it so quickly became predictable and ordinary contributed to its recuperation. The next time we see forms of algorithmic nostalgia being promoted – such as in Amazon’s recent announcement that its Alexa will be able to channel the voices of dead people in a bid to ‘make memories last’ (
Paúl, 2022) – there will likely be less surprise, friction, and most troublingly perhaps, even less attentiveness to its ethical ramifications. As Bory points out, it falls to us then as media and communications scholars to identify and understand ‘how corporate narratives are driving the symbolic and cultural integration of new intelligent systems in society’ (
Bory, 2019).