Research

Media Experience and Competence

is the title of a programme in collaboration with colleagues Monique Timmers, Anneke de Graaf and Reza Kartosen. I also closely collaborate with Valentijn Visch, associate professor at Delft Technical University.

As an example of the issues driving the research consider the paradox of negative emotion in entertainment. Many entertainment productions have been enjoyed coupling often intense unpleasant feeling with enjoyment and fascination. The grand guignol theatre, a special effects live theatre performance and current popular horror movies have in common that they provoke real fear but are liked nonetheless, or perhaps in part because they do so. Popular farce and current television comedy show nasty insults and painful embarrassment. Psychological drama can make you feel miserable, but you cannot wait to watch the next issue. Some computer games although requiring huge investments of time, effort and frustration, are nonetheless a sourceof tremendous pleasure to their aficionados.

One key to solving the paradox of negative feeling in entertainment is the role of user competence. "Frequency of exposure" counts in enjoying media entertainment.  Pleasure in entertainment, it seems, is for the initiated. But what competence develops from "habit" and how does it affect entertainment experiences?

Most generally it may be assumed that media users tacit competences of all kinds that users of media have in order to get gratifications from media use. For instance, in the use of media such as film, television and computer games for entertainment, enjoyment presupposes knowledge of genres, what experiences to expect, and conventions as to themes and style.

Working hypotheses underlying the research programmme Media Experience and Competence are 1) Engaging in media entertainment can be motivated by a desire for competent activity, autonomous sense-making, and social relatedness; 2) that regular users of certain entertainment genres share knowledge of typical content, structural and effect features of entertainment productions, and use it in optimalising their experience and 3) that this tacit knowledge overlaps in large part with explicit expert knowledge as contained in journalistic reviews, critical analyses and theories of film, television and games.

Important issues for research have to do with use and learning of tacit media entertainment knowledge. How does knowledge affect the use of say films, and how does it affect a person's entertainment experience? How do people acquire and update their knowledge, using their own accumulating experiences with continuously changing supplies, explicit specialist informationand communication with peers?

Understanding what user competencies are involved in media entertainment and how users acquire and update these is relevant for 1) mediadesign and production, both of "pure entertianment" productions and those targeted to fulfill other communication functions using entertainment as a vehicle or byproduct, 2) media literacy projects and policies, and last but not least 3) contributes to scholarly knowledge of the role of media in culture, society and the human mind.    

  Research areas

I cover three strands of research more in particular:

A. Emotion  and visuals : Pleasures of experiencing film, television, pictures, and videogames

B. Genre classification : Experience and structure-based categorization of media entertainment

C. New entertainment media : Designing entertainment and learning in the digital world

  A. Emotion and visuals: The pleasures of experiencing film, television and pictures  

Emotion is central to the entertainment experience. Emotions reflect goals, preferences and competences of media users. They are shaped by the structure and meaning of entertainment productions. They interface between subpersonal mechanisms and conscious, volitional interaction in the experience of entertainment. Finally, they link personal experience with entertainment as a social and cultural phenomenon.

Most entertainment productions are entirely visual or have a prominent visual component in their manifestation. Visual forms of entertainment include film, television programmes, comic strips, cartoons, video games and others. In our culture, the visual is often associated with non-serious use of media, notably for entertainment. One reason for this is that visuals are considered carriers of emotional expression rather than rationaldiscourse. The alleged emotionality of visuals is seen as a shortcoming or even a threat by some, and as a potential for liberation by others. Either intuition can be taken as the starting point for media psychological research. The following questions are addressed:

1. What emotions constitute entertainment and which do not? (Tan, 2004).

Making an inventory of the emotions in entertainment is a useful enterprise. But understanding why some emotions are typically associated with entertainment is another matter. The distinction between interactive entertainment (playing games, fantasy role play) and so-called lean back forms (reading stories, watching film and television) seems paramount in shaping emotion. Emotional interest is what all forms of entertainment experiences have in common, while different modes of (empathic) involvement colour specific experiences of various entertainment forms and genres (Tan 1996; 2008).

2. What does the visual, or more generally, the medium contribute to the emotions and to the entertainment experience, why and how? (Tan, 1995, 1996, 1999, 2000, 2005; Tan & Tollenaar, 2007)

The visual has traditionally been associated with superficial cognition, which in turn befits happy moods. In contrast to books, film, television, comic strips and games have been considered light media, suitable for lifting spirits by provoking happy moods. Apart from comparative response studies, in-depth analysis of the various media's workings as emotion machines is needed to accumulate evidence as to the validity of views such as these. One major  issue is how the visual and the medium interact with the user's capacity for imagination in creating emotional experiences that somehow refer to "reality". A related question is to what degree users are aware they are dealing with an artifact rather than with a "reality", and one that is the result of intentional agency.       

3. What is the role of pleasure in entertainment, and how are paradoxical pleasures of entertainment to be understood? (Tan & Frijda, 1999; Tan, 2004; 2008)

Most people agree that entertainment should be pleasurable. But what is pleasure? Do people want more and more of it all the time, or in leisure and entertainment? How does it relate to larger projects in life, and positively valued life goals such as happiness and fulfillment? Paradoxical mixtures of pleasure and pain abound in entertainment. Just think of fear plus fascination in horror, sorrow plus exhilaration in romantic comedy and frustration plus success in gaming. The 'bitter-sweet' emotions characteristic of melodrama are another very common example. And think of the efforts and frustration that dedicated gamers bear, and seem to like. At this point I collaborate with Joyce Neys who carries out her PhD research on pleasure in gaming and with Jeroen Jansz, who is also involved in Joyce's project.

4. What are typical conditions for entertainment emotions? (Tan, 1996; 2008; Tan & Jansz, 2008)

It has been attempted to define entertainment as a pleasurable state related to certain conditions. Lack of self-involvement, seriousness, profound empathy and sympathy are cases in point. Research into such conditions focuses on the role of perceived realism, empathy, and involvement in enjoying films, television and games. In one ESF-NWO project with Nico Frijda and Erik Rietveld, we attempt to grasp refined emotions and varieties of embodied empathic involvement in watching art and entertainment. (See link hereunder to the CONTACT project: http://linus.media.unisi.it/cirg/contact/ )

5. Is pleasure in entertainment dependent on experience, training and knowledge? (Tan, 1999; Tan, 2007; 2008)

A popular assumption is thatmost entertainment pleasures either do not need any training or require knowledge and skills that are easily acquired. But there are also indications to the contrary. For instance, movie fandom and gaming experience tend to contribute to entertainment pleasures.Therefore, more informed questions are subject of research, such as these:

- What competence do various forms of entertainment require?

- Is there pleasure without competence in entertainment?

- What is the nature of "entertainment intelligence"? Is it perhaps largely communicative?

- Does entertainment require assuming roles?

- How are competences acquired?

- Can pleasures be increased by instruction?

- How is competence used in attaining personal goals, such self-presentation and giving shape to one's identity?

B. Genre classification: Experience and structure-based categorization of media entertainment  

The aim of the research programme is to study the way people categorize their experiences with productsofmedia entertainment. Current categories suchas genres and formats used by the audience, institutions and the industry have been known for their conventional and ad hoc nature. These categories may be contrasted with the outcomes of empirical research of categories. I collaborate with Valentijn Visch in this research. He recently wrote a dissertation on the subject, see his website, below.   

The main question is: How is user knowledge of entertainment mentally organised and how does it become manifest in the experience of entertainment products? (Visch & Tan, 2007; 2008; 2009).

In this research individual recipients and users categorise entertainment products in various ways. Various aspects of categorisation are studied:

genre recognition, recognition tasks genre classification, sorting and labeling tasks genre production,  closed, structured, guided production tasks. We investigate this categorisation in relation to two factors:

1)       experiential processes of reception and use, in particular

sensation and perception comprehension appreciation and emotion of/ provoked by content and

2)       structural properties, in particular

content properties, incl. themes  style parameters multimedia analytical parameters Categorisation is approached through experimental research, using "natural" and artificial stimuli. Experiential processes are registered through questionnaires. Individual variation will be an important factor, but recipient and user factors are mainly studied as control variables. The aim of the research is to contribute to a psychologically realistic theory of entertainment genres. Research into ESB categorization has relevance for other areas of research in communications, including media appeal and mood management. It contributes to a knowledge basis for discourse on media genres in journalism and policy making.

There is also a relation with research into automated information retrieval. Categorization strategies and cues used by humans may have heuristic value for algorithms of IR, especially in the visual domain.

C. New media: Designing entertainment and learning in the digital world  

The introduction of broadband networks (BB) constitutes a major break in the technological infrastructure for media entertainment (ME). This research strand explores what the new technology may add to traditional and current Internet ME, focusing on interactivity and supply. As to the increase of interactivity, virtually unlimited storage, immediate server effects and high capacity bidirectional data traffic have the potential to remove some limitations to personalisation, activity, socializing and remaining gratifications inherent to current ME. As to supply, bandwidth and ubiquitous access make for dazzling numbers of entertainment titles being available at all times and places. However, the wealth of interactive possibilities and channel capacity of BB create a content void and a lack of uses and formats. Interdisciplinary research and development attempt to fill the gap.

The main question is: What are relevant design factors in new media applications combining entertainment with learning? (Tan & Müller, 2003; Tan et al., 2004, 2008; Tan & Oinonen, 2009)

The production of new media especially in Europe faces the challenge making entertainment media better in terms of potential for enhancing participation, social coherence, learning, identity exchange, community creativity.

The research programme consists, first, of projects that aim at developing prototypes of IT systems that pair an entertainment aspect with at least one serious function. My experience is mainly in interactive systems for learning about art, culture and entertainment. The emphasis is on user-centred design of such systems or components thereof. See for instance the SCALEX project link hereunder, http://www.scalex.info/

Second, together with Jeroen Jansz Iam involved in Joyce Neys' PhD research project Pleasure and Pain in Gaming, on motivation in gamers of varying expertise.  

Publications referred to  

Tan, E.S. (1995). Film induced affect as a witness emotion. Poetics , 23, special on Emotion and Cultural products, pp. 7-32.

Tan, E.S. (1996). Emotion and the structure of narrative film. Film as an emotion machine. Mahwah (NJ): Erlbaum.

Tan, E.S. (1999). Emotion and style as design. Proceedings of the First International Conference on Design and Emotion , Delft University of Technology,   Nov. 3-5 1999, pp. 55-64.

Tan, E.S. (2000). Emotion, art and the humanities. In M. Lewis and J.M. Haviland-Jones (Eds.), Handbook of Emotions , 2nd. Ed., pp. 116-136. New York : Guilford Press.

Tan, E.S. (2004). Het pleziervan media-entertainment . Oratie. Amsterdam: Vossius Pers, 2004. http://www.aup.nl/do.php?a=show_visitor_book&isbn=9789056293437

Tan, E.S. (2005). Three views of facial expression and its understanding in the cinema. In J.D. Anderson & B. Fisher Anderson (Eds.) Moving Image Theory: Ecological Considerations , pp. 107-127. Carbondale , Il.: Southern Illinois University Press.

Tan, E.S. (2007). On the cognitive ecology of the cinema. In M.A. Peterson, B. Gillam, & H.A. Sedgwick (Eds.), In the Mind's Eye: Julian Hochberg's Contributions to Our Understanding of the Perception of Pictures, Film, and the World , pp. 562-571.

Tan, E.S. (2008). Entertainment is emotion. The functional architecture of the entertainment experience. Media Psychology , 8 (1), 28-51. http://dare.uva.nl/record/306434

Tan, E.S., Chisalita, C., Oinonen, K., & Raijmakers, B. (2008). Learning and entertainment in museums: A case study.  In P. Ludes (Ed.), Convergence and fragmentation (pp. 91-110). Bristol: Intellect.

Tan, E.S., & Frijda,N.H. (1999). Sentiment in film viewing. In C. Plantinga & G.M. Smith (Eds .), Passionate views. Film, cognition, and emotion , pp. 48-64 . The Johns Hopkins University Press. Baltimore & London : Johns Hopkins University Press.

Tan, E.S. & Jansz, J. (2008), The game experience. In R. Schifferstein & P.P. Hekkert (Eds.) Product Experience (pp. 531-556). Amsterdam : Elsevier. http://dare.uva.nl/record/312093

Tan, E.S., & Müller, H. (2003). Integration of specialist tasks in the digital image archive. In H. van Oostendorp (Ed.), Cognition in a digital world , pp. 47-73 Mahwah(NJ): Erlbaum.

Tan, E.S., Oinonen, K., Trummer, C., Martin, J., Bernreiter, M., Schwarzinger, C., Hoffmann, A., Rauter, B., Moritsch, O., Reinhart, M., Mittendorfer, G., & Artiger, V. (2004). Final evaluation of SCALEX . Sixth Framework Technical report SLX-T7.3-D7.3.1-UVA-001-02, 2004.

Tan, E.S., & Oinonen, K. (2009). Personalising content presentation in museum exhibitions: A case study. In R. Sablatnig, M. Kampel, & M. Lettner (Eds.), VSMM 2009, Proceedings of the 15 th International Conference on Virtual Systems and Multimedia, pp. 232-238. September 9-12 2009, Vienna , Austria .

Tan, E.S. & Tollenaar, A., (2007). Aesthetic communication through posters. Empirical Studies of the Arts , 25, 1, 21-39.

Visch, V.T., & Tan, E.S. (2007). Effects of film velocity on genre recognition. Media Psychology , 59-75.

Visch, V.T. & Tan, E.S. (2008). Narrative versus style: Effect of genre-typical events versus genre-typical filmic realizations on film viewers' genre recognition. Poetics, 36 , 301-315. http://dare.uva.nl/record/306441

Visch, V.T. & Tan, E.S. (2009). Categorizing moving objects into film genres: The effect of animacy attribution, emotional response, and the deviation from non-fiction. Cognition , 110 , 265-272.