Animation and Live Action: Bringing Plasmatics to the Silver Screen_2

Luc Besson: The Extraordinary Adventures of Adele Blanc-Sec 2010 Luc Besson from The Fifth Element onwards has demonstrated the mastery of the kind of high-resolution imaginative interpretation that I associate with Total Cinema. Here the seamless integration of animatronic and digital pterodactyls and revivified mummies fit well in the Edwardian (Belle Epoque) imagined by Jacques Tardi in his comic books of the same name. But this is not just another example of the brilliant realisation and remediation of the strip form, it is Luc Besson tapping into the ‘steam-punk’ zeitgeist, and like Jean-Pierre Jeunet’s delightful MicMacs of the previous year (and Jeunet’s earlier Amélie (2001), Besson brilliantly evokes a poetic and magical France – here set in the Belle Epoque that marries the whimsical magic of Raymond Peynet, the suspension of disbelief of the fairy tale, with the francophile consciousness of Melies and Feuillade.

Just to recap: the phenomenon of Total Cinema is emerging out of the marriage (or convergence) of computer-generated imagery and filmed live action. This emerging hybrid (film/digital) art form was initially called virtual cinematography (a phrase coined, I think, by John Gaeta, the special effects director on The Matrix series). So virtual cinematography is the beginning of total cinema, and it comprises several technologies that I cover in this blog. To reiterate, these technologies include:

rotoscoping

photo-grammetry

motion capture

compositing

special effects

CGI – computer generated imagery – modelling and animation

crowd simulation software

time-slice video, and bullet-time

augmented reality – virtual reality

These are the founding technologies of Total Cinema, but the art forms of the 21st century (let’s call them Expanding Cinema – an adaption of the phrase coined by Gene Youngblood in Expanded Cinema his book of 1970, where he surveyed the multitudinous foundations of the new film-related arts emerging from the 1960s counter culture). Expanding Cinema will embrace a far greater range of speculative forms emerging from the digital world, and most especially perhaps the new spaces of augmented/enhanced reality, the world of games, the world of social media, and the contextualisation of film within our culture generally (Secret Cinema, ‘prosumer film-making’ and ventures of this sort).

We are in a melting pot of ideas, of technologies, of art practices and increasing audience empowerment that is changing the way we conceive of films, how we finance films, make films, distribute films, appreciate films, and respond to the cinematic experience. That moving pictures (film, animation and realtime-imagery) will be central to the multi-faceted face of 21st century art, I have no doubt. That these emerging new forms will be immersive, multi-sensory, multi-media, interactive and participative is also most likely. It is also extremely likely that we will never again have a standardised fixed form (as we did during the c90 years dominance of 35mm film technology). We are in a period of what Rosalind Krauss calls the post-medium condition. Expanding Cinema, like the digital computing/software world it now inhabits, is continually evolving. The artist’s palette of the 21st century is itself transforming. It is likely to continue to give us the richest, most accessible, most customisable, most available, continuously evolving, continually expanding cinema…

Peter Greenaway: Tulse Luper Suitcases 2003-04. This is a film-based multi-media project by Greenaway, comprising 4 films – 3 ‘source’ films and one feature-film (DVDs), as well as web-sites, books and CDROMs. “The project has been described by Greenaway as “a personal history of uranium” and the “autobiography of a professional prisoner”. It is structured around 92 suitcases allegedly belonging to Luper, 92 being the atomic number of uranium as well as a number used by Greenaway in the formal structure of his earlier work (most notably The Falls). Each suitcase contains an object “to represent the world”, which advances or comments upon the story in some way, although in many cases the contents are more metaphorical than real.” (wikipedia)

Over the last twenty years, several researchers, fascinated by these emerging forms, have attempted to provide suitable descriptors – Michael Nash in his Vision after Television: Technocultural Convergence, Hypermedia and the New Media Arts field (1996), traces the impact of digital on the ‘traditional’ (analogue) video arts; Peter Greenaway only last year (2014) reiterated his ‘Cinema is Dead, Long Live Cinema‘ mantra:

“Cinema’s death date was 31 September 1983, when the remote-control zapper was introduced to the living room, because now cinema has to be interactive, multi-media art,” and

“Every medium has to be redeveloped, otherwise we would still be looking at cave paintings… New electronic film-making means the potential for expanding the notions of cinema has become very rich indeed.”

(These pronunciations by Greenaway were made at a master class at the Korean Pusan Festival in 2014. http://www.independent.co.uk/news/world/asia/greenaway-announces-the-death-of-cinema–and-blames-the-remotecontrol-zapper-394546.html)

The diversity of the expanding cinema forms range from viral web videos, to stage illusions/performance, to spectacular films:

Jonas Akerlund and Lady Gaga: Telephone 2010. In this promop-video for her song Telephone, Gaga and Akerlund lean heavily on tropes from Quentin Tarantino’s Pulp Fiction, orchestrated in a short film story and syncopated by Gaga and Beyonce’s brilliant performance. A ‘pure pop for now people’ production (the descriptor comes from Davie Robinson’s Stiff Records button-badge of c1978), Telephone is a hymn to B movies, Noir film, pulp crime magazines and Captain America pop art. Its great!

Zack Snyder: Watchmen 2009. Another brilliant remediation of the comic-graphic novel form. As digital imaging technologies, CGI, and special effects arts grew to full maturity in the first decade of the 21st century, It became possible to do full justice to the new wave of reflective superhero comics from the likes of Alan Moore, Grant Morrison, and Frank Miller. Zack Snyder and Robert Rodriguez had worked together on Miller’s Sin City (2005) – a film that to me was the perfect synthesis of film and CGI for this remediated story. Watchmen explores the synergy in a somewhat less stylised form.

Chris Milk: The Wilderness Downtown 2009. In this multi-window, realtime, customisable package, where the narrative is driven by music, Milk and his Google colleagues create a showpiece ‘interactive movie’ that combines generic components composited in real-time with live Google Earth and Streetview feeds. Very powerful – because it is customisable to the individual user/viewer.

Improv Everywhere/Chad Nicholson: Frozen Grand Central 2007. Live participatory, immersive performance – like this and other events staged by from Improv Everywhere (see also Human Mirror 2001) demonstrate another strand of the various media-art experiments intertwining in Expanding Cinema.

Will-I-Am and Jesse Dylan: Yes We Can! 2008. This video went viral in 2008. It was composed by Will-I-am and Jesse Dylan (son of famous Bob), and is comprised of mostly monochrome clips and stills and audio quotes from the Barack Obama’s Presidential campaign of that year. Going viral means that it was recommended by burgeoning links through social media, quickly reaching over 22 million recipients. It was videos like this that awoke marketeers to the promotional potential of the viral, self-recommending nature of media artefacts in the social media matrix.

Musion: Eyeliner projection illusions 2009. Recreating stage presence by using h-tech versions of the 19th century Pepper’s Ghost illusion, Musion added another tool to the rapidly extending vocabulary of multi-media staging and lighting.

2012_Marshmallow-Laser-Feast_realtime-projection-mapping_c

Marshmallow Laser Feast: realtime-projection mapping 2012. In this short promo for Sony Playstation, Marshmallow – a team of artists and developers – create what used to be called ‘post-production fx’ in realtime, using clever projection-mapping, performance and digital smoke and mirrors. Phenomenal!

So this is a brief summary of the territory that 2014 film students will be entering. That it is a territory much dreamed about by experimental artists for the last 200 years (see Richard Wagner’s Das Kunstwerk der Zukunft of 1849), and envisioned in many different (analogue) forms by the Modernist avant garde (by El Lissitszky, Rodchenko, Kurt Schwitters, Lazlo Moholy Nagy etc etc), will not deter contemporary students from making their own explorations and innovations in this fantastically rich media palette.

Animation and Live Action: Bringing Plasmatics to the Silver Screen

It was Sergei Eisenstein, in a series of essays written in the early 1940s (later published as Eisenstein on Disney, and as Sergei Eisenstein: Disney – edited by Oksana Bulgakowa and Dietmar Hochmuth, Potemkin Press 2013), who described Disney’s films (and by extension all animated cartoons) as having the quality of plasmaticness:

“The rejection of the constraint of form, fixed once and for all, freedom from ossification, an ability to take on any form dynamically. An ability which I would call ‘plasmaticity,’ for here a being, represented in a drawing, a being of a given form, a being that has achieved a particular appearance, behaves itself like primordial protoplasm, not yet having a stable form, but capable of taking on any and all forms of animal life on the ladder of evolution.”

Walt Disney + Ub Iwerks: Steamboat Willy 1928. This was the short animated film that launched Disney’s most famous anthropomorphic character, Mickey Mouse. Eisenstein would have been thoroughly familiar with cartoons from Disney, Fleischer, Hannah-Barbera and other animators of this period.

Eisenstein envied the animator the freedom of the mythological shape-shifter – the ability to change the forms of his character to illustrate their evanescent identity, their dual or multiple nature, or to illustrate and express metaphorically their change in temperament or mood. Potemkin Press describe Eisenstein’s objective in the blurb for Sergei Eisenstein: Disney:

“His essay on Walt Disney was part of his unfinished book METHOD, a study of the relationship between archaic thought and art practice. Disney becomes the central subject of analysis, because in his work the synesthetic affects of sound and color, and the perfect visual rhythm unite with animism, totemism and the plasmatic qualities of form. Eisenstein discovered in these distinctive characteristics of form a long genealogical ancestry and a deep yearning for the freedom that could allow one to imagine and even represent differently the relations of man and nature.”

(http://us.potemkinpress.com/products/eisenstein-method-disney-edited-and-commented-by-oksana-bulgakowa-and-dietmar-hochmuth)

Walt Disney: Fantasia 1940. Disneys rather self-conscious masterpiece of this early period was of course Fantasia (1940), which drew upon the talents of his astonishing team of animators, the Philadelphia Orchestra, several classical compositions, and the arrangement and conducting skills of Leopold Stokowski. Thjis cel is from the section The Sorcerer’s Apprentice – one of eight animated sections in Fantasia.

Walt Disney: Three Little Pigs 1935 (from Silly Symphonies (1930-1940).

Eisenstein was so impressed with Disney’s work that in 1935, when he was a judge at the Cannes Film Festival, he insisted that Disney’s Three Little Pigs should get first prize. But it was the general ability of animators to create with total artistic (and ‘plasmatic’) freedom that was entrancing Eisenstein. Of course the possible synergies between animation and live-action cinematography had surprisingly been realised almost from the very beginnings of Film. As early as 1900, James Stuart Blackton had been employed as an artist on one of Thomas Edison’s shorts The Enchanted Drawing (1900) that illustrated this potential synergy.

1900_Blackton_Enchanted-Drawing_first-animation_c

James Stuart Blackton/Thomas Edison: The Enchanted Drawing 1900. Using stop-motion, the filming of an artist drawing where the drawing is made to animate in response to the artist’s drawing, was a major innovation in the history of film, pointing the way to the invention of rotoscoping by Max Fleischer in 1917, and flagging the potential of special effects.

Dave Fleischer + Max Fleischer: Mechanical Doll 1922. The Fleischer brothers invented a rostrum rig for creating rotoscoped animations in 1917. Rotoscoping involved back-projecting live action cine frames, one frame at a time, onto a translucent drawing board so that animators could trace-over the live-action image and convert the live-action to animation. Here, 5 years later the Fleischer’s explore the natural outcome of their rotoscoping process in this short animation integrating live-action and animation.

Dave and Max Fleischer: Mechanical Doll (1922)

For 1922, this is really innovative. Fleischer had invented a rotoscoping rig in 1917, and I guess that the use of this rig would suggest this delightful synthesis of live-action and cel animation – a practice made world famous some 60 years later by Disney in his Who Framed Roger Rabbit (1988)

Bob Sabiston/Flat Back Films: Rotoshop 2001. Sabiston’s Flat Back Films made quite a stir with this semi-automatic rotoscoping software. Experiments in computerised cartooning stem from at least 1990 (see Barlow, Blakemore, Weston-Smith: Images and Understanding (1990)), but Sabiston’s software is state of the art, vastly reducing time spent on the human interpretation of source material and manual rotoscoping.

Autoscope 2014 – this app is being developed right now in 2014, and is only available as a prototype. It follows Bob Sabiston’s clever Rotoshop (2001) as an automatic rotoscoping tool – rather successfully as far as the examples illustrate.

The modest developer of Autoscope (identified on his website as Paul, describes his Autoscope software:

“I turn normal videos into cartoons through the process of rotoscoping, or, to be more precise, autoscoping (don’t look it up; it won’t be in the dictionary … yet). My project, Autoscope, is the result of years of research, experimentation, trials, errors and correcting those errors. The videos I am putting up here show what Autoscope can do. These are just short clips but you could, if you wanted to, remake an entire feature film or an entire TV series in cartoon form.”

http://autoscope.atwebpages.com/index.htm

Richard Linklater: A Scanner Darkly 2006. From Philip K. Dicks ominous 1976 novella about surveillance, Linklater uses rotoscoping (Rotoshop) to semi-automate this high-budget, very successful animated feature. The advantage of rotoscope is that you can combine the best of live-action with the total control of the hand-drawn cel animation.

The most famous movies to benefit from automatic or semi-automated rotoscoping tools are of course Richard Linklater’s rotoscoped A Scanner Darkly (2006), and Ari Folman’s Waltz with Bashir (2008). You need to look at these award winning movies.

to be continued…. see following post

From Cel to CGI: Animation as the Protean Medium

In the early 1940s Sergei Eisenstein wrote a series of articles on the animated cartoons and films of Walt Disney, (published as Eisenstein on Disney) admiring and envying the protean nature of the animation media – the ability of the animator to draw (or create in whatever medium) whatever came into his or her imagination, and to express that directly in their films, often more-or-less single-handedly. In the period that Eisenstein was writing of (1930s-1940s) this total freedom was of course denied the live-action film-maker, who was limited to a palette of cinematography, in-camera effects, optical special effects, editing – and of course the essential métiers of fictional film-making – the production design, costume, make-up, styling, editing and (later) sound, music, colour, etc.

In animation, of course all these skills could be, and often are, under the control of a singular creative artist-animator, or at most a small team of animators who handled the in-betweening – the drawing of the frames that ensured smooth animation between master frames drawn by the chief animator. Other artists might be deployed painting backgrounds, making the sound-track, voice-recording, music and sound effects, etc.

For example, looking at the work of the most famous animators of the period 1920-1940, we have a plethora of examples, including Disney, of the total freedom of the animator. Of course, the pioneer of this form was Emile Cohl, working as early as 1908 with little masterpieces of animation like his Fantasmagorie:

Emile Cohl: Fantasmagorie 1908. Emile Cohl has been described as the ‘originator of the cartoon film’, and in their classic work The Technique of Film Animation (British Film Academy 1959), John Halas and Roger Manvell describe his work: “The French pioneer Emile Cohl put a series of black-line drawings on sheets of white paper and photographed them. On the screen he used the negative film, and so achieved an action performed by white figures on a black background. It might be claimed in his case that he was less concerned with developing any technical skill than with showing the simple charm possessed by his little match-stick creatures. Even at this primitive stage, the emphasis was artistic rather than technical.” Cohl was the first interesting animator, developing some of the basic techniques (principles) of animation – the idea of shooting a series of drawings and projecting these at a frame-rate fast enough (12 frames/sec) to fool our eyes and brains into seeing movement. At first working in France, he spent some time in the USA, teaching animation techniques. What Halas and Manvell have to say about Cohl is of course quite true, his animations were more a proof of concept than a technical demomnstration of how it could be done, However the illustration above shows that even this ‘humble pioneer’ was able to see how, and invent one of the stock-in-trade techniques that we now call ‘morphing’ (from morphology – study of form and shapes), whereby we gradually change one form into another in a sequence of minor changes) could create fascinating animations, and that animation itself could become not only a mass-cinema medium, but an art-form in itself.

Emile cohl: Un Drame 1908. Cohl’s surreal style of drawing, fully exploiting what Eisenstein called plasmaticness – the ability to change or mutate or transform one object into another at will – illustrates the complete freedom of the artist.

But at this period (1920s-1930s) it wasn’t just the animators who were exploring the seemingly infinite potential of drawing. New insights had come from the artist Paul Klee, based on his teaching practice at the Bauhaus c1920-1930. Klee published his ideas on drawing in two books: The Thinking Eye (covering 1921-22) and The Pedagogical Sketchbook. He annunciates the idea later deployed by the Surrealists of ‘taking a line for a walk’ – the Surrealists called this psychic automatism and invented randomised drawing games like the Exquisite Corpse to exploit this creativity. But other influences emerged from popular culture, notably in the then very young art of the newspaper comic strip, and in this metier, it is the drawing art of George Herriman and his remarkable series Krazy Kat that spring to the fore.

George Herriman: Krazy Kat 1938. Herriman is the great poet-artist of the comic form – a truly unique genius in this medium. His subtle, sardonic, insightful humour, married with his gentle surrealism – and the life with which he endows his tripartite cast of characters, makes this a really exceptional work. Ignatz the mouse hates Krazy Kat, who in turn really loves him, and interprets Ignatz’ incessant brick-throwing as a sign of affection.Krazy hates Offissa Bull Pup who in turn loves Krazy and hates Ignatz. With this slender yet richly fruitful plot-idea, Herriman explores the vagaries and frailties of the human condition and our attempts to relate to others – producing delightfully spare drawings in a series that picture the Jewish Ignatz, the black Kat and the WASP Pupp in a strangely urbanised Arizona Desert (Coccocino County) a landscape that often morphs radically from frame to frame. Herriman drew this strip from 1913 to 1944. I’ve chosen to situate this in 1924 because it was the year that the art critic Gilbert Seldes wrote an appreciative paper on Herriman’s Krazy Kat, that positioned it as ‘serious art’. With his playfully surreal plots, his inventive drawing and experimental frame layouts, Herriman exerted an enormous influence on the development of the comic-form. Herriman epitomises the artist-draftsman, adventurous, innovative, practising Paul Klee’s advice to draw by taking a line for a walk.

The freedom of the cel-animator and claymation animator to seamlessly change a shape from one thing into another – to metamorphose or just ‘morph’ objects – is one the characteristics of animation that particularly impressed Eisenstein. Dave Fleischer – the animator who pioneered rotoscoping (more on this later) in 1917 was an exemplar of animation. One of my favourites is his Betty Boop series, especially this one:

Dave Fleischer: Betty Boop in Snow White 1933. Fleischer – always an innovative animator – epitomises the expressive morphing and distortion of characters – here the Evil step-mother is asking her mirror who is the most beautiful one of all? when Betty (Snow White) enters the room.

While drawing a morph from one character to another was easy, creating credible, photo-realist transmutations or morphs was difficult, and it was an aspect of film-making that several directors played with before digitalisation – movies based on upon Robert Louis Stevenson’s novella The Strange Case of Dr Jekyll and Mr Hyde (1886) such as the 1912 version directed by Lucius Henderson, for example:

Lucius Henderson: Jekyll and Hyde (1912). As you can see Henderson relied on just about the simplest possible transition shot – Dr Jykell drinks the potion, bends down, there is a cut, and when he raises his head he is Mr Hyde. Other film-makers used short lap dissolves to achieve the transition – fading out on Jykell, cross-fading in with Hyde – a technique used on the very cusp of digital morphing software by Kevin Godley and Lol Creme in the elegant promo-video for their hit single Cry (1985). The first morph software that I came across was Gryphon Software’s Morph, available for the Mac around 1991-2.

Live-action film-makers utilised animation techniques of course, using stop-motion to gradually expose sequences that animated otherwise inanimate objects, or that ‘animate’ still photographs in a similar way. Or artists could paint, draw or scratch marks straight onto the film-stock, creating a camera-less animation technique. The Canadian Norman McLaren and New Zealander Len Lye excel at these kinds of animation (we’ll look at their work in a future post). Godley and Creme use the early portrait-photography head-rest in their 1985 promo-video Cry.

Godley and Creme: Cry 1985 Kevin Godley and Lol Creme were innovative film-makers as well as talented musicians. In their promo-video for the single Cry, they use a head-rest to help pin-register the eyes of several participants as they lap-dissolve between them. I talked to Kevin Godley a few years later about this video, and congratulated them on a successful use of an established optical effects to achieve this.

Of course, from about 1991-2 onwards, software companies like Gryphon had developed desktop versions of morphing tools (Gryphon’s Morph (c1992). This enabled the user to input two images, and control an automatic morph between them, controlling reference points for each image, specifying the number of in-between frames, and the output resolution and format.

Fantamorph desktop software. Most desktop morph solutions allowed you to input two resource images, then indicate which features should be retained in the final morph.

By the mid 1990s, the digital-film revolution was already underway on desktop computers like the Mac. We had software for non-linear editing (Adobe Premiere 1991) for compression (MPEG-2 1994, and Quicktime (1991). And with Gryphon’s Morph, we could make high-resolution animations from still images. Digital media has in the last twenty years or so, given us the tools by which we can successfully marry live-action cinematography with the effects taken for granted by hand-animators (cel and claymation animators) over the previous century. Another important step on the road to the development of Total Cinema.

To be continued….

Total Cinema and the Phantasmagoria

The blog is about Total Cinema, and like Total Cinema, it is a work-in-progress.

What do I mean by Total Cinema?

I am talking about films in which there appears to be no limit on the imagination of the director/creators in terms of their ability to visualise and illustrate their ideas. The amazing whirlpool battle in Pirates of the Caribbean is an example, the seamless and invisible scaling of Gandalf and the Hobbits in Lord of the Rings another, the low-Earth orbit space sequences in Gravity, – and so on.. Mostly Total Cinema exemplars are those films that rely upon a large component of computer-graphics (CGI) in their production. But they are ‘Total’ also in their potential to integrate – and to illustrate – the many different components of digital art (see later).

Ettiene-Gaspard Robertson: Phantasmagoria – 1802. Robertson’s ‘assembly of phantoms’ was a big hit in London in the very early 19th century. It utilised all the tricks of theatre and stage magician (much as Georges Melies did a century later) to bring ghosts, spirits, and phantoms to his audiences, whose screams and ejaculations would compliment Robertson’s own sound effects (the use of zithers, Chinese cymbals and other strange instruments). Robertson was a pioneer of the use of the new powerful (6-10 Candela) Argand Oil Lamps (invented 1780), used back projection onto theatrical scrim (thin gauze), and smoke and mirrors too…

The essential back-story of Total Cinema can be traced from early 19th century theatrical son et lumiere presentations like those of Ettiene-Gaspard Robertson and Paul Philipdor, called Phantasmagorias (‘assembly of phantoms’, from c1802). Tom Gunning has written an essay about Phantasmagoria (Illusions Past and Future), as theatres of the senses, using illusions, stage magic, smoke and mirrors and importantly the Argand oil-lamp-powered magic lantern projector, and traces it’s transmutations through the 19th century into Symbolist Theatre d’Art (1891), mentioning the mid-century impact of Wagner’s theories of the Gesamptkunstwerk (1849). By the end of the century of course we have the technologies of the gas-light and electric-light powered magic-lantern slide projectors, and the emergence of various moving-picture systems, culminating in the feast of immersive media technologies at the 1900 Paris Exposition. And the cultural historian Marina Warner examines the Phantasmagoria of Robertson, in her book of the same name examining the mediated products of fantasy.

1802_Philipdore_Phantasmagoria_magic-lantern_c

Paul Philipdor/Ettiene-Gaspard Robertson: Magic-Lantern projections in Phantasmagoria 1802. Robertson and Philipdor mounted their magic lantern on rails so that it could be moved backward and forwards to englarge and reduce the projected image. To an audience unused to large-scale moving images, this had a considerable shock effect.

“The phantasmagoria derived directly from the camera obscura and magic lantern shows and many displays had been staged before Robertson’s struck a chord with the public; but Robertson’s Gothic horror spectacular, and its many brilliant twists and devices, turned any spectator from a cool observer into a willing, excitable victim, Whereas the diorama and panoramas concentrated on battles, modern cityscapes, or exotic scenery, customs and people – they are the forerunners of the wide-screen epic film – the phantasmagoria shadows forth great silent movies like F. W. Murnau’s vampire movie Nosferatu (1919) and Robert Weine’s The Cabinet of Doctor Caligari (1919).”

Robertson’s ‘brilliant twists and devices’ included the Argand magic-lantern projector, mounting the projector on rails so that the projected image could be ‘zoomed’ in and out, the use of theatrical scrim (see-through muslin), back projection,

(Marina Warner: Phantasmagoria (2006) pps 147-148

Fura del Baus: Das Rheingold at Houston 2013. Fura del Baus are an adventurous and innovative production company. This is their recent interpretation of Wagner’s Das Rheingold. It illustrates in contemporary media some of Wagner’s aspirations for a total (composite) art work, that he called Gesamptkunstwerk in his 1849 paper: The Artwork of the Future Das Kunstwerk der Zukunft.

In 1967, I wrote a very juvenile but well-meaning (unpublished) essay on the Gesamptkunstwerk (Composite Art Work) for my degree year at Portsmouth College of Art. In summary it traced the immersive, multi-sensory and multimedia aspects of the theatrical arts from Greek Theatre through to 19th century music-hall, Richard Wagner, and into the 20th century via Edward Gordon Craig, Brecht, and Antonin Artaud, wrapping up with the then contemporary practitioners of immersive mediated environments, including Stan Vanderbeek (Moviedome 1963), John Cage and Al Hansen (Happenings 1959).

Stan Vanderbeek: MovieDome 1963 Vanderbeek studied at Black Mountain College where he met some of the great experimentalists of the 1960s – John Cage, Richard Buckminster Fuller, Merce Cunningham, Josef Albers. Vanderbeek was an impressive film-maker, impressing amongst others, Terry Gilliam and Eduardo Paolozzi with his collage-montage animations. The Moviedome was a construction inspired by Buckminter Fuller (the geodesic dome), that housed multi-projection displays of Vanderbeeks films and slide archives.

With the prescience of a 22 year-old, I knew straight away (I will trace some of the sources that inspired me in this essay) that this was the zeitgeist development of the time – the recurring themes of immersion, multi-media, multi-sensoria and synaesthesia (and later, increasingly interactivity), permeated the iterations of avant garde experimental art through the 20th century, culminating in the development of virtual realities, videogames, the internet and web in the millennial decade. Now of course, there are a proliferation of premonitory signals of the new art in what I am calling Total Cinema or the Cinema of Totality (currently characterised by spectacular fantasy and sci-fi movies of the ilk of Avatar (Cameron 2010), Gravity (Cuaron 2014), Pirates of the Caribbean (Verbinski et al 2003- present) and Lord of the Rings (Jackson, 2001-2003) – films where there has been apparently no technical limit to the imagination of the director.)

Edith Sitwell: preparation for Facade-An Entertainment from 1922. With music by William Walton and an improvised loud-hailer, Edith Sitwell performs her zeitgeist work, which received rave reviews for lots of formal innovations and later was made into ballet by Frederick Ashton. This kind of media-art-fusion, made highly contemporary by the improvised PA system, was another signpost on the multifold pathways to the art form of the 21st century.

2007_Verbinski_Pirates_Worlds-End_mocap_c

Gore Verbinski: Pirates of the Caribbean At Worlds End 2007. This was Verbinski’s third triumphant Pirates movie. I loved the way Verbinski compressed almost every possible cliche of the sea (Davy Jones Locker, Pieces of Eight, treasure-islands, ghost ships,) with new mythic hybrids, fusions of marine life and humans, – with all the facility of a great illustrator (see Mervyn Peake: Captain Slaughterboard Weighs Anchor, and Treasure Island). Here’s a binary shot of just one scene, showing the mo-cap performers rigged for motion capture, and the final scene.

I want to wrap this idea up now by saying, along with Tom Gunning, Lev Manovich, and recently Kristen Whissel, that the developments I am going to trace in this series of blogs are the seedlings of a 21st century art form that will draw from these themes and iterate them into a coherent and continually developing series of new art-forms that will be as particular to the 21st century as Cinema, Television and Computing were to the 20th. And all or most of these forms will include moving images of some kind. The cinema may be dead, but the Movies will develop and expand, enveloping us in a new immersive form that will integrate multiple media, incorporate social media, involve interactivity, and provide us with limitless scope for self-expression, for recording the world, for communicating with others, for exploring ideas. This is the media of our time.

Sergei Eisenstein: Eisenstein on Disney (1941,republished 1993)

Tom Gunning: Illusions Past and Future: The Phantasmagoria and its Specters 2004

Marina Warner: Phantasmagoria 2006

Lev Manovich: Software Takes Command 2014

Kristen Whissel: Spectacular Digital Effects – CGI and Contemporary Cinema 2014

Frank Rose: The Art of Immersion 2012

Stephen Keane: Cinetech – Film, Convergence and New Media 2007

Jean Cocteau: mythopoeic movies – La Belle et la Bete 1946

Jean Cocteau: La Belle et la Bete 1946. Josette Day as Belle and Jean Marais as the Beast. The costumes, by Christian Berard and Marcel Escoffier, (and with an uncredited Pierre Cardin as costume-maker); with production-design by Lucien Carre, Antonio Castilo, and Berard, compliment the (uncredited) brilliant prosthetic make-up of the Beast. The fabulous black and white cinematography is by Henri Alekan (who shot the GregoryPeck/Audrey Hepburn modern day fairy-tale Roman Holiday in 1953)

Cocteau’s poetic approach to film-making is summed-up in his collection of writings on Film, published as The Art of Cinema posthumously in 1994.

“A cinema studio is a factory for making ghosts. The cinema is a ghost language that has to be learned. It is incredible for a poet to know this. The day when a director understands that the author’s role is not limited to the script (to writing it) – the day when the author reveals himself – then the dead language of cinema will become a living language.”

Cocteau: Album de Cinema, 1943 from Cocteau: The Art of Cinema 1992

“The film unfolds to a perfectly poised slow tempo in surreal settings that gain intensity from the fabulous decor and costumes, the glittering lighting and the many moments of magic effects. The candle-lit shadow-play deepens the gothic atmosphere of the beast’s castle, harking back to early expressionist classics of the genre.”
— Marina Warner

“When Jean Cocteau made his film of La Belle et la Bête in l946, he imagined a beast who has no rival for hideous fascination among fairytale beasts before or since: Jean Marais’s growling, slowed, incantatory delivery, his sweeping, elaborately princely magnificence of apparel, his thick pelt curling out exuberantly from his lace collar and fine linen as he springs and lopes, and, above all, his staring pale eyes in the great leonine and brindled mask of his face with the two sharp incisors defining his mouth, has never been matched for erotic power. He captures a perfect and irresistible synthesis of repulsiveness and attractiveness, which evokes a genuine frisson of desire in me every time I see it; Cocteau’s realization of a dream about the power of love, as personified by Belle, exercises its seduction again and again over me, even while my common sense, my reason, and a long habit of skepticism about romance tell me I must resist. “

(Marina Warner from the DVD Notes for the BFI release of La Belle et la Bete)

Jean Cocteau: La Belle et la Bete 1946. The Prince aka the aspirant lover Avenant aka the beast, played by Cocteau’s lover Jean Marais with Josette Day as Belle.

Cocteau’s mythic films are his poetic response to the surge of interest in mythology, mythography and the psychology of myth that pervades this period. (A remarkable period for English writing too – in these years we have J.R.R. Tolkein writing his Lord of the Rings, Mervyn Peake creating Gormenghast, C.S. Lewis and his Narnia chronicles, J.R. Eddison and his Worm Ouroborus, as well as the mythographic analyses of Joseph Campbell (The Masks of God), Robert Graves (The White Goddess), etc), the components of Carl Jung’s writings (Psychology and Alchemy, Flying Saucers – the Myth of Things Seen in the Sky,), and of course Jackson Pollock’s paintings of the early-1940s, exploring Jungian archetypes (eg Stenographic Figure, 1942). Of course, with Cocteau, his primary mythopoeic drive is to interpret the mid-20th century zeitgeist in the forms and with a poetic awareness of the archetypal forms to be discovered in fairy stories, in mythologies, in dreams, in archetypes.

In Christian Berard and Lucien Carre’s gothic sets with the real arms holding the flickering and guttering candelabra, Belle (Josette Day) is lost in a dreamlike corridor in the Beast’s castle.

Interestingly in one of his last publications, the mythographer Joseph Campbell (famous by then not just for his vast body of mythographical work, but also because of his role in inspiring the mythology of Lucas’ Star Wars), describes the role that mythography played in his own quest for self-realisation (Pathways to Bliss – Mythology and Personal Transformation, 2004)

“Myth is not the same as history: myths are not inspiring stories of people who lived notable lives. No, myth is the transcendent in relationship to the present. Now, a folk hero is different from the subject of a biography, even when the hero may have been a real person once upon a time – John Henry or George Washington. The folk hero represents a transforming feature in the myth. When you have an oral mythic tradition, it’s right up to date. In the folktales of the American Indians, you have bicycles, you have the form of the Capitol dome in Washington. Everything gets incorporated into the mythology immediately. In our society of fixed texts and printed words, it is the function of the poet to see the life-value of the facts round about, and to deify them, as it were, to provide images that relate the everyday to the eternal.” (my italics)

(Joseph Campbell: Pathways to Bliss pps XVI 2004)

Cocteau is well aware of this poetical driver:

“A film is not the telling of a dream, but a dream in which we all participate together through a kind of hypnosis, and the slightest breakdown in the mechanics of the dream wakens the dreamer, who loses interest in a sleep that is no longer his own”

Jean Cocteau: The Art of Cinema pps 40)

Cocteau describes the experience of film, in a similar way to how William Gibson describes the experience of cyberspace: “Cyberspace. A consensual hallucination experienced daily by billions of legitimate operators, in every nation, by children being taught mathematical concepts… A graphic representation of data abstracted from banks of every computer in the human system. Unthinkable complexity. Lines of light ranged in the nonspace of the mind, clusters and constellations of data. Like city lights, receding…”

(from William Gibson: Neuromancer, 1983)

Jean Cocteau: Mythopoetic Movies – Orphee (1949)

Jean Cocteau, following the surrealists, really epitomised the magical realism or cine-poetic approach to film-making. Not many poets had tried their hand at movie-making before Cocteau, especially not poets particularly fascinated by the mytho-poetic dimensions of fable, fairy story and myth. Cocteau’s two signature post-WW2 movies – La Belle at La Bete (1946), and Orphee (1949) are absolutely must-see films in this refined genre – pioneer films that still stand pretty much on their own, despite the spate of fantasy and magic-realism films that have emerged coevally with modern CGI and digital sfx.

As a reminder of Cocteau’s ingenuity during this post-war period, when film stock was hard to come by, finance was difficult, Cocteau was 57 years old in 1946 when he made La Belle et la Bete, had recently recovered from an opium addiction. Cocteau could not afford the kind of expensive optical and special effects that Hollywood took for granted. He had to invent his own ‘effects’ to bring his re-telling of the story of Orpheus and the Underworld to the screen. In-camera effects and simple special effects were his narrative tools.

1949_Cocteau_Orphee_rubber-gloves_strip_c

Jean Cocteau: Orphee, 1949. Cocteau, strapped for finance, had to use his ingenuity in making the special effects necessary to illustrate Orphee’s transition from the real to the mythopoetic world of the underground. Cocteau chooses the device of the mirror, clever cutting, and a highly reflective basin of mercury. In this sequence we see Orphee approaching the mirror (or is it a pair of hands approaching Jean Marais, wearing the same rubber gloves? Also in this part of Orphee, we see Cocteau reversing the film so that Orphee’s rubber gloves seem to snap on instantly. These simple tricks are very impressive and disbelief-suspending.

Cocteau was making these movies in the context of a surge of interest in poetry and myth signalled by James Joyce: Finnegan’s Wake (1939), Erwin Panofsky: Studies in Iconology (1939), Joseph Campbell: A Skeleton Key to Finnegans Wake (1944), Robert Graves: The White Goddess (1948), Joseph Campbell:The Hero With A Thousand Faces (1949). In the wake of the mythic horrors and devastation of the Nazis and World War 2, and the dawn of the Atomic age with its apocalyptic threat, these timeless subjects came to the fore as mankind searched for new meaning as first peace, then a new ‘cold war’ era, dawned.

Although not a member of the ‘official’ Surrealists, Cocteau’s trilogy of films related to the legend of Orpheus: Blood of a Poet ((1931); Orphee (1949), and Testament d’Orphee (1960) fit well within the surrealist’s spectrum of interests (the subconscious, dreams, myths, universal archetypes, magic, chance, automatic writing, etc). Perhaps, like Appollinaire, Cocteau, in his twenties, had been influenced by the fantastic film series of Louis Feuillade (Fantomas (1913), Les Vampires (1915), Judex (1916). None of these films relied on much more special effects than George Melies had at his disposal, it was their dark mystery and hints of the supernatural that the Surrealists admired. Interestingly Georges Franju made a brilliant re-make of Judex in 1963.

Other significant surrealistic films that Cocteau almost certainly would have seen include:

Rene Clair: Entr-Acte 1924

Lotte Reiniger: Die Abenteur des Prinzen Achmed 1926

James Sibley Watson/Melville Webber: The Fall of the House of Usher 1928

Germain Dulac and Antonin Artaud: The Seashell and the Clergyman 1928,

Man Ray: L’Etoile de Mer (1928)

Jean Epstein: The Fall of the House of Usher 1928

Luiz Bunuel/Salvador Dali Un Chien Andalou 1929

Marx Brothers: Animal Crackers 1930

Carl Dreyer: Vampyr 1932

Marx Brothers: Duck Soup 1933

Dave Fleischer: Betty Boop as Snow White 1933

Marx Brothers: A Night at the Opera 1935

Charlie Chaplin: Modern Times 1936

Curtis Harrington: Fall of the House of Usher 1942

Maya Deren/Alexander Hammid: Meshes of the Afternoon 1943

Walt Disney/Salvador Dali: Destino 1945

While this is speculation (on what Cocteau saw), I wanted you to see that Cocteau’s work, though unique, fits within a broader tradition of artists exploring the unconscious, magic, myth and fable, and the absurdity of modern life.

Of course, making serious films in post war France, even with financing from the generous Duc de Noaille was difficult and to optimise his visualisation of the Underworld, Cocteau was forced by circumstance to be inventive with his special effects, resorting to the simplicity of the smoke and mirror ideas of Melies, and using in-camera effects.

Special effects can often be as simple as ‘in-camera’ effects – you know: the iris fade to black/or white, double exposure, running film in reverse, stop-motion, elapsed time, slow motion, rotating camera, inverting camera, using negative as positive, (etc).

Jean Cocteau: Orphee (1949). Heurtebise, Death’s chauffeur, follows Orpheus through the mirror into the Underworld, distance-fogging (under exposure) underlines the illusion.

Jean Cocteau: Orphee (1949) Heurtibise in the ‘reel’ world sees Orpheus in the Underworld.

http://www.criterion.com/films/610-orpheus

Doug Trumbull: Special Effects on 2001 A Space Odyssey

Modern special effects – certainly in my experience – really date from 1968 – the year that Stanley Kubrick’s 2001 A Space Odyssey was released. Of course, 2001 was made without any computer-generated imagery, but the aesthetics, the ‘visualisation of a near future that hadn’t quite got off the drawing board yet’ makes Kubrick one of those interesting artists who explored analogue prototypes of the digital (CGI) future that would be glimpsed a decade or so later – in the late 1970s (see previous post). Kubrick employed a 23 year old animator called Doug Trumbull to create some of the special effects he needed for 2001. Interestingly, Trumbull’s parents had been a mechanical engineer and an artist – a perfect gene-pool for a special-effects artist!

Previously Trumbull had been making detailed animations about space travel, for the company Graphic Films in LA, and when Kubrick approached his colleague Con Pederson about the sfx on 2001, Trumbull also pitched for the job (eventually both Pederson and Trumbull were employed by Kubrick,, along with Tom Howard and Wally Reevers).

in the 1960s special effects were still largely to do with model-making and clever cinematography, and what was clever about 2001 was two things: the way the camera was controlled mechanically, so that Kubrick could shoot the high-resolution sequences marrying together live-action foregrounds with front-projected very high resolution backgrounds; and secondly, the new ‘Slit-Scan’ process invented by Trumbull for the Star-Gate sequence in the film.

Trumbull lists all the important innovations and workflow practices for 2001:

• the creation of a control room/riling system/idea centre – a room where all the ideas, storyboards, visuals, status reports, etc were kept and filed.

“One of the most serious problems that plagued us throughout the production was simply keeping track of all ideas, shots, and changes and constantly re-evaluating and updating designs, storyboards, and the script itself. To handle all of this information, a “control room,” constantly manned by several people and with walls covered by pert charts, flow diagrams, progress reports, jog sheets, punch cards, and every conceivable kind of filing system, was used to keep track of all progress on the film.”

Trumbull: Creating Special Effects for 2001 A Space Odyssey (American Cinematographer 49(6):416-420,451-453. June 1968.)

For a flavour of this data-base room, see:

http://www.mach25media.com/2001lacma.html

• the use of front projection and retroreflective matting instead of glass-matte paintings to create more realistic backgrounds (since the 1990s this has been superseded by blue-screen/green-screen systems.

http://www.visual-memory.co.uk/sk/2001a/page2.html

Herb A. Lightman : Front Projection for “2001: A Space Odyssey” American Cinematographer

Front-Projection special effects set-up. Instead of a glass painting, Kubrick adapted a front projection technique to create high-resolution back-drops for his Ape-Man sequences and other sequences used throughout 2001 A Space Odyssey. The rear screen is made of a highly reflective material made by 3M – its called a retro-reflective screen. The backgrounds were shot on 10×8 Ektachrome transparencies, and Kubrick had to construct his own 10×8 size projector. These background images were projected onto a semi-reflective glass screen, so that in one take the camera could record both foreground and background images in one take.

”

On the set of “2001: A Space Odyssey,” producer-director Stanley Kubrick uses binoculars to check fine focus on the vast front-projection screen. Special 3M material reflects 100 times the light falling upon the screen, provided that projector and film camera lenses are aligned on precisely the same axis. Kubrick wanted to use an 8×10 transparency for maximum sharpness, found there was no projector for it in existence, built his own.”

• Back Projection (rear-screen or back-projection was used in 2001 for shots of instrument panels)

1968_Kubrick+Trumbull_back-projection-scene_c

“Interior view of the Orion commuter ship approaching the Space Station, with all of its scopes showing readouts. Preparation of these readouts for all the spacecraft ate up tons of technical material and required many months to animate.” Each instrument screen on the cockpit dashboard was a back-projected animation. Here the interface graphics (the cockpit screen readouts) were animated diagrams and schematics from hundreds of technical journals, coloured with gel filters.

• Rotoscoping – where other cheaper or easier effects could not be used, Trumbull, Pederson and co used manually intensive rotoscoping’:

Max Fleischer – the animator later responsible for Betty Boop, Talkartoons and Popeye among many other cartoons – invented this rotoscoping rig in 1917. The idea of tracing live action as an animation tool led to the inricate special effects and compositing technologies in use today. Fleischer also developed ‘karaoke’-style on-screen bouncing-ball lyrics in the 1920s. Rotoscoping involved back projecting a live-action sequence frame by frame onto a translucent drawing frame, where the main actions the animator was interested in could be traced off onto a transparent animation cel.

“Since we couldn’t afford to tie up the animation stand, or any camera, for very laborious and time-consuming rotascope jobs on so many shots, a unique rotascoping system was devised. Using ordinary darkroom enlargers, equipped with carriers for rolls of 70mm film, each frame-by-frame image was projected onto specially marked animation peg boards, to which the projected image of the perforations had to be visually aligned.” (Trumbull, op cit)

• The best technical advisors/the cooperation of leading research/R&D labs on the promise of product placement:

“On the recommendation of Clarke, Kubrick hired spacecraft consultants Frederick Ordway and Harry Lange, who had assisted some of the major contractors in the aerospace industry and NASA with developing advanced space vehicle concepts, as technical advisors on the film. Ordway was able to convince dozens of aerospace giants such as IBM, Honeywell, Boeing, General Dynamics, Grumman, Bell Telephone, and General Electric that participating in the production of “2001” would generate good publicity for them. Many companies provided copious amounts of documentation and hardware prototypes free of charge in return for “product placements” in the completed film. They believed that the film would serve as a big-screen advertisement for space technology and were more than willing to help out Kubrick’s crew in any way possible. Lange was responsible for designing much of the hardware seen in the film.”
George D. DeMet : The Special Effects of “2001: A Space Odyssey”  Originallly published in DFX, July 1999

• Doug Trumbull: more on Slit Scan

Slit-scan was a technique of gradually exposing film, one strip at a time while the camera and/or the background was moved in stages. Trumbull adopted the idea from panoramic camera, and from the idea of the ‘photo-finish’ cameras invented to determine which horse had actually won a race.

output from a modern ‘photo-finish’ camera system. The camera records a temporal event, just as the panoramic camera recorded a spatial event.

1843_Puchberger_Panoramic-slit-scan_camera_c

Joseph Puchberge: Panoramic ‘strip-scan’ camera invented in 1843, for the purpose of taking wide aspect ratio panoramas.

Slit-Scan – the progressive, labour intensive, special effects process now wholly superseded by digital effects.

I’d like to compare Trumbull’s final ‘Star-Gate’ sequence from 2001 A Space Odyssey with a recently made video explaining the slit-scan process.

Trumbull: Stargate Sequence from 2001 A Space Odyssey (1968)

“As the black monolith vanishes into a strangely symmetrical alignment of Jupiter and its moons, the camera pans up and the “Stargate” engulfs the screen. For this infinite corridor of lights, shapes, and enormous speed and scale, I designed what I called the Slit-Scan machine. Using a technique of image scanning as used in scientific and industrial photography, this device could produce two seemingly infinite planes of exposure while holding depth-of-field from a distance of fifteen feet to one and one-half inches from the lens at an aperture of F/1.8 with exposures of approximately one minute per frame using a standard 65mm Mitchell camera.” (Trumbull: op cit)

Slit-scan recreated: John Hess explains slit-scan and recreates a slit-scan rig:

https://www.youtube.com/watch?v=KhRo2WbWnKU

Bernard Lodge: original Dr Who titles 1963. Working from a technical innovation called Howl-Around discovered and used by BBC video technician Ben Palmer, Bernard Lodge began his own experiments with video feedback (howl-around), much as Delia Derbyshire and Dick Mills had utilised audio-feedback in their production of Ron Grainer’s Dr Who theme. This produced visuals that are the video equivalent of Trumbull’s slit-scan effects 5 years later – a mysterious, electronic, temporal-spatial voyage through the new space time of cybernetic (video-feedback) space.

Slit-scan used in original Dr Who titles, designed by Bernard Lodge (1963)

Virtual Cinematography and Total Cinema

Ed Zajac: Simulation of a Two-Giro Gravity Gradient Attitude Control System (1963) – the earliest 3d CGI film that I have found. It illustrates the hypothesis that only two gyro-servos would be needed to orientate a satellite in orbit.

James Cameron: Avatar 2010. Movies that rely on on about 80% CGI are commonplace in the last decade or so. Cameron’s Avatar, like Cuaron’s Gravity (2013), Jackson’s Lord of the Rings (2001-4), Verbinski’s Pirates of the Caribbean (2003), depend upon considerable CGI, and demonstrate just how far we have come from Ed Zajac’s 1963 film.

From Ed Zajac’s wireframe – the first 3d animated model – (1963) to James Cameron’s Avatar (2010) CGI is less than 50 years old. But look how far we’ve come! Of course, under all the rendered surfaces of Avatar, we’ll find the familiar wireframe and polygon models.

Overview: CGI and film

Its important to understand some of the fundamentals of CGI – how 3d models are made, how they are animated, how they made life-like or credible. So here’s my essential guide to CGI:

In MediArtInnovation I’ll be examining all the actual and likely components of the emerging total cinema, beginning with the tools that characterise what John Gaeta (vfx designer on The Matrix series) calls Virtual Cinematography. These will include motion-capture, blue-screen/green-screen live action, motion-tracking, expression-capture, crowd-simulation, character-simulation, augmented reality, projection mapping and previz software.

Overview: CGI and film

The very beginnings of CGI on film are in the 1950s, when cameras were put in front of cathode-ray-tube (CRT) screens and the images on the screen either filmed in runtime (for example, at 24 frames/sec) or shot frame-by-frame onto a customised camera or film-recorder. This is the technique John Whitney (one of the pioneers of computer-animation) used for his contribution to the titles sequence for Vertigo. John Whitney uses analog computer animation in short films and in 1958, for Saul Bass: titles sequence for Hitchcock’s Vertigo.

Saul Bass and John Whitney: Vertigo film titles 1958

http://www.youtube.com/watch?v=3YNCJ6uGg1I

3d computer graphics uses the three-dimensional coordinates of X, Y, and Z – with usually, X as horizontal, Y as vertical and Z as depth dimensions. By including a unit scale the position and size of objects like this blue cube can be identified by its X,Y,Z coordinates.

By shuffling these coordinates (changing the numbers), objects can be repositioned, stretched, moved, distorted in any way numerically possible. Computer graphics is about coordinates and calculations.

In CGI, a ‘virtual’ camera (VC) can be defined anywhere in XYZ space. Unlike a real camera, a virtual camera can have an infinitely adjustable (elastic) lens – from extreme telephoto to ultra wide-angle – to macro – just by adjusting these values in software. The virtual camera can be moved anywhere in XYZ space at whatever speed the animator desires. The virtual camera can shoot at whatever frames rates are desired – from ultra slo-mo to stop-frame, and it can shoot in stereo 3d too. The virtual camera can be set to any aperture, any depth-of-field, any filter, any colour-space. A VC is not limited by gravity or mechanics – it is infinitely flexible.

1963_Fetter_Boeing-man animation_wireframe

William Fetter: Boeing Man wireframe 1963 These 3d animations were used to provide ergonomic design data for the planning of airplane cockpit instrument layout. To my knowledge they weren’t committed to film.

Early computer graphics, though only simple wireframes, pushed processing power at that time to its limits. Early computers would take several seconds to compute a 3d wireframe , several minutes to compute a complex 3d model, often hours to compute a rendered image (with surface textures, point-source lighting etc) Each generation of computing power allowed further refinements to CGI – such as surface rendering, smooth-shading, texture-mapping, transparency, smoke and flame effects, cloud effects, skin-rendering, hair-rendering, fur-rendering, trees and forests, fractal landscapes, realistic humanoid and animal movement (motion-capture), and so on.

This sequence of 2d wireframes is by Charles Csuri, one of the great pioneers of CGI. He was experimenting here with hand-drawn images that have been digitised. Csuri demonstrates what can be done by algorithmically distorting the images (stretching, squeezing, copying, pasting etc). Yes – in the early days of computer graphics, all these basic operations had to be discovered and illustrated!

In this plate (like the others above, from Gene Youngblood: Expanded Cinema, 1970), we see samples from John Whitney’s mesmeric computer-generated films, produced using his adapted analog computer, tinted with filters, and presented as triple-image projections in 1967).

Indicative of the kind of experimental computer-imaging techniques of the mid 1960s, Ken Knowlton’s discovery that computers can be programmed to calculate the grey-scale of tonal images:

Ken Knowlton: Studies in Perception No1 1966. The realisation that the computer was a medium dawned very slowly, really by increments as computer-imaging, then computer animation, computer-poetry, computer-music (etc) emerged in the 1960s. The far-sighted Jascia Reichardt celebrated the computer as an art medium in her 1968 exhibition at London ICA – called Cybernetic Serendipity. Knowlton’s was one of the computer-images featured. This is also reproduced from Gene Youngblood’s brilliant 1970 overview: Expanded Cinema (1970).

Ken Knowlton developed the Beflix (Bell Flicks) animation system in 1963, which was used to produce dozens of artistic films by artists Stan VanDerBeek, Knowlton and Lillian Schwartz.[9] Instead of raw programming, Beflix worked using simple “graphic primitives”, like draw a line, copy a region, fill an area, zoom an area, and the like.”

Stan Vanderbeek: Poemfield 1964-1967. The radically innovative Vanderbeek partners with Ken Knowlton – a great pioneer of computer graphics – to create computer-generated typographic poetry. “Poem Field is the name of a series of 8 computer-generated animations by Stan VanDerBeek and Ken Knowlton in 1964-1967. The animations were programmed in a language called Beflix (short for “Bell Flicks”), which was developed by Knowlton.” (wikipedia)

Stan Vanderbeek + Ken Knowlton: Poemfield 1964 – 1967

Vanderbeek was very helpful to me when, as a student, he sent me a bundle of material to help me with my 1967 graduation thesis on the Gesampkunstwerk – the composite art work. He studied with the composer John Cage, the dancer Merce Cunningham and architect-philosopher Buckminster Fuller (at Black Mountain College) was a very influential film-maker, and developed his famous movie-dome studio in Stony Point, New York.

http://www.youtube.com/watch?v=BMaWOp3_G4A

Bell Labs – a leading digital research centre at that time, made this short (16 min) documentary: A Computer Technique for the Production of Animated Movies (c1964), recording some of Knowlton’s research.

Big steps in CGI came in the late 1960s, early 1970s. Wireframes could be clad with polygon surfaces, animation became more sophisticated. The earliest feature film to carry CGI images was Richard Heffron’s Futureworld (1976), in which we glimpse some of these then state of the art advances.

Richard Heffron: Futureworld, 1976. Heffron revisits the familiar territory of Michael Crichton’s Westworld (1973), where Yul Brynner plays a robotic replica of a cowboy-gunslinger. Brynner makes a cameo appearance in Futureworld too – alongside Peter Fonda. But Futureworld was the first major feature to incorporate 3d CGI – and the clip below illustrates the state of the art in the mid-1970s.

Futureworld – while not a Michael Crichton project like his WestWorld, follows a fairly predictable story trope – Westworld was a futuristic theme-park ‘world’ where robotic cowboys populate a small western town. Adventurous theme-park visitors dress in character, and can live among the cowboy robots – they can pick gun-fights with them, even shoot them. The cowboy robots only use blanks in their guns. But something goes wrong with one of the robots, and he starts shooting back at the visitors, with real bullets.

(You recognise this plot in Jurassic Park and many of Crichton’s other hugely successful movies)

Same thing with Futureworld, only this time it wasn’t by Crichton.

http://www.imdb.com/title/tt0074559/

Stephen Lisberger: Tron 1982 ERarly chromakey/blue-screen matting of live-action filmed characters into a CGI wireframe set.

The next big step came with Disney’s Tron (Stephen Lisberger, 1982). Here the plot entails our hero, played by Jeff Bridges, actually entering the cyberspace world of a computer and battling with evil virus-code villains. This is the earliest movie to feature colour-keyed live action film (heavily tinted here to give a CRT-blue colour), optically combined with a computer-generated, hand-coloured wireframe set.

Lisberger’s 1982 storyboard/visualisation of a frame for Tron. Xerox photocopy machines had a considerable impact on graphics at this time, but the marker-pen was the usual storyboard tool,

Despite its obvious shortcomings compared to CGI now, Tron was a box-office success and prepared the way for more computer animation – leading a dozen years later to John Lassiter’s Toy Story (1995) – the first completely computer generated feature.

To marry CGI with live action means that the position of objects and people in the real-world (in studio movie-sets or on location) has to be synchronised with the ‘people’ and objects in CGI space. (And of course the camera-lensing, lighting and aperture has to be synchronised too). This is where two technologies come in: motion-capture (MOCAP) and motion-control.

MOCAP is the capture of an actor’s motion and movement. It is usually done in a green-screen or blue-screen studio. Motion control is feeding data on camera position, aperture, filters etc from the real-world camera to the virtual camera, so that images from both worlds can be aligned in perfect synch.

Before Digital: Optical and Mechanical FX

Of course, the world of virtual cinematography has grown out of, and ultimately was based upon, the pre-digital world of optical special effects (opticals or mechanicals), where production designs, directors and cinematographers built real 3d sets and used either in-camera or laboratory-processed special effects. These early effects techniques – like this example of glass-matte painting – emerged in the 1930s (it was used in Gone With the Wind, Selznick 1939), and were still being used in Steven Spielberg’s Raiders of the Lost Ark in 1982.

Optical FX was the name given to in-camera effects and laboratory (processing) effects. In-camera effects include all the ways you could manipulate the filmic image inside the camera – such as double or multiple exposures, iris-fades, rewinding film, matte-shots (where part of the lens is blocked-off by a cut-out mask or glass matte – the film is shot with the matte, then the film is re-wound and a different matte used to expose the film.

Diagram showing the set-up for shooting with glass matte paintings – the real location and the glass background painting are married together in the camera.

Another glass matte rig The compositing together was an in-camera effect – the foreground scene is shot through a matte, so that painting and live-action are composited together on the film negative.

Norman Dawn: Glass matte technique used in film 1907 According to the blogger Yang, at the excellent thespecialeffectsblog.blogspot: “Norman Dawn made several improvements on the matte shot to apply it to motion picture, and was the first director to use rear projection in cinema. Dawn combined his experience with the glass shot with the techniques of the matte shot. Up until this time, the matte shot was essentially a double-exposure: a section of the camera’s field would be blocked with a piece of cardboard to block the exposure, the film would be rewound, and the blocked part would also be shot in live action. Dawn instead used pieces of glass with sections painted black (which was more effective at absorbing light than cardboard), and transferred the film to a second, stationary camera rather than merely rewinding the film. The matte painting was then drawn to exactly match the proportion and perspective to the live action shot. The low cost and high quality of Dawn’s matte shot made it the mainstay in special effects cinema throughout the century. [2] Traditionally, matte paintings were made by artists using paints or pastels on large sheets of glass for integrating with the live-action footage.[1] The first known matte painting shot was made in 1907 by Norman Dawn (ASC), who improvised the crumbling California Missions by painting them on glass for the movie Missions of California.[2] Notable traditional matte-painting shots include Dorothy’s approach to the Emerald City in The Wizard of Oz, Charles Foster Kane’s Xanadu in Citizen Kane, and the seemingly bottomless tractor-beam set of Star Wars Episode IV: A New Hope.” http://thespecialeffectsblog.blogspot.co.uk

Glass-mattes – This technique was in widespread use in the 1930s (notably Gone With the Wind – Selznick, 1939) and involves interposing a glass painting between the camera and the live-action to add landscape or other ‘location’ cues, skyscapes, background scenery etc). With careful control of depth-of-field, this becomes an in-camera effect.

Doug Trumbull: Slit-scan rig for Kubrick’s 2001 A Space Odyssey, 1968

Slit-scan is an analog or manual technique in which a physical optical slit is used to progressively expose a frame of film. It is used for special fx, especially for elongating objects. It was used in the early 1960s by Bernard Lodge to create the Dr Who titles-sequence (1963), and later by the optical fx wizard Doug Trumbull for the stargate sequence in Kubrick’s 2001 a Space Odyssey (1968).

Doug Trumbull/Stanley Kubrick: Star-gate sequence from 2001 A Space Odyssey (1968) – a 9.5 minute sequence engineered using slit-scan (and some old footage from Dr Strangelove). This is the most remarkable example – outside movies that are about the effect of drugs – of the influence of counter culture experimental film-making upon major Hollywood productions.

to be continued…

The Cinema of Totality

(This is the first in a series of posts about the emergence of indicators of a Cinema of Totality)

Oscar Schlemmer: Triadic Ballet, Bauhaus 1922 Ideas of the gesamptkunstwerk – the total art work – drove both Walter Gropius (founder and director of the Bauhaus), and Lazlo Moholy Nagy, who in 1923 took over as head of the Preliminary (Foundation) Course from Johannes kitten. Oscar Schlemmer had his ideas too, inspired by von Kliest’s essay on the Marionette Theatre (1798), and concurring with Moholy Nagy’s vision of a theatre of machines.

Back in 1924, in his book Theatre of the Bauhaus, the artist-designer-film-maker Lazlo Moholy Nagy, then in charge of the Foundation year at the Bauhaus, included a piece headed Theatre, Circus, Variety in which he described THE COMING THEATER – A THEATER OF TOTALITY:

“3. THE COMING THEATER: THEATER OF TOTALITY
Every form process or Gestaltung has its general as well as its particular premises, from which it must proceed in making use of its specific media. We might, therefore, clarify theatre production if we investigated the nature of its highly controversial media: the human word and the human action, and, at the same time, considered the endless possibie creator—man.”

“It must be said, however, that those ideas, in contradistinction to a literary-illustrative viewpoint, have unquestionably advanced creative theater precisely because they were diametrically opposed. They canceled out the predominance of the exclusively logical-intellectual values. But once the predominance has been broken, the associative processes and the language of man, and consequently man himself in his totality as a formative medium for the stage, may not be barred from it. To be sure, he is no longer to be pivotal—as he is in traditional theater—but is to be employed ON AN EQUAL FOOTING WITH THE OTHER FORMATIVE MEDIA.”

Examples of indicators of the emerging Cinema of Totality. The Wachowsky’s The Matrix (1999) in many ways mapped out the new ‘virtual cinematography’ – the intricate compositing of live-action-motion-capture-CGI, freeing the camera both in space and time.

I believe that we are witnessing the early stages of a Cinema of Totality. Very early stages, I suggest, as the true art of the 21st century is still only being hinted at in the great variety of emergent art-forms and media innovation that has marked the last twenty years. These nascent forms include interactive installation-art, augmented reality, virtual reality, e-books, web-sites, virtual worlds, projection-mapping, webcasting, geographical information systems, social media, fractals, CGI, robotics, simulation software, chatbots, machine-generated literature, games, MMORPGs, hypertext and hypermedia, multi-touch, gesture recognition, speech-recognition, machine-learning, video-art, artificial life, genetic algorithms, QR Quick-Response Codes, hologram projection, tamagotchi, exoskeletons, transmedia narrative, synthespians, software agents, Internet of Everything, wikipedia, internet archive, Skype, animatronics, interactive documentary, digital journalism, realtime motion capture, PreViz software, – and of course a swath of acronymic technologies, including: html, vrml, RSS, PHP, Java, CCTV, HD, QTVR, VVVV, RFID, GPS, GIS, etc.

Some key books have hinted at the possibilities here: Brenda Laurel: Computers as Theatre (1991), Howard Rheingold: Virtual Reality (1991), Clifford Pickover: Computers and the Imagination (1991), Steven Levy: Artificial Life (1992), Neil Stephenson: Snowcrash (1992), Noam Chomsky: Manufacturing Consent (1992), Scott Bukatman: Terminal Identity (1993), Howard Rheingold: The Virtual Community (1993), April Greiman: Hybrid Imagery (1993), Kevin Kelly: Out of Control (1994), Bob Cotton and Richard Oliver: The Cyberspace Lexicon (1994), Robert Pepperell: The Post-Human Condition (1995), Sherry Turkle: Life on the Screen (1996), Mitchell Resnick: Turtles, Termites and Traffic Jams (1997), Janet Murray: Hamlet on the Holodeck (1997), Bob Cotton and Richard Oliver: Understanding Hypermedia 2.000 (1998), Hans Moravec: Robot (1999), Scott McCloud: Reinventing Comics (2000), Bob Cotton and Malcolm Garrett: You Aint Seen Nothing Yet (2000), Jay David Bolter: Remediation (2000), Brenda Laurel: Utopian Entrepreneur (2001), Lev Manovich: The Language of New Media (2001), Martin Dodge and Rob Kitchin: Atlas of Cyberspace (2001), Lucien King: Game On (2002), Bob Cotton: Futurecasting Digital Media (2002), Howard Rheingold: Smart Mobs (2002), Jeffrey Shaw: Future Cinema (2002), Bill Moggeridge: Designing Interactions, Charlie Gere: Art, Time and Technology (2006), Stephen Keane: Cinetech (2007), Mike Faulkner: Audio-Visual Art and VJ Culture (2008), Jesse Schell: The Art of Game Design (2008), Erik Brynjolfsson and Andrew McFee: The Second Machine Age (2014), Lev Manovich: Software Takes Command (2014).

While most of these example films are blockbusters, and several are series (to amortise the original costs over several films), around 30% of the top 100 greatest grossing movies would fit in as illustrations of the totality of directorial freedom in the last decade or so. If you can imagine it, we can build it.

These were all hints as to the essential building blocks of a 21st century iteration of Wagner’s Gesamptkunstwerk (composite art work) – hints as to the ingredients of a new total art form. But I want to talk about a recent trend apparent in big-budget (blockbuster) movie-making. A trend that has several signs of a new Cinema of Totality.

Since the successful integration of (first of all) models and optical effects, then later, computer-generated imagery (CGI) with film (in movie-industry terms, going all the way back to Fritz Lang’s Metropolis (1927), Stanley Kubrick’s 2001 A Space Odyssey (1968), George Lucas’ Star Wars (1977), and Stephen Lisberger’s Tron (1982), but more particularly and perhaps more spectacularly, in James Cameron’s Terminator films (from 1984), then in a string of mostly hugely successful films that synthesise CGI, motion-capture, scenography, prosthetics, analglyptic 3d, virtual cinematography as well as multi-channel sound, costume design and other traditional filmic arts. These include: (in chronological order):

Luc Besson: The Fifth Element (1997)

Wachowskis: The Matrix series (from 1999)

Chris Columbus: Harry Potter and the Philosopher’s Stone (2001)

Peter Jackson: Lord of the Rings series (from 2001)

Gore Verbinski: Pirates of the Caribbean (2003)

Frank Miller, Robert Rodriguez: Sin City (2005)

Andrew Adamson: Chronicles of Narnia 2005

James McTeigue: V for Vendetta (2006)

Christopher Nolan: The Dark Knight (2008)

James Cameron: Avatar (2010)

Alfonso Cuaron: Gravity (2013)

(an indicative list only – you can think of many more films like this, in which the director seems to have total creative freedom)

In the last decade or so, it is computer graphics modelling and animation (CGI for short) that has catalysed this creative freedom – allowing director’s, production designers and cinematographers the freedom to think and to create entirely outside the constraints of gravity and space/time. From the very first high-level CGI I saw in the early 1980s (it was a much treasured showreel from John Whitney Jr’s Triple I studio) it was obvious that this was not only part of the future of animation, but an essential component in the future of movies.

Of course, CGI wasn’t a sudden phenomena – on the contrary, the art and science of computer graphics has a long history – with its foundations in the work of the MIT Lincoln Lab (light pen 1958), Boeing Aircraft Company (3d wireframes) in the early 1960s, and its breakthrough-moment came with the work of Ivan Sutherland and his 1963 doctoral hypothesis: the Sketchpad software (interactive, screen-based 3d modelling), and the work of the flight-simulation company he set up with David Evans (Evans and Sutherland) in 1968.

This is an abbreviated chronology of CGI:

Jay Forrester: Industrial (System) Dynamics 1958

William Fetter (then at Boeing)- coins ‘computer graphics’ in 1960

Steve Russell: Space War computer game 1961

William Fetter: wireframe 3d modelling including Boeing Man – ergonomic 3d wireframe of human body 1964

Arthur Appel: Hidden-line removal 1967

Arthur Appel: Ray-casting 1968

Henri Gouraud: Gouraud Shading – 1971

Michael Noll: raster-scan (bitmaps) 1971

Bui Tuong Phong: Phong Shading 1973

Ed Catmull: Texture mapping 1974

Martin Newell: Utah Teapot model 1975

Turner Whitted: Ray Tracing 1979

Benoit Mandelbrot: Fractal landscapes (used in Star Trek Wrath of Khan, 1982)

Rebecca Allen: Motion Capture for Twyla Tharp: Catherine Wheel (capturing and modelling realistic human motion)1983

William Reeves: Particle System (rendering fire, explosions, clouds, smoke etc)

Cornell University: Radiosity 1984

Jamnes Kajiya: Path-Tracing 1986

Will Wright: Sim City complex system simulation (based on Forrester’s System Dynamics) 1989

John Carmack and John Romero: Wolfenstein 3D realtime 3d first-person shooter 1992

John Carmack: Doom realtime 3d first-person shooter 1993

Kristin Dana: skin and cloth textures 1999

cel-shading 2000

Voxel Volume Ray-Casting

Stephen Regeleious: Massive/Prime crowd-rendering 2001

This is hardly a definitive list, but it gives some impression of the phenomenal development of computer graphics over the 54-year period: 1960 – present. The components of CGI – the hardware and software – are of course still developing. The software is a fascinating example of the success of the scientific collaborative sharing of knowledge aka the open-source ideal. The software packages used in the industry now (Maya for example) are the result of over 50 years of innovation by computer scientists, artists, CG specialists, and graphics programmers. The gradual accretion of algorithms for modelling and rendering is a phenomenal example of international collaboration, and has resulted in a set of immensely powerful tools for film-makers, (as well as for artists, architects, designers, scientists, medical researchers, etc, etc).

A scene from The Matrix – the universe of options echoes the range of potential techno-aesthetic components of the cinema of totality.

In this brief mapping of the trendings towards a Cinema of Totality, a few zeitgeist concerns resonate through the last 11 decades or so. One is immersion – the desire to surround an audience with light/sound /visuals/scenographics/information/signals/signs/meaning – another is interactivity – the trending towards active artist-audience-artist dialogue, machine-dialogue, interpersonal dialogue, intermedia and transmedia dialogue. Another is non-linearity – a zeit-form iterating through 11 decades in the atonal composers, in James Joyce (especially in Finnigans Wake, 1939), and in William Burroughs (early 1960s), and in Dziga Vertov (Man With A Movie Camera, 1929). And in Maturana, Cybernetics (Norbert Weiner, 1947), in Psycho-geography (Guy Debord 1954), in T.S. Eliot (The Wasteland, 1922). The other main recursive strand is multi-media, intermedia, transmedia (a mixing of media – film, ballet, literature, music, dance, poetry, theatre etc etc), and finally a kind of synaesthesia – a mixing of the senses.

These are the gradually evolving forms that will composite, catalyse and gell together into a radical 21st century art form, orchestrated by a student genius who gets it. Who gets the immediacy, the interlocution, the smartness, the genius, the richness of a form that marries sensory, kinaesthetic, physical, intellectual, algorithmic, lyrical and expressive modes of a form that is still being born, still nascent, still, like an unchecked lottery ticket, resonant with hope. We love it.

(best short history of computer graphics is at https://design.osu.edu/carlson/history/lesson2.html)

Alexandr Medvedkin: AgitProp Train 1923

Alexandr Medvedkin: Agit-prop Train scans from a German history of Russian Cinema, published c1935

“Medvedkin was head of the Russian propaganda train that travelled around the Soviet Union making Social realist documentaries, as the narrator ironically comments; “films made at the top to instruct the grass roots, footage gathered at the grass roots, to be edited at the top.” (from: http://www.acs.ucalgary.ca/~tstronds/nostalghia)

The Agit Trains were highly decorated railway carriages/goods wagons, each carriage a different form of agitprop (agitation+propaganda) – a theatre wagon, a cinema wagon, film-studio and film-processing, etc. There’s a wonderful recreation of an Agit-Train in Warren Beatty’s feature Reds (1981), which maps the life and work of the American communist and journalist John Reed.

Warren Beatty: Reds 1981 This brilliant feature is a biopic of John Reed, the American journalist and communist who covered the Russian revolution and civil war – 1919-1923

Medvedkin and Dziga Vertov were two of the film-makers actively involved in the Bolshevik AGITPROP trains during the Russian Civil War. These trains toured the Russian countryside distributing leaflets, staging plays, showing films, illustrating the benefits of the Revolution for an illiterate peasantry. The AGITPROP trains are wonderful examples of how multimedia – and film and electronic media, were creatively developed as ‘education, agitation and propaganda’ media.

Medvedkin’s Agit-train logotype from c1933

“…one of the primary aims of the agitprop trains was health education. Vertov and Medvedkin filmed people during the day, edited all night, then played the film back to their audience/participators next day. When they played another village the film, they found that these agitprop media had the power to unify…”(Henny 1983) at: http://books.google.com/books?id=6Ldyu8v6c-IC&pg=PA85&lpg=PA85&dq=medvedkin+agitprop&source=web&ots=d7iTgDScya&sig=crecn8NkdbvdJDJfoRe-g07CgWM#PPA85,M1

Alexandr Medvedkin: Happiness 1935 Medvedkins most successful film, a slapstick social comedy set in Russia before the revolution, and in the USSR before Stalin’s Collectivisation (mass-redistribution and state-ownership of land) in the early 1930s. “The original print featured an experimental color sequence illustrating a poor peasant’s dreams of becoming a king.[1] It was the first color production of the Mosfilm studio. According to Medvedkin, “At that time the peasant could not dream of anything multicoloured of beautiful. His dream was limited, and in that the technology helped”. The sequence was discarded on account of its poor technical quality and is apparently lost.” (wikipedia)

	Shea on Jacqueline de Jong: Situationi…
	Task 2.3: History of… on graphic design in the 1920s
	bobcotton on Richard Buckminster Fuller: Ge…
	Quels enjeux derrièr… on Richard Buckminster Fuller: Ge…
	Ep. 100 – 2001… on Doug Trumbull: Special Effects…

media+art+innovation

A forum about innovation in old and new media