Following the launch of National Theatre’s smart caption glasses in October, disabled writer, artist and curator, Sandra Alland, explores the development of captioning for live performance, highlighting a range of international innovations.
In this in-depth article Alland covers the following areas (follow links to go to relevant section):
Definitions and history of open and closed captioning – Alland describes the key differences between open and closed captioning by highlighting some of the earliest instances and innovations of each within the performing arts in Canada, the United States, The Sultanate of Oman and Austria.
Phone apps for closed captions – Alland compares the variation, in cost and options, of portable and flexible closed-captioning apps provided by GoTheatrical! (Australia), Talking Birds’ the Difference Engine (UK) and Softlab’s Stage-sync (Spain).
National Theatre’s smart caption glasses – Alland explores the high-tech innovation offered by the UK’s National Theatre, which automates and personalises closed captioning while also addressing issues like ‘visual ping-pong’.
Integrated captions as a political and artistic tool – Citing the UK’s Graeae and Birds of Paradise theatre companies and Germany’s Un-Label, Alland highlights integrated open captioning – which not only provides access and awareness, but also creatively enhances the show for all audience members.
Definitions and history of open and closed captioning
Open captioning presents captions of a theatrical production to the entire audience. Although there are usually seats with optimal viewing, the words are visible – or ‘open’ – to everyone.
During Canadian Opera Company‘s 1983 production of Elektra, Iranian-born General Director, Lotfollah Mansouri, oversaw the first-ever projected slides of a libretto. For many years after, opera translation for English-language audiences was the most-used form of open captioning.
Surtitles – text displayed above or beside the operatic action – didn’t really constitute accessible open captioning, as they didn’t include descriptions of onstage sounds for Deaf, deafened, hard of hearing and neurodivergent audiences. But, thanks to the work of Canadian Opera Company, opera became a dependable genre for text of what was said/sung onstage, much like subtitled ‘foreign-language’ films at the cinema.
As the demand for access to the arts increases, open captioning is becoming more standard throughout all genres of live performance. But unlike opera companies, theatres usually offer very few captioned performances throughout the run of a play. Based on access for some, rather than translation for all, theatrical open captioning is programmed similarly to sign language interpreting and audio description.
Its users, therefore, encounter similar problems. Having ‘special’ performances on select dates is not only limiting, but also othering – as if there’s a separate ‘D/deaf space’. This can additionally lead hearing or non-disabled audiences to view accessible performances as less-desirable experiences.
Closed captioning, in contrast with open captioning, is solely visible to people who want it. Best-known as a form of access for television, closed captions can usually be turned off, and sometimes users can personalise text size and colour.
Over the years, some opera companies responded to criticisms of surtitles – poor sightlines, lighting interference, and ‘distraction’ from the action – by installing small screens in the back of seats or on railings. In 1995, New York’s Metropolitan Opera launched the first electronic libretto system, Met Titles, for about $1,000 per seat. Many international opera houses now use similar closed caption/subtitle systems, often offering multiple languages. Screens are polarised to reduce glare, and can sometimes be turned off.
In 2011, Royal Opera House Muscat in the Sultanate of Oman introduced Radio Marconi multimedia interactive screens. Audiences could read subtitles and production information, watch related videos, or call for assistance. In Austria, the Lemon42- and Marconi-developed software at Vienna’s Wiener Staatsoper currently offers six languages and production information for each opera. This ‘infotainment’ system is available in front of your seat and on a large overhead screen (an interesting combination of closed and open captioning).
The Canadian Opera Company still uses its trademarked open surtitles, and it’s unclear whether seat-based screens – or more hidden alternatives like the rear window captioning system – make watching while reading easier or better. Spoiler alert: as several of my interviewees made clear, there’s no ‘best’ or ‘one size fits all’ solution to captioning. But it’s certainly thrilling that D/deaf and disabled people finally have options.
Phone apps for closed captions
Downloadable apps for phones and tablets are currently the most affordable and popular form of closed captioning for theatre. Audience members can use a device they’re already familiar with, and sit anywhere in a space.
In Adelaide, The Captioning Studio’s GoTheatrical! began offering closed captioning and audio description on personal apps in 2012. A main provider of open captioning in Australia since 2004, GoTheatrical serves theatres of all sizes, including Sydney Opera House. Whether choosing open captions on lightweight and affordable televisions, or closed captions via an app, companies can display up to eight lines of text at once, and employ various colours to show when different actors are speaking or signing.
4-minute video from The Captioning Studio’s YouTube channel, featuring the 2012 launch of their GoTheatrical closed captioning app, with testimonials from D/deaf audience members.
Yet, companies using GoTheatrical still seem to program few captioned shows, and with either open or closed captions. Melbourne Theatre Company has only one open-captioned performance per production (and no CC shows). In its six-week run of The Dance of Death, Belvoir St Theatre offers just one show with the GoTheatrical app (and no open captions).
Some Australian companies could be captioning sparsely because remotely-operated captions incur a cost per show. But I suspect there’s also consideration of the ‘distraction’ element for hearing and non-disabled audiences – who might complain about sitting next to people holding devices. Because access is still framed as a ‘special’ service, closed captions can, therefore, come with their own sense of othering – and one that can feel personal.
In 2010, Coventry’s Talking Birds Theatre Company created the Difference Engine, a phone app similar to GoTheatrical. But the Difference Engine is operated onsite by theatre companies themselves, from a laptop or Raspberry Pi. Designed by artists for artists, it delivers scripted or live closed captioning to mobile devices via app or browser, with the goal of offering captions for every performance of a show. It’s an affordable solution, especially for small-scale or site-specific companies, as you only need someone who knows the show’s cues (as opposed to a professional captioner on- or off-site). The app’s server can run without broadband, and on a battery pack.
‘We created the Difference Engine because we wanted to give more people the chance to access experimental, outdoor, small-scale or immersive performances,’ says General Manager Philippa Cross.
1-minute Talking Birds video, demonstrating their closed captioning app for personal devices, the Difference Engine.
Talking Birds offers training, but companies just have to divide scripts into sections of captions, then upload the text document to the system. Several lines of yellow words appear on a black background, with names of characters to differentiate between speakers. Talking Birds is developing a separate audio description app, like that of GoTheatrical, and testing a multi-channel version to deliver captions in translation. A single operator can deliver captions and AD through the same system.
Over in Spain, SoftLab at Charles III University of Madrid have taken things in a more automated, large-scale direction, with the development of Stage-sync. From GoAll’s single app on a personal device, you can choose between captions, Spanish Sign Language, audio description, or an impressive combination of captions and sign language. But what really sets this app apart is its use of artificial intelligence, deep learning and advanced audio processing.
‘What we’ve created’, says SoftLab research group leader Ángel García Crespo, ‘is a system that allows for automatic synchronisation with a theatrical production.’
After a successful inaugural tour of Spain with The Addams Family musical in 2017-18, UC3M’s Stage-sync is expanding to Brazil and the United States.
‘Theatre has a great level of complexity, because an actor can time their performance a bit too fast or slow,’ Crespo continues. ‘There can be a major pause, or applause. And all of this can de-synchronise (captions). There’s a server tasked with tracking where we are in the play. It learns from what’s transpired in other performances.’
1-minute YouTube video of Stage-sync’s Spanish Sign Language programming for The Addams Family musical. Captioned in Spanish, but you can still get the idea if you don’t read Spanish.
The Stage-sync app was also designed with creativity in mind. For The Addams Family musical, the character of ‘Thing’ (a disembodied hand) didn’t feature as much as in the film version. So, Thing was playfully incorporated into the app as the right hand of the sign language interpreter.
National Theatre’s smart caption glasses
One difficulty, either when reading open captions beside the stage or especially with closed captions from a phone or tablet, is looking back and forth from the action to the words. Perhaps personal device apps are therefore more ideal for non-static events, or lectures and tours that don’t focus as much on action. An innovation that addresses this issue of visual ping-pong comes from National Theatre, who have introduced software similar to Stage-sync’s – delivered in a vastly different form.
4-minute YouTube video of National Theatre’s new smart caption glasses, with feedback from D/deaf theatre practioners.
Enter smart caption glasses. Professor Andrew Lambourne’s speech-following software engine has been developed since 2014, in collaboration between National Theatre, Accenture, and the captioning service, StageTEXT. Glasses from Epson are fully adjustable, fit over personal glasses, and come with a tutorial and control pad. Individual users can adjust text size, position, colour and brightness, as well as choosing a background colour and border. Captions can be viewed as single lines or three scrolling lines. Best of all, you just turn your head to follow the action, and the captions travel with you.
Launched in October 2018 after a year of testing, closed captions via smart caption glasses are available from February 2019, at any performance and in every seat of all three National Theatre spaces. They’re subject to availability, but never fear – NT has 90 pairs.
2019 plans for the glasses include Leeds Playhouse, as well as a National Theatre tour of Macbeth. NT hopes to expand their purview to include sign language – like with Madrid’s Stage-sync handheld app. And, like the Difference Engine, they want to eventually include foreign language translation and live captioning. Lack of accuracy is often a downside of automated and deep-learning software, but National Theatre believes it can soon hit its goal of 95%.
Smart caption glasses, like some personal apps, are an option with wide flexibility – you’re not limited to one performance on a Tuesday afternoon. But, like personal devices, glasses can visibly other D/deaf and neurodivergent people. National Theatre’s online booking process also involves a lot of reading and isn’t yet intuitive. However, I imagine any booking quirks will soon be fixed, and National Theatre is impressively committed to providing options.
Jonathan Suffolk, Technical Director, says, ‘The NT doesn’t favour any one service, as we’re aware different types of services suit different people.’
Each production at National Theatre continues to also offer open captions on three to four performances. Suffolk adds, ‘The NT favours providing as many different ways of accessing our work as possible, so there is choice and as many people as possible can enjoy the work we make.’
Integrated captions as a political and artistic tool
Besides some D/deaf and neurodivergent people preferring open captions, there’s also value in open captioning as an educational and political device – and as a design element in its own right. Open captions make the rest of the audience aware of captions and the people who might need to use them. Significantly, if captions are always discreet, we risk invisibilising important achievements of disabled- and Deaf-led theatre companies who create works that are always accessible.
One such innovation is the creative use of open captions, so they not only provide access, but also add unique artistic effect. Take Graeae Theatre Company’s 2015 production, The Solid Life of Sugar Water. Captions were projected next to each person onstage, moving from the wall to other parts of the set. Like speech bubbles in a comic, the text was intimately connected to the characters, adding depth and humour.
‘We’re interested in the way accessibility enhances the work as a whole,’ says Lizzy Leggat, Access Manager at Graeae. ‘Given our focus on the aesthetics of access, we mostly use open captioning, particularly with our indoor, traditionally-staged performances.’
For Leggat and Graeae, the vital elements of captioning are placement, speed/timing, accuracy – and integration. But if a play is outside or in an experimental space, open captioning can become unwieldy and unpredictable. ‘For our most recent outdoor project we used Talking Birds’ app,’ Leggat continues. ‘For all productions, we consider how captioning and BSL will be best integrated into the performance right from the start.’
Graeae used creative open captioning in the style of newsprint for its 2010 production of Reasons to be Cheerful, similar to how Glasgow’s Birds of Paradise Theatre Company projected text messaging to augment the action (and comedy) in Wendy Hoose (2014). The artistic possibilities are endless.
Robert Softley-Gale, of Birds of Paradise, adds, ‘Creative open captions – when we get them right – allow a much wider audience to engage. By making captions part of the design of a show, we enhance the production for each audience member, demonstrating that access and high-quality theatre go hand-in-hand.’
Over in Cologne, Un-Label Performing Arts Company also integrates captions into their work, additionally offering local languages on tour. Creative Director, Lisette Reuter, explains that captions aren’t often offered in German theatre, and tend to be afterthoughts when they are. Reuter feels traditional open surtitles/captions are perceived as ‘alien elements’, and using them as an add-on neglects ‘emotional and artistic levels.’
‘We’re trying to take accessibility a step further with our international Creative Europe funded project, ImPArt,’ she continues. ‘Access becomes a source of inspiration and encourages experiments. For example, we’ll translate the artistic quality of a monologue by visualising its content through projected visual arts. We’re convinced art can be complex, subtle and accessible – without compromising on quality.’
Both open and closed captioning have come far, with exciting choices whatever the size and budget of a show. From immersing yourself in a large-scale production with smart glasses, to following on your phone as a play tours a riverside, to enjoying the creative projection of text and visuals onto stage sets – both technology and artistic imagination continue to inject theatre with transformative possibilities.
