At this year’s Game Developers Conference, Insomniac’s Lindsay Thompson discussed the process of bringing American Sign Language into Marvel’s Spider-Man 2.
Marvel’s Spider-Man 2 was praised in many areas when it was released in late 2023. The game’s traversal mechanics, dual-hero structure, and cinematic storytelling techniques were among the more obvious features to get recognition. Another equally impressive accomplishment was the accurate, on-screen representation of American Sign Language (ASL).
The returning character of Hailey Cooper, portrayed by Deaf actor Natasha Ofili, played an important role in the game’s story. Not only is she a close friend of Miles Morales, but she is also playable in a memorable sequence that allows players with hearing to experience the world from her perspective.
Accuracy and representation
In an animation panel at this year’s Game Developer’s Conference, associate animation director Lindsay Thompson said that including an accurate representation of deaf characters and ASL in the game was an important goal.
“Some of our key pillars for doing ASL work was that we had ASL consultants and interpreters,” she says. “We had deaf actors and body doubles on the mo-cap stage, and a different and more extensive animation polish work pipeline,” she explains.
Insomniac partnered with The Hypernovas, a US-based organisation that specialises in working within the entertainment industry to advocate, educate, and consult on ASL.
“They were there for our questions regarding deaf culture,” says Thompson. “They also worked with us on stage to make sure performances were clear and then worked with us to go over every scene and shot that had ASL to make sure that our signs were correct and clear.”
For each hearing cast member, a deaf body double was employed and on stage at Insomniac’s mocap studio. Thompson explains that the shooting method used to capture the hearing cast’s spoken performances and the deaf body double’s signing often varied. Depending on the scene’s requirements, it was sometimes better for them to work side-by-side. At other times, the actor would perform face-to-face with their body double.
“Each deaf cast member also had their own interpreter so we could all communicate on set. We also had the Hypernovas on stage to help interpret the scripts with the cast and plan out the signing performance,” Thompson explains.
Capturing proficiency
Thompson says it was an intentional choice to depict characters with different levels of ASL experience. To support this approach, the main cast would occasionally sign themselves, resulting in a less polished but more genuine performance.
“Seeing as they were playing a more inexperienced signer, it was authentic for them not to be signing as fluently as a deaf person would,” she says. “We had to consider each character’s experience and knowledge of ASL. So, Genke, Miles’ best friend, is working very hard to learn. Miles is probably the most comfortable and experienced in speaking with Hailey. Rio, Miles’ mom, is clearly just beginning to learn as Miles has just started dating Hailey.”
Another example – and beware some mild spoilers – comes with the unexpected appearance of Albert Moon at the end of the game.
“When Albert Moon says, nice to meet you in ASL, we approached it as though he had literally watched a YouTube video before coming over and saying something to her,” Thompson says. “So, our interpreter was like, well, that maybe wasn’t quite it, but that looks like somebody who’s just trying to say a nice phrase to somebody.”
Improving fidelity
In addition to capturing nuances of performance and character, there were technical complexities and challenges to overcome. Thompson highlights the specially adapted equipment the performers use to improve finger detection.
“You may also notice that normally mocap gloves don’t have as many markers on them. It’s harder to capture intricate finger movement most of the time, and we often keyframe our fingers after the fact, but for our ASL scenes, we added extra markers to our signing actors so that we would start off with better, more accurate data,” Thompson says.
The final in-game results are a combination of the two performances, with the arm and hand movement layers of the signer layered onto the body and face of the main cast member.
“Once we shoot the data for our scenes, we then move on to layout. Our animators do everything. They do cameras, layout, polish, and keyframe work. We are a department full of directors,” Thompson explains.
“After laying it out and getting data combined, we finally worked with The Hypernovas to go over this performance and video footage. We could get clarifications on any parts that may be confusing. We would often go through it sign by sign, and there were many times our animators had to go into the performances and add or remove signs after the fact to make it clear.”
This process could include holding the signs for an extra frame or two to ensure readability or improving the fluidity of the movements. Despite the additional mocap markers, Thompson says that nearly every sign in the game, particularly when there is quick fingerspelling, was enhanced to some degree.
Thompson is particularly proud of Hailey’s playable mission. The puzzle-based sequence manipulates the game’s usual soundscape and uses emojis as signifiers to her thoughts.
“Our designers and writers put together a truly unique experience as a deaf character. From the audio design, to the emotes, to the text conversation, they found unique ways to interpret emotions in a story without a voice. And bonus points, there’s a cat on this mission, so it’s really awesome.”
