Kristen Secora, Karen Emmorey, Visual-Spatial Perspective-Taking in Spatial Scenes and in American Sign Language, The Journal of Deaf Studies and Deaf Education, Volume 25, Issue 4, October 2020, Pages 447–456, https://doi.org/10.1093/deafed/enaa006
Navbar Search Filter Mobile Enter search term Search Navbar Search Filter Enter search term SearchAs spatial languages, sign languages rely on spatial cognitive processes that are not involved for spoken languages. Interlocutors have different visual perspectives of the signer’s hands requiring a mental transformation for successful communication about spatial scenes. It is unknown whether visual-spatial perspective-taking (VSPT) or mental rotation (MR) abilities support signers’ comprehension of perspective-dependent American Sign Language (ASL) structures. A total of 33 deaf ASL adult signers completed tasks examining nonlinguistic VSPT ability, MR ability, general ASL proficiency (ASL-Sentence Reproduction Task [ASL-SRT]), and an ASL comprehension test involving perspective-dependent classifier constructions (the ASL Spatial Perspective Comprehension Test [ASPCT] test). Scores on the linguistic (ASPCT) and VSPT tasks positively correlated with each other and both correlated with MR ability; however, VSPT abilities predicted linguistic perspective-taking better than did MR ability. ASL-SRT scores correlated with ASPCT accuracy (as both require ASL proficiency) but not with VSPT scores. Therefore, the ability to comprehend perspective-dependent ASL classifier constructions relates to ASL proficiency and to nonlinguistic VSPT and MR abilities.
In the effort to understand what cognitive abilities support typical and atypical language comprehension and production, sign language research can make important contributions to the discussion. Researching language in a modality other than speech helps researchers, clinicians, and teachers to understand the cognitive abilities that support general language comprehension and production (i.e., abilities required for language regardless of modality) and those that are specifically tied to either the spoken or signed modality. Sign languages (such as American Sign Language, ASL) are produced by the hands, face, and body, and utilize the space in front of the body, called signing space. Sign languages can use space topographically to schematically map the spatial arrangements of signs or spatial classifier constructions 1 within signing space onto real or imagined spaces and the objects in them. For example, a signer can produce classifier handshapes that refer to a person and a bicycle and place them next to each other in signing space to indicate a person standing next to a bicycle. The classifiers’ spatial arrangement maps directly onto the spatial relationship of those two objects in real or imagined space. This iconic use of space stands in contrast to the way spoken languages express spatial relations, namely via specific lexical items (e.g., prepositions) or spatial morphemes (e.g., locative affixes).
While knowing a sign language does not seem to result in a generalized advantage in spatial cognition (e.g., Marschark et al., 2015), there is evidence that sign experience can result in specific enhancements in spatial cognition. When adult deaf signers and hearing nonsigners were asked to generate the mental images of block letters, the deaf signers showed significantly faster reaction times than the hearing nonsigners when forming the mental images of complex letters ( Emmorey & Kosslyn, 1996; Emmorey, Kosslyn, & Bellugi, 1993). Similarly, during a task requiring a decision about whether two rotated block figures were mirror images of one another, deaf signing adults showed faster reaction times compared with hearing nonsigners but both groups were equally accurate ( Emmorey et al., 1993). Furthermore, Emmorey, Klima, and Hickok (1998) showed that adult deaf signers were better than nonsigners at remembering objects’ orientations within a scene and recreating a visual scene (e.g., a room with furniture) but only when a 180° mental transformation of the scene was required. The groups performed similarly when no transformation was required. Emmorey et al. (1998) suggest, as with others who report advantages in spatial cognition for deaf signers (e.g., Bavelier et al., 2001; Talbot & Haude, 1993), that the specific spatial processes involved in sign language processing can provide measurable advantages for similar nonlinguistic tasks. The enhanced performance seen in adult deaf signers for these spatial cognitive tasks suggests that nonlinguistic visual-spatial skills may support performance in spatial linguistic tasks.
Sign researchers have long recognized that the spatial nature of signed languages creates conflicting visual perspectives between the signer and the addressee when conversing in the canonical face-to-face arrangement: what is produced on the signer’s right is viewed on the addressee’s left (e.g., Emmorey et al., 1998; Emmorey & Tversky, 2002; Pyers, Perniss, & Emmorey, 2008; Pyers, Perniss, & Emmorey, 2015). This perspective conflict only arises when signers describe nonpresent scenes. When describing objects that are visible in their environment, the location of the objects and of the signer’s hands are both visible, and such spatial descriptions are produced in what has been termed Shared Space ( Emmorey & Tversky, 2002). For Shared Space, there is no need for perspective-taking because each interlocutor can maintain his or her perspective of the jointly viewed scene, and this perspective does not conflict with the view of the signer’s hands. When describing nonpresent scenes, the signer and addressee view the signer’s hands with differing perspectives (e.g., the sign produced on the signer’s left is viewed by the addressee to be on the right side). Therefore, some sort of mental process is required in order for both interlocutors to arrive at the same understanding of what is being signed. The convention in ASL (and many other unrelated sign languages) is for the signer to produce egocentric descriptions for nonjointly viewed scenes using what we will term “Signer Space” (i.e., from the signer’s perspective), which contrasts with Shared Space ( Brozdowski, Secora, & Emmorey, 2019; Pyers et al., 2008; Pyers et al., 2015). For descriptions in Signer Space, the addressee(s) must perform a mental transformation in order to understand spatial descriptions from the signer’s perspective. Although this perspective conflict has been discussed previously, little work has been done to examine the cognitive mechanisms that underlie the resolution of this conflict.
Emmorey et al. (1998) discussed the required transformation in terms of mental rotation (MR) of a scene (e.g., of the description of a room). While MR and perspective-taking abilities are highly related, they are dissociable processes ( Hegarty & Waller, 2004; Zacks, Mires, Tversky, & Hazeltine, 2000). Visual-spatial perspective-taking (VSPT) seems to involve an imagined self-projection through space to align with another’s physical location and their visual-spatial perspective of a target object or scene ( Kessler & Rutherford, 2010; Kessler & Thomson, 2010), which is in contrast to the process of mentally rotating a two- or three-dimensional object or array (e.g., Michelon & Zacks, 2006; Zacks et al., 2000). While gender effects on object-based MR ability have been well documented (e.g., Peters, 2005; Voyer, Voyer, & Bryden, 1995), reports are mixed as to the role of gender in VSPT tasks (e.g., Kessler & Wang, 2012; Mohr, Rowe, & Blanke, 2010; Zacks et al., 2000). Although gender is often an important factor in spatial tasks, particularly for MR type tasks, it may play less of a role during the comprehension of spatial language in ASL. Emmorey et al. (1998) found that males outperformed females on an object-based spatial task, but male and female signers performed similarly when the task involved comprehending spatial ASL descriptions.
To date, the nature of the spatial transformation necessary for the addressee to adopt the signer’s perspective remains unclear. Pyers et al. (2015) suggested that embodied or cognitive control mechanisms might underlie the ability for an addressee to ignore his or her own visual perspective of a signed (or gestured) production in order to adopt another’s visual perspective. One way to examine what processes contribute to this perspective transformation is to compare an individual’s relative abilities in MR and nonlinguistic visual-spatial perspective-taking tasks with their ability to perform the necessary perspective transformation within sign comprehension.
Evidence from atypical sign language development during childhood has provided some clues about whether nonlinguistic VSPT ability may be related to the spatial transformations seen in ASL. Quinto-Pozos et al. (2013) presented a case study of a native signing deaf teenager, pseudonym Alice, who achieved high levels of ASL proficiency, but selectively struggled with the aspects of ASL that depended on understanding perspectives (e.g., role/referential shift, classifier constructions). She also struggled with nonlinguistic visuospatial tasks (e.g., assembling puzzles) and with nonlinguistic perspective-taking tasks (e.g., imagining how an array would look from another perspective). Quinto-Pozos et al. hypothesized that Alice had a developmental perspective-taking visuospatial deficit which selectively impaired her ability to master the spatial aspects of ASL that depend on those nonlinguistic abilities.
Similarly, Shield and Meier (2012) reported data from native signing children who have been diagnosed with autism spectrum disorder (ASD). These children produced palm reversal errors, a pattern of errors not seen in typically developing native signing children. The children with ASD produced manual fingerspelled 2 letters with the palm facing inward, toward their own body, rather than with a correct outward-facing palm. Typically developing children learning ASL do not produce this kind of inward facing error. Such errors seem to reflect a lack of perspective shifting because the children with ASD copy what they visually perceive. For instance, if the child observes a parent fingerspelling with an outward-facing palm orientation (normal fingerspelling) and mimics the observed handshape and orientation, such a production would result in an incorrect inward-facing palm orientation. Shield and Meier suggest that this kind of error may arise from a difficulty with self-other mapping. One part of successful mapping between one’s own body and another’s body may depend on a VSPT process which seems to be impaired in individuals with ASD (e.g., Hamilton, Brindley, & Frith, 2009; Shield, Pyers, Martin, & Tager-Flusberg, 2016).
Thus, evidence from atypical sign language acquisition suggests that nonlinguistic perspective-taking abilities may underlie linguistic perspective-taking abilities in ASL, particularly for perspective-dependent structures in ASL. Furthermore, as described above, behavioral studies with adults suggest that extensive practice with a signed language can result in enhancements in nonlinguistic visuospatial abilities. However, whether nonlinguistic perspective-taking abilities relate to linguistic perspective-taking abilities in signing adults remain to be empirically tested. Therefore, the research question for the present study is: what cognitive abilities are related to the ability of adult signers to comprehend perspective-dependent structures in ASL?
MR processes may be involved in reconciling the spatial conflict between the signer’s and addressee’s perspective (as suggested by Emmorey et al., 1998). If so, we would expect to see a significant correlation between an individual’s performance on an MR task and performance on an ASL task involving spatial perspective transformation. On the other hand, nonlinguistic perspective-taking abilities may relate to an addressee’s ability to adopt the signer’s perspective. In this case, we would expect to see a correlation between performance on a nonlinguistic VSPT task and an ASL task requiring perspective-taking. Finally, both MR and nonlinguistic perspective-taking abilities may relate to a signer’s ability to reconcile the perspective conflict within ASL due to the highly related nature of MR and embodied perspective-taking (e.g., Hegarty & Waller, 2004). In that case, multiple regressions would be able to determine whether one of the two cognitive abilities relates more strongly with ASL perspective-taking performance. Since perspective-taking within ASL also involves the comprehension of linguistic structure (e.g., phonological, lexical, and syntactic processing), we expect general ASL proficiency to relate to the comprehension of perspective-dependent ASL structures. However, we would not expect overall ASL proficiency to relate to nonlinguistic VSPT abilities since only the ability to comprehend and produce specific ASL structures (e.g., classifier constructions) is hypothesized to relate to VSPT abilities. As Quinto-Pozos et al. (2013) showed that nonlinguistic VSPT deficits can disrupt the use of perspective-dependent ASL structures in spite of achieving a high proficiency in overall ASL ability.
Therefore, the present study examined the relationship between deaf signers’ performance on a nonlinguistic VSPT task, an MR task, and a task that taps proficiency in comprehending specific ASL structures hypothesized to rely on a visual-spatial transformation. Secora and Emmorey (2019) previously reported the VSPT and MR performance of this group of deaf signers in comparison to a group of hearing nonsigners—both groups performed similarly on the two tasks. The aim of the present study was to clarify what cognitive abilities play a role in reconciling the visual-perspective conflict that can arise as a result of the topographic use of space in ASL spatial descriptions.
A total of 33 deaf signers (16 F; Mage = 30.4 years, SD = 8.1; range: 20.4–48.5 years) were recruited from the community in San Diego, CA. Demographic information is presented in Table 1. Of the 33 total signers, 20 participants were native signers (learned ASL from birth from deaf parent/s; 13F) and 13 were early signers (learned ASL prior to age 6; 3F). Additionally, participants reported no history of ASD. ASD screening information was unavailable from one male participant. Participants also reported no history of significant neurological problems, such as head injuries, epilepsy, coma, etc. There were no significant differences in age between males and females or between native and early signers (all ps > .20).