The empirical grounding of linguistics has benefited greatly from research on multimodal communication, from video recordings of face-to-face interactions to elaborately composed ancient texts. What such expressivity has in common is the coordination of language and other semiotic resources to achieve complex social and cultural goals. From this perspective, an instance of “language” might be the outcome of diverse modalities of expression in simultaneous use, possibly stemming from previously distinct language traditions. As the relationship between speech, signed language, writing, and gesture is central to this developing understanding of language, there is a pressing need for basic analysis, documentation, and typologically oriented research that is — in this way — more inclusive.
I target this research gap through my current work to decipher hand signs in the figural art of Ancient Maya texts. This unique script is expressed through inscribed hand forms in the otherwise iconographic descriptions of people, deities, and other beings. The iconographic imagery and their embedded hand signs are integrated with hieroglyphic writing in semiotically rich and artistically masterful ancient texts. The texts thereby involve two modalities of semantic expression: the hieroglyphic script and the hand-sign script.
My research on the ancient hand-sign script builds on prior work to document and describe the bimodal talk of Arapaho speakers (from the Wind River Reservation, Wyoming). What characterizes Arapaho bimodalism is the complex mixing of signed language and spoken language within acts of everyday conversation. As such, manual and vocal articulations of language are integrated into grammatically coherent expressions. Notably, each modality usually carries its own semantic and pragmatic weight, so that co-operating hand signs and spoken words are complementary, mutually elaborative, and without redundancy.