PopSignAI Preview 0.1.4

PopSignAI is currently in preview mode, offering users access to its first seven levels.

This innovative application addresses an important educational gap: the need to teach sign language to infants, regardless of their hearing status. Developed by Georgia Tech and the RIT/NTID Center on Access Technology, PopSign is a bubble shooter game designed to facilitate the learning of American Sign Language (ASL). The game aligns with the MacArthur-Bates Communicative Development Inventories, which represent foundational concepts for language acquisition in children. While primarily aimed at hearing parents of deaf infants, PopSign is beneficial for anyone interested in learning ASL.

PopSignAI enhances the original game by integrating sign language recognition technology. This advancement stems from the collection of over 220,000 examples of 250 ASL signs gathered by the Deaf Professional Arts Network from 47 native ASL signers. The recognition capability has been bolstered by Google's support during a Kaggle machine learning competition that concluded in May 2023. The game is now incorporating Google's Mediapipe hand tracking and sign language recognition models, developed using TensorFlow Lite.

It is noteworthy that 95% of deaf infants are born to hearing parents who may lack knowledge of sign language. These infants are at risk for Language Deprivation Syndrome (LDS) due to insufficient exposure to language during critical periods of development, which can adversely affect their relationships, education, and future employment opportunities.

In PopSignAI, players engage in typical "bubble shooter" mechanics, matching three or more bubbles of the same type to burst them. Unique to this version, the game leverages a smartphone’s selfie camera to recognize the player's signing of one of five designated signs while they aim at bubbles to shoot. Future updates will introduce additional levels as the recognition system evolves.

We invite individuals interested in contributing examples of their signing to support the game's development to contact us at [email protected].