Assistant Professor of Computer Science and Engineering Ramtin Zand has received a National Science Foundation (NSF) CAREER Award to study computing systems for real-time machine learning technologies. Zand’s five-year, nearly $600,000 project is expected to produce smaller, faster and smarter machine learning computing systems for use in real-world applications without reliance on large servers.
Zand will lead his team in his Intelligent Circuits, Architectures and Systems (ICAS) Lab to create a heterogeneous machine learning system that will enable the implementation of computer vision and large language models on energy and resource-constrained devices.
“Our project is focused on real-world machine learning applications that can be processed in tiny, battery-powered devices,” Zand says. “You can’t always rely on having an Internet connection, so they should be able to process quickly, in real time, without access to large data centers.”
Zand’s research combines neuromorphic and edge computing to simulate how the brain works while using tiny, energy-efficient devices. Neuromorphic computing uses specialized hardware and software algorithms to process data more efficiently, while edge computing refers to making more efficient machine learning models that can run on small devices.
Sign language translation is one of the targeted applications in the project. Zand’s team will use vision and language models to create sign language translation technology that facilitates seamless communication between sign language users and others. After recognizing hand gestures and facial expressions, the team will convert signs to spoken language. All of these components will be assigned as tasks to computing systems and architectures, creating a heterogenous system that uses a combination of solutions to build a useful computing system.
Once developed, the technology will be deployed on portable, wearable devices for on-demand use. The outcome of this work can empower those with hearing and speech impairments, fostering inclusive communication and workforce diversity.
“In the last two decades, nearly 70% of the assistant professors in the department have secured this award, marking a noteworthy accomplishment,” says Homayoun Valafar, chair of the Department of Computer Science and Engineering. “Dr. Zand’s project and long-term goals align perfectly with the department's mission, and his work is essential to the success of our neuromorphic computing, machine learning and edge computing efforts.”
Per NSF CAREER Award requirements, Zand’s project will integrate research, education and outreach by developing cross-disciplinary curricula and online educational resources. He is offering a graduate course on edge commuting that, previously offered in spring 2020 as a Special Topics course, will become an official part of the department’s curriculum in spring 2025. He is also collaborating with American Sign Language instructors in the Linguistics program to create data sets for vision and language models. Students will perform hand gestures and collect data resulting in an unbiased, diverse data set. In addition, the team is creating educational content for YouTube, including five-minute presentations on various aspects of the research.
"My students, Brendan Reidy, Mohammadreza Mohammadi, Blake Seekings, Peyton Chandarana and Mohammed Elbtity, played an important role in the success of this project and proposal. From brainstorming ideas to collecting preliminary results, and engaging in outreach activities, their contributions were invaluable,” Zand says.
According to Zand, a variety of machine learning systems in applications such as robotics, self-driving cars and assistive technologies require real-time computation and low energy use to ensure extended battery life. Beyond sign language translation, the methodology and framework developed in this project can pave the way for real-time technology in many other domains.
“I am very interested in building things that are scalable and have a large impact,” he says. “If we can have small gadgets that can be used for different applications, it can change people’s lives. This kind of research makes that possible because it specifically focuses on converting large-scale AI and machine learning systems to small scale systems that can be accessible to anyone.”