Professor Fred Jiang & Team Work on Smart Headphone Prototype
EE Assistant Professor Fred Jiang and EE Professor Peter Kinget knew that they wanted to collaborate on a project together. They sat down and Jiang started explaining that he had recently purchased noise cancelling headphones and noticed that when walking around outside, he couldn’t hear anything around him, including approaching vehicles while crossing intersections. While that might be a nice feature on an airplane or in a noisy coffee shop, this feature wasn’t necessarily the safest for walking commuters. Cognizant of this, he began to notice that many pedestrians were listening to music or using their smartphones while walking, which is a major safety problem.
Kinget then explained that based on one of his personal hobbies, bike riding, he noticed that it was difficult for bikers to know when cars were approaching – even more so when the rider has headphones on; and that street noise is drowned out with these types of headphones which has likely caused many accidents over the years.
According to the National Highway Traffic Safety Administration (NHSTA), there were 6,283 pedestrians killed in traffic accidents last year – the most since 1990. And as technology advances, technology becomes more and more engrained in our everyday lives.
Jiang, whose background is in cyber physical systems and data analytics; and Kinget, whose background is focused on the design of analog and RF integrated circuits set out that day to develop headphones that could make people’s lives safer at reasonable cost. The research team includes Peter Kinget, Chair of Electrical Engineering; Shahriar Nirjon, a professor of computer science at the University of North Carolina (UNC) at Chapel Hill; and Joshua New, a psychology professor from Barnard College. Graduate students from both Columbia and UNC also work on the project.
“We are exploring a new area in developing an inexpensive and low-power technology that creates an audio-alert mechanism for pedestrians,” says Jiang. “Once this technology is developed, we can focus on other scenarios such as biker safety, construction worker safety, and railroad safety.”
The smart-headphone project was awarded a $1.2 million grant from the National Science Foundation in 2017, and the team has since published two conference papers as well as a journal paper in IEEE Internet of Things Journal on their research. They have also received several honors including a best demo award from an ACM conference and a best presentation award from an IEEE conference.
The research and development of the smart headphones involves embedding multiple miniature microphones in the headset as well as developing a low-power data pipeline to process sounds from oncoming vehicles into usable features that can be used to alert the pedestrian. It must also extract the correct cues that signal impending danger. The pipeline will contain an ultra-low power, custom-integrated circuit that directly converts analog audio signals into spatial features for helping with localizing the vehicles quickly and consuming very little power.
The researchers are also using a combination of signal processing and machine learning techniques to design the smart headset. Machine-learning models on the user’s smartphone will classify hundreds of acoustical cues from city streets and nearby vehicles and warn users when they are in danger. The mechanism will be designed so that people will recognize the alert and respond quickly. The team is now testing its design both in the lab and on the streets of New York – a city known for its congestion and its cacophony of sounds. New, the psychology professor from Barnard, says he will conduct perceptual and behavioral experiments with people to see how the alerts can be effectively provided to pedestrians who walk in cities wearing headphones.
The team’s aim is to develop a prototype of the smart headphone system at Columbia and then transfer the technology to a commercial company. “We hope that once refined,” he says, “the technology will be commercialized and mass produced in a way that will help cities reduce pedestrian fatalities.”
-------
In the Media:
Fast Company
Gizmodo
Science Daily
Good News Network
Traffic Technology Today
Hindustan Times
By Eliese Lissner
Posted:
Dec 19 2019