$12M grant to build animal-inspired sensing for autonomous drones and robots

UNIVERSITY PARK, Pa. — A research project, led by Penn State engineers to create a sort of digital brain inspired by the processes insects and small animals use to make smart decisions with limited information, has been awarded $12 million by the U.S. Army through the Army’s Research Laboratory.

Saptarshi Das, Ackley Professor of Engineering Science, professor of engineering science and mechanics and principal investigator on the project, explained how many powerful computing systems today require supercomputing clusters, data centers or a large amount of external power to function. Edge devices, or computer systems disconnected from a larger cloud service or power structure, can struggle to make smart, artificial intelligence (AI) informed decisions. If successful, the system could help AI-powered drones and ground robots make smarter decisions, particularly in remote or covert locations where information is limited, he said.

“Drones may be operating in a remote area where you don’t have connection, or you may not even want to communicate with a cloud in an effort to conceal your location,” said Das, who holds additional affiliations in electrical engineering and the Materials Research Institute. “Our proposed system seeks to grant edge devices the onboard power necessary to make smart decisions in complex environments, without requiring bulky components or a surplus of energy.”

The project proposes an intelligent sensing node that will eliminate the need to convert physical data collected by a drone’s onboard cameras and sensors into digital information.

“Saptarshi and his team are tackling one of the toughest challenges in technology today — making machines think smarter while using less power,” said Joshua Robinson, director of the Materials Research Institute at Penn State. “Their work will help keep our warfighters and our nation safer, and it’s exactly the kind of bold, high-impact research that will accelerate the technologies our national security depends on.”

To achieve this, the team plans to draw inspiration from unlikely sources: the biological systems that compose our bodies, as well as the tiny brains of animals like locusts and owls.

“These animals’ brains are all examples of edge devices, to some extent,” said Das. “Although their brains are very tiny, they can accurately process audio and visual cues because of how effectively they can filter out irrelevant stimulus from important information.”

The node will be equipped with sensors capable of processing the broadband electromagnetic waves emitted by electronic devices in a surrounding environment. According to Wooram Lee, associate professor of electrical engineering and co-principal investigator, these signals will be processed through a specialized structure similar to a cochlea, a system inside the human ear that collects vibrations and translates them into sensory inputs the body can interpret as sound.

“Instead of directly processing a broadband electromagnetic signal in the digital domain, which can be power-hungry and bandwidth-limited, we are taking inspiration from the architecture of the cochlea to pre-process the signal in the analog domain,” Lee said. “This approach can substantially improve both processing bandwidth and energy efficiency.”

According to Lee, the main power draw in drone computing systems comes from converting analog information, collected using optics and sensors, into digital data. Analog computing uses physical phenomena like electricity to perform the math problems powering computers instead of binary ones and zeros. Powering drones with analog computing would eliminate this energy conversion and tremendously cut power needs. The team plans to achieve this by using two-dimensional materials — namely graphene, a honeycomb-shaped layer of carbon only one-atom-thick — and traditional silicon semiconductors to transfer electricity into an array of memristors, components that can intake an electrical current and amplify the output.

In the team’s proposed design, analog currents will move through a circuit built to imitate how neurons interconnect and transmit electrical signals across the brain. Currents will be fed into graphene-based field-effect transistors — tiny components that can control the flow of electricity through a system. This process mimics specific neurons found in the brain known as coincidence detectors, which only trigger an output upon detecting two or more inputs simultaneously or in quick succession. This allows for the brain to filter out unimportant stimulus, and according to Ram Narayanan, distinguished professor of electrical engineering and co-principal investigator, applying it in their intelligent sensing node will allow drones to use less energy and make smarter decisions.

Narayanan, who holds additional affiliations in engineering science and mechanics and the Larson Transportation Institute, explained that another problem facing interconnected swarms of drones are that the radio waves they use to communicate can be easily intercepted or jammed. To help address this, the node will incorporate a noise radar–based system that allows for members of the same drone swarm to embed messages into a larger, encoded frequency that will appear as just signal noise to listeners. Information in the neuromorphic computing system will be represented by stochastic, or unpredictable, noise-like spike trains — a technique that exploits the correlation-based processing drones use to extract specific signals from a larger transmission.

Narayanan said this approach will reject signal interference, clutter and intentional deception, allowing drone systems to remain hidden even when communicating in high radio-traffic environments.

“By embedding messages within noise-like waveforms indistinguishable from the radio frequencies present in the ambient environment, we can counter our adversaries' use of decoys, deceptive frequency emissions and sensor spoofing,” Narayanan said. “This approach can improve the robustness, energy efficiency and stealthiness of autonomous systems operating in contested electromagnetic environments.”

According to Das, this framework will have incredibly broad applications beyond both drones and ground robotics, potentially allowing larger swarms of autonomous agents to covertly communicate and operate in tandem using relatively low power. The team plans to conduct yearly demonstrations of the system throughout development, transitioning between each different institution leading research on the project annually. Over time, the scale of each test will increase from testing a singular prototype to larger-scale tests of drone swarms powered by the node.

“A system like this is the holy grail of advanced sensing,” Das said. “We’ve long had the capabilities to achieve all of these behaviors separately, but putting them together in a single system that adheres to very tight size, weight and power constraints is a critical need and a new frontier.”

Additional researchers on the project include Joshual Yang, co-principal investigator and Arthur B. Freeman Chair Professor of Electrical and Computer Engineering at the University of Southern California; and Qingfei Xia, Dev and Linda Gupta Professor of Electrical and Computer Engineering at the University of Massachusetts Amherst.

This research is supported by the Department of Defenses’ U.S. Army Combat Capabilities Development Command Army Research Laboratory under award number W911NF-26-2-A161. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funders.

At Penn State, researchers are solving real problems that impact the health, safety and quality of life of people across the commonwealth, the nation and around the world.

For decades, federal support for research has fueled innovation that makes our country safer, our industries more competitive and our economy stronger. Recent federal funding cuts threaten this progress.

Learn more about the implications of federal funding cuts to our future at Research or Regress.