Bird researchers use virtual reality to bring fieldwork experience to classroom

UNIVERSITY PARK, Pa. — A flutter of blue and yellow darts through a field in late May. Trees, shrubs and summer flowers fill the landscape. A blue-winged warbler is just within reach, with one swift motion it can be gently grasped, banded and studied to understand the health and evolution of one of North America’s most colorful birds.  

A practice once reserved for scientists, this moment is now possible anywhere in the world thanks to a virtual reality experience developed by scientists at Penn State.    

The virtual reality (VR) program, named VRmivora, was developed under a five-year U.S. National Science Foundation (NSF) CAREER Award earned by David Toews, the Louis Martarano Career Development Professor of Biology in Penn State’s Eberly College of Science. Toews partnered with the Penn State Center for Immersive Experiences to conceptualize and advance VRmivora. 

The project was first started by an undergraduate in Toews’ lab, Lisa Wang, a Schreyer Scholar who said she wanted to improve science outreach and communication to students and the public. With a passion for birds and using art in education, she said she sought a way to help others understand and even experience the fieldwork typically done in ornithology.  

“With the work that we do, part of it is field work, part of it is in the lab and part of it is computation. Most of it is either not accessible to most people or, for the computational parts, kind of abstract,” Wang explained. 

She pitched the idea of a VR experience to Toews and created the original bird 3D model that was included in the NSF grant proposal and used to make the VR program, which is now available to download for free on SideQuest, a gaming and community VR platform.  

“It’s a great instructional tool for anyone high school age or older, as we work with a lot of natural field systems that aren't always readily accessible,” Toews said. 

The name VRmivora is a play on words from the genus of the blue-winged warbler, Vermivora cyanoptera, a colorful songbird with bluish wings and a yellow body. Toews’ research focuses on both golden-winged and blue-winged warblers and how they hybridize and cross genetically. Offspring from the crosses can have a combination of different colors and patterns, but what genes or diet factors affect this is a focus of their research. 

Toews and his team investigate this question by extracting DNA from warbler fecal samples to determine their diet. However, to collect these samples, they must have access to wild warblers. Unfortunately for student researchers, the migratory birds appear in Pennsylvania in the middle of May, which is when students are either taking finals or leaving campus, so they rarely get the opportunity to go into the field.  

“This is where VRmivora comes into the picture,” Toews said. “It doesn’t replace being able to go outside in the field and do this research, but it’s a close approximation. When I tell students about bird banding or using mist nets, they don’t have a visual to understand what I am talking about. It fills a gap for one part of the scientific process that isn’t always accessible for students.  

By rendering an outdoor environment that has trees, grass, woods and birds, the VR experience allows students to interact with the birds and the lab on a small scale, Toews explained.  

“You can make mistakes in the VR experience that you don’t want to make in real time,” Toews said. “You can also experience things that you probably wouldn’t be able to do in the field.  One of my favorite parts of the experience is when you can grab a bird and bring it up and you can see all of the different parts of the bird that you may not be able to in real life.”  

In the VR experience, students can approach warblers perched on low-level branches, listen to their song, and delicately hold them for closer observation and to take samples, comparing them to a virtual field guide. The simulation also includes a 360-degree video of Toews’ team performing the bird catching in real life, from setting up nets to measuring and banding a warbler. Then, students are taken to a virtual lab where they go through the steps to analyze the samples, like extracting and processing genetic material.  

The Center for Immersive Experiences developed VR models for the "pure” VRmivora warblers and their hybrids, with Bart Masters, Lead Programmer for XR, and Alex Fatemi, Lead 3D Modeler, working together on the project. Wang developed aninitial bird model, drawing on her 3D animation experience from high school and “a lot of trial and error,” Wang said.  

Toews includes the VR experience in his “BIOL445: Molecular Ecology” class, where students work through the simulation under Toews’ guidance, learning to study evolution through the lens of genetics. Toews said he also hopes to showcase VRmivora at DNA Day this coming spring, an event for high school students organized by the National Human Genome Research Institute to learn about DNA and genetics, contributing to science outreach.  

“I think for instructors, for my colleagues, seeing what is possible with VR will hopefully spark some creative juices for others in the field of biology and ecology and evolutionary biology,” Toews said. “This is one way to bring a bit of that experience to students. While it doesn't replace that opportunity, it supplements it for a larger group of students that may not have the ability or access to the birds that we are working with.” 

At Penn State, researchers are solving real problems that impact the health, safety and quality of life of people across the commonwealth, the nation and around the world.  

For decades, federal support for research has fueled innovation that makes our country safer, our industries more competitive and our economy stronger. Recent federal funding cuts threaten this progress.   

Learn more about the implications of federal funding cuts to our future at Research or Regress.