DALLAS, Pa. — Nathan “Nate” Crotts, who earned a bachelor’s of science in surveying engineering at Penn State Wilkes-Barre in May, and Salvatore Marsico, associate professor of engineering at the campus, visited the University of Campinas in Brazil to further their research on using artificial intelligence (AI) and large language models (LLMs) to map roadway assets at the end of the spring semester.
They partnered with Henrique Oliviera, a former surveying engineering faculty member at Penn State Wilkes-Barre who returned to his native Brazil and has continued to work on projects with prior colleagues and his current employer, the University of Campinas. The research trip was funded by the Office of Global Programs, Penn State Wilkes-Barre and University College.
The research project began in 2022 as a collaboration between Marsico and Oliveira, to examine electrical distribution lines and roadway networks. It expanded to include investigating how AI might be able to assist in mapping roadway assets, such as guiderails, roadway distresses like potholes and cracks, protective barriers, telecommunications and electrical poles and equipment.
The goal is to train an AI model to recognize specific assets and hindrances, including potholes, cracks that need to be repaired, and power lines or vegetation that need maintenance, from video recorded from a moving car.
“As part of our visit, we went out on local roads and filmed our drive, looking for potholes, road signs, communication lines and things of that nature,” Marsico explained. “Nate will then take the video and input the information into our software, which helps him identify and categorize any potential obstructions.”
At the University of Campinas, Crotts and Marsico met with master’s and doctoral students, who collected aerial data to map delivery systems, and shared their own ground measurement work. The idea is that utility companies or road managers could quickly identify and assess potential problems from the ground or air.
“We’re figuring out the most efficient and effective way to make a futuristic city better able to optimize certain ways using advancements in technology,” Marsico said.
Crotts said his dataset is on an open-source platform (Roboflow.com) that anyone can access for their own needs.
“For the utilities that own these assets, for example, we are able to provide them with a location or a GIS [geographic information system] map to see everywhere that is encumbered by vegetation or needs attention. They can then optimize the way they tackle those issues,” he said.
Two other Penn State Wilkes-Barre students, Mrigakshi Verma and JunJie Cao, also contributed to the project. Prior to the trip, Crotts, Verma and Cao presented the group’s research at the campus’ Celebration of Scholarship in April. Their poster was one selected to represent Penn State Wilkes-Barre during the Eastern Regional Undergraduate Research Symposium at Penn State Brandywine, where it placed third.
Crotts has conducted research since his first year at Penn State Wilkes-Barre.
“I wanted to take as many opportunities as I could beyond the classroom,” he said. “I was able to use a variety of new instruments and work on larger projects with Dr. Bolkas, including the levee at the Francis E. Walter Dam. It has been great to implement what I was learning in the classroom on real-world projects. The research experiences have provided me with a good way to learn and understand the process.”
While he had traveled internationally before, this was the first University-related international trip for Crotts, who said he felt “very fortunate and happy to have the opportunity.”
Crotts is continuing in his role with Crews Surveying, a job he started as a student, and working toward becoming a surveyor in training, the first step in becoming a licensed professional surveyor.
“I appreciate that I was able to do undergraduate research that is directly related to my future career,” Crotts said. “I’ve come out of college with many different experiences. As automation continues to develop, I hope to get involved in working with computer science majors and then work with software development and automation in surveying.”
Following the research portion of their trip, Crotts and Marsico spent time exploring Brazil, including a visit to Rio de Janeiro. Crotts said that in addition to being at the University of Campinas, the visit to Rio was a highlight of the trip for him.
“Seeing Rio was absolutely phenomenal. It’s hard to describe how beautiful it is. We were able to see the Christ the Redeemer statue and Sugar Mountain, go on a cable car and more,” he said. “One of my favorite parts came toward the end of our trip, when we were taken in by two families for a true Brazilian experience. Not only did we get to go to the beach with them, but we became more immersed in their culture and their society. We saw how they go about everyday life, right down to the way they cook and do meals. That was a great experience.”