A new Intel program will harness AI to help staff ‘walk through’ unfamiliar areas before they arrive

Written by Brandi Vincent

The Intelligence Community’s main research branch has launched a new program to develop algorithm-based software systems that will merge images captured from different altitudes and angles – including from traffic cameras, drones, satellites and other platforms – to create immersive and photorealistic virtual environments of unknown places. worldwide.

Thanks to its new Walk-Through Rendering Images of Varying Altitudes (WRIVA) programthe Intelligence Advanced Research Projects Activity (IARPA) will promote technology to notify government officials of potentially dangerous locations before they deploy there.

“Imagine if law enforcement, the military, or aid workers could virtually drop into a location and look around and familiarize themselves with it before they arrived. These groups often need to provide quick support and life-saving assistance in unfamiliar or dynamic areas. Allowing them to prepare ahead of time protects them from harm when they have to conduct these activities,” IARPA program manager Ashwini Deshpande said Friday during a project preview and coaching. General Agency Announcement (BAA) for federal funding, ahead of its release on Tuesday.

Inspired by a teenage girl

WRIVA marks the first research project Deshpande is leading for the agency. She was inspired to present it after an experience that is likely familiar to many parents.

“One day I was trying to explain to my pre-teen where I was going to pick her up when she was hanging out with her friends, but we ran into a problem because I couldn’t describe where I was going to be in a way that she could understand. And when we looked for more information, we discovered that part of the problem might be that the area had recently gone through a lot of changes and that we were working with outdated information,” she recalls.

The solution she and her daughter landed on — picking another place to meet — seemed easy. But the whole scenario has Deshpande thinking about similar challenges for government officials.

“For example, what if law enforcement needs to know where they might be vulnerable when responding to a threat? Or what if we need to meet needs in an area where the landscape has recently changed due to bombing or natural disaster? ” she says. “Finding a solution is really essential.”

With years of experience as chief scientist and technical adviser, Deshpande brought the problem to IARPA — a federal research center that works on projects for the Intelligence Community (IC) and other agencies — and launched the project successfully.

“WRIVA aims to help users visually see or plan a mission activity. And it will be a game-changer for the IC, the military, law enforcement, as well as humanitarian aid and disaster relief,” she said.

Piles of images and data from a number of sources on a specific location can be applied to create site models, which can then allow personnel to repeat missions with a better knowledge of where they are. are going to operate. But these tools usually require huge amounts of this kind of information to work properly. Using WRIVA, Deshpande and his team want to produce software systems to perform site modeling in scenarios where huge volumes of ground-level imagery with reliable metadata are not readily accessible or available.

“Our goal with WRIVA is to move beyond some recent advances in machine learning and computer vision and to advance the technology in the areas of reducing the number of viewpoints needed to create site models,” said she noted.

Those involved in the project also intend to accelerate the rate at which site templates are completed and approved for use within the federal government. Essentially, WRIVA’s envisioned results will be algorithms and methodologies that allow managers to quickly create site models without full 360 degree coverage of a location – and methodologies to repair corrupted images.

“If you think of the story of us wanting to learn about an area that we can’t access, it’s been around for so long. I mean, it goes back to [Greek mythology with] Icarus, and back in the days of World War I where we were fixing cameras on pigeons,” Deshpande said. “But I think what’s going to be really different here is the level of immersion these models will experience.”

Other site modeling capabilities exist within the IC and Defense department. However, the processes of creating such tools are often incredibly slow, tedious and time-consuming, she noted.

“I think the real advances in technology will be the speed of the model creation to allow us to react very quickly, as well as the fact that you won’t have to spend hours and need a ton of expertise to collect the base images to create these site templates,” Deshpande said. “It will really allow people to practice almost like they’re in a video game, you know, to practice their scenarios before they have to go and carry out their mission in person.”

A “transient” ability

The WRIVA program is expected to be a 42-month research effort with hands-on work beginning in fiscal year 2023.

The locations for which it will actually be applied will depend on the specific needs of the country in three and a half years, once the envisaged technology is fully developed.

“But I think we plan to run experiments that represent a variety of different scenarios,” Deshpande explained. “One of the areas of concern is improvised housing like refugee camps – we plan to do data collections that mimic scenarios like that. We also plan to do data collections that apply to environments We have a wide range of field experience and challenges that proposers look forward to.”

So far, IARPA has named MITER Corp., Johns Hopkins University’s Applied Physics Laboratory, and Massachusetts Institute of Technology’s (MIT) Lincoln Laboratory as test and evaluation partners for this effort.

Building on their extensive activities to create and refine transformative technologies for disaster relief and humanitarian assistance, MIT officials will work to generate applicable high-fidelity datasets and build an infrastructure to assess the performance of capabilities produced by IARPA partners.

“There’s an almost immediate application of technology like this in damage quantification – so you can think right after a disaster when things are damaged, and people are trying to figure out what’s broken and what’s needs to be fixed, a tool like this could actually help to not only relay that information, but also especially here in the United States, can expand the available manpower that could come in to respond,” said MIT Lincoln Laboratory group leader Adam Norige during Friday’s briefing.

Research agencies like IARPA often conduct exploratory projects and don’t play a big role in fully operationalizing the technology they create – but, given the applications that already exist, the people involved in WRIVA are ” quite confident” that it will eventually be put into service.

“Our goal is to create transitional capacity. We have several partners within DOD and IC who are interested and highly engaged as stakeholders, as members of our government advisory group and as potential transition partners, and we are working closely working with them now to ensure that their needs are addressed and that they also put in place the necessary infrastructure to be able to take the WRIVA program and implement it according to their own operational needs,” said Deshpande.

“We have frankly received a lot of interest from several organizations within the military, but we also work closely with elements of the IC who more directly support the combatant,” she noted. .

-In this story-

Adam Norige, artificial intelligence (AI), Ashwini Deshpande, data, IARPA, Johns Hopkins Applied Physics Lab, machine learning, MIT Lincoln Lab, MITER, modeling, research and development, WRIVA

Ryan H. Bowman