Mesgaran creates new tool: Google Weed View

AI enables low-cost tracking of invasive johnsongrass

Johnsongrass, far left, captured near a construction site using Google Street View. (Mohsen Mesgaran/UC Davis)

To manage johnsongrass, a noxious weed that crowds out cotton and sickens horses, farmers have tried herbicides, burning and hand-pulling. Now, researchers in the UC Davis Department of Plant Sciences have developed a more high-tech weapon against the invasive weed: artificial intelligence and machine learning.

Using photos from Google's Street View database, the researchers have tracked down more than 2,000 cases of johnsongrass in the Western United States for a fraction of the cost and time that it would take for drive-by or other in-person surveys. They call their tool Google Weed View.

Johnsongrass identified growing near agricultural land using Google Street View. The yellow boxes were designated by artificial intelligence; the red boxes were drawn by human hand. (Mohsen Mesgaran/UC Davis)

The advancement could help land managers easily and quickly survey for other problem plants.

“Once the model is trained, you can just go and run it on millions of images from Google Street View,” said Mohsen Mesgaran, an assistant professor in the department. “We have huge flexibility, and its capability can be scaled up very quickly.”

The technique can easily be extended to other plant species. All that is needed is to label the new item in Street View photos and train the algorithm to identify that object in the images.

By providing location information, Google Weed View also offers an opportunity to examine how climate affects the growth and spread of weeds and invasive plants at very large scales.

“I think it can be both useful for management and for people with interests in more basic questions in ecology,” Mesgaran said.

A colleague's query

Mesgaran began looking at Google's photo database of roadways, streets and highways after Kassim Al-Khatib, a professor of Cooperative Extension in the same department, asked if he could survey Western states for johnsongrass.

Al-Khatib studies where johnsongrass grows, ways to manage it and how this perennial has evolved to be so prevalent and resilient. He's also working with scientists at the University of Georgia to decode the genome of johnsongrass, which is one of the top 10 most invasive weeds worldwide.

Johnsongrass can crowd out native plants, harbor pathogens and affect agriculture. It grows up to 7 feet tall with flowers that are green, violet, dark red or purplish brown depending on maturity, according to a UC Statewide Integrated Pest Management Program briefing page.

“Johnsongrass is a major weed not just in California but worldwide,” Al-Khatib said. “It's very difficult to control. It's a problem on vineyards. It's a problem for cultivated crops. It's a problem on orchards.”

Google Weed View allows for rapid, convenient scanning. It is continuously updated via everyday users with compatible cameras and images collected by Google. “Instead of a day of in-person driving, we can use AI to determine if johnsongrass is in a county or not,” Al-Khatib said. 

Setting the parameters

Johnsongrass patches identified using Google Street View. The yellow boxes were designated by artificial intelligence; the red boxes were drawn by human hand. (Mohsen Mesgaran/UC Davis)

To find the weeds, Mesgaran went to Google Street View, which hosts billions of panoramic photos. It didn't take long to find johnsongrass.

“The pictures are really good quality,” he said. “You can see plants and flowers.”

Street View's photos offer a 360-degree view, so in his request, Mesgaran set parameters, based on street direction (bearing), to only see the side view. He also specified latitude, longitude and other factors. To train the deep, or machine learning, model, he chose Texas, where johnsongrass is prevalent.

A student sorted through more than 20,000 images from that request to find pictures with johnsongrass, then drew rectangular shapes around the weeds. They located 1,000 images.

The labeled photos were fed into a computer to train a deep learning algorithm capable of identifying johnsongrass in Google's images. The model was run again to capture potentially more images containing johnsongrass. These additional images were then labeled and used to further refine the model. With each iteration, the algorithm learned and became more accurate.

“This deep learning model was trained by these images,” Mesgaran said. “Once we had a semi-working model, we ran it against about 300,000 images.”

For Al-Khatib's request, researchers focused on 84,000 miles of main roads in California, Nevada, Oregon and Washington states. The team discovered 2,000 locations with johnsongrass.

Google Weed View cost less than $2,000 to purchase the images and teach the model. A traditional car survey to cover the same area would cost an estimated $40,000 in gas, hotel, food and other costs.

“In a matter of months, we came up with 2,000 records, and I can do it for the whole U.S.,” Mesgaran said.

Next up? The whole United States.