Robot generates images to prepare the soil for the future of agriculture

By creating open database of plant images, project supports more sustainable food production and helps Canada advance the field of digital agriculture

Researchers from the University of Winnipeg (UWinnipeg) and the University of Saskatchewan (USask) have taken on an ambitious challenge: build the ground for the next revolution in global farming and food production. With support from George Weston Limited and Mitacs, the team is filling a gap within the digital agriculture field by building a robotic system to create an open dataset of Canadian prairie crop plants and weeds.

“We want to enable the idea of creating automated vehicles that can distinguish plants to make decisions in the field like killing weeds and saving crops. But to do that, one first needs lots of labelled images of both crops and weeds,” explains Professor Christopher Bidinosti, of UWinnipeg’s Department of Physics.

After an unsuccessful search for such data, he and Associate Professor Christopher Henry, of UWinnipeg’s Department of Applied Computer Science, decided to embark on a complex journey to produce the information.

“We have identified the real crux of the problem,” Bidinosti says. “We took on what might not look like the most attractive job, but scientifically the most important one.”

Generating and labelling data in one go to produce more sustainable food

The first step of the project included building an articulated robot that could take thousands of pictures of plants every day in a growth chamber and feed a system that automatically labels the files. Since 2019, over one million images were generated, and another 5,000 to 10,000 are added to the system every weekday.

The construction and operation of the robot has been mostly conducted by Michael Beck, Mitacs intern and postdoctoral fellow at UWinnipeg. The other six interns are developing machine learning models, testing field data, and building data infrastructure. In addition to Bidinosti and Henry, they count on the academic support of USask’s Associate Professor Ian Stavness, of the Department of Computer Science.

Through Weston, the team also won a Seeding Food Innovation grant aimed at accelerating solutions to sustainable food challenges. That resource was added to the award from Mitacs and, more recently, to funding from Western Economic Diversification Canada, acknowledging the potential of the research.

“As Canada’s largest food company, we are acutely aware of the problems that the effects of climate change and an expanding global population will have on the world’s food supply, and want to play a small part to improve things,” says Tamara Rebanks, Vice-President of Community Investments at Weston.

“We hope that these technologies will allow farmers to produce safer and more sustainable food with fewer pesticides that will benefit all Canadian consumers. We see the potential for Canada to become a leader in data management in agriculture and hope others will build on this research.”

A huge team effort, the project also counts on collaborations with other academic institutions like the University of Manitoba and Red River College and industry partners such as Northstar Robotics, Sightline Innovations, Manitoba Pulse & Soybean Growers Association, and Canola Council of Canada.

From theoretical studies to impactful agriculture applications

For the supervisors, an exciting outcome of the project is that students from fields largely focused on academic research are gaining real-world experience through the several industry partnerships.

“All our students working on this will be in a wonderful position. Machine learning will certainly not go away, and they will be well trained for that,” says Bidinosti. Henry adds, “They can be pioneers in the industry once they leave this project.”

Beck, who studied mathematics at the University of Kaiserslautern, in Germany, and followed it up with a PhD in computer science, agrees with that perspective. After spending years dedicated to a theoretical and very focused area, he is now harnessing skills — like structuring, analyzing, and problem-solving — to do a work that touches on many applied fields in demand, including agriculture, robotics, machine vision, software development, and artificial intelligence.

“The last time I learned about so many different topics in parallel was likely in school. But besides acquiring new knowledge, I think this whole project just shows how valuable lifelong learning, transferring skills, and interdisciplinary work are in tackling today’s challenges,” he says.

As the work progresses, Beck, Bidinosti, Henry, and all the research team hope to give a long-lasting and impactful contribution to Canadian agriculture. The goal is to have all their assets open so other researchers and the industry can build on their work.

Part of the dataset was already shared in a research paper; now they are improving and escalating the work to release it in phases later. Other Canadian researchers and industry will have access first. Then, it will be shared globally.

The potential applications are limitless. “It used to be pure science fiction but it’s not anymore,” says Bidinosti. “I think of it as gardening but expanded to a giant Canadian crop farm that could have robotic gardeners taking care of each plant. That is still long away, but it won’t happen without this database that we’re providing.”

Mitacs’s programs receive funding from multiple partners across Canada. We thank the Government of Canada, the Government of Alberta, the Government of British Columbia, Research Manitoba, the Government of New Brunswick, the Government of Newfoundland and Labrador, the Government of Nova Scotia, the Government of Ontario, Innovation PEI, the Government of Quebec, Fonds de recherche du Québec – Nature et technologies, the Government of Saskatchewan, and the Government of Yukon for supporting us to foster innovation and economic growth throughout the country.

Do you have a business challenge that could benefit from a research solution? If so, contact Mitacs today to discuss partnership opportunities: BD@mitacs.ca

Photo featuring the research team (courtesy of the University of Winnipeg): MSc student Maryam Bafandkar, Christopher Henry, Ed Cloutis, Chris Bidinosti, MSc student Reid Lowden, PhD student Chen-Yi Liu, Jonathan Ziprick (Red River College), MSc student Pu Junyao, and postdoctoral fellow Michael Beck.

Mitacs Team

Mitacs’s website content is created by people throughout our organization, united in their passion for innovation and eager to share their perspectives with others in the innovation ecosystem. 

More posts by this author Contact Me