FruitPunch AI for Food: Developing high-precision agricultural robots that will increase land yield while reducing resources, with use of digital twins and reinforcement learning

When investigating existing software solutions for training my robot with the help of a digital twin, I noticed that these existing solutions had a very high entry level in terms of knowledge and skill. We though we could do better!

Vincent Fokker – Project lead Agrosim Tweet

The challenge

Artificial Intelligence (AI) and robotics can help farmers to produce higher-quality food and have less impact on the environment. However, training and testing of agriculture robots is time consuming and expensive, because this usually happens in a physical environment and often involves the robot breaking down. In our project, we create a digital version of the robot that is tested and trained in a simulated environment. This way, training and testing of robots becomes faster and cheaper.

Focus points of the project

  1. Developing a framework for training and testing of autonomous robots, focussed specifically on the agricultural domain.
  2. Building a demo setup for valdidating the applicability of the framework to a real-world setting.

We will do so by building a demo setup with a robot in an environment that will mimick a real-world secenario. At the same time we will train a digital twin of this robot in a simulated environment. Steps to take are:

  1. Building the environment based on sensor information of the robot.
  2. Doing object detection of fruits and mapping of surroundings for navigation with the help of minimal sensory information.
  3. Integrating the ability to use own, pre-trained object detection algorithms.
  4. Setting-up the training framework for training the robot in the siumulated environment in order to learn the robot proper behavior.

Succes indicators for the demo are when the robot is able to ..

🍒   proper navigation within the physical environement, without damaging itself.
🥝   detect fruits and classify them correctly.
🍍   grasp the fruit based on the sensory information it has.
🍎   autonomously retrieve the fruits, and is able to collect them, based on easy command.


The testing setup of Agrosim with a custom made physical and digital training environment for the turtlebot with gripper, Intel rear sense camera and lidar.

Quick facts: the problem

🥥   By 2050 there are expected to be 9 billion people on Earth. The planet must produce more food in the next four decades than all farmers in history have harvested over the past 8,000 years due to population growth. (Dutch Review)

🍉   Dutch agriculture exports rose 8% to the equivalent of €9.9 billion in 2019. Making The Netherlands number two in the world when it comes to agricultural export. (

🍊   Agricultural machines are becoming bigger and bigger, and with that, the burden on the soil by the weight of these machines increases. (

🍉   Existing soil cultivating solutions are focused on a Monoculture for efficiency of the work. Monoculture exhausts soil fertility, requiring costly applications of chemical fertilisers. (

🥝   Lost biodiversity. Industrial farms don’t support the rich range of life that more diverse farms do. As a result, the land suffers from a shortage of the ecosystem services, such as pollination, that a more diverse landscape offers. (

the solution

🥥   Robotic solutions such as Pixelfarming can increase the biodiversity and proper usage of the soil, decreasing the weigth of the machinery used for managing and cultivating this soil. (Pixel farming robotics)

🍉   Farmers’ interest in the latest technologies has turned agricultural robots and drones into a market that’s set to reach $23.06 billion by 2028. (PR news wire)

🍊   Agricultural robots can replace burdensome work for farmers. In some cases these robots even preform better then humans do in a specific task. (Octinion)

Apply to contribute