BioImage.IO Chatbot: recognition in Nature Methods and the next steps

Around this time last year we wrote an article about the newly developed BioImage.IO Chatbot developed by DDLS Fellow Wei Ouyang (KTH) and AICell Lab at SciLifeLab and KTH. At the time, the chatbot was in its infancy and just getting started. In August, it was featured in a special issue in Nature Methods focused on advanced AI in biology.

The BioImage.IO Chatbot is a result of a successful collaboration between the authors Wei Ouyang (AICell Lab, DDLS Fellow at SciLifeLab), Gabriel Reder (AICell Lab and KTH), Wanlu Lei (Ericsson and KTH), Caterina Fuster Barceló and Arrate Muñoz-Barrutia (both at Universidad Carlos III de Madrid and Instituto de Investigación Sanitaria Gregorio Marañón).

According to the authors, the BioImage.IO Chatbot is a community-driven platform designed to streamline access to an extensive array of bioimaging tools, databases, and services. Built with ease of use and intuitiveness in mind, the Chatbot functions as an AI-powered agent, leveraging OpenAI’s GPT-4. BioImage.IO Chatbot is special because of its ability to perform retrieval-augmented generation (RAG). RAG is a technique that enhances the Chatbot’s ability to search external sources like documentation and databases. This minimizes the risk of generating inaccurate information, commonly known as “hallucinations” in AI.

Moreover, the Chatbot employs ReAct (Reasoning and Acting), allowing it to refine its responses iteratively. For example, if the Chatbot initially struggles to find specific information (such as locating a particular protein in the Human Protein Atlas) it can adjust by trying different keywords or search strategies. On top of all this, it can generate Python code for complex bioimage analysis tasks, such as segmenting microscopy images using AI models.

Recognition in Nature Methods

Published in a special issue focusing on advanced AI in biology, the paper highlights the BioImage.IO chatbot as an impactful AI-based tool in the biological research field. “Wei Ouyang and colleagues bring the benefits of large language models directly to biologists analyzing images with their BioImage.IO chatbot, which enables users to benefit from community-wide knowledge and easily navigate complex bioimage analysis tasks through written prompts” (Embedding AI in biology).

What’s next?

With support from the DDLS program, Wei Ouyang’s lab is expanding the BioImage.IO Chatbot into a versatile AI agent capable of reading scientific papers, generating experimental plans, controlling microscopes, operating liquid handling systems. Their goal is to translate scientific SOPs into AI-driven workflows, paving the way for the next generation of automated scientific discovery.

The chatbot is currently available for testing at https://bioimage.io/chat and feel free to stay up-to-date by visiting their GitHub repository.

DOI: 10.1038/s41592-024-02370-y

Contact