Automatic detection and analysis of animal behaviour

Description
Even simple animals, like fruitflies or worms, can make complex decisions, for instance when they interact with predators, possible sex mates or explore food sources. Ethomics is a new discipline of neuroscience, attempting automatic and high-throughput analysis of animal behaviour, and investigating correlations and links to explore how genes drive behaviour. The purpose of this project is to create state of the art techniques of computer vision and machine learning to track animals activity and link them to stereotypical behaviours. Ultimately, we aim at building a system that can recognize animals status or intention and interfere with them using for instance laser or mechanical stimulations.

In particular, an initial application of the machine will be to create a new paradigm to study the links between sleep and learning in Drosophila. This new tool will be used for automatized sleep deprivation and learning conditioning. In the past, we created a software to track Drosophila locomotor activity and analyse sleep. The system, released as open source, is highly scalable, offers high resolution, it is affordable and easy to use (www.pysolo.net). With this project, we aim at expanding the existing system so that it would no longer be limited to passive detection of flies activity, but could actively interact with a group of animals, shining a heat-transducing infrared laser beam onto single flies whenever specific conditions are satisfied.

Project timeline.
Data analysis, computer vision, machine learning, component.
Techniques used: mathematics and statistics applied to data analysis and computer science

1. The existing software is already able to track the animals position in real time and it would need to use this information to direct one or more IR laser diodes connected to motors. However, it will be necessary to modify the video tracking to recognize multiple interacting flies with higher precision and target only those that satisfy predefined conditions . This part will be done in collaboration with Dr. Stefanos Zafeiriou of the computing department at Imperial College.
2. In particular, the software will be modified so that “conditions for shooting” will easily be programmed by the researcher. Also, the software should be able to recognize and output a complete data frame regarding the previous, current and predicted conditions of each flies. This will achieved using machine learning algorithms to identify stereotypic behaviours and act accordingly.

Building and testing the hardware component.
Techniques used: principle of electronics, basic physics of lasers

3. Once flies are successfully tracked and conditions established, the software will interact with a motor controller to aim and shoot one or more laser diodes onto single flies
4. Laser pulses will have to be theoretically selected and empirically calibrated for wavelength, power and duration, to use them as sleep deprivators or for learning conditioning
5. Final protocols will be established for the different possible uses of the machine that will combine part 1 and part 2 of the project

Additional material and links.

• Perona’s and Dickinson’s software for automated fly behaviour is found at this address and described in this paper.
• Another software, again from Perona, is described here.
• A software for ant tracking can be found here, and here are some samples of how it performs.
• A sample of my flies moving in a vial (~15 minutes video at VGA resolution) can be found here
• Other sample movies from the ctrax project can be found here and here

Funding.

This project is funded by a Royal Society research grant.