Zebrafish Detection (This work has been moved to this repo: gongqing/DLMA)

Abstract

This AI project is a detection task for identifying zebrafish organs and phenotypes in micrographs, which is based on the Meta AI project, Detectron2 version 0.4.1. It mainly used Mask R-CNN for training and validating. It has 16 detected objects, including 8 specific organs and 8 specific abnormal phenotypes.

Inference results by Mask R-CNN

How to start

To use our zebrafish AI detection, you have to learn how to install and utilize the detectron2, please refer the detectron2 instructions.

Firstly, install python3.8 or higher version, detectron2 (0.4.1) and other dependent libraries (see zebrafish_maskrcnn.py). Among all the libraries, fishutil and fishclass are two customized python libraries for transforming the model outputs to quantitative parameters of zebrafish.

Then put the pre-trained model weights in your specified path.

# Modify some necessary paths
# path to model weights
cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth")
...
# path to images that pending to detect
path = 'your/specified/path'
...
# save your results
df.to_csv(os.path.join('.../results/', 'results.csv'))

Finally, run the zebrafish_maskrcnn.py file in inference mode locally. To switch the code to inference mode, you have to add a "#" before each line of the Train and Evaluate section in our code, only reserve the Inference section to run the code.

If you want to use TensorMask to identify the objects, please refer to another python file, zebrafish_tensormask.py. Through this model you can also see the identification results in real-time, which is supported by Open CV.

We have opened our image library and annotations, if you want to re-train this model locally, please download the train.json, test.json, train images and test images(see images_url.txt). The json file contains annotations of COCO style. After downloading the images and files, register the coco instances in your code, modify the name and path of the register.

# train registry
from detectron2.data.datasets import register_coco_instances
register_coco_instances("zebrafish_train", {}, "/your/path/train.json",
                        "/your/path/train_images/")
register_coco_instances("zebrafish_train", {}, "/your/path/test.json",
                        "/your/path/test_images/")

Expected results

After the model inference, we can acquire a csv file which cotains the quantitative parameters of specific organs and abnormal phenotypes, these information demonstrate every detail of one specific zebrafish, and we can use them to analyze the developmental status of each zebrafish.

Developer comments

We will continually update this repository and add more images to our library, if you want to use this work, please cite our research article.