Neuon AI in PlantCLEF 2021

This repository contains the team's work in PlantCLEF 2021. The goal of the challenge was to identify plants in field pictures based on a training set of digitized herbarium specimens. We achieved an MRR of 0.181 on the primary metric and an MRR of 0.158 on the secondary metric (difficult species). The official results are released here.

Methodology

Stage 1: Construct and train networks

Herbarium-Field Triplet Loss Network (HFTL Network)

One-streamed Network (OSM Network)

Stage 2: Obtain herbarium dictionary

Herbarium Dictionary Construction

Stage 3: Compare feature similarity between field embedding and herbarium dictionary

Feature similarity comparison

Repository files

Training scripts

• HFTL Network
  • run_HFTL_network.py (Main script)
  • network_module_HFTL.py
• OSM Network
  • run_OSM_network.py (Main script)
  • network_module_OSM.py

Validation scripts

• HFTL Network

  • (1) get_herbarium_dictionary_HFTL.py
  • (2) validate_HFTL_network.py
  • (3) get_mrr_score.py

• OSM Network

  • (1) get_herbarium_dictionary_OSM.py
  • (2) validate_OSM_network.py
  • (3) get_mrr_score.py

Lists

• Herbarium Network

  • Train
    list/herbarium/clef2021_herbarium_combined_train_306005.txt
  • Test
    list/herbarium/clef2021_herbarium_combined_test_15221.txt

• Field Network (2017)

  • Train
    list/field_2017/clef2017_EOL_Web_train_server.txt
  • Test
    list/field_2017/clef2017_EOL_Web_test_server.txt

• Field Network (2021)

  • Train
    list/field_2021/clef2021_field_combined_train.txt
  • Test
    list/field_2021/clef2021_field_combined_test.txt

• HFTL Network

  • Herbarium Train
    list/HFTL/clef2020_known_classes_herbarium_train_added.txt
  • Herbarium Test
    list/HFTL/clef2020_known_classes_herbarium_test.txt
  • Field Train
    list/HFTL/clef2020_known_classes_field_train_added.txt
  • Field Test
    list/HFTL/clef2020_known_classes_field_test.txt

• OSM Network

  • Train
    list/OSM/clef2021_mixed_class_train.txt
  • Test
    list/OSM/clef2021_mixed_class_test.txt

• Herbarium Dictionary

  • Without field bias
    list/997_class_emb_sample_for_herbDict_server_removed_duplicates.txt
  • With field bias
    list/997_class_emb_sample_for_herbDict_new_field_bias.txt

• Test Sets

  • Test Set 1 (With field training images)
    list/HFTL/clef2020_known_classes_field_test.txt
  • Test Set 2 (Without field training images)
    list/missing_class_sample.txt
    (Note: The image files for Test Set 2 are sourced from Google Image queries)

Checkpoints

• HFTL Network
  • HFTL inception v4 model
• OSM Network
  • OSM inception v4 model