Machine Learning Models

The Discourse Analysis Tool Suite makes heavy use of various Machine Learning Models, especially during the import and pre-processing of documents. Here, we list all models that are currently in use.

Named Entity Recognition

• Spacy NER (eng): en_core_web_trf
• Spacy NER (de): de_core_news_lg
• Spacy NER (it): it_core_news_lg

Object Detection

• DETR model with ResNet-50 backbone by facebook

Image Caption Generation

• Vision Transformer + GPT-2 for Image Captioning by nlpconnect

Multi-modal Similarity Search

• Text Encoder: CLIP-based sentence-transformer
• Image Encoder: CLIP

Automatic Video & Audio Transcriptions

• Whisper: Whisper Timestamped

Large Language Models

• Gemini