experiments/3.1. Data Consolidation (ProstT5).ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "id": "666ea2fa",
            "metadata": {},
            "source": [
                "# PHIStruct: Improving phage-host interaction prediction at low sequence similarity settings using structure-aware protein embeddings\n",
                "\n",
                "<b>Mark Edward M. Gonzales<sup>1, 2</sup>, Jennifer C. Ureta<sup>1, 2, 3</sup> & Anish M.S. Shrestha<sup>1, 2</sup></b>\n",
                "\n",
                "<sup>1</sup> Bioinformatics Lab, Advanced Research Institute for Informatics, Computing and Networking, De La Salle University, Manila 1004, Philippines <br>\n",
                "<sup>2</sup> Department of Software Technology, College of Computer Studies, De La Salle University, Manila 1004, Philippines <br>\n",
"<sup>3</sup> Walter and Eliza Hall Institute of Medical Research, Melbourne, Victoria, 3052, Australia\n",
                "\n",
                "✉️ gonzales.markedward@gmail.com, jennifer.ureta@gmail.com, anish.shrestha@dlsu.edu.ph"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "164216b8",
            "metadata": {},
            "source": [
                "<hr>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "13703268",
            "metadata": {},
            "source": [
                "# 💡 Prerequisites\n",
                "\n",
                "### Option 1: Download the prerequisite files\n",
                "1. Download `rbp_prostt5_embeddings.h5` from this [link](https://drive.google.com/file/d/1oNJkzVwTJmy7D38KGOnzn3PsLfDBavmG/view?usp=sharing), and unzip it.\n",
                "1. Create a folder named `inphared` inside `data`. <br>\n",
                "   Create a folder named `structure`, this time inside `data/inphared`, and save `rbp_prostt5_embeddings.h5` inside `data/inphared/structure`.\n",
                "1. Download `consolidated.tar.gz` from this [link](https://drive.google.com/file/d/1yQSXwlb37dm2ZLXGJHdIM5vmrzwPAwvI/view?usp=sharing), and unzip it. This should result in a folder named `consolidated`. <br> Technically, this notebook only needs `consolidated/rbp.csv`.\n",
                "1. Save the extracted `consolidated` folder inside `data/inphared`. \n",
                "\n",
                "\n",
                "### Option 2: Generate the prerequisite files yourself (may take a couple of weeks!)\n",
                "1. Use ColabFold to predict the protein structures of the sequences in `data/inphared/fasta/complete`, following the instructions [here](https://github.com/YoshitakaMo/localcolabfold). <br>Refer to our paper for the parameters at which we ran ColabFold. <br>For reproducibility, we provide the results of running ColabFold [here](https://drive.google.com/file/d/1ZPRdaHwsFOPksLbOyQerREG0gY0p4-AT/view?usp=sharing).\n",
                "1. Encode the predicted structures using Foldseek, following the instructions [here](https://github.com/mheinzinger/ProstT5?tab=readme-ov-file#-how-to-derive-3di-sequences-from-structures). <br>For reproducibility, we provide the results of this encoding step [here](https://drive.google.com/file/d/1_GkC6NH2FiHh0AwqA7SARGjrkDHxE8dP/view?usp=sharing).\n",
                "1. Feed the results of the encoding step to ProstT5 in order to generate the structure-aware embeddings, following the instructions [here](https://github.com/mheinzinger/ProstT5/tree/main/scripts). \n",
                "1. Consolidate the embeddings into a single HDF5 file named `rbp_prostt5_embeddings.h5`. \n",
                "1. Create a folder named `inphared` inside `data`. <br>\n",
                "   Create a folder named `structure`, this time inside `data/inphared`, and save `rbp_prostt5_embeddings.h5` inside `data/inphared/structure`.\n",
                "1. Download `consolidated.tar.gz` from this [link](https://drive.google.com/file/d/1yQSXwlb37dm2ZLXGJHdIM5vmrzwPAwvI/view?usp=sharing), and unzip it. This should result in a folder named `consolidated`. <br> Technically, this notebook only needs `consolidated/rbp.csv`.\n",
                "1. Create a folder named `inphared` inside `data`, and save the extracted `consolidated` folder inside `data/inphared`. \n",
                "   \n",
                "### Resulting folder structure\n",
                "\n",
                "`experiments` (parent folder of this notebook) <br> \n",
                "↳ `data` <br>\n",
                "&nbsp; &nbsp;↳ `inphared` <br>\n",
                "&nbsp; &nbsp;&nbsp; &nbsp; ↳ `consolidated` <br>\n",
                "&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp; ↳ `rbp.csv` <br>\n",
                "&nbsp; &nbsp;&nbsp; &nbsp; ↳ `structure` <br>\n",
                "&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp; ↳ `rbp_prostt5_embeddings.h5` <br>\n",
                "↳ `3.1. Data Consolidation (ProstT5).ipynb` (this notebook) <br>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "4702fd1b",
            "metadata": {},
            "source": [
                "<hr>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "e3e3b2cd",
            "metadata": {},
            "source": [
                "# 📁 Output files\n",
                "\n",
                "1. If you would like to skip running this notebook, download `rbp_prostt5_relaxed_r3.csv` from this [link](https://drive.google.com/file/d/1PLrfpkUd37G8jbYInWFoghlw_SGHogSV/view?usp=sharing). This CSV file consolidates the embeddings.\n",
                "\n",
                "1. Create a folder named `inphared` inside `data`. <br>\n",
                "   Create a folder named `structure`, this time inside `data/inphared`, save `rbp_prostt5_relaxed_r3.csv` inside `data/inphared/structure`.\n",
                "\n",
                "1. Download `consolidated.tar.gz` from this [link](https://drive.google.com/file/d/1yQSXwlb37dm2ZLXGJHdIM5vmrzwPAwvI/view?usp=sharing), and unzip it. This should result in a folder named `consolidated`. <br> Technically, this notebook generates only `consolidated/rbp_embeddings_prostt5_relaxed_r3.csv`, which consolidates the phage-host information and the embeddings. \n",
                "\n",
                "1. Save the extracted `consolidated` folder inside `data/inphared`. \n",
                "\n",
                "\n",
                "### Resulting folder structure\n",
                "\n",
                "`experiments` (parent folder of this notebook) <br> \n",
                "↳ `data` <br>\n",
                "&nbsp; &nbsp;↳ `inphared` <br>\n",
                "&nbsp; &nbsp;&nbsp; &nbsp; ↳ `consolidated` <br>\n",
                "&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp; ↳ `rbp.csv` <br>\n",
                "&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp; ↳ `rbp_embeddings_prostt5_relaxed_r3.csv` <br>\n",
                "&nbsp; &nbsp;&nbsp; &nbsp; ↳ `structure` <br>\n",
                "&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp; ↳ `rbp_prostt5_embeddings.h5` <br>\n",
                "&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp; ↳ `rbp_prostt5_relaxed_r3.csv` <br>\n",
                "↳ `3.1. Data Consolidation (ProstT5).ipynb` (this notebook) <br>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "f1dbb929",
            "metadata": {},
            "source": [
                "<hr>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "827178f8",
            "metadata": {},
            "source": [
                "# Part I: Preliminaries"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "6c907ccc",
            "metadata": {},
            "source": [
                "Import the necessary libraries and modules."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "id": "4d080e9b",
            "metadata": {},
            "outputs": [],
            "source": [
                "import pandas as pd\n",
                "\n",
                "import ConstantsUtil\n",
                "import StructureUtil\n",
                "\n",
                "%load_ext autoreload\n",
                "%autoreload 2"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "id": "94773cb6",
            "metadata": {},
            "outputs": [],
            "source": [
                "constants = ConstantsUtil.ConstantsUtil()\n",
                "util = StructureUtil.StructureUtil()"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "2e0b8146",
            "metadata": {},
            "source": [
                "<hr>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "f0a7ff46",
            "metadata": {},
            "source": [
                "# Part II: Consolidation of the ProstT5 embeddings"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "ae946fd1",
            "metadata": {},
            "source": [
                "Consolidate the embeddings into a single data frame."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "id": "b448b7d6",
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": [
                        "100%|█████████████████████████████████████████████████████████████████████████| 144885/144885 [23:53<00:00, 101.07it/s]\n"
                    ]
                },
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>Protein ID</th>\n",
                            "      <th>s1</th>\n",
                            "      <th>s2</th>\n",
                            "      <th>s3</th>\n",
                            "      <th>s4</th>\n",
                            "      <th>s5</th>\n",
                            "      <th>s6</th>\n",
                            "      <th>s7</th>\n",
                            "      <th>s8</th>\n",
                            "      <th>s9</th>\n",
                            "      <th>...</th>\n",
                            "      <th>s1015</th>\n",
                            "      <th>s1016</th>\n",
                            "      <th>s1017</th>\n",
                            "      <th>s1018</th>\n",
                            "      <th>s1019</th>\n",
                            "      <th>s1020</th>\n",
                            "      <th>s1021</th>\n",
                            "      <th>s1022</th>\n",
                            "      <th>s1023</th>\n",
                            "      <th>s1024</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>0</th>\n",
                            "      <td>AAA74324_1_relaxed_r3_pdb</td>\n",
                            "      <td>-0.008888</td>\n",
                            "      <td>0.010757</td>\n",
                            "      <td>0.026901</td>\n",
                            "      <td>0.002495</td>\n",
                            "      <td>0.006329</td>\n",
                            "      <td>-0.001117</td>\n",
                            "      <td>0.018707</td>\n",
                            "      <td>-0.026306</td>\n",
                            "      <td>-0.034668</td>\n",
                            "      <td>...</td>\n",
                            "      <td>-0.001294</td>\n",
                            "      <td>-0.026688</td>\n",
                            "      <td>0.008217</td>\n",
                            "      <td>0.004044</td>\n",
                            "      <td>-0.000456</td>\n",
                            "      <td>0.013741</td>\n",
                            "      <td>0.004570</td>\n",
                            "      <td>0.031921</td>\n",
                            "      <td>0.003466</td>\n",
                            "      <td>-0.022720</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>1</th>\n",
                            "      <td>AAA74331_1_relaxed_r3_pdb</td>\n",
                            "      <td>0.008438</td>\n",
                            "      <td>-0.006050</td>\n",
                            "      <td>0.025681</td>\n",
                            "      <td>0.018616</td>\n",
                            "      <td>0.007584</td>\n",
                            "      <td>-0.031097</td>\n",
                            "      <td>0.020370</td>\n",
                            "      <td>-0.010178</td>\n",
                            "      <td>-0.002850</td>\n",
                            "      <td>...</td>\n",
                            "      <td>0.013016</td>\n",
                            "      <td>0.058044</td>\n",
                            "      <td>-0.033661</td>\n",
                            "      <td>-0.013657</td>\n",
                            "      <td>0.029449</td>\n",
                            "      <td>-0.039368</td>\n",
                            "      <td>-0.026581</td>\n",
                            "      <td>-0.004993</td>\n",
                            "      <td>0.021896</td>\n",
                            "      <td>0.000183</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>2</th>\n",
                            "      <td>AAA98578_2_relaxed_r3_pdb</td>\n",
                            "      <td>-0.026337</td>\n",
                            "      <td>0.014549</td>\n",
                            "      <td>0.033600</td>\n",
                            "      <td>0.019562</td>\n",
                            "      <td>0.055298</td>\n",
                            "      <td>0.024307</td>\n",
                            "      <td>0.049469</td>\n",
                            "      <td>0.012596</td>\n",
                            "      <td>0.010185</td>\n",
                            "      <td>...</td>\n",
                            "      <td>0.043640</td>\n",
                            "      <td>0.023865</td>\n",
                            "      <td>-0.026382</td>\n",
                            "      <td>-0.036011</td>\n",
                            "      <td>0.025574</td>\n",
                            "      <td>-0.017944</td>\n",
                            "      <td>0.064209</td>\n",
                            "      <td>0.006153</td>\n",
                            "      <td>0.002045</td>\n",
                            "      <td>-0.022614</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>3</th>\n",
                            "      <td>AAB09218_1_relaxed_r3_pdb</td>\n",
                            "      <td>-0.000671</td>\n",
                            "      <td>0.030960</td>\n",
                            "      <td>0.063782</td>\n",
                            "      <td>-0.007980</td>\n",
                            "      <td>0.036011</td>\n",
                            "      <td>0.007420</td>\n",
                            "      <td>0.086060</td>\n",
                            "      <td>-0.041656</td>\n",
                            "      <td>0.032410</td>\n",
                            "      <td>...</td>\n",
                            "      <td>0.027679</td>\n",
                            "      <td>0.028122</td>\n",
                            "      <td>-0.031738</td>\n",
                            "      <td>-0.007370</td>\n",
                            "      <td>-0.011604</td>\n",
                            "      <td>0.044434</td>\n",
                            "      <td>0.035614</td>\n",
                            "      <td>0.031067</td>\n",
                            "      <td>-0.033691</td>\n",
                            "      <td>0.008873</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>4</th>\n",
                            "      <td>AAB70057_1_relaxed_r3_pdb</td>\n",
                            "      <td>-0.002905</td>\n",
                            "      <td>-0.011444</td>\n",
                            "      <td>0.025513</td>\n",
                            "      <td>0.030548</td>\n",
                            "      <td>0.007416</td>\n",
                            "      <td>-0.010193</td>\n",
                            "      <td>0.056671</td>\n",
                            "      <td>-0.035767</td>\n",
                            "      <td>-0.015480</td>\n",
                            "      <td>...</td>\n",
                            "      <td>-0.002598</td>\n",
                            "      <td>0.052521</td>\n",
                            "      <td>-0.021255</td>\n",
                            "      <td>0.022614</td>\n",
                            "      <td>0.012123</td>\n",
                            "      <td>-0.007881</td>\n",
                            "      <td>0.025803</td>\n",
                            "      <td>0.020477</td>\n",
                            "      <td>-0.004837</td>\n",
                            "      <td>0.003382</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "<p>5 rows × 1025 columns</p>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "                  Protein ID        s1        s2        s3        s4  \\\n",
                            "0  AAA74324_1_relaxed_r3_pdb -0.008888  0.010757  0.026901  0.002495   \n",
                            "1  AAA74331_1_relaxed_r3_pdb  0.008438 -0.006050  0.025681  0.018616   \n",
                            "2  AAA98578_2_relaxed_r3_pdb -0.026337  0.014549  0.033600  0.019562   \n",
                            "3  AAB09218_1_relaxed_r3_pdb -0.000671  0.030960  0.063782 -0.007980   \n",
                            "4  AAB70057_1_relaxed_r3_pdb -0.002905 -0.011444  0.025513  0.030548   \n",
                            "\n",
                            "         s5        s6        s7        s8        s9  ...     s1015     s1016  \\\n",
                            "0  0.006329 -0.001117  0.018707 -0.026306 -0.034668  ... -0.001294 -0.026688   \n",
                            "1  0.007584 -0.031097  0.020370 -0.010178 -0.002850  ...  0.013016  0.058044   \n",
                            "2  0.055298  0.024307  0.049469  0.012596  0.010185  ...  0.043640  0.023865   \n",
                            "3  0.036011  0.007420  0.086060 -0.041656  0.032410  ...  0.027679  0.028122   \n",
                            "4  0.007416 -0.010193  0.056671 -0.035767 -0.015480  ... -0.002598  0.052521   \n",
                            "\n",
                            "      s1017     s1018     s1019     s1020     s1021     s1022     s1023  \\\n",
                            "0  0.008217  0.004044 -0.000456  0.013741  0.004570  0.031921  0.003466   \n",
                            "1 -0.033661 -0.013657  0.029449 -0.039368 -0.026581 -0.004993  0.021896   \n",
                            "2 -0.026382 -0.036011  0.025574 -0.017944  0.064209  0.006153  0.002045   \n",
                            "3 -0.031738 -0.007370 -0.011604  0.044434  0.035614  0.031067 -0.033691   \n",
                            "4 -0.021255  0.022614  0.012123 -0.007881  0.025803  0.020477 -0.004837   \n",
                            "\n",
                            "      s1024  \n",
                            "0 -0.022720  \n",
                            "1  0.000183  \n",
                            "2 -0.022614  \n",
                            "3  0.008873  \n",
                            "4  0.003382  \n",
                            "\n",
                            "[5 rows x 1025 columns]"
                        ]
                    },
                    "execution_count": 3,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "rbp_prostt5_relaxed_r3 = util.convert_prostt5_h5_to_df(\n",
                "    f\"{constants.INPHARED}/{constants.STRUCTURE_PROSTT5}\", \"_relaxed_r3_pdb\"\n",
                ")\n",
                "rbp_prostt5_relaxed_r3.head()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "id": "0dedf2a4",
            "metadata": {},
            "outputs": [],
            "source": [
                "rbp_prostt5_relaxed_r3.to_csv(\n",
                "    f\"{constants.INPHARED}/{constants.CSV_PROSTT5}_relaxed_r3.csv\", index=False\n",
                ")"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "ca09a07e",
            "metadata": {},
            "source": [
                "Combine the embeddings data frame with the data frame containing the phage-host information."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "id": "c5cc5f13",
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>Protein ID</th>\n",
                            "      <th>Accession</th>\n",
                            "      <th>Description</th>\n",
                            "      <th>Classification</th>\n",
                            "      <th>Genome Length (bp)</th>\n",
                            "      <th>Jumbophage</th>\n",
                            "      <th>molGC (%)</th>\n",
                            "      <th>Molecule</th>\n",
                            "      <th>Modification Date</th>\n",
                            "      <th>Number CDS</th>\n",
                            "      <th>...</th>\n",
                            "      <th>Genbank Division</th>\n",
                            "      <th>Isolation Host (beware inconsistent and nonsense values)</th>\n",
                            "      <th>Host Superkingdom</th>\n",
                            "      <th>Host Phylum</th>\n",
                            "      <th>Host Class</th>\n",
                            "      <th>Host Order</th>\n",
                            "      <th>Host Family</th>\n",
                            "      <th>Year-Month</th>\n",
                            "      <th>Protein Sequence</th>\n",
                            "      <th>Nucleotide Sequence</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>0</th>\n",
                            "      <td>BAD16801.1</td>\n",
                            "      <td>AB161975</td>\n",
                            "      <td>Wolbachia phage WOcauB1</td>\n",
                            "      <td>Wolbachia phage WOcauB1 Viruses</td>\n",
                            "      <td>20484</td>\n",
                            "      <td>False</td>\n",
                            "      <td>36.878</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2016-07-26</td>\n",
                            "      <td>27</td>\n",
                            "      <td>...</td>\n",
                            "      <td>PHG</td>\n",
                            "      <td>Wolbachia sp. wCauB</td>\n",
                            "      <td>bacteria</td>\n",
                            "      <td>pseudomonadota</td>\n",
                            "      <td>alphaproteobacteria</td>\n",
                            "      <td>rickettsiales</td>\n",
                            "      <td>anaplasmataceae</td>\n",
                            "      <td>2016-07</td>\n",
                            "      <td>MKEAIYQRIKDLAANSTPDQLAYLAKSLELIADKKAISNVVQMTEV...</td>\n",
                            "      <td>ATGAAAGAAGCAATATACCAAAGGATAAAGGATTTAGCAGCAAATA...</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>1</th>\n",
                            "      <td>BAF36105.1</td>\n",
                            "      <td>AB231700</td>\n",
                            "      <td>Microcystis phage LMM01</td>\n",
                            "      <td>Microcystis phage LMM01 Fukuivirus LMM01 Fukui...</td>\n",
                            "      <td>162109</td>\n",
                            "      <td>False</td>\n",
                            "      <td>45.953</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2023-08-22</td>\n",
                            "      <td>189</td>\n",
                            "      <td>...</td>\n",
                            "      <td>PHG</td>\n",
                            "      <td>Unspecified</td>\n",
                            "      <td>bacteria</td>\n",
                            "      <td>cyanobacteriota</td>\n",
                            "      <td>cyanophyceae</td>\n",
                            "      <td>chroococcales</td>\n",
                            "      <td>microcystaceae</td>\n",
                            "      <td>2023-08</td>\n",
                            "      <td>MHQNISKENRGNYNNGIRPRIFMITTIDFRDIQAACIKQLDDMSKD...</td>\n",
                            "      <td>GTGCATCAAAATATTTCAAAGGAGAATCGTGGAAACTATAACAACG...</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>2</th>\n",
                            "      <td>BAF36110.1</td>\n",
                            "      <td>AB231700</td>\n",
                            "      <td>Microcystis phage LMM01</td>\n",
                            "      <td>Microcystis phage LMM01 Fukuivirus LMM01 Fukui...</td>\n",
                            "      <td>162109</td>\n",
                            "      <td>False</td>\n",
                            "      <td>45.953</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2023-08-22</td>\n",
                            "      <td>189</td>\n",
                            "      <td>...</td>\n",
                            "      <td>PHG</td>\n",
                            "      <td>Unspecified</td>\n",
                            "      <td>bacteria</td>\n",
                            "      <td>cyanobacteriota</td>\n",
                            "      <td>cyanophyceae</td>\n",
                            "      <td>chroococcales</td>\n",
                            "      <td>microcystaceae</td>\n",
                            "      <td>2023-08</td>\n",
                            "      <td>MRIFYIHHPFLATHRYLLSNAYSTPYTDSITKLTTSYSSMPIILSV...</td>\n",
                            "      <td>GTGAGGATTTTTTATATCCACCATCCATTCCTCGCTACTCACCGAT...</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>3</th>\n",
                            "      <td>BAF36131.1</td>\n",
                            "      <td>AB231700</td>\n",
                            "      <td>Microcystis phage LMM01</td>\n",
                            "      <td>Microcystis phage LMM01 Fukuivirus LMM01 Fukui...</td>\n",
                            "      <td>162109</td>\n",
                            "      <td>False</td>\n",
                            "      <td>45.953</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2023-08-22</td>\n",
                            "      <td>189</td>\n",
                            "      <td>...</td>\n",
                            "      <td>PHG</td>\n",
                            "      <td>Unspecified</td>\n",
                            "      <td>bacteria</td>\n",
                            "      <td>cyanobacteriota</td>\n",
                            "      <td>cyanophyceae</td>\n",
                            "      <td>chroococcales</td>\n",
                            "      <td>microcystaceae</td>\n",
                            "      <td>2023-08</td>\n",
                            "      <td>MLTDVDIQALIDASISGLSGEMPIVANIAARNALSLTKNTQVLVLD...</td>\n",
                            "      <td>TTGCTGACAGATGTCGATATTCAGGCATTAATTGATGCCTCAATTT...</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>4</th>\n",
                            "      <td>BAF36132.1</td>\n",
                            "      <td>AB231700</td>\n",
                            "      <td>Microcystis phage LMM01</td>\n",
                            "      <td>Microcystis phage LMM01 Fukuivirus LMM01 Fukui...</td>\n",
                            "      <td>162109</td>\n",
                            "      <td>False</td>\n",
                            "      <td>45.953</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2023-08-22</td>\n",
                            "      <td>189</td>\n",
                            "      <td>...</td>\n",
                            "      <td>PHG</td>\n",
                            "      <td>Unspecified</td>\n",
                            "      <td>bacteria</td>\n",
                            "      <td>cyanobacteriota</td>\n",
                            "      <td>cyanophyceae</td>\n",
                            "      <td>chroococcales</td>\n",
                            "      <td>microcystaceae</td>\n",
                            "      <td>2023-08</td>\n",
                            "      <td>MFGVFIVRREGGYIGTQPNWDAANRPGNWDILDVYNRQRRNLWIQS...</td>\n",
                            "      <td>TTGTTCGGAGTTTTTATCGTGAGGCGTGAAGGCGGCTATATCGGAA...</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "<p>5 rows × 36 columns</p>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "   Protein ID Accession              Description  \\\n",
                            "0  BAD16801.1  AB161975  Wolbachia phage WOcauB1   \n",
                            "1  BAF36105.1  AB231700  Microcystis phage LMM01   \n",
                            "2  BAF36110.1  AB231700  Microcystis phage LMM01   \n",
                            "3  BAF36131.1  AB231700  Microcystis phage LMM01   \n",
                            "4  BAF36132.1  AB231700  Microcystis phage LMM01   \n",
                            "\n",
                            "                                      Classification  Genome Length (bp)  \\\n",
                            "0                    Wolbachia phage WOcauB1 Viruses               20484   \n",
                            "1  Microcystis phage LMM01 Fukuivirus LMM01 Fukui...              162109   \n",
                            "2  Microcystis phage LMM01 Fukuivirus LMM01 Fukui...              162109   \n",
                            "3  Microcystis phage LMM01 Fukuivirus LMM01 Fukui...              162109   \n",
                            "4  Microcystis phage LMM01 Fukuivirus LMM01 Fukui...              162109   \n",
                            "\n",
                            "   Jumbophage  molGC (%) Molecule Modification Date  Number CDS  ...  \\\n",
                            "0       False     36.878      DNA        2016-07-26          27  ...   \n",
                            "1       False     45.953      DNA        2023-08-22         189  ...   \n",
                            "2       False     45.953      DNA        2023-08-22         189  ...   \n",
                            "3       False     45.953      DNA        2023-08-22         189  ...   \n",
                            "4       False     45.953      DNA        2023-08-22         189  ...   \n",
                            "\n",
                            "   Genbank Division  Isolation Host (beware inconsistent and nonsense values)  \\\n",
                            "0               PHG                                Wolbachia sp. wCauB          \n",
                            "1               PHG                                        Unspecified          \n",
                            "2               PHG                                        Unspecified          \n",
                            "3               PHG                                        Unspecified          \n",
                            "4               PHG                                        Unspecified          \n",
                            "\n",
                            "   Host Superkingdom      Host Phylum           Host Class     Host Order  \\\n",
                            "0           bacteria   pseudomonadota  alphaproteobacteria  rickettsiales   \n",
                            "1           bacteria  cyanobacteriota         cyanophyceae  chroococcales   \n",
                            "2           bacteria  cyanobacteriota         cyanophyceae  chroococcales   \n",
                            "3           bacteria  cyanobacteriota         cyanophyceae  chroococcales   \n",
                            "4           bacteria  cyanobacteriota         cyanophyceae  chroococcales   \n",
                            "\n",
                            "       Host Family Year-Month  \\\n",
                            "0  anaplasmataceae    2016-07   \n",
                            "1   microcystaceae    2023-08   \n",
                            "2   microcystaceae    2023-08   \n",
                            "3   microcystaceae    2023-08   \n",
                            "4   microcystaceae    2023-08   \n",
                            "\n",
                            "                                    Protein Sequence  \\\n",
                            "0  MKEAIYQRIKDLAANSTPDQLAYLAKSLELIADKKAISNVVQMTEV...   \n",
                            "1  MHQNISKENRGNYNNGIRPRIFMITTIDFRDIQAACIKQLDDMSKD...   \n",
                            "2  MRIFYIHHPFLATHRYLLSNAYSTPYTDSITKLTTSYSSMPIILSV...   \n",
                            "3  MLTDVDIQALIDASISGLSGEMPIVANIAARNALSLTKNTQVLVLD...   \n",
                            "4  MFGVFIVRREGGYIGTQPNWDAANRPGNWDILDVYNRQRRNLWIQS...   \n",
                            "\n",
                            "                                 Nucleotide Sequence  \n",
                            "0  ATGAAAGAAGCAATATACCAAAGGATAAAGGATTTAGCAGCAAATA...  \n",
                            "1  GTGCATCAAAATATTTCAAAGGAGAATCGTGGAAACTATAACAACG...  \n",
                            "2  GTGAGGATTTTTTATATCCACCATCCATTCCTCGCTACTCACCGAT...  \n",
                            "3  TTGCTGACAGATGTCGATATTCAGGCATTAATTGATGCCTCAATTT...  \n",
                            "4  TTGTTCGGAGTTTTTATCGTGAGGCGTGAAGGCGGCTATATCGGAA...  \n",
                            "\n",
                            "[5 rows x 36 columns]"
                        ]
                    },
                    "execution_count": 5,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "rbp_nucleotide_seq = pd.read_csv(\n",
                "    f\"{constants.INPHARED}/{constants.CONSOLIDATED}/{constants.INPHARED_RBP_DATA}\"\n",
                ")\n",
                "rbp_nucleotide_seq.head()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "id": "26c96a3b",
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>Protein ID</th>\n",
                            "      <th>Accession</th>\n",
                            "      <th>Description</th>\n",
                            "      <th>Classification</th>\n",
                            "      <th>Genome Length (bp)</th>\n",
                            "      <th>Jumbophage</th>\n",
                            "      <th>molGC (%)</th>\n",
                            "      <th>Molecule</th>\n",
                            "      <th>Modification Date</th>\n",
                            "      <th>Number CDS</th>\n",
                            "      <th>...</th>\n",
                            "      <th>Isolation Host (beware inconsistent and nonsense values)</th>\n",
                            "      <th>Host Superkingdom</th>\n",
                            "      <th>Host Phylum</th>\n",
                            "      <th>Host Class</th>\n",
                            "      <th>Host Order</th>\n",
                            "      <th>Host Family</th>\n",
                            "      <th>Year-Month</th>\n",
                            "      <th>Protein Sequence</th>\n",
                            "      <th>Nucleotide Sequence</th>\n",
                            "      <th>Protein ID (Clean)</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>0</th>\n",
                            "      <td>BAD16801.1</td>\n",
                            "      <td>AB161975</td>\n",
                            "      <td>Wolbachia phage WOcauB1</td>\n",
                            "      <td>Wolbachia phage WOcauB1 Viruses</td>\n",
                            "      <td>20484</td>\n",
                            "      <td>False</td>\n",
                            "      <td>36.878</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2016-07-26</td>\n",
                            "      <td>27</td>\n",
                            "      <td>...</td>\n",
                            "      <td>Wolbachia sp. wCauB</td>\n",
                            "      <td>bacteria</td>\n",
                            "      <td>pseudomonadota</td>\n",
                            "      <td>alphaproteobacteria</td>\n",
                            "      <td>rickettsiales</td>\n",
                            "      <td>anaplasmataceae</td>\n",
                            "      <td>2016-07</td>\n",
                            "      <td>MKEAIYQRIKDLAANSTPDQLAYLAKSLELIADKKAISNVVQMTEV...</td>\n",
                            "      <td>ATGAAAGAAGCAATATACCAAAGGATAAAGGATTTAGCAGCAAATA...</td>\n",
                            "      <td>BAD16801_1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>1</th>\n",
                            "      <td>BAF36105.1</td>\n",
                            "      <td>AB231700</td>\n",
                            "      <td>Microcystis phage LMM01</td>\n",
                            "      <td>Microcystis phage LMM01 Fukuivirus LMM01 Fukui...</td>\n",
                            "      <td>162109</td>\n",
                            "      <td>False</td>\n",
                            "      <td>45.953</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2023-08-22</td>\n",
                            "      <td>189</td>\n",
                            "      <td>...</td>\n",
                            "      <td>Unspecified</td>\n",
                            "      <td>bacteria</td>\n",
                            "      <td>cyanobacteriota</td>\n",
                            "      <td>cyanophyceae</td>\n",
                            "      <td>chroococcales</td>\n",
                            "      <td>microcystaceae</td>\n",
                            "      <td>2023-08</td>\n",
                            "      <td>MHQNISKENRGNYNNGIRPRIFMITTIDFRDIQAACIKQLDDMSKD...</td>\n",
                            "      <td>GTGCATCAAAATATTTCAAAGGAGAATCGTGGAAACTATAACAACG...</td>\n",
                            "      <td>BAF36105_1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>2</th>\n",
                            "      <td>BAF36110.1</td>\n",
                            "      <td>AB231700</td>\n",
                            "      <td>Microcystis phage LMM01</td>\n",
                            "      <td>Microcystis phage LMM01 Fukuivirus LMM01 Fukui...</td>\n",
                            "      <td>162109</td>\n",
                            "      <td>False</td>\n",
                            "      <td>45.953</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2023-08-22</td>\n",
                            "      <td>189</td>\n",
                            "      <td>...</td>\n",
                            "      <td>Unspecified</td>\n",
                            "      <td>bacteria</td>\n",
                            "      <td>cyanobacteriota</td>\n",
                            "      <td>cyanophyceae</td>\n",
                            "      <td>chroococcales</td>\n",
                            "      <td>microcystaceae</td>\n",
                            "      <td>2023-08</td>\n",
                            "      <td>MRIFYIHHPFLATHRYLLSNAYSTPYTDSITKLTTSYSSMPIILSV...</td>\n",
                            "      <td>GTGAGGATTTTTTATATCCACCATCCATTCCTCGCTACTCACCGAT...</td>\n",
                            "      <td>BAF36110_1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>3</th>\n",
                            "      <td>BAF36131.1</td>\n",
                            "      <td>AB231700</td>\n",
                            "      <td>Microcystis phage LMM01</td>\n",
                            "      <td>Microcystis phage LMM01 Fukuivirus LMM01 Fukui...</td>\n",
                            "      <td>162109</td>\n",
                            "      <td>False</td>\n",
                            "      <td>45.953</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2023-08-22</td>\n",
                            "      <td>189</td>\n",
                            "      <td>...</td>\n",
                            "      <td>Unspecified</td>\n",
                            "      <td>bacteria</td>\n",
                            "      <td>cyanobacteriota</td>\n",
                            "      <td>cyanophyceae</td>\n",
                            "      <td>chroococcales</td>\n",
                            "      <td>microcystaceae</td>\n",
                            "      <td>2023-08</td>\n",
                            "      <td>MLTDVDIQALIDASISGLSGEMPIVANIAARNALSLTKNTQVLVLD...</td>\n",
                            "      <td>TTGCTGACAGATGTCGATATTCAGGCATTAATTGATGCCTCAATTT...</td>\n",
                            "      <td>BAF36131_1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>4</th>\n",
                            "      <td>BAF36132.1</td>\n",
                            "      <td>AB231700</td>\n",
                            "      <td>Microcystis phage LMM01</td>\n",
                            "      <td>Microcystis phage LMM01 Fukuivirus LMM01 Fukui...</td>\n",
                            "      <td>162109</td>\n",
                            "      <td>False</td>\n",
                            "      <td>45.953</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2023-08-22</td>\n",
                            "      <td>189</td>\n",
                            "      <td>...</td>\n",
                            "      <td>Unspecified</td>\n",
                            "      <td>bacteria</td>\n",
                            "      <td>cyanobacteriota</td>\n",
                            "      <td>cyanophyceae</td>\n",
                            "      <td>chroococcales</td>\n",
                            "      <td>microcystaceae</td>\n",
                            "      <td>2023-08</td>\n",
                            "      <td>MFGVFIVRREGGYIGTQPNWDAANRPGNWDILDVYNRQRRNLWIQS...</td>\n",
                            "      <td>TTGTTCGGAGTTTTTATCGTGAGGCGTGAAGGCGGCTATATCGGAA...</td>\n",
                            "      <td>BAF36132_1</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "<p>5 rows × 37 columns</p>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "   Protein ID Accession              Description  \\\n",
                            "0  BAD16801.1  AB161975  Wolbachia phage WOcauB1   \n",
                            "1  BAF36105.1  AB231700  Microcystis phage LMM01   \n",
                            "2  BAF36110.1  AB231700  Microcystis phage LMM01   \n",
                            "3  BAF36131.1  AB231700  Microcystis phage LMM01   \n",
                            "4  BAF36132.1  AB231700  Microcystis phage LMM01   \n",
                            "\n",
                            "                                      Classification  Genome Length (bp)  \\\n",
                            "0                    Wolbachia phage WOcauB1 Viruses               20484   \n",
                            "1  Microcystis phage LMM01 Fukuivirus LMM01 Fukui...              162109   \n",
                            "2  Microcystis phage LMM01 Fukuivirus LMM01 Fukui...              162109   \n",
                            "3  Microcystis phage LMM01 Fukuivirus LMM01 Fukui...              162109   \n",
                            "4  Microcystis phage LMM01 Fukuivirus LMM01 Fukui...              162109   \n",
                            "\n",
                            "   Jumbophage  molGC (%) Molecule Modification Date  Number CDS  ...  \\\n",
                            "0       False     36.878      DNA        2016-07-26          27  ...   \n",
                            "1       False     45.953      DNA        2023-08-22         189  ...   \n",
                            "2       False     45.953      DNA        2023-08-22         189  ...   \n",
                            "3       False     45.953      DNA        2023-08-22         189  ...   \n",
                            "4       False     45.953      DNA        2023-08-22         189  ...   \n",
                            "\n",
                            "   Isolation Host (beware inconsistent and nonsense values)  \\\n",
                            "0                                Wolbachia sp. wCauB          \n",
                            "1                                        Unspecified          \n",
                            "2                                        Unspecified          \n",
                            "3                                        Unspecified          \n",
                            "4                                        Unspecified          \n",
                            "\n",
                            "   Host Superkingdom      Host Phylum           Host Class     Host Order  \\\n",
                            "0           bacteria   pseudomonadota  alphaproteobacteria  rickettsiales   \n",
                            "1           bacteria  cyanobacteriota         cyanophyceae  chroococcales   \n",
                            "2           bacteria  cyanobacteriota         cyanophyceae  chroococcales   \n",
                            "3           bacteria  cyanobacteriota         cyanophyceae  chroococcales   \n",
                            "4           bacteria  cyanobacteriota         cyanophyceae  chroococcales   \n",
                            "\n",
                            "       Host Family Year-Month  \\\n",
                            "0  anaplasmataceae    2016-07   \n",
                            "1   microcystaceae    2023-08   \n",
                            "2   microcystaceae    2023-08   \n",
                            "3   microcystaceae    2023-08   \n",
                            "4   microcystaceae    2023-08   \n",
                            "\n",
                            "                                    Protein Sequence  \\\n",
                            "0  MKEAIYQRIKDLAANSTPDQLAYLAKSLELIADKKAISNVVQMTEV...   \n",
                            "1  MHQNISKENRGNYNNGIRPRIFMITTIDFRDIQAACIKQLDDMSKD...   \n",
                            "2  MRIFYIHHPFLATHRYLLSNAYSTPYTDSITKLTTSYSSMPIILSV...   \n",
                            "3  MLTDVDIQALIDASISGLSGEMPIVANIAARNALSLTKNTQVLVLD...   \n",
                            "4  MFGVFIVRREGGYIGTQPNWDAANRPGNWDILDVYNRQRRNLWIQS...   \n",
                            "\n",
                            "                                 Nucleotide Sequence Protein ID (Clean)  \n",
                            "0  ATGAAAGAAGCAATATACCAAAGGATAAAGGATTTAGCAGCAAATA...         BAD16801_1  \n",
                            "1  GTGCATCAAAATATTTCAAAGGAGAATCGTGGAAACTATAACAACG...         BAF36105_1  \n",
                            "2  GTGAGGATTTTTTATATCCACCATCCATTCCTCGCTACTCACCGAT...         BAF36110_1  \n",
                            "3  TTGCTGACAGATGTCGATATTCAGGCATTAATTGATGCCTCAATTT...         BAF36131_1  \n",
                            "4  TTGTTCGGAGTTTTTATCGTGAGGCGTGAAGGCGGCTATATCGGAA...         BAF36132_1  \n",
                            "\n",
                            "[5 rows x 37 columns]"
                        ]
                    },
                    "execution_count": 6,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "rbp_nucleotide_seq_clean = util.add_prostt5_id_df(rbp_nucleotide_seq)\n",
                "rbp_nucleotide_seq_clean.head()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "id": "2cfa4a45",
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>s1</th>\n",
                            "      <th>s2</th>\n",
                            "      <th>s3</th>\n",
                            "      <th>s4</th>\n",
                            "      <th>s5</th>\n",
                            "      <th>s6</th>\n",
                            "      <th>s7</th>\n",
                            "      <th>s8</th>\n",
                            "      <th>s9</th>\n",
                            "      <th>s10</th>\n",
                            "      <th>...</th>\n",
                            "      <th>s1016</th>\n",
                            "      <th>s1017</th>\n",
                            "      <th>s1018</th>\n",
                            "      <th>s1019</th>\n",
                            "      <th>s1020</th>\n",
                            "      <th>s1021</th>\n",
                            "      <th>s1022</th>\n",
                            "      <th>s1023</th>\n",
                            "      <th>s1024</th>\n",
                            "      <th>Protein ID (Clean)</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>0</th>\n",
                            "      <td>-0.008888</td>\n",
                            "      <td>0.010757</td>\n",
                            "      <td>0.026901</td>\n",
                            "      <td>0.002495</td>\n",
                            "      <td>0.006329</td>\n",
                            "      <td>-0.001117</td>\n",
                            "      <td>0.018707</td>\n",
                            "      <td>-0.026306</td>\n",
                            "      <td>-0.034668</td>\n",
                            "      <td>0.010445</td>\n",
                            "      <td>...</td>\n",
                            "      <td>-0.026688</td>\n",
                            "      <td>0.008217</td>\n",
                            "      <td>0.004044</td>\n",
                            "      <td>-0.000456</td>\n",
                            "      <td>0.013741</td>\n",
                            "      <td>0.004570</td>\n",
                            "      <td>0.031921</td>\n",
                            "      <td>0.003466</td>\n",
                            "      <td>-0.022720</td>\n",
                            "      <td>AAA74324_1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>1</th>\n",
                            "      <td>0.008438</td>\n",
                            "      <td>-0.006050</td>\n",
                            "      <td>0.025681</td>\n",
                            "      <td>0.018616</td>\n",
                            "      <td>0.007584</td>\n",
                            "      <td>-0.031097</td>\n",
                            "      <td>0.020370</td>\n",
                            "      <td>-0.010178</td>\n",
                            "      <td>-0.002850</td>\n",
                            "      <td>0.018906</td>\n",
                            "      <td>...</td>\n",
                            "      <td>0.058044</td>\n",
                            "      <td>-0.033661</td>\n",
                            "      <td>-0.013657</td>\n",
                            "      <td>0.029449</td>\n",
                            "      <td>-0.039368</td>\n",
                            "      <td>-0.026581</td>\n",
                            "      <td>-0.004993</td>\n",
                            "      <td>0.021896</td>\n",
                            "      <td>0.000183</td>\n",
                            "      <td>AAA74331_1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>2</th>\n",
                            "      <td>-0.026337</td>\n",
                            "      <td>0.014549</td>\n",
                            "      <td>0.033600</td>\n",
                            "      <td>0.019562</td>\n",
                            "      <td>0.055298</td>\n",
                            "      <td>0.024307</td>\n",
                            "      <td>0.049469</td>\n",
                            "      <td>0.012596</td>\n",
                            "      <td>0.010185</td>\n",
                            "      <td>-0.002304</td>\n",
                            "      <td>...</td>\n",
                            "      <td>0.023865</td>\n",
                            "      <td>-0.026382</td>\n",
                            "      <td>-0.036011</td>\n",
                            "      <td>0.025574</td>\n",
                            "      <td>-0.017944</td>\n",
                            "      <td>0.064209</td>\n",
                            "      <td>0.006153</td>\n",
                            "      <td>0.002045</td>\n",
                            "      <td>-0.022614</td>\n",
                            "      <td>AAA98578_2</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>3</th>\n",
                            "      <td>-0.000671</td>\n",
                            "      <td>0.030960</td>\n",
                            "      <td>0.063782</td>\n",
                            "      <td>-0.007980</td>\n",
                            "      <td>0.036011</td>\n",
                            "      <td>0.007420</td>\n",
                            "      <td>0.086060</td>\n",
                            "      <td>-0.041656</td>\n",
                            "      <td>0.032410</td>\n",
                            "      <td>0.046600</td>\n",
                            "      <td>...</td>\n",
                            "      <td>0.028122</td>\n",
                            "      <td>-0.031738</td>\n",
                            "      <td>-0.007370</td>\n",
                            "      <td>-0.011604</td>\n",
                            "      <td>0.044434</td>\n",
                            "      <td>0.035614</td>\n",
                            "      <td>0.031067</td>\n",
                            "      <td>-0.033691</td>\n",
                            "      <td>0.008873</td>\n",
                            "      <td>AAB09218_1</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>4</th>\n",
                            "      <td>-0.002905</td>\n",
                            "      <td>-0.011444</td>\n",
                            "      <td>0.025513</td>\n",
                            "      <td>0.030548</td>\n",
                            "      <td>0.007416</td>\n",
                            "      <td>-0.010193</td>\n",
                            "      <td>0.056671</td>\n",
                            "      <td>-0.035767</td>\n",
                            "      <td>-0.015480</td>\n",
                            "      <td>0.026581</td>\n",
                            "      <td>...</td>\n",
                            "      <td>0.052521</td>\n",
                            "      <td>-0.021255</td>\n",
                            "      <td>0.022614</td>\n",
                            "      <td>0.012123</td>\n",
                            "      <td>-0.007881</td>\n",
                            "      <td>0.025803</td>\n",
                            "      <td>0.020477</td>\n",
                            "      <td>-0.004837</td>\n",
                            "      <td>0.003382</td>\n",
                            "      <td>AAB70057_1</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "<p>5 rows × 1025 columns</p>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "         s1        s2        s3        s4        s5        s6        s7  \\\n",
                            "0 -0.008888  0.010757  0.026901  0.002495  0.006329 -0.001117  0.018707   \n",
                            "1  0.008438 -0.006050  0.025681  0.018616  0.007584 -0.031097  0.020370   \n",
                            "2 -0.026337  0.014549  0.033600  0.019562  0.055298  0.024307  0.049469   \n",
                            "3 -0.000671  0.030960  0.063782 -0.007980  0.036011  0.007420  0.086060   \n",
                            "4 -0.002905 -0.011444  0.025513  0.030548  0.007416 -0.010193  0.056671   \n",
                            "\n",
                            "         s8        s9       s10  ...     s1016     s1017     s1018     s1019  \\\n",
                            "0 -0.026306 -0.034668  0.010445  ... -0.026688  0.008217  0.004044 -0.000456   \n",
                            "1 -0.010178 -0.002850  0.018906  ...  0.058044 -0.033661 -0.013657  0.029449   \n",
                            "2  0.012596  0.010185 -0.002304  ...  0.023865 -0.026382 -0.036011  0.025574   \n",
                            "3 -0.041656  0.032410  0.046600  ...  0.028122 -0.031738 -0.007370 -0.011604   \n",
                            "4 -0.035767 -0.015480  0.026581  ...  0.052521 -0.021255  0.022614  0.012123   \n",
                            "\n",
                            "      s1020     s1021     s1022     s1023     s1024  Protein ID (Clean)  \n",
                            "0  0.013741  0.004570  0.031921  0.003466 -0.022720          AAA74324_1  \n",
                            "1 -0.039368 -0.026581 -0.004993  0.021896  0.000183          AAA74331_1  \n",
                            "2 -0.017944  0.064209  0.006153  0.002045 -0.022614          AAA98578_2  \n",
                            "3  0.044434  0.035614  0.031067 -0.033691  0.008873          AAB09218_1  \n",
                            "4 -0.007881  0.025803  0.020477 -0.004837  0.003382          AAB70057_1  \n",
                            "\n",
                            "[5 rows x 1025 columns]"
                        ]
                    },
                    "execution_count": 7,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "rbp_prostt5_relaxed_r3_clean = pd.read_csv(\n",
                "    f\"{constants.INPHARED}/{constants.CSV_PROSTT5}_relaxed_r3.csv\"\n",
                ")\n",
                "rbp_prostt5_relaxed_r3_clean = util.sanitize_prostt5_df(\n",
                "    rbp_prostt5_relaxed_r3_clean, \"_relaxed_r3_pdb\"\n",
                ")\n",
                "rbp_prostt5_relaxed_r3_clean.head()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 8,
            "id": "abe5c3af",
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>Protein ID</th>\n",
                            "      <th>Accession</th>\n",
                            "      <th>Description</th>\n",
                            "      <th>Classification</th>\n",
                            "      <th>Genome Length (bp)</th>\n",
                            "      <th>Jumbophage</th>\n",
                            "      <th>molGC (%)</th>\n",
                            "      <th>Molecule</th>\n",
                            "      <th>Modification Date</th>\n",
                            "      <th>Number CDS</th>\n",
                            "      <th>...</th>\n",
                            "      <th>s1015</th>\n",
                            "      <th>s1016</th>\n",
                            "      <th>s1017</th>\n",
                            "      <th>s1018</th>\n",
                            "      <th>s1019</th>\n",
                            "      <th>s1020</th>\n",
                            "      <th>s1021</th>\n",
                            "      <th>s1022</th>\n",
                            "      <th>s1023</th>\n",
                            "      <th>s1024</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>0</th>\n",
                            "      <td>BAD16801.1</td>\n",
                            "      <td>AB161975</td>\n",
                            "      <td>Wolbachia phage WOcauB1</td>\n",
                            "      <td>Wolbachia phage WOcauB1 Viruses</td>\n",
                            "      <td>20484</td>\n",
                            "      <td>False</td>\n",
                            "      <td>36.878</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2016-07-26</td>\n",
                            "      <td>27</td>\n",
                            "      <td>...</td>\n",
                            "      <td>0.007633</td>\n",
                            "      <td>-0.026031</td>\n",
                            "      <td>-0.017395</td>\n",
                            "      <td>-0.059082</td>\n",
                            "      <td>-0.006268</td>\n",
                            "      <td>-0.030869</td>\n",
                            "      <td>0.028748</td>\n",
                            "      <td>-0.049500</td>\n",
                            "      <td>-0.020996</td>\n",
                            "      <td>-0.002974</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>1</th>\n",
                            "      <td>BAF36105.1</td>\n",
                            "      <td>AB231700</td>\n",
                            "      <td>Microcystis phage LMM01</td>\n",
                            "      <td>Microcystis phage LMM01 Fukuivirus LMM01 Fukui...</td>\n",
                            "      <td>162109</td>\n",
                            "      <td>False</td>\n",
                            "      <td>45.953</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2023-08-22</td>\n",
                            "      <td>189</td>\n",
                            "      <td>...</td>\n",
                            "      <td>0.011124</td>\n",
                            "      <td>-0.023529</td>\n",
                            "      <td>-0.010574</td>\n",
                            "      <td>-0.023682</td>\n",
                            "      <td>0.024490</td>\n",
                            "      <td>0.000420</td>\n",
                            "      <td>0.022034</td>\n",
                            "      <td>-0.016678</td>\n",
                            "      <td>-0.037628</td>\n",
                            "      <td>-0.002535</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>2</th>\n",
                            "      <td>BAF36110.1</td>\n",
                            "      <td>AB231700</td>\n",
                            "      <td>Microcystis phage LMM01</td>\n",
                            "      <td>Microcystis phage LMM01 Fukuivirus LMM01 Fukui...</td>\n",
                            "      <td>162109</td>\n",
                            "      <td>False</td>\n",
                            "      <td>45.953</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2023-08-22</td>\n",
                            "      <td>189</td>\n",
                            "      <td>...</td>\n",
                            "      <td>-0.041595</td>\n",
                            "      <td>0.041870</td>\n",
                            "      <td>0.035736</td>\n",
                            "      <td>-0.003040</td>\n",
                            "      <td>0.004318</td>\n",
                            "      <td>0.036255</td>\n",
                            "      <td>0.028839</td>\n",
                            "      <td>0.037933</td>\n",
                            "      <td>-0.019272</td>\n",
                            "      <td>0.002094</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>3</th>\n",
                            "      <td>BAF36131.1</td>\n",
                            "      <td>AB231700</td>\n",
                            "      <td>Microcystis phage LMM01</td>\n",
                            "      <td>Microcystis phage LMM01 Fukuivirus LMM01 Fukui...</td>\n",
                            "      <td>162109</td>\n",
                            "      <td>False</td>\n",
                            "      <td>45.953</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2023-08-22</td>\n",
                            "      <td>189</td>\n",
                            "      <td>...</td>\n",
                            "      <td>-0.022156</td>\n",
                            "      <td>0.033630</td>\n",
                            "      <td>0.011124</td>\n",
                            "      <td>0.019257</td>\n",
                            "      <td>0.006161</td>\n",
                            "      <td>0.042145</td>\n",
                            "      <td>0.021805</td>\n",
                            "      <td>0.035278</td>\n",
                            "      <td>-0.013054</td>\n",
                            "      <td>-0.016052</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>4</th>\n",
                            "      <td>BAF36132.1</td>\n",
                            "      <td>AB231700</td>\n",
                            "      <td>Microcystis phage LMM01</td>\n",
                            "      <td>Microcystis phage LMM01 Fukuivirus LMM01 Fukui...</td>\n",
                            "      <td>162109</td>\n",
                            "      <td>False</td>\n",
                            "      <td>45.953</td>\n",
                            "      <td>DNA</td>\n",
                            "      <td>2023-08-22</td>\n",
                            "      <td>189</td>\n",
                            "      <td>...</td>\n",
                            "      <td>0.012329</td>\n",
                            "      <td>0.001997</td>\n",
                            "      <td>0.012733</td>\n",
                            "      <td>0.014519</td>\n",
                            "      <td>0.004818</td>\n",
                            "      <td>0.007980</td>\n",
                            "      <td>-0.002657</td>\n",
                            "      <td>0.021210</td>\n",
                            "      <td>-0.000913</td>\n",
                            "      <td>-0.048462</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "<p>5 rows × 1060 columns</p>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "   Protein ID Accession              Description  \\\n",
                            "0  BAD16801.1  AB161975  Wolbachia phage WOcauB1   \n",
                            "1  BAF36105.1  AB231700  Microcystis phage LMM01   \n",
                            "2  BAF36110.1  AB231700  Microcystis phage LMM01   \n",
                            "3  BAF36131.1  AB231700  Microcystis phage LMM01   \n",
                            "4  BAF36132.1  AB231700  Microcystis phage LMM01   \n",
                            "\n",
                            "                                      Classification  Genome Length (bp)  \\\n",
                            "0                    Wolbachia phage WOcauB1 Viruses               20484   \n",
                            "1  Microcystis phage LMM01 Fukuivirus LMM01 Fukui...              162109   \n",
                            "2  Microcystis phage LMM01 Fukuivirus LMM01 Fukui...              162109   \n",
                            "3  Microcystis phage LMM01 Fukuivirus LMM01 Fukui...              162109   \n",
                            "4  Microcystis phage LMM01 Fukuivirus LMM01 Fukui...              162109   \n",
                            "\n",
                            "   Jumbophage  molGC (%) Molecule Modification Date  Number CDS  ...  \\\n",
                            "0       False     36.878      DNA        2016-07-26          27  ...   \n",
                            "1       False     45.953      DNA        2023-08-22         189  ...   \n",
                            "2       False     45.953      DNA        2023-08-22         189  ...   \n",
                            "3       False     45.953      DNA        2023-08-22         189  ...   \n",
                            "4       False     45.953      DNA        2023-08-22         189  ...   \n",
                            "\n",
                            "      s1015     s1016     s1017     s1018     s1019     s1020     s1021  \\\n",
                            "0  0.007633 -0.026031 -0.017395 -0.059082 -0.006268 -0.030869  0.028748   \n",
                            "1  0.011124 -0.023529 -0.010574 -0.023682  0.024490  0.000420  0.022034   \n",
                            "2 -0.041595  0.041870  0.035736 -0.003040  0.004318  0.036255  0.028839   \n",
                            "3 -0.022156  0.033630  0.011124  0.019257  0.006161  0.042145  0.021805   \n",
                            "4  0.012329  0.001997  0.012733  0.014519  0.004818  0.007980 -0.002657   \n",
                            "\n",
                            "      s1022     s1023     s1024  \n",
                            "0 -0.049500 -0.020996 -0.002974  \n",
                            "1 -0.016678 -0.037628 -0.002535  \n",
                            "2  0.037933 -0.019272  0.002094  \n",
                            "3  0.035278 -0.013054 -0.016052  \n",
                            "4  0.021210 -0.000913 -0.048462  \n",
                            "\n",
                            "[5 rows x 1060 columns]"
                        ]
                    },
                    "execution_count": 8,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "rbp_structure_embeddings_relaxed_r3 = pd.merge(\n",
                "    rbp_nucleotide_seq_clean,\n",
                "    rbp_prostt5_relaxed_r3_clean,\n",
                "    how=\"inner\",\n",
                "    validate=\"one_to_one\",\n",
                "    on=\"Protein ID (Clean)\",\n",
                ")\n",
                "del rbp_structure_embeddings_relaxed_r3[\"Protein ID (Clean)\"]\n",
                "rbp_structure_embeddings_relaxed_r3.head()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "id": "2716889e",
            "metadata": {},
            "outputs": [],
            "source": [
                "rbp_structure_embeddings_relaxed_r3.to_csv(\n",
                "    f\"{constants.INPHARED}/{constants.CONSOLIDATED}/rbp_embeddings_prostt5_relaxed_r3.csv\",\n",
                "    index=False,\n",
                ")"
            ]
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3 (ipykernel)",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.10.9"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 5
}