fix: Problem in the description field when using the MariaDB or MySQL

[MarceloNunesAlves]

This pull request addresses an issue where the 'description' fields, defined as str in the SQLModel model, were being created as varchar(255) columns in MariaDB and MySQL databases. This behavior was inconsistent with the expected text data type for these fields.

Changes:

• Modified the data type of the 'description' fields in the SQLModel model to sqlalchemy.Text. This ensures that the corresponding columns in MariaDB and MySQL are created as text columns, allowing for larger text values.
• Note: PostgreSQL is not affected by this change as it does not require a specific size for text columns.

Reasoning:

• The varchar(255) data type imposes a character limit, which may not be sufficient for storing longer descriptions.
• Using the text data type provides more flexibility for storing variable-length text data without size constraints.

Below is the print with the error generated:

I think it is related to this Issuer that was closed: #1890

log_error_mysql_maria.log

[github-actions]

Pull Request Validation Report

This comment is automatically generated by Conventional PR

Whitelist Report

Whitelist	Active	Result
Pull request is a draft and should be ignored	✅	❌
Pull request is made by a whitelisted user and should be ignored	❌	❌
Pull request is submitted by a bot and should be ignored	✅	❌
Pull request is submitted by administrators and should be ignored	❌	❌

Result

Pull request does not satisfy any enabled whitelist criteria. Pull request will be validated.

Validation Report

Validation	Active	Result
All commits in this pull request has valid messages	❌	✅
Pull request does not introduce too many changes	❌	✅
Pull request has a valid title	✅	✅
Pull request has mentioned issues	❌	✅
Pull request has valid branch name	❌	✅
Pull request should have a non-empty body	✅	✅

Result

Pull request satisfies all enabled pull request rules.

_{Last Modified at 19 Aug 24 18:07 UTC}

[aws-amplify-sa-east-1]

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-3431.dmtpw4p5recq1.amplifyapp.com

[ogabrielluiz]

Hey @MarceloNunesAlves

Thanks for this PR. We just need to add the migration script to it.

The steps should be:
cd src/backend/base/langflow && poetry install
then:
poetry run alembic revision --autogenerate -m "Update description columns type"
then, in the newly created file, paste the following code:

"""Update description columns type

Revision ID: 4522eb831f5c
Revises: 0d60fcbd4e8e
Create Date: 2024-08-20 11:15:58.282318

"""

from typing import Sequence, Union

import sqlalchemy as sa
from alembic import op
from langflow.utils import migration
from sqlalchemy.engine.reflection import Inspector

# revision identifiers, used by Alembic.
revision: str = "4522eb831f5c"
down_revision: Union[str, None] = "0d60fcbd4e8e"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None


def upgrade() -> None:
    conn = op.get_bind()
    # ### commands auto generated by Alembic - please adjust! ###
    inspector = Inspector.from_engine(conn)  # type: ignore

    with op.batch_alter_table("flow", schema=None) as batch_op:
        if migration.column_exists(table_name="flow", column_name="description", conn=conn):
            columns = inspector.get_columns("flow")
            description_column = next((column for column in columns if column["name"] == "description"), None)
            if description_column is not None and isinstance(description_column["type"], sa.VARCHAR):
                batch_op.alter_column(
                    "description", existing_type=sa.VARCHAR(), type_=sa.Text(), existing_nullable=True
                )

    with op.batch_alter_table("folder", schema=None) as batch_op:
        if migration.column_exists(table_name="flow", column_name="description", conn=conn):
            columns = inspector.get_columns("folder")
            description_column = next((column for column in columns if column["name"] == "description"), None)
            if description_column is not None and isinstance(description_column["type"], sa.VARCHAR):
                batch_op.alter_column(
                    "description", existing_type=sa.VARCHAR(), type_=sa.Text(), existing_nullable=True
                )

    # ### end Alembic commands ###


def downgrade() -> None:
    conn = op.get_bind()
    # ### commands auto generated by Alembic - please adjust! ###
    inspector = Inspector.from_engine(conn)  # type: ignore
    with op.batch_alter_table("folder", schema=None) as batch_op:
        if migration.column_exists(table_name="folder", column_name="description", conn=conn):
            columns = inspector.get_columns("folder")
            description_column = next((column for column in columns if column["name"] == "description"), None)
            if description_column is not None and isinstance(description_column["type"], sa.VARCHAR):
                batch_op.alter_column(
                    "description", existing_type=sa.VARCHAR(), type_=sa.Text(), existing_nullable=True
                )

    with op.batch_alter_table("flow", schema=None) as batch_op:
        if migration.column_exists(table_name="flow", column_name="description", conn=conn):
            columns = inspector.get_columns("flow")
            description_column = next((column for column in columns if column["name"] == "description"), None)
            if description_column is not None and isinstance(description_column["type"], sa.VARCHAR):
                batch_op.alter_column(
                    "description", existing_type=sa.VARCHAR(), type_=sa.Text(), existing_nullable=True
                )
    # ### end Alembic commands ###

Make sure everything is working and commit ;)

[ogabrielluiz]

LGTM

we'll add the migrations

[MarceloNunesAlves]

@ogabrielluiz , I think everything is okay now.

[ogabrielluiz]

LGTM

[MarceloNunesAlves]

@ogabrielluiz , Do you know when this is going to be merged?

[xzqxnet0990]

@ogabrielluiz ogabrielluiz
I meet another problem on mysql, I think the script should also be changed at the beginning script "260dbcc8b680".

`024-09-04 16:27:44,552 - migration.py[line:218] - INFO: Will assume non-transactional DDL.
2024-09-04 16:27:44,564 - migration.py[line:623] - INFO: Running upgrade -> 260dbcc8b680, Adds tables
[09/04/24 16:27:44] ERROR 2024-09-04 16:27:44 - ERROR

• utils utils.py:127
  - (pymysql.err.OperationalError)
  (1170, "BLOB/TEXT column 'description'
  used in key specification without a
  key length")
  [SQL: CREATE INDEX ix_flow_description
  ON flow (description)]
  (Background on this error at:
  https://sqlalche.me/e/20/e3q8)
  ERROR 2024-09-04 16:27:44 - ERROR - api_server.py:129
  api_server -
  (pymysql.err.OperationalError)
  (1170, "BLOB/TEXT column
  'description' used in key
  specification without a key
  length")
  [SQL: CREATE INDEX
  ix_flow_description ON flow
  (description)]
  (Background on this error at:
  https://sqlalche.me/e/20/e3q8) `

[xzqxnet0990]

@ogabrielluiz ogabrielluiz I meet another problem on mysql, I think the script should also be changed at the beginning script "260dbcc8b680".

`024-09-04 16:27:44,552 - migration.py[line:218] - INFO: Will assume non-transactional DDL. 2024-09-04 16:27:44,564 - migration.py[line:623] - INFO: Running upgrade -> 260dbcc8b680, Adds tables [09/04/24 16:27:44] ERROR 2024-09-04 16:27:44 - ERROR

• utils utils.py:127
  • (pymysql.err.OperationalError)
    (1170, "BLOB/TEXT column 'description'
    used in key specification without a
    key length")
    [SQL: CREATE INDEX ix_flow_description
    ON flow (description)]
    (Background on this error at:
    https://sqlalche.me/e/20/e3q8)
    ERROR 2024-09-04 16:27:44 - ERROR - api_server.py:129
    api_server -
    (pymysql.err.OperationalError)
    (1170, "BLOB/TEXT column
    'description' used in key
    specification without a key
    length")
    [SQL: CREATE INDEX
    ix_flow_description ON flow
    (description)]
    (Background on this error at:
    https://sqlalche.me/e/20/e3q8) `

And I fix the code, I think it works.

260dbcc8b680_adds_tables.py line 105
with op.batch_alter_table("flow", schema=None) as batch_op:
    flow_columns = [col["name"] for col in inspector.get_columns("flow")]
    if "user_id" not in flow_columns:
        batch_op.add_column(
            sa.Column(
                "user_id",
                sqlmodel.sql.sqltypes.GUID(),
                nullable=True,  # This should be False, but we need to allow NULL values for now
            )
        )
    if "user.id" not in existing_fks_flow:
        batch_op.create_foreign_key("fk_flow_user_id", "user", ["user_id"], ["id"])
    if "ix_flow_description" not in existing_indices_flow:
        # Corrected way to specify key length for the description column index
        batch_op.create_index(batch_op.f("ix_flow_description"), [sa.text("description(255)")], unique=False)
    if "ix_flow_name" not in existing_indices_flow:
        batch_op.create_index(batch_op.f("ix_flow_name"), ["name"], unique=False)

[xzqxnet0990]

@ogabrielluiz ogabrielluiz I meet another problem on mysql, I think the script should also be changed at the beginning script "260dbcc8b680".
`024-09-04 16:27:44,552 - migration.py[line:218] - INFO: Will assume non-transactional DDL. 2024-09-04 16:27:44,564 - migration.py[line:623] - INFO: Running upgrade -> 260dbcc8b680, Adds tables [09/04/24 16:27:44] ERROR 2024-09-04 16:27:44 - ERROR

• utils utils.py:127

  • (pymysql.err.OperationalError)
    (1170, "BLOB/TEXT column 'description'
    used in key specification without a
    key length")
    [SQL: CREATE INDEX ix_flow_description
    ON flow (description)]
    (Background on this error at:
    https://sqlalche.me/e/20/e3q8)
    ERROR 2024-09-04 16:27:44 - ERROR - api_server.py:129
    api_server -
    (pymysql.err.OperationalError)
    (1170, "BLOB/TEXT column
    'description' used in key
    specification without a key
    length")
    [SQL: CREATE INDEX
    ix_flow_description ON flow
    (description)]
    (Background on this error at:
    https://sqlalche.me/e/20/e3q8) `

And I fix the code, I think it works.

260dbcc8b680_adds_tables.py line 105
with op.batch_alter_table("flow", schema=None) as batch_op:
    flow_columns = [col["name"] for col in inspector.get_columns("flow")]
    if "user_id" not in flow_columns:
        batch_op.add_column(
            sa.Column(
                "user_id",
                sqlmodel.sql.sqltypes.GUID(),
                nullable=True,  # This should be False, but we need to allow NULL values for now
            )
        )
    if "user.id" not in existing_fks_flow:
        batch_op.create_foreign_key("fk_flow_user_id", "user", ["user_id"], ["id"])
    if "ix_flow_description" not in existing_indices_flow:
        # Corrected way to specify key length for the description column index
        batch_op.create_index(batch_op.f("ix_flow_description"), [sa.text("description(255)")], unique=False)
    if "ix_flow_name" not in existing_indices_flow:
        batch_op.create_index(batch_op.f("ix_flow_name"), ["name"], unique=False)

I fix it in #3678

[sa411022]

The columns listed below may also need to be changed from VARCHAR(255) to TEXT.

• message.text
• variable.value
• vertex_build.params

[MarceloNunesAlves]

Hi @sa411022, how are you? Do you push a PR about this?