Pydantic v2.7+ fails to validate MultiValueEnum fields

[gbatagian]

Initial Checks

• I confirm that I'm using Pydantic V2

Description

Pydantic v2.7+ fails to accurately validate MultiValueEnum fields (from the aenum package).

MultiValueEnum fields could be used with no issues in Pydantic v2.6.4 and previous versions.

There is a related issue raised in the aenum repo, but the fix likely falls within Pydantic's scope, as the latest aenum version is compatible with Pydantic 2.6.4 but incompatible with 2.7.0 and later releases.

Example Code

# ------ under version 2.6.4 ------
import pydantic

print(pydantic.__version__) 

from aenum import MultiValueEnum
from pydantic import BaseModel

class BooleanField(MultiValueEnum):
    TRUE = "1", "True", "true", "Yes", 1
    FALSE = "0", "False", "false", "No", 0


class Sample(BaseModel):
    field_1: BooleanField
    field_2: BooleanField
    field_3: BooleanField
    field_4: BooleanField

print(Sample(**{"field_1": 1, "field_2": "true", "field_3": "Yes", "field_4": "1"}))

# Will print:
# 2.6.4
# field_1=<BooleanField.TRUE: '1'> field_2=<BooleanField.TRUE: '1'> field_3=<BooleanField.TRUE: '1'> field_4=<BooleanField.TRUE: '1'>

# ------ under version 2.7.0 ------
import pydantic

print(pydantic.__version__) 

from aenum import MultiValueEnum
from pydantic import BaseModel

class BooleanField(MultiValueEnum):
    TRUE = "1", "True", "true", "Yes", 1
    FALSE = "0", "False", "false", "No", 0


class Sample(BaseModel):
    field_1: BooleanField
    field_2: BooleanField
    field_3: BooleanField
    field_4: BooleanField

print(Sample(**{"field_1": 1, "field_2": "true", "field_3": "Yes", "field_4": "1"}))

# Will print:
# 2.7.0
# and then raise the following exception:
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[2], line 19
     16     field_3: BooleanField
     17     field_4: BooleanField
---> 19 print(Sample(**{"field_1": 1, "field_2": "true", "field_3": "Yes", "field_4": "1", "clienttype1": "individual", "clienttype2": "0"}))

File ~/.pyenv/versions/3.12.1/lib/python3.12/site-packages/pydantic/main.py:175, in BaseModel.__init__(self, **data)
    173 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    174 __tracebackhide__ = True
--> 175 self.__pydantic_validator__.validate_python(data, self_instance=self)

ValidationError: 3 validation errors for Sample
field_1
  Input should be '1' or '0' [type=enum, input_value=1, input_type=int]
    For further information visit https://errors.pydantic.dev/2.7/v/enum
field_2
  Input should be '1' or '0' [type=enum, input_value='true', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/enum
field_3
  Input should be '1' or '0' [type=enum, input_value='Yes', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/enum

Python, Pydantic & OS Version

pydantic version: 2.7.0
        pydantic-core version: 2.18.1
          pydantic-core build: profile=release pgo=true
                 install path: /home/gbatagiannis/.pyenv/versions/3.12.1/lib/python3.12/site-packages/pydantic
               python version: 3.12.1 (main, Feb  2 2024, 11:04:41) [GCC 11.4.0]
                     platform: Linux-6.5.0-35-generic-x86_64-with-glibc2.35
             related packages: typing_extensions-4.9.0 fastapi-0.109.2
                       commit: unknown

[sydney-runkle]

@gbatagian,

Yeah likely on our end, we made some major changes to enum handling in 2.7. Will look into a fix for this!

[samuelcolvin]

It'll be because before v2.7 we just called the enum with the value so MultiValueEnum took care of validating that the value was allowed.

As of 2.7 we check the value is valid in rust before returning associated the enum value. I've no idea how hard it is to get multiple input values to map to a single enum member, but we can try.

[victor-h-costa]

I also bumped into this. Is there any update or known workaround for this this issue?

[sydney-runkle]

Not at the moment, but I'll try to redo the enum validator in rust for v2.10 to support things like this. Thanks for following up!

[gbatagian]

There is sort of a workaround within the Pydantic scope by manually casting the values into a multivalue enum through a field_validator:

from aenum import MultiValueEnum
from pydantic import BaseModel, field_validator

class BooleanField(MultiValueEnum):
    TRUE = "1", "True", "true", "Yes", 1
    FALSE = "0", "False", "false", "No", 0


class Sample(BaseModel):
    field_1: BooleanField
    field_2: BooleanField
    field_3: BooleanField
    field_4: BooleanField
    
    @field_validator(
        "field_1",
        "field_2",
        "field_3",
        "field_4",
        mode="before"
    )
    @classmethod
    def convert_into_boolean_field(cls, v):
        return BooleanField(v)

print(Sample(**{"field_1": 1, "field_2": "true", "field_3": "Yes", "field_4": "1"}))

# Will print:
# field_1=<BooleanField.TRUE: '1'> field_2=<BooleanField.TRUE: '1'> field_3=<BooleanField.TRUE: '1'> field_4=<BooleanField.TRUE: '1'>

The value validation happens within the field_validator. If the input matches any of the accepted values of the MultiValueEnum, the validator returns the corresponding enum value. Otherwise, a validation error is raised:

print(Sample(**{"field_1": 2, "field_2": "true", "field_3": "Yes", "field_4": "1"}))


---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[2], line 1
----> 1 print(Sample(**{"field_1": 2, "field_2": "true", "field_3": "Yes", "field_4": "1"}))

File /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pydantic/main.py:192, in BaseModel.__init__(self, **data)
    190 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    191 __tracebackhide__ = True
--> 192 self.__pydantic_validator__.validate_python(data, self_instance=self)

ValidationError: 1 validation error for Sample
field_1
  Value error, 2 is not a valid BooleanField [type=value_error, input_value=2, input_type=int]
    For further information visit https://errors.pydantic.dev/2.8/v/value_error

This is the only method I have have found so far to work with MultiValueEnum with Pydantic versions > 2.7

[victor-h-costa]

Thanks @gbatagian . In my case I'm having trouble with using a MultiValueEnum as a FastAPI query parameter, and I can't attach a field_validator to it (FastAPI uses pydantic to serialize enums). For now I made a workaround using a string query parameter instead. Then I manually transform the string into the enum like this BooleanField("Yes")

[gbatagian]

@victor-h-costa One way to use a Pydantic model with MultiValueEnum fields as schema for query parameters on a url under the current MultiValueEnum validation issue on Pydantic v2.7+, is by utilising the FastAPI dependency pattern and a factory method, e.g. something like this:

from aenum import MultiValueEnum
from fastapi import Depends, FastAPI, Query, Request
from fastapi.responses import JSONResponse
from pydantic import BaseModel


class BooleanField(MultiValueEnum):
    TRUE = "1", "True", "true", "Yes", 1
    FALSE = "0", "False", "false", "No", 0


class Params(BaseModel):
    field_1: BooleanField | None = None

    @classmethod
    def from_request(cls, field_1: str | None = Query(None)):
        return cls(
            field_1=BooleanField(field_1) if field_1 is not None else None
        )


app = FastAPI()


@app.exception_handler(ValueError)
async def valuer_error_exception_handler(
    request: Request, exc: Exception
) -> JSONResponse:
    return JSONResponse(status_code=422, content={"message": f"{exc}"})


@app.get("/sample")
async def sample(q: Params = Depends(Params.from_request)) -> Params:
    return q

>> fastapi dev server.py

>> curl -X 'GET' \
  'http://127.0.0.1:8000/sample' \
  -H 'accept: application/json'
{"field_1":null}%    

>> curl -X 'GET' \
  'http://127.0.0.1:8000/sample?field_1=1' \
  -H 'accept: application/json'
{"field_1":"1"}%     

>> curl -X 'GET' \
  'http://127.0.0.1:8000/sample?field_1=Yes' \
  -H 'accept: application/json'
{"field_1":"1"}% 

>> curl -X 'GET' \
  'http://127.0.0.1:8000/sample?field_1=2' \
  -H 'accept: application/json'
{"message":"'2' is not a valid BooleanField"}%