Add an sql wrapper around passed value in an add_row function

[nextstopsun]

Last addition to a flex backend adds an sql_type, which is quite handy.
Another great feature could be addition of sql wrapper around passed column value in add_row function.
I guess all column values are being passed to a generated sql COPY command.
This process is unreachable for lua script. Can it be exposed?
For example something like this

function osm2pgsql.process_node(object)
    tables.mytable:add_row({
        tags = 'jsonb_strip_nulls(' .. object.tags .. ')',
        geom = { create = "point" }
    })
end

Concatenation is used just to give an idea, real implementation could of course be cleaner.

[joto]

As you just found out (see #1522) exposing database features to the osm2pgsql user is difficult to handle and error prone. There are just too many thing that can go wrong and they restrict what kind of changes we can later do in osm2pgsql. We therefore try to avoid this as much as possible.

In this case we can't even do what you want (if I understand you correctly), because of the way the COPY command works. Unlike with the INSERT command, you can only send data through a COPY, not call functions and the like.

Any kind of processing you want to do in the database is probably best done with database methods such as triggers and views. PostgreSQL supports many clever tricks and workarounds and they are, admittedly often difficult to use, but its only a small minority who needs these things. It would probably be interesting to start collecting these tricks and documenting them, though.

[nextstopsun]

You're right in saying things get complex and buggy when you add such features, but osm2pgsql users shouldn't be afraid of postgres exceptions.
There definitely are some workarounds, ie that case with "serial" could be solved with generated always column before your fix.

Triggers and views help a lot, and imo there should be a way to not overwrite existing tables in create mode. All temporary and final tables can be generated beforehand, so it's just a matter of one flag in the code and willpower:) This can be marked with "experimental" and "use with caution", so when postgres throws errors in this case - just let it be.

Regarding COPY: afaik it can use a SELECT or VALUES statement as a data source, can this be of any help?

There are some discussions here about allowing plain sql queries in lua script, this also can be handy. Imagine you allow user to transform records with sql right before copying imported data from temporary to final table. Such a feature reduces the need for geometry functions in lua scripts.

[nextstopsun]

The SELECT part of this INSERT statement is a good place for data processing with sql.

Is it possible to let build_sql_column_list function result be overridden from lua side?