-
Notifications
You must be signed in to change notification settings - Fork 829
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Timestamps round trip incorrectly on non UTC systems #1195
Comments
Setting the location to UTC on the timestamp does work, so this is not really a function of the system timezone but the way timestamps are serialized. |
Is the database column a |
It's timestamp. I can make a small reproduction sometime: been a bit busy lately. |
The same problem here (from ent/ent#2692): The column created in Postgres is: used_at timestamp with time zone NOT NULL, The value in Postgres is saved without timezone:
I'm using PG in Docker and using the PG query: show timezone I get:
I get from Ent (using pgx stdlib) the value:
This is a mess on the clients (where an UTC value is expected). Using pgdriver/pq I get the UTC value from DB. Am I wrong somewhere? |
I tried using this connection with this code too:
import (
"database/sql"
_ "github.com/jackc/pgx/v4/stdlib"
)
conn, err := sql.Open("pgx", "postgres://postgres:postgres@localhost/project?sslmode=disable&timezone=UTC")
//handle err The problem is here. I need a way to get back from DB the UTC values (that are laready stored in the DB). |
@jackc can you please help us here? 🙏 |
I asked on SO too: https://stackoverflow.com/questions/72771272/how-to-setup-pgx-to-get-utc-values-from-db. They suggest a custom type. More code, more issues! @wbl did you find a way? |
Okay, I see what's going on here. The SO thread basically has the answer, but here's a little more of the explanation of why.
That leaves us with the choice of using UTC or using the |
So Go time representation stores the timezone explicitly. Code that is working is setting the location to UTC on retrieval and adjusting the offsets on insertion. If the returned times are set to the UTC location, it won't break code that is working. If we don't want to fix it via what would technically be a breaking change, adding documentation could help. |
@jackc I'm asking you an OPTIONAL config to return UTC as stored in DB. Please. No breaking. |
It already is optional. Just not with a config change. Create a new type that has |
AFAICT a custom type works in all situations except when using the Lines 717 to 726 in d42b399
|
@jackc what is the minimal approach to implement the solution above in pgx v5? I would like to wrap existing types as much as possible and just take the scan result time and convert it to UTC. I thought the following could be a path type timestamptzCodec struct {
pgtype.TimestamptzCodec
}
func (t *timestamptzCodec) PlanScan(m *pgtype.Map, oid uint32, format int16, target any) pgtype.ScanPlan {
plan := t.TimestamptzCodec.PlanScan(m, oid, format, target)
if plan == nil {
return nil
}
return ×tamptzPlan{plan}
}
type timestamptzPlan struct {
pgtype.ScanPlan
}
func (t *timestamptzPlan) Scan(src []byte, target any) error {
err := t.ScanPlan.Scan(src, target)
fmt.Printf("Target info %v %T\n", target, target)
return err
} However Is there a simpler solution? |
I think what you would want to do is to create your own wrapper type that implements Then your choice is whether to have a custom |
Just ran into this issue. Currently if you define a timestampz field then marshal and unmarshal as normal using pgx you will get incorrect data in memory (but correct in db) unless your local timezone happens to match the timezone of the entry. Setting the TZ environment variable does not seem to change this. |
Ran into this issue too. I would have expected the driver to convert time.Time values to UTC. I don't care about storing time zone, just want the same timestamp returned (e.g., Unix() is equivalent for what I stored and retrieved). |
Hey there :) I am having the same issue.. Whats weird is that, when I use sql.Open or sqlx.Connect, everything works fine. Any idea? Thanks |
I seem to be running into this issue with a query that looks like this, where
When I pass in |
@dedalusj Did you end up getting this working? I'm sure a working example could help out a lot of folks. |
Ran into this as well with |
If ScanLocation is set, it will be used to convert the time to the given location when scanning from the database. The Codec interface is now implemented by *pgtype.TimestamptzCodec instead of pgtype.TimestamptzCodec. This is technically a breaking change, but it is extremely unlikely that anyone is depending on this, and if there is downstream breakage it is trivial to fix. #1195 #1945
If ScanLocation is set, the timestamps will be assumed to be in the given location when scanning from the database. The Codec interface is now implemented by *pgtype.TimestampCodec instead of pgtype.TimestampCodec. This is technically a breaking change, but it is extremely unlikely that anyone is depending on this, and if there is downstream breakage it is trivial to fix. #1195 #1945
This issue was originally filed with pgx v4. I don't see making any changes there. But think that PR #1948 would solve all the issues raised here for pgx v5. It allows specifying the desired conn.TypeMap().RegisterType(&pgtype.Type{
Name: "timestamptz",
OID: pgtype.TimestamptzOID,
Codec: &pgtype.TimestamptzCodec{ScanLocation: time.UTC},
}) Please review and give feedback. |
If ScanLocation is set, it will be used to convert the time to the given location when scanning from the database. The Codec interface is now implemented by *pgtype.TimestamptzCodec instead of pgtype.TimestamptzCodec. This is technically a breaking change, but it is extremely unlikely that anyone is depending on this, and if there is downstream breakage it is trivial to fix. #1195 #1945
If ScanLocation is set, the timestamps will be assumed to be in the given location when scanning from the database. The Codec interface is now implemented by *pgtype.TimestampCodec instead of pgtype.TimestampCodec. This is technically a breaking change, but it is extremely unlikely that anyone is depending on this, and if there is downstream breakage it is trivial to fix. #1195 #1945
#1948 has been merged. This should resolve this issue. |
When I am on a system not set to UTC, and store a timestamp to a database via the database/sql compatible interface, the value in the database is stored with the same numeric value but the zone set to UTC. When reading back the timestamp stored in the DB is correctly reconstituted, but alas this is not the same as the value that went in.
The text was updated successfully, but these errors were encountered: