In gomods#1124, a GCP lock type was added as a singleflight backend. As
part of this work, the GCP backend's Save() was made serial, likely
because moduploader.Upload requires a call to Exists() before it,
rendering the GCP lock less useful, by doubling the calls to GCS.

However, by doing this, the existence check was now only checking
the existence of the mod file, and not the info or zip. This meant
that if during a Save, the zip or info uploads failed, on subsequent
rquests, that when using the GCP singleflight backend, Athens would
assume everything had been stashed and saved properly, and then
fail to serve up the info or zip that had failed upload, meaning
the cache was in an unhealable broklen state, requiring a manual
intervention.

To fix this, without breaking the singleflight behavior, introduce
a metadata key that is set on the mod file during its initial upload,
indicating that a Stash is still in progress on subsequent files,
which gets removed once all three files are uploaded successfully,
which can be checked if it it is determined that the mod file
already exists. That way we can return a errors.KindAlreadyExists
if a Stash is in progress, but also properly return it when a Stash
is *not* currently in progress if and only if all three files exist
on GCS, which prevents the cache from becoming permanently poisoned.

One note is that it is possible the GCS call to remove the metadata
key fails, which would mean it is left on the mod object forever.
To avoid this, consider it stale after 2 minutes.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>

This is the proper way to make sure we do not write a partial file.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>

This is useful if, for example, the user is on a slow network.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>