Proposal: Error Adapter

[yordis]

Purpose

Errors are everywhere. However, error's producers tend to describe errors
differently.

The lack of a common way of describing errors means developers are continually
re-learning how to consume errors. That also limits the potential for libraries,
tooling, and infrastructure to aid of error data across delivery environments,
like SDK or logger systems. The portability and productivity that can be
achieved from error data are hindered overall.

This ADR is a specification for describing error data in standard formats to
provide interoperability across services, platforms, and systems.

[hiebj]

In practice, I am not sure if this will really work out. My team uses CloudEvents for error reporting as well, both as domain events in our pub/sub systems and also when reporting errors to API consumers.

We almost made a proposal very similar to this, but ultimately decided against it. I'll try to outline the reasons.

Ultimately, it really boils down to one problem: domain errors just don't have enough structural overlap - particularly, structural overlap that can't be handled by the existing CloudEvents spec and proper use of existing schemas.

prior art

This has actually been tried before, for HTTP requests specifically: see Problem JSON and JSON:API errors. However, those solutions are not suitable for all environments; they are too reliant on JSON and HTTP to be effective for an event-driven, cross-platform error scheme.

Even when compared to the above - probably the closest anyone has ever gotten to a universal error schema - there are essentially only two fields not explicitly covered by CloudEvents: a human-readable message and a detail link. The detail link can be covered with creative use of the existing attributes, however, such as type or subject, and in many cases it is sufficient to put a human-readable message directly in data with datacontenttype set to text/plain.

Other domain-specific error data must be provided as either an additional data object (as you have here) or additional attributes (the way Problem JSON handles it).

how to provide schema for domain-specific error data?

Lots of errors have highly domain-specific data that needs to be passed along, and that data is often crucial to the handling of the error - not simply "metadata" that can be dumped into a log for human debugging purposes. It is not sufficient to leave that data untyped - middleware and other abstract event processors need to be able to figure out what kind of data is there if you want to have any kind of generic handling, such as a common way of dealing with data rollback.

The problem is: how do you provide a schema definition for those properties? There really is no way to provide a schema definition for extra data attributes if the dataschema is fixed. With a metadata object, you could certainly provide an additional metadataschema attribute that defines an additional schema for the metadata - but to what avail? Why not simply provide a dataschema that captures your domain-specific error concerns?

suggestions

Given the above, I think the best thing that can be done to improve the CloudEvents spec in this regard is to create specific adapters for particular existing error structures, such as Problem JSON and JSON:API errors. In those cases though, I think it will probably turn out to be best to leave the data attribute alone, use the other attributes creatively, and allow people to write domain-specific schemas for their error data.

Beyond that, I think what the industry really needs is a collaborative effort on a broader mission of creating standardized error schemas for the myriad categories of domain errors that occur. Then we can simply attach those schemas to CloudEvents and be done with this once and for all.

[yordis]

particularly, a structural overlap that can't be handled by the existing CloudEvents spec and proper use of existing schemas.

Which schemas?

I agree with you that CloudEvents has most of the data that we would need. That is why I re-wrote my ADR (which is how the conversation started) to take advantage of CloudEvents's structure and add more alignment around the event's data.

they are too reliant on JSON and HTTP to be effective for an event-driven, cross-platform error scheme.

Would you mind expanding on that?

In many cases, it is sufficient to put a human-readable message directly in data with datacontenttype set to text/plain.

Totally,

Just that at that point, you lost the ability to add some data that would allow you to do i18n (if you want to send the event out to some clients that may care about that) or do i18n all together, which many people opt-in to do so, you can still use the errormessage if you would like to ignore i18n.

If that is your case, I think it is totally valid, but personally, I much rather bring more alignment. In case you don't care, you can ignore the field. It is not mandatory.

The problem is: how do you provide a schema definition for those properties?

That is why you have other fields that allow you to identify the error itself, such as the type field. type field will allow you to understand better the context of the "unstructured" metadata value. If you want a centralized way to deal with this, you will end up with a centralized mapping that understands all the contexts 🤷🏻 I am not sure how to avoid.

I think it will probably turn out to be best to leave the data attribute alone

We can't if we want to have some alignment and consider multiple concerns like i18n or helping programmers.

Which we could remove errormessage and errorlink for sure, leaving metadata (it could be just data) behind it.

Still, from my experience, there is always a balance between "what is technically correct" (whatever that means) and dealing with the fact that they are always a trade-off.
I much rather have alignment around giving programmers some useful information. In the past, we tried to be technically correct and brought resentment (for valid reasons) towards the decisions because it didn't take into consideration their needs as programmers.

use the other attributes creatively

That is where I strongly disagree. That is where misguided decisions are made since you rely on people's experiences and ability to "creative" (take my proposal as an example)

Would you mind helping me to identify how I accomplish the intention without following the proposal? What would be an alternative?

[hiebj]

You said you rewrote your ADR - out of curiosity, what was your original proposal?

Which schemas?

Would you mind helping me to identify how I accomplish the intention without following the proposal? What would be an alternative?

I think I'd love to see an open-source project called ErrorSchemas or something that does work similar to what CloudEvents has done here - gather together a huge collection of existing error shapes from various libraries and applications, group them into categories, and try to align on a limited number of error shapes - or perhaps a single error shape - that eliminates the need for an arbitrary untyped "metadata" object, either by giving us a single solution that works in every case, or else a set of schemas to choose from for different types of error cases.

So far every attempt to generalize on errors has basically gone the route of "it's a message and a link, and then a bucket of who-knows-what" and one of my favorite things about CloudEvents is the capability to use schema to define the shape of your payload.

I realize that's not a workable suggestion in any sort of near term. Perhaps instead I'll share what we were thinking of proposing.

My first instinct was actually to open a PR suggesting that we actually create a humanreadable extension attribute, or perhaps more simply, message. The type or subject field could contain a detail link (this is what I meant by "use the existing attributes creatively", though it's not really a stretch from the intended use) and data would contain your metadata, typed by the given dataschema.

Originally we had thought to name the attribute error which would allow you to use it for duck-typing, but in all of the use cases we could think of, it was sufficient to just pivot on the type to discern errors, the same as you do for any other event. And humanreadable or message is more flexible - the idea was, there are many types of events that might benefit from having a human-readable message attached, not just errors; and as an extension attribute, it wouldn't be too offensive to folks not interested in using it.

We elected not to suggest that after reading carefully the documentation, particularly this:

Extension attributes to the CloudEvent specification are meant to be additional metadata that needs to be included to help ensure proper routing and processing of the CloudEvent. Additional metadata for other purposes, that is related to the event itself and not needed in the transportation or processing of the CloudEvent, should instead be placed within the proper extensibility points of the event (data) itself.

It's hard to make the argument that a human-readable message falls under routing and processing concerns. It seems like it belongs in data, where you have placed errormessage and errorlink. However, given your comment here:

I much rather have alignment around giving programmers some useful information. In the past, we tried to be technically correct and brought resentment (for valid reasons) towards the decisions because it didn't take into consideration their needs as programmers.

It seems to me that adding an extension attribute humanreadable or message to the event body, then allowing your metadata be schema-backed in the data attribute would be a pretty ideal developer experience. It's not "technically correct" pursuant to the comments in the spec about extension attributes being meant for routing... but it seems so useful that I'm inclined to think it might be forgiven.

What do you think?

[hiebj]

Nevermind to my question above about your ADR, I found the PR against your ADR repository since you tagged this PR in a comment there.

Seems you are interested in essentially the same problem I am. I would be interested in pursuing some comprehensive research into existing error patterns as I mentioned above. I don't think the resulting work necessarily belongs in the CloudEvents spec, but I do think the projects would be complimentary - essentially providing a set of schemas that are used in CloudEvent data - particularly errors, since they can often be generalized to a significant extent across domains.

[cneijenhuis]

Can you define error?

An error could be of technical nature - something that a developer needs to have a look at and fix. Given that a human has to have a look, interoperability and portability don't strike me as the most important goals.

An error could also be a state in the domain. E.g. if you're trying to charge a credit card, your payment provider may return either a charge.succeeded or charge.failed event. Surely, the latter case is an error - we prefer the charge to be successful. However, the rejection is to be expected as well. Our code has to handle both cases.

If we encounter an event/case our code doesn't handle yet, we need to figure out how to handle the event - read the documentation, look at the schema etc. However, that is independent of whether the event represents a success or an error in our domain. I am not sure why we should treat these differently.
E.g. i18n - you need to solve it for both cases. If you want to inform the user the charge was successful and you'll e.g. send a package, the mechanism is the same as when informing the user the charge failed they'll e.g. need to provide a different payment method or cancel their order. Adding a textual description into the event may or may not be a good idea, but that doesn't depend on whether the event is an error state in your domain.

[yordis]

hat eliminates the need for an arbitrary untyped "metadata" object, either by giving us a single solution that works in every case, or else a set of schemas to choose from for different types of error cases.

Personally, I need to update my ADR to add even more constraints where the metadata is a Map<string, string>, so you can't have nested values, and everything is a string.

So far every attempt to generalize on errors has basically gone the route of "it's a message and a link, and then a bucket of who-knows-what" and one of my favorite things about CloudEvents is the capability to use schema to define the shape of your payload.

Right, agree, that is why I am leveraging the spec to handle errors.

suggesting that we actually create a humanreadable extension attribute,

The original spec was using extensions, I proposed the last meeting and some people suggest that we don't need to use extensions when we could use an adapter instead, and add the schema of the data.

[yordis]

interoperability and portability don't strike me as the most important goals.

We do 🤷🏻

I would like to hear more precise feedback since your comment is amazing, and I would like to remove the subjectivity as much as possible. Proposing some alternative would be amazing.

[hiebj]

An error could also be a state in the domain. E.g. if you're trying to charge a credit card, your payment provider may return either a charge.succeeded or charge.failed event. Surely, the latter case is an error - we prefer the charge to be successful. However, the rejection is to be expected as well. Our code has to handle both cases.

Exactly. There are basically two different types of errors: unexpected errors that the user can't do anything about, and errors that are actually expected domain results.

In the case of the former, there is really nothing useful to show to the user other than "whoops, something went wrong, try again later or call support with event ID xyz-1234". And IMO, that message does not actually need to be provided by the backend at all. If something unexpected, and therefore unrecoverable has happened, there is little value in even sending metadata about the error to the client. If you handle internationalization, you will have content in your UI that must be translated. This type of message is just more static UI content.

In the case of the latter, we're not really talking about an "error". An error in this case is, in fact, a domain result - indicating that the user's request couldn't be executed. When I said that we use CloudEvents to represent errors in pub/sub, this is what I meant - domain-specific, expected events (which nevertheless may be consumed by parties who would benefit from having a schema attached).

This is what I meant by "there is not sufficient structural overlap in error types". Errors that are actually domain results are going to have domain specific concerns.

If we're talking about messages sent to a client app, and your domain chooses to provide the user with a link to an error page, that's one approach; if your domain instead encodes a list of steps the user needs to take, such as a list of missing fields they need to fill out before their user profile is "complete", that's another approach. The UI has to know what that shape is and it is likely to be context-specific. If your API is consumed by third parties, it is useful to have a well-defined schema, and send it along with your message.

If we're talking about domain events being published to pub/sub, you're going to need to provide some information that allows consuming services to handle the event somehow... and that information should have a schema.

There are relatively few cases, notably in the case of developer tools, where you actually want to take "a bucket of who-knows-what" in the form of untyped metadata and just show it to the user. And if you are building a developer tool, "a bucket of who-knows-what" is domain specific!

So in the end, what we are left with is:

• an unexpected, unrecoverable error in which case providing additional information has questionable value; or
• an expected domain result, in which case the schema is domain-specific and local to your project or team

It's possible that sending errormessage and errorlink and metadata is an effective solution for all of the domain concerns encountered by your team. It's not really generalizable to "all errors, everywhere", though.

Really the only thing about errors that can be generalized is that there is a human-readable string involved - but even that is domain-specific, because some domains want to translate it, and some domains don't want to provide a message at all because the front-end decides what to show the user based on the encoded type. I'm not sure about you, but most "human readable" error messages I get from third party APIs are not something I actually want to show the user.

This is why we didn't go through with suggesting the humanreadable extension. Our domain failure shapes are specific to our domain and often don't even include a human-readable string. Many of our events do; in those cases, it's encoded as part of the domain-specific dataschema.

Proposing some alternative would be amazing.

I think the alternative is that we don't solve this problem with changes to the CloudEvents spec. Trying to create a schema to solve "errors" is out of scope for this spec.

Users of CloudEvents need to define whatever schemas are appropriate for the various domain events in their contexts. This is as true for failure events as it is for success events.

The error schema in your PR is specific to your domain and belongs in your application or a team repository. A Map of strings is not sufficient for all domain use cases, and the "general" solution is what already exists in CloudEvents - a data attribute that has an associated dataschema.

The goal of CloudEvents isn't to come up with a unified set of domain failure schemas. It's to come up with an envelope for portable and interoperable events, that is, to define a consistent envelope shape for inter-service communication (events) so that services utilizing pub/sub can be deployed to multiple platforms (portability), and so that systems that span platforms don't have to constantly be mapping between Azure/Google/AWS/IBM event shapes (interoperability).

If you want to tackle the problem of portability and interoperability of errors across domains, I think a lot more work needs to be done - specifically, analyzing existing errors across lots of systems and coming up with categories and schemas for various well-known and oft-used domain errors. The resulting schemas could be shared by the open source community and used as payloads in CloudEvents. But they are complimentary solutions - not two parts of the same solution.

[yordis]

I think the alternative is that we don't solve this problem with changes to the CloudEvents spec. Trying to create a schema to solve "errors" is out of scope for this spec.

Which CloudEvents spec? This is not part of CloudEvents spec directly but an adapter.

Does that mean CloudEvents shouldn't tell you how to define GitHub events https://github.com/cloudevents/spec/blob/v1.0/adapters/github.md#github-cloudevents-adapter or any other events for that matter https://github.com/cloudevents/spec/tree/v1.0/adapters?

I think it is important to understand the differences between CloudEvents Spec and Adapters.

analyzing existing errors across lots of systems and coming up with categories and schemas for various well-known and oft-used domain errors.

Hint where we are right now. It is hard for me to disagree with you, but for the wrong reasons.

Most of your concerns have been thought out before, I am not sure if you understand that, none of this comes from a misunderstanding of the situation.

Although what you are saying is completely valid, when it comes to trying to tackle the situation, most of your concerns fall into such schema decision (hint, as an adapter, that is not part of the spec), or at least my understanding from what you are saying suggest that. The problem is that you focusing on things that are related to errors, but no part of what this adapter is trying to tackle (Read Purpose section for more information about it).

I would like to hear more clear examples (potentially sharing some concrete examples based on real cases) where the Error Adapter would become a problem, and wouldn't work for you.

[hiebj]

You're right. As an adapter, this wouldn't have an effect on me unless I choose to use it. Even as an optional extension it would be opt-in. However, I think you know what I'm getting at here. You are proposing Error Adapter, not Yordis Error Adapter, because you are trying to create a generalized interface for all errors.

Your patronizing tone implies that you think I don't know what an Adapter is, but frankly, I don't think I'm the one that is confused. The intent of Adapters in the context of CloudEvents is to provide a mapping from various platform-specific event types to CloudEvents:

Not all event producers will produce CloudEvents natively. As a result, some "adapter" might be needed to convert these events into CloudEvents. This will typically mean extracting metadata from the events to be used as CloudEvents attributes.

Is there a Yordis or straw-hat-team PAAS I am not aware of that produces errors which need to be adapted to CloudEvents for interoperability with other platforms? Even if there were, I would still suggest that you be more specific and say that this is a Yordis Error Adapter.

Ultimately what this amounts to is a suggestion of one possible way to represent your team's error needs and other hypothetical use cases using CloudEvents. My issue with it is that it is not concrete enough to adapt a specific event type (Yordis Error Adapter) nor flexible enough to solve the general case of adapting cross-domain failure events to CloudEvents (CloudError or humanreadable extension).

I don't think this belongs here. The purpose of adapters is not to define new dataschemas for event payloads. It's for taking existing, widely-used event envelope schemas and making them compatible with CloudEvents. The fact that you have bundled your schema with your proposal rather than linking to external RFC or specification suggests that it is not that. You are not adapting an already widely-used schema and aligning it to CloudEvents. You're proposing a new and arbitrary way to extend the spec to represent a highly general concept with hypothetical use cases, and doing so in a way that actually limits the reusability of CloudEvents for the purpose of failure reporting.

An Adapter takes a specific existing interface A and applies an algorithm to make it compatible with another specific interface B (and ideally in this case, the resulting BfromA has no loss of fidelity with respect to the original A). It is by definition concrete and not abstract. What an Adapter does not do is add additional behavior to B in the hopes that it might be close enough to some set of hypothetical As such that the resulting B+ can subsequently be adapted to those As. That is called extension, and is what you are actually proposing here, even if you are calling it an Adapter.

An Adapter algorithm doesn't contain a comment like this: "You may use a subject that identify particular field from the input data." Adapters don't use "may".

Your stated purpose, by copying the purpose statement of CloudEvents with a few words changed, is to create a common format for representing generalized errors (As) by adding specific structure to CloudEvents (B) to create this new schema (B+). You want to create portability and interoperability for errors, and presumably you think this schema should be widely adopted because you've proposed it here rather than just keeping it within your team. Essentially, you are making the argument that from now on everyone should adapt their As to your B+. Except maybe not, because your response to my concerns about its lack of flexibility seems to be that I should simply ignore it, because it's optional and just an adapter.

You even mentioned that you discussed or considered or tried proposing this as an extension first, which suggests to me that you know that this is - in fact - an extension that you are proposing.

As I stated in my first comment, something that may be useful would be to instead provide actual adapters concretely showing how to map the JSON:API error structure and the Problem JSON error structure to CloudEvents. Those are well-known and (perhaps?) widely-used specs, one an entire, opinionated specification for building JSON APIs and the other a quite well-known RFC. In either case the obvious choice would be to map the flexible payload to the data attribute, which means keeping the dataschema flexible. If I were to try and adapt Problem JSON to your schema, it would either be impossible (if metadata is a Map of strings) or less powerful than straight CloudEvents (if metadata is an object with no backing schema).

[hiebj]

I would like to hear more clear examples (potentially sharing some concrete examples based on real cases) where the Error Adapter would become a problem, and wouldn't work for you.

Since you asked, let's talk about the specific case I mentioned briefly above.

• I have a REST API that allows users to toggle their professional profile and become "searchable" on my organization's job search platform. The back end performs a fairly common type of business rule validation and then comes back to the user with a response that is either successful, or contains an array of objects indicating, in detail, the set of minimum requirements they have not yet met with their profile. Perhaps they need to upload a picture. Maybe they haven't confirmed their email address. They may not have provided the minimum of 5 professional skills, or they may not have either education or work experience listed (we require at least one).

How exactly do I represent this with your schema? Particularly after your comment about constraining the metadata object to a Map of strings?

• Our API is used by third party integrations for job placement organizations that create user profiles automatically for their clients by parsing their resumes. It is important that these consumers be able to understand the data we are sending back to them, because it's complex and not necessarily obvious how it should be handled. On the client side, we provide a library that generates the reconciliation UI using the schema. This makes it possible for the user to fix errors and resubmit, and we don't have to keep updating our Web Component every time we add new requirements.
• Or another example: when an unexpected failure occurs and events go to a DLQ, we have a common middleware that inspects the schema of the failed payload and resolves/hydrates identifiers for logging purposes so that the poor soul debugging at 2am doesn't have to go look up every sparse identifier by hand to see the entity's full object graph.

All of these applications are failure events that require both complex data and a schema, and that's not to mention the basic point that it's just damn useful to deliver a meaningful schema with your payloads, for data integrity validation and debugging purposes.

In the end, it's not actually my job to decide what does or does not belong in this spec or its documented extensions and adapters, since I'm not a voting contributor. At this point I'll bow out and leave it up to the maintainers to decide what is best.

[sdatspun2]

all, I am running late in raising PR against @yordis's fork for JSON schema for error. Here are the schemas associated with event propagating an error. Imo, these should cover both validation errors (error in an event sent earlier that failed validation for example) as well as errors in a system that consumers might be interested in receiving. The errorinstance.json is derived from https://json-schema.org/draft/2019-09/output/schema. pl. review. thanks.

my assumption is that data would carry instance of error.json.

error.json

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "title": "Error",
  "description": "Error conveys one or more error conditions in CloudEvent",
  "properties": {
    "name": {
      "type": "string",
      "description": "Human readable, unique name of the error."
    },
    "id": {
      "type": "string",
      "description": "A unique identifier for this error for correlation purposes."
    },
    "message": {
      "type": "string",
      "description": "Message describing the error."
    },
    "errors": {
      "type": "array",
      "description": "Additional details for error instances if more than ones.",
      "items": {
        "$ref": "errorinstance.json"
      }
    },
    "links": {
      "type": "array",
      "description": "Links providing more information related to error(s). These links could point to FAQ or support pages providing help in order to resolve the error(s).",
      "items": {
        "$ref": "linkdescription.json"
      }
    }
  },
  "required": [
    "name",
    "id",
    "message"
  ]
}

errorinstance.json

{
	"$schema": "http://json-schema.org/draft-07/schema#",
	"type": "object",
	"title": "Error Instance",
	"description": "Error instance.",
	"properties": {
		"keywordlocation": {
			"type": "string",
			"description": "Location of keyword entity involved in error. If the keyword is in `data`, set this value to the JSON pointer to the property."
		},
		"instancelocation": {
			"type": "string",
			"description": "The location of the error instance if qualification is needed beyond the keyword location. Could be used to provide index of entry in array."
		},
		"instancevalue": {
			"type": "string",
			"description": "The value of the keyword entity in error, if available."
		},
		"in": {
			"type": "string",
			"description": "The location of the entity that is in error. A valid value is `context`, `data` or `extension`. Default is `data`."
		},
		"message": {
			"type": "string",
			"description": "Error message for the entity in error."
		},
		"link": {
			"description": "Link providing more information related to this error instance.",
			"$ref": "linkdescription.json"
		}
	},
	"required": [
		"in",
		"message"
	]
}

linkdescription.json

{
	"$schema": "http://json-schema.org/draft-07/schema#",
	"title": "Link Description Object (JSON Schema)",
	"type": "object",
	"required": [
		"href",
		"rel"
	],
	"properties": {
		"href": {
			"description": "a URI template, as defined by RFC 6570, with the addition of the $, ( and ) characters for pre-processing",
			"type": "string"
		},
		"rel": {
			"description": "relation to the target resource of the link.",
			"type": "string"
		},
		"title": {
			"description": "title for the link",
			"type": "string"
		},
		"mediatype": {
			"description": "media type (as defined by RFC 2046) describing the link target",
			"type": "string"
		}
	}
}

[yordis]

@sdatspun2 hey, happy to see you here. I have some concerns based on your response.

1. From the perspective of the ADR as of today, the event itself is an error event, rather than the payload of the event.

   • Why would you do that you may ask?

   Because most of the fields that you may propose could fall into the CloudEvent itself, and either the data or the event itself
   containing all the information is unimportant from my perspective.

   I don't mind adding all the metadata into data, but then we will be replicating most of the fields of the event into the data such as id, source, subject, time; and I can't see a strong reason to do so.

2. (help me here) Your example has a list of errors, where this adapter is for a single error event instance, which means that a list of error events is out of the scope. How CloudEvents handles a list of events sent at once? Could you do that?