Uncategorized

Adopting Problem Details for HTTP APIs By João Antunes

As for Problem Details representing errors other than validation, when needed, they can include an additional objectDetail field, which can contain any arbitrary object to provide additional context about the error to the client.

To illustrate, here’s a couple of examples, starting with a validation error:

YAML

{
  "type": "tag:openvia.io,2020:problems:shift-management/general/validation-error",
  "title": "Invalid request",
  "status": 400,
  "detail": "Invalid request",
  "traceId": "40000003-0009-fb00-b63f-84710c7967bb",
  "errors": [
    {
      "parameter": "shiftId",
      "description": "Invalid shift identifier"
    },
    {
      "pointer": "#/endDateTime",
      "description": "Invalid end date time"
    }
  ]
}
And another type of error:
{
  "type": "tag:openvia.io,2020:problems:shift-management/shift/vehicle-in-use-error",
  "title": "Vehicle in use",
  "status": 422,
  "detail": "Vehicle in use",
  "traceId": "40000003-0009-fb00-b63f-84710c7967bb",
  "objectDetail": {
    "vehicleId": "455bae74-eb9c-43fb-a175-7bda9cff1c2e"
  }
}

{
  "type": "tag:openvia.io,2020:problems:shift-management/general/validation-error",
  "title": "Invalid request",
  "status": 400,
  "detail": "Invalid request",
  "traceId": "40000003-0009-fb00-b63f-84710c7967bb",
  "errors": [
    {
      "parameter": "shiftId",
      "description": "Invalid shift identifier"
    },
    {
      "pointer": "#/endDateTime",
      "description": "Invalid end date time"
    }
  ]
}
And another type of error:
{
  "type": "tag:openvia.io,2020:problems:shift-management/shift/vehicle-in-use-error",
  "title": "Vehicle in use",
  "status": 422,
  "detail": "Vehicle in use",
  "traceId": "40000003-0009-fb00-b63f-84710c7967bb",
  "objectDetail": {
    "vehicleId": "455bae74-eb9c-43fb-a175-7bda9cff1c2e"
  }
}

As you can see, we still return a quick and easy error message, as part of the title and detail fields (returning the same here, but could have added something more to the detail), but we now have the type to clearly tell the client what the problem is, plus the errors or objectDetail to provide additional context.

As mentioned earlier, you can check out a sample implementation in this repository.

Documenting

Now that we discussed how we standardized the definition of the errors our APIs return, it’s probably as important to discuss how to document it. Having errors clearly defined without the clients knowing about them is the same as having defined nothing.

So, one possible approach, which wasn’t the one we took, would be to setup some server that included documentation about the errors. If we went with this, we could have made the type URIs dereferenceable, pointing directly to this server.

However, at this stage, we preferred not go this route, and instead centralize the API docs as much as possible in the OpenAPI docs each API provides, hence the usage of non-dereferenceable URIs.

Will it be good enough in the future? Not sure, but it was the more straightforward solution at this point, and I feel like we still have some margin for improvement within these constraints.

So, how we’re handling this, is by including not only the base Problem Details information in the OpenAPI doc, but also the various problem types possible, in which endpoints they may occur, as well as the structure of the potential objectDetail objects.

Let’s see some examples, starting with the base definition:

YAML

components:
  schemas:
    ProblemDetails:
      description: A tailored problem details schema, respecting the RFC `https://www.rfc-editor.org/rfc/rfc9457`
      type: object
      required:
        - type
        - title
        - status
      properties:
        type:
          type: string
          description: |
            A URI identifying the problem type.
            
            Possible values:
              - `tag:openvia.io,2020:problems:shift-management/general/not-found-error`
              - `tag:openvia.io,2020:problems:shift-management/shift/vehicle-in-use-error`, includes `VehicleInUseErrorDetail` as `objectDetail`
        title:
          type: string
          description: A short, human-readable summary of the problem (not localized).
        status:
          type: integer
          description: The HTTP status code for the request with the problem.
        detail:
          type: string
          nullable: true
          description: A human-readable explanation specific to this occurrence of the problem (not localized).
        traceId:
          type: string
          nullable: true
          description: The id of the trace associated with the request where the problem occurred.
        objectDetail:
          type: object
          nullable: true
          description: |
            An (optional) object containing more specific information about the error.

            Its structure depends on the `type` property, and can be found listed in this document.
    
    ValidationProblemDetails:
      description: A tailored problem details schema, respecting the RFC `https://www.rfc-editor.org/rfc/rfc9457`
      type: object
      required:
        - type
        - title
        - status
        - errors
      properties:
        type:
          type: string
          description: |
            A URI identifying the problem type.
            
            Possible values:
              - `tag:openvia.io,2020:problems:shift-management/general/validation-error
        title:
          type: string
          description: A short, human-readable summary of the problem (not localized).
        status:
          type: integer
          description: The HTTP status code for the request with the problem.
        detail:
          type: string
          nullable: true
          description: A human-readable explanation specific to this occurrence of the problem (not localized).
        traceId:
          type: string
          nullable: true
          description: The id of the trace associated with the request where the problem occurred.
        errors:
          type: array
          items:
            $ref: '#/components/schemas/ValidationError'
          description: A collection of validation errors.
    ValidationError:
      type: object
      description: |
        Includes details about validation errors.

        Only one of the properties `parameter`, `header` and `pointer` is present, depending on the origin (check property descriptions for more info).
      required:
        - description
      properties:
        description:
          type: string
          description: Non-localized description of the error.
        parameter:
          type: string
          nullable: true
          description: Included to identify a path or query parameter that is invalid.
        header:
          type: string
          nullable: true
          description: Included to identify a header parameter that is invalid.
        pointer:
          type: string
          nullable: true
          description: |
            Included to identify a property of a JSON payload that is invalid.

            JSON Pointer to the property that caused the error, following RFC `https://www.rfc-editor.org/rfc/rfc6901`, in its URI Fragment Identifier Representation.
    ConnectionTestFailedErrorDetail:
      type: object
      required:
        - originErrorMessage
      properties:
        originErrorMessage:
          type: string
An example objectDetail:
components:
  schemas:
    # ...
    VehicleInUseErrorDetail:
      type: object
      required:
        - vehichleId
      properties:
        vehichleId:
          type: string
          format: uuid
          description: The id of the vehicle which is already in use
And an example endpoint documenting what errors may occur:
paths:
  /shifts:
    post:
      tags:
        - shifts
      summary: Start a new shift
      operationId: startShift
      parameters:
        - name: X-Tenant-Id
          in: header
          description: The id of the tenant
          required: true
          schema:
            type: string
            format: uuid
            example: ff0f41fa-e4cc-48cd-9139-d742980d3313
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/StartShiftInput'
      responses:
        '201':
          description: Shift started successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/StartShiftResponse'
        '400':
          description: Invalid request
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ValidationProblemDetails'
        '401':
          description: Access token is missing or invalid
        '403':
          description: Authenticated user/application doesn't have permission for the requested operation
        '422':
          description: |
            An error occurred
            
            Problem details error type is one of:
              - `tag:openvia.io,2020:problems:shift-management/shift/vehicle-in-use-error`, includes `VehicleInUseErrorDetail` as `objectDetail`
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ProblemDetails'

components:
  schemas:
    ProblemDetails:
      description: A tailored problem details schema, respecting the RFC `https://www.rfc-editor.org/rfc/rfc9457`
      type: object
      required:
        - type
        - title
        - status
      properties:
        type:
          type: string
          description: |
            A URI identifying the problem type.
            
            Possible values:
              - `tag:openvia.io,2020:problems:shift-management/general/not-found-error`
              - `tag:openvia.io,2020:problems:shift-management/shift/vehicle-in-use-error`, includes `VehicleInUseErrorDetail` as `objectDetail`
        title:
          type: string
          description: A short, human-readable summary of the problem (not localized).
        status:
          type: integer
          description: The HTTP status code for the request with the problem.
        detail:
          type: string
          nullable: true
          description: A human-readable explanation specific to this occurrence of the problem (not localized).
        traceId:
          type: string
          nullable: true
          description: The id of the trace associated with the request where the problem occurred.
        objectDetail:
          type: object
          nullable: true
          description: |
            An (optional) object containing more specific information about the error.

            Its structure depends on the `type` property, and can be found listed in this document.
    
    ValidationProblemDetails:
      description: A tailored problem details schema, respecting the RFC `https://www.rfc-editor.org/rfc/rfc9457`
      type: object
      required:
        - type
        - title
        - status
        - errors
      properties:
        type:
          type: string
          description: |
            A URI identifying the problem type.
            
            Possible values:
              - `tag:openvia.io,2020:problems:shift-management/general/validation-error
        title:
          type: string
          description: A short, human-readable summary of the problem (not localized).
        status:
          type: integer
          description: The HTTP status code for the request with the problem.
        detail:
          type: string
          nullable: true
          description: A human-readable explanation specific to this occurrence of the problem (not localized).
        traceId:
          type: string
          nullable: true
          description: The id of the trace associated with the request where the problem occurred.
        errors:
          type: array
          items:
            $ref: '#/components/schemas/ValidationError'
          description: A collection of validation errors.
    ValidationError:
      type: object
      description: |
        Includes details about validation errors.

        Only one of the properties `parameter`, `header` and `pointer` is present, depending on the origin (check property descriptions for more info).
      required:
        - description
      properties:
        description:
          type: string
          description: Non-localized description of the error.
        parameter:
          type: string
          nullable: true
          description: Included to identify a path or query parameter that is invalid.
        header:
          type: string
          nullable: true
          description: Included to identify a header parameter that is invalid.
        pointer:
          type: string
          nullable: true
          description: |
            Included to identify a property of a JSON payload that is invalid.

            JSON Pointer to the property that caused the error, following RFC `https://www.rfc-editor.org/rfc/rfc6901`, in its URI Fragment Identifier Representation.
    ConnectionTestFailedErrorDetail:
      type: object
      required:
        - originErrorMessage
      properties:
        originErrorMessage:
          type: string
An example objectDetail:
components:
  schemas:
    # ...
    VehicleInUseErrorDetail:
      type: object
      required:
        - vehichleId
      properties:
        vehichleId:
          type: string
          format: uuid
          description: The id of the vehicle which is already in use
And an example endpoint documenting what errors may occur:
paths:
  /shifts:
    post:
      tags:
        - shifts
      summary: Start a new shift
      operationId: startShift
      parameters:
        - name: X-Tenant-Id
          in: header
          description: The id of the tenant
          required: true
          schema:
            type: string
            format: uuid
            example: ff0f41fa-e4cc-48cd-9139-d742980d3313
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/StartShiftInput'
      responses:
        '201':
          description: Shift started successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/StartShiftResponse'
        '400':
          description: Invalid request
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ValidationProblemDetails'
        '401':
          description: Access token is missing or invalid
        '403':
          description: Authenticated user/application doesn't have permission for the requested operation
        '422':
          description: |
            An error occurred
            
            Problem details error type is one of:
              - `tag:openvia.io,2020:problems:shift-management/shift/vehicle-in-use-error`, includes `VehicleInUseErrorDetail` as `objectDetail`
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ProblemDetails'

That’s it for this quick post. Little code, a lot of talk, but I don’t think the implementation details are the most important part of something like this. Even the fact that a framework has or hasn’t built-in, support for Problem Details is kind of irrelevant – it isn’t particularly hard to return a custom JSON payload when some error occurs.
For this reason, the post focused heavily on why I believe it’s important to be deliberate when standardizing and documenting something that’ll greatly impact the effective usage of the APIs we build. Problem Details by itself isn’t particularly helpful without the rest of the work.
Besides this, I also tried to provide a real world example of how this standard can be adopted effectively.

Relevant links:

Thanks for stopping by!