We noticed that JavaScript is disabled in your browser. We suggest enabling it for a better experience.
We noticed you're using an older version of Internet Explorer. We suggest you update to the latest version for a better experience.
Skip to main content

Code formats and examples for data connections middleware

OpenForms is phasing out support for version 1, 2, and 3 data connections. 

As of December 2023, we will no longer offer customer support for these data connections, however they will keep working until we deactivate older version data connections in June 2024. 

If you're building a new data connection today, make sure you're using version 4 or later. If your organization uses older version data connections, you will need to upgrade those to version 4 and update the middleware they access before June 2024. 

Not sure how to get started? Contact support and we'll point you in the right direction. 

How middleware works

Data connections are build-your-own integrations that send or receive OpenForms data to external services as respondents interact with your forms. Because we can’t predict what code format those services use, you need to write middleware to translate OpenForms data into something your external services can understand.

Data connection middleware highlighted between OpenForms and external applications

This article describes the JSON data that OpenForms can work with. Together with equivalent information about your external application (check its developer documentation for a similar article to this one), you'll have all the information you need to code middleware to translate between the two.

It's important to note that the data OpenForms can send and receive (in JSON format) is dictated by the version of data connections you are using. From time to time, we release new versions of data connections which recognize additional JSON properties, or change the syntax of field data within those properties. You can choose what version of data connections to use when you create a data connection.

Version 4.png

Keep reading to see what JSON properties you can send and receive in each version of data connections, as well as real world examples of how to use them.

Data connections version 1 JSON properties

Sending information from OpenForms

OpenForms can send the following JSON properties to middleware in data connections version 1. 

formVersionId

This property is automatically appended by OpenForms to indicate which form a data connection originates from. This is important as a single data connection can be reused on multiple forms.

sendFields

This is the field information from your form that is being sent to an external application via middleware. Each field is identified by the external ID you have assigned to it, and multiple fields can be sent by separating their external IDs with commas.
See Send and receive fields and syntax for more information.

receiveFields

This is the field information that you are requesting from an external application via middleware. Each field is identified by the external ID you have assigned to it in your form, and multiple fields can be sent by separating their external IDs with commas.

See Send and receive fields and syntax for more information.

Example JSON code sent by OpenForms

Let's say you want to create a data connection that sends a respondent’s ID - such as their staff number - to a CRM. This data connection receives back information linked to that ID to prefill other areas of the form - such as a first name, last name, and email address. 

For the sake of this example, let’s assume you’ve assigned the following external IDs:

personId

The respondent’s ID number, as entered into the form

first_name

The respondent’s first name, as held in the CRM

last_name

The respondent’s last name, as held in the CRM

email

The respondent’s email address, as held in the CRM

When triggered, the data connection generates the following JSON object and sends it to middleware:

{
  SendFields: {
    personId: "1"
  },
  ReceiveFields: [ 
    "first_name",
    "last_name",
    "email"
  ],
 FormVersionId: 5521
}

Receiving information from middleware

OpenForms can receive the following JSON properties from middleware in data connections version 1.

fields

This is a property that contains information requested from an external application via middleware. The information nested within should match the receiveFields property sent from OpenForms.

errors

These are any error messages that arise from requests made to your external application via middleware. General errors can be an array containing multiple error strings. Errors specific to a particular field comprise a single error string.

warnings

These are any warning messages that arise from requests made to your external application via middleware. General warnings can be an array containing multiple warning strings. Warnings specific to a particular field comprise a single warning string.

success

These are success messages that arise from requests made to your external application via middleware. These can only apply to individual fields (a general success message would be redundant), and each field can contain a single success string.

referenceId

This is a string of text that is used as an identifier for form submission. For example, this could be the ID of a submission in your CRM application.

status

This is a string of text that represents the status of this form submission. For example this can be "lodged" if a submission has been lodged in your CRM application ready for staff to work on.

Example JSON code received by OpenForms

The code below is an example of a valid response received from middleware, and contains examples of all of the JSON properties listed above. Note that warning and error properties exist as arrays outside of the fields property, and as strings when nested inside it.

{
  fields: {
    age: {
      answer: "24"
    },
    email: {
      answer: "somebody@somewhere.com.au"
    },
    first_name: {
      answer: "John"
    },
    last_name: {
      answer: "Doe"
    },
    gender: {
      answer: "",
      warning: "We didn't have this information in our systems. Please select an appropriate value"
    },
    street_address: {
      answer: "123 Main St, Suburbia, 3000, Australia",
      success: "Street address matched our records"
    },
    state: {
      answer: "Victoria"
    },
    phone: {
      answer: "",
      error: "Phone number was not found. Please enter your phone number"
    },
    favouriteFruit: {
      listItems: [
        { text: "Apples", value: "Apples" },
        { text: "Oranges", value: "Oranges" }
      ]
    }
  },
  warnings: [ "Warning message 1", "Warning message 2" ],
  errors: [ "Error message 1", "Error message 2" ],
  referenceId: "239898",
  status: "Submitted for processing"
}

Data connections version 2 JSON properties

Sending information from OpenForms

OpenForms can send the following properties to middleware in data connections version 2. 

In addition to adding new JSON properties, we’ve changed the names of some properties that OpenForms can recognize in version 2 of data connections . This means in some cases, middleware written for version 1 of data connections will not work with subsequent versions. 

formId

This property is automatically appended by OpenForms to indicate which form a data connection originates from. This identifier is unique to each form, but does not identify form versions.

formVersionId

This property is automatically appended by OpenForms to indicate which form version a data connection originates from. This identifier is unique to each version of a form.

formVersionNumber

This property is automatically appended by OpenForms to indicate which version of a form a data connection originates from.

This number is not unique across forms (multiple forms can have a formVersionNumber of “2” for example), but read together with the formId, can identify individual forms and versions.

externalRespondentId

This property is an identifier (such as a staff ID) that originates outside of OpenForms. 

This property can be populated in a variety of ways. It can be entered into a form field, requested from an external application, or appended to a URL or form link (see an example below of the URL method). 

responseReferenceId

This property is automatically appended by OpenForms, and is used to identify a particular response within OpenForms. Each time a respondent opens a form, a unique responseReferenceId is assigned.

dataConnectionId

This property is automatically appended by OpenForms, indicating which data connection is being used for a particular data transfer. Each data connection is given a unique ID.

sourceType

This property is automatically appended by OpenForms and identifies which part of a form a data connection is triggered from. These may be a field, a section, the end of a form, or the beginning of a form.

 

When the same data connection is used multiple times in a form, triggering across multiple fields or sections, sourceType and sourceElementId properties can be used together to determine which part of a form a particular data connection instance is triggering from.

sourceElementId

This property is automatically appended by OpenForms and identifies the specific element of a form that a data connection is triggered from (for example, the third field of a section, or the second section of a form).

Together with the sourceType property, this can be used to identify where individual instances of multiple-use data connections are triggered from.

externalId 

This property identifies a unique item in an external application (for example, a particular parking infringement or animal registration).

In data connections version 1, this object was named "referenceId."

externalStatus 

This property identifies the status of a particular item in an external application (for example, whether a fine is "paid" or "unpaid.") OpenForms can receive and update statuses (for example, changing a fine from "unpaid" to "paid" upon completion of a payment).
In data connections version 1, this object was named "status."

callCounter

This property is automatically appended by OpenForms each time a data connection is triggered from a particular location in a form.

This information can be used a number of ways - for example, to provide analytics data, to limit data connection requests, or to prevent duplication of data when the same data connection is triggered multiple times.

sendFields

This is the field information from your form that is being sent to an external application. Each field is identified by the external ID you have assigned to it, and multiple fields can be sent by separating their external IDs with commas.
See Send and receive fields and syntax for more information. 

receiveFields

This is the field information that you are requesting from an external application. Each field is identified by the external ID you have assigned to it in your form, and multiple fields can be sent by separating their external IDs with commas. 
See Send and receive fields and syntax for more information.

Example JSON code sent by OpenForms

Let’s say you’ve created a car maintenance form that let’s respondents track basic maintenance tasks, based on oil and tyre change information held in a CRM.

For the sake of this example, let’s assume you’ve assigned the following external IDs:

LicensePlate

The car’s license plate number, as entered into the form

CarModel

The car’s model, as held in the CRM

lastTyreReplacementDate

The date of the car’s last tyre change, as held in the CRM

lastOilChange

The date of the car’s last oil change, as held in the CRM

Let’s also say that this form is embedded into a website that users have to log into. When they click through to this form, the website appends their identity - in the form of an email address - to the form’s URL, like so:

https://localhost:44386/Form/6ad39b10-b984-4422-9df5-ccaeab24eef2?externalRespondentId=frankknight@pointrussell.com

When triggered, the data connection generates the following JSON object and sends it to middleware:

{
    "SendFields": {
    "LicensePlate": "Vic012"
    },
    "FormVersionId": 3096,
    "ReceiveFields": [
    "carModel",
    "lastTireReplacementDate",
    "lastOilReplacementDate"
    ],
    "FormId": 4066,
    "FormVersionNumber": 9,
    "ResponseReferenceId": "d4124407-f816-4df8-8a5a-7991e3aeee2d",
    "ExternalId": null,
    "ExternalStatus": null,
    "ExternalRespondentId": "frankknight@pointrussell.com",
    "SourceType": 3,
    "SourceElementId": "58599",
    "DataConnectionId": 10,
    "CallCounter": 1
}

In this example there’s a difference between the first call made to the CRM, and subsequent calls. The first time this data connection is triggered, the externalId and externalStatus properties are "null", because this information is held in the CRM, not the form. In subsequent calls, assuming this data is provided the first time the CRM is queried, those properties will contain data. 

Like so:

{
   "SendFields": {
     "LicensePlate": "Vic012"
    },
    "FormVersionId": 3096,
    "ReceiveFields": [
    "carModel",
    "lastTireReplacementDate",
    "lastOilReplacementDate"
    ],
    "FormId": 4066,
    "FormVersionNumber": 9,
    "ResponseReferenceId": "aa9507ad-e6e7-4e76-95cb-0ffb2ed36df9",
    "ExternalId": "car-222",
    "ExternalStatus": "broken",
    "ExternalRespondentId": "frankknight@pointrussel.com",
    "SourceType": 3,
    "SourceElementId": "58599",
    "DataConnectionId": 10,
    "CallCounter": 2
}

Receiving information from middleware

Openforms can receive the following JSON properties from middleware in data connections version 2.

fields

This is a property that contains information requested from an external application via middleware. The information nested within should match the “ReceiveFields” objects sent from OpenForms.

errors

These are any error messages that arise from requests made to your external application via middleware. General errors can be an array containing multiple error strings. Errors specific to a particular field comprise a single error string.

warnings

These are any warning messages that arise from requests made to your external application via middleware. General warnings can be an array containing multiple warning strings. Warnings specific to a particular field comprise a single warning string.

success

These are success messages that arise from requests made to your external application via middleware. These can only apply to individual fields (a general success message would be redundant), and each field can contain a single success string.

externalId

This property identifies a unique item in an external application (for example, a particular parking infringement or animal registration).
In data connections version 1, this object was named "referenceId."

externalStatus

This is a string of text that represents the status of this form submission. For example this might be "lodged" if a submission has been lodged in your CRM application ready for staff to work on.
In data connections version 1, this property was named "status."

externalRespondentId

This property is used to identify a user within a system that is not OpenForms, such as a CRM or DMS.

Example JSON Code Received by OpenForms

The code below is a sample of a valid response our car maintenance form may receive from middleware, and contains all of the JSON properties that data connections version 2 can understand. 

As with data connections version 1, warning and error property exist as arrays outside of the fields property, and as strings when nested inside it.

{
  "externalId": "car-222", 
  "externalStatus": "dueForService", 
  "externalRespondentId": "frankknight@pointrussel.com",
  "fields": {
    "carModel": {
      "answer": "Toyota Corolla", 
      "warning": "This model has a faulty airbag, please check with the manufacturer if your car is affected"
    }, 
    "lastOilReplacementDate": {
      "answer": "2 weeks ago", 
      "success": "You just changed your oil. This should be fine for another 11 months"
    }, 
    "lastTireReplacementDate": {
      "answer": "3 years ago", 
      "error": "You should change your tyres"
    }
  },
  "warnings": [
    "Your car was last serviced 2 years ago", 
  ],
}

Data connections version 3 JSON properties

Data connections version 3 uses the exact same JSON properties as data connections version 2, but contains an additional send field - transactionID - and updated send field syntax.

For more information on field syntax, see Send and receive fields

Data connections version 4 JSON objects

Data connections version 4 uses the same JSON properties as data connections version 3, with the addition of one new sendable JSON property:

SourceRepetitionIndex

Available only when a data connection is triggered within a repeatable group. The number indicates which repetition triggered the data connection.

 

Since introducing the ability for Reviewers to request changes to the submission from the Respondent (Oct 2023) as part of a Workflow review step we have also introduced an additional JSON property to denote whether the form is being re-submitted rather than the initial form submission:

isResubmission

To indicate if a data connection is triggered during a resubmission.

 

Field syntax has been updated to more closely align to the OpenForms API and to accommodate for repeatable groups.

For more information on field syntax, see Send and receive fields.

Send and receive fields and syntax

Send fields

The following form fields can be sent in each version of data connections. Note that the send field syntax has changed between versions 1/2, 3 and 4 of Data Connections, so you may need to update your middleware when moving to a new version. 

Send field Syntax
(Version 1 and 2)
Syntax
(Version 3)
Syntax 
(Version 4)
Text   
<field answer value>
{
     value: <field answer value> }
Number 
<field answer value>
{
     value: <field answer value>
}
Dropdown
<field answer value>
{
     selectedValue: <field answer value>
 
{
     value: <field answer value>
 
Checkbox
[
     <value of checked item>,
]
{
selectedValues: [
<value of checked item>,
]
}
{
multiValues: [
<value of checked item>,
]
}
Radio button
<field answer value>
{
selectedValue: <field answer value>
}
{
value: <field answer value>
}
Date 
<field answer value> 
{
     selectedDate: <field answer value>
}
{
value: <field answer value>
}
Email
<field answer value>
{
     value: <field answer value>
Rank 
[
     {
"label": <rank label>,
"rank": <rank>
},
]
{
selectedValues: [
{
label: <rank label>,
value: <rank>
},
]
}
{
multiValues: [
{
label: <rank label>,
value: <rank>
},
]
}
File Upload
[
{
id: <file identifier>,
name: <file name>
},
 
{
     files: [
{
id: <file identifier>,
name: <file name>,
isDeleted: <if the file was deleted>
},
]
}
{ 
files: [
{
fileId: <file identifier>,
fileName: <file name>,
},
]
}
Calculation
<field answer value>
{
value: <field answer value>
}
Signature
(File)
{
method: <signature type>,
name: <signature name>,
file: {
id: <file identifier>,
name: <file name>
isDeleted: <if the file was deleted>
}
}
{
type: <signature type>,
name: <signatory name>,
file: {
id: <file identifier>,
name: <file name>,
isDeleted: <if the file was deleted>
}
}
{
type: <signature type>,
signatoryName: <signatatory name>,
file: {
id: <file identifier>,
name: <file name>,
}
}
Signature
(Drawing)
{
method: <signature type>,
name: <signatory name>,
file: <null>
}
{
type: <signature type>
name: <signatory name>
file: {
id: <null>
name: <null>
isDeleted: <null>
}
{
{
type: <signature method>
signatoryName: <signatory name>
file: <null>
}
Location 
(Map enabled)
{ 
    address: <full address>,
latLng: {
lat: <latitude>,
long: <longitude>,
}
}
{
locationAddressFull: <full address>
locationLatLong: {
lat: <latitude>
lng: <longitude>
}
}
Location
(Map disabled)
{
address: <full address>
latLng: <null>
}
{
locationAddressFull: <full address>
locationLatLong: <null>
}
Payment
{
paymentAmount: <payment amount>
}
{
transactionID: <transaction ID from payment gateway>
paymentAmount: <payment amount>
paymentAmountSurcharge: <surcharge amount>
}

transactionID will be <null> for mid-form data connections. 

{
     paymentTransactionID:        <transaction ID from payment gateway>
paymentGateway: <payment gateway selected by respondent>
paymentAmount: <payment amount> paymentAmountSurcharge: <surcharge amount>
paymentAmountConvenienceFee: <convenience fee amount>
}

paymentTransactionID and paymentGateyway will be <null> for mid-form data connections. paymentAmountSurcharge and paymentAmountConvenienceFee will also be <null> for mid-form data connections when they are variable.

Radio Matrix
[
{
label: <option label>,
value: <option value>
},
]
{
selectedValues: [
{
label: <option label>,
value: <option value>
},
] }
{
multiValues: [
{
label: <option label>,
value: <option value>
},
]
}
Checkbox Matrix
[
{
label: <option label>,
value: [
<option value>,
]
},
]
{
selectedValues: [
{
label: <option label>,
values: [
<option value>,
]
},
]
}
{
multiValues: [
{
label: <option label>,
values: [
<option value>,
]
},
]
}
Content
<field answer value>
{
content: <content>
}
Image 
{
url: <url to the image>,
altText: <alternative text for the image>
}
Video
<video embed code>
{
content: <video embed code>
}
{
videoHtml: <video embed code>
videoTranscript: <video transcript text>
}
 

Repeatable send fields

Version 4 and above

Fields within repeatable groups contain some additional information. The syntax for repeatable send fields is:

   {
minRepeats: <the minimum number of repeats for this field>,
    maxRepeats: <the maximum number of repeats for this field>,
    values: [       
<Send field and value for each repetition>    
]
}

Each value represents a subsequent repetition and uses the complete send field syntax for that field type.

For example, this is what a text field might look like sent as a single field or a repeatable field:

Single field Repeatable field 
"text": {     "value": "John Doe"     }  
 
"text": {
  "minRepeats": 2,
  "maxRepeats": 4,
  "values": [
    {       "value": "John Doe"     },
    {       "value": "Jane Doe"     }
  ]
}

Receive fields

Receive fields are consistent across all versions of Data Connections. 

Receive Field Syntax Notes 
Text 
{
answer: <field answer value>,
success: <optional, success string>,
warning: <optional, warning string>,
error: <optional, error string>
}
  • To return multiple values that will be presented in a list for the form user to choose from, separate the values with \uE04D
Number
Date
Email
Payment
Content
Video
Image
{
answer: <field answer value>,
altText: <optional, image alt text>,
success: <optional, success string>,
warning: <optional, warning string>,
error: <optional, error string>
}
 
Dropdown
{
     answer: <field answer value>
listItems: [
{
text: <option text>,
value: <option value>
},
],
success: <optional, success string>,
warning: <optional, warning string>,
error: <optional, error string>
}
  • To check multiple values, separate values in the answer with a comma (,)
  • To rank multiple values, separate the values in the answer with a pipe (|)
Checkbox
Radio button
Rank

Repeatable receive fields

Version 4 and above

Fields within repeatable groups contain some additional information. The syntax for repeatable receive fields is:

[     
<receive field and value for each repetition>
]

Each value represents a subsequent repetition and uses the complete receive field syntax for that field.

For example, this is what a text field might look like received as a single field compared to repeatable field:

Single field  Repeatable field
 
"text": {     "answer": "John Doe"     }
"text": [
  {     "answer": "John Doe"   },
  {     "answer": "Jane Doe"   }
 
]

Using data connections to overwrite form data

Sometimes, you might want a data connection to overwrite a respondent’s answers to particular fields. 

 

For example, you might use a mid-form data connection to query an address database and autocomplete a respondent's answers to a series of address fields.

address.png

If a respondent subsequently changes their answer to a field like "street address" in the example above, a data connection will need to overwrite or clear multiple fields. 

To do this, your middleware will have to send back receive fields containing either an empty string or updated data for the field you'd like to clear or update, respectively. 

Here's an example of the data that will be sent to and from a form to clear or update a number field:

Send field Cleared receive field   Updated receive field
"Postcode": {  "value": "3000"  }  
"Postcode": {  "answer": ""  }
"Postcode": {  "answer": "3001"  }  

If the field being updated or cleared is a repeatable receive field, your middleware will have to populate the repetitions that are not being cleared or replaced with their original values. 

Here's an example of a repeatable field containing three postcode answers. The first answer needs to be retained, the second answer needs to be updated, and the third answer needs to be cleared. 

Send field Receive field
"Postcode": {
  "minRepeats": 1,
  "maxRepeats": 4,
  "values": [
    {       "value": "3000"     },
    {       "value": "4050"     }, { "value": "4000" }
  ]
}
"Postcode": [
{ "answer": "3000" },
{ "answer": "" },
{ "answer": "3050" }
]
Was this helpful?