Form Recognizer API

Form Recognizer extracts key value pairs and tables from documents and includes the following options:

  • Custom - Extracts information from forms (PDFs and images) into structured data based on a model created by a set of representative training forms. Form Recognizer learns the structure of your forms to intelligently extract text and data. It ingests text from forms, applies machine learning technology to identify keys and tables, and then outputs structured data that includes the relationships within the original file.
  • Prebuilt Receipt - Detects and extracts data from receipts using optical character recognition (OCR) and our receipt model, enabling you to easily extract structured data from receipts such as merchant name, merchant phone number, transaction date, transaction total, and more.


This API is currently available in:

  • Australia East - australiaeast.api.cognitive.microsoft.com
  • Canada Central - canadacentral.api.cognitive.microsoft.com
  • Central US - centralus.api.cognitive.microsoft.com
  • East Asia - eastasia.api.cognitive.microsoft.com
  • East US - eastus.api.cognitive.microsoft.com
  • East US 2 - eastus2.api.cognitive.microsoft.com
  • North Europe - northeurope.api.cognitive.microsoft.com
  • South Central US - southcentralus.api.cognitive.microsoft.com
  • Southeast Asia - southeastasia.api.cognitive.microsoft.com
  • UK South - uksouth.api.cognitive.microsoft.com
  • West Europe - westeurope.api.cognitive.microsoft.com
  • West US 2 - westus2.api.cognitive.microsoft.com

Get Receipt Result

Query the status and retrieve the result of an Analyze Receipt operation. The URL to this interface can be obtained from the 'Operation-Location' header in the Analyze Receipt response. We recommend calling this operation no more than once a second.

Select the testing console in the region where you created your resource:

Australia East Canada Central Central US Central US EUAP East Asia East US East US 2 North Europe South Central US Southeast Asia UK South West Europe West US 2

Request URL

Request parameters

string

Id returned by the Analyze Receipt operation, which can be obtained from the "Operation-Local" response header.

Request headers

string
Subscription key which provides access to this API. Found in your Cognitive Services accounts.

Request body

Response 200

JSON fields in the response body:

FieldTypeDescription
statusStringAnalyze Receipt operation status. Possible values:
  • NotStarted: The receipt analysis process has not started.
  • Running: The receipt is being analyzed.
  • Failed: The receipt analysis process has failed.
  • Succeeded: The receipt analysis process has succeeded.
If the status is Succeeded, the response JSON will include the receipt understanding and text recognition results. The receipt understanding result is organized as a dictionary of named field values, where each value contains the extracted text, normalized value, and corresponding OCR word elements. The text recognition result is organized as a hierarchy of Lines and Words, with text, bounding box and confidence information.
recognitionResults[Object]An array of objects, each representing the OCR result for a page in the input document.
understandingResults[Object]An array of objects, each representing a receipt detected in the input document.
pages[Integer]List of pages where the receipt is found. Currently, only single-page receipts are supported.
fieldsObjectDictionary of analyzed field values. If a field is not detected, the corresponding field value will be set to null.
valueTypeStringData type of the field value. Currently, possible types include "stringValue" and "numberValue".
valueString/NumberSemantic value of the recognized field. If the text value cannot be converted, value will be set to "" for stringValue or "NaN" for numberValue.
textStringExtracted field text in understandingResults, or a recognized word/line in recognitionResults.
elements[Object]List of references to OCR words comprising the detected field value.
$refStringJSON pointer to an OCR word element. For example, "#/recognitionResults/0/lines/1/words/2" refers to 3rd word in the 2nd line of the 1st page in the document.
lines[Object]An array of objects, each representing a line of recognized text. The maximum number of lines returned is 300 per page.
The lines are sorted top to bottom, left to right, although in certain cases proximity is treated with higher priority. As the sorting order depends on the detected text, it may change across images and OCR version updates. Thus, business logic should be built upon the actual line location instead of order.
words[Object]An array of objects, each representing a recognized word.
boundingBox[Number]The bounding box of a recognized line or word, depending on the parent object. It is represented by an array of eight numbers corresponding to four points, clockwise from the top-left corner. For image, the (x, y) coordinates are measured in pixels from the top-left corner of the image. For PDF, the (x, y) coordinates are measured in inches from the top-left corner of the each page.
confidenceStringThis flag appears as "Low" when the OCR recognition result of a word is of low confidence.
widthNumberThe width of the image/PDF in pixels/inches, respectively.
heightNumberThe height of the image/PDF in pixels/inches, respectively.
clockwiseOrientationNumberThe orientation of the image in clockwise direction, measured in degrees between [0, 360).
pageIntegerThe page number of the recognition result.
unitStringThe unit used by the width, height and boundingBox properties. For images, the unit is "pixel". For PDF, the unit is "inch".
Supported fields:

NameTypeExample
MerchantNameString"Contoso"
MerchantPhoneNumberString"+19876543210"
MerchantAddressString"123 Main St Redmond WA 98052"
TransactionDateString"2019-06-26"
TransactionTimeString"16:49:00"
TotalNumber14.34
SubtotalNumber12.34
TaxNumber2.00

{
  "status": "Succeeded",
  "recognitionResults": [
    {
      "page": 1,
      "clockwiseOrientation": 1.28,
      "width": 3024,
      "height": 4032,
      "unit": "pixel",
      "lines": [
        {
          "boundingBox": [
            772,
            506,
            1773,
            517,
            1771,
            732,
            769,
            721
          ],
          "text": "Contoso",
          "words": [
            {
              "boundingBox": [
                786,
                515,
                1766,
                517,
                1768,
                736,
                793,
                720
              ],
              "text": "Contoso"
            }
          ]
        },
        ...
      ]
    }
  ],
  "understandingResults": [
    {
      "pages": [
        1
      ],
      "fields": {
        "MerchantName": {
          "valueType": "stringValue",
          "value": "Contoso",
          "text": "Contoso",
          "elements": [
            {
              "$ref": "#/recognitionResults/0/lines/0/words/0"
            }
          ]
        },
        "MerchantPhoneNumber": {
          "valueType": "stringValue",
          "value": "+19876543210",
          "text": "( 987 ) 654 - 3210",
          "elements": [
            {
              "$ref": "#/recognitionResults/0/lines/2/words/0"
            },
            {
              "$ref": "#/recognitionResults/0/lines/2/words/1"
            },
            {
              "$ref": "#/recognitionResults/0/lines/2/words/2"
            },
            {
              "$ref": "#/recognitionResults/0/lines/2/words/3"
            },
            {
              "$ref": "#/recognitionResults/0/lines/2/words/4"
            },
            {
              "$ref": "#/recognitionResults/0/lines/2/words/5"
            }
          ]
        },
        "TransactionDate": {
          "valueType": "stringValue",
          "value": "2019-05-06",
          "text": "05/06/19",
          "elements": [
            {
              "$ref": "#/recognitionResults/0/lines/31/words/0"
            }
          ]
        },
        "Total": {
          "valueType": "numberValue",
          "value": 12.34,
          "text": "12.34",
          "elements": [
            {
              "$ref": "#/recognitionResults/0/lines/18/words/0"
            }
          ]
        },
        ...
      }
    }
  ]
}

Response 404

Invalid or expired operation ID.

{
  "error": {
    "code": "BadArgument",
    "message": "Operation ID is invalid or expired."
  }
}

Response 500

Internal server error.

{
  "error": {
    "code": "Unspecified",
    "message": "Internal server error."
  }
}

Response 503

Transient fault while querying Windows Azure storage services.

{
  "error": {
    "code": "StorageException",
    "message": "Transient fault occurred while querying Windows Azure storage services. Please try again later."
  }
}

Code samples

@ECHO OFF

curl -v -X GET "https://westeurope.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/prebuilt/receipt/operations/{operationId}"
-H "Ocp-Apim-Subscription-Key: {subscription key}"

--data-ascii "{body}" 
using System;
using System.Net.Http.Headers;
using System.Text;
using System.Net.Http;
using System.Web;

namespace CSHttpClientSample
{
    static class Program
    {
        static void Main()
        {
            MakeRequest();
            Console.WriteLine("Hit ENTER to exit...");
            Console.ReadLine();
        }
        
        static async void MakeRequest()
        {
            var client = new HttpClient();
            var queryString = HttpUtility.ParseQueryString(string.Empty);

            // Request headers
            client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", "{subscription key}");

            var uri = "https://westeurope.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/prebuilt/receipt/operations/{operationId}?" + queryString;

            var response = await client.GetAsync(uri);
        }
    }
}	
// // This sample uses the Apache HTTP client from HTTP Components (http://hc.apache.org/httpcomponents-client-ga/)
import java.net.URI;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.utils.URIBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

public class JavaSample 
{
    public static void main(String[] args) 
    {
        HttpClient httpclient = HttpClients.createDefault();

        try
        {
            URIBuilder builder = new URIBuilder("https://westeurope.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/prebuilt/receipt/operations/{operationId}");


            URI uri = builder.build();
            HttpGet request = new HttpGet(uri);
            request.setHeader("Ocp-Apim-Subscription-Key", "{subscription key}");


            // Request body
            StringEntity reqEntity = new StringEntity("{body}");
            request.setEntity(reqEntity);

            HttpResponse response = httpclient.execute(request);
            HttpEntity entity = response.getEntity();

            if (entity != null) 
            {
                System.out.println(EntityUtils.toString(entity));
            }
        }
        catch (Exception e)
        {
            System.out.println(e.getMessage());
        }
    }
}

<!DOCTYPE html>
<html>
<head>
    <title>JSSample</title>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js"></script>
</head>
<body>

<script type="text/javascript">
    $(function() {
        var params = {
            // Request parameters
        };
      
        $.ajax({
            url: "https://westeurope.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/prebuilt/receipt/operations/{operationId}?" + $.param(params),
            beforeSend: function(xhrObj){
                // Request headers
                xhrObj.setRequestHeader("Ocp-Apim-Subscription-Key","{subscription key}");
            },
            type: "GET",
            // Request body
            data: "{body}",
        })
        .done(function(data) {
            alert("success");
        })
        .fail(function() {
            alert("error");
        });
    });
</script>
</body>
</html>
#import <Foundation/Foundation.h>

int main(int argc, const char * argv[])
{
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
    
    NSString* path = @"https://westeurope.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/prebuilt/receipt/operations/{operationId}";
    NSArray* array = @[
                         // Request parameters
                         @"entities=true",
                      ];
    
    NSString* string = [array componentsJoinedByString:@"&"];
    path = [path stringByAppendingFormat:@"?%@", string];

    NSLog(@"%@", path);

    NSMutableURLRequest* _request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:path]];
    [_request setHTTPMethod:@"GET"];
    // Request headers
    [_request setValue:@"{subscription key}" forHTTPHeaderField:@"Ocp-Apim-Subscription-Key"];
    // Request body
    [_request setHTTPBody:[@"{body}" dataUsingEncoding:NSUTF8StringEncoding]];
    
    NSURLResponse *response = nil;
    NSError *error = nil;
    NSData* _connectionData = [NSURLConnection sendSynchronousRequest:_request returningResponse:&response error:&error];

    if (nil != error)
    {
        NSLog(@"Error: %@", error);
    }
    else
    {
        NSError* error = nil;
        NSMutableDictionary* json = nil;
        NSString* dataString = [[NSString alloc] initWithData:_connectionData encoding:NSUTF8StringEncoding];
        NSLog(@"%@", dataString);
        
        if (nil != _connectionData)
        {
            json = [NSJSONSerialization JSONObjectWithData:_connectionData options:NSJSONReadingMutableContainers error:&error];
        }
        
        if (error || !json)
        {
            NSLog(@"Could not parse loaded json with error:%@", error);
        }
        
        NSLog(@"%@", json);
        _connectionData = nil;
    }
    
    [pool drain];

    return 0;
}
<?php
// This sample uses the Apache HTTP client from HTTP Components (http://hc.apache.org/httpcomponents-client-ga/)
require_once 'HTTP/Request2.php';

$request = new Http_Request2('https://westeurope.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/prebuilt/receipt/operations/{operationId}');
$url = $request->getUrl();

$headers = array(
    // Request headers
    'Ocp-Apim-Subscription-Key' => '{subscription key}',
);

$request->setHeader($headers);

$parameters = array(
    // Request parameters
);

$url->setQueryVariables($parameters);

$request->setMethod(HTTP_Request2::METHOD_GET);

// Request body
$request->setBody("{body}");

try
{
    $response = $request->send();
    echo $response->getBody();
}
catch (HttpException $ex)
{
    echo $ex;
}

?>
########### Python 2.7 #############
import httplib, urllib, base64

headers = {
    # Request headers
    'Ocp-Apim-Subscription-Key': '{subscription key}',
}

params = urllib.urlencode({
})

try:
    conn = httplib.HTTPSConnection('westeurope.api.cognitive.microsoft.com')
    conn.request("GET", "/formrecognizer/v1.0-preview/prebuilt/receipt/operations/{operationId}?%s" % params, "{body}", headers)
    response = conn.getresponse()
    data = response.read()
    print(data)
    conn.close()
except Exception as e:
    print("[Errno {0}] {1}".format(e.errno, e.strerror))

####################################

########### Python 3.2 #############
import http.client, urllib.request, urllib.parse, urllib.error, base64

headers = {
    # Request headers
    'Ocp-Apim-Subscription-Key': '{subscription key}',
}

params = urllib.parse.urlencode({
})

try:
    conn = http.client.HTTPSConnection('westeurope.api.cognitive.microsoft.com')
    conn.request("GET", "/formrecognizer/v1.0-preview/prebuilt/receipt/operations/{operationId}?%s" % params, "{body}", headers)
    response = conn.getresponse()
    data = response.read()
    print(data)
    conn.close()
except Exception as e:
    print("[Errno {0}] {1}".format(e.errno, e.strerror))

####################################
require 'net/http'

uri = URI('https://westeurope.api.cognitive.microsoft.com/formrecognizer/v1.0-preview/prebuilt/receipt/operations/{operationId}')
uri.query = URI.encode_www_form({
})

request = Net::HTTP::Get.new(uri.request_uri)
# Request headers
request['Ocp-Apim-Subscription-Key'] = '{subscription key}'
# Request body
request.body = "{body}"

response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
    http.request(request)
end

puts response.body