Receipt data extraction with Python image preprocessing using UiPath

Image preprocessing

Let’s take a picture of a receipt, why not my groceries, and do it badly:

• The receipt is not centered, even worse, the orientation is wrong
• Have another element in the picture that has nothing to do with the receipt

The only requirement is that the text on the receipt is readable (when zooming). We don’t do magic here, maybe another day.. Here is the picture taken with my phone (old Android):

We want to clean it, transform it to be usable by removing any ‘noise’. For that we must first ‘detect’ the elements we need. We go with an easy and simple logic: we look for the largest element resembling a rectangular shape and we will assume that it is our receipt.  For this use case, I used Python to do all the image preparation. The detection of shapes will provide this result:

Now we want to make a geometrical transformation by changing the perspective of the image: the image should contain only the receipt and be in the correct orientation. That can be done with another simple assumption: longest length must be vertical. We do not want our robot to read diagonally, it will make no sense out of it. But a bit of distortion is acceptable. And as a last step, we apply a filter to make the text stand out more clearly (scanner effect). The final preproccessed image looks like this:

Receipt data extraction

After this preprocessing, our receipt is ready to be processed by the UiPath Intelligent Document Understanding module to extract the desired data like ‘date’ and ‘total amount’. If needed, we can also extract the items, but these will require more often some human intervention since the print quality is not always perfect. This can be done with the UiPath validation station described in this article. An example of extraction can be seen here:

As we can see, the amount and date are extracted correctly, but the printing quality for the small text of the items is not great and therefore generates some manual validation. This technique can be applied for other types of documents in pictures and for all of them, the gain in time and quality is huge when you use robotic assistance.

How to have your own receipt data extractor?

You will need the following elements and knowledge:

• Python experience in image transformations
• UiPath RPA experience in
  • Document Understanding and Document extraction
  • Long running processes
  • Action center

Or you can hire a veteran swiss RPA consultant that will do it for you