Automated Invoice Processing – A Beginners Guide

Invoices form the backbone of transactions between different businesses. With the advancement of technology, processes and data has been digitized with the use of ERP system and other technologies. Records in ERP systems are inserted manually using human force, a process which is laborious, time consuming and error prone. This blog discusses the need of automating the invoice process and the different challenges faced in the process of automation.

Why we need to automate invoice processing?

Progression in almost every industry is now enabled mainly by technology. With this advancement, businesses are exploring new domains, avenues & expanding their footprints. This translates directly into new business opportunities between different businesses and therefore, the volume of transactions has increased multi-fold. This huge volume of transactions is interpreted into invoices, which need processing. As mentioned earlier, manually handling these invoicing is a hassle and time taking process. It involves:

  1. Going through the invoice line by line
  2. Collecting the relevant data by verifying it multiple times (inv#, dates, amounts etc.)
  3. Inserting the data into the record (Excel sheets, ERP Systems)

Manual Invoice Processing

Figure 1 Manual Invoice Processing

This process of manual processing has the following issues:

  1. Human error
  2. Cost intensive
  3. More human effort
  4. Time consuming

Let’s imagine receiving an invoice from a vendor and the person handling the invoices mistakenly enters the wrong date, or perhaps enters the wrong invoice amount (adds or lessens one ‘0’ from the invoice total amount). This mistake might seem little in its entirety, but it will add discrepancy in your records.

This predicament is shared by almost all small-scale and large-scale businesses equally, especially it is a nightmare for their financial and accounting departments. They must manually handle volumes of invoices before closing the week or month because this is a time-sensitive business process. This has enabled them to explore other ways of processing invoices, which is less error prone, less time consuming and less expensive. By automating invoice processing, business can process huge volumes of data automatically, extract meaningful data, and insert the entries into the system (databases, excel sheets etc.)

Invoice Processing Automation

The integration of an automated system to process invoices can bring meaning to the businesses and add value to its processes. It can streamline business processes and can cut down cost, time and human effort.

Extracting structured or meaningful data from documents has always been devil of a task. The process comprises of different steps.

  • Pre-Processing: First step involves the extraction of text from invoices which may come in any format (PDF, doc, images). Depending upon different business processes, some steps might get added or removed. For example, some companies might receive invoices in print form, which need scanning to start the process, so it also becomes a part of pre-processing. Extraction of textual data can be done using:

 

  • Cleaning: After the extraction of data, it is desirable to clean the data to remove unwanted and junk data. For example, pdf2json library convert data into JSON format which also contains spaces and special characters. We might need to remove those and at some places convert the hex of those special characters before removing (e.g. %2F into ‘/’ for dates), so it won’t be desirable to remove all special characters.

 

  • Processing the extracted text: Processing on the extracted text is where the real challenge lies. Extracted text is useless until we attach meaning to it. Different approaches have been proposed, experimented with and are in use to tackle the problem. We have worked with 2 libraries, pdf2json &  invoice2data so far. In order to extract meaningful data from invoices, we need a context or some form of template which we can feed these libraries to extract fields like invoice number, invoice date, invoice total, line items etc. Otherwise there is no way, these libraries can extract the relevant fields on their own. This is a multi-step process and managing configuration/templates for these invoices form the basis of this step. Both libraries output the relevant fields which can be inserted in the ERP systems. Specific details can be found in the documentation of these libraries.

 

Conclusion: We have worked with both libraries and we found invoice2data to be more convenient and accurate in its results. One may prefer one over the other as per business needs. In the next blog of this series we will explain how to set-up invoice2data on your computers, and afterwards we will use it with different invoices to extract meaningful structured data.