Hybrid Approach to Document Attribute Classification

Abstract

Document processing and understanding have entered a new era with the development of computer vision and its combination with natural language processing. Businesses worldwide operate on various scales, including large, medium, and small, and in various modes, including business-to-business, business-to-consumer, and business-to government. Both the seller and the buyer keep account of their transactions in business records for financial processing, inventory control, auditing, and analyzing sales trends, among other purposes. Out of all the different kinds of documents, invoices are important in the business sector. The process of obtaining and managing the information in these documents by traditional methods that rely on human labor is very costly and labor intensive. In this research work we proposed a hybrid model that utilizes deep learning for structured data extraction from invoice images. The proposed model combines YOLOv5 deep neural network which is used to detect and generate bounding boxes for the key fields in the invoice images along with Optical Character Recognition (OCR) technology which is used to extract details from those specific regions. The dataset we used in this research is a custom collection of invoices which consist of 2225 images sourced from a Danish vendor, provided by Expert System Solution for training and evaluation. Annotation was done through Makesense.ai, an annotation tool powered by AI. The proposed model was extensively validated over this custom dataset and produced excellent results with an overall accuracy of 95.8%, F1 score of 95%, precision of 100%, and recall of 98%. These results highlight the effectiveness of the proposed hybrid approach in accurately identifying and extracting textual data from invoice images. The proposed system can significantly streamline data entry processes and improve automation in invoice processing. In future work, the model may be extended to accommodate more complex invoice layouts such as tables, multilingual content, and handwritten text. Also increasing the size of data with invoices coming x from various vendors and sectors would strengthen the model, and would help generalizing its use in more close-to-reality contexts