How to scan store receipts with React

Paper receipts mostly end up in the trash – they are hard to store for long periods of time, can’t be digitized easily, and formats differ across stores, locations, and regions. 

What if we could turn a store receipt image, extract all the useful information out of it, and keep it in our records? Well, that’s exactly what we’re going to do in this tutorial.

In this tutorial, we will:

  1. Build a React frontend project with Chakra UI components
  2. Let a user upload an image of a store receipt and pull out useful information such as item prices, store location, etc.
    as JSON using Taggun API
  3. Safely store the API key in an env file

Also, you can check out how to build a backend here – How to Scan Receipts in NodeJS

Pre-requisites

The Game Plan

Before we start building, here’s what are looking to achieve.

We will build a web app that lets a user select an image file to upload.

This can be an image of any store receipt. Once selected, it will be uploaded to Taggun API which converts the image to JSON data. All we have to do then is display it in a readable manner.

In a real-world scenario, you can use this same project to reward customers for the amount spent in-store, for data archiving purposes, or for simply converting nonstructured data to structured data.

Getting started

Let’s start by creating a new React project.

Create React App takes care of generating a boilerplate project for us with a sensible file structure, a basic layout, and core packages – saving us a ton of time.

npx create-react-app taggun-frontend
cd taggun-frontend

When the project is ready we can run it with:

npm start

Open up http://localhost:3000 to see it running in your browser.

Awesome! We have a basic React app going.

Add NPM packages

Let’s install some NPM packages to make our life easier.

npm i axios @chakra-ui/react @emotion/react@^11 @emotion/styled@^11 framer-motion@^6 @chakra-ui/icons @saas-ui/react 

Axios – Makes it easy to make a POST call to Taggun API
Chakra UI – Pre-built UI components
Chakra UI Icons – Icons by Chakra UI that look good with their components
SaaS UI – Specialized UI components like Datatable and Loader for a loading animation

Once installed, let’s update the ./src/index.js file so we can use the brand new components from Chakra UI and SaaS UI:

import React from 'react';
import ReactDOM from 'react-dom';
import './index.css';
import App from './App';
import reportWebVitals from './reportWebVitals';
import { SaasProvider } from '@saas-ui/react'

ReactDOM.render(
  <React.StrictMode>
    <SaasProvider>
      <App />
    </SaasProvider>
  </React.StrictMode>,
  document.getElementById('root')
);

// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();

Since it’s important the Provider is at the top-level file (index.js), all our React files will be able to correctly render the new components.

React UI – Upload image file

We are now ready to start building a UI that has:

1. Header section – explains what the project is and instructions on what the user should do next.

2. File upload section – a button to select a file from the computer and another to submit it.

I used a Ticket image as a logo in the Header section that you can download from here: https://github.com/Hannahroseoc/taggun_react_turorial/blob/main/public/outline.png

Make sure that you place it in the public folder for React to import it.

Update ./src/App.js like so:

import { Container, Center, Text, Box, Button, Image, VStack, Link } from '@chakra-
ui/react';
import { AttachmentIcon } from '@chakra-ui/icons'

function App() {

  return (
    <Container>
      <Box height="100vh" mt="30vh">
        <Box textAlign="center">
          <Center>
            <Image
              boxSize='45px'
              objectFit='cover'
              src='./outline.png'
            /></Center>
          <Text fontSize="xl" fontWeight="medium">Easiest way to scan your receipts using
Taggun API</Text>
          <Text fontSize="md" color="gray.500">Upload an image of a store receipt below to
see results</Text>

          <VStack mt={16}>
            <VStack spacing={6}>
              <Box>
                <label htmlFor="file-upload">
                  <AttachmentIcon /> Click to select receipt
                </label>
                <input id="file-upload" type="file" />
              </Box>
          </VStack>
            
            <Button size="md" colorScheme="primary">
              Upload photo
            </Button>

            <Link fontSize="xs" colorScheme="primary"
href="https://d7pdsiqo9rcig.cloudfront.net/wp-content/uploads/2019/04/receipt-
2.jpg">Download a sample one</Link></VStack>
        </Box>
      </Box>
    </Container>
  );
}

export default App;

Fantastic! Now we have a shell UI down that looks really good. The only problem is, it doesn’t do anything. Let’s fix that next.

Upload a file

We can use the onChange event on an input to listen to when a file has been selected by the user. The object is then is stored in a state variable.

 <input id="file-upload" type="file" onChange={handleUploadFile} />

Update the ./src/App.js :

import { useState } from 'react';

const [file, setFile] = useState();

const handleUploadFile = (evt) => {
    setFile(
      evt.target.files[0],
    )
}

Add a Submit button

Similar to the input, a Button also has an event onClick that gets triggered when someone clicks it. We can use this to upload the file using Axios to Taggun.

Taggun expects a POST request with a file attached and an apikey as the header.

<Button size="md" colorScheme="primary" onClick={submitPhoto}>
              Upload photo
</Button>

Update the ./src/App.js :

import axios from 'axios';

const [file, setFile] = useState();
const [isUploading, setIsUploading] = useState(false);

...

const submitPhoto = async () => {
    if (!file) { return; }
    setIsUploading(true)
    const data = new FormData()

    data.append('file', file)

    let url = "https://api.taggun.io/api/receipt/v1/verbose/file";

    try {
      const res = await axios.post(url, data, {
        headers: {
          'Content-Type': 'multipart/form-data',
          'apikey': "API_KEY"
        } 
      })
      console.log(res)
      setIsUploading(false);
    } catch (e) {
      console.error(e);
    }
}

.

Note: Remember to replace the API_KEY with yours from Taggun.

Run the project with npm start and try to upload the receipt image and press Submit. If everything goes well, you should see a JSON response in your console from Taggun.

Here’s what mine looks like:

React UI – Polish up

Now that we have data being returned from the API, we can polish up our UI so the user can easily interpret the data. Here’s how I chose to do it:

1. Show merchant’s address as Text

2. Show items sold in a Data Table

3. Show full response as JSON

I also added a loading animation for when we are waiting for the results to be processed.

import { Loader, DataTable } from '@saas-ui/react'
import { Container, Center, Text, Box, Button, Image, VStack, HStack, Divider, Link } from
'@chakra-ui/react';
import axios from 'axios';
import { useState } from 'react';
import { AttachmentIcon } from '@chakra-ui/icons'

function App() {
  const [res, setRes] = useState();
  const [file, setFile] = useState();
  const [isUploading, setIsUploading] = useState(false);
  const handleUploadFile = (evt) => {
    setFile(
      evt.target.files[0],
    ) 
  }

  const submitPhoto = async () => {
    if (!file) { return; }
    setIsUploading(true)
    const data = new FormData()
    data.append('file', file)

    let url = "https://api.taggun.io/api/receipt/v1/verbose/file";

    try {
      const res = await axios.post(url, data, {
        headers: {
          'Content-Type': 'multipart/form-data',
          'apikey': "API_KEY"
        }
      })
      console.log(res)
      setRes(res)
      setIsUploading(false);
    } catch (e) {
      console.error(e);
    }
  }
  return (
    <Container>
      <Box height="100vh" mt="30vh">
        <Box textAlign="center">
          <Center>
              <Image
              boxSize='45px'
              objectFit='cover'
              src='./outline.png'
            /></Center>
          <Text fontSize="xl" fontWeight="medium">Easiest way to scan your receipts using
Taggun API</Text>
          <Text fontSize="md" color="gray.500">Upload an image of a store receipt below to
see results</Text>
          <VStack mt={16}>
            {file ? <Box><Text>{file.name}</Text></Box> : <VStack spacing={6}><Box>
              <label htmlFor="file-upload">
                <AttachmentIcon /> Click to select receipt
              </label>
              <input id="file-upload" type="file" onChange {handleUploadFile} />
            </Box>
            </VStack>}
            {isUploading ? <Loader thickness="2px" /> : <Button size="md"
colorScheme="primary" onClick={submitPhoto}>
              Upload photo
            </Button>
}
            <Link fontSize="xs" colorScheme="primary"
href="https://d7pdsiqo9rcig.cloudfront.net/wp-content/uploads/2019/04/receipt-
2.jpg">Download a sample one</Link></VStack>
</Box>
        {res && <Box>
          <Box mt={10}>
            <Center>
              <Text fontWeight="medium">Successfully analyzed receipt</Text></Center>
            <Text fontWeight="medium">Details</Text>
            <Divider mb={4} />
            <HStack>
              <Text width={100}>
                Receipt #
              </Text>
              <Text>
                {res.data.entities.receiptNumber.data}
              </Text>
</HStack>

              <HStack>
              <Text width={100}>
Tax </Text>
              <Text>{res.data.entities.multiTaxLineItems[0].data.taxAmount.text}</Text>
            </HStack>
            <HStack>
              <Text width={100}>
                Merchant
              </Text>
              <Text>{res.data.merchantName.data}</Text>
              <Text>{res.data.merchantCity.text}</Text>
            </HStack>
          </Box>
          <Box overflowX="auto" mt={10}>
            <Text fontWeight="medium">Items</Text>
            <Divider mb={4} />
            <Box>
              <DataTable size="xs" columns={[{ "accessor": "data", "Header": "Currency" }, {
"accessor": "currencyCode", "Header": "Name" }, { "accessor": "text", "Header": "Item" }]}
data={res.data.amounts} />
</Box>

            <Box overflowX="auto" mt={10}>
              <Text fontWeight="medium">Full JSON response</Text>
              <Divider mb={4} />
              <pre style={{ "fontSize": "12px", "height": "400px", overflow: "scroll", }}>
                {JSON.stringify(res, null, 2)}
              </pre>
            </Box> </Box>
        </Box>}
      </Box>
    </Container>
  ); 
}
export default App;

Here’s what the UI looks like with the polish changes applied. Not too shabby right?

Environment Variables

It is a bad practice to save your API Key with the rest of your source code in case it gets leaked when you commit your code to Github. Let’s use env variables to hide it away.

Create a new file called .env and add your own API Key:

REACT_APP_TAGGUN_API_KEY=123445 #replace with your own here

Now we can update the submitPhoto in ./src/App.js with:

headers: {
          'Content-Type': 'multipart/form-data',
          'apikey': process.env.REACT_APP_TAGGUN_API_KEY
        }

Congratulations! You’ve now built a fully functioning React project that can scan receipts and pull out important information such as items bought, price, tax amount, store name, etc.