[FIXED] How to handle Credit Cards fonts with OpenCV and Tesseract in Python

Issue

I'm trying to read cards and output card numbers and expiry date with OpenCV.

import cv2
import pytesseract

filename = 'image1.png'
img = cv2.imread(filename)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
canny = cv2.Canny(gray, 50, 150, apertureSize=3)
result = pytesseract.image_to_string(canny)
print(f"OCR Results: {result}")

cv2.imshow('img', img)
cv2.imshow('canny', canny)

if cv2.waitKey(0) & 0xff == 27:
    cv2.destroyAllWindows()

1. Image before processing

2. Image after Canny

The result text does not look good. See the screenshot below:

Question: How can I properly handle the cards fonts well for better results. Any idea is highly appreciated.

Thanks.

Solution

It looks like the OCR is not working well when passing the edges of the text.
You better apply threshold instead of using Canny.

I suggest the following stages:

• Convert from BGR to HSV color space, and get the S (saturation) color channel of HSV.
  All gray pixels in S are zero, and colored pixels are above zero.
• Convert to binary using automatic threshold (use cv2.THRESH_OTSU).
• Crop the contour with the maximum size.
  Because the image you posted contains some background.
• Apply OCR on the cropped area.

Here is the code:

import numpy as np
import cv2
import imutils  # https://pypi.org/project/imutils/
import pytesseract

pytesseract.pytesseract.tesseract_cmd = r'C:\Program Files\Tesseract-OCR\tesseract.exe'  # I am using Windows

img = cv2.imread('image1.png')  # Read input image

# Convert from BGR to HSV color space
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

# Get the saturation color channel - all gray pixels are zero, and colored pixels are above zero.
s = hsv[:, :, 1]

# Convert to binary using automatic threshold (use cv2.THRESH_OTSU)
ret, thresh = cv2.threshold(s, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

# Find contours (in inverted thresh)
cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cnts = imutils.grab_contours(cnts)

# Find the contour with the maximum area.
c = max(cnts, key=cv2.contourArea)

# Get bounding rectangle
x, y, w, h = cv2.boundingRect(c)

# Crop the bounding rectangle out of thresh
thresh_card = thresh[y:y+h, x:x+w].copy()

# OCR
result = pytesseract.image_to_string(thresh_card)
print(f"OCR Results:\n {result}")


# Show images for debugging
cv2.imshow('s', s)
cv2.imshow('thresh', thresh)
cv2.imshow('thresh_card', thresh_card)
cv2.waitKey(0)
cv2.destroyAllWindows()

---

OCR Result:

 Visa Classic

| By)

4000 1234 Sb18 9010

CARDHOLDER MARE
VISA

Still not perfect...

---

s:

thresh:

thresh_card:

Answered By - Rotem