Large scale scene text verification with guided attention

Dafang He, Yeqing Li, Alexander Gorban, Derrall Heath, Julian Ibarz, Qian Yu, Daniel Kifer, C. Lee Giles

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Many tasks are related to determining if a particular text string exists in an image. In this work, we propose a new framework that learns this task in an end-to-end way. The framework takes an image and a text string as input and then outputs the probability of the text string being present in the image. This is the first end-to-end framework that learns such relationships between text and images in scene text area. The framework does not require explicit scene text detection or recognition and thus no bounding box annotations are needed. It is also the first work in scene text area that tackles such a weakly labeled problem. Based on this framework, we developed a model called Guided Attention. Our designed model achieves better results than several state-of-the-art scene text reading based solutions for a challenging Street View Business Matching task. The task tries to find correct business names for storefront images and the dataset we collected for it is substantially larger, and more challenging than existing scene text dataset. This new real-world task provides a new perspective for studying scene text related problems.

Original languageEnglish (US)
Title of host publicationComputer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers
EditorsC.V. Jawahar, Konrad Schindler, Greg Mori, Hongdong Li
PublisherSpringer Verlag
Number of pages16
ISBN (Print)9783030208721
StatePublished - 2019
Event14th Asian Conference on Computer Vision, ACCV 2018 - Perth, Australia
Duration: Dec 2 2018Dec 6 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11365 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference14th Asian Conference on Computer Vision, ACCV 2018

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Large scale scene text verification with guided attention'. Together they form a unique fingerprint.

Cite this