Large-scale Tag-based Font Retrieval with Generative Feature Learning

Tianlang Chen and Jiebo Luo, Department of Computer Science, University of Rochester

Zhaowen Wang, Ning Xu and Hailin Jin, Adobe Research

Abstract

Paper[PDF]

DataSet

Font selection is one of the most important steps in a design workflow. Traditional methods rely on ordered lists which require significant domain knowledge and are often difficult to use even for trained professionals. In this paper, we address the problem of large-scale tag-based font retrieval which aims to bring semantics to the font selection process and enable people without expert knowledge to use fonts effectively. We collect a large-scale font tagging dataset of high-quality professional fonts. The dataset contains nearly 20,000 fonts, 2,000 tags, and hundreds of thousands of font-tag relations. We propose a novel generative feature learning algorithm that leverages the unique characteristics of fonts. The key idea is that font images are synthetic and can therefore be controlled by the learning algorithm. We design an integrated rendering and learning process so that the visual feature from one image can be used to reconstruct another image with different text. The resulting feature captures important font design details while is robust to nuisance factors such as text. We propose a novel attention mechanism to re-weight the visual feature for joint visual-text modeling. We combine the feature and the attention mechanism in a novel recognition-retrieval model. Experimental results show that our method significantly outperforms the state-of-the-art for the important problem of large-scale tag-based font retrieval.

Cite us

dataset.tar.gz

Tianlang Chen, Zhaowen Wang, Ning Xu, Hailin Jin, and Jiebo Luo. "Large-scale Tag-based Font Retrieval with Generative Feature Learning", IEEE International Conference on Computer Vision(ICCV), 2019.

The dataset contains two parts. The first part contains the original font dataset we collected from Myfonts.com. There are totally 18,815 fonts labeled by 1,824 tags. The tags are preprocessed. In addition, we provide the training/validation/test set of the fonts. The second part contains the AMT-test set we used in our experiment. We collected 1,661 high confidence groups for the top-300 frequent tags, each group corresponds to a tag and is labeled one best matching font.
If you have any questions, please let me know.