Automatic Image Semantic Interpretation using Social Action and Tagging Data

Neela Sawant, Jia Li, James Z. Wang
The Pennsylvania State University, University Park, PA 16802
Abstract:

The plethora of social actions and annotations (tags, comments, ratings) from online media sharing Websites and collaborative games have induced a paradigm shift in the research on image semantic interpretation. Social inputs with their added context represent a strong substitute for expert annotations. Novel algorithms have been designed to fuse visual features with noisy social labels and behavioral signals. In this survey, we review nearly 200 representative papers to identify the current trends, challenges as well as opportunities presented by social inputs for research on image semantics. Our study builds on an interdisciplinary confluence of insights from image processing, data mining, human computer interaction, and sociology to describe the folksonomic features of users, annotations and images. Applications are categorized into four types: concept semantics, person identification, location semantics and event semantics. The survey concludes with a summary of principle research directions for the present and the future.


Full Paper

(PDF, 0.7MB)

On-line Info


Citation: Neela Sawant, Jia Li and James Z. Wang, `` Automatic Image Semantic Interpretation using Social Action and Tagging Data,'' Multimedia Tools and Applications, Special Issue on Survey Papers in Multimedia by World Experts, vol. 51, no. 1, pp. 213-246, 2011.

Copyright 2011 Springer-verlag. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from Springer-verlag. The final publication is available at www.springerlink.com .
Copyright 2011 Sawant, Li, and Wang.

Last Modified: Oct 11, 2010.
© 2010