Rating Image Aesthetics using Deep Learning
Xin Lu
The Pennsylvania State University
Zhe Lin, Hailin Jin, Jianchao Yang
Adobe Systems Inc.
James Z. Wang
The Pennsylvania State University
Abstract:
This paper investigates unified feature learning and
classifier training approaches for image aesthetics assessment.
Existing methods built upon handcrafted or generic image
features and developed machine learning and statistical modeling
techniques utilizing training examples. We adopt a novel deep
neural network approach to allow unified feature learning and
classifier training to estimate image aesthetics. In particular, we
develop a double-column deep convolutional neural network to
support heterogeneous inputs, i.e., global and local views, in order
to capture both global and local characteristics of images. In
addition, we employ the style and semantic attributes of images
to further boost the aesthetics categorization performance.
Experimental results show that our approach produces significantly
better results than the earlier reported results on the AVA dataset
for both the generic image aesthetics and content-based image
aesthetics. Moreover, we introduce a 1.5 million image dataset
(IAD) for image aesthetics assessment and we further boost the
performance on the AVA test set by training the proposed deep
neural networks on the IAD dataset.
Full Paper
(PDF, 19MB)
Photo.net dataset
(gzip, 6MB)
DPChallenge dataset
(gzip, 3MB)
AVA dataset
(tar, 13MB)
Citation:
Xin Lu, Zhe Lin, Hailin Jin, Jianchao Yang and James Z. Wang, ``Rating
Pictorial Aesthetics using Deep Learning,'' IEEE Transactions on
Multimedia, vol. 17, no. 11, pp, 2021-2034, 2015.
© 2015 IEEE. Personal use of this material is permitted. However,
permission to reprint/republish this material for advertising or
promotional purposes or for creating new collective works for resale
or redistribution to servers or lists, or to reuse any copyrighted
component of this work in other works must be obtained from the IEEE.
Last Modified:
August 31, 2015
© 2015