Specialized Web Robot for Objectionable Web Content Classification

Authors: SuGil Choi, SeungWan Han, Chi-Yoon Jeong, TaekYong Nam

Abstract:

This paper proposes a specialized Web robot to automatically collect objectionable Web contents for use in an objectionable Web content classification system, which creates the URL database of objectionable Web contents. It aims at shortening the update period of the DB, increasing the number of URLs in the DB, and enhancing the accuracy of the information in the DB.

Keywords: Web robot, objectionable Web content classification, URL database, URL rating

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1328378

References:

[1] http://www.robotstxt.org/wc/faq.html#what
[2] SeungMin Lee, TaekYong Nam, JongSu Jang. http://kidbs.itfind.or.kr/WZIN/jugidong/1161/116101.htm. IITA itfind, 2004
[3] Soumen Chakrabarti, Martin van den Berg, and Byron Dom. Focused crawling: a new approach to topic-specific Web resource discovery, 8th International World Wide Web Conference, 1999.
[4] C. C. Aggarwal, F. Al-Garawi, P. Yu. Intelligent Crawling on the World Wide Web with Arbitrary Predicates, WWW Conference, 2001
[5] Porno Robot, http://www.allworldsoft.com/software/9-556-porno-robot.htm