skip to main content
10.1145/1013367.1013529acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
Article

Distributed location aware web crawling

Published: 19 May 2004 Publication History

Abstract

Distributed crawling has shown that it can overcome important limitations of the today's crawling paradigm. However, the optimal benefits of this approach are usually limited to the sites hosting the crawler. In this work, we propose a location-aware method, called IPMicra, that utilizes an IP address hierarchy, and allows crawling of links in a near optimal location aware manner.

References

[1]
Odysseas Papapetrou, Stavros Papastavrou, and George Samaras. Distributed indexing of the web using migrating crawlers. In Proceedings of the Twelfth International World Wide Web Conference (WWW), 2003.
[2]
Odysseas Papapetrou, Stavros Papastavrou, and George Samaras. Ucymicra: Distributed indexing of the web using migrating crawlers. In Proceedings of the 7th East-European Conference on Advanced Databases and Information Systems, Dresden, Germany, 2003.

Cited By

View all
  • (2018)Label Count Algorithm for Web Crawler-Label CountSoftware Engineering10.1007/978-981-10-8848-3_50(525-531)Online publication date: 13-Jun-2018
  • (2009)Topical web crawling using weighted anchor text and web page change detection techniquesWSEAS Transactions on Information Science and Applications10.5555/1512690.15127026:2(263-275)Online publication date: 1-Feb-2009

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
WWW Alt. '04: Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters
May 2004
532 pages
ISBN:1581139128
DOI:10.1145/1013367
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 May 2004

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. distributed web crawling
  2. location aware web crawling

Qualifiers

  • Article

Conference

WWW04
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2018)Label Count Algorithm for Web Crawler-Label CountSoftware Engineering10.1007/978-981-10-8848-3_50(525-531)Online publication date: 13-Jun-2018
  • (2009)Topical web crawling using weighted anchor text and web page change detection techniquesWSEAS Transactions on Information Science and Applications10.5555/1512690.15127026:2(263-275)Online publication date: 1-Feb-2009

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media