- Topic Directories: Overview
- Topic Directories: Facts and Information
- Topic Directories: Tutorial and Course
- Topic Directories: References
Topic Directories: Overview
Topic Directories: Facts and Information
During the early days of the Internet, directory submissions used to play a substantial role in search engine rankings. In today's more complex world, search engines have gotten much more sophisticated in the way they determine rankings. That being said, there is still some value to submitting your website to directories.
Topic Directories: Tutorial and Course
The first wave of innovations was related to basic infrastructure comprising crawlers and search engines. Topic directories were the next significant feature to gain visibility.
In 1994, Jerry Yang and David Filo, Ph.D. students at Stanford University, created the Yahoo! directory to help people locate useful Web sites, growing by the thousands each day. Srinija Srinivasan, another Stanford alum, provided the expertise to create and maintain the treelike branching hierarchies that steer people to content. By April 1995, the project that had started out as a hobby was incorporated as a company.
The dazzling success of Yahoo! should not make one forget that organizing knowledge into ontologies is an ancient art, descended from philosophy and epistemology. An ontology defines a vocabulary, the entities referred to by elements in the vocabulary, and relations between the entities. The entities may be fine-grained, as in WordNet, a large lexical database for English, which is developed by Princeton University, or they may be relatively coarse-grained topics, as in the Yahoo! topic directory.
The paradigm of browsing a directory of topics arranged in a tree where children represent specializations of the parent topic is now pervasive. The average computer user is familiar with hierarchies of directories and files, and this familiarity carries over rather naturally to topic taxonomies. Following Yahoo!'s example, a large number of content portals have added support for hosting topic taxonomies. Some organizations employ a team of editors to maintain the taxonomy; others (e.g., the DMOZ is more decentralized and work through a loosely coupled network of volunteers. A large fraction of Web search engines now incorporate some form of taxonomy-based search as well.
Topic directories offer value in two forms. The obvious contribution is the cataloging of Web content, which makes it easier to search (e.g., SEO is an abbreviation of Search Engine Optimization is easier to distinguish from a Korean family name). Collecting links into homogeneous clusters also offers an implicit “more like this” mechanism: once the user has located a few sites of interest, others belonging to the same, sibling, or ancestor categories may also be of interest. The second contribution is in the form of quality control. Because the links in a directory usually go through editorial scrutiny, however cursory, they tend to reflect the more authoritative and popular sections of the Web. As we shall see, both of these forms of human input can be exploited well by Web mining programs.