Notice: This material is excerpted from Special Edition Using HTML, 2nd Edition, ISBN: 0-7897-0758-6. This material has not yet been through the final proof reading stage that it will pass through before being published in printed form. Some errors may exist here that will be corrected before the book is published. This material is provided "as is" without any warranty of any kind.
by Robert Meegan
Current estimates indicate that the Web has more than 20 million pages. Even using the fastest connection and taking just seconds to glance at each page would take a reader the better part of a decade to see them all, by which time perhaps ten times as many new pages would have appeared. Because of the scale of the Web, it is very unlikely that many people will find your page by pure chance. In fact, if your page has no links leading to it from other pages, it is very likely that no one will ever find it!
Because the reason to create a Web page in the first place is to exchange information, you probably want to encourage other people to visit your page. The best way to bring people to your page is to make the job of finding it as easy as possible.
In this chapter, you will learn about the following:
The type of the advertising that you do depends greatly on the nature of your page. If you are doing a page as a hobby, paying a thousand dollars to get a week's worth of exposure on one of the popular sites probably isn't worth the cost to you. On the other hand, if your site is the home page of a major multinational corporation, the attention that a professionally designed advertisement can bring more than justifies the expense.
Although I use the term advertising extensively in this chapter, most of the methods listed here are free. A better term might be Web page promotion.
With this point in mind, you should take the first step to advertising on the Web, which is to answer the following questions about your site:
If your page is just a hobby, where you share information with others who share your interests, you can mount a low-key advertising campaign. Most of the people who find your page will do so through links with other pages that cover the same topics. Think hard about the sites that you like to visit, and you'll probably find that most of your visitors like the same sites.
Even if you are starting a page just as a hobby, it doesn't need to end there. Many of the most successful aspects of the Web began as part-time activities.
Non-profit organizations can achieve tremendous exposure on the Web, far out of proportion to the amount of money invested. These organizations often have enough manpower to find a large number of free locations to advertise the site.
For a small business, the Web can be an excellent place to advertise. On the Web, unlike most other forms of advertising, even a small company can produce a presence that is equally impressive as is that of a huge conglomerate. Unlike the print world, the Web allows anyone access to full-color images regardless of budget. In the democratic world of the Web, all addresses are equally impressive, giving your company real estate that is just as valuable as that of your larger competitors.
If your company is a mail-order or service business that can support customers around the country, or even the world, investing a greater proportion of money and energy in Web advertising may well be worthwhile. If you work at one of these companies, you may want to consider using a commercial marketing service.
Most people find what they're looking for on the Web using one of the many available search tools. These systems are huge databases containing as many as 20 million Web pages, coupled with powerful indexing software that allows for quick searches. Many of the searchers are run on mainframe computers or large parallel processors that can handle hundreds of searches simultaneously.
In the beginning of the Web, the first search servers were run by universities, but now most of these early efforts have been taken over by private companies. What benefit do these companies find in providing free searches on these expensive computers? Advertising! The index sites are some of the most frequently visited on the Web, and the maintainers of these sites can charge high rates to the companies that advertise on these pages.
In WebTrack's study of Web advertising, they discovered that five of the top ten sites in terms of revenues from advertising were search tools.
In addition to the older sites, more than a hundred newer indexes are available. Some of them are restricted to a specific topic, and others are still very small, but all offer the opportunity to get your site noticed.
As you explore the Web, you soon discover that it possesses its own collection of fauna. The wildlife of the Web consists of autonomous programs that work their way across the millions of links that connect the sites, gathering information along the way.
These programs are known by such colorful names as robots, crawlers, walkers, spiders, worms, and (in the case of one Australian program) wombats. What do these wanderers do? Almost without exception, they arrive at a page and search it for any links to other pages. These new links are recorded and followed in turn. When all the links on a particular chain have been followed, the next path is restored from the database, and the process continues. Examples of these searchers are Lycos and Webcrawler. A large number of special-purpose wanderers are also used to generate statistics regarding the Web. These programs do not generate databases that can be used for text searching.
The alternatives to these crawlers are the structured systems. They store Web pages indexed against a series of categories and subcategories. The hierarchical nature of these systems appeals to many people who are more comfortable using an index where they can see all the categories.
The type of system on which you perform your searches is entirely up to your personal preferences. From the standpoint of advertising your site, you need to be aware of the differences. Some of the structured systems restrict you to a limited number of index entries. This limitation can mean that people who are looking for just the things that you offer may not find you because they are looking in the wrong place.
A complete listing of indexes would be out of date as soon as it was finished. New sites are added monthly, and even sites that are maintained by large corporations have disappeared. I have listed a few of the main sites in this section, but you should take the time to do some of your own searching when you decide to publish your pages.
In the big league is a handful of sites that can claim to have indexed a sizable portion of the Web. These sites are the most popular systems used by the majority of Web surfers. You need to register with these servers first to maximize your exposure. The following sites are the cream of the index crop at this time:
Fig. 20.1
Webcrawler is a Web spider that searches the Web for links to pages not in its index.
Fig. 20.2
On the other hand, Yahoo is a database that organizes sites based on a structure of categories.
Just below this first tier is a larger group of very good sites. These sites don't have the traffic that the bigger sites do, but that can change quickly. These sites tend to have loyal audiences who use them instead of the primary sites, often because of their indexing categories.
Registering with most of these servers is easy. Almost all sites have a page that you can access from the main page, with a form for adding your site to the system. If the server is a Web crawler, all you need to do is enter the URL for your page. The system then dispatches a robot to your page to examine it. This robot returns all the keywords found on your pages. Some servers also record the titles of your pages as well.
For the structured systems, you need to enter the categories under which you want your site listed. Most sites allow you to enter multiple categories, so you should prepare a list of the items that are most relevant before you start your submission. Some servers have a list of categories that are supported, whereas others leave the choice up to you. Before you enter your page, take some time to explore some of the sites that are already in the database. This way, you can get a feel for how the categories are arranged. Try to find at least a couple of sites that are similar to yours so that your page will not be lost in the wrong sections.
In this example, I walk you through the registration of a Web page on Yahoo! This example is a Web page that lists the naval battles in the American Civil War located at www.civilwar.org/pub/html/naval/home.html.
You add URLs to the Yahoo database using a forms-based interface.
After you select the categories, you need to write a brief description of your page.
Maybe this process seems like a lot of work to you. After all, you just spent the last three months huddled over your keyboard perfecting the last details on the perfect Web page, and now you want to take it easy for a while. Why worry about advertising your page when a handy little Web crawler is bound to wander by and index the whole thing for you?
The first problem with this approach is that a Web robot can't find your site unless it's linked to a page that the robot already knows about and is traversing.
The second problem is more subtle. When a Web crawler explores your page, it can get the information used in the index from many sources. Some servers use the page titles, others use the headings, still others actually scan the entire body and attempt to make sense of it. The problem with all these methods is that they may not find all the categories that can refer to your site. Often, the words that are used in headings or titles are not in their simplest form. It would be a shame not to be listed because the title of your page uses an irregular verb.
The location problem can be worked around in one of several ways, depending on the nature of the crawler. Some systems scan the newsgroups looking for new URLs to add to their search list. Others accept URLs that are e-mailed to them. Finally, you can just go to the searcher's home page and submit your page. Any or all of these methods invite the robots to pay a visit to your site. Of these methods, the most reliable by far is to drop in on the Web page of one or two of the more popular servers and submit your URL. Most of the Web crawlers are far easier to register with than are the structured systems.
Some of the earliest robots were poorly written and could swamp a server with hundreds of requests within seconds of each other. Fortunately, most recent robots are courteous enough not to overload their hosts. If your server does crash, check the logs for a single site that retrieved many documents within a short period of time. If such a site exists, try to contact the postmaster at the site that made the requests and let him or her know about the problems that you saw.
The problem with misindexing is a little more difficult to solve. Fortunately, most of the crawlers search the HTML source for a Web page looking for the META tag. If a keyword list is assigned to a META field, the crawler parses the keywords and uses them for the index.
Troubleshooting
I don't really want to have little autonomous programs wandering around in my system. How can I keep Web robots away?
You real cannot keep a robot out of a Web page that's visible to the rest of the Internet. However, many of the robot writers follow an unofficial standard. If you include the file /robots.txt on your server with the proper information in it, many robots read the file and follow your wishes.
The syntax for robots.txt is as follows: Create a line that begins with the field name User-agent:. This field must then contain the name of the robot that you want to restrain. You can have multiple User-agent fields, or if you want to exclude all agents not specifically mentioned in a User-agent field, you can use a field value of *. The line following each User-agent field should begin with the field name Disallow:. This field should contain a URL path. Any URL that begins with the path specified in the Disallow field should be ignored by the robot named in the User-agent field.
Some examples of this file are
# Any text that begins with a pound-sign is treated as a comment
User-agent: Webcrawler # This restriction will only apply to the robot named Webcrawler
Disallow: /webpages/data/ # Any URLs that begin with the path shown here will be skipped by Webcrawler
# This example is the universal "do not disturb" sign
User-agent: * # All robots
Disallow: / # Every URL begins with a / in the path
You do not always have to do all the work yourself. Several good services submit your pages to the major search systems for you. Many of these services have a charge for this function, but a few services are available for free.
Submit-it is a nice forms-based system that allows you to enter all the relevant data for your page, after which it registers you with your choices of more than a dozen popular search tools (see fig. 20.5). This service is provided for free and can help you to hit most of the major search sites. You can found it at http://www.submit-it.com.
Submit-it! Is a powerful tool for registering with multiple search engines quickly.
Just as with the search tools, any list of services on the Web is obsolete almost as fast as it is generated. Your best bet is to do a little checking on your own to see what else is out there. A good place to start looking is at the Web Announcements topic on Yahoo.
The big advantage of a submission server is that it cuts down on the amount of work that you have to do. The disadvantage is that your submissions are made automatically, using the same categories and keywords for each database. This is probably sufficient if your page is personal or is intended for a specific audience.
If your page is the Web presence for your company or your organization, spending the time learning each of the databases yourself is probably better, so that you can ensure that your listing ends up under the right headings. After all, the time required to submit your page is nothing compared to the hard work that you've put into making it as good as it is.
Many servers are dedicated to posting new listings. These systems may display a new site for many weeks or for just a few hours, depending on the nature of the server and the number of new submissions that it receives (see fig. 20.6). Some of these servers have strict rules regarding acceptability, so be sure to check them out thoroughly before submitting your site.
What's new pages are popular but can't be counted on for long-term results.
The advantage of these "what's new" services is that many people visit them on a regular basis, so getting listed can bring visitors to your site who might not have found it otherwise. The disadvantages are that your stay on the server is for a limited time, the traffic tends to be casual browsers rather than people with a strong interest, and some of the servers have an annoying habit of listing sites in alphabetical order. This last disadvantage may seem unimportant, but the Aardvarks Anonymous site is much more likely to get noticed than is the Zoological Society of Aardvark Fanciers.
One of the more amazing things to come out of the Web has been the tremendous proliferation of "Best of the Web" sites. These systems generate listings under a variety of names, such as What's Cool, What's Hot, Top 5%, Best of the Web, Hot Picks, and so on (see fig. 20.7). In practice, of course, the selection of pages for these lists is completely arbitrary. For example, with the rapid growth of the Web, it is unlikely that anyone has ever even visited 5 percent of the sites currently available, let alone enough to make a reasonable judgment of which are the very best.
Several organizations present "Best of the Web" awards.
So how are these lists maintained? In most cases, you can submit the URL of your site to the administrator of the list, and he or she visits your site and reviews it. If your site meets whatever selection criteria the list is based on, you get added to the list.
Some of these lists provide you with a small graphic that you can display on your page to indicate that you have been awarded the honor, and virtually all the lists include links to your page after you have been accepted.
What is the real value of these lists? In the cosmic scheme of things, very little. But some of these lists are well known, and many people use them as launching points for random surfing. If you have a general interest site, getting it listed on a couple of these pages can really boost your traffic.
Some examples of these sites are
Even more than the Web crawlers and structured systems, the primary method for traversing the Web is by using links found on other pages (see fig. 20.8). To expose your page to the maximum number of potential visitors, you should make an effort to get as many sites as possible to include links to your site.
Many sites have long lists of links to other related sites.
Most sites that cover a specific topic are more than pleased to include links to other sites that cover the same topic. By including as many links as possible, they make themselves more useful and, hence, more popular. To encourage people to link to your page, you need to identify sites that might be interested in linking to yours and then contact the site administrator.
The best way to find sites to contact about linking to your site is to surf the Web. Find sites that are of interest to you, and you'll probably find the sites that are of interest to people who would visit your site. Where should you start surfing? The same places that your visitors would.
Start with the indexes and see what's out there. Try several of the more popular ones, and be sure to try both structured systems and Web crawlers. If one of indexes is particularly useful, you know that it is a good place to register your site. The ones that aren't useful can wait before you submit to them.
After you find some sites, visit them and see what they have to offer. You're looking for sites that have a theme that is similar to yours, without being identical. For example, if your page contains links to everything that a person might ever want to know about hog farming, pages that might make good links to your page include general farming pages, pages that cover animal husbandry, and pages for companies that do business with hog farmers, including both suppliers and consumers. Other pages that also cover aspects of hog farming but that are not duplicates of yours would also be worth linking to.
The best way to get a link to your Web page is to simply contact the owner of the page that you'd like to be linked from and ask him or her to create the link. You can most easily accomplish this task by sending e-mail to the page author. In most cases, you should be able to find the address of the person who maintains the link on one of the pages at the site. Failing this, try sending e-mail to the address Webmaster at the site that interests you. Finally, if all else fails, you can examine the HTML source for the site's main page to see whether the author's address is included in a comment field.
Be sure to explain what your site is all about and to include the URL of the home page in your message. If the page that you want the other site to link to is not your home page, let the Webmaster know what the correct URL is. A brief (one-line) description of your page can save him or her some time when adding the link. Remember that the Webmaster is just as busy as you are and that anything that you can do to make his or her life easier will increase the chance that he or she links to your page.
Of course, you can expect that the person in charge of the other site will check out your page before adding a link. He or she will want to make sure that your page actually is what you say it is and that the quality is such that it will improve his or her site to be linked to yours.
To make your site more worthwhile for others to link to, the first step is to ensure that it is free of HTML errors and that it loads correctly. Have your site examined by other people from outside your site to check that all the images are available and that all the tags display in the proper format. No one wants to be associated with a site that is filled with sloppy work.
Second, include useful, current, and interesting information and images. No one wants to spend time downloading a site just to find that it contains a mess of outdated or boring gibberish. Links to shareware programs can also make your page more popular.
Before adding a link to download any program, be sure that the program explicitly states that it is for freeware or shareware distribution, particularly if it is not stored on your server.
Finally, make it attractive. Ask yourself if the page makes you want to read it; then get the opinions of some people you can trust.
An important step toward making your site successful is to include a number of links to other sites that might be of interest to visitors to your site. The entire concept of the Web revolves around the interconnection of millions of sites. Don't make your page a dead end.
Check occasionally that all the links on your page still lead somewhere. Pages maintained by other people may disappear, often without notice.
If your site is a personal page, include connections to pages of your friends and colleagues. A hobby site should include as many links to other sites with similar interests as you can find. Check the links to make sure that they point to pages that you want to be associated to; then include them.
Although you can certainly add a link to a page without the prior consent of the owner of the page, contacting the maintainer of the page to let him or her know of the new link is courteous. He or she may also have a preference as to which page you establish the link to.
Business and organization pages can include links to other sources of information related to your site. Including links to your competitors is not necessary, but having links that point to your suppliers and customers might be very effective. Encourage them to include reciprocal links back to your page. Remember that the most effective form of advertising is networking and that a link to your page is an implicit recommendation.
If your pages are focused on a specific topic, registering with any specialized index pages that cover your area of interest is well worth the time.
At present, you can find many sites for business-related topics. This fact isn't surprising, but what is amazing is the incredible variety of index pages available for other interests as well. A search of the Web turns up many specialized pages that contain dozens of links. The following are a few examples of these pages:
You can use specialized search tools such as Art Planet to locate your page successfully.
In the business world, pages exist for many different types of companies. You can see some of the tremendous variety in the following pages:
A Web site is very difficult to find in the vast reaches of the Internet. Fortunately, you can use public bulletin boards to broadcast information to a number of people at the same time. These public areas are known as newsgroups, and they serve as public forums for communications and debate.
Much like everything else on the Internet, these groups have their own rules and customs. Very broadly, they fall into two categories: open and moderated groups. Open groups are pretty much what they sound like, in that anyone can post a message. Unfortunately, this freedom often leads to a very low signal-to-noise ratio. Moderated groups require that all postings are passed through a moderator (or group of moderators) who screens the messages and removes off-topic messages. This process greatly improves the proportion of postings that are relevant to the subject of the newsgroup.
Regardless of the type of newsgroup, proper use can greatly increase the traffic at your Web site. By the same token, however, improper use can cause ill feelings and will not attract the visitors that you are looking for.
The first newsgroups to use when spreading the word about your new Web site are the announcement groups. These groups are dedicated to the purpose of broadcasting messages dealing with new sites and services (see fig. 20.10). Most of these groups are moderated and do an excellent job of keeping messages on topic.
Comp.infosystems.www.announce is the number one site for posting new sites.
The number one group for new Web page announcements is comp.infosystems.www.announce, which lists virtually every site that is submitted to it. The rules of this group are standard for many of the announcement groups. Postings should be relevant to the purpose of the group and should not have a commercial purpose other than the announcement of a Web site that provides further information about a commercial product or service. The message announcing the site should have the URL of the page clearly listed in the message, preferably on a separate line. The message should also include a clear but brief description of the nature of the site. Finally, the subject of the message should be clear and precise. It is recommended that the subject begin with a word or two that clearly defines your site. See the example shown in figure 20.11.
Fig. 20.11
This announcement is clear and concise, and it lets potential visitors know what they can expect.
Other good groups to announce in are comp.internet.net-happenings and misc.entrepreneurs (for business sites).
After you have posted on the announcement newsgroups, you should take some time to find any other groups that may involve topics covered in your Web site. Far more than 20 thousand newsgroups are operating right now, although your Internet provider may cache only a fraction of this number. With this kind of diversity present, finding the groups that most closely match your interests is normally not difficult.
After you narrow down the field to a small handful of groups, the next step is to read the various messages that are posted. Try to identify people who are regular posters, and look for threads that have a long life. This practice of reading messages on a group without posting is known as lurking. You lurk in a group to become more familiar with it before you post.
One of the features of many groups is the occasional posting of what is called a FAQ. This message is a list of Frequently Asked Questions, and reading it carefully can help you avoid asking any questions that might have been answered repeatedly in the past.
The primary benefit to lurking is that when you are ready to post messages, you can do so in a manner that is perceived as highly competent and professional.
After you do start posting, you should make a special effort to ensure that your posts are well written and on topic. Remember that you are not just carrying on a friendly conversation, but rather you are advertising your page. Avoid mentioning your Web site in the body of your posting, but include your signature at the end of the message. If your postings are worth reading, people will make an effort to visit your pages, too.
On the other hand, flames, messages that are rude or offensive, and those that are off the subject only make people upset.
One form of message posting is not recommended: Sending large numbers of messages to post on many different newsgroups regardless of the group's topic is known as spamming (see fig. 20.12). This kind of posting is a tremendous waste of bandwidth, and many people, particularly those who pay for their access based on time spent logged in, do not appreciate your postings.
The message about saving on long-distance charges is off-topic for this group and is an example of spamming.
An unfortunate side effect of spamming is that it tends to attract retaliation from residents of the Internet. This retaliation can begin at the annoyance level and rapidly escalate. To avoid any unpleasantness, you should follow the rules and act in a responsible manner.
The primary offense caused by spamming is the waste of huge amounts of storage space on computers around the world. To help curb the problem, many newsgroups have programs that can excise spamming messages automatically from the group, often before most people even see the message. One side effect of spamming is that you may become blacklisted, which can expose you to remarkable levels of harassment from cybervigilantes.
Although the idea of advertising on the Web is just starting to become popular, the potential is tremendous. A recent survey by WebTrack identified more than $12.4 million of spending in the fourth quarter of 1995. More than 250 active advertisers were found, some of whom had Web advertising budgets of over a half million dollars.
Although competing against a budget many times larger than yours may seem impossible, remember that in no other major media is the amount of money needed to reach thousands of potential customers so small. For a small- to medium-sized company, the Web can be an outstanding advertising bargain-if it is used carefully.
Many sites have more than one million visitors per day. This represents more potential customers than any method of advertising other than major magazines and newspapers. Most of these heavily visited sites recognize the value that their pages represent and have started to lease space on the pages. This space can vary from $250 per month up to $50,000 or more depending on the traffic at the site.
Although advertising at a site like one of the major search tools or on Netscape's home page might seem to be most effective, the most cost-effective sites may be those that have lower traffic counts. Look for sites that have traffic of at least a couple of thousand hits per day (see fig. 20.13). A number of services audit the number of visits (hits) that a site receives in a given period. Sites that subscribe to these services may be more expensive up front, but at least you'll know what you are paying for.
Advertising on a busy site can bring many visitors to your site.
When you sponsor a site, make certain that your ad falls within a part of the page that people will spend most of their time looking at. For the most part, this is the top of the screen, but if the primary feature of the page is a form field that needs to be filled out, this could be the best area. Also check that a text anchor to your site is visible if the graphic image is not loaded. Many people working on slower connections regularly turn off automatic loading of images. Finally, if a text-only version of the page exists, have the site put an anchor on that page as well.
A recent phenomena on the Web is the growth of the malls. These sites maintain links or pages for up to a couple of dozen different retail companies. The primary difference between these sites and the specialized directories is that the directories provide links to non-commercial sites as well.
At this time, the traffic on these sites has not yet reached the high levels that would command the top advertising rates. If your company deals primarily in mail-order retail or in services that can be performed remotely, exploring the possibility of opening a store front in one of these malls may be worthwhile (see fig. 20.14).
A storefront in an Internet mall can be a good way to raise the visibility of your site.
Here are some examples of Internet malls:
A Restaurant Netguide is a chance to show your menu to people before they even arrive in your city.
As you may have noticed, developing an effective Web advertising campaign can be a lot of work. For someone working on a page for a hobby, all this work might not matter much, but for a busy professional who is trying to build up business through the Web, the time required may be more than he or she can afford. One solution to this problem is to use a publicity consultant. This person is an agent who handles the details necessary for getting your site noticed.
Some of the functions that a publicity consultant should perform are as follow:
Before you enter into a contract with anyone, you should ask for references from previous clients. Contact the references and ask if the agent was aggressive in promoting their sites and prompt in communicating with the clients. Also take the time to look at the client sites themselves. Are they professional in appearance? Well-designed pages indicate that the client's recommendation should carry a solid weight.
The cost of consultant services can range from a few hundred dollars to tens of thousands, depending on the scope of the work. When you do contract with one of these consultants, consider the possibility of basing his or her fee on the amount of traffic that is received at your site. This approach requires that the agent put his or her money on the line and increases the incentive to provide good service. Be aware that this tactic may increase the cost of services to you because the consultant is now sharing some of the risk.
In this chapter, you've examined a number of ways that you can advertise your site on the Internet. This certainly does not mean that you can't increase your exposure through other methods. Indeed, if you don't use these other techniques, you may miss out on many opportunities.
The simplest of all of these methods is to include your site's URL in the signature on all your e-mail. Including your URL costs nothing and has the advantage of appearing before an audience that is (or at least should be) receptive to your message already.
You should also add your Web site URL to your business cards and stationery. In effect, your Web site is your office in cyberspace, and you should include its address alongside your physical office. Before you do so, however, remember to walk through your site carefully to check for a professional and finished appearance. You wouldn't invite potential clients into a half-finished office covered with graffiti, and you shouldn't show them your work in progress on the Web either.
Finally, include your URL on any ads that you might place in magazines, newspapers, or trade journals. For many people, the discovery of a Web site in an advertisement is an illicit thrill. It's a way of letting people with access to the Web feel that they're an exclusive group and that you're catering especially to them. Take advantage of the cachet that comes with being on the Web whenever you can.
Internet & New Technologies Home Page - Que Home Page
For technical support for our books and software contact support@mcp.com
© 1996, Que Corporation