Recommended Affiliate Programs
Robots, Spiders and Crawlers - Oh My!
By - Jd Reilly
From the
title you're probably thinking, "What does this have to do with internet
marketing?" Actually, it has everything to do with internet marketing. And
it has to do with search engines or you may know it as web search engines. Still
don't know? Of course you have heard of Google, Yahoo and others that bring your
website to the notice of the prospective customers. They present information to
your potential customers initiating a search.
So, what are
these robots, spiders and crawlers? Robots or bots are software applications
that run automated tasks over the Internet. Our friends at Wikipedia state,
"...bots perform tasks that are both simple and structurally repetitive, at
a much higher rate than would be possible for a human alone." These robots
are also being used as an access to content for media delivery.
Spiders are
automated programs run by the search engine systems and are used to index
websites. Basically it works like this, when you submit your website to a search
engine a spider or automated program will index your entire site. It does so by
reading the content or your site, your site's meta tags and will follow the
links that the site connects. The spider then returns all that information back
to a central depository, where the data is indexed. It will visit each link you
have on your website and index those sites as well. Some spiders will only index
a certain number of pages on your site, so don't create a site with 500 pages!
The spider
will periodically return to your site to check for any information that has
changed. The frequency with which this happens is determined by the moderators
of the search engine.
A spider is
almost like a book where it contains the table of contents, the actual content
and the links and references for all the websites it finds during its search,
and it may index up to a million pages a day.
Crawlers or
web crawlers are similar to spiders but as explained in Wikipedia, "...are
mainly used to create a copy of all the visited pages for later processing by a
search engine that will index the downloaded pages to provide fast searches.
Crawlers can also be used for automating maintenance tasks on a website, such as
checking links or validating HTML code. Also, crawlers can be used to gather
specific types of information from web pages, such as harvesting e-mail
addresses (usually for spam)."
What Does All
of This Mean?
When you ask
a search engine to locate information, it will actually search through the index
which it has created and not actually searching the Web. Different search
engines produce different rankings because not every search engine uses the same
algorithm to search through the indices.
One of the
things that a search engine algorithm scans for is the frequency and location of
keywords on a web page, but it can also detect artificial keyword stuffing or
spamdexing (this is where you repeat your keywords or phrases to manipulate the
relevancy or prominence of your resources.) Then the algorithms analyze the way
that pages link to other pages in the Web.
A search
engine can determine what a page is about and whether the keywords or phrases of
the linked pages are the same as the keywords on the original page. How do they
accomplish this? By checking how pages link to each other. It is basically a
check and balance system.
With this
knowledge you now understand the meaning of "search engine
optimization." You will be better equipped in your content writing with no
need to fear...robots, spiders and crawlers.
----------------------------------------------------------------
About The
Author:
JD Reilly of http://jdreilly.biz is committed to helping others succeed at internet marketing with tested and proven products and strategies. The success is built upon-A focused target market, A product people are hungry for, A marketing strategy and Automation. Discover the Easy way to become a Successful Internet Marketer with Expert Online Video Training at http://www.ebizdynasties.com
********************************************
Home Marketing Tips Helpful Resources Added Resources Articles
Helpful Links Recommended Affiliate Programs Free Newsletter
Contact Us Reciprocal Links Submit An Article
Copyright © AHomeBusinessOpportunity.net, All Rights
Reserved.
Unauthorized use of the contents or design of this website is prohibited.
You may not duplicate, or publish any of our materials without our express legal
consent.