Re: Will Googlebit crawl non-indexed files?
- From: Tonnie Lubbers <t.prasing@xxxxxxxxx>
- Date: Thu, 21 Feb 2008 18:49:46 +0100
dgarrard@xxxxxxxxxxxxxx schreef:
Hi, I have a database that generates stories and an associated image
that I wish to have indexed by Google and Google Images. Here is an
example: http://www.nufocusinc.com/sightings/aurifrons_KAN_3.html
These pages are autogenerated in the /sightings/ directory and I do
not want to have to maintain an index.html file that lists all the
files.
- My site root is successfully being crawled.
- My directory listing access in Apache for /sightings/ is forbidden
as in:
http://www.nufocusinc.com/sightings/
- I have added a link in my home page to a dummy file:
http://www.nufocusinc.com/sightings/google.html
So are questions are:
1) Do I have to allow directory listing access to that directory for
googlebot?
NO
2) Is the dummy file idea of any use?
NO
3) Do I have to have an index.html file in /sightings/ ?
YES. Preferably with links. The bot follows links, nothing else. It might pick up textual placed URl's and try to access them.
My goal is that the pages that are placed in the /sightings/ directory
are crawled and indexed by googlebot.
Then make a page where the bot can pick them up.
--
Webdesign: http://vision2form.nl/webontwerp/
Korte handleiding zoekmachine optimalisatie / gevonden worden:
http://vision2form.nl/webontwerp/gevonden-worden.html
Lifestyle - wonen reizen en genieten : http://vision4living.com
.
- References:
- Will Googlebit crawl non-indexed files?
- From: dgarrard
- Will Googlebit crawl non-indexed files?
- Prev by Date: Re: blocking robots.txt from non-robots
- Next by Date: Re: blocking robots.txt from non-robots
- Previous by thread: Re: Will Googlebit crawl non-indexed files?
- Next by thread: Re: Will Googlebit crawl non-indexed files?
- Index(es):
Relevant Pages
|
