Re: Do HTTP headers get spidered?



__/ [ canadafred ] on Friday 31 March 2006 18:44 \__

"Sorcerdon" <sorcerdon@xxxxxxxxx> wrote in message
news:1143823597.711292.23840@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Im using classic ASP.
Do strings in my HTTP headers get spidered.


No, but they should be interpreted for instructions, I imagine.


Code should always be considered visible to an intelligent spider.


By "spidered", I suppose the intent is to say "recorded", in which case the
answer is probably not. However, it would be nice to keep track of
redirection information. That might get stored in the crawler's server, but
never made public. No-one knows. Not me anyway...


Should I have my HTTP headers show
search engine optimization type strings?

I suppose it depends on the intent; whether you want to manipulate the
search engine or simply guide it.


How do you reckon you can SEO HTTP header? Identify oneself as the key words?

Best wishes,

Roy

--
Roy S. Schestowitz
http://Schestowitz.com | SuSE Linux ¦ PGP-Key: 0x74572E8E
7:50pm up 23 days 9:35, 10 users, load average: 0.67, 1.12, 1.05
http://iuron.com - Open Source knowledge engine project
.



Relevant Pages

  • Re: Do HTTP headers get spidered?
    ... Do strings in my HTTP headers get spidered. ... redirection information. ... search engine or simply guide it. ...
    (alt.internet.search-engines)
  • Re: Do HTTP headers get spidered?
    ... Do strings in my HTTP headers get spidered. ... redirection information. ... search engine or simply guide it. ... associated with server configuration and the network layer. ...
    (alt.internet.search-engines)
  • Do HTTP headers get spidered?
    ... Do strings in my HTTP headers get spidered? ... Should I have my HTTP headers show search engine optimization type ...
    (alt.internet.search-engines)
  • Re: fast text parsing
    ... >>Say I'm parsing stuff like http headers, what is going to give better ... Strings with regular expressions? ... It all 'depends' :-) If you're doing http header parsing, ... a parser class to the mongrel parser to do this based on the existing ...
    (comp.lang.ruby)