Re: Boolean query algorithms

Sherrie Laraurens wrote:

Hi Brain,

I'm not interested in the variations of logical predicates you
presented, also my question relates to how said queries be them sql
or something else are actually executed in a timely fashion.

The phrase "I have written a boolean search engine myself" followed
by "It simply generates the SQL queries" basically means you didn't
write anything of importance just a DB front end and sql generator
for an actual db which is doing all the dirty work for you, I guess
anyone with a couple of hours on their hands and an SQL compatible db
can say that they too have written a search engine of sorts, but alas
this was not the answer(s) I was looking for.


Then your question relates to scaling and is nothing to do with boolean
searches at all.

Google use 'chunck servers'. The data is split into say 10,000 parts, each
part stored on 1 server. Each of those servers is then replicated by say
10 others. When you a query need answering it is sent to the relevant
chunk servers which process in parallel and then feed the results back
where they are combined.

For a single query this is slightly less than 10,000 times faster than
storing everything on a single machine with a large disk array.

For multiple parellel queries it is even faster as the query can be sent to
any of the 10 chunck servers that hold each chunk of data.

That would be just 1 datacentre, google then copy the whole lot over dozens
of datacentres (not a copy in reality as they all seem to hold different
data).

Note: The numbers I have quoted are guesses.


--
Brian Wakem
Email: http://homepage.ntlworld.com/b.wakem/myemail.png
.

Relevant Pages

  • Re: Official Status of SQLServer 2005 ADP
    ... I have said that the support for SQL passthrough ... queries under MDB was bad and worst than the one offered by ADP while you ... > attempt to "pass through" every Access query against a linked ODBC ...
    (microsoft.public.access.adp.sqlserver)
  • Re: "Query Too Complex" Errors
    ... few dozens of queries, in the middle of which there's a long chain of ... we've been having a lot of those "Query Too ... some of the complexity in the SQL ... SQL statement you are working on. ...
    (microsoft.public.access.forms)
  • Re: Dynamic query problem
    ... On Oct 17, 9:41 am, Andy Hull ... If we were to provide a fully featured dynamic query generator we would have ... Provided with already built queries which they can edit ... SQL and see where it is the same as for the other queries and where it ...
    (microsoft.public.access.queries)
  • Re: CONTAINS performance
    ... mark, FTS needs to be very carefully tuned to achieve second response times, ... see SQL Server 2000 BOL title "Full-text Search Recommendations" for more ... When you include the "TOP 100" in your query, you are in fact limiting the ... valid for SQL queries, they often do not apply to FTS queries because the FT ...
    (microsoft.public.sqlserver.fulltext)
  • Re: Populating a list -- learning Access
    ... It is kinda funny -- I started using databases in the early 80's with dBase and, for years, never knew that I knew SQL! ... If you are on a form or report, the most important property is the NAME, because that is how you refer to it in code. ... I don't mean to skip your responce below, but thanks to your help with SQL, I was able to get the query to pull the info the way I needed. ... Queries (just shows the QBE grid for convenience -- ...
    (microsoft.public.access.forms)
Loading