Re: VERY chalanging question

Post the SQL you have so far.

Also post the hardware specification you are using this on.

I regularly deal with queries that consume tables with multi-millions of
rows in seconds without problem, the size of your data looks to be around
190MBytes based on 4 columns of 25 characters, basically its piddly.

Tony.

--
Tony Rogerson
SQL Server MVP
http://sqlblogcasts.com/blogs/tonyrogerson - technical commentary from a SQL
Server Consultant
http://sqlserverfaq.com - free video tutorials


"groupy" <liav.ezer@xxxxxxxxx> wrote in message
news:1149010763.810314.63670@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
input: 1.5 million records table consisting users with 4 nvchar
fields:A,B,C,D
the problem: there are many records with dublicates A's or duplicates
B's or duplicates A+B's or duplicates B+C+D's & so on. Mathematicly
there are 16-1 posibilities for each duplication.

aim: find the duplicates & filter them, leave only the unique users
which don't have ANY duplication.

We can do it by a simple select query that logicly checks the
duplication in a OR operator.
But it takes about 16 days in a very fast PC.
The DB is in sql-server, converting it to Oracle might acomplish it to
8 days.

How can i do it in a few hours?
Remeber that filtering first the users with parameter A & than by
parameter B & so on will result an error in the final result because it
will loose the information regarding the filtered users - maybe in
parameter C they are equal to other users in the table...

THANK YOU



.

Relevant Pages

  • Re: Equivalent of Get Next Extended to retrieve only unique keys?
    ... Do you know of any good resources for the ins & outs of accessing a btrieve database through SQL to supplement Pervasive's documentation? ... sort by various columns of data and to also filter on the columns. ... doesn't contain dozens of entries for the same patient. ... level of duplication is low, then perhaps you can do a regular GNE ...
    (comp.databases.btrieve)
  • Re: Equivalent of Get Next Extended to retrieve only unique keys?
    ... The SQL interface will give you an easier way to sort the data by ... sort by various columns of data and to also filter on the columns. ... level of duplication is low, then perhaps you can do a regular GNE ...
    (comp.databases.btrieve)
  • Re: VERY chalanging question
    ... Moving it to Oracle won't buy you anything. ... SQL Server MVP ... there are 16-1 posibilities for each duplication. ... find the duplicates & filter them, ...
    (comp.databases.ms-sqlserver)
  • Re: Named Mistakes and Questionable Practices
    ... we would have Dr. Codd's "Degree of Duplication" operator that ... This is how he handled dups in RM II. ... lost out to Standard SQL. ... are stuck with the family legacy. ...
    (comp.databases.theory)
  • Re: please explain
    ... > the 2003 tests are in MCSA and MCDBA as is one of the SQL test. ... > duplication be allowed? ...
    (microsoft.public.cert.mcdba)