Re: Sorted arrays

ruby@xxxxxxxxxx wrote:
> Thanks for the help. I switched it to use a Hash, and that performed
> MUCH better. The data is not coming from any SQL server--it is just
> a flat file. The Ruby script processes it into a format so that it
> can be BCP'd into MS SQL Server. And when the table that it gets
> BCP'd into has a unique constraint on the MD5 hash field--hence the
> need for me to eliminate the duplicate values beforehand. Granted, I
> could have changed the process of importing the data so that this
> unique constraint was not necessary, but it seemed better to me to
> eliminate the duplicates using Ruby in this case.

If you're on MS SQL Server there's another option: you can use an index
with IGNORE DUPLICATES: that way you don't have to preprocess the file at
all:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/tsqlref/ts_create_64l4.asp

Kind regards

robert

.

Relevant Pages

  • Re: Sorted arrays
    ... > unique constraint altogether and then just write something in T-SQL ... > to remove any duplicates after the data is BCP'd in. ... > duplicates from the .DAT file. ... >>> a format so that it can be BCP'd into MS SQL Server. ...
    (comp.lang.ruby)
  • Re: Modeling/Constraint question
    ... there are no duplicates in the keys; ... Enter: the surrogate key. ... there is a customer number. ... SQL Server community, why should believe anything else you say? ...
    (comp.databases.ms-sqlserver)
  • Re: Need More SQL Query NewID Help
    ... Looks like you have to refine the query to weed out the duplicates then. ... Hugo is Hugo Kornelis, another SQL Server MVP. ... Books Online for SQL Server 2005 at ...
    (comp.databases.ms-sqlserver)
  • how can I make this script shorter?
    ... Lowell Kirsh wrote: ... > I have a script which I use to find all duplicates of files within a ... existing python file duplicate detector. ... a hash. ...
    (comp.lang.python)
  • Re: File Duplication check
    ... Well, using a hash is the right way to go, but I don't understand why ... be duplicates (you really have to check both files against each other at ... tried MD5 hash, CRC check, SHA1. ... What is happening is MS Word is having some set of Metadata ...
    (microsoft.public.dotnet.languages.csharp)