Re: comparison of large data sets
- From: "Darek" <dooverone@xxxxx>
- Date: 27 Sep 2005 01:56:21 -0700
I try to describe whole situation.
I have 5 systems with similar amount of data.
One of them is a master, so I want to check differences between master
and each of the others. There must be 4 comparisons.
So Master system takes a part in 4 comparisons.
I have flat files of exported data from each systems. I'm not able to
access the data directly from databases.
Report is executed once in a month. Every month I get new data, which
partly are different, especially states.
> Yes a database will likely do it faster than a simple standalone
> program. A good choice would be (if available) one of the better DBMS
> products like ORACLE, DB2, SQL Server, FIrebird/Interbase, and a few
> others. THey all should be able to handle this volume of data.
I do some tests now.
It is difficult to persuade the customers to buy oracle database, so I
try something open source now:)
>
> >
> > Do you have any point of view, what shell I use to do this
>
> If you really are using shell scripts, I'd say avoid those. If you
> really want to "script" this, use PERL.
I was thinking of Perl, or python:)
I wonder if it should be faster to do it in database or use sort
algorithms implemented in C or Perl.
.
- References:
- comparison of large data sets
- From: Darek
- Re: comparison of large data sets
- From: Ed Prochak
- comparison of large data sets
- Prev by Date: database or text-retrieval system
- Next by Date: upgrade from Oracle 8.1.6.3.0 to 10g. Help required
- Previous by thread: Re: comparison of large data sets
- Next by thread: Re: comparison of large data sets
- Index(es):
Relevant Pages
|
|
