Re: Help Please - Searching a large text file

Thanks ever so much Mike.

I take on board what you say about the way I presented my code and I thank
you for your suggestions, I will do my best to do as you suggest in the
future. I'm afraid it's purely a case of inexperience on my part.

Thank you also for your code. Again I really appreciate you taking the time
to help me out.

Many many thanks.

Jem


"Mike Williams" <Mike@xxxxxxxxxxxxxxxxx> wrote in message
news:e8h8rs$7g6$1@xxxxxxxxxxxxxxxx
"Jem" <xxxxxxxxxx> wrote in message
news:ArSdnTqz8f50jDHZnZ2dnUVZ8qidnZ2d@xxxxxxxxxxxx

I'm a novice VB hobbyist/programmer and i'm trying my
best to write a small program for . . .

First let me give you a tip. If you want the best chance of getting a
response then it is wise to post your code samples in such a way that it
is easy for others to paste them into a VB project and run them. When I
paste your code into a project there are all sorts of errors, most caused
by the fact that many of your lines of code have been "hard wrapped" by
your newsreader. To avoid this it is always best to adjust your code lines
by inserting the "space underscore" character pair to break longer lines
up into multiple shorter lines. For example, see how the following single
line of VB code is broken up into three smaller lines:

lstItemsToOrder.AddItem ItemData(0) & vbTab & _
ItemData(1) & vbTab & ItemData(2) & vbTab & _
ItemData(3) & vbTab & ItemData(4) & vbTab & ItemData(5)


The file takes a short while to read into memory as it contains
approx 60,000 separate product lines. The time this takes isn't
really too much of a problem as it's at program startup . . .

Actually you can speed that part of it up about three or four times by
writing it in such a way that it does not Redim the array every single
time through the loop. This wastes a lot of time, and is unnecessary.
Instead, for the amount of lines you are dealing with, you should create
an array with about 500 elements to start with, and only Redim it when the
loop counter exceeds 500, adding a further 500 elements each time. Finally
Redim at the end to the correct size. In this way the number of Redims
will be only a tiny fraction of the original. This is how to do it:

Dim x As Long, DataString As String
Dim LensDataArray() As String, FileNumber As Long
FileNumber = FreeFile
Open "c:\Barcode Data.txt" For Input As #FileNumber
x = 0
ReDim LensDataArray(0 To 500)
Do While Not EOF(1)
If x > UBound(LensDataArray) Then
ReDim Preserve LensDataArray(UBound(LensDataArray) + 500)
End If
Line Input #FileNumber, LensDataArray(x)
DataString = LensDataArray(x)
x = x + 1
Loop
ReDim Preserve LensDataArray(x - 1)
Close #FileNumber

Also, when searching through the array you are using Split on every line
to split it up into an array of separate items. That is going to be very
time consuming, and again is unnecessary. You only need to do this Split
on elements that match your search criteria, which will save you a lot of
time. So, instead of using . . .

ItemData() = Split(LensDataArray(x), ",")
If txtBarcodeData.Text = ItemData(0) Then

you should do . . .

If txtBarcodeData.Text = Left$(LensDataArray(x), _
InStr(LensDataArray(x), ",") - 1) Then

The above code compares the textbox text to the first part of the string
(up to but not including the comma) using Left$ in conjunction with Instr,
and thereby avoids performing a Split.

Having said that, your search would work very much faster if the array was
ordered. The first part of the string (the barcode) is the part you are
searching on, so write a small app to load the file and sort the lines and
then save the file in sorted order. Then your main app when it loads the
file can perform a binary search, which will be orders of magnitude faster
than a line by line search. To find a specific string in 60,000 strings
takes an average of 30,000 compares, whereas to do exactly the same on an
ordered array of 60,000 strings can take as few as 16 compares.

Mike





.

Relevant Pages

  • Re: Help Please - Searching a large text file
    ... Actually you can speed that part of it up about three or four times by writing it in such a way that it does not Redim the array every single time through the loop. ... Dim LensDataArrayAs String, FileNumber As Long ...
    (comp.lang.basic.visual.misc)
  • Re: ReDim an array AFTER its loaded into another array
    ... '.RteInfo2 as string 'etc depending on how many pieces of data you need ... Redim vRteFactsAy ... ' RFA is the route facts array ... Once a "master" array is formed, it looks like the component arrays ...
    (microsoft.public.excel.programming)
  • Re: Error on UBound with Dynamic Array
    ... determine the difference in length of the string. ... Doug Steele, Microsoft Access MVP ... ReDim Preserve arrSplit+ 1) ... need to use a dynamic array and may be checking the boundries, ...
    (microsoft.public.access.modulesdaovba)
  • Re: Error on UBound with Dynamic Array
    ... Hadn't realized ReDim was 'expensive'. ... determine the difference in length of the string. ... need to use a dynamic array and may be checking the boundries, ... If locEnd = 0 Then ...
    (microsoft.public.access.modulesdaovba)
  • Re: Finding out if array has been dimensioned??
    ... You have to redim the array to the appropriate size before you start reading ... > Problem is when I come accross a doc that has no custom styles ... > Open filename For Output As #filenumber ...
    (microsoft.public.word.vba.general)