Re: Help Please - Searching a large text file
- From: "Mike Williams" <Mike@xxxxxxxxxxxxxxxxx>
- Date: Wed, 5 Jul 2006 21:58:08 +0100
"Jem" <xxxxxxxxxx> wrote in message news:ArSdnTqz8f50jDHZnZ2dnUVZ8qidnZ2d@xxxxxxxxxxxx
I'm a novice VB hobbyist/programmer and i'm trying my
best to write a small program for . . .
First let me give you a tip. If you want the best chance of getting a response then it is wise to post your code samples in such a way that it is easy for others to paste them into a VB project and run them. When I paste your code into a project there are all sorts of errors, most caused by the fact that many of your lines of code have been "hard wrapped" by your newsreader. To avoid this it is always best to adjust your code lines by inserting the "space underscore" character pair to break longer lines up into multiple shorter lines. For example, see how the following single line of VB code is broken up into three smaller lines:
lstItemsToOrder.AddItem ItemData(0) & vbTab & _
ItemData(1) & vbTab & ItemData(2) & vbTab & _
ItemData(3) & vbTab & ItemData(4) & vbTab & ItemData(5)
The file takes a short while to read into memory as it contains
approx 60,000 separate product lines. The time this takes isn't
really too much of a problem as it's at program startup . . .
Actually you can speed that part of it up about three or four times by writing it in such a way that it does not Redim the array every single time through the loop. This wastes a lot of time, and is unnecessary. Instead, for the amount of lines you are dealing with, you should create an array with about 500 elements to start with, and only Redim it when the loop counter exceeds 500, adding a further 500 elements each time. Finally Redim at the end to the correct size. In this way the number of Redims will be only a tiny fraction of the original. This is how to do it:
Dim x As Long, DataString As String
Dim LensDataArray() As String, FileNumber As Long
FileNumber = FreeFile
Open "c:\Barcode Data.txt" For Input As #FileNumber
x = 0
ReDim LensDataArray(0 To 500)
Do While Not EOF(1)
If x > UBound(LensDataArray) Then
ReDim Preserve LensDataArray(UBound(LensDataArray) + 500)
End If
Line Input #FileNumber, LensDataArray(x)
DataString = LensDataArray(x)
x = x + 1
Loop
ReDim Preserve LensDataArray(x - 1)
Close #FileNumber
Also, when searching through the array you are using Split on every line to split it up into an array of separate items. That is going to be very time consuming, and again is unnecessary. You only need to do this Split on elements that match your search criteria, which will save you a lot of time. So, instead of using . . .
ItemData() = Split(LensDataArray(x), ",")
If txtBarcodeData.Text = ItemData(0) Then
you should do . . .
If txtBarcodeData.Text = Left$(LensDataArray(x), _
InStr(LensDataArray(x), ",") - 1) Then
The above code compares the textbox text to the first part of the string (up to but not including the comma) using Left$ in conjunction with Instr, and thereby avoids performing a Split.
Having said that, your search would work very much faster if the array was ordered. The first part of the string (the barcode) is the part you are searching on, so write a small app to load the file and sort the lines and then save the file in sorted order. Then your main app when it loads the file can perform a binary search, which will be orders of magnitude faster than a line by line search. To find a specific string in 60,000 strings takes an average of 30,000 compares, whereas to do exactly the same on an ordered array of 60,000 strings can take as few as 16 compares.
Mike
.
- Follow-Ups:
- References:
- Prev by Date: Help Emailing File attachment Please
- Next by Date: Re: Help Please - Searching a large text file
- Previous by thread: Re: Help Please - Searching a large text file
- Next by thread: Re: Help Please - Searching a large text file
- Index(es):
Relevant Pages
|
