Re: dealing with a time series piece wise (with gaps)
- From: "Ashish Uthama" <first.last@xxxxxxxxxxxxx>
- Date: Wed, 18 Mar 2009 13:24:17 -0400
You do not mention the size of the text file, if its small enough to fit in memory, using LOAD and then working off the data in memory might be faster than:
Another approach would be to read the file line by line, and processing one segment at at time. However, since you will not know how much data to expect in a segment you will not be able to pre-allocate, this might slow your code.
On Tue, 17 Mar 2009 09:04:42 -0400, Yaj Bhattacharya <yajnaval@xxxxxxxxx> wrote:
Could someone help translate the following pseudocode to matlab?
You might be better off giving it a try and asking for help with any specific code you have written.
1. Read in a time series in a columnar format from an external file
(very easy, but looking to optimize the rest of the steps, load
command may or may not be inefficient here?)
given data in three columns in an external file: time, value,
error
Look at TEXTSCAN, this lets you read in one line at a time.
2. Check for discontinuities in data and store in pieces:
if [time(i+1) - time(i)] > some_value, then
store this piece of triplet array (time,
value, error) and assign this piece an address (say piece_1)
continue reading until another discontinuity is
encountered
store the new piece of triplet array with
new address (piece_2) until end of file
If you just work with one segment at a time, you wouldnt have to worry about 'new addresses'. Keep reading in the next line and storing it in a buffer till you hit a discontinuity.
4. Detrend each piece deduced in step 2 by subtracting a line-of-best-
fit, taking into account the errors
Without knowing much, I would suggest looking at the function DETREND (wasnt that easy?)
5. Find the Fourier coefficients for each piece after detrend step 4.
6. Write out each piece from step 5 as separate files with meaningful
filenames based on the time of the array above, so the file written
for piece_1 would be called "time(i)_time(i+n).dat" where the time
array is a continuous array (no gaps greater than some_value deduced
in step 2).
You could use the time (assuming its a number) as a string for the file name, look at NUM2STR.
fileName=['dataFor',num2str(time(i)),'_',num2str(time(i+n))];
then use FOPEN (look at 'wt' mode), and FPRINTF to write the data to the file.
Now you can continue with the TEXTSCAN loop above to read in the next data segment.
.
- References:
- dealing with a time series piece wise (with gaps)
- From: Yaj Bhattacharya
- dealing with a time series piece wise (with gaps)
- Prev by Date: Re: Calculating the Product of Two Gaussians
- Next by Date: Filters in digital signal processing
- Previous by thread: dealing with a time series piece wise (with gaps)
- Next by thread: Hidden Markovian Model
- Index(es):
Relevant Pages
|
Loading
