Re: significant test of the 2 approaches

nan wrote:
Well because I would like to know whether the results calculating
from the two methods are different or not.

I think what you mean is, "over different datasets, how do the two estimation methods differ?" That is, you are trying to characterize the difference between the two methods, and not the difference between two values you get from applying the two methods to any one dataset.

I can certainly repeat the calculation each time with the new 50 data
points. Then P(T>t) calculated using method A is sometimes larger
and sometimes smaller depending on the 50 data points selected.
That's why I was hoping to use bootstrap to determine whether
P(T>t) using both methods are statistically different.

It sounds like you're confusing the notion of "statistical significance" with the notion of the sampling distribution of an estimation method. It sounds like what you really want to do is a simulation experiment.

Let's say your estimation methods A and B were ways to estimate the mean of the distribution from which you are drawing data. To begin, you might pick a particular distribution, such as N(4,1). Then you'd generate a set of n values from that distribution and use the two methods to compute two estimates a_1 and b_1. Do that, say, 9999 more times to get {a_1, ... a_10000} and {b_1, ... b_10000}. You now have empirical estimates of the sampling distribution of your methods under the assumption of N(4,1) for sample size n. Use KSDENSITY to visualize. You can choose any way you like to decide whether A or B or neither is better. For example, which has a smaller bias? Which has a smaller mean squared error? And so on.

Now consider using different sample sizes or parameter values or distributions. The sampling distributions may well be different. And you get to decide what conditions are relevant to your decision of which method is better.

Hope this helps.

- Peter Perkins
The MAthworks, Inc.
.

Relevant Pages

  • Re: Help with BUGS model
    ... With the model where the Lambdas are drawn from only one gamma prior I ... sure why your MLE estimation procedure had problems. ... the expected value of the cell under the assumption of independence of ... The posterior distribution of lambda given N has a closed form. ...
    (sci.stat.math)
  • Least Squares solution for fitting beta distribution to empirical distrbution
    ... can use as a probability distribution using kernel density estimation. ... I'm wondering if there is an analytic solution to this. ...
    (sci.stat.math)
  • Bayesian estimation of structured correlation/covariance
    ... there are many delay elements that have are normally ... positive-definite correlation matrix symbolically and the ... I would like to apply bayesian parameter estimation for the unknown ... distribution with a chi-square distribution - in which I can formulate ...
    (sci.stat.math)
  • Re: percentiles estimation
    ... The estimation depends on the data. ... If the data set is quite nice, say has a shape of known distribution, ... using parametric method to estimate the distribution and calculating ... Subj: percentiles estimation ...
    (sci.stat.math)
  • Re: What are the Parameters for Algorithm AS 62 APPL. STATIST. (1973) VOL.22, NO.2
    ... > The description of array FRQNCY given is: ... >> c Users are much more likely to need the distribution function. ... >> C the two given sample sizes ...
    (sci.stat.math)