The researcher should always keep in mind the following limitations of the tests:
In the first place, test should not be used mechanically. It is not just decision making tool; in fact it is the only decision making mechanism. Therefore, proper interpretation of statistical evidence is important to make intelligent decisions.
Tests are unable to explain the reason for the existing differences like between the means of the two samples. They only show whether the difference is because of fluctuations of sampling or due to other reasons but fail to tell as to which the other reason is causing the difference.
Results coming up from the significance tests are based on probabilities and therefore cannot be expressed with full certainty. If a test shows that a difference is statistically important, it simply suggests that the difference in all probabilities is not due to chance.
Statistical inferences which are based on the significance tests cannot be said to be completely correct evidence regarding the truth of the hypotheses.It is all the more because of small samples where the probability of drawing erring inferences happens to be generally high. Therefore, for greater reliability, the size of samples should be sufficiently large.
All these limitations advice that in problems related to statistical importance, the inference techniques should be combined with sufficient knowledge of the subject matter with the capability of good judgment.