In statistics, an effect size is a measure of the strength of the relationship between two variables in a statistical population, or a sample-based estimate of that quantity. Sample-based effect sizes are distinguished from test statistics used in hypothesis testing, in that they estimate the strength of an apparent relationship, rather than assigning a significance level reflecting whether the relationship could be due to chance. In scientific experiments and observational studies, it is often useful to know not only whether a relationship is statistically significant, but also the size of the observed relationship. In practical situations, effect sizes are helpful for making decisions, since a highly significant relationship may be uninteresting if its effect size is small. Effect size measures play an important role in meta-analysis studies that summarize findings from a specific area of research, as well as in statistical power analyses.
Summary
The concept of effect size appears in everyday language. For example, a weight loss program may boast that it leads to an average weight loss of 30 pounds. In this case, 30 pounds is an indicator of the claimed effect size. Another example is that a tutoring program may claim that it raises school performance by one letter grade. This grade increase is the claimed effect size of the program. These are both examples of "absolute effect sizes," meaning that they convey the average difference between two groups without any discussion of the variability within the groups. For example, if the weight loss program results in an average loss of 30 pounds, we do not know if every participant loses exactly 30 pounds, or if half the participants lose 60 pounds and half the participants lose no weight at all.
In inferential statistics, an effect size helps to determine whether a statistically significant difference is a difference of practical concern. Given a sufficiently large sample size, a statistical comparison will always show a significant difference unless the difference in the population from which the data are sampled is exactly zero. The effect size conveys whether an observed difference is substantively important. This is in contrast to a statistical significance test, which assesses whether a relationship could be due to chance, regardless of the strength of the apparent relationship in the data. In meta-analysis, effect sizes are used as a common measure that can be calculated for different studies and then combined into an overall summary.
The term effect size can refer to a statistic calculated from a sample of data, or to a parameter of a hypothetical statistical population. Conventions for distinguishing sample from population effect sizes follow standard statistical practices — one common approach is to use Greek letters like ρ to denote population parameters and Latin letters like r to denote the corresponding statistic; alternatively, a "hat" can be placed over the population parameter to denote the statistic, e.g. with ρ̂ being the estimate of the parameter ρ.
The term effect size can refer to a standardized measures of effect (such as r , Cohen's d , and odds ratio), or to an unstandardized measure (e.g., the raw difference between group means and unstandardized regression coefficients). Standardized effect size measures are typically used when the metrics of variables being studied do not have intrinsic meaning to the reader (e.g., a score on a personality test on an arbitrary scale), when results from multiple studies are being combined when some or all of the studies use different scales, or when it is desired to convey the size of an effect relative to the variability in the population.
Reporting effect sizes is considered good practice when presenting empirical research findings in many fields . Effect sizes are particularly prominent in social and medical research. Relative and absolute measures of effect size convey different information, and can be used complementarily. A prominent task force in the psychology research community expressed the following recommendation:
Always present effect sizes for primary outcomes...If the units of measurement are meaningful on a practical level (e.g., number of cigarettes smoked per day), then we usually prefer an unstandardized measure (regression coefficient or mean difference) to a standardized measure (r or d).
– L. Wilkinson and APA Task Force on Statistical Inference (1999, p. 599)
Like any statistical estimate, effect sizes are estimated with error, and may be biased unless the effect size estimator that is used is appropriate for the manner in which the data were sampled and the manner in which the measurements were made. Another issue that arises in many statistical settings is publication bias, which occurs when scientists only report results when the estimated effect sizes are large or are statistically significant. As a result, if many researchers are carrying out studies under low statistical power, the reported results are biased to be stronger than the true effects.
Types
Pearson r correlation
Pearson's correlation, often denoted r and introduced by Karl Pearson, is widely used as an effect size when paired quantitative data are available; for instance if one were studying the relationship between birth weight and longevity. The correlation coefficient can also be used when the data are binary. Pearson's r can vary in magnitude from −1 to 1, with −1 indicating a perfect negative linear relation, 1 indicating a perfect positive linear relation, and 0 indicating no linear relation between two variables. Cohen gives the following guidelines for the social sciences: small effect size, r = 0.1-.23; medium, r = 0.24-.36; large, r = 0.37 or larger .
A related effect size is the coefficient of determination (the square of r , referred to as " r -squared"). In the case of paired data, this is a measure of the proportion of variance shared by the two variables, and varies from 0 to 1. An r 2 of 0.21 means that 21% of the variance of either variable is shared with the other variable. The r 2 is positive, so does not convey the polarity of the relationship between the two variables.
Effect sizes based on means
A (population) effect size θ based on means usually considers the standardized mean difference between two populations
where μ 1 is the mean for one population, μ 2 is the mean for the other population, and σ is a standard deviation based on either or both populations.
In the practical setting the population values are typically not known and must be estimated from sample statistics. The several versions of effect sizes based on means differ with respect to which statistics are used.
This form for the effect size resembles the computation for a t-test statistic, with the critical difference that the t-test statistic includes a factor of
. This means that for a given effect size, the significance level increases with the sample size. Unlike the t-test statistic, the effect size aims to estimate a population parameter, so is not affected by the sample size.
Cohen's d
Cohen's d is defined as the difference between two means divided by a standard deviation for the data
What precisely the standard deviation s is was not originally made explicit by Jacob Cohen because he defined it (using the symbol "σ") as "the standard deviation of either population (since they are assumed equal)". Other authors make the computation of the standard deviation more explicit with the following definition for a pooled standard deviation
with
and s k as the mean and standard deviation for group k , for k = 1, 2.
This definition of "Cohen's d " is termed the maximum likelihood estimator by Hedges and Olkin, and it is related to Hedges' g (see below) by a scaling
Glass's Δ
In 1976 Gene V. Glass proposed an estimator of the effect size that uses only the standard deviation of the second group
The second group may be regarded as a control group, and Glass argued that if several treatments were compared to the control group it would be better to use just the standard deviation computed from the control group, so that effect sizes would not differ under equal means and different variances.
Under an assumption of equal population variances a pooled estimate for σ is more precise.
Hedges' g
Hedges' g , suggested by Larry Hedges in 1981, is like the other measures based on a standardized difference
but its pooled standard deviation s * is computed slightly differently from Cohen's d
As an estimator for the population effect size θ it is biased. However, this bias can be corrected for by multiplication with a factor
Hedges and Olkin refer to this unbiased estimator g * as d , but it is not the same as Cohen's d . The exact form for the correction factor J() involves the gamma function
<
Polaris Sample Size Calculators For Meaningful Surveys
Polaris presents some sample size calculator options for you to decide how big a survey you will need to obtain meaningful results. Try one!
Sample Size Calculator by Raosoft, Inc.
The sample size doesn't change much for populations larger than 20,000. What is the ... If you'd like to see how we perform the calculation, view the page source.
JavaStat -- Binomial Proportion Differences
Proportion Difference Power / Sample Size Calculation (Revised 10/30/2009 - Also display results of uncorrected ("classical") calculation.)
Sample Size Calculation - Proportions - Wolfram Demonstrations Project
This Demonstration uses a normal approximation to the binomial distribution to estimate the minimum sample size, , for detecting a change in the proportion of a population with a ...
Medical Statistics Online Help
There are many packages dedicated to performing sample size/power calculations. It is often possible to perform such calculations within standard statistical software, and stand ...
Sample Size Calculation
I am in the Measure phase of a project and used a 1-sample T-test to calculate if the target I have defined is significant different then the current baseline.
Vanderbilt Biostatistics : Main.PowerSampleSize
PS: Power and Sample Size Calculation PS: Power and Sample Size Calculation Get PS (3.3 MB) version 3.0, 2009 Release Notes. by William D. Dupont and Walton D. Plummer, Jr.
Rollin Brant's Home Page
My collection of Sample Size Calculators provides a set of simple-to-use JavaScript utilities for doing basic sample size calculations. I also provide some resources for ...
Sample Size Calculations
The leading source for professional development COURSES in statistics
Sample Size Calculation
I want to take out a random sample out of population of size N and I want to fix a characteristic rate say p in that sample (i.e. teh rate of possesing that character in sample is ...