Harvey Motulsky's GraphPad Blog

Events (deaths) at time zero in survival analysis.

2010-03-15T17:48:09-07:00

When analyzing survival data, Prism simply ignores any rows with X=0. Our thinking is simple. If alternative treatments begin at time zero, then a death right at the moment treatment begins provides no information to help you decide which of two treatments is better. There is no requirement that X be an integer. If a death occurs half a day into treatment, and X values are tabulated in days, enter 0.5 for that subject.

Some fields (pediatric leukemia is one) do consider events at time zero to be valid. These studies to not simply track death, but track time until recurrence of the disease. But disease cannot recur until it first goes into remission. In the case of some pediatric leukemia trials, the treatment begins 30 days before time zero. Most of the patients are in remission at time zero. Then the patients are followed until death or recurrence of the disease. But what about the subjects who never go into remission? Some investigators consider these to be events at time zero. Some programs, we are told, take into account the events at time zero, so the Kaplan-Meier survival curve starts with survival (at time zero) of less than 100%. If 10% of the patients in one treatment group never went into remission, the survival curve would begin at Y=90% rather than 100%.

We have not changed Prism to account for deaths at time zero for these reasons:

We have seen no scientific papers, and no text books, that explains what it means to analyze deaths at time zero. It seems far from standard.
It seems wrong to combine the answers to two very different questions in one survival curve: What fraction of patients go into remission? How long do those in remission stay in remission?
If we included data with X=0, we are not sure that the results of the survival analysis (median survival times, hazard ratios, P values, etc.) would be meaningful.

The fundamental problem is this: Survival analysis analyses data expressed as the time it takes until an event occurs. Often this event is death. Often it is some other well defined event that can only happen once. But usually the event is defined to be something that could possibly happen to every participant in the trial. With these pediatric leukemia trials, the event is defined to be recurrence of the disease. But, of course, the disease cannot recur unless it first went into remission. That leads to the problem of how to analyze the data from patients who never go into remission.

We are willing to reconsider our decision to ignore, rather than analyze, survival data entered with X=0. If you think we made the wrong decision, please let us know. Provide references if possible.

There is a simple work around if you really want to analyze your data so deaths at time zero bring down the starting point below 100%, enter some tiny value other than zero. Enter these X values, say, as 0.000001. An alternative is to enter the data with X=0, and then use Prism's transform analysis with this user-defined transform:

X=IF(X=0, 0.000001, X)

In the results of this analysis, all the X=0 values will now be X=0.000001. From that results table, click Analyze and choose Survival analysis.

Independence

2010-03-15T12:58:27-07:00

The concept of independence is key to almost all statistical inference. But it is a tricky concept to understand. Here is an amusing article that explains the concept in a fun context: Eggs-actly what ARE the chances of a double-yolker?

Skewness

2010-03-04T06:45:37-07:00

Skewness quantifies the asymmetry of a distribution of a set of values. GraphPad Prism can compute the skewness as part of the Column Statistics analysis.

How skewness is computed

Understanding how skewness is computed can help you understand what it means. These steps compute the skewness of a distribution of values:

We want to know about symmetry around the sample mean. So the first step is to subtract the sample mean from each value, The result will be positive for values greater than the mean, negative for values that are smaller than the mean, and zero for values that exactly equal the mean.
To compute a unitless measures of skewness, divide each of the differences computed in step 1 by the standard deviation of the values. These ratios (the difference between each value and the mean divided by the standard deviation) are called z ratios. By definition, the average of these values is zero and their standard deviation is 1.
For each value, compute z³. Note that cubing values preserves the sign. The cube of a positive value is still positive, and the cube of a negative value is still negative.
Average the list of z³ by dividing the sum of those values by n-1, where n is the number of values in the sample. If the distribution is symmetrical, the positive and negative values will balance each other, and the average will be close to zero. If the distribution is not symmetrical, the average will be positive if the distribution is skewed to the right, and negative if skewed to the left. Why n-1 rather than n? For the same reason that n-1 is used when computing the standard deviation.
Correct for bias. For reasons that I do not really understand, that average computed in step 4 is biased with small samples -- its absolute value is smaller than it should be. Correct for the bias by multiplying the mean of z³by the ratio n/(n-2). This correction increases the value if the skewness is positive, and makes the value more negative if the skewness is negative. With large samples, this correction is trivial. But with small samples, the correction is substantial.

Interpreting skewness

The basics:

A symmetrical distribution has a skewness of zero.
An asymmetrical distribution with a long tail to the right (higher values) has a positive skew.
An asymmetrical distribution with a long tail to the left (lower values) has a negative skew.
The skewness is unitless.
Any threshold or rule of thumb is arbitrary, but here is one: If the skewness is greater than 1.0 (or less than -1.0), the skewness is substantial and the distribution is far from symmetrical.

How useful is it to assess skewness? Not very, I think. The numerical value of the skewness does not really answer any of these questions:

Does the distribution deviate enough from a Gaussian distribution, that parametric tests will give invade results?
Would the distribution be closer to Gaussian if the data were transformed by taking the logarithm (or reciprocal, or another transform) of all the values?
Is the skewness due to one or a few outliers?

The skewness doesn't directly answer any of those questions. Note that the D' Agostino and Pearson omnibus normality test (a choice within Prism's column statistics analysis) is a normality test that combines the skewness with the kurtosis (a measure of how far the shape of the distribution deviates from the bell shape of a Gaussian distribution), and so tries to answer the first question.

The definition of the skewness is part of a mathematical progression. The standard deviation is computed by first summing the squares of he differences each value and the mean. The skewness is computed by first summing the cube of those distances. And the kurtosis is computed by first summing the fourth power of those distances.

While there are good reasons for computing the standard deviation by squaring the deviations, there doesn't appear to be a deeper meaning to summing the cube of the differences between each value and the mean. Since the skewness is computed based on cubes, a value that is twice as far from the mean as another value increases the skewness eight times as much as that other value (because 2³=8). I don't see why alternative definitions of skewness where that factor is some other value (4, or 7 or 10 or any other value greater than 1) wouldn't be just as informative and useful.

Multiple definitions of skewness

Skewness has been defined in multiple ways. The method used by Prism (and described above) is the most common method. It is identical to the skew() function in Excel. This value of skewness is often abbreviated g₁.

The confidence interval of skewness

Whenever a value is computed from a sample, it helps to compute a confidence interval. In most cases, the confidence interval is computed from a standard error. The standard error of skewness (SES) depends on sample size. Prism does not calculate it, but it can be computed easily by hand using this formula:

The margin of error equals 1.96 times that value, and the confidence interval for the skewness equals the computed skewness plus or minus the margin of error. This table gives the standard error and margin of error for various sample sizes.

n	SE of skewness	Margin of error
3	1.225	2.400
4	1.014	1.988
5	0.913	1.789
6	0.845	1.657
7	0.794	1.556
8	0.752	1.474
9	0.717	1.406
10	0.687	1.347
15	0.580	1.137
20	0.512	1.004
25	0.464	0.909
50	0.337	0.660
100	0.241	0.473
200	0.172	0.337
300	0.141	0.276
400	0.122	0.239
500	0.109	0.214
1000	0.077	0.152
2500	0.049	0.096
5000	0.035	0.068
10000	0.024	0.048

The unequal variance (Welch) t test

2010-03-03T14:18:03-07:00

Two unpaired t tests

When you choose to compare the means of two nonpaired groups with a t test, you have two choices:

Use the standard unpaired t test. It assumes that both groups of data are sampled from Gaussian populations with the same standard deviation.
Use the unequal variance t test, also called the Welch t test. It assues that both groups of data are sampled from Gaussian populations, but does not assume those two populations have the same standard deviation.

These choices are offered by GraphPad InStat, GraphPad Prism, the GraphPad free web t test QuickCalc, as well as many other programs.

The usefulness of the unequal variance t test

To interpret any P value, it is essential that the null hypothesis be carefully defined. For the unequal variance t test, the null hypothesis is that the two population means are the same but the two population variances may differ. If the P value is large, you don't reject that null hypothesis, so conclude that the evidence does not persuade you that the two population means are different, even though you assume the two populations have (or may have) different standard deviations. What a strange set of assumptions. What would it mean for two populations to have the same mean but different standard deviations? Why would you want to test for that? Swailowsky points out that this situation simply doesn't often come up in science (1).

I think the unequal variance t test is more useful when you think about it as a way to create a confidence interval. Your prime goal is not to ask whether two populations differ, but to quantify how far apart the two means are. The unequal variance t test reports a confidence interval for the difference between two means that is usable even if the standard deviations differ.

How the unequal variance t test is computed

Both t tests report both a P value and confidence interval. The calculations differ in two ways:

Calculation of the standard error of the difference between means. The t ratio is computed by dividing the difference between the two sample means by the standard error of the difference between the two means. This standard error is computed from the two standard deviations and sample sizes. When the two groups have the same sample size, the standard error is identical for the two t tests. But when the two groups have different sample sizes, the t ratio for the Welch t test is different than for the ordinary t test. This standard error of the difference is also used to compute the confidence interval for the difference between the two means.
Calculation of the df. For the ordinary unpaired t test, df is computed as the total sample size (both groups) minus two. The df for the unequal variance t test is computed by a complicated formula that takes into account the discrepancy between the two standard deviations. If the two samples have identical standard deviations, the df for the Welch t test will be identical to the df for the standard t test. In most cases, however, the two standard deviations are not identical and the df for the Welch t test is smaller than it would be for the unpaired t test. The calculation usually leads to a df value that is not an integer. InStat, Prism, and our QuickCalc all round the df down to next lower integer, as is common. Future versions will use the fractional df, which is more accurate.

When to chose the unequal variance (Welch) t test

Deciding when to use the unequal variance t test is not straightforward.

It seems sensible to first test whether the variances are different, and then choose the ordinary or Welch t test accordingly. In fact, this is not a good plan. You should decide to use this test as part of the experimental planning.

References

1. S.S. Sawilowsky. Fermat, Schubert, Einstein, and Behrens-Fisher: The Probable Difference Between Two Means With Different Variances. J. Modern Applied Statistical Methods (2002) vol. 1 pp. 461-472

The meaning of the term "nonparametric".

2010-03-03T13:54:41-07:00

The term nonparametric is used inconsistently.

Nonparametric method or nonparametric data?

The term nonparametric should only refers to an analysis method. A statistical test can be nonparametric or not, although the distinction is not as crisp as you'd guess.

It makes no sense to describe data as being nonparametric, and the phrase "nonparametric data" should never ever be used. The term nonparametric simply does not describe data, or distributions of data. That term should only be used to describe the method used to analyze data.

Which methods are nonparametric?

Methods that analyze ranks are uniformly called nonparametric. These tests are all named after their inventors, including: Mann-Whitney, Wilcoxon, Kruskal-Wallis, Friedman, and Spearman.

Beyond that, the definition gets slippery.

What about modern statistical methods including randomization, resampling and bootstrapping? These methods do not necessarily assume any assumption about the population. They do not assume sampling from a Gaussian distribution. They analyze the actual data, and not the ranks. Are these methods nonparametric? Wilcox and Manly have each written texts about modern methods, but they do not refer to these methods as "nonparametric". Four texts of nonparametric statistics (by Conover, Gibbons, Lehman, and Daniel) don't mention randomization, resampling or bootstrapping at all, but the texts by Hollander and Wasserman do.

What about chi-square test, and Fisher's exact test? Are they nonparametric? Daniel and Gibbons include a chapter on these tests their texts of nonparametric statistics, but Lehman and Hollander do not.

What about survival data? Are the methods used to create a survival curve (Kaplan-Meier) and to compare survival curves (log-rank or Mantel-Haenszel) nonparametric? Hollander includes survival data in his text of nonparametric statistics, but the other texts of nonparametric statistics don't mention survival data at all. I think everyone would agree that fancier methods of analyzing survival curves (which involve fitting the data to a model) are not nonparametric.

Two purposes for analyzes ranks rather than data

I think the confusion arises because there are two distinct reasons to choose rank-based tests (like the Mann-Whitney test):

To avoid making assumptions about the distribution of the population. This also implies that there is no strong model describing the population.
To create a method that is robust to outliers.

Once you get beyond the rank-based tests, these two goals do not always go together. Modern methods can be distribution free, but not robust to outliers. And some robust methods are not distribution free.The term 'nonparametric' can be confusing because it can be used as a synonym for three different phrases:

Distribution free (the method makes no assumption, or at least no strong assumption, about the distribution of the population)
Robust (the method is not much influenced by one or a few outliers.
Rank based (the method works by first ranking the values, and then analyzing those ranks)

Because of these ambiguities, I would suggest avoiding the term nonparametric when possible. Instead, write up your analyses with the name of the test used.

	Practical Nonparametric Statistics (Wiley Series in Probability and Statistics)
	by W. J. Conover
	IBSN:0471160687. List price:
	Buy from amazon.com for $81.00

	Applied Nonparametric Statistics
	by Wayne W. Daniel
	IBSN:0534919766. List price:$59.95
	Buy from amazon.com for $59.95

	Nonparametric Statistical Inference (Statistics: a Series of Textbooks and Monogrphs)
	by Jean Dickinson Gibbons, Subhabrata Chakraborti
	IBSN:0824740521. List price:$104.95
	Buy from amazon.com for $102.33

	Nonparametric Statistical Methods, 2nd Edition
	by Myles Hollander, Douglas A. Wolfe
	IBSN:0471190454. List price:$166.00
	Buy from amazon.com for $98.49

	Nonparametrics: Statistical Methods Based on Ranks, Revised
	by Erich L. Lehmann
	IBSN:013997735X. List price:$89.33
	Buy from amazon.com for $74.99

	Randomization, Bootstrap and Monte Carlo Methods in Biology, Second Edition
	by Bryan F.J. Manly
	IBSN:0412721309. List price:$89.95
	Buy from amazon.com for $89.95

	All of Nonparametric Statistics
	by Larry Wasserman
	IBSN:1441920447. List price:$89.95
	Buy from amazon.com for $89.95

	Applying Contemporary Statistical Techniques
	by Rand R. Wilcox
	IBSN:0127515410. List price:$106.00
	Buy from amazon.com for $80.31

50% of what? How exactly are IC50 and EC50 defined?

2010-01-20T09:23:11-07:00

The definition of EC50 and IC50

The concepts of IC50 and EC50 are fundamental to pharmacology. The EC50 is the concentration of a drug that gives half-maximal response. The IC50 is the concentration of an inhibitor where the response (or binding) is reduced by half.

Seems simple enough. But when you actually go to fit data to determine these values, there are several complexities and ambiguities.

The rest of this article is about IC50 (I for inhibition, for downward sloping dose-response curves). All the ideas can be applied to stimulatory curves and EC50 (E for effective) as well. Just stand on your head when you view the figures.

The ideal situation

This figure shows an ideal situation:

The green symbols show measurements made with controls. The ones on the left (Blank) have no inhibitor, so define "100%". The ones on the right are in the presence of a maximal concentration of a standard inhibitor, so define "0%". The data of the experimental dose-response curve (red dots) extend all the way between the two control values.

When fitting this curve, you need to decide how to fit the top plateau of the curve. You have three choices:

Fit the data only, ignoring the Blank control values.
Average the Blank control values, and set the parameter Top to be a constant value equal to the mean of the blanks.
Enter the blank values as if they were part of the dose-response curve. Simply enter a low dose, perhaps 10^-10 or 10^-11. You can't enter zero, because zero is not defined on a log scale.

The results will be very similar with any of these methods, because the data form a complete dose-response curve with a clear top plateau that is indistinguishable from the blank. I prefer the third method, as it analyzes all the data, but that is not a strong preference.

Similarly, there are three ways to deal with the bottom plateau: Fit the data only, set Bottom to be a constant equal to the average of the NS controls, and put the NS controls into the fit as if they were a very high concentration of inhibitor.

That is the ideal situation. There is no ambiguity about what IC50 means.

A situation where IC50 can be defined in two ways

This figure shows an unusual situation where the inhibition curve plateaus well above the control values (NS) defined by a high concentration of a standard drug. This leads to alternative definitions of IC50.

Clearly, a single value cannot summarize such a curve. You'd need at least two values, one to quantify the middle of the curve (the drug's potency) and one to quantify how low it gets (the drug's maximum effect).

The graph above shows two definitions of the IC50.

The relative IC50 is by far the most common definition, and the adjective relative is usually omitted. It is the concentration required to bring the curve down to point half way between the top and bottom plateaus of the curve. The NS values are totally ignored with this definition of IC50. This definition is the one upon which classical pharmacological analysis of agonist and antagonist interactions is based. With appropriate consideration of the biological system and concentrations of interacting ligands, estimated Kd values can often be derived from the IC50 value defined this way (not so for the "so-called absolute IC50" mentioned below).

The concentration that provokes a response halfway between the Blank and the NS value is sometimes called the absolute IC50, The horizontal dotted lines show how 100% and 0% are defined, which then defines 50%. This term is not very standard, and is a bit misleading as there is nothing absolute about an "absolute IC50". Since this value does not quantify the potency of a drug, I think it is more miselading than helpful. Authors of the International Union of Pharmacology Committee on Receptor Nomenclature (1) agree that the concept of absolute IC50 (and that term) is not useful (R. Neubig, personal communication).

If you really want to use the absolute IC50, here are instructions for fitting a curve to find it.

Incomplete dose-response curves

Any attempt to determine an IC50 by fitting a curve to the data in the graph above will be useless. A curve fitting program might, or might not, be able to fit a dose-response curve to the data. But if the curve fits, the value of the IC50 is likely to be meaningless and have a very wide confidence interval. The data simply don't form a top plateau (which would define 100) or a bottom plateau (which would define 0). If data haven't defined 100 or 0, then 50 is undefined too, as is the IC50.

If you also have control values that define 100 and 0, then the curve can be easily fit. The curve below was created by fitting a dose response curve, but constraining the Top plateau to be a constant value equal to the mean of the Blanks values, and the Bottom plateau equal to the mean of the NS values.

The value of the IC50 fit this way only makes sense if you assume that higher concentrations of the inhibitor would eventually inhibit down to the NS values. That is an assumption that can't be tested with the data at hand.

The distinction between relative and absolute IC50 doesn't really apply to these data. Because the data don't define a bottom plateau, the IC50 must be defined relative to the NS control values.

Fitting normalized data

As you can see from all the examples above, it is not necessary to normalize the data to run from100% down to 0%. You can fit curves using data in their natural units. A common mistake is to assume that fitting dose-response curves requires that data first be normalized.

If you choose to normalize your data, it is essential that you think through carefully (and document in methods sections of papers) how 100% and 0% are defined. There are three strategies you can use:

From external controls (Blank and NS in the figures above). Since these values are so important, consider measuring these controls with more replicates than used for the rest of the experiment.
From very low and very high concentrations of the test substance.
From the plateaus of nonlinear regression. Fit the curve first, and then use the best-fit values of the Top and Bottom plateau to normalize the data.

If you fit normalized data, you probably want Prism to force the curve to go from 100 down to 0. It won't know to do this, unless you tell it. Don't make the common mistake of normalizing your data, but not constraining the curve to go from 100 down to 0. You can constrain the curve in two ways:

Choose to fit to a normalized model. The normalized models built in to Prism always go between 0 and 100.
Use a more general model, butt go to the Constrain tab, and set Bottom to a constant value of 0.0 and Top to a constant value of 100.0.

Summary

The concept of IC50 (or EC50) is a bit ambiguous unless you clearly specify which values define 100% and 0%.

Reference

1. R. R. Neubig et al. International Union of Pharmacology Committee on Receptor Nomenclature and Drug Classification. XXXVIII. Update on terms and symbols in quantitative pharmacology. Pharmacol Rev (2003) vol. 55 (4) pp. 597-606

Download

Download the Prism file used to create all the graph in this article.

Pooled SD in ANOVA

2010-01-15T16:25:31-07:00

ANOVA (one- and two-way) assumes that all the groups are sampled from populations that follow a Gaussian distribution, and that all these populations have the same standard deviation, even if the means differ. Based on this assumption, ANOVA computes a pooled standard deviation. This value is used in post tests.

The ANOVA results in Prism (and most programs) don't report this pooled standard deviation. But it is easy to calculate. As part of the ANOVA table, Prism reports several Mean Square values. One of these is the residual Mean Square (some programs use the term error rather than residual). The mean square values are essentially variances. The square root of the residual Mean Square is the pooled SD.

How is this a pooled SD?

First, review how a SD of one group is computed: Calculate the difference between each value and the group mean, square those differences, add them up, and divide by the number of degrees of freedom (df), which equals n-1. That value is the variance. Its square root is the SD.

To compute the pooled SD from several groups, calculate the difference between each value and its group mean, square those differences, add them all up (for all groups), and divide by the number of df, which equals the total sample size minus the number of groups. That value is the residual mean square of ANOVA. Its square root is the pooled SD.

This case study uses the concept of pooled SD.

Why does a normality test of residuals from nonlinear regression give different results than a norma

2009-11-22T08:56:18-07:00

Prism offers normality tests in two places:

As part of the Column Statistics analysis. This tests the normality of a stack of data. This analysis is intended to be used for Column data tables. If you entered data onto a Grouped or XY data table with subcolumns, these are averaged, and the calculations are performed only on the set of averages.
As part of the Nonlienar regression analysis. This tests the normality of the residuals. A residual is the distance of a value from the best-fit curve. If you entered replicate values into subcolumns, and chose the default option in nonlinear regression to fit each value individually, then the normality test is based on each individual value.

If you run both normality tests on the same data, they ask different quesitons and so give different answers.

As an example, create a new XY data table and choose the Michaelis-Menten enzyme kinetics example. There are 10 rows of data in triplicate with two missing values, so 28 Y values in all.

The graphs below show both analyses. The bottom left shows a normality test as part of nonlinear regression (a choice in the Diagnostics tab), testing the null hypothesis that the 28 residuals from the best fit curve are sampled from a Gaussian distribution. The bottom right shows the results of a normality test chosen in the Column statistics analysis, Prism first averaged the triplicates to compute ten means (one for each row). It then tests the null hypothesis that those ten means are sampled from a Gaussian distribution.

The two analyses give different results. If it makes sense to fit a curve (as it does here), then the normality test performed as part of nonlienar regression is helpful, because nonlinear regression is based on the assumption that the residuals are Gaussian. The P value is high, so you conclude that the data are consistent with the assumption that the residuals are Gaussian. In contrast, the normality test which is part of Column statistics really is not helpful. It tests whether the means of the triplicates are Gaussian. The low P value leads you to reject the assumption that the triplcates are Gaussian. But this is really not a relevant quesiton, so the answer is not useful.

Download the Prism file.

Fitting dose-response curve when X is dose, rather than log(dose).

2009-11-18T10:59:16-07:00

The dose-response equations built-in to Prism all assume that the X values are log(dose). You can either enter the data with X values as logarithms of doses, or use the Transform analysis to create a results table with the data arranged that way which can then be graphed and fit.

It is possible to fit data where X values are concentrations, rather than log(concentrations). It is necessary to adjust the equation accordingly.

Here is the equation built-in to Prism for fitting a variable slope (four-parameter) log(dose) response curve:

Y=Bottom + (Top-Bottom)/(1+10^((LogEC50-X)*HillSlope))

Here is the equation modified to expect X values to be concentrations, not logarithms, so the concentration does not need to be raised to the tenth power to antilog it:

Y=Bottom + (Top-Bottom)/(1+ (10^logEC50 /X)^HillSlope)

The equation still fits the logEC50, rather than the EC50. Why? Because the confidence intervals computed by Prism are always symmetrical around the parameter value. But the true uncertainty is only symmetrical on a logEC50 scale.

Download this Prism file to see how it works. The same data are fit and graphed twice.

In one version the X values are transformed to logarithms, and then fit to the equation built-in to Prism. Here the graph has a linear X axis, but the numbering is converted to powers-of-ten to show that the X values represent logarithms.
In the other version, the data are fit with the X values remaining as concentrations and fit to the equation showed above. Here the X axis is stretched to a logarithmic scale (top right of Format Graph dialog).

The two graphs look identical. The results of the two fits are identical.

Note that the second graph will only look good in Prism 5, which is smart about plotting curves on axes stretched to a logarithmic scale. Prism 4 was not smart about this, and the resulting curve looks very choppy.

Why is the HillSlope applied to the EC50 as well as the X values?

Why doesn't Prism report the standard error of the EC50?

GraphPad programs and OSX 10.6 (Snow Leopard)

2009-08-31T08:35:40-07:00

Apple released a new version of OSX, 10.6 Snow Leopard, on Aug. 28 2009.

Prism 5

We know of one problem using Prism 5.0b on Snow Leopard: Fill patterns don't render well. We recommend that you use solid fills for bars, and simply avoid fill patterns altogether if you use Snow Leopard (and even otherwise, fill patterns are a hold over from the days of plotters, and solid fills look better). It is likely that Apple will fix this glitch in Snow Leopard. If they don't, we'll try to bypass the problem in release 5.0c.

Other minor glitches:

Black colors appear gray when the graph is exported to a pdf file using CMYK colors, and viewed in Preview. Choose RGB colors instead, and the pdfs look fine. Or export a tiff file with CMYK colors. Note that the pdf file is fine, but is just rendered incorrectly by the new version of Preview.
The Send-to-Powerpoint button and command don't work. Use Copy and Paste instead.
When running a Prism script, the script log is always empty.
Editing sheet names in the navigator looks ragged.
The slider on the info page separating info constants from notes looks corrupted.
Exporting to the PICT format doesn't work.
Exporting using the monochrome color model (to export colorful graphs as black and white) doesn't work.
If you save a Prism file as XML, its icon is blank.
One person found that the updater from 5.0a to 5.0b did not work under Snow Leopard. But the full 5.0b installer worked fine.

We will investigate these problems, and any others we discover or are told about, and fix in release 5.0c coming soon. Let us know if you encounter other problems.

InStat, StatMate, and Prism 4

InStat 3, StatMate 2 and Prism 4 use an older style of Mac programming. They run perfectly on current macs using an Intel chip, but do so by relying on Apple's Rosetta system. Apple created Rosetta so programs written for the earlier generation of Macs that use a PowerPC chip will also work on newer Intel Macs. This is truly amazing software that just works. You don't even know it is there.

With OSX 10.4 (Tiger) and 10.5 (Leopard), Rosetta was automatically installed and simply works when it is needed. You don't have to configure it, and won't even know when it is running. The only exception is that a few people have had problems after updating to OSX 10.5.6. This page from the Apple web site explains how to fix the problem, which requires running the 'combo update' rather than the 'incremental update' .

Rosetta is not automatically installed with OSX 10.6 (Snow Leopard). If you are updating to Snow Leopard and plan to run InStat 3, StatMate 2, or Prism 4, click the "Customize" button in the Mac OS X Snow Leopard installer and select the option to install Rosetta.

If you don't install Rosetta at the time you install Snow Leopard, or get a new Mac without it, InStat, StatMate and Prism 4 will still work just fine. The first time you run one of these programs under Snow Leopard, OSX detects that you need Rosetta and provides an easy way to install it. You only have to do this once. Rosetta will be installed from Apple's server if you are connected to the internet. Otherwise, you'll need to insert your Mac OS X Snow Leopard installation disc, open the Optional Installs folder, and double-click Optional Installs.

Guidelines for presenting statistics in published papers.

2009-08-25T09:08:30-07:00

Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publications is a lengthy document with guidelines for authors and publishers. But it has only one paragraph about statistics:

"Describe statistical methods with enough detail to enable a knowledgeable reader with access to the original data to verify the reported results. When possible, quantify findings and present them with appropriate indicators of measurement error or uncertainty (such as confidence intervals). Avoid relying solely on statistical hypothesis testing, such as P values, which fail to convey important information about effect size. References for the design of the study and statistical methods should be to standard works when possible (with pages stated). Define statistical terms, abbreviations, and most symbols. Specify the computer software used."

These two papers give sensible guidelines for presenting statistical calculations and conclusions:

Curran-Everett and Benos. Guidelines for reporting statistics in journals published by the American Physiological Society. AJP - Gastrointestinal and Liver Physiology (2004) vol. 287 (2) pp. G307. Those authors later published a sequel, with additional comments. This sequel references a bunch of papers which critique the guidelines.

Ludbrook. The presentation of statistics in Clinical and Experimental Pharmacology and Physiology. Clin Exp Pharmacol Physiol (2008) vol. 35 (10) pp. 1271-4). Ludbrook has also self published a two-page set of guidelines for mathematical operators and statistical symbols.

These authors agree on two points (regarding style, not substance) that I was not aware of, so the GraphPad manuals and help screens (and my book Intuitive Biostatistics) have done differently:

They say that the standard error of the mean should be abbreviated as SE, rather than SEM.
They say that the mean and standard deviation should be written as mean (SD), rather than mean ± SD. if the mean is 11.2 and the standard deviation is 2.4, they suggest reporting 11.2 (2.4) rather than 11.2 ± 2.4. They recommend using that latter syntax only for standard errors, not standard deviations.

How does Prism compute and plot residuals from nonlinear regression?

2009-08-20T08:31:48-07:00

If you choose (or accept the default) standard weighting, then the residuals are the difference between the actual Y value you entered and the Y value predicted by the model. If the data point is above the curve, the residual is positive. If the data point is below the curve, the residual is negative. Least-squares regression works to minimize the sum of the squares of these residuals.

If you choose another weighting scheme, Prism 5 adjusts the definition of the residuals accordingly. The residual that Prism tabulates and plots equals the residual defined in the prior paragraph, divided by the weighting factor. The most common common alternative weighting is "Weight by 1/Y² (minimize relative distances squared)". In this case, the residual is defined to be the distance of the point from the curve divided by the Y value of the curve. Weighted nonlinear regression minimizes the sum of these residuals squared.

Note the ambiguity in defining weighting. The Prism dialog gives the choice to weight by 1/Y². This means that the squared residual is divided by Y². The weighted residual is defined as the residual divided by Y. Prism minimizes the sum of the squares of these weighted residuals.

Earlier versions of Prism (up to Prism 4) always plotted basic unweighted residuals, even if you chose to weight the points unequally.

When performing linear regression, Prism does not offer weighting so the residuals are always unweighted residuals as defined in the first paragraph above.

How Prism computes R² with weighted nonlinear regression.

How weighted nonlinear regression works.

Adjusted P values as part of multiple comparisons.

2009-08-17T18:47:45-07:00

Many people ask why multiple comparisons tests following one-way (or two-way) ANOVA can't report individual P values for each comparison. When you correct for multiple comparisons, it really doesn't make much sense to talk about individual P values. All you can do is divide the comparisons into two groups -- statistically significant and not -- at some defined significance level (usually 5%) that applies to the entire family of comparisons.

However SAS reports adjusted P values, and these are explained in the book by Westfall (citation below).

The idea is pretty simple. There is nothing special about 0.05 or 0.01... You can set the significance level to any probability you want. The adjusted P value is the smallest probability at which a particular comparison will be declared statistically significant (as part of the multiple comparison testing). Each comparison will have a unique adjusted P value. But these P values are computed from all the comparisons, and really can't be interpreted for just one comparison.

Computing the adjusted P value is trivial for Bonferroni multiple comparison tests. It is harder for Tukey and Dunnett tests, as it requires computing critical values beyond those that have been tabulated.

We are considering including adjusted P values as part of the reporting of one-way ANOVA with Tukey, Bonferroni or Dunnett multiple comparison tests. I hesitate to do this (the programming is far from trivial) because I don't really know how to explain how to interpret the results, and fear that they will be misinterpreted.

I don't know of any program except SAS that computes adjusted P values, and don't know of any book except Westfall that explains them. They do not seem to be mainstream.

Please let us know if you would like to see adjusted P values in future versions of Prism, and explain why.

	Multiple Comparisons and Multiple Tests (Text and Workbook Set)
	by Peter H. Westfall, Randall D. Tobias, Dror Rom
	IBSN:1580258336. List price:$62.32
	Buy from amazon.com for $50.18

Bug with Fisher's Exact test in Prism 5.02 and 5.0b

2009-08-10T11:31:45-07:00

Prism 5.02 (Windows) and 5.0b (Mac) included a fix to a trivial bug in Fisher's exact test (when the two groups are identical, the P value should be 1.00 but earlier versions of Prism sometimes reported P values slightly greater than 1.0). Unfortunately, that fix introduced a new bug that occurs only when:

You are using Prism 5.02 (Windows) or 5.0b (Mac). Earlier versions did not have this bug. Neither does InStat 3.0 or 3.1.
You have entered a symmetrical contingency table. A table is symmetrical when either the two row totals are identical, or the two column totals are identical.
You have chosen a two-tail (two-sided) P value. One-tail P values are computed correctly.

The result of the bug is that the P value will be too low in some, but not all cases. In many cases, the discrepancy is tiny and won't affect your conclusions. In other cases, the discrepancy is larger and may affect your conclusion.

Of course, we will fix the bug in the next release of Prism: 5.03 and 5.0c.

It is easy to determine whether you have encountered this bug, and to compute the correct two-tail P value. With symmetrical contingency tables, the two-tail P value is exactly twice the one-tail P value (that is not always true with contingency tables that are not symmetrical). Therefore, to bypass the bug, ask Prism to compute a one-tail (one-sided) P value. This is a choice in the Parameters dialog for analyzing contingency table. To compute a two-tail P value, simply double the one-tail P value.

More details and example.

GraphPad InStat 3.1 is available

2009-07-14T07:56:51-07:00

If you own a license for InStat 3, please update free to version 3.1. Updates are available for both Mac and Windows versions. The biggest change is that we've increased the size of the data table, which can now have 10,000 rows and 52 columns. Other changes are listed here. If you are not familiar with InStat, it is a very simple statistics program -- so simple, anyone can master it in just a few minutes. Learn more.