Trying out HAD: 10. Before testing for differences in means
Tests for differences in means, such as t-tests and analysis of variance (ANOVA), are staples of statistical analysis, but before writing about them, let's touch on a few points regarding filter settings and HAD2R code. To summarize briefly, always pay attention to the filter setting information that is output when using filter settings. I see, so that might be why it's in red text. That makes sense. (You can't make text red in note, right?)
Does HAD not have t-tests?
In the HAD analysis menu, the item following "Estimation of Means" is "Test for Difference in Means". This corresponds to what are known as "t-tests" and the like, but the words "t-test" do not appear in the analysis menu. Interesting, isn't it? In statistics textbooks, they are usually explained separately: t-tests for comparing two groups (like men and women) and ANOVA for comparing three or more groups (like first to third graders, or four stages from 30s to 60s). When calculating by hand, the calculations feel quite different, but in reality, both tests are tests with the null hypothesis that "there is no difference in the mean values of all levels", so it can be said that they are analyses with the same purpose. Therefore, for example, if you try to examine gender differences and click OK expecting a t-test because there are two levels (men and women), but for some reason the results of a 3-level ANOVA are output, you might panic for a moment. This happens if you forget that someone selected "other" for gender. The "assumption" that there are two levels because it is gender is often the cause of the panic. Putting aside the question of whether that is acceptable for someone studying psychology, let's just say that HAD is great for quietly pointing out, "
you forgot about the people who chose 'other' for gender, didn't you?".
Let's try testing for now
Anyway, let's give it a try. I have done both t-tests and ANOVA many times, but when you are in "try this and that" mode, you sometimes discover various things. The data used is iris. The procedure is as follows.
Set the variables to use (dependent variable first, independent variable second) => [Analysis] => Select "Test for Difference in Means" => Select "Unpaired" (default)
You can specify multiple dependent variables (!), but let's put that aside for a while; this time I set the dependent variable to "Sepal.L" and the independent variable to "Species". The result looks like this. Since there are 3 types of "Species" in the iris data, naturally, a 3-level ANOVA is executed.
I remember it took a little while to get used to the HAD ANOVA sheet because it outputs a huge amount of information all at once, but I'll write about that another time. The thing I was happiest about was this plot. This one.
It automatically draws the plot and even adds error bars. Isn't that kind? The reason I thought "HAD is great" is almost 100% thanks to this plot. Thank you, HAD. It's quite a hassle to draw this in Excel yourself, and when I was doing my graduation research, I didn't know how to add error bars to R plots.
By the way, the error bars in the plot above show the "standard error". If you think, "Wait, isn't that the 95% CI?", you can change it with the following procedure.
[HAD Settings] => [Graph Settings] => "Graph Display Settings" or [Analysis] => [Graph Settings] => "Bar Chart"
Even after outputting, the HAD plot is essentially an Excel graph, so you can change it however you like by tweaking the "Error Bars" formatting.
Trying to force a t-test
Now, let's try to force (?) a t-test using the same data. The idea is to test using only two of the three types of "Species". So, we use the "Filter" function. In the [Filter] column of the "Species" row, enter "3" as a half-width number. This specifies that "data where Species is 3 will not be used for analysis".
I can't remember that, so I click the [Filter] button and use the GUI. It's a GUI like this. If you click OK here, it will be set in the same way as the screen above.
In this state, perform the same analysis as before.
Set the variables to use (dependent variable "Sepal.L" first, independent variable "Species" second)
⇒ [Analysis]
⇒ Select "Test for difference in means"
⇒ Select "Unpaired" (default)
The results look like the above. Even though I am running the same menu, the output is completely different. Here, too, the plot really shines. I am very grateful for that. Another thing I am grateful for, or perhaps I should say, something that makes me think "that's right," is that it runs both the Welch test and the t-test. I am not knowledgeable enough to discuss the technical details of the differences between the two, but the t-test assumes that the variances of the two groups are equal, while the Welch test does not make such an assumption.
In some statistics textbooks, it was previously explained that one should first test whether the variances are equal, and based on that result, use the appropriate one, either the t-test or the Welch test. The textbook I first studied also had explanations in that direction, but such a procedure is unnecessary, and one should always perform the Welch test, and Professor Shimizu's blog also writes the following.
A commonly used method is to perform a test for equality of variances, and if equal variances hold, use a standard t-test, and if not, use the Welch test. However, regarding this as well, as Professor Aoki's site simulation clarifies, the Welch test always produces correct results whether equal variances hold or not, so one should always perform the Welch test. Also, the test for equal variances followed by a t-test has the problem of test duality, which makes it impossible to maintain the significance level. Therefore, for testing the difference in means between two groups, let's always use the Welch test. (https://norimune.net/1585)
The site of Professor Aoki mentioned is here.
Then why does it also output the results of the t-test? The question arises, but well, there might be various circumstances.
Be careful with filters
By the way, at the beginning of the output result earlier, there was a display like this in red letters.
It is displaying the filter settings as they are. A filter of "3" is set for the Species variable. In other words, it is alerting you that data where Species=3 is excluded from the analysis. However, there are things to be careful about regarding this filter setting. Let's look at two cases here. One is from the perspective of using analysis history. The other is from the perspective of using HAD2R.
Analysis history does not record filters
HAD output always comes with an "analysis code." It is this, printed small in the upper right of the sheet. I wanted it to stand out, so I enlarged the text and pasted it.
The analysis settings are recorded in numbers, and based on this code, the same analysis can be reproduced. This is a feature implemented since HAD12.21. How to use it is,
Right-click the cell where the analysis code is output
⇒ Select "Run analysis code" from the menu
or ⇒ Click "Reflect analysis code to HAD"
If you select the former, "Run analysis code," it will silently execute the same analysis. At this time, it is not affected by how the modeling form is currently set, and the sheet with the same name is not overwritten. This is convenient. However, filter settings are not included in the analysis code. This is important. In other words,
Run "Scatter plot" analysis with filter "Species(3)" ⇒
"Scatter" sheet <<analysis without filter>> is output ⇒
Change filter setting to "Species(1)" on the "Modeling" sheet ⇒
Run the "analysis code" of the "Scatter" sheet from earlier <<You ran it without a filter earlier, right?>> ⇒
"Scatter plot" analysis is executed with filter "Species(1)"
Therefore, an analysis like "that filter setting once more" cannot be done just by running the "analysis code." "Once more with a different filter setting" is OK by changing the filter setting and then running the "analysis code." It's a bit complicated. If you understand the quirks and use them conveniently, it is convenient. Surely. (Or is that obvious?)
Also, if you click the latter, "Reflect analysis code to HAD," it will correct the variable settings to match the analysis code, but it will not be executed, and it will just return to the modeling sheet. Therefore, this would be convenient when using the same variables for a different filter or a different analysis.
In any case, if you think this filter setting will be important in the future, it is considered wise to delete unnecessary rows of data, create new data with the necessary rows (be careful not to lose the original data), and then continue the analysis.
Applying filters to data
Since I'm at it, I will organize how to execute this using HAD. Suppose you want to continue the analysis by excluding only the data where Species=3 from the iris data. Rather than continuing to use the "Species(3)" filter, creating a new dataset that excludes the rows where Species=3 can reduce simple mistakes in subsequent analyses. The procedure is as follows:
Set all necessary variables in the variables to use
⇒ Set filter "Species=3"
⇒ [Dataset]
⇒ [Output dataset]
⇒ Check the top option "Output selected variables" and click OK
Then, data like this will be output to the "DataOutput" sheet.
In the figure above, rows 2 to 95 are hidden. In the original data, the data for Species=3 starts from row 101, but that entire section has become missing values. I will use this as the new dataset. Continuing the procedure.
Select all columns of the output variables
⇒ Right-click on the selection
⇒ Click "Use as main dataset"
Then, the newly created variables will be inserted into the "Data" sheet. The previous data remains on the right side. In this example, column H is blank, so anything to the right of it is not considered data for analysis.
You can click [Load data] to continue the analysis. In this state, if you perform the same analysis as before, the t-test will be executed even without setting a filter. This is because the Species variable now has only 2 levels, as all data for Species=3 became missing values through the procedure above. All's well that ends well.
Alternatively, in the filter setting GUI mentioned earlier, you can click "Create filter variable," add the output variable to the data, and set a filter of "1" for that variable. In any case, you must always pay attention to which subjects are included in your analysis.
HAD2R also does not record filters
Finally, regarding HAD2R. When you perform a test for the difference in means, the following code is output. The code is naturally different for 2 levels versus 3 levels, but this is the code for when there are 2 levels, i.e., a t-test.
dat <- read.csv(".../csvdata.csv")
t.test(Sepal.L~Species, dat)
And, to write this in a small voice (?) at the beginning,the "～" (tilde operator) in the output above is a full-width character, which is an error. You cannot analyze correctly unless you use the half-width "~" (I said small voice, but I made it bold. Because it's important). Also, as is already obvious, since Species is originally a 3-level variable, you cannot perform a t-test with the code above.This is because the filter setting is not reflected in the code, so Species remains at 3 levels (it would be fine if it were a new dataset with the filter applied, though).
To reflect the filter setting (excluding Species=3), you need to write it as follows, for example. If you also correct the tilde operator to half-width,
subdat <- dat[dat$Species!=3, ]
t.test(Sepal.L ~ Species, subdat)
Also, the t-test in R is named t.test, butit performs a Welch test by default. If you want to perform a standard t-test, you need to set the option var.equal = TRUE. Well, since it is said that the Welch test is practically sufficient, it is not strictly necessary, but if you really want to use it, you need to set the option.
So, as a result of trying to force a t-test using the 3-level Species as an independent variable, I learned a lot about filter settings and it became a long explanation, so I will stop here for now. I will write about t-tests and ANOVA separately.