Factor Analysis (FA) is a statistical method that is used to analyze the underlying structure of a set of variables. It is a data reduction technique that attempts to account for the intercorrelations among a large number of variables in terms of fewer unobservable (latent) variables or factors. In R Programming Language, the psych package offers a range of functions to conduct factor analysis.
Factor analysis involves several steps
- Data preparation: The data is usually scaled so that all the variables are on a similar scale.
- Factor Extraction: Factors are extracted based on how well they explain the variance in the data. Common extraction methods include maximum likelihood estimate (MLE), Principal Axis Factoring (PAF) and Minimum Residual (MR). Although principal components analysis (PCA)is sometimes used for dimensionality reduction, it is not a true factor extraction method in factor analysis.
- Factor Rotation: The factors are usually rotated to make their interpretation easier. The most common method of rotation is Varimax rotation, which tries to maximize the variance of the factor loadings.
- Factor interpretation: Interpreting the factors by examining the factor loadings (correlation between variables and factors). High loadings indicate a strong relationship between a variable and a factor.
Loading the Data
We will use the Iris dataset, which is built into R, for this example. The dataset contains measurements of the sepal length, sepal width, petal length and petal width of three different iris species.
data(iris)
# View the first few rows of the dataset
head(iris)
Output:
Data Preparation
Prior to performing factor analysis, we must prepare the data by scaling the variables to a mean of zero and a standard deviation of one. This is necessary because factor analysis is sensitive to scale differences
# Scale the data
iris_scaled <- scale(iris[,1:4])
Determining the Number of Factors
The next step is to determine the number of factors to extract from the data. This can be done using methods such as the Kaiser criterion, scree plot or parallel analysis. In this example, we use the Kaiser criterion, which suggests selecting factors with eigenvalues greater than one.
# Perform factor analysis
library(psych)
fa <- fa(r = iris_scaled,
nfactors = 4,
rotate = "varimax")
summary(fa)
Output:
The output shows the results of factor analysis performed with 4 factors.
- The negative degrees of freedom (−4) indicate that the model is over-specified, meaning too many factors were extracted for the given number of variables. This suggests that fewer factors (e.g., 2 or 3) may provide a more reliable and interpretable solution
- The RMSEA value of 0 suggests a perfect fit, but this is not meaningful due to the over-specified model.
- The Tucker Lewis Index (TLI ≈ 1.009) indicates an excellent fit, but values greater than 1 typically occur in overfitted models and should be interpreted cautiously
This summary shows that the factor analysis was performed with 4 factors and provides the standardized loadings for each variable on each factor. It also reports the variance explained and model fit statistics.
Interpreting the Results of Factor Analysis
After the factor analysis is finished, we can interpret the findings by looking at the factor loadings, which are the correlations between the observed variables and the factors that were extracted. Generally, loadings of more than 0.4 or 0.5 are significant.
# View the factor loadings
fa$loadings
Output:
Loadings:
MR1 MR2 MR3 MR4
Sepal.Length 0.997
Sepal.Width -0.108 0.757
Petal.Length 0.861 -0.413 0.288
Petal.Width 0.801 -0.317 0.492
MR1 MR2 MR3 MR4
SS loadings 2.389 0.844 0.332 0.000
Proportion Var 0.597 0.211 0.083 0.000
Cumulative Var 0.597 0.808 0.891 0.891
In this case, MR1 (the first factor) is strongly associated with Petal.Length and Petal.Width, while MR2 is associated with Sepal.Length and Sepal.Width.
Validating the Results of Factor Analysis
It is important to validate the factor structure by checking the assumptions and comparing results across different subsets of the data.
# Examine factor structure for different subsets
subset1 <- subset(iris[,1:4],
iris$Sepal.Length < mean(iris$Sepal.Length))
fa1 <- fa(subset1, nfactors = 4)
print(fa1)
Output:
Factor Analysis using method = minres
Call: fa(r = subset1, nfactors = 4)
Standardized loadings (pattern matrix) based upon correlation matrix
MR1 MR2 MR3 MR4 h2 u2 com
Sepal.Length 0.66 0.61 -0.12 0 0.82 0.178 2.1
Sepal.Width -0.68 0.61 0.11 0 0.85 0.150 2.0
Petal.Length 1.00 0.00 0.00 0 1.00 0.005 1.0
Petal.Width 0.97 0.01 0.16 0 0.97 0.031 1.1
MR1 MR2 MR3 MR4
SS loadings 2.85 0.74 0.05 0.00
Proportion Var 0.71 0.18 0.01 0.00
Cumulative Var 0.71 0.90 0.91 0.91
Proportion Explained 0.78 0.20 0.01 0.00
Cumulative Proportion 0.78 0.99 1.00 1.00
Mean item complexity = 1.5
Test of the hypothesis that 4 factors are sufficient.
The degrees of freedom for the null model are 6 and the objective function was
4.57 with Chi Square of 351.02
The degrees of freedom for the model are -4 and the objective function was 0
The root mean square of the residuals (RMSR) is 0
The df corrected root mean square of the residuals is NA
The harmonic number of observations is 80 with the empirical chi square 0 with prob < NA
The total number of observations was 80 with Likelihood Chi Square = 0 with prob < NA
Tucker Lewis Index of factoring reliability = 1.018
Fit based upon off diagonal values = 1
Measures of factor score adequacy
MR1 MR2 MR3 MR4
Correlation of (regression) scores with factors 1.00 0.91 0.69 0
Multiple R square of scores with factors 1.00 0.82 0.47 0
Minimum correlation of possible factor scores 0.99 0.64 -0.05 -1
By examining the factor loadings for different subsets, we ensure the results are stable and reliable.
Using factanal() Function for Factor Analysis
The factanal() function is used to perform factor analysis on a data set. The factanal() function takes several arguments described below
Syntax:
factanal(x, factors, rotation, scores, covmat)
where,
- x : The data set to be analyzed.
- factors : The number of factors to extract.
- rotation : The rotation method to use. Popular rotation methods include varimax, oblimin and promax.
- scores : Whether to compute factor scores for each observation.
- covmat : A covariance matrix to use instead of the default correlation matrix.
Here is an example code snippet that demonstrates how to use factanal() function in R:
install.packages("psych")
library(psych)
data(mtcars)
# Perform factor analysis on the mtcars dataset
factor_analysis <- factanal(mtcars,
factors = 3,
rotation = "varimax")
print(factor_analysis)
Output:
Call:
factanal(x = mtcars, factors = 3, rotation = "varimax")
Uniquenesses:
mpg cyl disp hp drat wt qsec vs am gear carb
0.135 0.055 0.090 0.127 0.290 0.060 0.051 0.223 0.208 0.125 0.158
Loadings:
Factor1 Factor2 Factor3
mpg 0.643 -0.478 -0.473
cyl -0.618 0.703 0.261
disp -0.719 0.537 0.323
hp -0.291 0.725 0.513
drat 0.804 -0.241
wt -0.778 0.248 0.524
qsec -0.177 -0.946 -0.151
vs 0.295 -0.805 -0.204
am 0.880
gear 0.908 0.224
carb 0.114 0.559 0.719
Factor1 Factor2 Factor3
SS loadings 4.380 3.520 1.578
Proportion Var 0.398 0.320 0.143
Cumulative Var 0.398 0.718 0.862
Test of the hypothesis that 3 factors are sufficient.
The chi square statistic is 30.53 on 25 degrees of freedom.
The p-value is 0.205
In this example, we load the psych package, which provides functions for data analysis and visualization and the mtcars data set, which contains information about different car models. We then use the factanal() function to perform factor analysis on the mtcars data set, specifying that we want to extract three factors and use the varimax rotation method. Finally, we print the results of the factor analysis.