Copying "Introduction to Multivariate Analysis" in Python Vol. 3 - Chapter 2 "First Steps in Multiple Regression Analysis" ② Goodness of Fit, Standardized Partial Regression Coefficients

Chapter 2 "First Steps in Multiple Regression Analysis"

Authors of the book: Dr. Sadao Ishimura, Dr. Koshiro Ishimura

This is a Python coding record of Chapter 2 "First Steps in Multiple Regression Analysis" from the book "Introduction to Multivariate Analysis".

This is a coding series where we learn the basics of multivariate analysis along with Python.

This article covers model evaluation and interpretation of partial regression coefficients. Specifically, we will focus on topics such as the coefficient of determination, multiple correlation coefficient, AIC, and standardized partial regression coefficients.

We will proceed with ChatGPT-assisted learning!
Now, let's open the book and embark on a journey into multivariate analysis 🚀

Introduction

This blog series introduces the "fun of multivariate analysis" gained through Python coding of the book "Introduction to Multivariate Analysis" (Tokyo Tosho, referred to as the "text").

The introduction of the book and citation notation are posted in the linked article.

Chapter 2: First Steps in Multiple Regression Analysis

This article covers the following sections of Chapter 2.

The data used in this article is cited directly from the data published in the text.
For data with a small number of entries, we register the data in the code, and for data with a large number of entries, we convert it to a CSV file and load it.

We will import the libraries used in this article.

### インポート

# 数値計算
import numpy as np
import pandas as pd

# 統計
import pingouin as pg
import statsmodels.formula.api as smf
import statsmodels.api as sm

# 描画
import matplotlib.pyplot as plt
import seaborn as sns
from graphviz import Digraph
plt.rcParams['font.family'] = 'Meiryo'  # または import japanize_matplotlib

Preparation for Analysis

We will cite Table 2.1.1 "Data on Temperature, Pressure, and Orientation" on page 35 of the text.
We have added the explanatory variable "Time" to the data from the previous article.

■ Loading Data
We will set the data into a pandas DataFrame.

### 温度・圧力・配向度のデータ p.35 表2.1.1

# データの登録
data1 = pd.DataFrame(
    {'配向度': [45, 38, 41, 34, 59, 47, 35, 43, 54, 52],
    '温度': [17.5, 17.0, 18.5, 16.0, 19.0, 19.5, 16.0, 18.0, 19.0, 19.5],
    '圧力': [30, 25, 20, 30, 45, 35, 25, 35, 35, 40],
    '時間': [20, 20, 20, 20, 15, 20, 20, 20, 20, 15],
    }, index=range(1, 11))
data1.index.name = 'サンプルNo.'

# 結果の表示
data1

【Execution Result】
The number of data points (sample size) is 10.

The objective variable is "Orientation," and the candidate explanatory variables are "Temperature," "Pressure," and "Time."

■ Obtaining multiple regression analysis results with Python libraries
We will perform multiple regression analysis using the libraries statsmodels and pingouin, using "temperature" and "pressure" as explanatory variables.
These two libraries were covered in the previous article.

🖲️statsmodels

# statsmodels利用
result1_sm = smf.ols(formula='配向度 ~ 温度 + 圧力', data=data1).fit()
result1_sm.summary()

【Execution Results】
The coef values for "Intercept", "temperature", and "pressure" in the bottom section are the partial regression coefficients.

🖲️pingouin

# pingouin利用
result1_pg = pg.linear_regression(X=data1[['温度', '圧力']], y=data1['配向度'])
result1_pg

【Execution Results】
The coef column contains the partial regression coefficients.

Goodness of fit of the multiple regression equation

Although we have obtained a multiple regression equation from the data, can the multiple regression equation properly estimate the predicted values of the objective variable...?

What is called "goodness of fit of the multiple regression equation" or "goodness of fit of the model" is, roughly speaking, checking the "goodness" based on the "degree/index to which the explanatory variables explain the objective variable" in relation to the analyzed data.
The text explains "indices indicating goodness of fit," focusing on the coefficient of determination and AIC.

Coefficient of determination p.50~

The coefficient of determination $${R^2}$$ is, roughly speaking, an index of the "degree to which the multiple regression equation explains the variation in the objective variable."
It takes a value from 0 to 1, and the higher the degree of explanation = the better the fit, the closer it gets to 1.

I will borrow the formulas for the coefficient of determination from pages 50 and 51 of the text.

📊 Japanese version

In the first line, the denominator is a quantity related to "observed values," and the numerator is a quantity related to "predicted values" = "multiple regression equation."
I think you can imagine that something about the observed values can be explained by something from the multiple regression equation.

📊 Mathematical formula version
The true nature of the "sum of squares" is revealed!

The formulas are composed of objective variables.

• $${y_i}$$: Objective variable (observed value) of the $${i}$$-th data point

• $${\bar{y}}$$: Mean of the target variable

• $${Y_i}$$: Predicted value of the target variable for the $${i}$$-th data point (predicted using the multiple regression equation)

The meaning of each sum of squares is roughly as follows.

• The variation indicated by the sum of squares of observed values (total sum of squares) is the sum of the squares of the 'difference between the observed value and the mean,' which represents the 'variation in the target variable' (degree of deviation from the mean).

• The sum of squares of predicted values (regression sum of squares) is the sum of the squares of the 'difference between the predicted value from the multiple regression equation and the mean,' which represents the 'variation in the target variable explained by the explanatory variables.'

• The residual sum of squares is the sum of the squares of the 'difference between the observed value and the predicted value = residual,' which represents the 'variation in the target variable not explained by the explanatory variables.'

🔢 Calculation of the three sums of squares
We will calculate the 'three sums of squares' from Table 2.4.1 in the text.
First, we calculate the 'observed value,' 'predicted value,' and 'residual' of the target variable for each data point.

### ３つの平方和 p.50 表2.4.1

# 実測値・予測値・残差の表の作成
data1_pred = pd.concat([data1['配向度'].rename('実測値'),
                        result1_sm.fittedvalues.rename('予測値'),
                        result1_sm.resid.rename('残差')], axis=1)
data1_pred

【Execution Result】

Let's visualize the degree of deviation between the observed values and predicted values to get an intuitive image.

# 実測値と予測値のプロット

# 描画領域の設定
fig, ax = plt.subplots(figsize=(5, 5))
# 実測値と予測値の散布図の描画
sns.scatterplot(data=data1_pred, x='実測値', y='予測値', s=70, ax=ax)
# 実測値＝予測値となる45度直線の描画
ax.plot([32, 60], [32, 60], color='tab:red', ls='--')
# 修飾
ax.set(xlim=(32, 60), ylim=(32, 60));

【Execution Result】
The 45-degree line (red dotted line) indicates observed value = predicted value.
The closer the data points (blue dots) are to this line, the higher the accuracy of the predicted values.
In this figure, the data points are generally scattered around the 45-degree line, so it seems to have decent prediction accuracy.

We will calculate the 'mean' of each column in Table 2.4.1 and the three sums of squares.

# 平均値の表示
data1_pred_mean = data1_pred.mean(axis=0).rename('平均値').to_frame()
data1_pred_mean.round(10).T

【Execution Result】
The mean of the observed values and predicted values is the 'mean of the observed values.'
The mean of the residuals is 0.

Using these mean values, we calculate the three sums of squares.

# 平方和の算出 

# 上記の平均値を計算に使いやすいように加工
means = data1_pred_mean.values.flatten()

# 平方和の算出
sum_square1 = ((data1_pred.apply(lambda x: x - means, axis=1)**2)
                 .sum(axis=0)
                 .rename('平方和').to_frame().T)
sum_square1.round(2)

【Execution Result】
As shown in the text,
'Total sum of squares = Regression sum of squares + Residual sum of squares'.

🔢 Calculation of the coefficient of determination
We calculate the coefficient of determination using the total sum of squares and the regression sum of squares.

# 決定係数の算出 p.51
print((sum_square1['予測値'] / sum_square1['実測値']).values[0])

【Execution Result】
The coefficient of determination is $${0.858}$$.
The fit of this multiple regression equation seems good.

Let's extract the coefficient of determination from the multiple regression analysis results of statsmodels and pingouin.

🖲️statsmodels

# statsmodels利用
result1_sm.rsquared

【Execution Result】

🖲️pingouin

# pingouin利用
result1_pg.r2[0]

[Execution Results]

Adjusted R-squared p.52

The coefficient of determination has the property of increasing as the number of explanatory variables increases.
The "adjusted R-squared" $${\widehat{R}^2}$$ can eliminate the influence of the number of explanatory variables.

I will borrow the formula for the adjusted R-squared $${\widehat{R}^2}$$ from the text.

$${N}$$ is the number of data points (sample size), and $${p}$$ is the number of explanatory variables.

🔢 Calculating the adjusted R-squared
Let's calculate the adjusted R-squared using the sum of squares of observed values (total sum of squares) $${S_T}$$ and the residual sum of squares $${S_E}$$ calculated earlier.

# 自由度調整済み決定係数の算出 p.53

# 設定と準備
p = 2           # 説明変数の個数
N = len(data1)  # 標本サイズ

# SSR, SSE, SSTの算出
SSR = sum_square1['予測値']
SSE = sum_square1['残差']
SST = SSR + SSE

# 自由度調整済み決定係数の算出
R2_adj = 1 - (SSE/(N - p - 1)) / (SST/(N - 1))
R2_adj.values[0]

[Execution Results]
The value is smaller than the coefficient of determination $${0.858}$$.

Let's extract the adjusted R-squared for the two-variable model from statsmodels and pingouin.

🖲️statsmodels

# statsmodels利用
result1_sm.rsquared_adj

[Execution Results]

🖲️pingouin

# pingouin利用
result1_pg.adj_r2[0]

[Execution Results]

🔢 Comparing multiple regression analyses with different numbers of explanatory variables
As in Table 2.4.2 "Results by SPSS" on p.53 of the text, let's compare the coefficients of determination for two models with explanatory variables:
・Two variables: "Temperature" and "Pressure"
・Three variables: "Temperature", "Pressure", and "Time"

# 独立変数に時間を加えない場合と加える場合の決定係数 p.53 表2.4.2

# 独立変数に加える場合の回帰分析 ※statsmodels利用
result1_3_sm = smf.ols(formula='配向度 ~ 温度 + 圧力 + 時間', data=data1).fit()

# 表の作成
data1_r2_df = pd.DataFrame(
    {'温度 圧力': [result1_sm.rsquared, result1_sm.rsquared_adj],
     '温度 圧力 時間': [result1_3_sm.rsquared, result1_3_sm.rsquared_adj]},
     index=['決定係数', '自由度調整済み決定係数'])
data1_r2_df.loc['差'] = data1_r2_df.iloc[0, :] - data1_r2_df.iloc[1, :]
data1_r2_df.columns.name = '説明変数'

# 結果の表示
data1_r2_df.round(3)

[Execution Results]
The coefficient of determination is larger for the "Temperature, Pressure, Time" model, which has more explanatory variables.
On the other hand, the adjusted R-squared is larger for the "Temperature, Pressure" model, which has fewer explanatory variables.

Multiple correlation coefficient p.54

The multiple correlation coefficient $${R}$$ is the correlation coefficient between the observed values and predicted values of the objective variable.
The square of the multiple correlation coefficient is the coefficient of determination $${R^2}$$.

The "Multiple R" included in the execution results of Excel's "Regression" analysis tool is the multiple correlation coefficient.

🔢 Calculating the multiple correlation coefficient
Let's calculate the multiple correlation coefficient for the two-variable model.
Using pandas' corr(), we will calculate the correlation coefficient between the observed values and the predicted values.

### 重相関係数 p.54
multi_corr1 = data1_pred[['実測値', '予測値']].corr().iloc[0, 1]
multi_corr1

[Execution Result]
The multiple correlation coefficient is 0.926.

Let's confirm that squaring the multiple correlation coefficient matches the coefficient of determination.

### 重相関係数の二乗が決定係数
multi_corr1**2

[Execution Result]
It matches the previously calculated coefficient of determination of 0.858.

AIC (Akaike Information Criterion) p.55

AIC the smaller the value, the better the model's predictive performance.
Note that the text introduces AIC as a statistic indicating the lack of fit.

I will borrow the AIC formula for the multiple regression model from the text.

$${N}$$ is the number of data points (sample size), and $${p}$$ is the number of explanatory variables.

🔢 Calculating AIC
Let's calculate the AIC for the two-variable model.

### AICの算出 p.55

# 設定と準備
N = len(data1)  # 標本サイズ
p = 2           # 説明変数の個数

# 残差平方和の算出
SSE = sum((data1_pred['実測値'] - data1_pred['予測値'])**2)

# AICの算出
AIC1 = N * (np.log(2 * np.pi * (SSE/N)) + 1) + 2 * (p + 2)
AIC1

[Execution Result]

Let's extract the AIC for the two-variable and three-variable models from statsmodels.

🖲️statsmodels
This is the two-variable model.

# statsmodels利用 説明変数が２つ
result1_sm.aic

[Execution Result]

The reason it differs from the value calculated using the text's formula is that it uses a different calculation method than the text.
According to the statsmodels documentation, for models including an intercept, it is calculated using the following formula:

I feel that statsmodels is more common (this is my personal opinion).

This is the AIC for the three-variable model.

# statsmodels利用 説明変数が３つ
result1_3_sm.aic

[Execution Result]

The two-variable model with the smaller AIC value can be evaluated as having a better fit.

(Bonus) ANOVA Table
The "ANOVA table" sits at the end of page 55 of the textbook.
Let's create an ANOVA table using the regression analysis results of the two-variable model from statsmodels.

# 重回帰の分散分析表の例 statsmodels利用
sm.stats.anova_lm(result1_sm)

[Execution Result]
The variation due to regression is displayed separately for each explanatory variable.
The variation due to residuals is shown in the Residual row.

More details in the next article!

What Partial Regression Coefficients Mean p.56

What do the coefficients of each explanatory variable, the "partial regression coefficients," mean in a multiple regression equation...?

Let's verify this meaning with actual calculations!

■ The regression coefficient of simple regression analysis and the partial regression coefficient of multiple regression analysis are different
Let's draw the "path diagram" from page 56 onwards of the textbook and confirm thatthe regression coefficient of simple regression analysis and the partial regression coefficient of multiple regression analysis are different.
As preparation, I will execute simple regression analysis and multiple regression analysis using statsmodels.

### 重回帰分析のパス図 p.56 図2.5.1～図2.5.3

## 係数の推定 by statsmodels

# 説明変数: 温度
result1_sm_temp = smf.ols(formula='配向度 ~ 温度', data=data1).fit()
b1_temp = result1_sm_temp.params.iloc[1]

# 説明変数: 圧力
result1_sm_press = smf.ols(formula='配向度 ~ 圧力', data=data1).fit()
b1_press = result1_sm_press.params.iloc[1]

# 説明変数: 温度、圧力
result1_sm_temp_press = smf.ols(formula='配向度 ~ 温度 + 圧力', data=data1).fit()
b1, b2 = result1_sm_temp_press.params.iloc[1:]

[Execution Result] None

I will draw a path diagram for the multiple regression analysis of the objective variable and two explanatory variables.
I will use the directed graph Digraph from the graphviz library.

## 重回帰分析のパス図 図2.5.1

## 設定
# 有向グラフオブジェクトの生成、neatoでnodeの位置調整を実施
g = Digraph(engine='neato')
# nodeの基本属性の設定
g.attr('node', shape='box', fontname='Meiryo UI')

## node：頂点の作成、posで位置固定
g.node('温度x1', pos='0, 1!')
g.node('圧力x2', pos='0, 0!')
g.node('配向度y', pos='2, 0.5!')

## edge：辺の作成
g.edge('温度x1', '配向度y', label=f'b1={b1:.3f}')
g.edge('圧力x2', '配向度y', label=f'b2={b2:.3f}')

## グラフの表示
g

[Execution Result]
The equations near the arrows are the partial regression coefficients.

Next is the path diagram for simple regression analysis using only temperature as an explanatory variable.

## 単回帰分析のパス図 図2.5.2

## 設定
# 有向グラフオブジェクトの生成、neatoでnodeの位置調整を実施
g = Digraph(engine='neato')
# nodeの基本属性の設定
g.attr('node', shape='box', fontname='Meiryo UI')

## node：頂点の作成、posで位置固定
g.node('温度x1', pos='0, 0!')
g.node('配向度y', pos='2, 0!')

## edge：辺の作成
g.edge('温度x1', '配向度y', label=f'b={b1_temp:.3f}')

## グラフの表示
g

[Execution Result]
It is a different value from the partial regression coefficient $${b_1}$$ of the multiple regression analysis.

Next is the path diagram for simple regression analysis using only pressure as an explanatory variable.

## 単回帰分析のパス図 図2.5.3

## 設定
# 有向グラフオブジェクトの生成、neatoでnodeの位置調整を実施
g = Digraph(engine='neato')
# nodeの基本属性の設定
g.attr('node', shape='box', fontname='Meiryo UI')

## node：頂点の作成、posで位置固定
g.node('圧力x2', pos='0, 0!')
g.node('配向度y', pos='2, 0!')

## edge：辺の作成
g.edge('圧力x2', '配向度y', label=f'b={b1_press:.3f}')

## グラフの表示
g

[Execution Result]
This is also a different value from the partial regression coefficient $${b_2}$$ of the multiple regression analysis.

Regarding the difference in coefficients between simple regression analysis and multiple regression analysis, the textbook states that it is "because the explanatory variables are exerting some influence on each other."
Let's check the correlation coefficient between temperature and pressure.

### 説明変数間の相関係数 p.57 ※pandas利用
data1.corr().loc['温度', '圧力']

[Execution Result]
There was a positive correlation!

■ Connecting the regression coefficient of simple regression analysis and the partial regression coefficient of multiple regression analysis
Following the textbook, I will confirm the connection between the regression coefficient of simple regression analysis and the partial regression coefficient of multiple regression analysis through the regression coefficient and partial regression coefficient of temperature.

I will confirm the influence of pressure on temperature using simple regression analysis.

### 温度と圧力の単回帰式 p.57 ※statsmodels利用

result1_sm_press2temp = smf.ols(formula='温度 ~ 圧力', data=data1).fit()
b0, b1 = result1_sm_press2temp.params
print(f'温度 = {b1:.4f} x 圧力 + {b0:.4f}')

[Execution Result]
The regression coefficient is $${0.1029}$$.

Removing the effect of pressure from temperature means finding the residuals (the difference between the actual and predicted values) from the simple regression analysis of temperature and pressure.
Let's try it.

### 温度と圧力の単回帰分析における残差V p.58 図2.5.1

# データフレームの作成
data1_V = data1[['温度', '圧力']].copy()
data1_V['予測値'] = result1_sm_press2temp.fittedvalues
data1_V['残差V'] = data1_V['温度'] - data1_V['予測値']

# 結果の表示
data1_V

[Execution Result]
The residual $${V}$$ is the value obtained by subtracting the predicted value from the actual temperature value.
This residual becomes the "temperature with the effect of pressure removed."

Similarly, let's remove the effect of pressure from the degree of orientation, which is the objective variable.

### 配向度と圧力の単回帰分析における残差W p.58 図2.5.1

# データフレームの作成
data1_W = data1[['配向度', '圧力']].copy()
data1_W['予測値'] = result1_sm_press.fittedvalues
data1_W['残差W'] = data1_W['配向度'] - data1_W['予測値']

# 結果の表示
data1_W

[Execution Result]
The residual $${W}$$ is the value obtained by subtracting the predicted value from the actual degree of orientation value.
This residual becomes the "degree of orientation with the effect of pressure removed."

Finally, we perform a simple regression analysis of the two residuals—that is, a simple regression analysis of the degree of orientation with the effect of pressure removed and the temperature—to confirm the connection between the coefficients.

### 残差Wと残差Vの単回帰式の回帰係数の算出 p.59 ※statsmodels利用

# 残差Wと残差Vのデータフレームの作成: statsmodels用
result1_WV_df = pd.concat([data1_W['残差W'], data1_V['残差V']], axis=1)
# 単回帰分析の実行
result1_sm_V2W = smf.ols(formula='残差W ~ 残差V', data=result1_WV_df).fit()
# 係数の推定値を取得して表示
b0, b1 = result1_sm_V2W.params
print(f'残差W = {b1:.3f} x 残差V + {b0:.3f}')

[Execution Result]
The regression coefficient $${3.470}$$ for residual $${V}$$ = temperature (after removing the effect of pressure) matched the partial regression coefficient $${3.470}$$ for temperature in the multiple regression analysis of the two variables!

As the text states, it was found that the "partial regression coefficient $${3.470}$$ indicating the degree of influence of temperature on the degree of orientation after removing the effect of pressure" is equal to the "regression coefficient $${3.470}$$ of the simple regression analysis of temperature on the degree of orientation."

🛸 A little detour: Partial Regression Plots 🛸

So far, we have confirmed the relationship between "temperature and degree of orientation" with the effect of pressure removed through calculations.
You want to visualize it more casually with a chart, right?
You can visualize it with a "partial regression plot"!
You can see the relationship between one explanatory variable and the objective variable while "removing the influence of the remaining explanatory variables."
Actually, I learned about the existence of this plot from ChatGPT.

You can easily draw it using statsmodels' plot_partregress.

### 部分回帰プロット

# 追加インポート
from statsmodels.graphics.regressionplots import plot_partregress

# 変数の設定
target = '配向度'        # 目的変数名
vars = ['温度', '圧力']  # 説明変数名

# 描画領域の設定
fig, axes = plt.subplots(1, 2, figsize=(8, 4), tight_layout=True)
# 説明変数ごとに部分回帰プロット描画を繰り返し処理
for var, ax in zip(vars, axes.flat):
    # 部分回帰プロットの描画
    plot_partregress(
        endog=target,   # 目的変数
        exog_i=var,     # 説明変数
        exog_others=[exc_v for exc_v in vars if exc_v != var], # 除外する説明変数
        data=data1,     # データフレーム
        obs_labels=False,  # 散布図にインデックスを付記するかどうか 
        ax=ax,
    )
    # 修飾
    ax.set_title(f'{var}の部分回帰プロット')
    ax.set_xlabel(f'残差化した{var}', fontsize=12)
    ax.set_ylabel(f'残差化した{target}', fontsize=12)
plt.show()

[Execution Result]
The slope of the line on the left is the partial regression coefficient for temperature.
The slope of the line on the right is the partial regression coefficient for pressure.

Here is the code to draw a partial regression plot using "simpler code" based on the regression analysis results from statsmodels' ols.

### 部分回帰プロット グリッド版

# 追加インポート
from statsmodels.graphics.regressionplots import plot_partregress_grid

## 描画
# 描画領域の設定
fig = plt.figure(figsize=(8, 4))
# 部分回帰プロットの描画
plot_partregress_grid(result1_sm, exog_idx=vars, fig=fig)
# 修飾
plt.tight_layout();

[Execution Result]

I asked ChatGPT how to read partial regression plots!

1. Overview of partial regression plots

• This is a graph to see "how much only this variable is effective."

• The procedure is roughly 3 steps:

  1. Remove the influence of other variables

     • Subtract the part that can be explained by the "remaining explanatory variables" from the objective variable y to get the residuals

     • Similarly, subtract the part that can be explained by the "remaining explanatory variables" from the target explanatory variable X₁ to get the residuals

  2. Plot the remaining "residuals" against each other

     • Align the "residuals of X₁" on the horizontal axis and the "residuals of y" on the vertical axis

  3. Fit a straight line

     • The slope of that line almost matches the regression coefficient of X₁ when other variables are fixed.

2. How to read it

• If the scatter is small and follows a straight line
  → Even if the influence of other variables is removed, X₁ explains y well

• If the points are scattered or there are many outliers
  → X₁ alone may have weak explanatory power, or the influence of outliers may be significant

• Direction and magnitude of the slope of the straight line
  → If '+', y increases as X₁ increases; if '-', the direction is reversed
  → The magnitude of the slope represents the amount of change per unit

Standardized Partial Regression Coefficient p.60

Between temperature and pressure, which has a greater influence on the degree of orientation...?
Partial regression coefficients in multiple regression analysis cannot be compared simply because the units of the explanatory variables differ.

This is where the 'standardized partial regression coefficient' comes in.
It is the partial regression coefficient of a multiple regression equation using standardized objective and explanatory variables.

Note that while the textbook uses the term 'standard partial regression coefficient', this article uses 'standardized partial regression coefficient', which I am personally more accustomed to.

Data standardization is the operation of subtracting the mean of the data from the data and dividing by the standard deviation of the data.
Standardized data has a mean of 0 and a variance of 1.

🔢 Calculation of standardized partial regression coefficients
Let's standardize the data for orientation, temperature, and pressure, calculate the standardized partial regression coefficients, and compare the magnitude of the influence of temperature and pressure.

First, data standardization.
This corresponds to Table 2.5.7 'Data Standardization' on p.61 of the textbook.

### データの標準化 p.62 表2.5.7
data1_std = (data1[['配向度', '温度', '圧力']]
             .apply(lambda x: (x - x.mean()) / x.std(ddof=1), axis=0))
data1_std

[Execution Result]
Each variable has become 'unitless'.

Perform multiple regression analysis using the standardized data.
This corresponds to Table 2.5.8 on p.62 of the textbook.
We will use statsmodels.

### 重回帰分析の実行 標準化偏回帰係数の算出 p.62 表2.5.8 ※statsmodels利用
result1_std_sm = smf.ols(formula='配向度 ~ 温度 + 圧力', data=data1_std).fit()
result1_std_sm.summary2().tables[1].round(4)

[Execution Result]
Coef. is the standardized partial regression coefficient.
Comparing the standardized partial regression coefficients, temperature is $${0.5576}$$ and pressure is $${0.4836}$$.
Temperature has a greater influence on the degree of orientation!

🔢 Calculation of standardized partial regression coefficients using correlation coefficients
I will borrow the formulas from the textbook.

📊 Formula for partial regression coefficients using the variance-covariance matrix

📊 Formula for standardized partial regression coefficients using the correlation matrix
$${b_1^*, b_2^*}$$ are the standardized partial regression coefficients.

We will calculate it using the formula for standardized partial regression coefficients using the correlation matrix.

### 相関行列を用いた標準化偏回帰係数の算出 p.63

# x1とy, x2とyの相関係数の算出
corr_vec = data1.corr().loc['配向度', ['温度', '圧力']].values
print('x1とy, x2とyの相関係数:')
print(corr_vec, '\n')

# x1とx2の相関行列の算出
corr_mtx = data1[['温度', '圧力']].corr().values
print('x1とx2の相関行列:')
print(corr_mtx, '\n')

# 標準化偏回帰係数の算出
b1, b2 = np.linalg.inv(corr_mtx) @ corr_vec
print('標準化偏回帰係数:')
print(f'b1: {b1:.4f},  b2: {b2:.4f}')

【Execution Result】
We were able to calculate the standardized partial regression coefficients!

🔢 Calculating standardized partial regression coefficients using partial regression coefficients
This is a method that does not require standardizing the data 🙆‍♂️.

Let's convert the statsmodels partial regression coefficients (non-standardized data) into standardized partial regression coefficients.

### 標準化しないデータの偏回帰係数を標準化偏回帰係数に変換

# 説明変数Xと目的変数yの標準偏差を算出
std_X = data1[['温度', '圧力']].std(ddof=1)
std_y = data1['配向度'].std(ddof=1)

# 標準化偏回帰係数 ＝ 通常の偏回帰係数 × Xの標準偏差 ÷ yの標準偏差
(result1_sm.params[1:] * std_X / std_y).rename('標準化偏回帰係数').to_frame()

【Execution Result】
They have transformed into standardized partial regression coefficients!
Now you can get standardized partial regression coefficients without having to standardize the data!

ChatGPT will wrap up the end of the article.
This time, I'll compare it to a morning routine.

📘 A word from ChatGPT:

That is all for this copy-coding session.

Series Articles

Next Article

Previous Article

Table of Contents

Blog Introduction

I am writing 7 series articles on note.
Please take a look!

1. Relaxed Statistics

This is a blog that roughly explores probability and statistics using the Statistical Test Grade 2 problem collection as a guide.
Feel free to treat it like casual conversation. Please take a look.
It corresponds to the Statistical Test Grade 2 Official Problem Collection CBT Edition.
Sample code for Python and EXCEL is also available.

2. Experiment! Fun Bayesian Modeling 1 & 2 with PyMC Ver. 5

I will draw and analyze the Bayesian models used in psychological research from the books "Fun Bayesian Modeling" and "Fun Bayesian Modeling 2" using PyMC Ver. 5.
Like this book, many Bayesian models are written in R + Stan.
I will strive to explore the possibilities of PyMC and make Bayesian modeling easy to practice.
Since these are familiar and easy-to-visualize themes, please try running them with PyMC and let's enjoy it together!

3. Experiment! Bayesian Modeling from Iwanami Data Science 1 with PyMC Ver. 5

I will draw and analyze the Bayesian models by four Bayesians from the book "Experiment! Iwanami Data Science Vol. 1" using PyMC Ver. 5.
This book is a great resource for learning the basics of Bayesian programming.
I feel like I've become friends with Bayesian methods by happily running PyMC models.
Everyone, please try running them with PyMC and let's play and learn together!

4. Fun Copying: Bayesian, Python, etc.

I will blog about the results of my "book copying activities" for Bayesian, Python, and others.
I am mainly working on translations into Python.
I hope this serves as sample code for fellow learners who are also copying code. 🍀

5. Introduction to Time Series Analysis for Psychology with R and Stan, using Python and PyMC Ver. 5

I will practice the time series analysis from the book "Introduction to Time Series Analysis for Psychology with R and Stan" using Python and PyMC Ver. 5.
This book is packed with themes on time series analysis!
I realized the depth of time series analysis.
I will enjoy learning time series analysis with my favorite language, Python.

6. Writing about Data Science-like things

I write columns on statistics, data analysis, AI, machine learning, and Python on an irregular basis.
There are many articles related to statistics and data science books.
Series on "Statistics," "Python," "Mathematics and Python," and "R" have been created.

7. Python Machine Learning Programming Practice Log

I wrote articles about my various thoughts when studying the book "Python Machine Learning Programming: PyTorch & scikit-learn Edition."
This book is a textbook for scikit-learn and PyTorch.
Please feel free to give it a try.

Thank you very much for reading until the end.