Financial Machine Learning: Feature Importance - Feature Orthogonalization

When features are correlated, their importance tends to be underestimated in MDI or MDA estimation due to substitution effects.
By using PCA, which is used for feature dimensionality reduction, and orthogonalizing the features to eliminate linear relationships between them, it becomes possible to analyze feature importance without substitution effects.
Consider a stationary feature matrix $${X_{t,n}}$$, where $${t=1,\dots,T, n=1,\dots.N}$$ for time bars and the number of features, respectively.
Before orthogonalization, this matrix is standardized using the time (T) mean $${{\mu_n}}$$ and variance $${{\sigma_n}}$$ of each feature.
$${Z_{t,n}=\displaystyle{\frac{X_{t,n}-\mu_nu}{\sigma_n}}}$$
This process ensures that the results are not influenced by the variance of the features, and centering by the mean clarifies the principal directions of the features.
The diagonalization of $${{\bf Z}}$$ via eigenvalue decomposition is
$${{\bf Z}^{T}\{bf ZW}=W\varLambda}$$
where $${{\bf W}}$$ is the eigenvector matrix and $${{\bf \varLambda}}$$ is the diagonal matrix of eigenvalues, given by $${{\bf P}={\bf ZW}}$$.
The implementation of this diagonalization is provided in Snippet 8.5.
Here, a variance threshold of 95% is set, limiting the orthogonal features to those provided by this threshold.

def get_eVec(dot,varThres):
    # compute eVec from dot proc matrix, reduce dimension
    eVal,eVec=np.linalg.eigh(dot)
    idx=eVal.argsort()[::-1] # arugments for sorting eVal desc.
    eVal,eVec=eVal[idx],eVec[:,idx]
    #2) only positive eVals
    eVal=(pd.Series(eVal,index=['PC_'+str(i+1)
                                for i in range(eVal.shape[0])]))
    eVec=(pd.DataFrame(eVec,index=dot.index,columns=eVal.index))
    eVec=eVec.loc[:,eVal.index]
    #3) reduce dimension, form PCs
    cumVar=eVal.cumsum()/eVal.sum()
    dim=cumVar.values.searchsorted(varThres)
    eVal,eVec=eVal.iloc[:dim+1],eVec.iloc[:,:dim+1]
    return eVal,eVec

def getOrthoFeats(dfx,varThres=0.95):
    # given a DataFrame, dfx, of features, compute orthofeatures dfP
    dfZ=dfx.sub(dfx.mean(),axis=1).div(dfx.std(),axis=1) # standardize
    dot=(pd.DataFrame(np.dot(dfZ.T,dfZ),
                      index=dfx.columns,
                      columns=dfx.columns))
    eVal,eVec=get_eVec(dot,varThres)
    dfP=np.dot(dfZ,eVec)
    return pd.DataFrame(dfP).add_prefix("ORT_")

Through this process, features with small eigenvalues are omitted, and the feature dimensionality is reduced. The new feature dimensions differ from the originally provided features, and the ranking by PCA importance analysis is based on the values of the eigenvalues; unlike MDI, MDA, and SFI in supervised learning, this is a result of unsupervised learning.
By calculating Kendall's weighted $${{\tau}}$$ between the importance ranking obtained from MDI, MDA, and SFI analysis applied to these orthogonalized features and the PCA ranking, the consistency between these two rankings can be determined.
The reason for using weighted $${{\tau}}$$ here is to place greater emphasis on the most important features.
The calculation of Kendall's weighted $${{\tau}}$$ uses scikit-learn code (Snippet 8.6).

from scipy.stats import weightedtau
featImp=np.array([0.55,0.33,0.07,0.05]) # feature importance
pcRank=np.array([1,2,3,4],dtype=np.float64) # PCA rank
weightedtau(featImp,pcRank**-1)[0]