Outer Product on Vector

Last Updated : 8 May, 2025

The outer product is a fundamental operation in linear algebra that constructs a matrix from two vectors. Unlike the inner (dot) product, which results in a scalar, the outer product produces a matrix, capturing pairwise multiplicative interactions between elements of the two vectors. This operation plays a crucial role in tensor algebra, physics, and is especially prevalent in fields like machine learning and deep learning.

Outer-Product
Geometrical Interpretation of Outer Product

Mathematical Definition of the Outer Product

Given two vectors:

\mathbf{u} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_m \end{bmatrix}, \quad\mathbf{v} = \begin{bmatrix} v_1 & v_2 & \cdots & v_n \end{bmatrix}

their outer product is given by:

\mathbf{A} = \mathbf{u} \otimes \mathbf{v}

where:

A_{ij} = u_i v_j, \quad \text{for } i = 1,2,\dots,m \text{ and } j = 1,2,\dots,n

Thus, the outer product results in an m×n matrix.

Numerical example

Let’s consider the vectors:

\mathbf{u} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}, \quad\mathbf{v} = \begin{bmatrix} 4 & 5 \end{bmatrix}

The outer product is:

\mathbf{A} =\begin{bmatrix}1 \times 4 & 1 \times 5 \\2 \times 4 & 2 \times 5 \\3 \times 4 & 3 \times 5\end{bmatrix}=\begin{bmatrix}4 & 5 \\8 & 10 \\12 & 15\end{bmatrix}

Python Implementation

Using NumPy’s outer() Function

Python 
import numpy as np

# Define vectors
u = np.array([1, 2, 3])
v = np.array([4, 5])

# Compute the outer product
A = np.outer(u, v)

print(A)

Output:

outer-product-2
Outer Product

NumPy's np.outer() function efficiently computes the outer product of two vectors.

Using Matrix Multiplication

The outer product can also be computed by treating the vectors as matrices—specifically, by reshaping one vector as a column vector and the other as a row vector. The outer product is then given by:

\mathbf{A} = \mathbf{u} \mathbf{v}^T

Here, 𝑢 is a column vector and 𝑣⊤ is the transpose of a row vector, resulting in a matrix 𝐴 whose entries are all pairwise products 𝑢𝑖𝑣𝑗.

Python 
# Reshape vectors into column and row format
A_alternative = u.reshape(-1, 1) @ v.reshape(1, -1)

print(A_alternative)

Output:

outer-product-2
Outer Product

Numerical Example in Python

Let’s compute the covariance matrix using the outer product.

\mathbf{C} = \frac{1}{n-1} \sum_{i=1}^{n} (\mathbf{x}_i - \bar{\mathbf{x}})(\mathbf{x}_i - \bar{\mathbf{x}})^T

Python 
import numpy as np

X = np.array([[1, 2], [2, 3], [3, 4]])  

# Compute mean of each column (feature)
mean = np.mean(X, axis=0)

# Center the data by subtracting mean
X_centered = X - mean

# Compute covariance matrix using the outer product
cov_matrix = (X_centered.T @ X_centered) / (X.shape[0] - 1)

print("Covariance Matrix:\n", cov_matrix)

Output:

outer-product-2
Covariance Matrix

This method is equivalent to np.cov(X.T, bias=False).

Difference Between Inner and Outer Products

  • The inner product (dot product) is:

\mathbf{u} \cdot \mathbf{v} = \sum_{i=1}^{n} u_i v_i

It produces a scalar.

  • The outer product is:

\mathbf{A} = \mathbf{u} \otimes \mathbf{v}

It produces a matrix.

Example:

\begin{bmatrix} 1 & 2 & 3 \end{bmatrix} \cdot \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix} = 1(4) + 2(5) + 3(6) = 32

while

\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} \otimes \begin{bmatrix} 4 & 5 \end{bmatrix} =\begin{bmatrix} 4 & 5 \\ 8 & 10 \\ 12 & 15 \end{bmatrix}

Applications of the Outer Product

Tensor Algebra and Physics

  • Used in quantum mechanics to represent density matrices.
  • Helps in constructing higher-order tensors.

Machine Learning and Neural Networks

  • Used in Hebbian learning for weight updates.
  • Plays a role in covariance matrix computations.

Image Processing and Computer Vision

  • Used in low-rank approximations.
  • Helps in feature engineering.

Statistical Analysis

  • The covariance matrix of a dataset is derived using the outer product.
Comment

Explore