Linear Algebra and Dimensionality Reduction

Structure of Scalar, vector, matrix and tensor

scalar -> Row vector -> Column Vector -> Matrix -> Tensor(high dimensional matrix) 

Figure 1. Scalar - Vector - Matrix - Tensor Structure

Figure 1 shows the structure of scalar->vector->matrix->tensor.

• A scalar is a single value, but it can be considered a particular vector case where the vector has only one element.
• A vector is an array of values in a row or a column. A row vector is a set of x values, while a column vector is a set of y values.
• A matrix is a two-dimensional array of numbers where a row and column identify each element. A matrix can indeed be thought of as a set of (x, y) pairs, but it is important to note that the elements of a matrix can also be scalars, not just pairs of values.
• A tensor is a multi-dimensional array where multiple indices identify each element. Tensors can have any number of dimensions, but the most common ones are scalars (0-dimensional), vectors (1-dimensional), and matrices (2-dimensional). The concept of a set of matrix sets is not well-defined in the context of tensors. Instead, it is more accurate to think of a tensor as a multi-dimensional array of scalar values.

Linear Algebra: Matrices

• n x p matrix M is an array of n rows and p columns
• The matrix can be written with double underlining
• The matrix transposes exchange rows and columns