Wiki

Hermitian Matrices
HermitianMatrix
Hermitian Matrices vs. Symmetric Matrices
Special Properties of Hermitian Matrices
Arithmetic Operations
Performance Considerations

Hermitian Matrices

In addition to Symmetric Matrices, Blaze also provides an adaptor for Hermitian matrices. Hermitian matrices provide the compile time guarantee to be square matrices with pair-wise conjugate complex values. Mathematically, this means that an Hermitian matrix is always equal to its conjugate transpose and that all non-diagonal values have a complex conjugate counterpart. Within the Blaze library, Hermitian matrices are realized by the HermitianMatrix class template.

HermitianMatrix

The blaze::HermitianMatrix class template is an adapter for existing dense and sparse matrix types. It inherits the properties and the interface of the given matrix type MT and extends it by enforcing the additional invariant of Hermitian symmetry (i.e. the matrix is always equal to its conjugate transpose). It can be included via the header files

#include <blaze/Blaze.h>
// or
#include <blaze/Math.h>
// or
#include <blaze/math/HermitianMatrix.h>

and forward declared via the header file

#include <blaze/Forward.h>

The type of the adapted matrix can be specified via template parameter:

namespace blaze {

template< typename MT >
class HermitianMatrix;

} // namespace blaze

MT specifies the type of the matrix to be adapted. HermitianMatrix can be used with any non-cv-qualified, non-reference, non-pointer, non-expression dense or sparse matrix type. Also, the given matrix type must have numeric element types (i.e. all integral types except bool, floating point and complex types). Note that the given matrix type must be either resizable (as for instance blaze::HybridMatrix or blaze::DynamicMatrix) or must be square at compile time (as for instance blaze::StaticMatrix).

The following examples give an impression of several possible Hermitian matrices:

using blaze::unaligned;
using blaze::unpadded;
using blaze::rowMajor;
using blaze::columnMajor;

// Definition of a 3x3 row-major dense Hermitian matrix with static memory
blaze::HermitianMatrix< blaze::StaticMatrix<int,3UL,3UL,rowMajor> > A;

// Definition of a resizable column-major dense Hermitian matrix based on HybridMatrix
blaze::HermitianMatrix< blaze::HybridMatrix<float,4UL,4UL,columnMajor> B;

// Definition of a resizable row-major dense Hermitian matrix based on DynamicMatrix
blaze::HermitianMatrix< blaze::DynamicMatrix<std::complex<double>,rowMajor> > C;

// Definition of a fixed size row-major dense Hermitian matrix based on CustomMatrix
blaze::HermitianMatrix< blaze::CustomMatrix<double,unaligned,unpadded,rowMajor> > D;

// Definition of a compressed row-major single precision complex Hermitian matrix
blaze::HermitianMatrix< blaze::CompressedMatrix<std::complex<float>,rowMajor> > E;

The storage order of an Hermitian matrix is depending on the storage order of the adapted matrix type MT. In case the adapted matrix is stored in a row-wise fashion (i.e. is specified as blaze::rowMajor), the Hermitian matrix will also be a row-major matrix. Otherwise, if the adapted matrix is column-major (i.e. is specified as blaze::columnMajor), the Hermitian matrix will also be a column-major matrix.

Hermitian Matrices vs. Symmetric Matrices

The blaze::HermitianMatrix adaptor and the blaze::SymmetricMatrix adaptor share several traits. However, there are a couple of differences, both from a mathematical point of view as well as from an implementation point of view.

From a mathematical point of view, a matrix is called symmetric when it is equal to its transpose and it is called Hermitian when it is equal to its conjugate transpose. For matrices of real values, however, these two conditions coincide, which means that symmetric matrices of real values are also Hermitian and Hermitian matrices of real values are also symmetric.

From an implementation point of view, Blaze restricts Hermitian matrices to numeric data types (i.e. all integral types except bool, floating point and complex types), whereas symmetric matrices can also be block matrices (i.e. can have vector or matrix elements). For built-in element types, the HermitianMatrix adaptor behaves exactly like the according SymmetricMatrix implementation. For complex element types, however, the Hermitian property is enforced (see also The Hermitian Property is Always Enforced).

using blaze::DynamicMatrix;
using blaze::DynamicVector;
using blaze::HermitianMatrix;
using blaze::SymmetricMatrix;

// The following two matrices provide an identical experience (including performance)
HermitianMatrix< DynamicMatrix<double> > A;  // Both Hermitian and symmetric
SymmetricMatrix< DynamicMatrix<double> > B;  // Both Hermitian and symmetric

// The following two matrices will behave differently
HermitianMatrix< DynamicMatrix< complex<double> > > C;  // Only Hermitian
SymmetricMatrix< DynamicMatrix< complex<double> > > D;  // Only symmetric

// Hermitian block matrices are not allowed
HermitianMatrix< DynamicMatrix< DynamicVector<double> > > E;  // Compilation error!
SymmetricMatrix< DynamicMatrix< DynamicVector<double> > > F;  // Symmetric block matrix

Special Properties of Hermitian Matrices

An Hermitian matrix is used exactly like a matrix of the underlying, adapted matrix type MT. It also provides (nearly) the same interface as the underlying matrix type. However, there are some important exceptions resulting from the Hermitian symmetry constraint:

Hermitian Matrices Must Always be Square

In case a resizable matrix is used (as for instance blaze::HybridMatrix, blaze::DynamicMatrix, or blaze::CompressedMatrix), this means that the according constructors, the resize() and the extend() functions only expect a single parameter, which specifies both the number of rows and columns, instead of two (one for the number of rows and one for the number of columns):

using blaze::DynamicMatrix;
using blaze::HermitianMatrix;
using blaze::rowMajor;

// Default constructed, default initialized, row-major 3x3 Hermitian dynamic matrix
HermitianMatrix< DynamicMatrix<std::complex<double>,rowMajor> > A( 3 );

// Resizing the matrix to 5x5
A.resize( 5 );

// Extending the number of rows and columns by 2, resulting in a 7x7 matrix
A.extend( 2 );

In case a matrix with a fixed size is used (as for instance blaze::StaticMatrix), the number of rows and number of columns must be specified equally:

using blaze::StaticMatrix;
using blaze::HermitianMatrix;
using blaze::columnMajor;

// Correct setup of a fixed size column-major 3x3 Hermitian static matrix
HermitianMatrix< StaticMatrix<std::complex<float>,3UL,3UL,columnMajor> > A;

// Compilation error: the provided matrix type is not a square matrix type
HermitianMatrix< StaticMatrix<std::complex<float>,3UL,4UL,columnMajor> > B;

The Hermitian Property is Always Enforced

This means that the following properties of an Hermitian matrix are always guaranteed:

The diagonal elements are real numbers, i.e. the imaginary part is zero
Element a_ij is always the complex conjugate of element a_ji

Thus modifying the element a_ij of an Hermitian matrix also modifies its counterpart element a_ji. Also, it is only possible to assign matrices that are Hermitian themselves:

using blaze::CompressedMatrix;
using blaze::DynamicMatrix;
using blaze::StaticMatrix;
using blaze::HermitianMatrix;
using blaze::rowMajor;

using cplx = std::complex<double>;

// Default constructed, row-major 3x3 Hermitian compressed matrix
HermitianMatrix< CompressedMatrix<cplx,rowMajor> > A( 3 );

// Initializing the matrix via the function call operator
//
//  ( (1, 0) (0,0) (2,1) )
//  ( (0, 0) (0,0) (0,0) )
//  ( (2,-1) (0,0) (0,0) )
//
A(0,0) = cplx( 1.0, 0.0 );  // Initialization of the diagonal element (0,0)
A(0,2) = cplx( 2.0, 1.0 );  // Initialization of the elements (0,2) and (2,0)

// Inserting three more elements via the insert() function
//
//  ( (1,-3) (0,0) (2, 1) )
//  ( (0, 0) (2,0) (4,-2) )
//  ( (2,-1) (4,2) (0, 0) )
//
A.insert( 1, 1, cplx( 2.0,  0.0 ) );  // Inserting the diagonal element (1,1)
A.insert( 1, 2, cplx( 4.0, -2.0 ) );  // Inserting the elements (1,2) and (2,1)

// Access via a non-const iterator
//
//  ( (1,-3) (8,1) (2, 1) )
//  ( (8,-1) (2,0) (4,-2) )
//  ( (2,-1) (4,2) (0, 0) )
//
*A.begin(1UL) = cplx( 8.0, -1.0 );  // Modifies both elements (1,0) and (0,1)

// Erasing elements via the erase() function
//
//  ( (0, 0) (8,1) (0, 0) )
//  ( (8,-1) (2,0) (4,-2) )
//  ( (0, 0) (4,2) (0, 0) )
//
A.erase( 0, 0 );  // Erasing the diagonal element (0,0)
A.erase( 0, 2 );  // Erasing the elements (0,2) and (2,0)

// Construction from an Hermitian dense matrix
StaticMatrix<cplx,3UL,3UL> B{ { cplx(  3.0,  0.0 ), cplx(  8.0, 2.0 ), cplx( -2.0,  2.0 ) },
                              { cplx(  8.0,  1.0 ), cplx(  0.0, 0.0 ), cplx( -1.0, -1.0 ) },
                              { cplx( -2.0, -2.0 ), cplx( -1.0, 1.0 ), cplx(  4.0,  0.0 ) } };

HermitianMatrix< DynamicMatrix<double,rowMajor> > C( B );  // OK

// Assignment of a non-Hermitian dense matrix
StaticMatrix<cplx,3UL,3UL> D{ { cplx(  3.0, 0.0 ), cplx(  7.0, 2.0 ), cplx( 3.0, 2.0 ) },
                              { cplx(  8.0, 1.0 ), cplx(  0.0, 0.0 ), cplx( 6.0, 4.0 ) },
                              { cplx( -2.0, 2.0 ), cplx( -1.0, 1.0 ), cplx( 4.0, 0.0 ) } };

C = D;  // Throws an exception; Hermitian invariant would be violated!

The same restriction also applies to the append() function for sparse matrices: Appending the element a_ij additionally inserts the element a_ji into the matrix. Despite the additional insertion, the append() function still provides the most efficient way to set up an Hermitian sparse matrix. In order to achieve the maximum efficiency, the capacity of the individual rows/columns of the matrix should to be specifically prepared with reserve() calls:

using blaze::CompressedMatrix;
using blaze::HermitianMatrix;
using blaze::rowMajor;

using cplx = std::complex<double>;

// Setup of the Hermitian matrix
//
//       ( (0, 0) (1,2) (3,-4) )
//   A = ( (1,-2) (2,0) (0, 0) )
//       ( (3, 4) (0,0) (0, 0) )
//
HermitianMatrix< CompressedMatrix<cplx,rowMajor> > A( 3 );

A.reserve( 5 );         // Reserving enough space for 5 non-zero elements
A.reserve( 0, 2 );      // Reserving two non-zero elements in the first row
A.reserve( 1, 2 );      // Reserving two non-zero elements in the second row
A.reserve( 2, 1 );      // Reserving a single non-zero element in the third row

A.append( 0, 1, cplx( 1.0, 2.0 ) );  // Appending an element at position (0,1) and (1,0)
A.append( 1, 1, cplx( 2.0, 0.0 ) );  // Appending an element at position (1,1)
A.append( 2, 0, cplx( 3.0, 4.0 ) );  // Appending an element at position (2,0) and (0,2)

The Hermitian property is also enforced for Hermitian custom matrices: In case the given array of elements does not represent an Hermitian matrix, a std::invalid_argument exception is thrown:

using blaze::CustomMatrix;
using blaze::HermitianMatrix;
using blaze::unaligned;
using blaze::unpadded;
using blaze::rowMajor;

using CustomHermitian = HermitianMatrix< CustomMatrix<double,unaligned,unpadded,rowMajor> >;

// Creating a 3x3 Hermitian custom matrix from a properly initialized array
double array[9] = { 1.0, 2.0, 4.0,
                    2.0, 3.0, 5.0,
                    4.0, 5.0, 6.0 };
CustomHermitian A( array, 3UL );  // OK

// Attempt to create a second 3x3 Hermitian custom matrix from an uninitialized array
std::unique_ptr<double[]> memory( new double[9UL] );
CustomHermitian B( memory.get(), 3UL );  // Throws an exception

Finally, the Hermitian property is enforced for views (rows, columns, submatrices, ...) on the Hermitian matrix. The following example demonstrates that modifying the elements of an entire row of the Hermitian matrix also affects the counterpart elements in the according column of the matrix:

using blaze::DynamicMatrix;
using blaze::HermtianMatrix;

using cplx = std::complex<double>;

// Setup of the Hermitian matrix
//
//       ( (0, 0) (1,-1) (0,0) (2, 1) )
//   A = ( (1, 1) (3, 0) (4,2) (0, 0) )
//       ( (0, 0) (4,-2) (0,0) (5,-3) )
//       ( (2,-1) (0, 0) (5,3) (0, 0) )
//
HermitianMatrix< DynamicMatrix<int> > A( 4 );
A(0,1) = cplx( 1.0, -1.0 );
A(0,3) = cplx( 2.0,  1.0 );
A(1,1) = cplx( 3.0,  0.0 );
A(1,2) = cplx( 4.0,  2.0 );
A(2,3) = cplx( 5.0,  3.0 );

// Setting all elements in the 1st row to 0 results in the matrix
//
//       ( (0, 0) (0,0) (0,0) (2, 1) )
//   A = ( (0, 0) (0,0) (0,0) (0, 0) )
//       ( (0, 0) (0,0) (0,0) (5,-3) )
//       ( (2,-1) (0,0) (5,3) (0, 0) )
//
row( A, 1 ) = cplx( 0.0, 0.0 );

The next example demonstrates the (compound) assignment to submatrices of Hermitian matrices. Since the modification of element a_ij of an Hermitian matrix also modifies the element a_ji, the matrix to be assigned must be structured such that the Hermitian symmetry of the matrix is preserved. Otherwise a std::invalid_argument exception is thrown:

using blaze::DynamicMatrix;
using blaze::HermitianMatrix;

std::complex<double>  cplx;

// Setup of two default 4x4 Hermitian matrices
HermitianMatrix< DynamicMatrix<cplx> > A1( 4 ), A2( 4 );

// Setup of the 3x2 dynamic matrix
//
//       ( (1,-1) (2, 5) )
//   B = ( (3, 0) (4,-6) )
//       ( (5, 0) (6, 0) )
//
DynamicMatrix<int> B( 3UL, 2UL );
B(0,0) = cplx( 1.0, -1.0 );
B(0,1) = cplx( 2.0,  5.0 );
B(1,0) = cplx( 3.0,  0.0 );
B(1,1) = cplx( 4.0, -6.0 );
B(2,1) = cplx( 5.0,  0.0 );
B(2,2) = cplx( 6.0,  7.0 );

// OK: Assigning B to a submatrix of A1 such that the Hermitian property is preserved
//
//        ( (0, 0) (0, 0) (1,-1) (2, 5) )
//   A1 = ( (0, 0) (0, 0) (3, 0) (4,-6) )
//        ( (1, 1) (3, 0) (5, 0) (6, 0) )
//        ( (2,-5) (4, 6) (6, 0) (0, 0) )
//
submatrix( A1, 0UL, 2UL, 3UL, 2UL ) = B;  // OK

// Error: Assigning B to a submatrix of A2 such that the Hermitian property isn't preserved!
//   The elements marked with X cannot be assigned unambiguously!
//
//        ( (0, 0) (1,-1) (2,5) (0,0) )
//   A2 = ( (1, 1) (3, 0) (X,X) (0,0) )
//        ( (2,-5) (X, X) (6,0) (0,0) )
//        ( (0, 0) (0, 0) (0,0) (0,0) )
//
submatrix( A2, 0UL, 1UL, 3UL, 2UL ) = B;  // Assignment throws an exception!

The Elements of a Dense Symmetric Matrix are Always Default Initialized

Although this results in a small loss of efficiency (especially in case all default values are overridden afterwards), this property is important since otherwise the Hermitian property of dense Hermitian matrices could not be guaranteed:

using blaze::DynamicMatrix;
using blaze::HermitianMatrix;

// Uninitialized, 5x5 row-major dynamic matrix
DynamicMatrix<int,rowMajor> A( 5, 5 );

// Default initialized, 5x5 row-major Hermitian dynamic matrix
HermitianMatrix< DynamicMatrix<int,rowMajor> > B( 5 );

Arithmetic Operations

A HermitianMatrix can be used within all numerical operations in any way any other dense or sparse matrix can be used. It can also be combined with any other dense or sparse vector or matrix. The following code example gives an impression of the use of HermitianMatrix within arithmetic operations:

using blaze::HermitianMatrix;
using blaze::DynamicMatrix;
using blaze::HybridMatrix;
using blaze::StaticMatrix;
using blaze::CompressedMatrix;
using blaze::rowMajor;
using blaze::columnMajor;

using cplx = complex<float>;

DynamicMatrix<cplx,rowMajor> A( 3, 3 );
CompressedMatrix<cplx,rowMajor> B( 3, 3 );

HermitianMatrix< DynamicMatrix<cplx,rowMajor> > C( 3 );
HermitianMatrix< CompressedMatrix<cplx,rowMajor> > D( 3 );

HermitianMatrix< HybridMatrix<cplx,3UL,3UL,rowMajor> > E;
HermitianMatrix< StaticMatrix<cplx,3UL,3UL,columnMajor> > F;

E = A + B;     // Matrix addition and assignment to a row-major Hermitian matrix (includes runtime check)
F = C - D;     // Matrix subtraction and assignment to a column-major Hermitian matrix (only compile time check)
F = A * D;     // Matrix multiplication between a dense and a sparse matrix (includes runtime check)

C *= 2.0;      // In-place scaling of matrix C
E  = 2.0 * B;  // Scaling of matrix B (includes runtime check)
F  = C * 2.0;  // Scaling of matrix C (only compile time check)

E += A - B;    // Addition assignment (includes runtime check)
F -= C + D;    // Subtraction assignment (only compile time check)
F *= A * D;    // Multiplication assignment (includes runtime check)

Note that it is possible to assign any kind of matrix to an Hermitian matrix. In case the matrix to be assigned is not Hermitian at compile time, a runtime check is performed.

Performance Considerations

When the Hermitian property of a matrix is known beforehands using the HermitianMatrix adaptor instead of a general matrix can be a considerable performance advantage. This is particularly true in case the Hermitian matrix is also symmetric (i.e. has built-in element types). The Blaze library tries to exploit the properties of Hermitian (symmetric) matrices whenever possible. However, there are also situations when using an Hermitian matrix introduces some overhead. The following examples demonstrate several situations where Hermitian matrices can positively or negatively impact performance.

Positive Impact: Matrix/Matrix Multiplication

When multiplying two matrices, at least one of which is symmetric, Blaze can exploit the fact that A = A^T and choose the fastest and most suited combination of storage orders for the multiplication. The following example demonstrates this by means of a dense matrix/sparse matrix multiplication:

using blaze::DynamicMatrix;
using blaze::HermitianMatrix;
using blaze::rowMajor;
using blaze::columnMajor;

HermitianMatrix< DynamicMatrix<double,rowMajor> > A;        // Both Hermitian and symmetric
HermitianMatrix< CompressedMatrix<double,columnMajor> > B;  // Both Hermitian and symmetric
DynamicMatrix<double,columnMajor> C;

// ... Resizing and initialization

C = A * B;

Intuitively, the chosen combination of a row-major and a column-major matrix is the most suited for maximum performance. However, Blaze evaluates the multiplication as

C = A * trans( B );

which significantly increases the performance since in contrast to the original formulation the optimized form can be vectorized. Therefore, in the context of matrix multiplications, using a symmetric matrix is obviously an advantage.

Positive Impact: Matrix/Vector Multiplication

A similar optimization is possible in case of matrix/vector multiplications:

using blaze::DynamicMatrix;
using blaze::DynamicVector;
using blaze::CompressedVector;
using blaze::HermitianMatrix;
using blaze::rowMajor;
using blaze::columnVector;

HermitianMatrix< DynamicMatrix<double,rowMajor> > A;  // Hermitian and symmetric
CompressedVector<double,columnVector> x;
DynamicVector<double,columnVector> y;

// ... Resizing and initialization

y = A * x;

In this example it is not intuitively apparent that using a row-major matrix is not the best possible choice in terms of performance since the computation cannot be vectorized. Choosing a column-major matrix instead, however, would enable a vectorized computation. Therefore Blaze exploits the fact that A is symmetric, selects the best suited storage order and evaluates the multiplication as

y = trans( A ) * x;

which also significantly increases the performance.

Positive Impact: Row/Column Views on Column/Row-Major Matrices

Another example is the optimization of a row view on a column-major symmetric matrix:

using blaze::DynamicMatrix;
using blaze::HermitianMatrix;
using blaze::Row;
using blaze::rowMajor;
using blaze::columnMajor;

using DynamicHermitian = HermitianMatrix< DynamicMatrix<double,columnMajor> >;

DynamicHermitian A( 10UL );  // Both Hermitian and symmetric
Row<DynamicHermitian> row5 = row( A, 5UL );

Usually, a row view on a column-major matrix results in a considerable performance decrease in comparison to a row view on a row-major matrix due to the non-contiguous storage of the matrix elements. However, in case of symmetric matrices, Blaze instead uses the according column of the matrix, which provides the same performance as if the matrix would be row-major. Note that this also works for column views on row-major matrices, where Blaze can use the according row instead of a column in order to provide maximum performance.

Negative Impact: Assignment of a General Matrix

In contrast to using an Hermitian matrix on the right-hand side of an assignment (i.e. for read access), which introduces absolutely no performance penalty, using an Hermitian matrix on the left-hand side of an assignment (i.e. for write access) may introduce additional overhead when it is assigned a general matrix, which is not Hermitian at compile time:

using blaze::DynamicMatrix;
using blaze::HermitianMatrix;

HermitianMatrix< DynamicMatrix< complex<double> > > A, C;
DynamicMatrix<double> B;

B = A;  // Only read-access to the Hermitian matrix; no performance penalty
C = A;  // Assignment of an Hermitian matrix to another Hermitian matrix; no runtime overhead
C = B;  // Assignment of a general matrix to an Hermitian matrix; some runtime overhead

When assigning a general, potentially not Hermitian matrix to an Hermitian matrix it is necessary to check whether the matrix is Hermitian at runtime in order to guarantee the Hermitian property of the Hermitian matrix. In case it turns out to be Hermitian, it is assigned as efficiently as possible, if it is not, an exception is thrown. In order to prevent this runtime overhead it is therefore generally advisable to assign Hermitian matrices to other Hermitian matrices.

In this context it is especially noteworthy that in contrast to additions and subtractions the multiplication of two Hermitian matrices does not necessarily result in another Hermitian matrix:

HermitianMatrix< DynamicMatrix<double> > A, B, C;

C = A + B;  // Results in an Hermitian matrix; no runtime overhead
C = A - B;  // Results in an Hermitian matrix; no runtime overhead
C = A * B;  // Is not guaranteed to result in an Hermitian matrix; some runtime overhead

Previous: Symmetric Matrices ---- Next: Triangular Matrices