Provide support for the softmax function
Issue #192
resolved
Description
The Blaze library should provide a softmax function for both dense vectors and dense matrices. More specifically, it should be possible to ...
- ... evaluate
softmax()
on dense vectors; - ... evaluate
softmax()
on dense matrices.
Conceptually, the softmax()
should work as follows:
Vector Example
// Creating the vector ( 1 2 3 4 1 2 3 )
blaze::StaticVector<double,7UL> x{ 1.0, 2.0, 3.0, 4.0, 1.0, 2.0, 3.0 };
blaze::StaticVector<double,7UL> y;
// Evaluating the softmax function
y = softmax( x ); // Results in ( 0.024 0.064 0.175 0.475 0.024 0.064 0.175 )
Matrix Example
// Creating the matrix
// ( 1 2 3 )
// ( 4 1 2 )
// ( 3 4 1 )
blaze::StaticMatrix<double,3UL,3UL> A{ { 1.0, 2.0, 3.0 }
, { 4.0, 1.0, 2.0 }
, { 3.0, 4.0, 1.0 } };
blaze::StaticMatrix<double,3UL,3UL> B;
// Evaluating the softmax function
B = softmax( A ); // Results in ( 0.0157764 0.0428847 0.116573 )
// ( 0.316878 0.0157764 0.0428847 )
// ( 0.116573 0.316878 0.0157764 )
Tasks
- implement the softmax function for dense vectors
- implement the softmax function for dense matrices
- provide a full documentation of the softmax function
- ensure compatibility with all existing dense vector and matrix classes
- ensure compatibility with all existing dense vector and matrix expressions
- guarantee maximum performance for the operations
- add test cases for the entire softmax functionality
Comments (4)
-
reporter -
reporter -
assigned issue to
- edited description
-
assigned issue to
-
reporter - changed status to open
-
reporter - changed status to resolved
Summary
The feature has been implemented, tested, optimized, and documented as required. It is immediately available via cloning the Blaze repository and will be officially released in Blaze 3.4.
softmax()
The softmax function, also called the normalized exponential function, of a given dense vector or matrix can be computed via
softmax()
. The resulting dense vector or matrix consists of real values in the range(0..1]
, which add up to 1.Vector Example
blaze::StaticVector<double,7UL,rowVector> x{ 1.0, 2.0, 3.0, 4.0, 1.0, 2.0, 3.0 }; blaze::StaticVector<double,7UL,rowVector> y; // Evaluating the softmax function y = softmax( x ); // Results in ( 0.024 0.064 0.175 0.475 0.024 0.064 0.175 ) double s = sum( y ); // Results in 1
Matrix Example
blaze::StaticMatrix<double,3UL,3UL> A{ { 1.0, 2.0, 3.0 } , { 4.0, 1.0, 2.0 } , { 3.0, 4.0, 1.0 } }; blaze::StaticMatrix<double,3UL,3UL> B; // Evaluating the softmax function B = softmax( A ); // Results in ( 0.0157764 0.0428847 0.116573 ) // ( 0.316878 0.0157764 0.0428847 ) // ( 0.116573 0.316878 0.0157764 ) double s = sum( B ); // Results in 1
- Log in to comment
The following two functions implement the
softmax()
function for both vectors and matrices:Please note that both functions make use of the
sum()
function, which is available on master and will be officially released in Blaze 3.4.Please feel free to use these functions until they have been integrated into Blaze and are officially supported.