Vector/Scalar and Matrix/Scalar Addition & Subtraction

[PhilSche]

It appears that Blaze is currently missing functionality for vector/scalar and matrix/scalar addition and subtraction. While this operation is mathematically not defined, it could be convenient to provide such functionality as an element-wise addition/subtraction.

You can find an example code below:

// element wise addition of a double to each element of a vector from the right side
blaze::DynamicVector<double> operator+( const blaze::DynamicVector<double> &lhs , const double rhs ) {   
    blaze::DynamicVector<double> new_vector = lhs;
    for (size_t i = 0; i < lhs.size(); ++i){
        new_vector[i] += rhs;
    }
    return new_vector;
}

// element wise addition of a double to each element of a vector from the left side
blaze::DynamicVector<double> operator+( const double lhs , const blaze::DynamicVector<double> &rhs ) {         
    blaze::DynamicVector<double> new_vector = rhs;
    for (size_t i = 0; i < rhs.size(); ++i){
        new_vector[i] += lhs;
    }
    return new_vector;
}

Of course, the same proposal also applies Matrix-types.

[Klaus Iglberger]

Hi Philipp!

Thanks a lot for the proposal. We will consider this for one of the next releases of Blaze. In the mean time you might consider using a UniformVector (*) for the addition. This would help to speed up the computation by means of vectorization and possibly parallelization:

blaze::DynamicVector<double> a;
blaze::UniformVector<double> b;
// ... 
a + b;  // Vector/vector addition as replacement for a vector/scalar addition

Best regards,

Klaus!

(): Please note that UniformVector is one of the new features of Blaze 3.5, which is already available in the master branch of the Blaze* repository but will officially be released in a few weeks.

[Klaus Iglberger]

Summary

The feature has been implemented, tested, optimized, and documented as required. It is immediately available via cloning the Blaze repository and will be officially released in Blaze 3.6.

Scalar Addition

For convenience it is possible to add a scalar value to a dense vector or dense matrix, which has the same effect as adding a uniform vector or matrix (see UniformVector and UniformMatrix). In Blaze it is possible to use all built-in/fundamental data types except bool as scalar values. Additionally, it is possible to use std::complex values with the same built-in data types as element type. Examples:

blaze::DynamicVector<int> v1{ 3, 2, 5, -4, 1, 6 };

// Add 3 to all elements of v1; Results in
//
//    ( 6, 5, 8, -1, 4, 9 )
//
blaze::DynamicVector<int> v2( v1 + 3 );

blaze::DynamicMatrix<int> M1{ {  3, 2, 5 },
                              { -4, 1, 6 } };

// Add 3 to all elements of M1; Results in
//
//    (  6, 5, 8 )
//    ( -1, 4, 9 )
//
blaze::DynamicMatrix<int> M2( M1 + 3 );

Scalar Subtraction

Similar to addition it is also possible to subtract a scalar value from a dense vector or dense matrix, which has the same effect as subtracting a uniform vector or matrix (see UniformVector and UniformMatrix). In Blaze it is possible to use all built-in/fundamental data types except bool as scalar values. Additionally, it is possible to use std::complex values with the same built-in data types as element type. Examples:

blaze::DynamicVector<int> v1{ 3, 2, 5, -4, 1, 6 };

// Subtract 3 from all elements of v1; Results in
//
//    ( 0, -1, 2, -7, -2, 3 )
//
blaze::DynamicVector<int> v2( v1 - 3 );

blaze::DynamicMatrix<int> M1{ {  3, 2, 5 },
                              { -4, 1, 6 } };

// Subtract 3 from all elements of M1; Results in
//
//    (  0, -1, 2 )
//    ( -7, -2, 3 )
//
blaze::DynamicMatrix<int> M2( M1 - 3 );

[Johannes Czech]

@Klaus Iglberger I highly support your decision on this.

Other common numerical libraries such as numpy or eigen also support this operation, so most people expect this behaviour.

‌

One small thing, I noticed is that the new operations aren’t commutative:

blaze::DynamicMatrix<int> M2( M1 + 3 ); // works
blaze::DynamicMatrix<int> M3( 3 + M1 ); // doesn't work

Also, scalar operations aren’t applicable when using them on intermediate results, but Matrices/Vectors are:

blaze::DynamicMatrix<int> M4( M1 + (M1 + 3) ); // works
blaze::DynamicMatrix<int> M5( 1 + (M1 + 3) ); // doesn't work

Can these operations be supported, too?

‌

Best regards,

Johannes Czech

[Klaus Iglberger]

Hi Johannes!

I deliberately ignored scalar/vector addition and scalar/vector subtraction when implementing these operations. From my point of view they don’t feel natural and intuitive. However, thinking about this a second time I realized that scalar/vector subtraction can indeed make a difference in terms of readability and performance:

blaze::DynamicVector<int> a{ 3, 5, -1, 2 };
blaze::DynamicVector<int> b( 4 - a );   // Efficient way to express scalar/vector subtraction
blaze::DynamicVector<int> c( -a + 4 );  // More inefficient way to express the same

Therefore the two commits b441d35 and 36dab40 retrofit the requested operations. You are now be able to perform scalar addition and subtraction all variations you mentioned.

Best regards,

Klaus!

[Johannes Czech]

Thank you very much @Klaus Iglberger ! I think many people will appreciate this.

The remaining operator which isn’t supported in both directions is scalar/vector division.

In numpy and xtensor this operation is interpreted as element-wise division.

blaze::DynamicVector<float> V1{3, 2, 5 };
blaze::DynamicVector<float> V2(2.0f * V1);        // works
blaze::DynamicVector<float> V3(2.0f * (V1 + 3));  // works
blaze::DynamicVector<float> V4(V1 / (V1 + 3));    // works
blaze::DynamicVector<float> V5(V1 / 2.0f);        // works
blaze::DynamicVector<float> V8((V1 + 3) / 2.0f);  // works
blaze::DynamicVector<float> V9(2.0f / V1);        // doesn't work
blaze::DynamicVector<float> V10(2.0f / (V1 + 3)); // doesn't work

Can this be integrated in Blaze as well or should I create my own operator for this use case?

Best regards,

Johannes Czech

[Klaus Iglberger]

Hi Johannes!

You are correct, for consistency reasons it should also be possible to perform an elementwise scalar/vector division. Commits 58fba06 and c671027 retrofit this operation for both dense vectors and dense matrices, respectively.

Best regards,

Klaus!

[Johannes Czech]

Great! Thank you for prompt reply.

[Johannes Czech]

Hello once again @Klaus Iglberger ,

I noticed that the following operator overloads are still missing in Blaze 3.7 for theCompressedVector / CompressedMatrix class:

blaze::CompressedVector<float> V1{3, 2, 5 };
blaze::CompressedVector<float> V2(2.0f * V1);        // works
blaze::CompressedVector<float> V3(2.0f * (V1 + 3));  // doesn't work
blaze::CompressedVector<float> V4(V1 / (V1 + 3));    // doesn't work
blaze::CompressedVector<float> V5(V1 / 2.0f);        // works
blaze::CompressedVector<float> V8((V1 + 3) / 2.0f);  // doesn't work
blaze::CompressedVector<float> V9(2.0f / V1);        // doesn't work
blaze::CompressedVector<float> V10(2.0f / (V1 + 3)); // doesn't work

Can this be added or are there certain issues which don’t allow these overloads for compressed variable types?

Best wishes,

Johannes

[Klaus Iglberger]

Hi Johannes!

As the wiki states, scalar addition and subtraction are only available for dense vectors and matrices. Thus the operations were not forgotten, but deliberately not implemented. The reason is the semantic ambiguity of these operations for sparse vectors and matrices:

blaze::CompressedVector<int> a{ 1, 0, 2, 0, 3 };

a + 1;  // What is the expected result?; could be (2, 1, 3, 1, 4) or (2, 0, 3, 0, 4)
a - 1;  // What is the expected result?; could be (0, -1, 1, -1, 2) or (0, 0, 1, 0, 2)

Semantically it is not clear whether the +1 and -1 refer to all elements or only the non-zero elements. Multiplication and division, on the other hand, work because the result is the same for both approaches.

Depending on your expectation, you can easily provide the according implementation yourself. The following addition operator contains the necessary code for both the addition to the non-zero elements and the addition to all elements:

template< typename VT  // Type of the left-hand side dense vector
        , bool TF      // Transpose flag of the left-hand side dense vector
        , typename ST  // Type of the right-hand side scalar
        , EnableIf_t< IsNumeric_v<ST> >* = nullptr >
inline decltype(auto) operator+( const SparseVector<VT,TF>& vec, ST scalar )
{
   // Addition to the non-zero elements
   using ScalarType = AddTrait_t< UnderlyingBuiltin_t<VT>, ST >;
   return map( ~vec, blaze::bind2nd( Add{}, ScalarType( scalar ) ) );

   // Addition to all elements
   //return (~vec) + uniform( (~vec).size(), scalar );
}

I hope this helps,

Best regards,

Klaus!

[Johannes Czech]

Thanks for the reply and the sample code.

Intuitively, I expected it to behave as in the first case (addition to all elements), but I understand your point of view regarding ambiguity.

Best regards,

Johannes