Addition of row/column vector to whole matrix in row/column-wise fashion (broadcasting)

[Marcin Junczys-Dowmunt]

Hi, Eigen allows for broadcasting when elementwise operations between matrices and vectors are performed that match for the significant dimension, for instance:

A - 30x50 Matrix

X - 50x100 Matrix

b - 1x100 Row vector

(A*X).rowwise() + b

Adds b to all 30 rows of the multiplication result, the equivalent for columns works as well. NumPy for instance, does this automatically for row vectors. Unless I am mistaken there is no such functionality in BLAZE (other than iterating through rows). Would be nice to have with good multi-threading.

[Marcin Junczys-Dowmunt]

I guess a very blaze way to do it would be to add a proxy that allows the following:

C = A * X + broadcast(b);

where b is added to all rows if it is a row vector, or to all columns if b is a column vector.

[Klaus Iglberger]

Thanks, Marcin, for the great idea.

Your are correct, currently the best way to approach the problem with Blaze is to iterate over rows:

M = A*X;
for( size_t i=0UL; i<M.rows(); ++i ) {
   row( M, i ) += b;
}

However, this is neither the fastest possible way nor the most convenient one. Therefore we will consider your idea in one of the next releases. Thanks again,

Best regards,

Klaus!

[Klaus Iglberger]

Summary

The feature has been implemented, tested, optimized, and documented as required. It is immediately available via cloning the Blaze repository and will be officially released in Blaze 3.5.

The expand() function

Via the expand() function it is possible to convert a dense or sparse vector into a matrix. A column vector is expanded into a column-major matrix, a row vector is expanded into a row-major matrix. As demonstrated by the following examples, expand() can be used with both runtime and compile time parameters:

blaze::DynamicVector<int,columnVector> a{ 1, 2, 3 };
blaze::CompressedVector<int,rowVector> b{ 1, 0, 3, 0, 5 };

// Expand the dense column vector ( 1 2 3 ) into a dense 3x5 column-major matrix
//
//   ( 1 1 1 1 1 )
//   ( 2 2 2 2 2 )
//   ( 3 3 3 3 3 )
//
expand( a, 5 );  // Runtime parameter
expand<5>( a );  // Compile time parameter

// Expand the sparse row vector ( 1 0 3 0 5 ) into a sparse 3x5 row-major matrix
//
//   ( 1 0 3 0 5 )
//   ( 1 0 3 0 5 )
//   ( 1 0 3 0 5 )
//
expand( b, 3 );  // Runtime parameter
expand<3>( b );  // Compile time parameter

[Ecolss Logan]

Hi @Klaus Iglberger , good to see this issue has been discussed here before.

I read the doc about vector expansion, but want to double confirm one use-case.

Say, I have 2 matrices as follows

DynamicMatrix<float> A(10, 5);
DynamicMatrix<float> B(10, 1);

A += B;  // expecting numpy-ish row broadcasting

I was expecting that B has only 1 column, and expecting it could be broadcasted when added to A.

But looks like we cannot do it this way, so 2 questions to ask,

1. Instead, take the column of B and expand it, then add to A, right?
2. Since A is row-major, and if I take the column of B and expand, then the resulting expanded matrix is actually column-major, adding them would be OK? Still cache-friendly?

‌

[Klaus Iglberger]

Hi Ecolss!

There is no numpy-ish broadcasting in Blaze. In Blaze it is always expected that matrices are properly sized and there is no implicit assumptions that a user wants to do broadcasting in case the sizes don’t match.

Assuming the two matrices A and B

blaze::DynamicMatrix<int> A( 10, 5 );
blaze::DynamicMatrix<int> B( 10, 1 );

you have three options how to perform this operation.

---

Option 1: Repeating a Matrix

A += repeat<1,10>( B );

You can repeat a matrix, which results in another matrix. See the wiki about repeating matrices for a more detailed explanation.

---

Option 2: Expanding a Row/Column

A += expand<10UL>( column<0>( B ) );

The expand() operation promotes a vector to a matrix. See the wiki about vector expansion to get more information.

You are correct, in this example the matrix resulting from the expand() operation would be considered a column-major matrix. Still, even for large matrices, Blaze performs the addition in a cache-friendly manner by means of blocking. It cannot apply vectorization, though.

---

Option 3: Using a row/column selectionS

A += columns( B, { 0, 0, 0, 0, 0 } );                 // Runtime indices
A += columns<0,0,0,0,0>( B );                         // Compile time indices
A += columns( B, []( size_t ){ return 0UL; }, 5UL );  // Index producer

Via row/column selections you can select arbitrary rows/columns. This also enables you to repeat individual rows/columns or patterns of rows/columns. See the wiki about column selections to get a complete overview.

‌

I hope this helps,

Best regards,

Klaus!

[Ecolss Logan]

Hi Klaus!

Thanks a lot!