Addition of row/column vector to whole matrix in row/column-wise fashion (broadcasting)

Issue #43 resolved
Marcin Junczys-Dowmunt created an issue

Hi, Eigen allows for broadcasting when elementwise operations between matrices and vectors are performed that match for the significant dimension, for instance:

A - 30x50 Matrix

X - 50x100 Matrix

b - 1x100 Row vector

(A*X).rowwise() + b

Adds b to all 30 rows of the multiplication result, the equivalent for columns works as well. NumPy for instance, does this automatically for row vectors. Unless I am mistaken there is no such functionality in BLAZE (other than iterating through rows). Would be nice to have with good multi-threading.

Comments (9)

  1. Marcin Junczys-Dowmunt reporter

    I guess a very blaze way to do it would be to add a proxy that allows the following:

    C = A * X + broadcast(b);

    where b is added to all rows if it is a row vector, or to all columns if b is a column vector.

  2. Klaus Iglberger

    Thanks, Marcin, for the great idea.

    Your are correct, currently the best way to approach the problem with Blaze is to iterate over rows:

    M = A*X;
    for( size_t i=0UL; i<M.rows(); ++i ) {
       row( M, i ) += b;

    However, this is neither the fastest possible way nor the most convenient one. Therefore we will consider your idea in one of the next releases. Thanks again,

    Best regards,


  3. Klaus Iglberger


    The feature has been implemented, tested, optimized, and documented as required. It is immediately available via cloning the Blaze repository and will be officially released in Blaze 3.5.

    The expand() function

    Via the expand() function it is possible to convert a dense or sparse vector into a matrix. A column vector is expanded into a column-major matrix, a row vector is expanded into a row-major matrix. As demonstrated by the following examples, expand() can be used with both runtime and compile time parameters:

    blaze::DynamicVector<int,columnVector> a{ 1, 2, 3 };
    blaze::CompressedVector<int,rowVector> b{ 1, 0, 3, 0, 5 };
    // Expand the dense column vector ( 1 2 3 ) into a dense 3x5 column-major matrix
    //   ( 1 1 1 1 1 )
    //   ( 2 2 2 2 2 )
    //   ( 3 3 3 3 3 )
    expand( a, 5 );  // Runtime parameter
    expand<5>( a );  // Compile time parameter
    // Expand the sparse row vector ( 1 0 3 0 5 ) into a sparse 3x5 row-major matrix
    //   ( 1 0 3 0 5 )
    //   ( 1 0 3 0 5 )
    //   ( 1 0 3 0 5 )
    expand( b, 3 );  // Runtime parameter
    expand<3>( b );  // Compile time parameter
  4. Ecolss Logan

    Hi @Klaus Iglberger , good to see this issue has been discussed here before.

    I read the doc about vector expansion, but want to double confirm one use-case.

    Say, I have 2 matrices as follows

    DynamicMatrix<float> A(10, 5);
    DynamicMatrix<float> B(10, 1);
    A += B;  // expecting numpy-ish row broadcasting

    I was expecting that B has only 1 column, and expecting it could be broadcasted when added to A.

    But looks like we cannot do it this way, so 2 questions to ask,

    1. Instead, take the column of B and expand it, then add to A, right?
    2. Since A is row-major, and if I take the column of B and expand, then the resulting expanded matrix is actually column-major, adding them would be OK? Still cache-friendly?

  5. Klaus Iglberger

    Hi Ecolss!

    There is no numpy-ish broadcasting in Blaze. In Blaze it is always expected that matrices are properly sized and there is no implicit assumptions that a user wants to do broadcasting in case the sizes don’t match.

    Assuming the two matrices A and B

    blaze::DynamicMatrix<int> A( 10, 5 );
    blaze::DynamicMatrix<int> B( 10, 1 );

    you have three options how to perform this operation.

    Option 1: Repeating a Matrix

    A += repeat<1,10>( B );

    You can repeat a matrix, which results in another matrix. See the wiki about repeating matrices for a more detailed explanation.

    Option 2: Expanding a Row/Column

    A += expand<10UL>( column<0>( B ) );

    The expand() operation promotes a vector to a matrix. See the wiki about vector expansion to get more information.

    You are correct, in this example the matrix resulting from the expand() operation would be considered a column-major matrix. Still, even for large matrices, Blaze performs the addition in a cache-friendly manner by means of blocking. It cannot apply vectorization, though.

    Option 3: Using a row/column selectionS

    A += columns( B, { 0, 0, 0, 0, 0 } );                 // Runtime indices
    A += columns<0,0,0,0,0>( B );                         // Compile time indices
    A += columns( B, []( size_t ){ return 0UL; }, 5UL );  // Index producer

    Via row/column selections you can select arbitrary rows/columns. This also enables you to repeat individual rows/columns or patterns of rows/columns. See the wiki about column selections to get a complete overview.

    I hope this helps,

    Best regards,


  6. Log in to comment