Private Types | List of all members
blaze::ThreadPool< TT, MT, LT, CT > Class Template Reference

Implementation of a thread pool. More...

#include <ThreadPool.h>

Inherits blaze::NonCopyable.

Public Member Functions

Constructor
 ThreadPool (size_t n)
 Constructor for the ThreadPool class. More...
 
Destructor
 ~ThreadPool ()
 Destructor for the ThreadPool class. More...
 
Get functions
bool isEmpty () const
 Returns whether any tasks are scheduled for execution. More...
 
size_t size () const
 Returns the current size of the thread pool. More...
 
size_t active () const
 Returns the number of currently active/busy threads. More...
 
size_t ready () const
 Returns the number of currently ready/inactive threads. More...
 
Scheduling functions
template<typename Callable >
void schedule (Callable func)
 Scheduling the given zero argument function/functor for execution. More...
 
template<typename Callable , typename A1 >
void schedule (Callable func, A1 a1)
 Scheduling the given unary function/functor for execution. More...
 
template<typename Callable , typename A1 , typename A2 >
void schedule (Callable func, A1 a1, A2 a2)
 Scheduling the given binary function/functor for execution. More...
 
template<typename Callable , typename A1 , typename A2 , typename A3 >
void schedule (Callable func, A1 a1, A2 a2, A3 a3)
 Scheduling the given ternary function/functor for execution. More...
 
template<typename Callable , typename A1 , typename A2 , typename A3 , typename A4 >
void schedule (Callable func, A1 a1, A2 a2, A3 a3, A4 a4)
 Scheduling the given four argument function/functor for execution. More...
 
template<typename Callable , typename A1 , typename A2 , typename A3 , typename A4 , typename A5 >
void schedule (Callable func, A1 a1, A2 a2, A3 a3, A4 a4, A5 a5)
 Scheduling the given four argument function/functor for execution. More...
 
Utility functions
void resize (size_t n, bool block=false)
 Changes the total number of threads in the thread pool. More...
 
void wait ()
 Waiting for all scheduled tasks to be completed. More...
 
void clear ()
 Removing all scheduled tasks from the thread pool. More...
 

Private Types

typedef Thread< TT, MT, LT, CT > ManagedThread
 Type of the managed threads.
 
typedef PtrVector< ManagedThreadThreads
 Type of the thread container.
 
typedef threadpool::TaskQueue TaskQueue
 Type of the task queue.
 
typedef MT Mutex
 Type of the mutex.
 
typedef LT Lock
 Type of a locking object.
 
typedef CT Condition
 Condition variable type.
 

Private Member Functions

Thread functions
void createThread ()
 Adding a new thread to the thread pool. More...
 
bool executeTask ()
 Executing a scheduled task. More...
 

Private Attributes

Member variables
volatile size_t total_
 Total number of threads in the thread pool.
 
volatile size_t expected_
 Expected number of threads in the thread pool. More...
 
volatile size_t active_
 Number of currently active/busy threads.
 
Threads threads_
 The threads contained in the thread pool.
 
TaskQueue taskqueue_
 Task queue for the scheduled tasks.
 
Mutex mutex_
 Synchronization mutex.
 
Condition waitForTask_
 Wait condition for idle threads.
 
Condition waitForThread_
 Wait condition for the thread management.
 

Detailed Description

template<typename TT, typename MT, typename LT, typename CT>
class blaze::ThreadPool< TT, MT, LT, CT >

Implementation of a thread pool.

General

The ThreadPool class template represents a thread pool according to the thread pool pattern (see for example http://en.wikipedia.org/wiki/Thread_pool_pattern). It manages a certain number of threads in order to process a larger number of independent tasks.

threadpool.png

The primary purpose of a thread pool is the reuse of system resources: instead of creating a single thread for every individual task, threads are reused to handle several tasks. This increases the performance in comparison to different threading strategies, as illustrated in the graph below. The first bar indicates the sequential performance of 1000 matrix-matrix multiplications of arbitrarily sized square matrices. The second bar shows the performance of the same work performed by 1000 distinct threads (i.e. one thread for each matrix-matrix multiplication) on a quad-core system. In this case, all cores of the system can be used, but the additional overhead of creating and managing new threads prevents the expected performance increase by a factor of four. The third bar illustrates the performance of four threads distributing the work between them (i.e. 250 matrix-matrix multiplications per thread), again using the same quad-core system. This approach nearly achieves four times the performance of the sequential execution. The fourth bar represents the performance of the ThreadPool class using fourth threads for the execution of the 1000 individual multiplications.

threadpool2.png

Additionally, the thread pool approach simplifies load balancing and increases the stability of the system.

Class Definition

The implementation of the ThreadPool class template is based on the implementation of standard thread functionality as provided by the C++11 standard or the Boost library. Via the four template parameters it is possible to configure a ThreadPool instance as either a C++11 thread pool or as Boost thread pool:

template< typename TT, typename MT, typename LT, typename CT >
class ThreadPool;

The following example demonstrates how to configure the ThreadPool class template as either C++11 standard thread pool or as Boost thread pool:

typedef blaze::ThreadPool< boost::thread
, boost::mutex
, boost::unique_lock<boost::mutex>
, boost::condition_variable > BoostThreadPool;
typedef blaze::ThreadPool< std::thread
, std::mutex
, std::unique_lock<std::mutex>
, std::condition_variable > StdThreadPool;

For more information about the standard thread functionality, see [1] or [2] or the current documentation at the Boost homepage: www.boost.org.

Using the ThreadPool class

The following example demonstrates the use of the ThreadPool class. In contrast to the setup of individual threads (see the Thread class description for more details), it is not necessary to create and manage individual threads, but only to schedules tasks for the accordingly sized thread pool.

// Definition of a function with no arguments that returns void
void function0() { ... }
// Definition of a functor (function object) taking two arguments and returning void
struct Functor2
{
void operator()( int a, int b ) { ... }
};
int main()
{
// Creating a thread pool with initially two working threads
StdThreadPool threadpool( 2 );
// Scheduling two concurrent tasks
threadpool.schedule( function0 );
threadpool.schedule( Functor2(), 4, 6 );
// Waiting for the thread pool to complete both tasks
threadpool.wait();
// Resizing the thread pool to four working threads
threadpool.resize( 4 );
// Scheduling other concurrent tasks
...
threadpool.schedule( function0 );
...
// At the end of the thread pool scope, all tasks remaining in the task queue are removed
// and all currently running tasks are completed. Additionally, all acquired resources are
// safely released.
}

Note that the ThreadPool class template schedule() function allows for up to five arguments for the given functions/functors.

Throwing exceptions in a thread parallel environment

It can happen that during the execution of a given task a thread encounters an erroneous situation and has to throw an exception. However, exceptions thrown in the usual way cannot be caught by a try-catch-block in the main thread of execution:

// Definition of a function throwing a std::runtime_error during its execution
void task()
{
...
throw std::runtime_error( ... );
...
}
// Creating a thread pool executing the throwing function. Although the setup, the scheduling
// of the task, the wait() function and the destruction of the thread pool are encapsuled
// inside a try-catch-block, the exception cannot be caught and results in an abortion of the
// program.
try {
StdThreadpool threadpool( 2 );
thread.schedule( task );
threadpool.wait();
}
catch( ... )
{
...
}

For a detailed explanation how to portably transport exceptions between threads, see [1] or [2]. In case of the Boost library, the according Boost functionality as demonstrated in the following example has to be used. Note that any function/functor scheduled for execution is responsible to handle exceptions in this way!

#include <boost/bind.hpp>
#include <boost/exception_ptr.hpp>
// Definition of a function that happens to throw an exception. In order to throw the
// exception, boost::enable_current_exception() is used in combination with throw.
void throwException()
{
...
throw boost::enable_current_exception( std::runtime_error( ... ) );
...
}
// Definition of a thread function. The try-catch-block catches the exception and uses the
// boost::current_exception() function to get a boost::exception_ptr object.
void task( boost::exception_ptr& error )
{
try {
throwException();
error = boost::exception_ptr();
}
catch( ... ) {
error = boost::current_exception();
}
}
// The function that start a thread of execution can pass along a boost::exception_ptr object
// that is set in case of an exception. Note that boost::current_exception() captures the
// original type of the exception object. The exception can be thrown again using the
// boost::rethrow_exception() function.
void work()
{
boost::exception_ptr error;
StdThreadPool threadpool( 2 );
threadpool.schedule( boost::bind( task, boost::ref(error) ) );
threadpool.wait();
if( error ) {
std::cerr << " Exception during thread execution!\n\n";
boost::rethrow_exception( error );
}
}

Known issues

There is a known issue in Visual Studio 2012 and 2013 that may cause C++11 threads to hang if their destructor is executed after the main() function:

http://connect.microsoft.com/VisualStudio/feedback/details/747145

In order to circumvent this problem, for Visual Studio compilers only, it is possible to explicitly resize a ThreadPool instance to 0 threads and to block until all threads have been destroyed:

int main()
{
static StdThreadPool threadpool( 4 );
// ... Using the thread pool
threadpool( 0, true );
}

Note that this should ONLY be used before the end of the main() function and ONLY if the threadpool will not be used anymore.

References

[1] A. Williams: C++ Concurrency in Action, Manning, 2012, ISBN: 978-1933988771
[2] B. Stroustrup: The C++ Programming Language, Addison-Wesley, 2013, ISBN: 978-0321563842

Constructor & Destructor Documentation

template<typename TT , typename MT , typename LT , typename CT >
blaze::ThreadPool< TT, MT, LT, CT >::ThreadPool ( size_t  n)
explicit

Constructor for the ThreadPool class.

Parameters
nInitial number of threads $[1..\infty)$.

This constructor creates a thread pool with initially n new threads. All threads are initially idle until a task is scheduled.

template<typename TT , typename MT , typename LT , typename CT >
blaze::ThreadPool< TT, MT, LT, CT >::~ThreadPool ( )

Destructor for the ThreadPool class.

The destructor clears all remaining tasks from the task queue and waits for the currently active threads to complete their tasks.

Member Function Documentation

template<typename TT , typename MT , typename LT , typename CT >
size_t blaze::ThreadPool< TT, MT, LT, CT >::active ( ) const
inline

Returns the number of currently active/busy threads.

Returns
The number of currently active threads.
template<typename TT , typename MT , typename LT , typename CT >
void blaze::ThreadPool< TT, MT, LT, CT >::clear ( )

Removing all scheduled tasks from the thread pool.

Returns
void

This function removes all currently scheduled tasks from the thread pool. The total number of threads remains unchanged and all active threads continue completing their tasks.

template<typename TT , typename MT , typename LT , typename CT >
void blaze::ThreadPool< TT, MT, LT, CT >::createThread ( )
private

Adding a new thread to the thread pool.

Returns
void
template<typename TT , typename MT , typename LT , typename CT >
bool blaze::ThreadPool< TT, MT, LT, CT >::executeTask ( )
private

Executing a scheduled task.

Returns
true in case a task was successfully finished, false if not.

This function is repeatedly called by every thread to execute one of the scheduled tasks. In case there is no task available, the thread blocks and waits for a new task to be scheduled.

template<typename TT , typename MT , typename LT , typename CT >
bool blaze::ThreadPool< TT, MT, LT, CT >::isEmpty ( ) const
inline

Returns whether any tasks are scheduled for execution.

Returns
true in case task are scheduled, false otherwise.
template<typename TT , typename MT , typename LT , typename CT >
size_t blaze::ThreadPool< TT, MT, LT, CT >::ready ( ) const
inline

Returns the number of currently ready/inactive threads.

Returns
The number of currently ready threads.
template<typename TT , typename MT , typename LT , typename CT >
void blaze::ThreadPool< TT, MT, LT, CT >::resize ( size_t  n,
bool  block = false 
)

Changes the total number of threads in the thread pool.

Parameters
nThe new number of threads $[1..\infty)$.
blocktrue if the function shall block, false if not.
Returns
void
Exceptions
std::invalid_argumentInvalid number of threads.

This function changes the size of the thread pool, i.e. changes the total number of threads contained in the pool. If n is smaller than the current size of the thread pool, the according number of threads is removed from the pool, otherwise new threads are added to the pool. Via the block flag it is possible to block the function until the desired number of threads is available.

Note that there is a known issue in Visual Studio 2012 and 2013 that may cause C++11 threads to hang if their destructor is executed after the main() function:

http://connect.microsoft.com/VisualStudio/feedback/details/747145

In order to circumvent this problem, for Visual Studio compilers only, it is possible to explicitly resize a ThreadPool instance to 0 threads and to block until all threads have been destroyed:

int main()
{
static ThreadPool< std::thread
, std::mutex
, std::unique_lock< std::mutex >
, std::condition_variable > threadpool( 4 );
// ... Using the thread pool
threadpool( 0, true );
}

Note that this should ONLY be used before the end of the main() function and ONLY if the threadpool will not be used anymore.

template<typename TT , typename MT , typename LT , typename CT >
template<typename Callable >
void blaze::ThreadPool< TT, MT, LT, CT >::schedule ( Callable  func)

Scheduling the given zero argument function/functor for execution.

Parameters
funcThe given function/functor.
Returns
void

This function schedules the given function/functor for execution. The given function/functor must be copyable, must be callable without arguments and must return void.

template<typename TT , typename MT , typename LT , typename CT >
template<typename Callable , typename A1 >
void blaze::ThreadPool< TT, MT, LT, CT >::schedule ( Callable  func,
A1  a1 
)

Scheduling the given unary function/functor for execution.

Parameters
funcThe given function/functor.
a1The first argument.
Returns
void

This function schedules the given function/functor for execution. The given function/functor must be copyable, must be callable with one argument and must return void.

template<typename TT , typename MT , typename LT , typename CT >
template<typename Callable , typename A1 , typename A2 >
void blaze::ThreadPool< TT, MT, LT, CT >::schedule ( Callable  func,
A1  a1,
A2  a2 
)

Scheduling the given binary function/functor for execution.

Parameters
funcThe given function/functor.
a1The first argument.
a2The second argument.
Returns
void

This function schedules the given function/functor for execution. The given function/functor must be copyable, must be callable with two arguments and must return void.

template<typename TT , typename MT , typename LT , typename CT >
template<typename Callable , typename A1 , typename A2 , typename A3 >
void blaze::ThreadPool< TT, MT, LT, CT >::schedule ( Callable  func,
A1  a1,
A2  a2,
A3  a3 
)

Scheduling the given ternary function/functor for execution.

Parameters
funcThe given function/functor.
a1The first argument.
a2The second argument.
a3The third argument.
Returns
void

This function schedules the given function/functor for execution. The given function/functor must be copyable, must be callable with three arguments and must return void.

template<typename TT , typename MT , typename LT , typename CT >
template<typename Callable , typename A1 , typename A2 , typename A3 , typename A4 >
void blaze::ThreadPool< TT, MT, LT, CT >::schedule ( Callable  func,
A1  a1,
A2  a2,
A3  a3,
A4  a4 
)

Scheduling the given four argument function/functor for execution.

Parameters
funcThe given function/functor.
a1The first argument.
a2The second argument.
a3The third argument.
a4The fourth argument.
Returns
void

This function schedules the given function/functor for execution. The given function/functor must be copyable, must be callable with four arguments and must return void.

template<typename TT , typename MT , typename LT , typename CT >
template<typename Callable , typename A1 , typename A2 , typename A3 , typename A4 , typename A5 >
void blaze::ThreadPool< TT, MT, LT, CT >::schedule ( Callable  func,
A1  a1,
A2  a2,
A3  a3,
A4  a4,
A5  a5 
)

Scheduling the given four argument function/functor for execution.

Parameters
funcThe given function/functor.
a1The first argument.
a2The second argument.
a3The third argument.
a4The fourth argument.
a5The fifth argument.
Returns
void

This function schedules the given function/functor for execution. The given function/functor must be copyable, must be callable with five arguments and must return void.

template<typename TT , typename MT , typename LT , typename CT >
size_t blaze::ThreadPool< TT, MT, LT, CT >::size ( ) const
inline

Returns the current size of the thread pool.

Returns
The total number of threads in the thread pool.
template<typename TT , typename MT , typename LT , typename CT >
void blaze::ThreadPool< TT, MT, LT, CT >::wait ( )

Waiting for all scheduled tasks to be completed.

Returns
void

This function blocks until all scheduled tasks have been completed.

Member Data Documentation

template<typename TT , typename MT , typename LT , typename CT >
volatile size_t blaze::ThreadPool< TT, MT, LT, CT >::expected_
private

Expected number of threads in the thread pool.

This number may differ from the total number of threads during a resize of the thread pool.


The documentation for this class was generated from the following files: