ThreadSanitizer reported several data races
Issue #467
new
When building x265 with ThreadSanitizer, it reports multiple data races with the following command:
./x265 --input x265-report/test.y4m --pools 2 -F 2 --lookahead-threads 0 --output /tmp/test.265
The messages are like:
==================
WARNING: ThreadSanitizer: data race (pid=26420)
Read of size 8 at 0x7b1000000048 by main thread:
#0 x265::ThreadPool::tryBondPeers(int, unsigned long, x265::BondedTaskGroup&) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threadpool.cpp (libx265.so.169+0x5b0c87)
#1 x265::BondedTaskGroup::tryBondPeers(x265::ThreadPool&, int) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threadpool.h:152:26 (libx265.so.169+0x679b4)
#2 x265::Lookahead::slicetypeDecide() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/slicetype.cpp:1222 (libx265.so.169+0x679b4)
#3 x265::Lookahead::findJob(int) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/slicetype.cpp:989:5 (libx265.so.169+0x67653)
#4 x265::Lookahead::getDecidedPicture() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/slicetype.cpp:1019:9 (libx265.so.169+0x68fe7)
#5 x265::Encoder::encode(x265_picture const*, x265_picture*) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/encoder.cpp:1451:37 (libx265.so.169+0xc7ed7)
#6 x265_encoder_encode /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/api.cpp:280:31 (libx265.so.169+0xe27ae)
#7 main /home/ubuntu/work/x265/x265-tsan/source/BUILD/../x265.cpp:955:26 (x265+0x4c8209)
Previous atomic write of size 8 at 0x7b1000000048 by thread T2:
#0 __tsan_atomic64_fetch_or <null> (x265+0x4725a0)
#1 x265::WorkerThread::threadMain() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threadpool.cpp:131:5 (libx265.so.169+0x5b0475)
#2 x265::stackAlignMain(x265::Thread*) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threading.cpp:91:15 (libx265.so.169+0x5b0299)
#3 x265::ThreadShim(void*) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threading.cpp:131 (libx265.so.169+0x5b0299)
Location is heap block of size 56 at 0x7b1000000040 allocated by main thread:
#0 operator new[](unsigned long) <null> (x265+0x4bc236)
#1 x265::ThreadPool::allocThreadPools(x265_param*, int&, bool) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threadpool.cpp:419:25 (libx265.so.169+0x5b1123)
#2 x265::Encoder::create() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/encoder.cpp:173:24 (libx265.so.169+0xbfa52)
#3 x265_encoder_open_169 /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/api.cpp:129:14 (libx265.so.169+0xe1793)
#4 main /home/ubuntu/work/x265/x265-tsan/source/BUILD/../x265.cpp:854:29 (x265+0x4c7854)
Thread T2 (tid=26631, running) created by main thread at:
#0 pthread_create <null> (x265+0x42d396)
#1 x265::Thread::start() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threading.cpp:138:9 (libx265.so.169+0x5b023e)
#2 x265::ThreadPool::start() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threadpool.cpp:523:27 (libx265.so.169+0x5b1873)
#3 x265::Encoder::create() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/encoder.cpp:231:29 (libx265.so.169+0xbff02)
#4 x265_encoder_open_169 /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/api.cpp:129:14 (libx265.so.169+0xe1793)
#5 main /home/ubuntu/work/x265/x265-tsan/source/BUILD/../x265.cpp:854:29 (x265+0x4c7854)
SUMMARY: ThreadSanitizer: data race /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threadpool.cpp in x265::ThreadPool::tryBondPeers(int, unsigned long, x265::BondedTaskGroup&)
==================
==================
WARNING: ThreadSanitizer: data race (pid=26420)
Write of size 1 at 0x7b9400000020 by thread T3:
#0 x265::WaveFront::findJob(int) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/wavefront.cpp (libx265.so.169+0x5b21d4)
#1 x265::WorkerThread::threadMain() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threadpool.cpp:146:31 (libx265.so.169+0x5b0614)
#2 x265::stackAlignMain(x265::Thread*) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threading.cpp:91:15 (libx265.so.169+0x5b0299)
#3 x265::ThreadShim(void*) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threading.cpp:131 (libx265.so.169+0x5b0299)
...
...
==================
WARNING: ThreadSanitizer: data race (pid=26420)
Read of size 1 at 0x7f0eef382800 by thread T3:
#0 x265::Analysis::topSkipMinDepth(x265::CUData const&, x265::CUGeom const&) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/analysis.cpp:3417:22 (libx265.so.169+0x30bd0)
#1 x265::Analysis::compressInterCU_rd0_4(x265::CUData const&, x265::CUGeom const&, int) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/analysis.cpp:1175:29 (libx265.so.169+0x1ea0a)
#2 x265::Analysis::compressCTU(x265::CUData&, x265::Frame&, x265::CUGeom const&, x265::Entropy const&) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/analysis.cpp:302:13 (libx265.so.169+0x1ba8e)
#3 x265::FrameEncoder::processRowEncoder(int, x265::ThreadLocalData&) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/frameencoder.cpp:1482:35 (libx265.so.169+0x7ff6c)
#4 x265::FrameEncoder::processRow(int, int) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/frameencoder.cpp:1286:9 (libx265.so.169+0x7ea76)
#5 x265::WaveFront::findJob(int) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/wavefront.cpp:106:17 (libx265.so.169+0x5b21c3)
#6 x265::WorkerThread::threadMain() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threadpool.cpp:146:31 (libx265.so.169+0x5b0614)
#7 x265::stackAlignMain(x265::Thread*) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threading.cpp:91:15 (libx265.so.169+0x5b0299)
#8 x265::ThreadShim(void*) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threading.cpp:131 (libx265.so.169+0x5b0299)
Previous write of size 4 at 0x7f0eef382800 by thread T4:
#0 bcast4(unsigned char*, unsigned char) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/cudata.cpp:40:70 (libx265.so.169+0x5cbd18)
#1 x265::CUData::setQPSubParts(signed char, unsigned int, unsigned int) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/cudata.h:255:97 (libx265.so.169+0x9e1fb)
#2 x265::Entropy::finishCU(x265::CUData const&, unsigned int, unsigned int, bool) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/entropy.cpp:899 (libx265.so.169+0x9e1fb)
#3 x265::Entropy::encodeCU(x265::CUData const&, x265::CUGeom const&, unsigned int, unsigned int, bool&) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/entropy.cpp (libx265.so.169+0x9debc)
#4 x265::Entropy::encodeCU(x265::CUData const&, x265::CUGeom const&, unsigned int, unsigned int, bool&) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/entropy.cpp:794:13 (libx265.so.169+0x9db87)
#5 x265::Entropy::encodeCU(x265::CUData const&, x265::CUGeom const&, unsigned int, unsigned int, bool&) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/entropy.cpp:794:13 (libx265.so.169+0x9db87)
#6 x265::Entropy::encodeCU(x265::CUData const&, x265::CUGeom const&, unsigned int, unsigned int, bool&) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/entropy.cpp:778:17 (libx265.so.169+0x9d99c)
#7 x265::Entropy::encodeCTU(x265::CUData const&, x265::CUGeom const&) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/entropy.cpp:758:5 (libx265.so.169+0x9d7f3)
#8 x265::FrameEncoder::encodeSlice(unsigned int) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/frameencoder.cpp:1259:24 (libx265.so.169+0x7e45b)
#9 x265::FrameEncoder::compressFrame() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/frameencoder.cpp:970:9 (libx265.so.169+0x7b03d)
#10 x265::FrameEncoder::threadMain() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/frameencoder.cpp:343:9 (libx265.so.169+0x7627c)
#11 non-virtual thunk to x265::FrameEncoder::threadMain() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/frameencoder.cpp (libx265.so.169+0x7c454)
#12 x265::stackAlignMain(x265::Thread*) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threading.cpp:91:15 (libx265.so.169+0x5b0299)
#13 x265::ThreadShim(void*) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threading.cpp:131 (libx265.so.169+0x5b0299)
Location is heap block of size 368640 at 0x7f0eef336000 allocated by main thread:
#0 posix_memalign <null> (x265+0x43de52)
#1 x265::x265_malloc(unsigned long) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/common.cpp:81:9 (libx265.so.169+0x5b9938)
#2 x265::CUDataMemPool::create(unsigned int, unsigned int, unsigned int, x265_param const&) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/cudata.h:378:9 (libx265.so.169+0x34b06)
#3 x265::FrameData::create(x265_param const&, x265::SPS const&, int) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/framedata.cpp:43:36 (libx265.so.169+0x5ca759)
#4 x265::Frame::allocEncodeData(x265_param*, x265::SPS const&) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/frame.cpp:125:26 (libx265.so.169+0x5c9b84)
#5 x265::Encoder::encode(x265_picture const*, x265_picture*) /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/encoder.cpp:1505:27 (libx265.so.169+0xc854f)
#6 x265_encoder_encode /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/api.cpp:280:31 (libx265.so.169+0xe27ae)
#7 main /home/ubuntu/work/x265/x265-tsan/source/BUILD/../x265.cpp:987:26 (x265+0x4c874c)
Thread T3 (tid=26648, running) created by main thread at:
#0 pthread_create <null> (x265+0x42d396)
#1 x265::Thread::start() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threading.cpp:138:9 (libx265.so.169+0x5b023e)
#2 x265::ThreadPool::start() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threadpool.cpp:523:27 (libx265.so.169+0x5b1873)
#3 x265::Encoder::create() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/encoder.cpp:231:29 (libx265.so.169+0xbff02)
#4 x265_encoder_open_169 /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/api.cpp:129:14 (libx265.so.169+0xe1793)
#5 main /home/ubuntu/work/x265/x265-tsan/source/BUILD/../x265.cpp:854:29 (x265+0x4c7854)
Thread T4 (tid=26709, running) created by main thread at:
#0 pthread_create <null> (x265+0x42d396)
#1 x265::Thread::start() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../common/threading.cpp:138:9 (libx265.so.169+0x5b023e)
#2 x265::Encoder::create() /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/encoder.cpp:365:28 (libx265.so.169+0xc0c20)
#3 x265_encoder_open_169 /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/api.cpp:129:14 (libx265.so.169+0xe1793)
#4 main /home/ubuntu/work/x265/x265-tsan/source/BUILD/../x265.cpp:854:29 (x265+0x4c7854)
SUMMARY: ThreadSanitizer: data race /home/ubuntu/work/x265/x265-tsan/source/BUILD/../encoder/analysis.cpp:3417:22 in x265::Analysis::topSkipMinDepth(x265::CUData const&, x265::CUGeom const&)
$ ./x265 --version
x265 [info]: HEVC encoder version 3.0+13-g348266a7d
x265 [info]: build info [Linux][clang 4.0.1][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
The relevant input file and the tsan output are attached.