Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data race on writing_rollback variable. #447

Open
wants to merge 30 commits into
base: master
Choose a base branch
from
Open

Data race on writing_rollback variable. #447

wants to merge 30 commits into from

Conversation

prohaska7
Copy link
Contributor

When running the FT tests, some DRD runs report a data race on the 'writing_rollback' variable when it is being incremented and decremented. This can cause the value of the variable to be indeterminate. See commit 95515e8 for details.
Tools:
Ubuntu 19.10, Clang 7,8,9, Gcc 7,8,9, Cmake 3.14

Copyright (c) 2020, Rik Prohaska
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

@prohaska7
glibc 2.30 defines the gettid system call, which conflicts with the
65753a9 
tokuft test version.  rename tokuft test version to avoid conflict.
@prohaska7
Remove ctest timeout property so that we can use the ctest --timeout …
1bbdec1 
…parameter instead.
@prohaska7
Fix ydb tests that fail due to loose file permissions. Note that file…
9be7dda 
… permissions were changed in commit 5aca29f.
@prohaska7
Remove obsolete debug code that was left in the tests by accident.
9af3125
@prohaska7
Add locktree and ydb tests cases that expose the lock request retry
64b819c 
bug when the lock memory use is over the limit.  The ydb tests
show that DB->put normally returns TOKUDB_OUT_OF_LOCKS when the
lock memory use is over the limit.
@prohaska7
When a lock is released, pending lock requests are retried. If the
ca0df3a 
retry occurs and the lock memory use is over the limit, then pending
lock requests should be completed with TOKUDB_OUT_OF_LOCKS result.
The current code does not handle this case and crashes.  This commit
fixes this case and is sufficient to fix PS-4328.
@prohaska7
Speed up helgrind_test_partitioned_counter test by running helgrind
ea536c2 
and drd scheduler with fairness (--fair-sched=try).  This should
reduce execution time from minutes to seconds.
@prohaska7
Fix a test that crashed in put_callback due to a null deref. The test
fddf161 
should intentially abort instead.
@prohaska7
Ignore mallopt error return in memory_init when running the address s…
a3c530c 
…anitizer. This allows the tests to run.
@prohaska7
Suppress pthread_mutex_destroy race with pthread_mutex_unlock again.
54c2ac8 
It causes helgrind to report false positive data races on tests,
specifically the helgrind_helgrind tests.

4101: ==17113== Possible data race during read of size 1 at 0x52FA980 by thread #1
4101: ==17113== Locks held: 1, at address 0x4C126C0
4101: ==17113==    at 0x483E99C: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
4101: ==17113==    by 0x483EC4A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
4101: ==17113==    by 0x4A95654: toku_mutex_destroy(toku_mutex_t*) (toku_pthread.h:427)
4101: ==17113==    by 0x4B6CF5C: bjm_destroy(background_job_manager_struct*) (background_job_manager.cc:67)
4101: ==17113==    by 0x4ABE119: write_dirty_pairs_for_close(cachetable*, cachefile*) (cachetable.cc:2406)
4101: ==17113==    by 0x4ABE308: cachetable_flush_cachefile(cachetable*, cachefile*, bool) (cachetable.cc:2493)
4101: ==17113==    by 0x4AB99BE: toku_cachefile_close(cachefile**, bool, __toku_lsn) (cachetable.cc:529)
4101: ==17113==    by 0x4ACD7C2: toku_ft_evict_from_memory(ft*, bool, __toku_lsn) (ft.cc:499)
4101: ==17113==    by 0x4ACEADD: toku_ft_remove_reference(ft*, bool, __toku_lsn, void (*)(ft*, void*), void*) (ft.cc:949)
4101: ==17113==    by 0x4AE9B21: ft_handle_close(ft_handle*, bool, __toku_lsn) (ft-ops.cc:3272)
4101: ==17113==    by 0x4AE9B54: toku_ft_handle_close(ft_handle*) (ft-ops.cc:3280)
4101: ==17113==    by 0x4A8CD68: toku_db_close(__toku_db*) (ydb_db.cc:212)
4101: ==17113==
4101: ==17113== This conflicts with a previous write of size 4 by thread #10
4101: ==17113== Locks held: none
4101: ==17113==    at 0x4C557A1: __pthread_mutex_unlock_usercnt (pthread_mutex_unlock.c:51)
4101: ==17113==    by 0x4C557A1: pthread_mutex_unlock (pthread_mutex_unlock.c:357)
4101: ==17113==    by 0x483F458: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
4101: ==17113==    by 0x4A84508: toku_mutex_unlock(toku_mutex_t*) (toku_pthread.h:256)
4101: ==17113==    by 0x4B6D0EC: bjm_remove_background_job(background_job_manager_struct*) (background_job_manager.cc:98)
4101: ==17113==    by 0x4ABDCDD: cachetable_flush_pair_for_close(void*) (cachetable.cc:2327)
4101: ==17113==    by 0x4B7D785: work_on_kibbutz(void*) (kibbutz.cc:154)
4101: ==17113==    by 0x4841B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
4101: ==17113==    by 0x4C51668: start_thread (pthread_create.c:479)
@prohaska7
Suppress memcheck uninitialized data jump in lz_encoder_prepare. See h…
e0dcade 
…ttps://prohaska7.blogspot.com/2015/11/uninitialized-data-problem-in-lzma.html for details.
@prohaska7
Some ydb tests leak memory for pfs keys. Fix the ydb library init and…
bfd3a72 
… destroy functions.
@prohaska7
Suppress memory leak in libjson on ubuntu 19.10 when running cmake.
875a173
@prohaska7
Run try-leak-reachable with RelWithDebInfo build type by avoiding gcc…
0b201fd 
… optimization.
@prohaska7
glibc 2.30 defines the gettid system call, which conflicts with the
d4f11d2 
tokuft test version.  rename tokuft test version to avoid conflict.
@prohaska7
Remove ctest timeout property so that we can use the ctest --timeout …
e1ed3f1 
…parameter instead.
@prohaska7
Fix ydb tests that fail due to loose file permissions. Note that file…
5a5c253 
… permissions were changed in commit 5aca29f.
@prohaska7
Remove obsolete debug code that was left in the tests by accident.
d383221
@prohaska7
Add locktree and ydb tests cases that expose the lock request retry
6a38ee4 
bug when the lock memory use is over the limit.  The ydb tests
show that DB->put normally returns TOKUDB_OUT_OF_LOCKS when the
lock memory use is over the limit.
@prohaska7
When a lock is released, pending lock requests are retried. If the
1fede89 
retry occurs and the lock memory use is over the limit, then pending
lock requests should be completed with TOKUDB_OUT_OF_LOCKS result.
The current code does not handle this case and crashes.  This commit
fixes this case and is sufficient to fix PS-4328.
@prohaska7
Speed up helgrind_test_partitioned_counter test by running helgrind
3c3f6a0 
and drd scheduler with fairness (--fair-sched=try).  This should
reduce execution time from minutes to seconds.
@prohaska7
Fix a test that crashed in put_callback due to a null deref. The test
853be0b 
should intentially abort instead.
@prohaska7
Ignore mallopt error return in memory_init when running the address s…
477b516 
…anitizer. This allows the tests to run.
@prohaska7
Suppress pthread_mutex_destroy race with pthread_mutex_unlock again.
5bee63f 
It causes helgrind to report false positive data races on tests,
specifically the helgrind_helgrind tests.

4101: ==17113== Possible data race during read of size 1 at 0x52FA980 by thread #1
4101: ==17113== Locks held: 1, at address 0x4C126C0
4101: ==17113==    at 0x483E99C: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
4101: ==17113==    by 0x483EC4A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
4101: ==17113==    by 0x4A95654: toku_mutex_destroy(toku_mutex_t*) (toku_pthread.h:427)
4101: ==17113==    by 0x4B6CF5C: bjm_destroy(background_job_manager_struct*) (background_job_manager.cc:67)
4101: ==17113==    by 0x4ABE119: write_dirty_pairs_for_close(cachetable*, cachefile*) (cachetable.cc:2406)
4101: ==17113==    by 0x4ABE308: cachetable_flush_cachefile(cachetable*, cachefile*, bool) (cachetable.cc:2493)
4101: ==17113==    by 0x4AB99BE: toku_cachefile_close(cachefile**, bool, __toku_lsn) (cachetable.cc:529)
4101: ==17113==    by 0x4ACD7C2: toku_ft_evict_from_memory(ft*, bool, __toku_lsn) (ft.cc:499)
4101: ==17113==    by 0x4ACEADD: toku_ft_remove_reference(ft*, bool, __toku_lsn, void (*)(ft*, void*), void*) (ft.cc:949)
4101: ==17113==    by 0x4AE9B21: ft_handle_close(ft_handle*, bool, __toku_lsn) (ft-ops.cc:3272)
4101: ==17113==    by 0x4AE9B54: toku_ft_handle_close(ft_handle*) (ft-ops.cc:3280)
4101: ==17113==    by 0x4A8CD68: toku_db_close(__toku_db*) (ydb_db.cc:212)
4101: ==17113==
4101: ==17113== This conflicts with a previous write of size 4 by thread #10
4101: ==17113== Locks held: none
4101: ==17113==    at 0x4C557A1: __pthread_mutex_unlock_usercnt (pthread_mutex_unlock.c:51)
4101: ==17113==    by 0x4C557A1: pthread_mutex_unlock (pthread_mutex_unlock.c:357)
4101: ==17113==    by 0x483F458: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
4101: ==17113==    by 0x4A84508: toku_mutex_unlock(toku_mutex_t*) (toku_pthread.h:256)
4101: ==17113==    by 0x4B6D0EC: bjm_remove_background_job(background_job_manager_struct*) (background_job_manager.cc:98)
4101: ==17113==    by 0x4ABDCDD: cachetable_flush_pair_for_close(void*) (cachetable.cc:2327)
4101: ==17113==    by 0x4B7D785: work_on_kibbutz(void*) (kibbutz.cc:154)
4101: ==17113==    by 0x4841B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
4101: ==17113==    by 0x4C51668: start_thread (pthread_create.c:479)
@prohaska7
Suppress memcheck uninitialized data jump in lz_encoder_prepare. See h…
6dd081f 
…ttps://prohaska7.blogspot.com/2015/11/uninitialized-data-problem-in-lzma.html for details.
@prohaska7
Some ydb tests leak memory for pfs keys. Fix the ydb library init and…
603a917 
… destroy functions.
@prohaska7
Suppress memory leak in libjson on ubuntu 19.10 when running cmake.
4cc6aaf
@prohaska7
Run try-leak-reachable with RelWithDebInfo build type by avoiding gcc…
d442692 
… optimization.
@prohaska7
Merge branch 'apr2020' of github.com:prohaska7/tokuft into apr2020
17f40f8
@prohaska7
Fix data race on the writing_rollback variable detected by the DRD an…
95515e8 
…d the thread sanitizer.

Multiple threads are allowed to increment and decrement this variable without
any atomicity.  The solution is to change the type of the writing_rollback
variable from an 'int' to an 'atomic_int'.

One may need to run the following test cases multiple times to stumble upon
the data race.

ctest -R ydb/drd_test_groupcommit_count --verbose

4104: ==21516== Thread 43:
4104: ==21516== Conflicting load by thread 43 at 0x04c22440 size 4
4104: ==21516==    at 0x4B4EB26: rollback_log_create(tokutxn*, blocknum_s, rollback_log_node**) (rollback.cc:160)
4104: ==21516==    by 0x4B4F4D9: toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) (rollback.cc:327)
4104: ==21516==    by 0x4B6F45F: toku_logger_save_rollback_cmdinsert(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2007)
4104: ==21516==    by 0x4AF3117: ft_txn_log_insert(ft*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, ft_msg_type) (ft-ops.cc:2406)
4104: ==21516==    by 0x4AF32B0: toku_ft_maybe_insert(ft_handle*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, __toku_lsn, bool, ft_msg_type) (ft-ops.cc:2423)
4104: ==21516==    by 0x4AA2AA9: db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, int, bool) (ydb_write.cc:221)
4104: ==21516==    by 0x4AA2CB3: toku_db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, unsigned int, bool) (ydb_write.cc:248)
4104: ==21516==    by 0x4AA68FC: autotxn_db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, unsigned int) (ydb_write.cc:1078)
4104: ==21516==    by 0x10B099: start_a_thread(void*) (test_groupcommit_count.cc:64)
4104: ==21516==    by 0x48414BA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4104: ==21516==    by 0x4C5D668: start_thread (pthread_create.c:479)
4104: ==21516==    by 0x4D99322: clone (clone.S:95)
4104: ==21516== Allocation context: BSS section of /home/rfp/projects/tokuft-Debug-gcc-7/src/libtokufractaltree.so
4104: ==21516== Other segment start (thread 42)
4104: ==21516==    at 0x484587C: pthread_mutex_unlock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4104: ==21516==    by 0x4B8BE37: pc_unlock() (partitioned_counter.cc:151)
4104: ==21516==    by 0x4B8C583: get_or_alloc_thread_local_counter(partitioned_counter*) (partitioned_counter.cc:366)
4104: ==21516==    by 0x4B8C5C0: increment_partitioned_counter (partitioned_counter.cc:375)
4104: ==21516==    by 0x4ACD9B5: evictor::add_pair_attr(pair_attr_s) (cachetable.cc:3769)
4104: ==21516==    by 0x4AC684E: cachetable_insert_pair_at(cachetable*, ctpair*, pair_attr_s) (cachetable.cc:871)
4104: ==21516==    by 0x4AC689E: cachetable_put_internal(cachefile*, ctpair*, void*, pair_attr_s, void (*)(blocknum_s, void*, ctpair*)) (cachetable.cc:897)
4104: ==21516==    by 0x4AC7410: toku_cachetable_put(cachefile*, blocknum_s, unsigned int, void*, pair_attr_s, CACHETABLE_WRITE_CALLBACK, void (*)(blocknum_s, void*, ctpair*)) (cachetable.cc:1207)
4104: ==21516==    by 0x4B4EC6F: rollback_log_create(tokutxn*, blocknum_s, rollback_log_node**) (rollback.cc:170)
4104: ==21516==    by 0x4B4F4D9: toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) (rollback.cc:327)
4104: ==21516==    by 0x4B6F45F: toku_logger_save_rollback_cmdinsert(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2007)
4104: ==21516==    by 0x4AF3117: ft_txn_log_insert(ft*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, ft_msg_type) (ft-ops.cc:2406)
4104: ==21516== Other segment end (thread 42)
4104: ==21516==    at 0x484482C: pthread_mutex_lock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4104: ==21516==    by 0x4A90590: toku_mutex_lock_with_source_location(toku_mutex_t*, char const*, int) (toku_pthread.h:267)
4104: ==21516==    by 0x4AC49BB: pair_lock(ctpair*) (cachetable.cc:117)
4104: ==21516==    by 0x4AC8D38: cachetable_unpin_internal(cachefile*, ctpair*, cachetable_dirty, pair_attr_s, bool) (cachetable.cc:1899)
4104: ==21516==    by 0x4AC8F5B: toku_cachetable_unpin(cachefile*, ctpair*, cachetable_dirty, pair_attr_s) (cachetable.cc:1932)
4104: ==21516==    by 0x4B4ED32: toku_rollback_log_unpin(tokutxn*, rollback_log_node*) (rollback.cc:181)
4104: ==21516==    by 0x4B6F5CE: toku_logger_save_rollback_cmdinsert(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2029)
4104: ==21516==    by 0x4AF3117: ft_txn_log_insert(ft*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, ft_msg_type) (ft-ops.cc:2406)
4104: ==21516==    by 0x4AF32B0: toku_ft_maybe_insert(ft_handle*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, __toku_lsn, bool, ft_msg_type) (ft-ops.cc:2423)
4104: ==21516==    by 0x4AA2AA9: db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, int, bool) (ydb_write.cc:221)
4104: ==21516==    by 0x4AA2CB3: toku_db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, unsigned int, bool) (ydb_write.cc:248)
4104: ==21516==    by 0x4AA68FC: autotxn_db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, unsigned int) (ydb_write.cc:1078)
4104: ==21516==
4104: ==21516== Conflicting store by thread 43 at 0x04c22440 size 4
4104: ==21516==    at 0x4B4EB32: rollback_log_create(tokutxn*, blocknum_s, rollback_log_node**) (rollback.cc:160)
4104: ==21516==    by 0x4B4F4D9: toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) (rollback.cc:327)
4104: ==21516==    by 0x4B6F45F: toku_logger_save_rollback_cmdinsert(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2007)
4104: ==21516==    by 0x4AF3117: ft_txn_log_insert(ft*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, ft_msg_type) (ft-ops.cc:2406)
4104: ==21516==    by 0x4AF32B0: toku_ft_maybe_insert(ft_handle*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, __toku_lsn, bool, ft_msg_type) (ft-ops.cc:2423)
4104: ==21516==    by 0x4AA2AA9: db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, int, bool) (ydb_write.cc:221)
4104: ==21516==    by 0x4AA2CB3: toku_db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, unsigned int, bool) (ydb_write.cc:248)
4104: ==21516==    by 0x4AA68FC: autotxn_db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, unsigned int) (ydb_write.cc:1078)
4104: ==21516==    by 0x10B099: start_a_thread(void*) (test_groupcommit_count.cc:64)
4104: ==21516==    by 0x48414BA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4104: ==21516==    by 0x4C5D668: start_thread (pthread_create.c:479)
4104: ==21516==    by 0x4D99322: clone (clone.S:95)
4104: ==21516== Allocation context: BSS section of /home/rfp/projects/tokuft-Debug-gcc-7/src/libtokufractaltree.so
4104: ==21516== Other segment start (thread 42)
4104: ==21516==    at 0x484587C: pthread_mutex_unlock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4104: ==21516==    by 0x4B8BE37: pc_unlock() (partitioned_counter.cc:151)
4104: ==21516==    by 0x4B8C583: get_or_alloc_thread_local_counter(partitioned_counter*) (partitioned_counter.cc:366)
4104: ==21516==    by 0x4B8C5C0: increment_partitioned_counter (partitioned_counter.cc:375)
4104: ==21516==    by 0x4ACD9B5: evictor::add_pair_attr(pair_attr_s) (cachetable.cc:3769)
4104: ==21516==    by 0x4AC684E: cachetable_insert_pair_at(cachetable*, ctpair*, pair_attr_s) (cachetable.cc:871)
4104: ==21516==    by 0x4AC689E: cachetable_put_internal(cachefile*, ctpair*, void*, pair_attr_s, void (*)(blocknum_s, void*, ctpair*)) (cachetable.cc:897)
4104: ==21516==    by 0x4AC7410: toku_cachetable_put(cachefile*, blocknum_s, unsigned int, void*, pair_attr_s, CACHETABLE_WRITE_CALLBACK, void (*)(blocknum_s, void*, ctpair*)) (cachetable.cc:1207)
4104: ==21516==    by 0x4B4EC6F: rollback_log_create(tokutxn*, blocknum_s, rollback_log_node**) (rollback.cc:170)
4104: ==21516==    by 0x4B4F4D9: toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) (rollback.cc:327)
4104: ==21516==    by 0x4B6F45F: toku_logger_save_rollback_cmdinsert(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2007)
4104: ==21516==    by 0x4AF3117: ft_txn_log_insert(ft*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, ft_msg_type) (ft-ops.cc:2406)
4104: ==21516== Other segment end (thread 42)
4104: ==21516==    at 0x484482C: pthread_mutex_lock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4104: ==21516==    by 0x4A90590: toku_mutex_lock_with_source_location(toku_mutex_t*, char const*, int) (toku_pthread.h:267)
4104: ==21516==    by 0x4AC49BB: pair_lock(ctpair*) (cachetable.cc:117)
4104: ==21516==    by 0x4AC8D38: cachetable_unpin_internal(cachefile*, ctpair*, cachetable_dirty, pair_attr_s, bool) (cachetable.cc:1899)
4104: ==21516==    by 0x4AC8F5B: toku_cachetable_unpin(cachefile*, ctpair*, cachetable_dirty, pair_attr_s) (cachetable.cc:1932)
4104: ==21516==    by 0x4B4ED32: toku_rollback_log_unpin(tokutxn*, rollback_log_node*) (rollback.cc:181)
4104: ==21516==    by 0x4B6F5CE: toku_logger_save_rollback_cmdinsert(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2029)
4104: ==21516==    by 0x4AF3117: ft_txn_log_insert(ft*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, ft_msg_type) (ft-ops.cc:2406)
4104: ==21516==    by 0x4AF32B0: toku_ft_maybe_insert(ft_handle*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, __toku_lsn, bool, ft_msg_type) (ft-ops.cc:2423)
4104: ==21516==    by 0x4AA2AA9: db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, int, bool) (ydb_write.cc:221)
4104: ==21516==    by 0x4AA2CB3: toku_db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, unsigned int, bool) (ydb_write.cc:248)
4104: ==21516==    by 0x4AA68FC: autotxn_db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, unsigned int) (ydb_write.cc:1078)

ctest -R ydb/drd_tiny_test_stress0 --verbose

4107: ==21390== Thread 52:
4107: ==21390== Conflicting load by thread 52 at 0x04c22440 size 4
4107: ==21390==    at 0x4B4EB26: rollback_log_create(tokutxn*, blocknum_s, rollback_log_node**) (rollback.cc:160)
4107: ==21390==    by 0x4B4F4D9: toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) (rollback.cc:327)
4107: ==21390==    by 0x4B6FF22: toku_logger_save_rollback_cmdupdate(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2168)
4107: ==21390==    by 0x4AF3741: toku_ft_maybe_update(ft_handle*, __toku_dbt const*, __toku_dbt const*, tokutxn*, bool, __toku_lsn, bool) (ft-ops.cc:2485)
4107: ==21390==    by 0x4AA2FB8: toku_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:290)
4107: ==21390==    by 0x4AA69C5: autotxn_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:1094)
4107: ==21390==    by 0x113C8F: update_op_db(__toku_db*, __toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1488)
4107: ==21390==    by 0x113D55: update_op(__toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1508)
4107: ==21390==    by 0x10F64D: worker(void*) (threaded_stress_test_helpers.h:537)
4107: ==21390==    by 0x48414BA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4107: ==21390==    by 0x4C8B668: start_thread (pthread_create.c:479)
4107: ==21390==    by 0x4DC7322: clone (clone.S:95)
4107: ==21390== Allocation context: BSS section of /home/rfp/projects/tokuft-Debug-gcc-7/src/libtokufractaltree.so
4107: ==21390== Other segment start (thread 53)
4107: ==21390==    at 0x484E3FB: pthread_rwlock_unlock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4107: ==21390==    by 0x4A9095C: toku_pthread_rwlock_wrunlock(toku_pthread_rwlock_t*) (toku_pthread.h:496)
4107: ==21390==    by 0x4ACCE5D: pair_list::write_list_unlock() (cachetable.cc:3580)
4107: ==21390==    by 0x4AC742C: toku_cachetable_put(cachefile*, blocknum_s, unsigned int, void*, pair_attr_s, CACHETABLE_WRITE_CALLBACK, void (*)(blocknum_s, void*, ctpair*)) (cachetable.cc:1215)
4107: ==21390==    by 0x4B4EC6F: rollback_log_create(tokutxn*, blocknum_s, rollback_log_node**) (rollback.cc:170)
4107: ==21390==    by 0x4B4F4D9: toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) (rollback.cc:327)
4107: ==21390==    by 0x4B6FF22: toku_logger_save_rollback_cmdupdate(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2168)
4107: ==21390==    by 0x4AF3741: toku_ft_maybe_update(ft_handle*, __toku_dbt const*, __toku_dbt const*, tokutxn*, bool, __toku_lsn, bool) (ft-ops.cc:2485)
4107: ==21390==    by 0x4AA2FB8: toku_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:290)
4107: ==21390==    by 0x4AA69C5: autotxn_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:1094)
4107: ==21390==    by 0x113C8F: update_op_db(__toku_db*, __toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1488)
4107: ==21390==    by 0x113D55: update_op(__toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1508)
4107: ==21390== Other segment end (thread 53)
4107: ==21390==    at 0x484482C: pthread_mutex_lock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4107: ==21390==    by 0x4A90590: toku_mutex_lock_with_source_location(toku_mutex_t*, char const*, int) (toku_pthread.h:267)
4107: ==21390==    by 0x4AC49BB: pair_lock(ctpair*) (cachetable.cc:117)
4107: ==21390==    by 0x4AC8D38: cachetable_unpin_internal(cachefile*, ctpair*, cachetable_dirty, pair_attr_s, bool) (cachetable.cc:1899)
4107: ==21390==    by 0x4AC8F5B: toku_cachetable_unpin(cachefile*, ctpair*, cachetable_dirty, pair_attr_s) (cachetable.cc:1932)
4107: ==21390==    by 0x4B4ED32: toku_rollback_log_unpin(tokutxn*, rollback_log_node*) (rollback.cc:181)
4107: ==21390==    by 0x4B70091: toku_logger_save_rollback_cmdupdate(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2190)
4107: ==21390==    by 0x4AF3741: toku_ft_maybe_update(ft_handle*, __toku_dbt const*, __toku_dbt const*, tokutxn*, bool, __toku_lsn, bool) (ft-ops.cc:2485)
4107: ==21390==    by 0x4AA2FB8: toku_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:290)
4107: ==21390==    by 0x4AA69C5: autotxn_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:1094)
4107: ==21390==    by 0x113C8F: update_op_db(__toku_db*, __toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1488)
4107: ==21390==    by 0x113D55: update_op(__toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1508)
4107: ==21390== Other segment start (thread 53)
4107: ==21390==    at 0x484587C: pthread_mutex_unlock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4107: ==21390==    by 0x4A90508: toku_mutex_unlock(toku_mutex_t*) (toku_pthread.h:256)
4107: ==21390==    by 0x4B521CD: rollback_log_node_cache::get_rollback_log_node(tokutxn*, rollback_log_node**) (rollback_log_node_cache.cc:101)
4107: ==21390==    by 0x4B4F478: toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) (rollback.cc:316)
4107: ==21390==    by 0x4B6FF22: toku_logger_save_rollback_cmdupdate(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2168)
4107: ==21390==    by 0x4AF3741: toku_ft_maybe_update(ft_handle*, __toku_dbt const*, __toku_dbt const*, tokutxn*, bool, __toku_lsn, bool) (ft-ops.cc:2485)
4107: ==21390==    by 0x4AA2FB8: toku_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:290)
4107: ==21390==    by 0x4AA69C5: autotxn_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:1094)
4107: ==21390==    by 0x113C8F: update_op_db(__toku_db*, __toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1488)
4107: ==21390==    by 0x113D55: update_op(__toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1508)
4107: ==21390==    by 0x10F64D: worker(void*) (threaded_stress_test_helpers.h:537)
4107: ==21390==    by 0x48414BA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4107: ==21390== Other segment end (thread 53)
4107: ==21390==    at 0x484482C: pthread_mutex_lock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4107: ==21390==    by 0x4A90590: toku_mutex_lock_with_source_location(toku_mutex_t*, char const*, int) (toku_pthread.h:267)
4107: ==21390==    by 0x4B3A2D1: block_table::_mutex_lock() (block_table.cc:72)
4107: ==21390==    by 0x4B3BE7D: block_table::allocate_blocknum(blocknum_s*, ft*) (block_table.cc:682)
4107: ==21390==    by 0x4B4EBBB: rollback_log_create(tokutxn*, blocknum_s, rollback_log_node**) (rollback.cc:167)
4107: ==21390==    by 0x4B4F4D9: toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) (rollback.cc:327)
4107: ==21390==    by 0x4B6FF22: toku_logger_save_rollback_cmdupdate(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2168)
4107: ==21390==    by 0x4AF3741: toku_ft_maybe_update(ft_handle*, __toku_dbt const*, __toku_dbt const*, tokutxn*, bool, __toku_lsn, bool) (ft-ops.cc:2485)
4107: ==21390==    by 0x4AA2FB8: toku_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:290)
4107: ==21390==    by 0x4AA69C5: autotxn_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:1094)
4107: ==21390==    by 0x113C8F: update_op_db(__toku_db*, __toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1488)
4107: ==21390==    by 0x113D55: update_op(__toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1508)
4107: ==21390==
4107: ==21390== Conflicting store by thread 52 at 0x04c22440 size 4
4107: ==21390==    at 0x4B4EB32: rollback_log_create(tokutxn*, blocknum_s, rollback_log_node**) (rollback.cc:160)
4107: ==21390==    by 0x4B4F4D9: toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) (rollback.cc:327)
4107: ==21390==    by 0x4B6FF22: toku_logger_save_rollback_cmdupdate(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2168)
4107: ==21390==    by 0x4AF3741: toku_ft_maybe_update(ft_handle*, __toku_dbt const*, __toku_dbt const*, tokutxn*, bool, __toku_lsn, bool) (ft-ops.cc:2485)
4107: ==21390==    by 0x4AA2FB8: toku_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:290)
4107: ==21390==    by 0x4AA69C5: autotxn_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:1094)
4107: ==21390==    by 0x113C8F: update_op_db(__toku_db*, __toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1488)
4107: ==21390==    by 0x113D55: update_op(__toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1508)
4107: ==21390==    by 0x10F64D: worker(void*) (threaded_stress_test_helpers.h:537)
4107: ==21390==    by 0x48414BA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4107: ==21390==    by 0x4C8B668: start_thread (pthread_create.c:479)
4107: ==21390==    by 0x4DC7322: clone (clone.S:95)
4107: ==21390== Allocation context: BSS section of /home/rfp/projects/tokuft-Debug-gcc-7/src/libtokufractaltree.so
4107: ==21390== Other segment start (thread 53)
4107: ==21390==    at 0x484E3FB: pthread_rwlock_unlock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4107: ==21390==    by 0x4A9095C: toku_pthread_rwlock_wrunlock(toku_pthread_rwlock_t*) (toku_pthread.h:496)
4107: ==21390==    by 0x4ACCE5D: pair_list::write_list_unlock() (cachetable.cc:3580)
4107: ==21390==    by 0x4AC742C: toku_cachetable_put(cachefile*, blocknum_s, unsigned int, void*, pair_attr_s, CACHETABLE_WRITE_CALLBACK, void (*)(blocknum_s, void*, ctpair*)) (cachetable.cc:1215)
4107: ==21390==    by 0x4B4EC6F: rollback_log_create(tokutxn*, blocknum_s, rollback_log_node**) (rollback.cc:170)
4107: ==21390==    by 0x4B4F4D9: toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) (rollback.cc:327)
4107: ==21390==    by 0x4B6FF22: toku_logger_save_rollback_cmdupdate(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2168)
4107: ==21390==    by 0x4AF3741: toku_ft_maybe_update(ft_handle*, __toku_dbt const*, __toku_dbt const*, tokutxn*, bool, __toku_lsn, bool) (ft-ops.cc:2485)
4107: ==21390==    by 0x4AA2FB8: toku_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:290)
4107: ==21390==    by 0x4AA69C5: autotxn_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:1094)
4107: ==21390==    by 0x113C8F: update_op_db(__toku_db*, __toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1488)
4107: ==21390==    by 0x113D55: update_op(__toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1508)
4107: ==21390== Other segment end (thread 53)
4107: ==21390==    at 0x484482C: pthread_mutex_lock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4107: ==21390==    by 0x4A90590: toku_mutex_lock_with_source_location(toku_mutex_t*, char const*, int) (toku_pthread.h:267)
4107: ==21390==    by 0x4AC49BB: pair_lock(ctpair*) (cachetable.cc:117)
4107: ==21390==    by 0x4AC8D38: cachetable_unpin_internal(cachefile*, ctpair*, cachetable_dirty, pair_attr_s, bool) (cachetable.cc:1899)
4107: ==21390==    by 0x4AC8F5B: toku_cachetable_unpin(cachefile*, ctpair*, cachetable_dirty, pair_attr_s) (cachetable.cc:1932)
4107: ==21390==    by 0x4B4ED32: toku_rollback_log_unpin(tokutxn*, rollback_log_node*) (rollback.cc:181)
4107: ==21390==    by 0x4B70091: toku_logger_save_rollback_cmdupdate(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2190)
4107: ==21390==    by 0x4AF3741: toku_ft_maybe_update(ft_handle*, __toku_dbt const*, __toku_dbt const*, tokutxn*, bool, __toku_lsn, bool) (ft-ops.cc:2485)
4107: ==21390==    by 0x4AA2FB8: toku_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:290)
4107: ==21390==    by 0x4AA69C5: autotxn_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:1094)
4107: ==21390==    by 0x113C8F: update_op_db(__toku_db*, __toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1488)
4107: ==21390==    by 0x113D55: update_op(__toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1508)
4107: ==21390== Other segment start (thread 53)
4107: ==21390==    at 0x484587C: pthread_mutex_unlock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4107: ==21390==    by 0x4A90508: toku_mutex_unlock(toku_mutex_t*) (toku_pthread.h:256)
4107: ==21390==    by 0x4B521CD: rollback_log_node_cache::get_rollback_log_node(tokutxn*, rollback_log_node**) (rollback_log_node_cache.cc:101)
4107: ==21390==    by 0x4B4F478: toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) (rollback.cc:316)
4107: ==21390==    by 0x4B6FF22: toku_logger_save_rollback_cmdupdate(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2168)
4107: ==21390==    by 0x4AF3741: toku_ft_maybe_update(ft_handle*, __toku_dbt const*, __toku_dbt const*, tokutxn*, bool, __toku_lsn, bool) (ft-ops.cc:2485)
4107: ==21390==    by 0x4AA2FB8: toku_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:290)
4107: ==21390==    by 0x4AA69C5: autotxn_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:1094)
4107: ==21390==    by 0x113C8F: update_op_db(__toku_db*, __toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1488)
4107: ==21390==    by 0x113D55: update_op(__toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1508)
4107: ==21390==    by 0x10F64D: worker(void*) (threaded_stress_test_helpers.h:537)
4107: ==21390==    by 0x48414BA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4107: ==21390== Other segment end (thread 53)
4107: ==21390==    at 0x484482C: pthread_mutex_lock (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_drd-amd64-linux.so)
4107: ==21390==    by 0x4A90590: toku_mutex_lock_with_source_location(toku_mutex_t*, char const*, int) (toku_pthread.h:267)
4107: ==21390==    by 0x4B3A2D1: block_table::_mutex_lock() (block_table.cc:72)
4107: ==21390==    by 0x4B3BE7D: block_table::allocate_blocknum(blocknum_s*, ft*) (block_table.cc:682)
4107: ==21390==    by 0x4B4EBBB: rollback_log_create(tokutxn*, blocknum_s, rollback_log_node**) (rollback.cc:167)
4107: ==21390==    by 0x4B4F4D9: toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) (rollback.cc:327)
4107: ==21390==    by 0x4B6FF22: toku_logger_save_rollback_cmdupdate(tokutxn*, FILENUM, BYTESTRING*) (log_code.cc:2168)
4107: ==21390==    by 0x4AF3741: toku_ft_maybe_update(ft_handle*, __toku_dbt const*, __toku_dbt const*, tokutxn*, bool, __toku_lsn, bool) (ft-ops.cc:2485)
4107: ==21390==    by 0x4AA2FB8: toku_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:290)
4107: ==21390==    by 0x4AA69C5: autotxn_db_update(__toku_db*, __toku_db_txn*, __toku_dbt const*, __toku_dbt const*, unsigned int) (ydb_write.cc:1094)
4107: ==21390==    by 0x113C8F: update_op_db(__toku_db*, __toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1488)
4107: ==21390==    by 0x113D55: update_op(__toku_db_txn*, arg*, void*, void*) (threaded_stress_test_helpers.h:1508)

Also observed by the thread sanitizer when running
ctest -R ydb/checkpoint_fairness --verbose

4793: WARNING: ThreadSanitizer: data race (pid=10298)
4793:   Write of size 4 at 0x7fd7194ac2b8 by thread T35 (mutexes: read M11, write M1048777):
4793:     #0 rollback_log_create(tokutxn*, blocknum_s, rollback_log_node**) /home/rfp/projects/tokuft/ft/txn/rollback.cc:160:21 (libtokufractaltree.so+0x18b990)
4793:     #1 toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) /home/rfp/projects/tokuft/ft/txn/rollback.cc:327:13 (libtokufractaltree.so+0x18b4c9)
4793:     #2 toku_logger_save_rollback_cmdinsert(tokutxn*, FILENUM, BYTESTRING*) /home/rfp/projects/tokuft-tsan/ft/log_code.cc:2007:3 (libtokufractaltree.so+0x1c7912)
4793:     #3 ft_txn_log_insert(ft*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, ft_msg_type) /home/rfp/projects/tokuft/ft/ft-ops.cc:2406:9 (libtokufractaltree.so+0xd2c2c)
4793:     #4 toku_ft_maybe_insert(ft_handle*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, __toku_lsn, bool, ft_msg_type) /home/rfp/projects/tokuft/ft/ft-ops.cc:2423:5 (libtokufractaltree.so+0xd3135)
4793:     #5 db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, int, bool) /home/rfp/projects/tokuft/src/ydb_write.cc:221:13 (libtokufractaltree.so+0x45d33)
4793:     #6 toku_db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, unsigned int, bool) /home/rfp/projects/tokuft/src/ydb_write.cc:248:13 (libtokufractaltree.so+0x456e3)
4793:     #7 autotxn_db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, unsigned int) /home/rfp/projects/tokuft/src/ydb_write.cc:1078:9 (libtokufractaltree.so+0x4ab47)
4793:     #8 start_txns(void*) /home/rfp/projects/tokuft/src/tests/checkpoint_fairness.cc:73:16 (checkpoint_fairness.tdb+0x4b6f5b)
4793:
4793:   Previous write of size 4 at 0x7fd7194ac2b8 by threa T34 (mutexes: read M11, write M1048756):
4793:     #0 rollback_log_create(tokutxn*, blocknum_s, rollback_log_node**) /home/rfp/projects/tokuft/ft/txn/rollback.cc:175:22 (libtokufractaltree.so+0x18bc94)
4793:     #1 toku_get_and_pin_rollback_log_for_new_entry(tokutxn*, rollback_log_node**) /home/rfp/projects/tokuft/ft/txn/rollback.cc:327:13 (libtokufractaltree.so+0x18b4c9)
4793:     #2 toku_logger_save_rollback_cmdinsert(tokutxn*, FILENUM, BYTESTRING*) /home/rfp/projects/tokuft-tsan/ft/log_code.cc:2007:3 (libtokufractaltree.so+0x1c7912)
4793:     #3 ft_txn_log_insert(ft*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, ft_msg_type) /home/rfp/projects/tokuft/ft/ft-ops.cc:2406:9 (libtokufractaltree.so+0xd2c2c)
4793:     #4 toku_ft_maybe_insert(ft_handle*, __toku_dbt*, __toku_dbt*, tokutxn*, bool, __toku_lsn, bool, ft_msg_type) /home/rfp/projects/tokuft/ft/ft-ops.cc:2423:5 (libtokufractaltree.so+0xd3135)
4793:     #5 db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, int, bool) /home/rfp/projects/tokuft/src/ydb_write.cc:221:13 (libtokufractaltree.so+0x45d33)
4793:     #6 toku_db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, unsigned int, bool) /home/rfp/projects/tokuft/src/ydb_write.cc:248:13 (libtokufractaltree.so+0x456e3)
4793:     #7 autotxn_db_put(__toku_db*, __toku_db_txn*, __toku_dbt*, __toku_dbt*, unsigned int) /home/rfp/projects/tokuft/src/ydb_write.cc:1078:9 (libtokufractaltree.so+0x4ab47)
4793:     #8 start_txns(void*) /home/rfp/projects/tokuft/src/tests/checkpoint_fairness.cc:73:16 (checkpoint_fairness.tdb+0x4b6f5b)
4793:
4793:   Location is global 'writing_rollback' of size 4 at 0x7fd7194ac2b8 (libtokufractaltree.so+0x0000002972b8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@prohaska7