Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

confinement: share geometry data between threads #94

Merged
merged 1 commit into from
May 10, 2024

Conversation

ManuelHu
Copy link
Collaborator

This fixed a very common memory safety error in MT mode with vertex confinement.

ASan report before this fix:

=================================================================
==1149==ERROR: AddressSanitizer: attempting double-free on 0x6020006e8ff0 in thread T2:
    #0 0x55ebc1f0143f in operator delete(void*, unsigned long) (/work/remage/build/src/remage+0x26943f)
    #1 0x7fcd575cf1b2 in G4SolidStore::DeRegister(G4VSolid*) (/opt/geant4/lib/libG4geometry.so+0x1541b2)
    #2 0x7fcd575d1742 in G4VSolid::~G4VSolid() (/opt/geant4/lib/libG4geometry.so+0x156742)
    #3 0x7fcd5763d36d in G4SubtractionSolid::GetCubicVolume() (/opt/geant4/lib/libG4geometry.so+0x1c236d)
    #4 0x7fcd5ef62227 in RMGVertexConfinement::SampleableObject::SampleableObject(G4VPhysicalVolume*, CLHEP::HepRotation, CLHEP::Hep3Vector, G4VSolid*, bool) /work/remage/src/RMGVertexConfinement.cc:61
    #5 0x7fcd5efc205b in void __gnu_cxx::new_allocator<RMGVertexConfinement::SampleableObject>::construct<RMGVertexConfinement::SampleableObject, G4VPhysicalVolume*&, CLHEP::HepRotation const&, CLHEP::Hep3Vector const&, G4VSolid*&, bool&>(RMGVertexConfinement::SampleableObject*, G4VPhysicalVolume*&, CLHEP::HepRotation const&, CLHEP::Hep3Vector const&, G4VSolid*&, bool&) /usr/include/c++/11/ext/new_allocator.h:162
    #6 0x7fcd5efb063d in void std::allocator_traits<std::allocator<RMGVertexConfinement::SampleableObject> >::construct<RMGVertexConfinement::SampleableObject, G4VPhysicalVolume*&, CLHEP::HepRotation const&, CLHEP::Hep3Vector const&, G4VSolid*&, bool&>(std::allocator<RMGVertexConfinement::SampleableObject>&, RMGVertexConfinement::SampleableObject*, G4VPhysicalVolume*&, CLHEP::HepRotation const&, CLHEP::Hep3Vector const&, G4VSolid*&, bool&) /usr/include/c++/11/bits/alloc_traits.h:516
    #7 0x7fcd5efb0bd5 in void std::vector<RMGVertexConfinement::SampleableObject, std::allocator<RMGVertexConfinement::SampleableObject> >::_M_realloc_insert<G4VPhysicalVolume*&, CLHEP::HepRotation const&, CLHEP::Hep3Vector const&, G4VSolid*&, bool&>(__gnu_cxx::__normal_iterator<RMGVertexConfinement::SampleableObject*, std::vector<RMGVertexConfinement::SampleableObject, std::allocator<RMGVertexConfinement::SampleableObject> > >, G4VPhysicalVolume*&, CLHEP::HepRotation const&, CLHEP::Hep3Vector const&, G4VSolid*&, bool&) /usr/include/c++/11/bits/vector.tcc:449
    #8 0x7fcd5ef9d667 in RMGVertexConfinement::SampleableObject& std::vector<RMGVertexConfinement::SampleableObject, std::allocator<RMGVertexConfinement::SampleableObject> >::emplace_back<G4VPhysicalVolume*&, CLHEP::HepRotation const&, CLHEP::Hep3Vector const&, G4VSolid*&, bool&>(G4VPhysicalVolume*&, CLHEP::HepRotation const&, CLHEP::Hep3Vector const&, G4VSolid*&, bool&) /usr/include/c++/11/bits/vector.tcc:121
    #9 0x7fcd5ef64d7b in RMGVertexConfinement::SampleableObjectCollection::emplace_back(G4VPhysicalVolume*, CLHEP::HepRotation const&, CLHEP::Hep3Vector const&, G4VSolid*, bool) /work/remage/src/RMGVertexConfinement.cc:118
    #10 0x7fcd5ef679d4 in RMGVertexConfinement::InitializePhysicalVolumes() /work/remage/src/RMGVertexConfinement.cc:163
    #11 0x7fcd5ef71b29 in RMGVertexConfinement::ActualGenerateVertex(CLHEP::Hep3Vector&) /work/remage/src/RMGVertexConfinement.cc:410
    #12 0x7fcd5ef71474 in RMGVertexConfinement::GenerateVertex(CLHEP::Hep3Vector&) /work/remage/src/RMGVertexConfinement.cc:398
    #13 0x7fcd5eed4d9f in RMGMasterGenerator::GeneratePrimaries(G4Event*) /work/remage/src/RMGMasterGenerator.cc:57
    #14 0x7fcd5a190bfe in G4WorkerTaskRunManager::GenerateEvent(int) (/opt/geant4/lib/libG4run.so+0xc2bfe)
    #15 0x7fcd5a18f093 in G4WorkerTaskRunManager::ProcessOneEvent(int) (/opt/geant4/lib/libG4run.so+0xc1093)
    #16 0x7fcd5a18ed95 in G4WorkerTaskRunManager::DoEventLoop(int, char const*, int) (/opt/geant4/lib/libG4run.so+0xc0d95)
    #17 0x7fcd5a18ef7c in G4WorkerTaskRunManager::DoWork() (/opt/geant4/lib/libG4run.so+0xc0f7c)
    #18 0x7fcd5a1666c1 in std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, std::__future_base::_Task_state<PTL::TaskGroup<void, void, 0l>::exec<G4TaskRunManager::AddEventTask(int)::{lambda()#1}, , void>(G4TaskRunManager::AddEventTask(int)::{lambda()#1})::{lambda()#1}, std::allocator<int>, void ()>::_M_run()::{lambda()#1}, void> >::_M_invoke(std::_Any_data const&) (/opt/geant4/lib/libG4run.so+0x986c1)
    #19 0x7fcd5a16c3ac in std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*) (/opt/geant4/lib/libG4run.so+0x9e3ac)
    #20 0x7fcd56653ee7 in __pthread_once_slow nptl/pthread_once.c:116
    #21 0x7fcd5a16c521 in PTL::Task<void>::operator()() (/opt/geant4/lib/libG4run.so+0x9e521)
    #22 0x7fcd57161930 in PTL::ThreadPool::execute_thread(PTL::VUserTaskQueue*) (/opt/geant4/lib/libG4ptl.so.2+0x16930)
    #23 0x7fcd571623db in PTL::ThreadPool::start_thread(PTL::ThreadPool*, std::vector<std::shared_ptr<PTL::ThreadData>, std::allocator<std::shared_ptr<PTL::ThreadData> > >*, long) (/opt/geant4/lib/libG4ptl.so.2+0x173db)
    #24 0x7fcd56ffb252  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xdc252)
    #25 0x7fcd5664eac2 in start_thread nptl/pthread_create.c:442
    #26 0x7fcd566dfa03 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x125a03)

0x6020006e8ff0 is located 0 bytes inside of 8-byte region [0x6020006e8ff0,0x6020006e8ff8)
freed by thread T1 here:
    #0 0x55ebc1f0143f in operator delete(void*, unsigned long) (/work/remage/build/src/remage+0x26943f)
    #1 0x7fcd575cf1b2 in G4SolidStore::DeRegister(G4VSolid*) (/opt/geant4/lib/libG4geometry.so+0x1541b2)

previously allocated by thread T2 here:
    #0 0x55ebc1f003d7 in operator new(unsigned long) (/work/remage/build/src/remage+0x2683d7)
    #1 0x7fcd575cff63 in G4SolidStore::Register(G4VSolid*) (/opt/geant4/lib/libG4geometry.so+0x154f63)

Thread T2 created by T0 here:
    #0 0x55ebc1ea2895 in pthread_create (/work/remage/build/src/remage+0x20a895)
    #1 0x7fcd56ffb328 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/lib/x86_64-linux-gnu/libstdc++.so.6+0xdc328)

Thread T1 created by T0 here:
    #0 0x55ebc1ea2895 in pthread_create (/work/remage/build/src/remage+0x20a895)
    #1 0x7fcd56ffb328 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/lib/x86_64-linux-gnu/libstdc++.so.6+0xdc328)

SUMMARY: AddressSanitizer: double-free (/work/remage/build/src/remage+0x26943f) in operator delete(void*, unsigned long)
==1149==ABORTING

to reproduce

$ cmake -DCMAKE_BUILD_TYPE=ASan ..
$ make
$ export ASAN_OPTIONS=detect_leaks=0:log_path=asan.log
$ ctest -R confinement-mt --repeat-until-fail 100

There are some crashes apart from this, but they also disappeared after this change.

Performance impact: not measured, but waiting time should be negligible compared to total simulation time, as this should only be used once.

@ManuelHu ManuelHu merged commit b60ea1a into legend-exp:main May 10, 2024
6 checks passed
@ManuelHu ManuelHu deleted the vertex-mt branch May 10, 2024 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant