Advanced Debugging Crash or Memory

Experiencing a Crash or Memory Leak with libopenshot?

This page includes many common commands and techniques for troubleshooting a tricky crash or memory leak in libopenshot or openshot-qt. We primarily use GDB and gperftools for many of these techniques. These are both great tools for attaching to the openshot-qt executable, when needing to troubleshoot an issue.

All of these commands require a local build of OpenShot: Build instructions here.

GDB Debugging (Learn More)

If you are experiencing a freeze or crash, the following GDB commands are usually the best way to quickly see what is happening.

cd [openshot-qt-folder]
gdb python3

(gdb) run src/launch.py
(gdb) CTRL + C      (to manually break out   OR   wait for a crash / seg fault) 
(gdb) info threads  (to view all threads, and what they are doing. Look for `__lll_lock_wait` for Mutex/deadlocks)
(gdb) thread 35     (Switch to thread #)
(gdb) bt            (Print stack trace for the current thread #)

Find Heap addresses using GDB

If you are looking to a Analyze the HEAP memory (perhaps curious what we are storing in the memory), use the following command to access the starting and ending memory address of the heap.

(gdb) info proc mappings     
      0x565012e3a000     0x565072169000 0x5f32f000        0x0 [heap]

Dump all memory to 1 big file (and print alpha words in memory)

NOTE: this is not usually very helpful, as most of our HEAP memory is large pixel arrays / large arrays of numbers. However, it is still an interesting technique.

(gdb) dump memory /home/USERNAME/output 0x565012e3a000 0x565072169000
strings -10 /home/USERNAME/output     # analyze memory for strings 10+ chars

Create core dump file

If you want to create a core dump file using GDB, which allows you to snapshot the debug info and memory for a future analysis. For the debug symbols, use the associated EXE (for example, openshot-example.exe or openshot-player.exe)

(gdb) generate-core-file      # creates core dump file in working directory, i.e. ~/apps/libopenshot-git/cmake-build-debug/examples

Heap analysis:

NOTE: The command pprof is replaced by google-pprof on some distros. You might need to replace some file paths in these commands to match your local distro paths. We can analyze the memory usage (# of memory allocations AND total MB allocated) using gperftools. We need to add the following prefix before our OpenShot executable. Run the executable with this prefix until a crash, or until the end of the program. This will hopefully generate a HUGE log file full of interesting analysis.

PPROF_PATH=/usr/bin/google-pprof LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libtcmalloc.so.4" HEAPCHECK=normal HEAP_CHECK_IDENTIFY_LEAKS=1 HEAP_CHECK_TEST_POINTER_ALIGNMENT=1 HEAP_CHECK_MAX_POINTER_OFFSET=-1 ./examples/openshot-example

Chart by # of leaks (i.e. frequencey of memory leak)

google-pprof ./examples/openshot-example "/tmp/openshot-example.368411._main_-end.heap" --inuse_objects --lines --heapcheck  --edgefraction=1e-10 --nodefraction=1e-10 --gv

Chart by MB of leaks (i.e. Size of memory leak)

google-pprof ./examples/openshot-example "/tmp/openshot-example.368411._main_-end.heap" --lines --heapcheck  --edgefraction=1e-10 --nodefraction=1e-10 --gv

By utilizing one of these 2 charting techniques, you should be able to see where the most memory allocations are happening, and where the largest MB of allocations are happening.

Keywords: libopenshot, memory, profiling, profile, debugging, heap, analysis, google-perf, gdb, freeze, seg fault, crash