Enabling core dumps v16

You can use core dumps to diagnose or debug errors. A core dump is a file containing a process's address space (memory) when the process terminates unexpectedly. Core dumps can be produced on demand, such as by a debugger, or upon termination.

Enabling core dumps on a RHEL or CentOS or Rocky Linux or AlmaLinux host

On RHEL/CentOS 7.x or RHEL/Rocky Linux/AlmaLinux 8.x, core file creation is disabled by default. To enable the core file generation:

  • Identify the system's current limit using the ulimit -c or ulimit -a command. 0 indicates that core file generation is disabled.

    # ulimit -c
    0
    
    # ulimit -a
    core file size          (blocks, -c) 0
    data seg size           (kbytes, -d) unlimited
    scheduling priority             (-e) 0
    file size               (blocks, -f) unlimited
    pending signals                 (-i) 3756
    max locked memory       (kbytes, -l) 64
    max memory size         (kbytes, -m) unlimited
    open files                      (-n) 1024
    pipe size            (512 bytes, -p) 8
    POSIX message queues     (bytes, -q) 819200
    real-time priority              (-r) 0
    stack size              (kbytes, -s) 8192
    cpu time               (seconds, -t) unlimited
    max user processes              (-u) 3756
    virtual memory          (kbytes, -v) unlimited
    file locks                      (-x) unlimited
  • Create a directory to store the core dumps, and modify kernel.core_pattern to store the dumps in a specified directory:

    mkdir -p /var/coredumps
    chmod a+w /var/coredumps
    
    sysctl kernel.core_pattern=/var/coredumps/core-%e-%p
    kernel.core_pattern = /var/coredumps/core-%e-%p
  • Persist the kernel.core_pattern setting across reboots:

    echo 'kernel.core_pattern=/var/coredumps/core-%e-%p' >> /etc/sysctl.conf
  • Enable core dumps in /etc/security/limits.conf to allow a user to create core files. Each line describes a limit for a user in the following form:

    <domain>  <type>  <item>  <value>
       *       soft    core    unlimited

    Use * to enable the core dump size to unlimited.

  • Set the limit of core file size to UNLIMITED:

    ulimit -c unlimited
    
    ulimit -c
    unlimited
  • To set a core limit for the services, add the following setting in /usr/lib/systemd/system/edb-as-15.service:

    [Service]
    LimitCore=Infinity
  • Reload the service configuration:

    systemctl daemon-reload
  • Modify the global default limit using systemd. Add the following setting in /etc/systemd/system.conf:

    DefaultLimitCORE=Infinity
  • Restart systemd:

    systemctl daemon-reexec
  • Stop and then start EDB Postgres Advanced Server:

    systemctl stop edb-as-15
    systemctl start edb-as-15
  • Now, the core dumps are enabled. Install the gdb tool and debug packages:

    yum install gdb
    debuginfo-install edb-as15 edb-as15-server-contrib edb-as15-server edb-as15-libs
  • Replace the path to a core dump file. Then, get a backtrace using the bt command to analyze output:

    gdb /usr/edb/as15/bin /var/coredumps/core-edb-postgres-65499
    (gdb) bt full

Enabling core dumps on a Debian or Ubuntu host

On Debian 10 or Ubuntu 18 and 20, core file creation is disabled by default. To enable the core file generation:

  • Identify the system's current limit using the ulimit -c or ulimit -a command. 0 indicates that core file generation is disabled.

    # ulimit -c
    0
    
    # ulimit -a
    core file size          (blocks, -c) 0
    data seg size           (kbytes, -d) unlimited
    scheduling priority             (-e) 0
    file size               (blocks, -f) unlimited
    pending signals                 (-i) 7617
    max locked memory       (kbytes, -l) 65536
    max memory size         (kbytes, -m) unlimited
    open files                      (-n) 1024
    pipe size            (512 bytes, -p) 8
    POSIX message queues     (bytes, -q) 819200
    real-time priority              (-r) 0
    stack size              (kbytes, -s) 8192
    cpu time               (seconds, -t) unlimited
    max user processes              (-u) 7617
    virtual memory          (kbytes, -v) unlimited
    file locks                      (-x) unlimited
  • Create a new directory to store the core dumps, and modify kernel.core_pattern to store the dumps in a specified directory:

    mkdir -p /var/coredumps
    chmod a+w /var/coredumps
    
    sysctl kernel.core_pattern=/var/coredumps/core-%e-%p
    kernel.core_pattern = /var/coredumps/core-%e-%p
  • Persist the kernel.core_pattern setting across reboots:

    echo 'kernel.core_pattern=/var/coredumps/core-%e-%p' >> /etc/sysctl.conf
  • To allow a user to create core files, enable core dumps in /etc/security/limits.conf. Each line describes a limit for a user in the following form.

    <domain>  <type>  <item>  <value>
       *       soft    core    unlimited

    Use * to enable the core dump size to unlimited.

  • Set the limit of core file size to UNLIMITED:

    ulimit -c unlimited
    
    ulimit -c
    unlimited
  • To set a core limit for the services, add the following setting in /lib/systemd/system/edb-as@.service:

    [Service]
    LimitCore=Infinity
  • Reload the service configuration:

    systemctl daemon-reload
  • Modify the global default limit using systemd. Add the following setting in /etc/systemd/system.conf:

    DefaultLimitCORE=Infinity
  • Restart systemd:

    systemctl daemon-reexec
  • Stop and then start EDB Postgres Advanced Server:

    systemctl stop edb-as@15.service
    systemctl start edb-as@15.service
  • Now, the core dumps are enabled. Install the gdb tool and debug symbols:

    apt-get install gdb
    apt-get install edb-as15 edb-as-contrib edb-as15-server edb-debugger-dbgsym
  • Replace the path to a core dump file. Then get a backtrace using the bt command to analyze output:

    gdb /usr/lib/edb-as/15/bin /var/coredumps/core-edb-postgres-21638
    (gdb) bt full
Note
  • The debug info packages name on a Debian or Ubuntu host can vary and include a -dbgsym or -dbg suffix. For more information about setting sources.list and installing the debug info packages, see the Debian wiki or the Ubuntu wiki.
  • The core files can be huge, depending on the memory usage. Enabling the core dumps on a system might fill up its mass storage over time.