Hello. Lately I’ve been noticing that websites stop working randomly and there are issues all over the place. When investigating the issue, a reocurring thing I see is that the 500/500 process limit I’ve set on accounts have been reached. It’s strange because all they have is a wordpress site + mail accounts.
Checking the tasks it’s filled with things like this:
crond is generating an email, which gets sent using $SENDMAIL, /usr/sbin/sendmail in this case. sendmail is a Postfix wrapper that calls postdrop to inject the message into Postfix’s queue in /.socket/postfix/postdrop. postdrop is unable to do so, so let’s work backwards:
First, look for any postdrop errors in /var/log/maillog, e.g. something like this informs you that the user is over quota and thus the file cannot be written to disk:
Feb 1 12:29:22 delta postfix/postdrop[25952]: warning: mail_queue_enter: create file maildrop/383820.25952: Disk quota exceeded
In this case, the user generating the mail delivered onto itself needs more storage. This can be done in User > Manage Users.
Perhaps there’s a permission error - look at /usr/sbin/postdrop within the vfs. This binary requires suid permission:
# ls -la /home/virtual/site12/fst/usr/sbin/postdrop
-rwxr-sr-x 2 root postdrop 294304 Dec 22 2023 /home/virtual/site12/fst/usr/sbin/postdrop
There’s a rare chance crond isn’t seeing the full filesystem; I’m not aware of this being the case any further but you can examine the root directory inode to determine this:
# lsof -p $(cat /home/virtual/site110/fst/var/run/crond.pid)
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
crond 16904 root cwd DIR 0,50 95 34188985 /home/virtual/site110/fst
crond 16904 root rtd DIR 0,50 95 34188985 /home/virtual/site110/fst
If the NAME field reports “deleted” this location no longer exists. Restarting the virtualcron service would clear the file handle:
Run cpcmd misc:release-fsghost /usr/sbin/postdrop. Given the timestamp of that file and last package release September 2024, it’s an old file handle.
# ls -l /usr/sbin/postdrop /home/virtual/FILESYSTEMTEMPLATE/siteinfo/usr/sbin/postdrop /home/virtual/site268/{fst,shadow}/usr/sbin/postdrop
ls: cannot access '/home/virtual/site268/shadow/usr/sbin/postdrop': No such file or directory
-rwxr-sr-x 2 root postdrop 312512 Nov 21 23:47 /home/virtual/FILESYSTEMTEMPLATE/siteinfo/usr/sbin/postdrop
-rwxr-sr-x 2 root postdrop 312512 Nov 21 23:47 /home/virtual/site268/fst/usr/sbin/postdrop
-rwxr-sr-x 2 root postdrop 312512 Nov 21 23:47 /usr/sbin/postdrop
File attributes should agree between FST, vfs, and rootfs as in the above example.
The fact there’s zero hardlinks would be your first hint something is askew, which means there’s no physical reference to that file on disk presently. Terminating all processes that hold an open file handle will allow the layer to refresh and that should fix the setuid permissions as well once the updated postdrop binary propagates.
Runtime of that command isn’t very good right now, so no. Ideally OverlayFS should release stale handles once the process goes out of scope rather than keeping the file in memory.
