a lot of problemos

Hi all,

last nite did a reboot of my server.
Apache wont start.

reboot again.
Same issu.

looking in the file system i dont see any of the hosted website roots.
on the nexus screen no list of domaint. in the top bar I do see 25 accounts and 36 domains.

Please help me out.
Can some one point me to the right direction?

Hmm booted to an ealer kernal and looks like the websites are working again…

Please tell me where to look for errors logs etc.
Thanks in advance :slight_smile:

Which version of the kernel did you change to? Which version were you on? Which OS? What does ip addr list report for the old and new kernel? If device names have changed, run upcp -sbf followed by EditDomain --reconfig --all.

Kernal 6.6.9-1
ip add list: 5 nics with ip addresses etc

upcp -sbf
is giving errors about an package mangement
FAILD centos is 8+ packagement
error cannot open packages database
error rpmdv open maild

I would need to see the exact error messages generated to provide further assistance.

Understand:

PLAY [localhost] ****************************************************************************************************************************************************************************************************************************
included: /usr/local/apnscp/resources/playbooks/roles/common/update-config/tasks/setup-resolved.yml for localhost
included: /usr/local/apnscp/resources/playbooks/roles/common/tasks/write-config.yml for localhost
FAILED - RETRYING: CentOS 8+ package management (2 retries left).
FAILED - RETRYING: CentOS 8+ package management (1 retries left).

TASK [common/update-config : CentOS 8+ package management] **********************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"attempts": 2, "changed": true, "cmd": ["dnf", "remove", "-y", "--duplicates", "--setopt=protected_packages="], "delta": "0:00:00.168362", "end": "2024-03-10 18:21:17.551682", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2024-03-10 18:21:17.383320", "stderr": "error: rpmdb: BDB3037 /var/lib/rpm/Packages: file size not a multiple of the pagesize\nerror: cannot open Packages index using db5 - Invalid argument (22)\nerror: cannot open Packages database in /var/lib/rpm\nError: Error: rpmdb open failed", "stderr_lines": ["error: rpmdb: BDB3037 /var/lib/rpm/Packages: file size not a multiple of the pagesize", "error: cannot open Packages index using db5 - Invalid argument (22)", "error: cannot open Packages database in /var/lib/rpm", "Error: Error: rpmdb open failed"], "stdout": "", "stdout_lines": []}

PLAY RECAP **********************************************************************************************************************************************************************************************************************************
localhost                  : ok=34   changed=0    unreachable=0    failed=1    skipped=19   rescued=0    ignored=0


Smells like your system had an abrupt shutdown during package updates with writethrough caching that didn’t have a battery-backup unit, so the database (/var/lib/rpm/Packages) experienced an incomplete write.

pushd /var/lib/rpm
# This most likely will fail
/usr/lib/rpm/rpmdb_verify Packages
# Backup corrupted "Packages"
mv Packages Packages.orig
/usr/lib/rpm/rpmdb_dump Packages.orig | /usr/lib/rpm/rpmdb_load Packages
rpm -vv --rebuilddb

thanks :slight_smile:
i dont have a raid setup or something with a battery.
still tho executed the commands.
everything looks ok no errors
rebooted.
on the boot from the os i see a lot of starting errors.
also still the same error messages that i have copy pased as before.

on the server console on boot i see - its fast so i cannot realy see them
for example
2 of 2 a start of job is runnig for php -fm pool activation MAIN
openibd - configure mellanox devices
also a bunch of [FAILED] for the hosted websites

See Troubleshooting > Pool fails to start. I’d also advise not using the experimental 6.x kernel if you are experiencing issues. cpcmd scope:set system.kernel system

i have an website with no php just html also not working…
there is more then only the php pools…

can u please check the server remotely?
i can pay for the support?

Then go from there. Obviously you want to work backward to see what caused the issue; running an experimental kernel would be the first thing I’d look at. Once that’s been addressed, look at the logfile for Apache to determine why it’s not starting up.

Start with the easiest thing to change first.

understand.
when i want to set the system.kernal

 cpcmd scope:set system.kernel system
(PDOException) EXCEPTION: SQLSTATE[08006] [7] connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused
        Is the server running on that host and accepting TCP/IP connections?

6.x blew out your network stack on RHEL8. You’ll need to manually revert back to 4.x either at boot or using grubby. Here’s what a quick Google search yielded.

Remember: experimental means just that, you are at risk of breaking something.

thanks :slight_smile:
i dont want to be expirimental only on dev system this is a production system. will check it out :slight_smile:

now booted with kernal 8.9 4.18.0-513.5.1
still the same issus :frowning:

journalctl -n150 -u httpd assuming Apache is still giving you issues. Additional a scrub may be beneficial, upcp -sbf

journactl -n150 -u httpd
-bash: journactl: command not found
journalctl -n150 -u httpd
Mar 10 20:02:37 <servername>, systemd[1]: Starting The Apache HTTP Server...
Mar 10 20:02:37 <servername>, httpd.init[3525]: chown: invalid user: 'apache:apache'
Mar 10 20:02:37 <servername>, httpd.init[3533]: chown: invalid user: 'apache'
Mar 10 20:02:38 <servername>, systemd[1]: httpd.service: Got notification message from PID 3562, but reception only permitted for main PID which is currently not known
Mar 10 20:02:38 <servername>, httpd.init[3481]: Starting httpd:
Mar 10 20:02:38 <servername>, httpd.init[3562]: AH00543: httpd: bad user name apache
Mar 10 20:02:38 <servername>, httpd.init[3539]: [34B blob data]
Mar 10 20:02:38 <servername>, httpd.init[3633]: lsof: can't get UID for apache
Mar 10 20:02:38 <servername>, httpd.init[3633]: lsof 4.93.2
Mar 10 20:02:38 <servername>, httpd.init[3633]:  latest revision: https://github.com/lsof-org/lsof
Mar 10 20:02:38 <servername>, httpd.init[3633]:  latest FAQ: https://github.com/lsof-org/lsof/blob/master/00FAQ
Mar 10 20:02:38 <servername>, httpd.init[3633]:  latest (non-formatted) man page: https://github.com/lsof-org/lsof/blob/master/Lsof.8
Mar 10 20:02:38 <servername>, httpd.init[3633]:  usage: [-?abhKlnNoOPRtUvVX] [+|-c c] [+|-d s] [+D D] [+|-E] [+|-e s] [+|-f[gG]]
Mar 10 20:02:38 <servername>, httpd.init[3633]:  [-F [f]] [-g [s]] [-i [i]] [+|-L [l]] [+m [m]] [+|-M] [-o [o]] [-p s]
Mar 10 20:02:38 <servername>, httpd.init[3633]:  [+|-r [t]] [-s [p:s]] [-S [t]] [-T [t]] [-u s] [+|-w] [-x [fl]] [--] [names]
Mar 10 20:02:38 <servername>, httpd.init[3633]: Use the ``-h'' option to get more help information.
Mar 10 20:02:38 <servername>, httpd.init[3654]: lsof: can't get UID for apache
Mar 10 20:02:38 <servername>, httpd.init[3654]: lsof 4.93.2
Mar 10 20:02:38 <servername>, httpd.init[3654]:  latest revision: https://github.com/lsof-org/lsof
Mar 10 20:02:38 <servername>, httpd.init[3654]:  latest FAQ: https://github.com/lsof-org/lsof/blob/master/00FAQ
Mar 10 20:02:38 <servername>, httpd.init[3654]:  latest (non-formatted) man page: https://github.com/lsof-org/lsof/blob/master/Lsof.8
Mar 10 20:02:38 <servername>, httpd.init[3654]:  usage: [-?abhKlnNoOPRtUvVX] [+|-c c] [+|-d s] [+D D] [+|-E] [+|-e s] [+|-f[gG]]
Mar 10 20:02:38 <servername>, httpd.init[3654]:  [-F [f]] [-g [s]] [-i [i]] [+|-L [l]] [+m [m]] [+|-M] [-o [o]] [-p s]
Mar 10 20:02:38 <servername>, httpd.init[3654]:  [+|-r [t]] [-s [p:s]] [-S [t]] [-T [t]] [-u s] [+|-w] [-x [fl]] [--] [names]
Mar 10 20:02:38 <servername>, httpd.init[3654]: Use the ``-h'' option to get more help information.
Mar 10 20:02:38 <servername>, systemd[1]: httpd.service: Got notification message from PID 3682, but reception only permitted for main PID which is currently not known
Mar 10 20:02:38 <servername>, httpd.init[3682]: AH00543: httpd: bad user name apache
Mar 10 20:02:38 <servername>, httpd.init[3539]: [FAILED]
Mar 10 20:02:38 <servername>, systemd[1]: httpd.service: Can't open PID file /run/httpd.pid (yet?) after start: No such file or directory
Mar 10 20:02:38 <servername>, systemd[1]: httpd.service: Failed with result 'protocol'.
Mar 10 20:02:38 <servername>, systemd[1]: Failed to start The Apache HTTP Server.

says my hostname of the server :slight_smile:

upcp -sbf

FAILED - RETRYING: CentOS 8+ package management (2 retries left).
FAILED - RETRYING: CentOS 8+ package management (1 retries left).

TASK [common/update-config : CentOS 8+ package management] **********************************************************************************************************************************
fatal: [localhost]: FAILED! => {"attempts": 2, "changed": true, "cmd": ["dnf", "remove", "-y", "--duplicates", "--setopt=protected_packages="], "delta": "0:00:01.210679", "end": "2024-03-10 20:20:02.315117", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2024-03-10 20:20:01.104438", "stderr": "Unable to detect release version (use '--releasever' to specify release version)\nErrors during downloading metadata for repository 'apnscp':\n  - Status code: 404 for https://yum.apnscp.com/os/$releasever/x86_64/repodata/repomd.xml (IP: 45.32.193.57)\nError: Failed to download metadata for repo 'apnscp': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", "stderr_lines": ["Unable to detect release version (use '--releasever' to specify release version)", "Errors during downloading metadata for repository 'apnscp':", "  - Status code: 404 for https://yum.apnscp.com/os/$releasever/x86_64/repodata/repomd.xml (IP: 45.32.193.57)", "Error: Failed to download metadata for repo 'apnscp': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried"], "stdout": "ApisCP Packages for CentOS Linux $releasever -  250  B/s | 239  B     00:00    ", "stdout_lines": ["ApisCP Packages for CentOS Linux $releasever -  250  B/s | 239  B     00:00    "]}

PLAY RECAP **********************************************************************************************************************************************************************************
localhost                  : ok=34   changed=0    unreachable=0    failed=1    skipped=19   rescued=0    ignored=0