Server went down overnight push alert on mobile app - restart - process is not running

#1

Result of systemctl status apnscp.service

● apnscp.service - ApisCP backend service
Loaded: loaded (/etc/systemd/system/apnscp.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/apnscp.service.d
└─override.conf
Active: failed (Result: exit-code) since Fri 2022-03-04 09:53:32 MST; 1min 42s ago
Process: 5933 ExecStopPost=/usr/sbin/httpd -f /usr/local/apnscp/config/httpd.conf -k stop $HTTPD_ARGS (code=exited, status=0/SUCCESS)
Process: 5919 ExecStart=/usr/local/apnscp/bin/apnscpd start (code=exited, status=255)
Main PID: 5919 (code=exited, status=255)
Status: “Initializing”
Tasks: 4 (limit: 101027)
Memory: 74.5M
CGroup: /system.slice/apnscp.service
└─3396 redis-server 127.0.0.1:0

#2

Check /usr/local/apnscp/storage/logs/start.log for the shutdown reason.

#3

Log file is too big, a few of the errors from last night. Let me know if this does not help.

WARNING: constant(): Couldn’t find constant Redis::COMPRESSION_ZSTD
[/usr/local/apnscp/lib/config.php:287]

0B. Error_Reporter::handle_error(2, “constant(): Couldn’t find constant Redis:$
[n/a]
1B. constant(“Redis::COMPRESSION_ZSTD”)
[/usr/local/apnscp/lib/config.php:287]
2B. include(”/usr/local/apnscp/lib/config.php")
[/usr/local/apnscp/lib/constants.php:61]
3B. {closure}()
[/usr/local/apnscp/lib/constants.php:69]
4B. include("/usr/local/apnscp/lib/constants.php")
[/usr/local/apnscp/lib/apnscpcore.php:57]
5B. include("/usr/local/apnscp/lib/apnscpcore.php")
[/usr/local/apnscp/lib/lservice.php:25]

WARNING: session_set_save_handler(): Cannot change save handler when headers al$
[/usr/local/apnscp/lib/apnscpsession.php:146]
WARNING: session_set_save_handler(): Cannot change save handler when headers al$
[/usr/local/apnscp/lib/apnscpsession.php:146]

0B. Error_Reporter::handle_error(2, "session_set_save_handler(): Cannot change$
[n/a]
1B. session_set_save_handler(apnscpSession, false)
[/usr/local/apnscp/lib/apnscpsession.php:146]
2B. apnscpSession::init()
[/usr/local/apnscp/lib/Auth.php:40]
3B. Auth->__construct()
[/usr/local/apnscp/lib/Auth/CLI.php:25]
4B. Auth_CLI->__construct()
[/usr/local/apnscp/lib/Auth.php:94]
5B. Auth::import(“CLI”)
[/usr/local/apnscp/lib/Auth.php:68]
6B. Auth::autoload()
[/usr/local/apnscp/lib/apnscpfunction.php:68]
7B. apnscpFunctionInterceptor::init()
[/usr/local/apnscp/lib/lservicelib.pht:133]
8B. ListenerServiceCommon->__construct()
[/usr/local/apnscp/lib/lservicelib.pht:1375]
9B. ListenerServiceCommon::init()
[/usr/local/apnscp/lib/lservice.php:28]
[Fri Mar 04 07:04:42 2022] EXCEPTION: fatal(): invalid session `aa0b09cd4cbb061$
[/usr/local/apnscp/lib/error_reporter.php:655]

0B. Error_Reporter::get_debug_bt()
[/usr/local/apnscp/lib/error_reporter.php:271]
1B. Error_Reporter::handle_error(128, "fatal(): invalid session `aa0b09cd4cbb0$
[/usr/local/apnscp/lib/error_reporter.php:159]
2B. Error_Reporter::handle_exception(apnscpException)
[n/a]

EXCEPTION: fatal(): invalid session `aa0b09cd4cbb0617e173822f5078ab36’
[/usr/local/apnscp/lib/error_reporter.php:655]

#4

You’re behind on migrations. This was introduced in v3.2.30 and activated in v3.2.31.

cd /usr/local/apnscp
git checkout v3.2.30
./artisan migrate --force

Let me know what errors are reported during migration.

#5

No errors, server is running now. How often should I run migration?

#6

Run upcp to come up current to what the code is.

Migrations run automatically whenever upcp runs if there’s a change in resources/playbooks/migrations/. upcp runs every night by default (cpcmd scope:get cp.update-policy). Based on the dialog yesterday in Discord, you fast-forwarded code in accordance with the Dovecot hotfix, which reset code to the current release and bypassed migrations.