Fresh AlmaLinux 10.x fails on PostgreSQL 16 incorrect timezone format in .conf file

Bug reports
1

Here are some helpful considerations when filing a bug:

  • What version of apnscp are you running?
    cpcmd not found (Fresh install)

  • Which version of RHEL or CentOS are you using?
    AlmaLinux release 10.1 (Heliotrope Lion)

  • Can you reproduce this bug reliably?

  1. Fresh install on an AWS EC2 instance running AlmaLinux 10.x.
  2. curl https://raw.githubusercontent.com/apisnetworks/apiscp-bootstrapper/master/bootstrap.sh | bash
  3. Profit (or not…)
  • Enable debug mode (cpcmd config_set apnscp.debug true). Restart backend by hand,
    cd /usr/local/apnscp/bin ; ./bin/apnscpd -f restart

RUNNING HANDLER [common : Restart postgresql] ***************************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => changed=false
msg: |-
Unable to restart service postgresql: Job for postgresql-16.service failed because the control process exited with error code.
See “systemctl status postgresql-16.service” and “journalctl -xeu postgresql-16.service” for details.

PLAY RECAP **************************************************************************************************************************************************************************************************************************************************
localhost : ok=200 changed=2 unreachable=0 failed=1 skipped=142 rescued=0 ignored=0

[root@ip-172-31-17-132 playbooks]# sed -n ‘603p’ /var/lib/pgsql/16/data/postgresql.conf
log_timezone = …/…/…/etc/localtime
[root@ip-172-31-17-132 playbooks]# systemctl status postgresql-16
× postgresql-16.service - PostgreSQL 16 database server
Loaded: loaded (/usr/lib/systemd/system/postgresql-16.service; indirect; preset: disabled)
Drop-In: /etc/systemd/system/postgresql-16.service.d
└─override.conf
Active: failed (Result: exit-code) since Sat 2026-01-31 02:19:16 UTC; 1min 3s ago
Duration: 4min 43.535s
Invocation: 0931007c91fa48a99890880a30dd0b74
Docs: PostgreSQL: Documentation: 16: PostgreSQL 16.11 Documentation
Process: 15117 ExecStartPre=/usr/pgsql-16/bin/postgresql-16-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
Process: 15124 ExecStart=/usr/pgsql-16/bin/postgres -D ${PGDATA} (code=exited, status=1/FAILURE)
Main PID: 15124 (code=exited, status=1/FAILURE)
Mem peak: 2.3M
CPU: 18ms

Jan 31 02:19:16 ip-172-31-17-132.us-east-2.compute.internal systemd[1]: Starting postgresql-16.service - PostgreSQL 16 database server…
Jan 31 02:19:16 ip-172-31-17-132.us-east-2.compute.internal postgres[15124]: 2026-01-31 02:19:16.513 GMT [15124] LOG: syntax error in file “/var/lib/pgsql/16/data/postgresql.conf” line 603, near token “.”
Jan 31 02:19:16 ip-172-31-17-132.us-east-2.compute.internal postgres[15124]: 2026-01-31 02:19:16.514 GMT [15124] FATAL: configuration file “/var/lib/pgsql/16/data/postgresql.conf” contains errors
Jan 31 02:19:16 ip-172-31-17-132.us-east-2.compute.internal systemd[1]: postgresql-16.service: Main process exited, code=exited, status=1/FAILURE
Jan 31 02:19:16 ip-172-31-17-132.us-east-2.compute.internal systemd[1]: postgresql-16.service: Failed with result ‘exit-code’.
Jan 31 02:19:16 ip-172-31-17-132.us-east-2.compute.internal systemd[1]: Failed to start postgresql-16.service - PostgreSQL 16 database server.
[root@ip-172-31-17-132 playbooks]# cpcmd misc_cp_version
-bash: cpcmd: command not found
[root@ip-172-31-17-132 playbooks]# cat /etc/redhat-release
AlmaLinux release 10.1 (Heliotrope Lion)

If I change it to ‘UTC’ postgres starts fine, but then I run:

cd /usr/local/apnscp/resources/playbooks && env ANSIBLE_LOG_PATH=/root/apnscp-bootstrapper.log BOOTSTRAP_SH=/root/resume_apnscp_setup.sh ansible-playbook -l localhost -c local bootstrap.yml

and it changes it back and fails again.

2

It’s a bad EC2 image. I’ve received a report on this in the past.

3

Bandaid fix:

[root@ip-172-31-17-132 playbooks]# nano roles/pgsql/install/defaults/main.yml
log_timezone: “{{ lookup(‘pipe’, ‘timedatectl show -p Timezone --value’) }}”

Next error:

TASK [apnscp/initialize-filesystem-template : Copy /etc/localtime to /home/virtual/FILESYSTEMTEMPLATE/siteinfo] *********************************************************************************************************************************************
fatal: [localhost]: FAILED! => changed=false
cmd: /usr/bin/rsync --delay-updates -F --compress --archive --out-format=‘<>%i %n%L’ /etc/localtime /home/virtual/FILESYSTEMTEMPLATE/siteinfo//etc/localtime
msg: |-
rsync: [sender] link_stat “/etc/localtime” failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1338) [sender=3.4.1]
rc: 23

PLAY RECAP **************************************************************************************************************************************************************************************************************************************************
localhost : ok=1006 changed=290 unreachable=0 failed=1 skipped=358 rescued=0 ignored=12

fixed it with:

nano roles/apnscp/initialize-filesystem-template/tasks/oneshot-files.yml

Copy files once on provisioning and never again

  • name: “Check if destination file exists”
    stat:
    path: “{{ apnscp_filesystem_template }}/{{ item.service | default(‘siteinfo’) }}/{{ item.file }}”
    register: st

  • name: “Ensure destination directory exists”
    file:
    path: “{{ apnscp_filesystem_template }}/{{ item.service | default(‘siteinfo’) }}/{{ item.file | dirname }}”
    state: directory
    recurse: yes
    when: not st.stat.exists

  • name: “Check if source file exists”
    stat:
    path: “{{ (item.file == ‘/etc/localtime’) | ternary(‘/usr/share/zoneinfo/UTC’, item.file) }}”
    register: src_st

  • name: “Synchronize {{ item.file }}”
    synchronize:
    src: “{{ (item.file == ‘/etc/localtime’) | ternary(‘/usr/share/zoneinfo/UTC’, item.file) }}”
    dest: “{{ apnscp_filesystem_template }}/{{ item.service | default(‘siteinfo’) }}/{{ item.file }}”
    recursive: yes
    copy_links: “{{ item.copy_links | default(False) }}”
    notify: Reload filesystem template
    when:

    • not st.stat.exists
    • src_st.stat.exists
4

Next error:

TASK [system/nss : Remove systemd from FST nsswitch.conf] **************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => changed=false
msg: Path /home/virtual/FILESYSTEMTEMPLATE/siteinfo/etc/nsswitch.conf does not exist !
rc: 257

PLAY RECAP *************************************************************************************************************************************************************************************************************************************************
localhost : ok=851 changed=33 unreachable=0 failed=1 skipped=493 rescued=0 ignored=0

Going to try on RHEL 10, started this at 5:11PM EST, still wasting time on debugging this.

5

Good to know it wasnt me. X|

6

Build updated for systems without /etc/localtime, so it’ll build without incident going forward.