Reboot VCSA and press e in the photon boot menu (grub):

Add in the linux line at the end rw init=/bin/bash

Then go to /etc/pam.d directory.

Open vi and file system-password:

vi /etc/pam.d/system-password

Change rember=5 (or other count) to 0:

Now start passwd and enter your old root password and reboot the VCSA with

reboot