XRAY: What to do when you notice a Masterkey mismatch

The master key is set per product say ( Artifactory, Xray, etc..), and must be the same for all the nodes in the same service (when using HA).

During the Xray migration or fresh installation(s) yet times we might encounter an error with decryption of master key. In case of fresh installation, you can stop Xray, remove/truncate the current Xray DB, then start Xray again It will generate a fresh master.key.

We can follow the below steps to restore the master key in the DB when we don’t have a backup of the existing master key file.

You will find below entries in the logs indicating a mismatch of master.key during the procedure. 

[jfxr ] [ERROR] [a314de91e8330833] [bin_mgr_service:300           ] [main                ] Failed to decrypt password for binmgr default: Cipher text is not well formatted
 [jfxr ] [ERROR] [a314de91e8330833] [ha_service:119                ] [main                ] failed to register HA node--- at /go/src/jfrog.com/xray/internal/service/ha/ha_service.go:118 (haService.ValidateLicenseAndRegisterNodeForServer) ---
Caused by: master keys do not match

Steps to be followed: 

NOTE: Take a DB snapshot/backup before performing the following steps

1. Remove the master.key from the $XRAY_HOME/etc/security/master.key:

mv master.key master.key.bkp

*Note: this does not actually remove the file, but rather renames it so it can no longer be detected by Xray

2. Enter the PostgreSql db:
 Navigate to your postgresql installation location and execute the following command:

psql xraydb xray

3. To list all tables: \dt
 
4. Remove the master.key from the database: 

SELECT * FROM key_id; 
 DELETE FROM key_id;

5. Remove high availability documents(if applicable):

DELETE FROM high_availability WHERE version = 'xray_version'; 

6. Remove the binary manager indexed resources data from below table first(We should be adding them from UI once the instance is up):

SELECT * FROM binary_managers_indexed_resources;
DELETE FROM binary_managers_indexed_resources;

7. Remove the binary manager:

SELECT * FROM binary_managers;
DELETE FROM binary_managers where bin_mgr_id = 'your_instance_id'

8. Remove the old admin account (this table may or may not be empty):
#This is in case the users' password got encrypted, it'll regenerate with "password" as the default

Delete from users where name = 'admin'

9. If any of the password fields are encrypted in system.yaml , set them to their decrypted state/ plain text. They will get re-encrypted on restart. 
 
10. Restart Xray and everything should start working again.