ARTIFACTORY: Federated repositories are out of sync

In some cases, we may run into issues where federated member repositories are out of sync with each other. In this article, we'll explore some potential causes and solutions when you're faced with this issue. 

"The Federated repository settings are not synchronized"

When we're faced with this error message in the UI, it is easily confused with the federated repository data being out of sync. This refers to any configuration items that may have fallen out of sync between the members. This is usually fixed by performing "Push Configuration" in the repository configuration. 




Before we start anything, we want to check the status of federation sync. You can do this by running the Get Federation Sync State API call. 

One possible cause for federated repositories being out of sync is simply network failures. 

When we experience these network failures, it is possible that the events failed to get propagated and are in a hanging state. Starting with release 7.71.1, Artifactory now features an auto-healing mechanism that checks Federated repositories at regular intervals for exhausted queues (queues that have exceeded the maximum number of attempts to send events to other Federation members). This mechanism resets the failed events automatically and tries again to sync with the target mirror. Auto healing is for when federation is stuck in a hanging state which happens after a number of errors occur and retries are exhausted. 

The second best way to sync these repositories up again is to perform a Federated Repository Force Full Sync.

Another possible cause for federated repositories being out of sync is due to federation being turned off at some point and turned back on again. 

When federation is off and a delete action is performed in a federated repository, and then federation is turned back on again, the delete actions will not sync to the other members. Instead, if we try to perform full sync from the instance with no files deleted, federation will try to restore the deleted files from the instance with the non-deleted files. The solution to this is to run our replicationDiff script which will shine light on the delta files between the repositories. And from there, we can proceed by deleting the delta files from the other repository.