Workers: Creating your First Worker - A Step By Step Example

Workers is a JFrog Cloud Platform service that you can use to create and run plugins that 
extend JFrog Platform functionality via a built-in serverless execution environment. You can create Workers that are triggered when events occur in the JFrog Platform - similar to the behavior of AWS Lambda functions.

Let’s examine the workflow for creating, testing, and using a worker. For this example I want to prevent the upload of a file on Artifactory if the target path does not match a given pattern. In this case the pattern is defined with a regular expression
 

STEP 1 - Accessing Workers in your JFrog Instance

To find, edit, and create new workers you must be logged in as an admin user. Select the Administration tab. There you’ll see a new button called “Workers” as shown below:

 
Figure 1 - Access the worker service by selecting Administration tab >  Workers.

Once in the Workers section, you will see a set of cards representing 3 types of Workers:

• Event Driven Worker - triggered when the event you have selected happens in the system.
• Scheduled Worker - triggered at predefined times or intervals, relying on a Cron expression to define its schedule.
• HTTP-Triggered Worker - triggered manually using REST API or JFrog CLI.

The workflow to use Workers may remind you of the Webhook service, but there are a few differences between Events and Workers:

• A worker is a script hosted on the JFrog platform
• The script is provided with support tools to make workers easy to use
• The service that launches an action that triggers a worker will wait for the worker's completion, so a worker can perform activities on the platform before the action actually occurs and is thus able to impact the result of the action.

The worker flow is summarized in the following figure:



Figure 2 - Workflow of a worker service

STEP 2 - Choose the Event that Triggers the Worker

We want to create a worker that checks an upload path before a file is uploaded. 

Start creating an Event Driven Worker by clicking on the corresponding button (see Figure 3)

 
Figure 3 - Start creating an “Event Driven Worker” from the empty state (left) or from a populated state (right).

We see that there is a worker type that is triggered before the file-upload action, named Before upload.  So we will create a worker of this type by clicking “Add” inside the card dedicated to the upload action:

Figure 4 - Create a new Worker that reacts to an upload.

STEP 3 - Write the Action

After you click Add, you reach the edit page, where you can enter typescript code to define how the worker will behave.

 
Figure 5 - The edit page of a worker.

The code editor contains a small working sample script that is a template for a worker’s script. 
This sample pings Artifactory and returns a status according to the HTTP response.

The worker service is expecting you to export as default an anonymous function that takes two inputs: context and data.
 

For more information see the typescript API documentation.


Figure 6 - The data structure for the request payload the worker will receive.

With the above script, the worker would return an object with the UploadStatus that would tell Artifactory whether to accept an upload or not. 

For example,the following script implements a worker that accepts uploads only if the artifact layout matches a given pattern:
 

export default async (context: PlatformContext, data: BeforeUploadRequest): Promise<BeforeUploadResponse> => {
    // This RegExp will match all repopaths that start with 'org/company/' and end with the extension .jar OR .war
    // For instance those paths will match the regex :
    // - org/company/src/app.jar
    // - org/company/package1/subPackage/webapp.war
    const authorizedPathRegEx = /^org\/company\/(?:\w+.\/)+[\w\-\.]+\.(?:jar|war)$/;
    let status: UploadStatus = UploadStatus.UPLOAD_UNSPECIFIED;
    let message = "";


    try {
        if (authorizedPathRegEx.exec(data.metadata.repoPath.path)) {
            status = UploadStatus.UPLOAD_PROCEED;
            message = `RepoPath '${data.metadata.repoPath.path}' is acceptable for the repository '${data.metadata.repoPath.key}'`;
        } else {
            status = UploadStatus.UPLOAD_STOP; 
            message = `RepoPath '${data.metadata.repoPath.path}' does not match the regex ${authorizedPathRegEx} for the repository '${data.metadata.repoPath.key}'`;
        }
    } catch(error) {
        status = UploadStatus.UPLOAD_WARN;
        console.error(`could not check the path: ${error}`);
        message = `An error occurred during the check. Proceed with warning.`;
    }


    return {
        status,
        message,
        modifiedRepoPath: data.metadata.repoPath
    }
}

STEP 4 - Test the Action

After pasting in the code for the worker, click “Run". The worker runs and you can see the result in the Execution Results pane in the bottom right hand side of your screen:


Figure 7 - Result of the first execution.

You can see that the script returned an object containing:

• status: The script determines the value for this property. In this result,  2 corresponds to the enum property UploadStatus.UPLOAD_STOP
• message: The script determines this property also. We see that the  repo path does not match the regex value.
• modifiedRepoPath: This property is set with the value of data.metadata.repoPath that we received from the payload as we want to keep the original value.
• executionStatus: The worker service determines this value. The value tells you that the script was successfully  executed.

Now before we  enable our worker, we want to test it a bit more, to ensure that once enabled we will not block all uploads by mistake. According to the regex we should be able to upload a file with the following repoPath value: “org/company/package1/subPackage/webapp.war”. To simulate this upload request we can use the data structure in the upper right area and modify its path property as shown in the figure below:



Figure 8 - Mock of the payload for testing.

If we press the “Run” button again we will see that the status value has changed to 1 (which equals UploadStatus.UPLOAD_PROCEED) and the message tells us that the repoPath value is acceptable and the worker can be executed:



Figure 9 - Result of the second execution, this time we provide a value to data.metadata.repoPath.path (see figure 8 above).

So far so good!
Let’s step forward by giving a name to our worker as in the screenshot:

 
Figure 10 - Setting the name of the worker.

Then click on “Save” and “Close”. The Workers page should now show your new configured worker :

 
Figure 11 - Display of the list of configured workers.

STEP 5 - Choose the Repositories

Currently, your Worker is configured but not enabled. The enable switch is is under the 'Enabled' column on the left.

If you hover over the switch button, you will see that you can’t enable it until you select at least one repository to associate with the worker. The worker is then triggered for the given repository or set of repositories.

Create a repository of type Maven and give it a name (for this example we used the name “maven”).

Click the worker's name and the wheel button up right configure your worker. 
From there you may add a description (optional) and add the repository you created by clicking the "plus" icon
 


Figure 12 - Adding new repositories that must trigger events for this worker.

To select the Maven repository navigate to the Select Filters window. Then in the Select Repositories tab, move the “maven” item to the right (see the figure below). After this every upload action to the repository “maven” will trigger this worker.

You can optionally use a path pattern to select the repository. To create a pattern, use the tab “Set patterns”. To fine tune your trigger condition for the selected repositories, use a pattern that matches the repository path (for more detail see the documentation). In this example, we are not using a filter as we want this worker to be triggered for any repository path.

To accept your settings, click Ok.

  
Figure 13 - Select the repository “maven” to trigger events for this worker.

STEP 6 - Enable the Worker

Enable your Worker by clicking Enable switch (1) and then click Save (2). Then close (3) the edit page.

 
Figure 14 - Enabling and saving your worker.

STEP 7 - Test the Worker End-to-End

Your Worker is enabled and you can test it in a real context.
Go to the Artifacts page and upload a jar file to the repository “maven” using the following information, which does not match the repository path that we defined for the worker:


Figure 15 - The artifact deployment popup with a configuration that uses a layout that does not match the pattern we defined in our worker.

ֿThe result is that the upload is canceled and an error message is generated:


Figure 16 - The artifact deployment popup showing an error message for the deployment.
  
Next, run a test using the following configuration. This time your upload should succeed:



Figure 17 - An example of a deployment configuration that matches the layout pattern we defined in our worker.

NOTE: Multiple uploads trigger multiple execution requests, one per file. If you send a JSON file and a JAR file at the same time, Artifactory will handle them as if they were independent uploads and it will result in only one file being uploaded. In this case, the deployment will fail because not all the files would have been downloaded successfully.



Figure 18 - Example of a multi-file deployment.



Figure 19 - The multi-file deployment result. It shows that the file with the “json” extension has not been deployed because it does not match the layout pattern we defined in our worker.

Conclusion

In this Article, we have demonstrated how to create and use a worker to more accurately administer your JFrog platform by providing conditions for uploading artifacts. But you could easily create many other workers for other actions and conditions.

JFrog workers provide the ability to easily add quality gates to your platform and give you the capability to control the JFrog platform operational flow. Note that given how powerful workers are, it is important to plan them carefully so that they are productive and carefully designed. If you add workers that would, for instance, block all downloads for even one second, you could easily imagine how long a command like `npm install` can take to complete. 

We recommend that you use the powerful worker functionality wisely and use it first on non-production repositories dedicated to tests. Take the opportunity to discover this new service, and feel free to get in touch with JFrog with feedback and any requests you might have for improvement. 

References

If you want to dig further into this new service, you can refer to the official documentation:
https://jfrog.com/help/r/jfrog-platform-administration-documentation/workers 

Here is the REST API Documentation:
https://jfrog.com/help/r/jfrog-rest-apis/workers-rest-apis  

We'll be adding more articles and examples as the service grows.