ARTIFACTORY: Integrating JFrog Artifactory Logs & Metrics with Splunk Using Fluentd (Docker)

Introduction 

This guide outlines the steps to integrate JFrog Artifactory with Splunk for centralized log and metric collection using a Fluentd sidecar container (docker installation). The setup pulls logs from Artifactory and pushes them to Splunk via the HTTP Event Collector.

Prerequisites

Before starting, ensure the following:

• A running JFrog Artifactory instance
• Docker installed on the host
• Network connectivity between Fluentd and the Splunk HEC endpoint

Install splunk

1. Create a Splunk account. If you don’t already have a Splunk account, sign up at Splunk.com.
2. Download Splunk (I have done it for Debian-based systems)

   wget -O splunk-9.4.3-237ebbd22314-linux-amd64.deb "https://download.splunk.com/products/splunk/releases/9.4.3/linux/splunk-9.4.3-237ebbd22314-linux-amd64.deb"

3. Install Splunk : Below command installs the Splunk .deb package and then resolves any missing dependencies if the installation was incomplete.

   sudo dpkg -i splunk-*.deb
   sudo apt-get --fix-broken install   # only if dependencies are missing

4. Start Splunk for the first time by accepting the license.

   sudo /opt/splunk/bin/splunk start --accept-license

5. Now you should be able to access Splunk Web Interface : http://<host>:8000

Configure Splunk HEC Tokens & Indexes 

1. Create Indexes 
Before setting up HTTP Event Collector (HEC) tokens, create dedicated indexes for logs and metrics:
Navigate to: Settings → Indexes → New Index and Create two indexes.Refer here for more information on creating indexes. 

2. Create HEC tokens
Configure HTTP Event Collector (HEC) tokens by navigating to Settings → Data Inputs → HTTP Event Collector and selecting the indexes you created. Click here for more information.

Prepare the Docker Setup Directory

1. Create a working directory containing:

• The Dockerfile for Fluentd
• A docker.env file holding environment variables needed for runtime

2. Build the Fluentd Docker Image 

docker build --build-arg SOURCE="JFRT" --build-arg TARGET="SPLUNK" -t jfrog/fluentd-splunk-rt . 

3. Configure Environment Variables in docker.env file.

4. Run the Fluentd Container.

docker run -it --name jfrog-fluentd-splunk-rt -v $JFROG_HOME/artifactory/var/log:/var/opt/jfrog/artifactory/log --env-file docker.env jfrog/fluentd-splunk-rt

5. Verify Data in Splunk.

index=<your_index> 

 
 
 
 


Conclusion

You have now successfully integrated JFrog Artifactory with Splunk using a Fluentd sidecar container in Docker. This integration allows for centralized and efficient collection of Artifactory logs and metrics within Splunk.