ARTIFACTORY: Deep dive into Artifactory Database connections and Http Max thread
In this article, we will talk about the Artifactory Database and Max thread configuration in detail.
For starters, I would suggest you go over the following article. These articles include the process of tuning the Artifactory.
Artifactory version 7x -
https://jfrog.com/help/r/how-do-i-tune-artifactory-for-heavy-loads/how-do-i-tune-artifactory-for-heavy-loads
Tomcat HTTP connection pool
Tomcat manages an HTTP connection pool for the incoming requests to Artifactory via port 8081.
It is configured in the server.xml [$JFROG_HOME/artifactory/app/artifactory/tomcat/conf] as "maxThreads" in the configuration file and the default is 200 for the 8081 connector.
To monitor this you you can integrate any monitoring tool with JMX, and to monitor the currentThreadsBusy in relation to the maxThreads.
Therefore as mentioned above, the HTTP thread pool for the 8081 connector, in simple words is how much concurrent connections Tomcat will forward to Artifactory when accessed via port 8081.
The same configuration and monitoring can be used for the 8040 connector, which usually would be used only by Artifactory when interacting with the Access service.
We usually recommend that the maxThreads for the 8040 connector will be 1/4 when compared to the 8081 maxThreads.
Artifactory async Thread Pool
The above stands for all synchronised requests that are coming via the 8081 port.
In addition, Artifactory also manages his own "async" thread pool. By default, it will be 4 * CPU cores.
The Async thread pool will be used for any internal "async" action performed by Artifactory.
Among these actions are: replications, metadata calculations, export/import, backup, GC, etc.
This async thread pool can be monitored using JMX Mbeans that Artifactory exposes:
https://jfrog.com/help/r/jfrog-platform-administration-documentation/artifactory-jmx-mbeans
https://jfrog.com/help/r/how-to-enable-jmx-monitoring-in-artifactory-7-x/how-to-enable-jmx-monitoring-in-artifactory-7.x
Specifically by "ActiveCount" VS "CorePoolSize".
Database Connection
With regards to DB connections, the Database max open connection defines how many connections Artifactory is going to forward to the Database to handle various operations. There are 3 connection pools - Artifactory, Access, and Metadata.
Following are the operations handle with the connection pools by Artifactory, Access and Metadata services.
For Artifactory connection pool - Handling Upload/download operation
For Access - Security information from the Database. Users, group, permission, authentication etc.
For Metadata - Calculation and Indexing of the packages within the Artifactory.
You can monitor the DB connection pools via JMX as well. Both an HTTP-8081 thread and an async thread can consume a DB connection.
The total number of Database connections opened from the Artifactory level is calculated using the below Thumb rule.
Total # of connections = (number of nodes) * ((artifactory.database.maxOpenConnections * 2) + access.database.maxOpenConnections + metadata.database.maxOpenConnections) + 50;
And it is always advised that the total number of Database connections opened from the Artifactory level should align with the maximum number of connections allowed at the Database level.
HTTP Client
Coming to HTTP connection pools that Artifactory manages for outgoing HTTP requests.
Every remote repository in Artifactory has its own HTTP connection pool.
By default, the pool is for 50 connections and this can be modified globally (not per repository) by:
artifactory.http.client.max.total.connections = 50 artifactory.http.client.max.connections.per.route = 50
For each remote repository, this HTTP connection pool will be used for all outgoing requests to that repository.
If there is a pull replication configured for that remote repository, the "pull" request as part of the replication will be used from the same connection pool.
There are also other HTTP connection pools, such as for HA propagation, Xray connections client.
There are various monitoring tool available to configure with the Artifactory in order to monitor the Database and Artifactory performance. Below articles you may find helpful.
https://jfrog.com/help/r/jfrog-platform-administration-documentation/log-analytics