Issue Description When attempting to upload large files to an Artifactory repository, you might face a "No space left on device" error associated with the cache _pre directory. Below is an example of the error logs that could be encountered:
2024-11-23T04:32:19.563Z [jfrt ] [ERROR] [trace_id] [currentReadWhileWriteStream:36] [c41e3|art-exec-24284] - An error has occurred while creating stream, destination path:$JFROG_HOME/var/data/artifactory/cache/xx/<sha1>
java.io.FileNotFoundException: $JFROG_HOME/var/data/artifactory/cache/_pre/dbRecordxxxxxxxx-<trace_id>-<file_name>.bin (No space left on device)
at java.base/java.io.FileOutputStream.open0(Native Method)
......
......
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Issue Cause
If enable the cache-fs settings in the Artifactory Filestore, when uploading the files to Artifactory repos and the file size is greater than the cache size, the files will temporarily store in the cache temporary directory named "_pre", and the path of the "_pre" is like "$JFROG_Home/artifactory/var/data/artifactory/cache/_pre".
The example is like following:
- The cache-fs is enabled in the Artifactory Filestore and the cache dir is in the "/opt/jfrog/artifactory/var/data/artifactory/cache/".
- Upload a 5G file to Artifactory repo.
If the file to be uploaded is larger than the free space of the disk where the cache resides, the upload will be failed and the preceding error will be displayed in the log.
Resolution When the Artifactory version is 7.71.1 or over, the Artifactory cache-fs template supports maxCacheSize, maxFileSizeLimit and skipDuringUpload parameters.
- maxCacheSize: The maximum storage allocated for the cache in bytes.
- Please note that maxCacheSize does not include files that are in progress of being uploaded (which is saved under cache/_pre); thus it is recommended to keep extra spaces for _pre folder.
- If the uploaded file is larger than maxCacheSize, the file will not be temporarily stored in the _pre directory under the cache directory
- maxFileSizeLimit: The maximum limit in bytes for a binary to be saved in the cache-fs layer. For example, if maxFileSizeLimit = 1000000000 (1GB), then any binary bigger than 1GB will not be saved in the cache-fs.
- When the upload file size is greater than the maxFileSizeLimit, the upload file will not be stored in the _pre dir. You can set the maxFileSizeLimit to limit the size of files stored in cache-fs to solve this problem.
- The parameter maxFileSizeLimit must be set to a value lower than maxCacheSize; otherwise, it serves no purpose.
- skipDuringUpload: If this flag is set to true, binaries will not be saved in the cache-fs during upload requests but will be saved in the cache-fs when downloading a binary.
- When this parameter is set, the uploaded files are not stored in the cache-fs _pre directory, which can solve this problem.
When Artifactory version is less than 7.71.1, no matter whether the size of the uploaded file is larger than the size set by cache-fs, the file will be stored in the cache-fs _pre temporarily. Advised to mount cache-fs to an independent disk and continuously monitor the cache-fs usage.