I currently have 1 remote conversion server that is working properly or seems to be at least that is using FTP. I have the local conversion server disabled as it is my web server as well. The remote conversion server is also a storage server and is treated like a CDN, but is running Nginx and seems to serve content fine. The problem seems to be that once I disabled my local conversion server it no longer is able to move files from the remote conversion server to the remote storage server (they are the same server) using FTP. I think it was all working fine when I was using NFS, but I switched to FTP because of the latency between data centers is around ~100ms and NFS was very slow. Should this setup work? Is it supposed to pull the video from the remote conversion server back to the local server then upload it again to the FTP server running on the remote storage server? I don't know what the problem is. I may look into switching to SSHFS instead of FTP. Any suggestions on making this work? All my background tasks are failing with the message "Failed to put XXXXX.mp4 file to storage server" and error code "04 - Storage server connection error".
Example of one of the many failed task logs:
[2023-11-04 04:03:11] INFO Starting conversion task 6470 for video 4784 [PH-C]
[2023-11-04 04:03:11] INFO Downloading source files [PH-C-1]
[2023-11-04 04:03:11] INFO ....Downloading source file from: https://somedomain/get_file/0/ef497997f62551dcadd20402e807c221/4000/4784/4784.tmp/
[2023-11-04 04:03:14] INFO ....Downloaded 67088176 bytes
[2023-11-04 04:03:14] INFO Pre-processing source file [PH-C-3]
[2023-11-04 04:03:14] INFO Creating video format "MP4 480p" [PH-C-5:MP4 480p]
[2023-11-04 04:03:14] INFO ....Creating video file [PH-C-5-2]
[2023-11-04 04:04:29] INFO ........Created video file 4784.mp4: 670 seconds, 67235407 bytes
[2023-11-04 04:04:29] INFO ....Moving metadata to the file beginning [PH-C-5-3]
[2023-11-04 04:04:29] INFO ........Skipped, already done by FFmpeg
[2023-11-04 04:04:29] INFO ....Copying video file to "Remote" [PH-C-5-4:Remote]
[2023-11-04 04:04:29] ERROR ........Failed to put 4784.mp4 file to storage server "Remote"
[2023-11-04 04:04:29] INFO Conversion task 6470 for video 4784 is finished in 78 sec [PH-CE]
[2023-11-04 04:04:29] INFO Removing 6481 task dir as it is being marked to be deleted
[2023-11-03 23:05:12] ERROR(4) Failed to put 4784.mp4 file to storage server "Remote" [PH-E]