Jump to content

Search the Community

Showing results for tags 'conversion'.

  • Search By Tags

    Type tags separated by commas.
  • Search By Author

Content Type


  • General Discussion
    • Announcements
    • Business Discussion
    • Business Services
    • Sell and Buy
  • Themes & templates
    • KVS default theme
    • KVS paysite theme
  • Educational / support
    • Technical Support
    • FAQ
    • Educational Series

Find results in...

Find results that contain...

Date Created

  • Start


Last Updated

  • Start


Filter by number of...


  • Start



About Me

Found 6 results

  1. Multiple conversion servers can be used only in KVS ultimate package. They will let you reduce CPU load on main server and make video conversion faster, but only to some point. It is a common misunderstanding to assume that the more conversion servers you add, the more videos can be converted for the same time. This is not always true. The overall conversion speed depends not only on the actual conversion (e.g. video encoding), but on many other factors as well that will be explained in this post. KVS conversion engine works in 3 phases, in some cases only 1 or 2 phases are executed and the rest are skipped. But if we speak of processing newly uploaded videos, this is always a 3-phase process: Main server: prepares conversion task for conversion server and uploads this task to conversion server Conversion server: executes task Main server: verifies and finalizes task execution Main server cron and conversion server cron are both triggered each 1 minute. It means that any new video will require at least 2 minutes before its conversion is finalized. Even if conversion itself takes couple of seconds, you will have to wait for 2 minutes due to cron limits. Conversion phase 1 When phase 1 is executed, in most cases this phase is very fast and all it needs to do is to copy task info and the uploaded video file to conversion server. This may take a while if your source files are huge and your conversion server is remote. The most important thing to understand here is that this phase is always processed in 1 thread. If you have 10 new videos scheduled and 10 free conversion servers, the main cron will have to copy each video to each conversion server 1 by 1 (not in parallel). If copying each video takes 1 minute, your conversion engine will be busy with spending 10 minutes to copy 10 tasks to conversion servers. In this case there is no big difference how many conversion servers you have, because each task takes the whole minute to simply copy it. Your video queue will not be processed faster than 1 * 60 * 24 = 1440 videos per day. Remote conversion server settings have an option to cope with this issue, which allows conversion server to download video source files from the main server instead of require main server to copy them to conversion server: This changes the whole picture. Main server will not need to spend 1 minute to copy video source files anymore, instead it will spend only 1 second to copy a small task file. Thus it can copy 10 tasks to 10 conversion servers in less than 10 seconds, so that for the whole minute it can copy up to 60 tasks. Then each conversion server will download source files from main server, but now this will happen in parallel. If you have only 2 remote conversion servers, then they both will download 2 video source files at the same time. But if you have 10 - they will download 10 in parallel. This is a huge speed increase. The only issue here is to make sure that your main server is able to serve so many files at the same time, you should also remember that main server serves your website for the end users and these parallel downloads can affect the overall speed of your main server. Conversion phase 2 During phase 2 conversion servers will download video source from main server (if this option enabled in their settings, see the previous section) and will do all the necessary conversion logic programmed. Here each conversion server works independently of everything else. If you have 10 conversion servers, they will do phase 2 for 10 tasks in parallel, so phase 2 can be easily scaled to any speed you want by just adding more conversion servers. It is a common practice to create multiple conversion servers (at least 2) for the same physical server. You can easily do that by simply using separate conversion folders, so you can create 2 or more conversion servers in KVS using the same physical server box, e.g.: Convertor - 1: /home/ftpuser/conversion1 Convertor - 2: /home/ftpuser/conversion2 Convertor - 3: /home/ftpuser/conversion3 Why does this make sense? The whole phase 2 conversion consists of multiple conversion steps, here are some examples of them: Downloading source file from the main server Using ffmpeg to convert video to another format or quality Using ffmpeg to take screenshots from video Using imagemagick to create thumbs for the screenshots Copying result video files to storage server(s) If you look at these steps, some of them are using much CPU and some of them are using much network to download or upload. Also when taking screenshots from video files ffmpeg is using less CPU in comparison to when converting, taking screenshots can result in some delays depending on source file and how easily it can be navigated based on keyframes. If you use only 1 conversion server for 1 physical server, there will be intervals when your CPU load is 0% or when your network load is 0%. This is a waste of resources. By creating at least 2 conversion servers for the same physical server you will increase its conversion speed and overall resources utilization will be close to 100%. But this doesn't mean that adding more conversion servers here will increase the speed to x2, or x3 - this is just a matter of +20% or +30%. Therefore we think that having 2 or max 3 conversion servers per 1 physical server is more than enough. Conversion phase 3 This final phase is executed back again on the main server. When conversion server finishes processing a task, main conversion engine will pull this task result from conversion server. All high-CPU operations have already been done on conversion server and this phase is just to pull result screenshots and result video files if needed. Copying files may take time, especially on low quality network connections. This phase is like phase 1 in term of time - it can take much time to copy big or many files and the number of conversion servers you have cannot speed this up. Similar to what we said in phase 1, phase 3 can also have optimized copying. Each conversion server has an option to enable it to copy video files to storage servers and thus main server will not have to do that: This will result in parallel video files copying and you can benefit here from having multiple conversion servers. There is one case however, when this option cannot be used - if you have a local storage server. The reason is that local storage server is defined as a directory path (e.g. /var/www/domain.com/contents/videos), and this directory makes sense for the main server only. Remote conversion servers cannot use this path, so they won't be able to copy result video files. In such cases KVS will automatically disable this option and will force all result video files to be copied via main server. Then you have no much sense to add many remote conversion servers, since they will probably have a lot of spare resources. Summary: what can affect conversion speed Here is a summary of factors that can affect conversion speed in different scenarios: Connection latency / network speed between main server and conversion servers - bad network conditions will reduce conversion speed, that's why remote conversion servers should be located ideally with the same datacenter to avoid network roundtrips. CPU and disk speed on conversion servers - converting big video files can utilize much CPU (when scaling, watermarking) and also disk subsystem. Very slow disks may result in CPU resources not fully utilized. RAM is not very significant, however we've seen some cases when ffmpeg or imagemagick was using more than 1GB of RAM. For a conversion server you should have at least 2GB of free RAM. If you use multiple conversion servers on the same physical server, then your physical server should have 2GB * number of conversion servers you plan there. In multiple remote conversion servers scenario if servers are allowed to download source files from main server, it is critically important that main server is able to serve all these files in parallel as well as it is able to serve your end users at the same time. Otherwise the benefit from using multiple conversion servers will be only in reducing CPU usage on the main server and not speeding up conversion process much. If you have any local storage server, which is used to host newly uploaded videos, then you will not get 100% benefit from using multiple remote conversion servers, as main server will have to pull all result video files from them in 1 thread, which will mean that your overall system conversion speed will be limited to the speed of pulling an average video file from a remote conversion server. You can still use 1 or 2 remote conversion servers for reducing main server's CPU usage. When uploading video files without conversion (e.g. you upload them not as source files, but as ready video formats, which is very popular when using tube scrapper plugin), CPU resources are mostly not being used. Thus in such scenarios the key component to speeding up new content processing is reducing network utilization. No much sense to use remote conversion servers, since no much CPU resources are being utilized. Instead remote conversion servers will increase network utilization and thus may reduce overall speed. If you are using local storage - you are fine with 1 local conversion server. But if you are using remote storage, you may need to add 1 more local conversion server (with another directory on filesystem), to parallel the process of copying video files to storage server(s) and thus to increase processing speed. How to analyze performance You can find conversion logs in Administration -> Background tasks log section: By looking into these stats you can have some understanding of what is the problem, if you have one. If most of the time is spent on creating video files, it may indicate that your CPU is quite slow. If much time is spent on copying or downloading files, it may indicate that your network connection is poor, or network is overloaded at some port (main, conversion or storage server), or disks are overloaded. Please pay attention to "Waiting for xxx to start phase Y" messages. They do not mean that conversion engine is stuck and "actively waiting", absolutely no - conversion engine may switch to another task during this period, which is the most likely scenario when you have many tasks. These waiting timings may be big enough, up to hours, but they are not considered when calculating "effective" time spent on task processing. Transcoding performance In most cases when you upload videos into KVS (not using embed codes or hotlinking 3rd-party video files) and your external storage / conversion servers do not have network bottlenecks, the major time is being spent on video transcoding. This is what background task log details names as "Creating format file". The transcoding is the process of decoding video file, resizing it to another size and compressing it back. This process uses a lot of CPU resources, and in some cases when your videos are in Full HD / 4K quality it may also utilize HDD due to big size of source and target files. Transcoding in KVS is typically used to create different resolutions of the same video, so that users can watch video in different qualities. Another reason of using video transcoding is to add watermark(s) on top of it. If transcoding takes much time in your setup and you are not happy with the number of videos that are being processed during each day, then there are few options to increase transcoding speed: 1) Add more external conversion servers so that more videos can be processed at once in parallel (possible with KVS Ultimate only). This is the most obvious, and the most expensive option. You need to buy some more server(s) for the time that you need to transcode your videos. 2) Use GPU-powered server and configure GPU conversion. GPUs are working faster on video transcoding due to various technical reasons, however the main issue with them is that you need vendor-specific ffmpeg codecs that will support transcoding on GPU. These codecs are never part of standard ffmpeg distributions (ffmpeg is the server library that does video transcoding), and they need to be built manually by your server support. Another problem is that you should change ffmpeg options in video format settings in KVS admin panel to get these video codecs working. 3) Avoid using transcoding and upload already prepared (transcoded) video files. This approach is typically used with grabbers and importing feeds, when the videos that you need to import are already transcoded into multiple qualities. All that KVS needs to do is to download them and put into storage system. No transcoding will happen in this case, so no CPU / GPU power is needed. However, your video files must be already present in all qualities you want to support, and also you will not be able to put any watermarks on them. You can find out more about uploading already transcoded video files in this article: If you are using grabbers, then check this article about speeding up grabbing: 4) Finally, if none of the above options works for you (you don't want to pay for conversion servers, or you are not using grabbers and don't have video transcoded in different video qualities), then the only way to avoid transcoding or reduce its time is to reduce the number of video qualities that your site offers. You can find them in Settings -> Video formats section of admin panel. By default KVS typically installs the following qualities: MP4 480p (the lowest and required) MP4 720p (for videos 720p+) MP4 1080p (for videos 1080p+) MP4 4K (for videos 4K+) For each of these video formats you can see the number of videos that have it. This can let you identify, if you win much time if you decide to stop using MP4 4K or MP4 1080p qualities for example. Are there many videos that have them in % to your videos that have MP4 480p? If yes, then you can increase transcoding speed in several times by simply disabling these formats. If most of your videos are 480p and 720p, then you won't win much from disabling them and have to do another choice: Delete 720p quality and keep all your videos in 480p. This would result in low visual quality and could become problem for your site in general. Actually 480p quality is too small for modern devices. Delete 720p quality and change 480p resize option to Do not resize (keep source size). This would result in each video will have the same quality as originally uploaded source file. No quality change will be possible, and all users will see video in the same quality. This could be good option, but you should keep in mind that if some of your source files are 1080p or 4k, they will be transcoded with the same size, and could take much disk space on your storage and utilize much network when somebody is watching them. So this option is good when you know for sure most of your source video files are of 720p / 480p quality; or you are OK with the fact that many videos will be served in 1080p+ quality from your servers. Also this option allows you completely avoid transcoding by uploading your source video not in Source file field, but in 480p format field (or better rename it to some other name to avoid confusion). Make 720p quality required (WARNING! this will create a set of tasks to create 720p for all videos that don't have it), and after it is created for all videos, delete 480p quality. So all your videos will be transcoded into 720p format only. If the source file is bigger than 720p (e.g. 1080p or 4K), its quality will be reduced to 720p; and if the source file is initially smaller - the quality will remain the same as source. This option is a good choice, when you want to serve decent video quality to your users, and you know that your source files have many of 1080p+, and you don't want to serve 1080p quality to all users.
  2. Multiple conversion servers can be used only in KVS ultimate package. They will let you reduce CPU load on main server and make video conversion faster, but only to some point. It is a common misunderstanding to assume that the more conversion servers you add, the more videos can be converted for the same time. This is not always true. The overall conversion speed depends not only on the actual conversion (e.g. video encoding), but on many other factors as well. Please read more about KVS conversion engine here. Adding new conversion server in KVS System requirements for a conversion server in KVS: PHP 7.1+ IonCube Loader 10.2+ (not required if your have open source code option with your license) PHP Client URL Library (curl) PHP FTP PHP Image Processing and GD PHP.ini disable_functions should not contain "exec" FFmpeg 1.0+ with libx264, libavfilter and AAC codec (libfaac, libfdk_aac or native ffmpeg aac codec) ImageMagick You can add a content server in admin panel in Settings -> Conversion servers. For Maximum tasks option it is recommended to keep it in 5-10 interval. This option doesn't mean how many tasks are executed in parallel; instead it affects how many tasks are uploaded to this conversion server. High values for this option may result on overall conversion performance degradation. For Optimize content copying options we provided complete explanation in this post. The next set of options are related to server connection. KVS uses these settings to copy tasks to conversion servers. In most cases you will need to specify FTP connection details here. The most confusing option here can be FTP folder. In order to specify it correctly you should first login to your FTP using standard FTP client and see where you come into. It can be possible that your FTP is directly configured to the needed folder, but in most cases FTP will be configured into home folder of FTP user or www root folder. NOTE: conversion folder doesn't require any HTTP access to it. Therefore it can be ANY folder on your remote server, but it should be unique and should belong to only 1 conversion server in KVS. Here is an example configuration for a remote conversion server: Configuring cron on remote conversion server NOTE: on conversion server with local connection you don't need to add any cron schedule for it, it will be automatically executed by KVS engine. After conversion server is created in KVS and its connection is validated, KVS will copy remote_cron.php file into the specified remote folder. This file is needed to perform remote conversion tasks. In order for it to start working, you should put it on cron every minute using the following command (please note that you should specify the valid filesystem path to remote_cron.php script): php /path/to/remote_cron.php > /dev/null 2>&1 Once configured, this script will create several files in the same folder, which will mean that this command works fine. KVS will change status for this conversion server from Initializing to Active within 5 minutes. After switching to Active status this server will be utilized by KVS conversion engine and will start processing conversion tasks. Troubleshooting issues KVS will validate basic issues when you try to save server settings in admin panel. After you add server, KVS will validate it on background every 5 minutes to check if it operates successfully. Even if initially you didn't have any errors, new errors may appear later due to various factors. First, FTP connection or remote server configuration may be changed externally and some functionality will simply stop working. All issues found on background are immediately reported to KVS start page as they all are considered CRITICAL and may indicate that your remote server is not working correctly and conversion may be blocked. You should make sure to fix them ASAP. Errors from background checks are not very detailed and just indicate some set of issues. In order to get more details you should try to save server settings in admin panel. NOTE: if server reports an error and you have already fixed it, please give it 5 minutes to do re-check and verify this error is fixed. Until that, server may still show this error. This only applies for background checks on existing servers. When you save server settings in admin panel, all issues are validated right a way without any delay. If saving server in admin panel still shows validation errors, these errors are not yet fixed. Here is the list of errors you can get when saving server settings: - This connection folder is already used by another server. This issue means that you are trying to configure this server into a folder, which is already used by another storage or conversion server. You should never do this, each storage or conversion server in KVS should be configured into a separate folder on filesystem. - Unable to connect to host : post. KVS is not able to connect to the specified FTP host or port. If the entered data is valid, most probably your FTP is protected by firewall and you should make sure that your main server's IP is whitelisted. Also sometimes your FTP may be configured for using non standard FTP port, you should check this as well. - Unable to login with credentials provided. Either FTP username or password is not valid. - Put / get / chmod / delete operations failed, insufficient permissions possible. When validating server connection, KVS will try to put a test file to the server and then delete it. You will get this error if any of the mentioned operations failed, which probably indicates wrong FTP folder or FTP configuration issue. Try to log in to FTP manually using desktop FTP client and try to copy any file into the specified folder. Can you do this manually? Also this error can happen if /tmp folder on your main KVS installation does not exist or is not writable. In order to check that go to Plugins -> Audit and run installation check. It will show error for tmp folder if anything is wrong. - No PHP extension for FTP was found. Your remote server's PHP doesn't have support for FTP module and therefore it is not possible to enable allow this server to copy content to storage servers option for this server. - No PHP extension for CURL was found. Your remote server's PHP doesn't have support for CURL module and therefore it is not possible to enable allow this server to pull source files from primary server option for this server. - Conversion script not configured / not working on this server. - Conversion script is not working. - Conversion script executed more than 15 minutes ago. These errors indicate that your remote server either doesn't have a cron task to execute remote_cron.php script every minute, or executing this script results in critical PHP error. Please check if you have cron task set. If you have it, then try executing this command manually from command line to see its output (please note that you should specify the valid filesystem path to remote_cron.php script): php /path/to/remote_cron.php In some cases if your remote server has another timezone than your main server, KVS may think that its remote_cron.php script is not executed, while it does. Open conversion server settings in KVS admin panel and specify its timezone offset under Configuration. Save conversion server settings and give it 5 minutes to re-check and remove this error. - Some libraries are not configured correctly on this server. This error indicates that KVS was not able to locate ffmpeg / imagemagick executables on your remote server. First of all you should make sure that these libraries are installed on the server. If not - please install them. If the libraries are installed, then most probably their binaries are not located using standard paths. Open conversion server settings in KVS admin panel and specify the correct paths for libraries that were not detected correctly under Configuration. Save conversion server settings and give it 5 minutes to re-check and remove this error.
  3. Hi, Could you guys give some suggestions of what kind of conversion servers and settings are working best for you? I'm currently using local but in the past I've tried servers with amd and i9s, with i9-9900k I was roughly able to encode 100 videos in 24 hours for 1080p, 720p and 480p with AMD Ryzen 5 I was able to do 40 or 50 in a day for same 3 formats -vcodec libx264 -movflags +faststart -acodec aac -strict -2 -ar 44100 -ab 128k -f mp4 is my settings for 1080p. Would love to hear what others are doing and if there's a way for me to optimize is further
  4. KVS features extremely powerful tube grabbers plugin (Plugins -> Grabbers). With this plugin you can import tons of content, even if you don't have any. We will not speculate on whether it is ethic or not to grab content from 3rd party sites, you should decide this yourself. For basic understanding on how to use grabbers plugin please check the corresponding section in this post: There are 3 different ways of how to import content from grabbers and as usual there is no perfect solution and each of them has its advantages and issues. Grabbing as embed codes This is the fastest way to fill your site with many content with few simple clicks. Install grabbers that support embed codes and set Mode of each grabber to Embed. Choose the other needed data you want to grab and submit lists where to grab videos from. Voila! Pros: The fastest way to import tons of content, as only screenshot will be download for each video. No conversion power is required, can be used in virtually any server. No storage and traffic payment is required, since no content is hosted within your server. Cons: You have no control on what is displayed inside embed code. The source site may opt to stop displaying embed codes, or to delete videos, or to redirect all visitors from embed codes to their site. You cannot control advertising that is displayed inside video player. But you can still use your advertising outside the player. No support for video previews. The whole idea of using embed codes is to avoid video files being downloaded to your server. Therefore there is nothing to create video previews from. See also: What video types are supported in KVS tube script and how they are different. Downloading videos as a single source file The idea here is that for each video KVS will download the file of the desired (typically maximum desired) quality, and then treat this file as a video source file to create all the needed formats from it (e.g. 720p, 480p, 1080p and video previews). In order to configure this you need to set grabber Mode to Download option and then choose which Quality you want to download as a source quality: So this option tells KVS that you want grabber to download 1080p file as a source file, and if a video doesn't have this quality - skip it. This will mean that only videos that have at least 1080p quality will be added by grabber. And if a video even has 4K quality, it will be added by grabber and KVS will download its 1080p file (not 4K file). This is normal, because you don't need to download 4K file, which has bigger size if your maximum video format is 1080p. Make sure you save your resources here. The most important fact here is that the downloaded file will be treated as Source file with processing. This will mean that all video files will be fully created by KVS, and you can even configure to cut intros from them and add your own watermark. Pros: You control your videos and player, you can show any advertising that you want inside the player. You do not depend on whether videos are removed from the source site or not. You can cut intro rendered by source site, you can put your watermark on top of theirs. Only 1 file needs to be downloaded, and everything else is created by KVS (but you still depend on the downloading speed for this file). Cons: Slower import speed than for embed codes, as you depend on the speed throttling imposed by source site. Huge conversion power will be required to create all video formats. Video files may have higher size after re-conversion, as KVS by default sets better compression quality then other sites. You may need to adjust compression quality, otherwise it may be a situation that the downloaded source file of 1080p quality is 100Mb, and KVS creates 720p quality (smaller resolution) that is 150Mb on disk - this makes absolutely no sense and should be checked and avoided. Downloading videos as multiple ready-made files This concept is different from the previous one in requiring KVS to download multiple video files of different qualities and uploading them as ready-made files without them being re-converted. In order to use it you need to set grabber Mode to Download and then choose Multiple Quality options similar to this: Configured like this KVS will download 480p, 720p and 1080p files (3 files in total) and will upload them into the corresponding video format slots. 240p and 2160p qualities will be skipped, even if source videos have them. The key fact here is that video files will be uploaded "as-is" without any conversion. This brings certain benefits and certain drawbacks. Pros: You still have full control on videos and player advertising in comparison to embed option. And you don't need any processing power for this, so virtually any server will allow this. Cons: Since KVS needs to download multiple files, the importing speed will be much slower. Since video files will be uploaded "as-is", you can't adjust them in any way, e.g. remove intros or do any watermarking. How grabber download works in KVS and how to make it faster Basically the last 2 scenarios are mostly different on what will be slower in your server environment: video conversion or downloading multiple files. Where download limit comes from? It is normal that source sites don't want to lose their network channel on your project downloading their videos, so they typically impose speed throttling, either per download or per IP. In the first case when you download one file, this file's download speed is limited to 100 mbit/s, when you download another file in parallel, its download speed is limited to 100 mbit/s as well, but the summary of your download speed is now 200 mbit/s. In the latter case no matter how many downloads your start from the same IP, the summary download speed will be limited to 100 mbit/s. So technically there are 2 ways of increasing download speed: Run downloads in parallel (will work if server only restricts speed per download). Run downloads in parallel + from different IPs (will work in all cases). In KVS each grabber creates a separate import task that can itself be executed in multiple threads. You can configure the number of threads per grabber either in grabber settings (when you enable auto-pilot), or for the whole import, when you import set of lists manually. So from the start if you grab videos via multiple grabbers, each grabber downloads its videos in a separate thread and thus you have multiple parallel downloads. You can see how this happens in Administration -> Background tasks -> filter by Status = In process: Here you can see 2 import tasks running in parallel. Each task comes from a separate grabber, each grabber has some download speed limit imposed by its source site, but the whole download speed of your system in increased. If you grab from 8 different sites, then the whole download speed will be ~8 times faster. Then you can also configure multiple threads per each import task, for example 2 threads. This will mean that each import task will be executed in KVS in 2 separate threads and will simply grab different videos from the same source site. This will definitely work to increase download speed for sites that do not have speed throttling by IP. But even for sites that limit download by IP, it may help in some cases. The download limit is configured at streaming servers, and theoretically when you try to download different videos from the same site you may reach different streaming servers, and downloading from 2 different streaming servers using the same IP will not trigger any IP limit, as every streaming server has its own IP limit. In practice if you import random videos from huge sites, you will most likely reach different streaming servers. So increasing the number of threads per grabber may also work to increase the overall download speed. However this comes with an increased risk of your server IP being blocked by source site, since the increased number of import threads will send an increased number of requests to their main site (not only to streaming servers). In order to reduce this risk we recommend to increase Timeout option available in each grabber settings. Timeout means the waiting interval for grabber to send next request to the source site. If your server sends 100s of requests within several minutes, its IP will be most likely banned, as no real user behaves like that. Therefore you should use some reasonable timeout time, which is set to 5 seconds by default, but should be definitely increased if you plan to grab videos 24/7. And each time you increase the number of threads per grabber, you also need to increase timeout so that the total number of requests within 1 hour or so is not increased. Here are simple calculations. Let's say a grabber downloads 1 video in 10:00 minutes. And you have timeout set to 30 seconds. So the whole time for grabber to import a video is 10:30. If you increase the number of threads per grabber to 2, then assuming that each thread will download video the same 10 minutes, the average download speed will be reduced to 5 minutes per video. In order to keep the number of requests same, you will need to increase timeout twofold from 30 to 60. This means that each thread will import 1 video every 11 minutes, but having 2 threads in parallel results in average time per video equals to 11:00/2 = 5:30. If you increase the number of threads to 4 and increase timeout x4 to 120 seconds, then it will be 12:00 minutes (the full time for each thread to process and download video) / 4 threads = 3:00 minutes per video in average. That's much better! And finally if you grab from 3 different sites in parallel you can increase your average importing speed to 1:00 minute per video, or 1440 videos per day. Even if downloading a single video takes 10:00 minutes. Of cause these calculations do not include some unexpected delays, when your site network is overloaded, or when source streaming server is overloaded and etc. The importing speed actually means how fast videos are parsed and downloaded into your site. You can follow importing speed by looking into the number of new video tasks in Administration -> Background tasks section. If the number of new video tasks is small and doesn't grow with time, then importing speed is slower than your conversion engine can consume. You may need to think about further increasing the number of threads per grabber, or adding more grabbers for other sites. If the number of new video tasks is constantly growing, this means that your conversion engine is overloaded and makes no sense to increase importing speed anymore. In the latter case you will need to understand how KVS conversion system works.
  5. Hi, I am helping my client build new website. KVS is truly powerful and build for website owners. Everything is fine except larger files. Its fine on files less than 2-3GB. However if files large than 5GB or more. Most likely fail when process. Is that any specific configured to prevent large files transfer and convert? Common error from log : "ERROR ....Download failed: only 1083907113 bytes of 6625368994 was downloaded" Current configure: PHP upload and input limit: 10GB Memory limit : 2048M Max input and execution time : 1200 (both on main and conversion servers) Thanks.
  6. KVS supports different content types for videos (uploading, embedding, hotlinking and other), here is full information about them: What video types are supported in KVS tube script and how they are different. When using upload content type, you upload a source video file into KVS and then let KVS process it and store it using KVS storage system. During video processing phase KVS will create all required video formats defined in your settings. KVS will also try to create all conditionally-optional video formats, if the uploaded source file meets their size and duration limits. The most common example here is when you have multiple video formats defined for multiple qualities, e.g. 360p, 480p and 720p. In such scenario 360p should be configured as required format, 480p and 720p should be configured as conditionally-optional formats. So that 480p and 720p are only created for source video files that allow resizing them to 480p or 720p respectfully. For example if you upload source file in 720p - all 3 formats will be created. If you upload source file in 480p - only 360p and 480p will be created, 720p format will be skipped. For source files with 360p or less quality KVS will only create 360p format, because this is the only required format in this configuration. When using the described approach you don't need to think about source file original quality, all you need is to upload it as a source file with processing and KVS will then create all necessary formats. This processing will however take much time, because operation to create one video file from another uses your server resources very intensively. There can be scenarios when you will want to skip this processing: You don't need to have multiple video formats, you need to show video in only one quality. You already have video files in the needed format(s), for example you already prepared them using desktop software or you import existing video files from external sources. You don't need any resize or watermarking on video files. KVS provides ability to skip video processing for all or some files. In order to do that you should upload your video files not as source files, but directly into the needed video format(s). When a file is uploaded as a video format, KVS will not do any re-conversion with this file, the file will be saved in the same form as you've uploaded it. By doing this you can significantly decrease video processing time, but there are certain limitations you should be aware of: No watermarking will be applied to the uploaded file as well as no resize and duration trim. You should make sure that the uploaded file has valid format (e.g. if you upload it into MP4 format, the file should be a valid MP4 file, not any other video file). If you try to upload an invalid file, most probably there will be an error thrown during processing phase, however this depends on your ffmpeg version and it is also possible that there will be no error and you will end up with a video that is not playable via web player even if you can play it with your desktop. If you have multiple video formats, you can choose to upload them all and fully skip conversion, or upload only one of them and let KVS create the rest. Depending on which functionality you typically use to upload videos into KVS, they all provide ability to upload in both ways. Naming convention for this option is as following: Source file (with processing): means that the uploaded video file will be considered as a source for all video formats that will be created by KVS. Format "XXX" (without processing): means that the uploaded video file will be uploaded directly as the given format, so it won't be processed by KVS (other formats may be created from it if required). Manual video adding When adding videos manually you can see a field to upload as a source file (Source file) and separate fields to upload into each specific video format (e.g. Format "XXX"): Videos mass import In mass import for each field you can either select if it should be considered as a source file (Source video file (with processing)), or as a specific video format (Video file "XXX" (without processing)): Videos importing feeds In importing feed settings you can choose whether all video files from this feed should be considered as source files (Source file (with processing)), or as files of a specific format (Format "XXX" (without processing)). Importing feeds do not support uploading multiple files in multiple video formats: FTP content uploader plugin Using FTP content uploader plugin is quite tricky if you have multiple video formats and want to upload them as formats without processing. In order to do that you should name your video files based on postfixes you have in video formats. For example if you want to upload only 1 format per each video, you can still put all video files into 1 upload folder, but make sure that their file extension match the postfix of the video format you want it to be uploaded to, e.g. test.mp4 will match video format with .mp4 postfix. Also you should set Video formats option to detect postfixes: If you want to upload multiple video formats for each video, then you should put files for the same video into a subfolder under your upload folder. Each file should have filename based on video format postfix you want it to be uploaded to, e.g. test.mp4, test_480p.mp4 and test_720p.mp4 for formats with .mp4, _480p.mp4 and _720p.mp4 postfixes. Then on summary page you can check if all the formats have been detected as expected: Grabbers plugin In grabbers plugin if you want to download only 1 video file, you can simply select whether it should be considered as a source file or specific format file under upload as option: If the source site provides multiple video formats, you can download as many of them as needed and upload as specific video formats. In order to get these fields displayed you should switch Quality field to Multiple value: NOTE: downloading multiple files from grabber could be slowly due to bandwidth throttling on the source site. If your project's conversion power is high, it may be faster to download only 1 file and then create other formats via KVS. Summary Uploading as a source file with processing: Should be used by inexperienced users. You can upload any source file with any quality (480p, 720p and etc), KVS will automatically decide which formats to create. KVS will create all the needed formats with all their settings, e.g. watermarks, quality, duration trim. Works slow due to video conversion utilized. Uploading as a format file without processing: Should be used by experienced users if all restrictions are not a problem (you don't need watermark and etc. see below). Video file format and quality (480p, 720p and etc.) should exact match video format settings. No watermarking, quality change and duration trim will be done. Works incredibly faster.
  • Create New...