Jump to content

Search the Community

Showing results for tags 'cdn'.

  • Search By Tags

    Type tags separated by commas.
  • Search By Author

Content Type


Forums

  • General Discussion
    • Announcements
    • Business Discussion
    • Business Services
    • Sell and Buy
  • Themes & templates
    • KVS default theme
    • KVS paysite theme
  • Educational / support
    • Technical Support
    • FAQ
    • Educational Series

Find results in...

Find results that contain...


Date Created

  • Start

    End


Last Updated

  • Start

    End


Filter by number of...

Joined

  • Start

    End


Group


About Me

Found 2 results

  1. KVS serves and stores different types of static content, like video files, photo images, video screenshots, user avatars, theme design styles / scripts. For video files and album photos KVS supports multi-server architecture and this is completely another story described here. This topic describes how to move static to CDN or another server for the rest of static content. Theme design files These are typically files located under /static folder on your main server. In most cases you will need to move these design elements to CDN for faster load. You should do that manually following any guideline you have for CDN. Please note that you should also check with CDN guide / support about crossdomain security issues (for example fonts can be only loaded from another domain if they provide Access-Control-Allow-Origin "*" header and etc.) From KVS side you only need to adjust one option in /admin/include/setup.php file: $config['statics_url']="$config[project_url]"; By default it points to the project URL, you should change it to your CDN: $config['statics_url']="//cdn.domain.com"; It will force all design static files to be loaded from this domain. NOTE: If your project is running under HTTPS, you should use CDN which also supports HTTPS. In most cases CDN will cache files from your original server for some time (can be forever). If you do any changes in these files on your server, your site will still continue using old versions. You may need to reset your CDN cache so that it pulled new versions. Another option is to increment version of the modified file when it is linked from template (not sure if all CDNs support this). For example templates use this tag to include CSS style file: <link href="{{$config.statics_url}}/styles/all-responsive-white.css?v=5.0" rel="stylesheet" type="text/css"/> If this file is loaded from CDN and you modify it, you can increment its version so that CDN will update it from your server again. The actual numbers don't matter, it should be different from the old one: <link href="{{$config.statics_url}}/styles/all-responsive-white.css?v=5.0.1" rel="stylesheet" type="text/css"/> Video screenshots from CDN By default video screenshots are stored under this folder on your main server: /contents/videos_screenshots You can move them to CDN as well, however you should consider the problem of invalidation here. Screenshots can be changed from time to time in admin panel. For example if you create 20 screenshots per each video and then leave only 10 best screenshots, or if rotator automatically removes badly clickable screenshots, your CDN will still show the old ones incorrectly. There is no solution for this issue from KVS side, you should check with your CDN provider on which options are available. For example they can ping files from your server from time to time and check if they were changed or not. But if you don't commit any screenshots editing you will probably be fine with leaving this as it is, since in your case the files will unlikely be changed. In order to switch screenshots to CDN you should first configure CDN per their guidelines and then in /admin/include/setup.php file of your KVS installation you should modify this URL: $config['content_url_videos_screenshots']="$config[project_url]/contents/videos_screenshots"; Please make sure to modify the correct option, as there are some similarly spelling options around. Your change should be something like: $config['content_url_videos_screenshots']="//cdn.domain.com/contents/videos_screenshots"; NOTE: If your project is running under HTTPS, you should use CDN which also supports HTTPS. If you want to make screenshot loading even faster, you can define alternate URLs for screenshots via different subdomains. The reason is that browsers do have limits to the number of concurrent connections for a single server (~6 at the same time). If there are many screenshots on your page they will be loaded in portions, but you can make this happen in parallel by loading them from different subdomains. They should all be configured as aliases for the same server and should serve the same content. In order to do that there is another option in /admin/include/setup.php config file: $config['alt_urls_videos_screenshots']=array(); You can list several alternate URLs there in the following format (make sure you retain the correct syntax or you may brake the code): $config['alt_urls_videos_screenshots']=array("//cdn2.domain.com/contents/videos_screenshots", "//cdn3.domain.com/contents/videos_screenshots"); This will force KVS to randomly load screenshots from these 3 URLs: //cdn.domain.com/contents/videos_screenshots (configured as main URL) //cdn2.domain.com/contents/videos_screenshots (configured as alternate URL 1) //cdn3.domain.com/contents/videos_screenshots (configured as alternate URL 2) Video screenshots from another server but not CDN If you want to use another server for serving video screenshots, you have 2 options: Option 1. Use rsync to push updates in KVS /contents/videos_screenshots folder to another server. This should be configured by your server admins. In this case you need to modify screenshots URL (see Video screenshots from CDN section). However this approach is not recommended, because rsync works asynchronously with some delay, so the common problem is that for new videos the screenshots may not yet been synced to remote server. If you know the interval between each rsynс push, you can workaround this problem by configuring KVS to postpone new content for some time. You should then change this option in /admin/include/setup.php: $config['post_dates_offset']="0"; It indicates the number of minutes all new content will be delayed. For example if rsync works every 1 hour you can configure it to be 70 or 80 minutes, to make sure that screenshots of new videos have been pushed to remote server in the moment when video appears on your site. Option 2. The second option is to move the whole screenshots storage folder to a remote server using network filesystem (NFS mount). This can be a good solution if remote server is located with the same datacenter for stability reason. You should ask your admins to create writable NFS mount to the remote server and move all existing content from /contents/videos_screenshots there. Then you should modify these 2 options in /admin/include/setup.php file: $config['content_path_videos_screenshots']="$config[project_path]/contents/videos_screenshots"; ... there are some other options in between ... $config['content_url_videos_screenshots']="$config[project_url]/contents/videos_screenshots"; The first option is a filesystem path and the second one is URL. Your changes should look like this: $config['content_path_videos_screenshots']="/mnt/server2/contents/videos_screenshots"; // should be a real path of the mounted folder ... $config['content_url_videos_screenshots']="//server2.domain.com/contents/videos_screenshots"; NOTE: If your PHP is configured with open_basedir restriction you should also update it to include /mnt/server2 path, otherwise KVS will not be able to write new screenshots into the mounted folder. You can check if this folder is writable or not with KVS audit plugin. Using the 2nd approach is more preferred, as there won't be any delay in screenshots availability. But it needs stable connection between servers. Other static files Other static files are normally user avatars and images of categorization objects like categories, channels, models. You can use the same approach for them as for screenshots, either move them to CDN (see Video screenshots from CDN), or move to another server (see Video screenshots from another server but not CDN). The only difference is in what options you should change in /admin/include/setup.php. Here they are: $config['content_url_categories']="$config[project_url]/contents/categories"; // URL for categories $config['content_url_content_sources']="$config[project_url]/contents/content_sources"; // URL for content sources $config['content_url_models']="$config[project_url]/contents/models"; // URL for models $config['content_url_dvds']="$config[project_url]/contents/dvds"; // URL for channels / series / DVDs $config['content_url_posts']="$config[project_url]/contents/posts"; // URL for posts $config['content_url_avatars']="$config[project_url]/contents/avatars"; // URL for users $config['content_url_referers']="$config[project_url]/contents/referers"; // URL for referers, not used in most cases $config['content_url_other']="$config[project_url]/contents/other"; // URL for uncategorized static files By default these all point to your main server, you can switch them to CDN or another server in the same way.
  2. By default KVS comes with local storage servers for videos and albums (if ultimate package). When you add a new storage server the first thing you should ask yourself is whether you want it to be in a new server group or in an existing server group. The difference is described in details in this article: Storage system in KVS tube script Here is short summary. New server group: You want to extend storage space. You want new content to be copied to new server, while old content should remain on old server. Existing server group: You want to load balance content traffic between multiple servers. You want to migrate existing content to a new server. NOTE: content servers for videos and albums work in exactly same way, so we will only describe that for videos. How to migrate your existing content to a new server or CDN The most common scenario is when you already have local storage and want to migrate to a remote server or CDN. In order to do that you should do the following steps: Step 1. Add a new content server into the same storage group where your old server is located. Don't worry, the new server will be added in Disabled status, so it won't affect your site users. You can find more info about adding new storage servers below. Step 2. Start content sync process for the new storage server. This process can be started in servers list in KVS admin panel. It will automatically copy all content stored in the group to a new server. NOTE: sync process will create a background task and you will be able to see its progress in the list of background tasks. Based on how many content you have, this task may take days and even weeks and it will be interrupted by processing new videos if any. If you want to manually copy content files via FTP, you can do that and skip this step. Step 3. When sync process is over, if possible verify the size of content folders on both servers to make sure they have the same size. At the same time verify that KVS is not showing any errors for the new server. There will be errors until all content is synced, but there should not be any errors afterwards. Step 4. Enable new server in storage group and disable the old one. Now your users will see content from the new server. Step 5. When you are sure there are no issues, delete old server. NOTE: KVS will stop copying new videos to the old server only after you delete it from KVS. Please make sure you remember where video files are stored on filesystem before deleting old server from KVS, see step 6 for more details. Step 6. When deleting server from KVS in step 5, KVS won't physically delete any video files there. Deleting a storage server from KVS only removes it from KVS records and doesn't affect anything on filesystem. This is done for security reasons, otherwise you could accidentally remove storage server from KVS and delete all your videos files. So the only way to delete video files from the old server (e.g. free the disk space) is to do that manually from server side. We recommend you to wait for 5-7 days after deleting old storage server from KVS so that you are 100% sure that all your site is working without any issues and all videos are playing fine. When you are sure everything is OK, you are good to delete old video files from server HDD. How to add a new content server System requirements for a content server in KVS: PHP 7.1+ No specific PHP modules required Apache and / or Nginx (recommended for better performance and content protection) NOTE: Apache is NOT strictly required for remote content server, you can have Nginx + PHP. You can add a content server in admin panel in Settings -> Storage servers. For server URL you should specify any URL that you want your content to be served from. It is a common practice to use IPs and subdomains here, as well as subfolders. It is not recommended to store content in www root folder, as this may bring some problems in future. So here is the recommended URL patterns: http://123.124.125.126/videos http://storage1.domain.com/videos http://storage2.domain.com/videos NOTE: if your site works via HTTPS, all your storage servers should also use HTTPS. This is why it is better to use subdomains here, as SSL certificates can support wildcard (*) subdomains and you can install the same certificate on all storage servers. Streaming type is an important option, which indicates the type of your storage server. If this server is external CDN provider, then you should select CDN (CDN configuration for videos and albums is a topic of another article, will not be covered here). If your server doesn't have nginx installed, then you should choose HTTP 302 redirect. This option means that KVS will redirect to video files directly without any protection. Video files which are available via direct links can be easily hotlinked by other sites and someone can be stealing your traffic. This is the common issue with wordpress and other scripts, they do not allow protecting your video files. KVS keeps this option for specific cases, but its usage is not recommended. In most cases your storage server should have nginx installed and in this case you should select nginx streaming type. The next set of options are related to server connection. KVS uses these settings to copy files to remote servers. In most cases you will need to specify FTP connection details here. The most confusing option here can be FTP folder. In order to specify it correctly you should first login to your FTP using standard FTP client and see where you come into. It can be possible that your FTP is directly configured to the needed folder, but in most cases FTP will be configured into home folder of FTP user or www root folder. Since your content will be stored somewhere in subfolder (e.g. http://storage1.domain.com/videos), you should specify RELATIVE path to this subfolder in this field. For example if your FTP is configured into www root (http://storage1.domain.com), then you need to specify: videos If your FTP is configured into user's home folder, then you will need to specify more subfolders to come to /videos: public_html/videos or www/videos or even data/domain.com/public_html/videos NOTE: each storage or conversion server in KVS should have unique folder. Do not configure different servers into the same folder on filesystem, this may result in data loss. Sometimes if you specify wrong FTP folder KVS will show error that it is not able to validate connection. However in many cases there will be no error, and the files will be copied to the wrong place and will not be available to the outside. Then after you add some videos to this server KVS will show an error on it, indicating that video files are not working from this server. Control script URL field is not editable. This field is built automatically from the URL field you specified above. It indicates where you should put KVS control script on this server, so that it is integrated with KVS. Control script serves 2 things: it provides info about server (its load and disk space) and implements protection for your content. There is a link near by to download this script and you should copy it manually to the storage server so that it is available at the displayed Control script URL. So if your storage server has this URL: http://storage1.domain.com/videos Then you should copy control script here: http://storage1.domain.com/remote_control.php You can manually run this script via browser and make sure it is working and returns "connected" word. Until this happens, you won't be able to save this server. Finally, there is Time offset field. In order to implement content protection KVS generates links which expire after some time (usually after 1 or 2 hours). In order to have this working, it is required that timezones on both servers are synced. If you don't want to configure the same timezone on your remote server, then you need to specify time offset, e.g. how many hours your storage server's timezone is behind or ahead in comparison to main server. However due to different daylight saving concepts in US and Europe it can be possible that after some time this difference will be shifted 1 hour back or forth. So the best way to ensure that remote server will not have any problem is to configure the same timezone on main server and remote server. Then you will not need to specify any time offset. Here is an example configuration for a remote storage server: Finalizing protection for nginx After the server is added, open it for editing. You may see that in server settings there is a textarea with nginx config displayed (if your Streaming type is set to nginx): You should add this piece of code into nginx config of your storage server to make sure that protected files are not available via direct links. After modifying nginx config you should restart nginx. There is no common guide on where do you find your nginx config. It depends on your server panel vendor. Most server panels do allow restarting nginx from panel GUI, but do not allow modify nginx config directly. You should figure this out with your host support. But if you do not configure this, your video files will still be available via direct links and no protection will be possible. CDN storage When adding CDN storage you should use CDN option in Streaming type field. Then you also need to put your CDN control script into /admin/cdn folder (create this folder as it doesn't exist by default), specify its name in CDN control script field and specify your secret key in Streaming key field: CDN control script is a PHP script that should be implemented by your CDN provider. This script implements content protection logic by creating temporary links to video or image files. Moreover, this script is responsible for invalidating content on CDN servers when files are replaced (modified) or deleted. NOTE: invalidation process may be very important for a tube site. Consider you have DMCA claim that requires you to delete copyrighted content. If invalidation is not working correctly, your CDN will continue to stream video files that should return 404 instead. Thus you may experience troubles with providing proof of deletion. Each CDN provider has its own API for protection and invalidation, there are no standards in this field. Therefore KVS requires that each provider implements their API and provides their CDN control script to customers for use in KVS. If no such script is provided, it will still be possible to use this CDN in KVS with default CDN control script, but no content protection or invalidation will be implemented and you will be missing these important features. CDN providers: if you are willing to implement CDN control script for your CDN, just ask customer to send you default CDN control script that can be downloaded in admin panel. This script provides a basic skeleton with all the needed functions having empty or basic implementation. If you want to use default CDN control script: Download default CDN control script and upload it to /admin/cdn folder. Specify cdnapi.php in CDN control script field. Specify any text in Streaming key field as this value is not used by default control script. Troubleshooting issues KVS will validate basic issues when you try to save server settings in admin panel. After you add server, KVS will validate it on background every 5 minutes to check if it operates successfully. Even if initially you didn't have any errors, new errors may appear later due to various factors. First, FTP connection or remote server configuration may be changed externally and some functionality will simply stop working. Also when you add a server which doesn't have any videos yet, KVS is not able to check if videos are streamed correctly or not. Only after you add some videos, this check will happen on background. All issues found on background are immediately reported to KVS start page as they all are considered CRITICAL and may indicate that your remote server is not working correctly and users are not able to see videos or photos. You should make sure to fix them ASAP. Errors from background checks are not very detailed and just indicate some set of issues. In order to get more details you should try to save server settings in admin panel. NOTE: if server reports an error and you have already fixed it, please give it 5 minutes to do re-check and verify this error is fixed. Until that, server may still show this error. This only applies for background checks on existing servers. When you save server settings in admin panel, all issues are validated right a way without any delay. If saving server in admin panel still shows validation errors, these errors are not yet fixed. Here is the list of errors you can get when saving server settings: - This connection folder is already used by another server. This issue means that you are trying to configure this server into a folder, which is already used by another storage or conversion server. You should never do this, each storage or conversion server in KVS should be configured into a separate folder on filesystem. - Unable to connect to host : post. KVS is not able to connect to the specified FTP host or port. If the entered data is valid, most probably your FTP is protected by firewall and you should make sure that your main server's IP is whitelisted. Also sometimes your FTP may be configured for using non standard FTP port, you should check this as well. - Unable to login with credentials provided. Either FTP username or password is not valid. - Put / get / chmod / delete operations failed, insufficient permissions possible. When validating server connection, KVS will try to put a test file to the server and then delete it. You will get this error if any of the mentioned operations failed, which probably indicates wrong FTP folder or FTP configuration issue. Try to log in to FTP manually using desktop FTP client and try to copy any file into the specified folder. Can you do this manually? Also this error can happen if /tmp folder on your main KVS installation does not exist or is not writable. In order to check that go to Plugins -> Audit and run installation check. It will show error for tmp folder if anything is wrong. - Control script failure. As we mentioned in this article, for remote storage servers you need to manually copy control script to make it available under the displayed Control script URL. You will get this error if KVS is not able to connect to it. Try opening control script URL in browser, do you see "connected" word printed? If you don't see it, then most probably you didn't copy control script to the mentioned location, or copied it to the wrong FTP location, or there is an issue with running PHP scripts on your storage server. If script works in browser but you still see this error, it can indicate that your main server is not able to connect to this URL due to DNS or other routing issue on your main server. You can check that by using server command line and typing the following command with your control script URL: curl -I http://storage1.domain.com/remote_control.php - Remote server time is not synchronized with primary server time. As we said already, it is recommended that PHP timezones on both servers are synced. You can get rid of this error by specifying timezone offset either positive (e.g. 2) or negative (e.g. -2). This error will show timestamps on both servers to give you clue how many hours you should specify. - This script is configured with another secret key, please update its secret key. Each remote control script has secret key hardcoded in it for protection. In some cases, specifically if you use multiple KVS installations and for each of them configure the same remote storage server, you can get this error. If you already had remote_control.php script on your storage server, then most probably it is already used by other KVS installation. If you indeed have other KVS configured with the same storage server, better contact support to make sure you don't harm another installation. If you modify secret key in remote_control.php and it turns out that this file is also used by another KVS, it will stop working there. Troubleshooting content check found errors The final validation that happens for every storage server is verifying that random videos can be played. If this check fails, you will see content check found errors issue displayed for server. You can further open content test page to see test logs, which include HTTP headers logs. Here are possible reasons: Your server doesn't have nginx installed but in KVS settings you set its streaming type as nginx. In this is true, then you need to change server's streaming type option to 302 redirect. Your nginx is not configured correctly, you should ask your host support to make sure that nginx configuration printed in KVS server settings was correctly added to nginx config for this server domain. CDN API script is not working as expected or the issue is at CDN provider's end. If your server is CDN, please open content test log for this server and send HTTP headers log to your CDN provider to describe the issue. Your Apache is configured with MultiViews option enabled, which prevents KVS from serving content correctly. In this case content test HTTP headers log will show 404 error. The requested file is missing on your server and thus returns 404 error which you can see in content test HTTP headers log. In most cases this happens because your FTP configuration has been changed externally and suddenly FTP user's home folder was changed to another one. KVS started copying new files to another location and of cause these new files are no more available under the old URL, configured in KVS server settings. You should login with FTP manually and check if its location is the same as what you have specified in server URL. If the location is different, then you should check with your host support where the correct location is and ask them to move content files from this new location to the old location and re-configure FTP user back to have home folder at the old location. For example your URL is configured as http://storage1.domain.com/videos and FTP access was configured directly into this folder. As time passed, KVS put many content into this folder but then suddenly your host support changed FTP access directly to domains www folder, e.g. this one: http://storage1.domain.com. The new files are no more copied under /videos folder, but copied under / folder instead. These new files will no more be available under http://storage1.domain.com/videos URL and KVS will report content check error.
×
×
  • Create New...