Tech Support Posted December 1, 2017 Share Posted December 1, 2017 KVS admin panel supports all possible ways to add videos. All you need is to stick to the way which best suits your situation, whether you have existing content or not. There are different content types supported for videos (upload, embedding, hotlinking and other), you may also need to understand how they are different from each other in What video types are supported in KVS tube script and how they are different post. When using upload content type, it is also possible to upload files differently depending on whether you want them to be re-converted or not, please check Different ways to upload video files into KVS for more details. Adding videos manually This is the most obvious way to add videos. The form can be found in Videos -> Add video section of admin panel. You can specify all possible options there and send new video to conversion engine. You don't need to log in to website area as a specific user to add videos from this user - you can do that in admin panel by selecting the needed user in add video form. In Settings -> Content Settings section under Video adding / editing settings you can select some options that affect this form: the most valuable are default user and default status. Based on which video content type you want to add, KVS will show only fields that are related to the selected content type: Adding videos manually is very simple and straightforward process, but this won't work well if you need to add many videos. Therefore please check different mass importing options available in KVS. Mass importing videos from a text file or Excel spreadsheet This feature should be used in the following cases: You have a text file created by some other website or program with the needed list of videos to import (KVS export feature can also generate such text file to export videos from one KVS to another). You have a list of video files uploaded to your server and you want to import them into KVS with specifying all additional info. For accessing this functionality you should go to Videos -> Import videos. There you can copy-paste your text field into a data textarea, make sure to configure Field separator correctly (\t for copy-paste from Excel, semicolon (;) for CSV, or other separator if you are using plain text file). In most cases Line separator should not be changed. Then you need to select list of Fields in the exact order you have in your import data. IMPORTANT! Some CSV files may have separator at the end of every line, which actually means that there is one more empty field at the end of every line. In this case you should add one more additional field and select * Skip this field * option there. If you still have error indicating wrong number of fields in every single line, you should carefully check whether you've selected the correct separator and whether the number of fields matches the number of separators in the line + 1. The number of fields should be +1 all the time! Here are some examples. Single text file import with pipe (|) separator: Specifying data in Excel and then copy-paste: When using import functionality it is very important to understand which data you need to provide based on which video content type you are trying to import. Video upload: you should either provide video file URL in Source video file (with processing) field, or one / several Video file "XXX" (without processing) fields. Video hotlink: you should provide video file URL in Hosted file URL (hotlinking) field. Additionally you may specify Duration and Overview screenshots (sources comma-separated). If you don't provide duration or screenshots in import data, KVS will have to download the file you are hotlinking for detecting its duration and screenshots. This may significantly increase video processing time. Video embed: you should specify Embed code (embedded video) field. In addition if you also specify Hosted file URL (hotlinking) field, KVS will temporarily download file from this URL and will detect duration and screenshots automatically from it. Otherwise if you don't know video file URL or you don't want KVS to spend time on its download, you should specify Duration and Overview screenshots (sources comma-separated) fields. Pseudo video: you should specify Outgoing URL (pseudo video) field. In addition if you also specify Hosted file URL (hotlinking) field, KVS will temporarily download file from this URL and will detect duration and screenshots automatically from it. Otherwise if you don't know video file URL or you don't want KVS to spend time on its download, you should specify Duration and Overview screenshots (sources comma-separated) fields. Import feature supports the huge list of other options, but for novice users in most cases you don't need to touch them and better use everything by default. If you import many videos, you may need to look at post date randomization options, so that you can import videos to be published in some future or past period. For example you can import 1000 videos as they have been added one by one since 2010. Anyway, you can always do the same via massedit feature, so not something should be really important during import phase. After you submit the import, KVS will validate each single line and will show you summary so that you can confirm it. The validation may take much time depending on your input data. By default KVS validates all URLs to be valid, which may take some time. You can disable URL validation in import options. It is recommended to start with several lines first to make sure you understand the concept. When looking into import validation summary you will be able to see all errors and warnings. Warnings are not critical, they just indicate potential issues in your input data, which are not necessary real issues. Errors are critical and indicate the lines that can't be imported. If there are any lines without errors, you can still confirm import and lines which have errors will be simply skipped from being imported. If all lines have errors, you will not be able to confirm the import. After you confirm import summary, KVS will create a background task for import. You can see import tasks in Administration -> Imports log. There you can see data submitted with each import as well as import status and log. All imports are processed on background. You don't have to wait for anything here. Import tasks may have errors, even if validation didn't find any issues. There may be issues with downloading files, network issues and some other. IMPORTANT! If you submitted an import but you don't see any videos added, you may need to go to Administration -> Imports log and look at what is going on there. It may be that all import lines failed. Looking into import log will help you understand the issue. It may also be a case that your import is stuck due to lack of free space on your main server (since import in most cases download new files to your server there is a protection to check disk space, so that your HDD is not 100% filled). In this case import will automatically continue as soon as more free space is available. When you use this functionality really often, it may help to create an import pattern, so that you do not select the same options every time. You may define as much different import patterns as needed, then you just need to select the needed pattern and copy-paste import data. The rest of options will be defaulted from import pattern. Uploading video files via FTP and use FTP content uploader plugin If you already have a list of video files and want to quickly upload them into KVS, the easiest way is to use FTP content uploader plugin. You just need to create a folder somewhere under your www root and upload all your video files there. Then in FTP content uploader plugin you should specify full server path to that folder, or relative path under your www root (not starting with /, e.g. upload for http://domain.com/upload folder). Each video file in this folder will be treated as a standalone video and will be uploaded into KVS with just a few clicks: In advanced scenario here if you want to upload multiple formats for the same video as well as screenshots and / or description, you may need to create subfolders for each video. All files in a subfolder will be considered to belong to each video. The following rules will be applied: Subfolder name means title of a video. ZIP file means list of screenshots uploaded for a video. JPG file means main screenshot uploaded for a video. TXT file means description specified for a video. Video files in subfolder may be treated differently based on your selection in Video formats option: -- If you want to upload as a source file with processing, then the biggest video file will be chosen as a source file, other video files will be ignored. -- If you want to upload as format files without processing, the plugin will compare file names with video formats to find postfix matching. As a result, plugin will show you a summary of all detected files and after you confirm that, they will be uploaded as videos into KVS. It is recommended to enable delete option (if not a problem that files will be deleted), because this will use rename operation on filesystem, which works instantly in comparison to copy operation. Using this plugin will not allow you to specify other data, such as categories, tags and etc. It is expected that you want to create videos first and then you may manually edit them in admin panel, after they are imported. If you want to specify all this data at import time, you need to consider using import feature (2). Since your video files are already uploaded on your server, you just need to specify their URLs for the import to take them. At the same time you can also specify other data needed. Automating mass importing videos with a feed This feature should be used in the following cases: Import data is regularly updated and you want to automate importing the updates. You want to automatically sync content between 2 or more KVS projects. You have a software that uploads video files to your FTP and you want then to auto-import them (for example CCTV or webcams). To access this functionality go to Videos -> Add importing feed section. Feeds are very similar to mass import described in previous section, but the main purpose of a feed is to automate importing without your intervention. Every feed can be configured to query a text file from the given URL every given interval and add all new content that appears in this text file. Many options here are very similar to mass importing options: for plain text file format you need to specify field and line separators and select list of data columns. When a feed gets executed for the first time, it will add all available content from the given file if you don't specify any limit. Starting from the next execution feed will only add new content that appears in the text file. Here it is very important to understand how duplicates are being calculated. First of all you should specify Duplicates prefix - any unique string for this feed. Then if using CSV / plain text format you should also indicate the key field from this file - a field which is unique for every line. It can be video ID, or video URL or title - any unique field. Duplicates prefix is needed for cases when you need to separate content between different feeds. Imagine that you used video ID as a key field. If you import data from multiple providers by using multiple feeds, IDs may overlap there, e.g. video #1234 may exist at both provider 1 and provider 2. If KVS only used ID to identify duplicates, it would end up by considering video #1234 from provider 1 and video #1234 from provider 2 as duplicates, so one of these videos wouldn't get added. But if you specify Duplicates prefix unique for each feed - this will not be a problem anymore. Video #1234 from feed 1 will not be a duplicate of video #1234 from feed 2. At the same time, you may have several different feeds adding content from the same source (for example 1 feed is pulling all videos from Category 1 and another feed is pulling all videos from Category 2). Since these feeds pull content from the same source and the content may overlap, you should specify the same value for Duplicates prefix for both feeds. Because a single video #1234 may have both Category 1 and Category 2 and thus it will appear in both feeds. If you've specified different prefixes, this video will be added twice. But if prefixes are the same - videos #1234 in both feeds will be considered as duplicates. Grabbing videos from 3rd party sites KVS features extremely powerful tube grabbers plugin. With this plugin you can import tons of content, even if you don't have any. We will not speculate on whether it is ethic or not to grab content from 3rd party sites, you should decide this yourself. In order to use grabbers you first need to install and configure some. KVS provides a list of most popular tube grabbers and from time to time we expand it with new grabbers. There is also a default video grabber which can download videos from abstract tube sites. However specific grabbers are more powerful: They may support embed codes if source site provides them. They may support additional video data, such as categories, tags, models, rating, views, date, etc. They may support downloading video files in multiple qualities, so that you don't need to convert them on your server. Most of video grabbers require youtube-dl library to be installed on your server. If you don't have it, please ask host support to install it for you, otherwise it won't be possible to use many grabbers. After you install some grabbers, you should configure them. You should select Mode option to indicate whether you want this grabber to download, embed or add videos as pseudo videos. Then you also need to choose which data fields you want to grab; the set of fields is different for each grabber and is based on which data can be parsed from the grabbed site. Finally there are filtering options, that help you to maintain grabbed content quality. IMPORTANT! when you want to use grabbers to download videos to your server, like with any other methods you can decide whether you want the downloaded video files to be processed, or you want them to be saved without processing. Here you can use 2 different tactics based on whether your project has enough computing power or not: If your project doesn't have much computing power, it makes sense to download all files without processing. In order to configure this you should set Multiple value for Quality option and then based on the number of video files provided by grabber match your video formats: If your project has enough computing power, it may be faster for you to download only 1 single file from grabber and have other files created by KVS. This may work faster, because download from grabbers is typically slow due to speed throttling by the source site. In this case you should select the needed file under Quality option and assign it to the correct format; so that all other formats will be created by KVS:NOTE: in some cases if you want to process all files, even the downloaded one, you will need to set upload as Source file. One important issue you'll find when using grabbers is how to manage your categorization efficiently. Different sites may have many similar categories named differently, so that grabbers may create too many categories in your database. There are several workarounds to deal with this described in Categorization best practices article. There are 2 ways you can import videos from grabbers plugin: you can import videos manually or you can configure grabbers to ping video lists from time to time and add videos from these lists. When using grabbers for the first time we recommend running manual imports first to understand how grabbers work and whether the grabbed videos are in the way you want them to be added. In order to import videos manually you should go to Upload content using grabbers manually section in grabbers plugin. There you need to specify list of URLs you want to grab. In this list you can specify both individual videos (if you want to pick each video yourself) and also you can specify video list URLs or video RSS URLs to grab the whole lists. Here is where you can feel the real power of using site-specific grabbers - you can use your browser to navigate source site and find a page which lists the needed videos - this can be a page listing all top rated videos, or most viewed videos for the last week, or videos from a specific category, or videos from a specific user and etc. You can then copy-paste URL you see in browser address field and grabber will open that page and will add all videos from it. Many tube grabbers also support pagination mode, which means that grabber will be able not only to add all videos you see on that URL, but also to navigate to page #2, to page #3 and etc. of a paginated list and find all videos there as well. In order to enable that, you need to specify a number of videos you want to grab right after the URL separated by pipe symbol, e.g.: https://www.kvs-demo.com/top-rated/|1000 This command tells KVS grabber to import 1000 videos from this page. If this page is a paginated list and shows only 50 videos per page, and the grabber supports pagination, KVS will query 20 more pages from this list to add 1000 videos in total. Using this approach you can mass import 100 top rated videos from every category on the site by simply specifying URLs to category pages and number of videos you want to grab from them. If you do not specify a number after list URL, KVS will add only those videos that are displayed initially on this page, e.g. only 50 videos for this sample top videos list: https://www.kvs-demo.com/top-rated/ Finally, when you have imported sample list of videos and you are happy with them, you can move forward and configure auto-pilot. Auto-pilot is exactly the same as importing manually via specifying list URLs and counts. The only difference is that auto-pilot will do that automatically for you, e.g. it will query the specified pages every N hours as you configure and will check if there are any new videos there. Therefore if you want it to import new videos every time, you should make sure to provide list URLs, which are sorted by new videos first. If you recall our previous example the top.html will not work here, since top lists are rarely updated. Instead in auto-pilot settings you should specify URLs which are updated every N hours you plan to query for new videos, for example latest videos page: https://www.kvs-demo.com/latest-updates/|100 By setting the above URL to auto-pilot you will require KVS to grab new videos from this website every N hours. Then you simply do not need to upload any videos manually, because they will be added and added every N hours you configure (if the source site is updated of cause with the new videos). If you want to grab only from specific categories, or tags, or models or whatever, you will need to specify multiple URLs, each for the list you want to grab, e.g. like this for grabbing from 3 categories: https://www.kvs-demo.com/categories/rock-music/|100 https://www.kvs-demo.com/categories/house-music/|100 https://www.kvs-demo.com/categories/pop-music/|100 While you can also use RSS with grabbers, using the URLs gives you much more power than RSS. In most cases you can adjust sorting or even do some filtering in the URL, so that grabbers only import content you'd like to appear on your site. Here are some important highlights with using tube grabbers in KVS: Use specific grabbers designed for specific sites, they support more options. If no specific grabber is available for the site you want to parse, you can try using default video grabber if it suits you (this grabber only supports download mode). But this grabber won't support 100% sites, it can only grab from sites that do have video URLs clearly printed in HTML code. When downloading from other sites, your IP may be blacklisted for frequent downloads. KVS has no control on that other than letting you specify timeout to avoid frequent requests. Some grabbers support proxies that can be listed in their settings. Try importing manually first before you configure auto-pilot. If you want to grab initial set of content, grab it manually and use auto-pilot for pulling new videos. Use web URLs instead of RSS to download from video lists. Make sure videos in the list are filtered and sorted in the way you want to grab them. You can still use RSS if you want to. Default grabber only supports RSS lists. Grabbers may become broken from time to time if source sites gets updated. If you find this issue, please report to KVS support and we will update the broken grabber to be working again. With KVS you can create your own grabbers and use them on your projects. You can find more details in How to create custom tube video grabber for KVS article. Combining grabbers and mass import Grabbers plugin can import many videos from other sites, but sometimes you may need more flexible way to grab video files and at the same time specify your own title, description, categorization and so on. In this case you can combine grabbers functionality to download files and mass import flexibility to specify data. However in this case you need to manually select videos and specify their direct page URLs, grabbing multiple videos from lists will not be possible. In order to use grabbers in mass import you should specify video page URL in Video page URL (grabbers) field. Then you can also specify any additional data you need. The key point here is that your custom fields specified in import data will override fields from grabbers. For example you can specify title and categories, so these fields will be taken from import. All other data will be taken from grabber. Using this approach will take much of your time to prepare data, but you will also benefit from more accurate and unique SEO, which is best for your project. Quote Link to comment Share on other sites More sharing options...
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.