Jump to content

Importing video from Obj Storage into KVS transcoding etc

Mr. Pervect

Recommended Posts

I have all my footage uploaded to a cloudserver through FTP.
How am I suppose to tell KVS to import the individual MP4 scenes so I can "Add movie" from KVS with the right settings (Movietitle, model name, description etc).
If anyone could point me in the right direction that would be great!

Message from my webhost:
The latest version of KVS supports S3 directly out of the box, but since we have an older version here we have to do a workaround to make it work with obj storage.

On your server we have an application called SFTPGo installed. That app connects directly to your bucket, it opens an FTP port to connect to it and translates the FTP input to S3 calls.

Tell them that you need to upload a lot of files directly to Obj Storage, and you don't want to use FTP because it would take too much time, and after they are uploaded you need to know how can you add them to KVS so the videos can be published.

That should help them understand this setup and offer a solution to it.

Link to comment
Share on other sites

It is not possible to upload videos to KVS directory from some S3 storage, there is no such function.

The only way would be to use hotlink content type (e.g. hotlink the video files directly uploaded into your storage). But in this case KVS will not be responsible for these files, for example it won't be possible to move them, or delete when video is deleted in KVS. Hotlinked videos are videos that are stored somewhere else.

This article summarizes about hotlink:


Link to comment
Share on other sites

You can use import to import all the data you uploaded, and in import data you can specify different video formats (e.g. full files, trailers, previews) so that KVS will download them all and upload into its storage. But you should understand that KVS will duplicate these files and will upload them into it own storage structure, as you have defined in Settings -> Storage servers. If you have some S3 storage defined there, then KVS will upload the files into that S3 storage.

But as we understood from your initial post, your S3 provider doesn't support standard S3 API, so it can't be added into KVS as a storage server.

Link to comment
Share on other sites

This is what my webhost told me.
Would you advice this? I should be able to add this through the "Hosted file" section in add video right?
And yes, in Storage server I see "CDN / Object Storage Videos"
I am only missing the : 1. Add trailer preview field 2. Alternative resolutions etc when I choose to add a video through this field.



I created a public CDN zone. All the videos in the (name-com) bucket are accessible from there.
For example, the video you uploaded to the (name-com) bucket at contents/videos/0/39/VIDEONAME_FULLSCENE.mp4 is available here

If URLs will not work for you, we can temporarily mount the bucket as some local folder on the server. Just let me know where you expect it to be mounted.



Edited by Mr. Pervect
Link to comment
Share on other sites

Please see another message from my webhost.
So If I understand correctly now I can make a list and upload all scenes/trailers/mouseover video's under VIDEOS > Import video's.
Then KVS will import and process the video's (Create alternative resolutions etc 1080/720) to our S3 storage.
When this is done I can add all relevant information (Model name/tags/publish date etc.)

Thanks for your help!



It looks like there was some sort of confusion. The video storage server is configured to store the files on S3 at the moment. You can check the settings.

The KVS doesn't support S3 directly. That's why we installed and configured the SFTPgo account on the server. It works like an FTP-to-S3 proxy. KVS connects to the SFTPgo account by FTP and SFTPgo proxy these uploads to the sitename-com bucket.


Link to comment
Share on other sites

3 hours ago, Mr. Pervect said:

The KVS doesn't support S3 directly. That's why we installed and configured the SFTPgo account on the server. It works like an FTP-to-S3 proxy. KVS connects to the SFTPgo account by FTP and SFTPgo proxy these uploads to the sitename-com bucket.

KVS supports S3 now starting from 6.1.0 version. But again I'm not sure we are talking about the same things here, because when we say KVS supports S3, this means KVS can create its OWN storage structure on S3 servers. But what you have is some video files that are ALREADY stored in S3, they are not connected to KVS in any way, they are just some 3rd-party files stored somewhere. And for KVS it doesn't matter where they are stored: either on your local disk, or local server, or some other server. KVS will either download and process them (and potentially push into its own CDN S3 structure), or will just link from where they are located right now. And this kind of "linking" is named hotlink in KVS terminology.

Link to comment
Share on other sites

I was misinformed by my webhost. or a miscommunication.
I have 500GB worth of content on the S3 server I wanted to import into KVS.
When I tried all scenes it got cancelled because of storage issues.
But when I did 1 everything went great.
I have 2 CDN zones figured, 1 pubic, 1 private.

But there are 2 problems:
- 1: I get warning from KVS saying "Video source files are not protected from direct access." this is because apparently the links are public.
- 2: Host didn't understand that by importing from the url KVS would not take those video's, re-encode them and upload them into another directory.


Message rom host:
When you first mentioned that you importing the videos with the URL (e.g.: https://dgeg544gee.ssl.server.net/TEST.mp4) i thought KVS was taking those videos, re-encoding them, and uploading them to another directory. Since it's not doing that then that option to import videos won't work here.What option did the KVS team originally recommended to take those videos and upload them?

Please advice the best way I can batch import my 24 scenes.
They are currently stored on an S3 server as my VPS has only like 250GB.


Edited by Mr. Pervect
made a mistake
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...