Importing video from Obj Storage into KVS transcoding etc

[Mr. Pervect]

I have all my footage uploaded to a cloudserver through FTP.
How am I suppose to tell KVS to import the individual MP4 scenes so I can "Add movie" from KVS with the right settings (Movietitle, model name, description etc).
If anyone could point me in the right direction that would be great!

Message from my webhost:
The latest version of KVS supports S3 directly out of the box, but since we have an older version here we have to do a workaround to make it work with obj storage.

On your server we have an application called SFTPGo installed. That app connects directly to your bucket, it opens an FTP port to connect to it and translates the FTP input to S3 calls.

Tell them that you need to upload a lot of files directly to Obj Storage, and you don't want to use FTP because it would take too much time, and after they are uploaded you need to know how can you add them to KVS so the videos can be published.

That should help them understand this setup and offer a solution to it.

[Tech Support]

It is not possible to upload videos to KVS directory from some S3 storage, there is no such function.

The only way would be to use hotlink content type (e.g. hotlink the video files directly uploaded into your storage). But in this case KVS will not be responsible for these files, for example it won't be possible to move them, or delete when video is deleted in KVS. Hotlinked videos are videos that are stored somewhere else.

This article summarizes about hotlink:

 

[Mr. Pervect]

Thanks for the speedy reply.
That's a shame because I just uploaded 500GB to the S3 storage.
What would be the best way to batch upload all the: Full Scenes, Trailers, Mouseover video's into KVS?

Thanks again, I love the software you've made!

[Tech Support]

You can use import to import all the data you uploaded, and in import data you can specify different video formats (e.g. full files, trailers, previews) so that KVS will download them all and upload into its storage. But you should understand that KVS will duplicate these files and will upload them into it own storage structure, as you have defined in Settings -> Storage servers. If you have some S3 storage defined there, then KVS will upload the files into that S3 storage.

But as we understood from your initial post, your S3 provider doesn't support standard S3 API, so it can't be added into KVS as a storage server.

[Mr. Pervect]

Thanks.
This is what my webhost told me.
Would you advice this? I should be able to add this through the "Hosted file" section in add video right?
And yes, in Storage server I see "CDN / Object Storage Videos"
I am only missing the : 1. Add trailer preview field 2. Alternative resolutions etc when I choose to add a video through this field.
 

Quote

 

I created a public CDN zone. All the videos in the (name-com) bucket are accessible from there.
For example, the video you uploaded to the (name-com) bucket at contents/videos/0/39/VIDEONAME_FULLSCENE.mp4 is available here
https://94837hjhjss.ssl.servername.net/contents/videos/0/39/VIDEONAME_FULLSCENE.mp4

If URLs will not work for you, we can temporarily mount the bucket as some local folder on the server. Just let me know where you expect it to be mounted.

 

 

Edited August 1, 2023 by Mr. Pervect

[Mr. Pervect]

Please see another message from my webhost.
So If I understand correctly now I can make a list and upload all scenes/trailers/mouseover video's under VIDEOS > Import video's.
Then KVS will import and process the video's (Create alternative resolutions etc 1080/720) to our S3 storage.
When this is done I can add all relevant information (Model name/tags/publish date etc.)

Thanks for your help!

 

Quote

It looks like there was some sort of confusion. The video storage server is configured to store the files on S3 at the moment. You can check the settings.

The KVS doesn't support S3 directly. That's why we installed and configured the SFTPgo account on the server. It works like an FTP-to-S3 proxy. KVS connects to the SFTPgo account by FTP and SFTPgo proxy these uploads to the sitename-com bucket.

 

[Tech Support]

3 hours ago, Mr. Pervect said:

The KVS doesn't support S3 directly. That's why we installed and configured the SFTPgo account on the server. It works like an FTP-to-S3 proxy. KVS connects to the SFTPgo account by FTP and SFTPgo proxy these uploads to the sitename-com bucket.

KVS supports S3 now starting from 6.1.0 version. But again I'm not sure we are talking about the same things here, because when we say KVS supports S3, this means KVS can create its OWN storage structure on S3 servers. But what you have is some video files that are ALREADY stored in S3, they are not connected to KVS in any way, they are just some 3rd-party files stored somewhere. And for KVS it doesn't matter where they are stored: either on your local disk, or local server, or some other server. KVS will either download and process them (and potentially push into its own CDN S3 structure), or will just link from where they are located right now. And this kind of "linking" is named hotlink in KVS terminology.

[Mr. Pervect]

I was misinformed by my webhost. or a miscommunication.
I have 500GB worth of content on the S3 server I wanted to import into KVS.
When I tried all scenes it got cancelled because of storage issues.
But when I did 1 everything went great.
I have 2 CDN zones figured, 1 pubic, 1 private.

But there are 2 problems:
- 1: I get warning from KVS saying "Video source files are not protected from direct access." this is because apparently the links are public.
- 2: Host didn't understand that by importing from the url KVS would not take those video's, re-encode them and upload them into another directory.

Quote

Message rom host:
When you first mentioned that you importing the videos with the URL (e.g.: https://dgeg544gee.ssl.server.net/TEST.mp4) i thought KVS was taking those videos, re-encoding them, and uploading them to another directory. Since it's not doing that then that option to import videos won't work here.What option did the KVS team originally recommended to take those videos and upload them?

Please advice the best way I can batch import my 24 scenes.
They are currently stored on an S3 server as my VPS has only like 250GB.

Thanks!

Edited August 6, 2023 by Mr. Pervect
made a mistake

[Tech Support]

This was already replied in support ticket.

[Mr. Pervect]

Thanks!