Jump to content

Recommended Conversion Servers?


Emilia
 Share

Recommended Posts

Hi,

Could you guys give some suggestions of what kind of conversion servers and settings are working best for you?

I'm currently using local but in the past I've tried servers with amd and i9s,

with i9-9900k I was roughly able to encode 100 videos in 24 hours for 1080p, 720p and 480p
with AMD Ryzen 5 I was able to do 40 or 50 in a day for same 3 formats

-vcodec libx264 -movflags +faststart -acodec aac -strict -2 -ar 44100 -ab 128k -f mp4 is my settings for 1080p.

Would love to hear what others are doing and if there's a way for me to optimize is further

Link to comment
Share on other sites

Hello

I have tried many ways of remote servers for encoding.

We had local servers consisting of I9 10core/20 thread but the cost of power plus dedicated fiber cable was not worth it.  So we found best solution to be 8C/16T servers from so you start which are pretty cheap.

Our numbers 

Local (our office servers) 

Dedicated fiber cable  1Gb -  €120 / month

Electricity costs to run two servers 24/7 -  € 90 +

Two servers at so you start - €110 euro.

So easy choice. If you need info on how to setup  the servers let me know we could help

 

 

 

 

  • Thanks 1
Link to comment
Share on other sites

2 hours ago, phseven said:

Two servers at so you start - €110 euro.

Is it safe to assume that you are using 2 servers with Intel  Xeon D-1540?

What ffmpeg flags do you use for all your formats and how many videos does each encoding server finishes in a day?

 

I've always gone with intel's i9 and amd's series, I didn't consider xeons for encoding until now.

 

2 hours ago, phseven said:

So easy choice. If you need info on how to setup  the servers let me know we could help

Setting up server is fine, thank you. What I'm really after is optimal ffmpeg flags for formats.

 

4K -vcodec libx264 -movflags +faststart -acodec aac -strict -2 -ar 44100 -ab 128k -f mp4

1080p -vcodec libx264 -movflags +faststart -acodec aac -strict -2 -ar 44100 -ab 128k -f mp4

720p -vcodec libx264 -movflags +faststart -acodec aac -strict -2 -ar 44100 -ab 128k -f mp4

480p -vcodec libx264 -movflags +faststart -acodec aac -strict -2 -ar 44100 -ab 128k -f mp4

 

As you see, I haven't tried to change the bitrate and I'm not usinf crf or any preset. I just wanted to have encoding done as quick as possible but now I'm inclined more towards getting a good quaity/size ratio.

Any tips regarding that would be greatly appreciated

 

6 hours ago, Tech Support said:

The most powerful conversion servers are using nvidia graphic cards for conversion:

Thank you, but I'm going for optimized vs powerful and expensive

Link to comment
Share on other sites

I really never counted how many videos per day are encoded because videos are always different in size and duration and depends how many videos users upload or we upload.

But both servers usually encode way more than 500 videos per day.

For encoding we use CRF at 25 for free tubes and CRF 21 for premium videos and pay-sites.

 

 

Link to comment
Share on other sites

2 hours ago, Emilia said:

Is it safe to assume that you are using 2 servers with Intel  Xeon D-1540?

What ffmpeg flags do you use for all your formats and how many videos does each encoding server finishes in a day?

 

I've always gone with intel's i9 and amd's series, I didn't consider xeons for encoding until now.

 

Setting up server is fine, thank you. What I'm really after is optimal ffmpeg flags for formats.

 

4K -vcodec libx264 -movflags +faststart -acodec aac -strict -2 -ar 44100 -ab 128k -f mp4

1080p -vcodec libx264 -movflags +faststart -acodec aac -strict -2 -ar 44100 -ab 128k -f mp4

720p -vcodec libx264 -movflags +faststart -acodec aac -strict -2 -ar 44100 -ab 128k -f mp4

480p -vcodec libx264 -movflags +faststart -acodec aac -strict -2 -ar 44100 -ab 128k -f mp4

 

As you see, I haven't tried to change the bitrate and I'm not usinf crf or any preset. I just wanted to have encoding done as quick as possible but now I'm inclined more towards getting a good quaity/size ratio.

Any tips regarding that would be greatly appreciated

 

Thank you, but I'm going for optimized vs powerful and expensive

 

 

Hello, I see that I have something different, what difference would there be with my code:

format 480p: -vcodec libx264 -movflags +faststart -threads 0 -r 25 -g 50 -crf 25 -me_method hex -trellis 0 -bf 8 -acodec aac -strict -2 -ar 44100 -ab 128k -f mp4

Link to comment
Share on other sites

9 hours ago, phseven said:

But both servers usually encode way more than 500 videos per day.

That sounds unrealistic to me. I've never seen performance like that an older cpu like Xeon D-1540.

Do you compile ffmpeg from source and make modifications to kernel as well?

Link to comment
Share on other sites

3 hours ago, Emilia said:

That sounds unrealistic to me. I've never seen performance like that an older cpu like Xeon D-1540.

This depends on many factors, such as the number of video formats (e.g. 1080p, 720p, 480p) you create and their duration. So the plain number doesn't actually mean anything. Also check if CPU is the actual bottleneck in your case, as it can be possible that slow HDD reducing ffmpeg speed, while CPU is not even fully loaded.

Link to comment
Share on other sites

8 hours ago, Emilia said:

That sounds unrealistic to me. I've never seen performance like that an older cpu like Xeon D-1540.

Do you compile ffmpeg from source and make modifications to kernel as well?

We use bare centos 7 with static ffmpeg builds .  XEON processors even if older run more efficient than desktop processors. 

 

4 hours ago, Tech Support said:

This depends on many factors, such as the number of video formats (e.g. 1080p, 720p, 480p) you create and their duration. So the plain number doesn't actually mean anything. Also check if CPU is the actual bottleneck in your case, as it can be possible that slow HDD reducing ffmpeg speed, while CPU is not even fully loaded.

 

That's correct  it depends on many factors like video formats and video duration , we never looked really into how many videos it encoded per day but what worked best for us.  SSD tend to be faster than mechanical HD on normal 480p, 720p, 1080p but not on 4k which is about the same.

Link to comment
Share on other sites

Update: I've ordered 3 servers to run some tests on. I'll be activating all 3 servers at the same time once the setup is done, they'll be encoding videos from same batch and I'll monitor them for a week. In the mean time I'll just make sure to keep the content coming and in bulk quantities for them stay busy at all times. Thinking about adding 2 active encoders on each server as mentioned here.

 

On 10/11/2017 at 12:31 PM, Tech Support said:

if you use only 1 conversion server for 1 physical server, there will be intervals when your CPU load is 0% or when your network load is 0%. This is a waste of resources. By creating at least 2 conversion servers for the same physical server you will increase its conversion speed and overall resources utilization will be close to 100%. But this doesn't mean that adding more conversion servers here will increase the speed to x2, or x3 - this is just a matter of +20% or +30%. Therefore we think that having 2 or max 3 conversion servers per 1 physical server is more than enough.

 

CPU - Benchmark Score

Intel Core i7-8700 - 13180
AMD Ryzen 7 1700X - 15622
Intel Xeon E5-1650V3 - 10382

I've selected these particular CPUs because their benchmark score is fairly close to each other. Xeon in particular is lower then the rest because I was told that they perform better, so I thought it'd be a nice idea to give it some handicap.

Hoping for some exciting results ✌

  • Like 1
Link to comment
Share on other sites

  • 2 weeks later...

Hi,

So here are my results. These won't be 100% accurate but they do gave me some clarity.

After 10 days of encoding non stop on all 3 servers here are the number of completed tasks each server finished. I added another server a day later so I'll include that too

I use video semi processing so for each video I got 3-4 tasks for each video format.

Intel Core i7-8700 - 1762 total - completed 176 tasks/day on average (10 running days)
AMD Ryzen 7 1700X - 1134 total - completed 113 tasks/day on average (10 running days)
Intel Xeon E5-1650V3 - 1342 total - completed 134 tasks/day on average (10 running days)
AMD Ryzen 9 3900 - 2006 total - completed 222 tasks/day on average (9 running days)

Apart from these 4 servers I also installed 10 small linux vps with 2 cores, each vps cost me 3.5eur/mo and have completed 40 tasks/day on average. Some of the longer videos uploaded by users, 1hr + did take between 6-8 hours on these small vps to make a 1080p format. But that was expected. But combining 10 of those small vps to run tasks simultaneously I was able to complete over 4k tasks in 10 days for a lot less cost compared to those dedicated servers

  • Thanks 3
Link to comment
Share on other sites

Hint: we asked KVS support if it is an OK idea to buy servers, put them in the basement of my office on a gigabit LAN so the actual video encoding would happen locally. 

They said they have nothing against this.

And this solution seems to be the best cost-wise.

A second-hand server costs 200-300 USD / EUR, you need to buy graphics card for it [need to buy proper server for such cards].

We are just about to build this out, if anyone wants to join in in brainstorming, etc: get in touch with me!

Kind Regards,

Matyko Berenyi

Owner/CEO @ NetConvert Ltd [ http://www.netconvert.net ]
LinkedIn: https://www.linkedin.com/in/matykoberenyi/
Skype: live:matyko42
Cell/WhatsApp: +36304573092

  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...