whats new KVS 6.3.0 update

[Tech Support]

Update status

6.3.0 update status is ALPHA. Due to huge background code changes we are cautious to provide updates and we only install for new projects. Beta updates should be available soon after some initial testing.

 

Update requirements

• KVS 5.0.0+ (earlier versions should be updated to KVS 5.0.1 first).
• PHP 7.1 - 7.4, PHP 8.1.

If you haven't yet updated KVS to 5.0.1, then please do, you can find KVS 5.0.1 update information here: KVS 5.0.1 update

Important notes for 6.3.0

• Due to security concerns we disabled execution of {{php}} blocks in site templates if allow_php_in_templates setup option is not enabled. KVS update plugin should warn about this if your project is affected.
• Please check important notes for 6.2.0 if you didn't yet update to this version.
• Please check important notes for 6.0.1 if you didn't yet update to this version.
• Please check KVS 5.5.0 and vertical screenshots issue in old projects issue if you didn't yet update to this version.
• Please check important notes for 5.4.0 if you didn't yet update to this version.
• Please check important notes for 5.2.0 if you didn't yet update to this version.

 

6.3.0 update procedure (same as usual, nothing special)

Any project starting from 5.0.0 can be updated to 6.3.0 with one only procedure. Depending on your current KVS version, update procedure may contain additional steps for older versions.

You must use KVS update plugin in order to update your project. For update you will need:

• KVS update ZIP archive for your project and its hash code for update plugin (contact support in beta phase to get these).
• FTP connection to your project so that you can upload files.

Update procedure has the following steps:

• Create backup using Project backup plugin. You may skip manual backup if you have automated daily backups enabled and you see the backup archive from today or yesterday in the list of available backups.
• Upload update ZIP into KVS update plugin and specify hash code.
• KVS update plugin will notify you if there are any custom changes in KVS system files, which will be overridden by update (player skins are not checked here, so they will be silently updated if have custom changes).
• KVS update plugin will update database automatically.
• KVS update plugin will ask you to copy files from the archive on top of your project using FTP or filesystem copy. Please make sure you are NOT USING sync functionality in your FTP client, which will delete many files on your server, because obviously update archive contains only part of all files. What you need is just to drag and drop files from update archive on top of your project root folder and confirm their re-writing (excluding _INSTALL folder, it doesn't need to be copied).
• KVS update plugin will verify the updated files and finalize update procedure.
• Use System audit plugin to verify that everything looks good.

 

Whats new in KVS 6.3.0

Project-wide background code changes, AVIF support in videos and albums, better stats and charts, rotator satellite
support, static files editor, and dozen of smaller enhancements.

1) We continue with background code refactoring. This version has fully revamped system and network calls across the
whole product.

2) Our Nextgen architecture is now started being introduced in admin panel in several places. One of the new editors
is System settings, where you can specify timezone, memory limits for different processes and set of networking options.
Some of these options were previously available in Content settings, some of them are brand new.

 

3) We finally added Email settings, so that you can now set up using 3d-party email service without our assistance.
Email template editing is coming in the next updates.

 

4) Screenshot formats now support AVIF in addition to WebP and JPG formats. This could be controversial for thumbs,
since AVIF images are created 10 times longer than JPG and provide not a big difference in comparison to WebP (~55%
smaller size vs ~33% smaller size for WebP). However what is more important is player poster images, which are bigger
than thumbs and using AVIF for them may be beneficial for your SEO. Player poster images are typically considered
as LCP in Google performance evaluation. Screenshot formats now allow you to configure player poster settings as well,
and our recommendation would be to switch them to AVIF. Previously KVS would automatically convert all player poster
images to WebP (if supported by imagemagick), but with this update you will need to switch them to AVIF manually if you
want.

 

5) WebP and AVIF support was also added into album formats as well.

6) In screenshot and album formats KVS would now show a popup with visual confirmation of the options, where you can
see the modified options applied to some random thumbs. This should be useful to understand if the new options are fine
or need some adjustments.

 

7) Stats section in admin panel has some crucial changes. The index page is now a list of configurable charts where you
can customize from 7 available types of charts, which will likely be extended in future. Each chart supports filtering
and data display options. Hourly stats are now supported, and one of the charts allows comparing today's data with
yesterday's. Some of the traffic stats now switched to Nextgen code, and they allow now filtering by multiple different
values for the same filter, for example it is possible now to filter combined stats for several countries or devices.

 

8) In Stats Settings you can now choose to not collect content or search stats from a specific set of countries, or
collect only from a specific set of countries. This may be useful to reduce garbage stats and only collect from your
primary set of countries. This is also useful when you pay tokens for video views and you don't want to pay for specific
countries. Same for search queries, you can avoid queries that are submitted by users from certain countries, that may
later appear on your site and in SEO.

 

9) In traffic stats KVS will now populate separate stats for known SEO bots. Also requests from SEO bots will now be
excluded from content stats (video views, file requests and etc.).

10) Rotator logic was finalized to support satellites. Now if enabled, rotator will collect combined stats from primary
project and all satellites. Also fixed issue with broken screenshots in some cases when rotator is configured to
partially delete screenshots.

11) When vertical screenshots are merged together, KVS would now use screenshots from different parts of videos, not
from the beginning only.

12) Added support for billing.creditcard (this is subsidiary of Verotel). Verotel was extended with chargebacks and
refunds transaction types.

13) Import and export (including feeds) will now consider using display name instead of username for users.

14) In Website UI section it will now be possible to modify and upload static files, such as CSS, JS and images. This
may reduce the necessity of providing FTP access to theme developers, however there is still need to modify .htaccess
file directly from server filesystem if URL pattern changes are required.

 

15) Due to security concerns we disallowed execution of PHP fragments in templates by default, unless you have system
option allow_php_in_templates activated in /admin/include/setup.php. Previously this option was only required to save
templates with {{php}} directives from admin panel, but it didn't prevent the code from being executed (for example if
added via FTP). Now such code will not be executed if the option is not activated manually. This is incompatibility
change and KVS update plugin should warn about such cases during update.

 

16) In Memberzone Settings it is now possible to configure unconfirmed accounts auto-removal after N days.

 

17) In logon block it will be possible to choose between list of member statuses that are allowed to log in.

 

18) In search queries we added ability to specify redirect for each query individually. This can be used to redirect
some indexed queries with abused terminology to other search results.

 

19) Bugs that have been fixed:

- [SEVERE] Several new bugs related to PHP 8 support.
- [MEDIUM] A player poster could be not rendered correctly in some cases.
- [MEDIUM] There was an issue with correct detection of Chrome browser in mobile Apple devices.
- [LOW] Feed auto-deletion logic didn't run correctly with its schedule.
- [LOW] Fixed several issues with player subtitles display.
- [LOW] It was possible for users to add new videos into playlists that were locked by admin.

 

[viper]

so its not final update?

[Anna034546]

These are cool features, but you haven't fixed the crucial issues of the script. It has a hard limit on scaling. There is no way to load balance the server that hosts the KVS and also the database. Once users reach 200K visitors a day, all shiny features need to be turned off one by one to keep the website up and reduce the server load.

[AsianViralHub]

when will it be out?

[Tech Support]

7 hours ago, viper said:

so its not final update?

 

1 hour ago, AsianViralHub said:

when will it be out?

We will need 1-2 weeks to test on new projects and then will start providing beta updates.

 

3 hours ago, Anna034546 said:

These are cool features, but you haven't fixed the crucial issues of the script. It has a hard limit on scaling. There is no way to load balance the server that hosts the KVS and also the database. Once users reach 200K visitors a day, all shiny features need to be turned off one by one to keep the website up and reduce the server load.

There are no issues with 200K visitors per day, or even 2000K. If you have issue with 200K, something is definitely wrong.

[Mich]

Email settings without the need for support, a new Stats section, and auto-deletion of unconfirmed items.
Really nice!

 

13 hours ago, Anna034546 said:

These are cool features, but you haven't fixed the crucial issues of the script. It has a hard limit on scaling. There is no way to load balance the server that hosts the KVS and also the database. Once users reach 200K visitors a day, all shiny features need to be turned off one by one to keep the website up and reduce the server load.

For my part, I find KVS to be over-optimized for 100k unique visitors per day (KVS Stats).
If the theme is well-designed and no functions are causing issues, I've seen 2.00 Load Average on a Ryzen 3600.
Of course, everything has been optimized on my end, and recently compression (brotli/gzip) has been disabled since Cloudflare is in use with ZSTD.

[TheBeast]

Any updates regarding HLS?

[Anna034546]

12 hours ago, Tech Support said:

 

We will need 1-2 weeks to test on new projects and then will start providing beta updates.

 

There are no issues with 200K visitors per day, or even 2000K. If you have issue with 200K, something is definitely wrong.

There is no tube website built on vanilla KVS that gets 2 million visitors a day, unless you mean serving website_unavailable.html. Undoubtedly, big tube websites use KVS, but they have found internal workarounds and have heavily edited the script. This allows them to either use several servers in load balancing or architect the script to spread each function across remote servers.

Every time we hit 250k-300k visitors a day, the website experienced a good 10-15 minutes of downtime per day, and the load average at best was between 20-30. The server specifications weren't the highest end, but it was high-mid range. Everything was on CDN, the database was on another server, and media and conversion were on remote servers.

But the issue is that many processes still happen on the KVS main server, such as copying all media from the main server to media/conversion servers and rotator settings. Even some images are hosted on KVS; I believe those video screenshots on the player bar are hosted on the KVS main server.

What support suggested to us was disabling KVS features one by one and adding an aggressive caching policy to pages. Things like disabling search options, similar videos, extended features, and stats. There were a few other suggestions that limit KVS features to reduce the website's load, I need to look up all the details. But at the end of the day, there was a hard limit, and the response was that KVS is not designed for load balancing the main server.

So I guess your target audience is customers with less than 200K visitors a day, which is logical. Out of 1,000 customers, maybe 5 of them may go above that, and they need to find a custom solution by themselves.

3 hours ago, Mich said:

Email settings without the need for support, a new Stats section, and auto-deletion of unconfirmed items.
Really nice!

 

For my part, I find KVS to be over-optimized for 100k unique visitors per day (KVS Stats).
If the theme is well-designed and no functions are causing issues, I've seen 2.00 Load Average on a Ryzen 3600.
Of course, everything has been optimized on my end, and recently compression (brotli/gzip) has been disabled since Cloudflare is in use with ZSTD.

The problem didn't start at 100k; the issues started appearing above 200k. Although if your website is just embedding videos, it uses much fewer resources. I experienced that if users are able to upload high-quality full videos like 4K, it puts a lot of load on the server, even with remote conversion and media servers, as everything needs to go back and forth from the main KVS server.

[Tanjiro]

Wow, that's a lot of updates, must be tough for KVS team.

Waiting for final update 😀

[Mich]

5 hours ago, Mich said:

Email settings without the need for support, a new Stats section, and auto-deletion of unconfirmed items.
Really nice!

 

For my part, I find KVS to be over-optimized for 100k unique visitors per day (KVS Stats).
If the theme is well-designed and no functions are causing issues, I've seen 2.00 Load Average on a Ryzen 3600.
Of course, everything has been optimized on my end, and recently compression (brotli/gzip) has been disabled since Cloudflare is in use with ZSTD.

unconfirmed items --> unconfirmed accounts.

If my last post can be edited :)

[hbarnetworks]

On 11/24/2024 at 10:36 PM, Anna034546 said:

These are cool features, but you haven't fixed the crucial issues of the script. It has a hard limit on scaling. There is no way to load balance the server that hosts the KVS and also the database. Once users reach 200K visitors a day, all shiny features need to be turned off one by one to keep the website up and reduce the server load.

The database is always scalable. This has nothing to do with the application. The front-end can easily be run on multiple servers with shared storage. The only problem is that the caching folders needs to be mapped to a localdrive or else the shared storage will flunk. But other than that its definitly possible to scale. But it requires alot of prep work. but that is always the case with scaling there is no easy solution to that.
 

Source: I am running ~1m to 1,5m users a Day.

But sadly no HLS yet. :(

Edited November 26 by hbarnetworks
Typos

[Tech Support]

20 hours ago, Anna034546 said:

There is no tube website built on vanilla KVS that gets 2 million visitors a day, unless you mean serving website_unavailable.html. Undoubtedly, big tube websites use KVS, but they have found internal workarounds and have heavily edited the script. This allows them to either use several servers in load balancing or architect the script to spread each function across remote servers.

This is not true. I don't know why you have this opinion, but here is for example a tube which is 1 of 3 similar tubes running on the same server (3 tubes on the same server). Server is powerful, yes. As you can see the overall daily traffic (total counted + bots) is more than 2M. So server handles around 7-8M of daily visits in total. The number of videos on this tube is 300k. They use separate server with sphinx for related and search. No other code customizations are used, just vanilla KVS, 100% know this.

 

20 hours ago, Anna034546 said:

Every time we hit 250k-300k visitors a day, the website experienced a good 10-15 minutes of downtime per day, and the load average at best was between 20-30. The server specifications weren't the highest end, but it was high-mid range. Everything was on CDN, the database was on another server, and media and conversion were on remote servers.

So then you may be need to investigate why your project experienced this. It may be simply because your caching was disabled, or somewhere not optimal, or sometimes theme customization team may add some aggressive logic into templates or PHP extension files, there should be definitely a reason. 300k visitors should not be an issue for middle range server, specifically with database moved to remote.

 

20 hours ago, Anna034546 said:

But the issue is that many processes still happen on the KVS main server, such as copying all media from the main server to media/conversion servers and rotator settings. Even some images are hosted on KVS; I believe those video screenshots on the player bar are hosted on the KVS main server.

There is a way to workaround this. You need to buy a satellite license and move your main license to a subdomain like admin.domain.com, and satellite would be issued to domain.com. Then your main admin panel with all conversion and video screenshots will be located on a server with admin.domain.com, and your primary frontend will run as a satellite on another server on domain.com. This is a good way to split content processing. I would even suggest that satellite runs on the same server where database is located, because as a frontend it will need fast network connection with database. Some customers are using this approach.

 

20 hours ago, Anna034546 said:

What support suggested to us was disabling KVS features one by one and adding an aggressive caching policy to pages. Things like disabling search options, similar videos, extended features, and stats. There were a few other suggestions that limit KVS features to reduce the website's load, I need to look up all the details. But at the end of the day, there was a hard limit, and the response was that KVS is not designed for load balancing the main server.

This sounds a bit strange. From what you said here is that your database is moved to remove server. Then disabling search options, similar videos, caching and etc. doesn't affect your primary server load, it will only affect DB server load. If what you are saying is true (e.g. your primary server has big load, and this is not because of database as database is not located on the primary server), you need to provide more details of what is causing the high load. It can be disks or even network channel overload that results in high LA on the server. In case of high CPU usage from Apache or PHP the first recommendation would be to purchase open source code option if you don't have it yet.

 

20 hours ago, Anna034546 said:

So I guess your target audience is customers with less than 200K visitors a day, which is logical. Out of 1,000 customers, maybe 5 of them may go above that, and they need to find a custom solution by themselves.

Actually not. I can admit KVS has issues with big databases (500k+), as you need very specific optimizations to support millions of content, and normally these optimizations would mean less options and less flexibility. But no issues with dealing with big traffic, which KVS was specifically designed for.

[Tech Support]

21 hours ago, TheBeast said:

Any updates regarding HLS?

No, not yet.

[Mich]

5 hours ago, hbarnetworks said:

The database is always scalable. This has nothing to do with the application. The front-end can easily be run on multiple servers with shared storage. The only problem is that the caching folders needs to be mapped to a localdrive or else the shared storage will flunk. But other than that its definitly possible to scale. But it requires alot of prep work. but that is always the case with scaling there is no easy solution to that.
 

Source: I am running ~1m to 1,5m users a Day.

But sadly no HLS yet. :(

Source: I am running ~1m to 1,5m users a Day.
How much monthly on Similarweb?

Waiting for HLS here as well, or an equivalent solution.

Edited November 26 by Mich

[Matyko42]

Besides the exciting new features, this thread has a Very Interesting and important discussion regarding the traffic ~ load. It would be super-cool to have a separate, detailed discussion between KVS users with significant traffic and videos, we could help each other and the KVS Tech Support could add real value to such a discussion / workshop.

Also, it would be nice to get familiar with each other, and probably do business as well. 

I even gave a name to this: "Advanced Tube Alliance" :) 
[[ ...Maybe its not the best idea to do such a serious discussion in a open forum here... Maybe KVS could open a separate 'discreet' forum/part for us? ]]

Let's get in touch with each other! → We have a network of 14 tubes, shared db with ~350k videos, 1M+ daily traffic on the network. 

Matyko Berenyi / matyko@netconvert.net

Owner/CEO @ NetConvert Ltd [ https://www.netconvert.net ] 
LinkedIn: https://www.linkedin.com/in/matykoberenyi/
Skype: live:matyko42 [not the kazakh account!]
Cell/WhatsApp: +36304573092

 

[Jackass]

Great updates!
It would be great to start the video with sound when setting auto play, so far I have to mute the sound to start the video automatically.

[Tech Support]

1 hour ago, Jackass said:

It would be great to start the video with sound when setting auto play, so far I have to mute the sound to start the video automatically.

This is not possible, browsers do not allow autoplay videos with sound by default. Some browsers may allow auto-play with sound on some well-known video websites, such as youtube.com for example, but not for all sites.

[hbarnetworks]

On 11/26/2024 at 5:22 PM, Mich said:

Source: I am running ~1m to 1,5m users a Day.
How much monthly on Similarweb?

Waiting for HLS here as well, or an equivalent solution.

Why would that matter? similairweb is just an indication its not reality.

[Mich]

On 12/2/2024 at 4:50 AM, hbarnetworks said:

Why would that matter? similairweb is just an indication its not reality.

Because SimilarWeb is quite accurate, I can compare it with other websites I own that don't use KVS. (Like ecommerce)