development KVS 4.0 implementation (previously was 3.9.2)

[Tech Support]

NOTE: this version was initially planned as 3.9.2, but finally resulted in versioning change to 4.0. There will no be 3.9.2 version available, after 3.9.1 we will switch to 4.0.

 

We are very close to finalizing 3.9.1 update and pushing it to beta testing mode.

 

We will use this topic to post updates on 4.0 implementation. Meanwhile here are our high level plans for 4.0, please feel free to post your comments and proposals for the highlighted areas, as it will be much easier to consider them initially then to re-do in future.

 

We are not sure if all of these areas will be done in 4.0, it depends on their implementation effort, but if not they will be moved to 4.1 version then.

 

AREA 1. Mail system.

 

Complete emailing refactoring. We know this is annoying issue for many people and would like to finally close it.

 

Requirements that we plan here:

• Ability to easily set up public email service to send emails (gmail and etc.)
  
• Ability to configure email templates in admin panel, extend support for tokens that can be used in emails.
  
• Ability to translate email templates to multiple languages similarly to other content in KVS.
  
• Ability to enable / disable certain emails from one place.
  
• Support for new emails: welcome email on signup, notification about new messages, notification about comments posted to the user's content, notification about payout sent to user. It should be possible to configure the frequency of sending some notifications in admin panel: if they should be sent immediately or should be summarized on a daily / weekly basis.
  
• A new site block should be added to allow users subscribe or unsubscribe certain notifications. Should be possible to include unsubscribe link in every email sent. By default users will be subscribed to all enabled emails in the system.
  
• Ability to configure scheduled emails with updates on the site - here you should be able to use standard template functions like adding blocks and rendering email similar to any page on your site. For example it should be possible to render top videos / albums of the last week and send to your users each weekend.
  

AREA 2. Automated audit and admin notifications.

 

In KVS you can run audit plugin in manual mode to check your project for any issues. KVS can check up to 95% of all possible issues and in most cases if something is not working, audit plugin will report that. However in some cases you may not even notice that something is wrong until you manually run audit plugin.

 

Requirements that we plan here:

• Create log table to log any issues that arise from time to time.
  
• Extend audit plugin with some more checks.
  
• Run audit plugin on scheduled basis to make sure that any issues will be reported ASAP.
  
• Report summary email to admin on a daily basis (traffic stats, content stats, project status).
  
• Report critical issues to admin's email immediately when they are identified.
  

AREA 3. Comments.

 

Requirements that we plan here:

• Ability to reply to other comments (hierarchy).
  
• Ability to show users comments to their uploaded content.
  
• Ability to disable comments to specific content (from admin panel and user area).
  
• Ability to sort comments by votes (show top rated comments).
  

AREA 4. Video conversion.

 

Requirements that we plan here:

• Optimize trailer creation with new ffmpeg features.
  
• Investigate into applying dynamic watermarks.
  
• Investigate into possibility to embed pre-roll / post-roll video fragments into each video file.
  

AREA 5. VAST support.

 

Add support for VAST advertising into KVS player.

Edited February 14, 2018 by Tech Support
Changed versioning from 3.9.2 to 4.0

[steward]

Comments on Mail:

 

Reminder to ensure we can hook into this. We are already using third party APIs to send email. Sending from the site is a matter of queuing the message. A cron job does the sending.

We already have opt-in lists and users have set preferences. We use several different ESP depending on conditions (eg SendInBlue has SMS for account recovery, MailJet has drip campaigns).

The choice of where you put the hook and how much information is provided can make the transition easy or difficult. We will never go back to SMTP on our own servers.

 

Responsive emails for mobile are hard to design, implement and test. There are many third-party services that sell templates. It would be best if we can purchase them if we want.

Please do not lock us into a proprietary kvs-only templating system for emails.

 

 

[Tech Support]

The choice of where you put the hook and how much information is provided can make the transition easy or difficult. We will never go back to SMTP on our own servers.

 

Thanks for noting this. We are planning of cause use cron solution to send emails, except for immediate needs (signup and so on). We are planning to allow multiple sending options, e.g. sendmail (the currently used), phpmailer and we can also include custom script into this list, which will be called by KVS in case of need to send email.

 

Please do not lock us into a proprietary kvs-only templating system for emails.

 

Not sure I understand this point, but we will definitely not support any other templating than smarty. An email is HTML format, and any service which sells email templates, it sells in HTML format, correct? So any HTML code can be a valid template for smarty. But for example if you want to send emails with updates you will have to use KVS list_videos / list_albums blocks to render content, this is where you can use smarty functionality. I'm not sure however how we will technically implement this for now. Most probably we will have to stick to Website UI section, so that each such template should be added as a site page first and then it can be selected as a mail template. Since we already have the needed rendering engine we will reuse it for faster development.

 

[steward]

except for immediate needs (signup and so on)

 

Forget cron. No exceptions. Every outgoing mail must go through an ESP API. The SMTP server is not available to kvs. There is no other way to send mail.

Outgoing mail is sent from a different server.

 

All that matters is that you have one point where it can be intercepted, and all the necessary data is on hand at that point. For all outgoing email.

 

Hope that is clear.

 

An email is HTML format,

 

And CSS. The css may be based on Zurb or Bootstrap and may be included or converted to inline. The website page approach sounds fine.

The styling will not be compatible with the website.

 

Thank you for your consideration.

 

[Tech Support]

Forget cron. No exceptions. Every outgoing mail must go through an ESP API. The SMTP server is not available to kvs. There is no other way to send mail.

Outgoing mail is sent from a different server.

 

All that matters is that you have one point where it can be intercepted, and all the necessary data is on hand at that point. For all outgoing email.

 

Hope that is clear.

 

Not clear yet... Not sure how is cron related to the way of sending emails.

 

The reason for using cron emailing is the following. If you want to notify users about new private messages or new comments posted to their content, it makes no sense to do that immediately after the message was sent or after a comment was written. The reason is: there can be many of such messages or comments. I think it is not a good idea to send a separate email for each new message or comment. Instead it will make more sense to send something like:

- You have 10 new private messages

- You have 20 comments posted on your videos

- You have 30 purchases of your content today (if user is configured to be paid in tokens)

and etc.

 

For such notifications we will need to collect them in database and to be grouped together, so that they are sent once a day or other period.

 

With regard to API for sending emails, I was thinking that using phpmailer + external email service like google is kind of standard for small businesses. Is there any other way? What do you mean by ESP API?

[Tech Support]

In 3.9.2 we have updated categories and tags auto-selection plugins to allow lenient mode for compound tags of categories. In lenient mode the compound expression will be tested for match of its each component word instead of exact expression match.

 

[steward]

With regard to API for sending emails, I was thinking that using phpmailer + external email service like google is kind of standard for small businesses. Is there any other way? What do you mean by ESP API?

 

ESP API = Email Service Provider Application Programming Interface.

 

"ESP" has a broad definition and multiple meanings. Most people think gmail is an Email Service Provider, and they are correct. It provides emails !

You can google for Wikipedia or other pages with "What is an ESP?" But it is best to look at the sites themselves:

https://www.inkthemes.com/transactional-email-service-providers/

All ESPs recommend *not* using gmail (eg https://app.mailjet.com/support/how-to-set-up-mailjet-s-smtp-with-gmail,110.htm).

But all that is debatable.

 

ESPs are not just for marketing and campaigns. They also handle transactional mail (signup, forgot password, receipt for payment, ...)

KVS cannot begin to duplicate the services they provide. They keep our member list clean, so we never email anyone that bounced or marked us as spam.

And more.

 

Sample ESP API listed at https://apidocs.sendinblue.com/tutorial-sending-transactional-email/

That is for SendInBue PHP. GitHub has dozens (hundreds) more (Mailjet/php, Mailgun/Ruby etc etc).

 

Adult webmasters can get blacklisted by using scripts that repeatedly send to dead mails and spam traps. I speak from experience.

We must know when to stop sending (we need to act on the result: delivered, bounced, opened, clicked, marked a spam...).

Or eventually you cannot even deliver a password recovery request.

 

KVS can use cron to batch transactional emails, that makes sense.

 

ESPs also batch emails. We can send them the batch in one call. They have algorithms tuned to make delivery appear normal to the outside world, so the site does not look like a spammer.

 

End of lesson. You guys probably knew this anyways. The point is this: at whatever point you invoke phpmailer, provide an override or hook. Allow us to use other mailers.

 

From your point of view, an ESP is just another php sending system like SwiftMailer or phpmailer. The API can accept a a batch or a single email.

 

I now store this information with each user: (a) what notifications have they opted in? (b) when was the last date/time they confirmed/validated their email, and © is it valid now (marked invalid when we get a notification from the ESP that an email has bounced).

 

In some cases ESPs can reply with bad news instantly. Or in the case of a previously recorded bad email, we can. So instead of saying "Please check your inbox" we can say "Sorry we were unable to deliver to that email". That is a result phpmailer cannot provide.

 

Email deliverability is a huge subject area. From KVS end it is not so complicated. Just don't lock us into phpmailer is all I'm saying ;)

 

Thank you for listening.

[Tech Support]

Email deliverability is a huge subject area. From KVS end it is not so complicated. Just don't lock us into phpmailer is all I'm saying ;)

Thank you for listening.

 

Thanks for the info! We will consider this when re-designing email area.

[Tech Support]

In 3.9.2 we have added a new option to video crop settings, which can be used to force crop for screenshots uploaded manually.

 

Prior to that KVS never applied crop to the screenshots uploaded manually and this also included grabbers / feeds when importing embed codes or pseudo videos: the grabbed screenshots couldn't be cropped. Starting from 3.9.2 it be possible to enable this.

 

[enricoeur]

You should add an easy way to have Google reCaptcha to protect LOGIN / SIGNUP form. In my case someone created a script that use to automatic register / login / and send massive messages to all subscribed users.

 

My community is free and so somone studied the situation to find a profitable way to monetize my community.

 

No any way to stop them, they use multiple Proxy around the globe to continue and contiue with this spam operation.

 

 

Thanks

[Tech Support]

You should add an easy way to have Google reCaptcha to protect LOGIN / SIGNUP form.

 

Thanks

 

We will check that. We had the same issue with this KVS forum and using reCaptcha didn't help from spam registrations. Registrations can be done manually with easily completing reCaptcha manually, do you want to enable captcha on sending messages as well? Only this captcha can prevent from sending massive spam, but it will also bring inconvenience to normal users.

 

There should be other solution like for example prevent from sending many similar messages, or messages with URLs, or require minimum 5 days account duration in order to be able to send messages. Probably the combination of these will give more effect.

 

[Tech Support]

In 3.9.2 we have updated categories and tags auto-selection plugins to allow lenient mode for compound tags of categories. In lenient mode the compound expression will be tested for match of its each component word instead of exact expression match.

 

[ATTACH=JSON]{"data-align":"none","data-size":"full","title":"tags_autoselection.png","data-attachmentid":297}[/ATTACH]

 

After some feedback we changed GUI for this implementation. It was noted that in many cases you will need lenient match only for specific expressions, while for the rest of compound expressions you will better not use it. Therefore we added ability to enable lenient match only for the specified expressions:

 

[PornLover]

Hi,

 

You should really add a plugin for auto channel selection based on plugin auto model selection...

 

It will be really helpful as now we have to manually add new videos to the channels which is painful...

 

So please try to add the "Auto channel selection" plugin...

 

Thanks

[Tech Support]

Hi,

 

You should really add a plugin for auto channel selection based on plugin auto model selection...

 

It will be really helpful as now we have to manually add new videos to the channels which is painful...

 

So please try to add the "Auto channel selection" plugin...

 

Thanks

 

But how do you expect it to work? Models / tags / categories autoselection are based on title / description and they look for specific words in content title or desc. Do you mean that for channels it should also use the same logic? Please post an example of video title and channel title that should be selected for this video, to make sure we understand this correctly. From my experience I don't recall that video titles have channel names, normally this is not true. But maybe you mean description, not title?

[PornLover]

 

But how do you expect it to work? Models / tags / categories autoselection are based on title / description and they look for specific words in content title or desc. Do you mean that for channels it should also use the same logic? Please post an example of video title and channel title that should be selected for this video, to make sure we understand this correctly. From my experience I don't recall that video titles have channel names, normally this is not true. But maybe you mean description, not title?

 

Hi,

 

Maybe we should add an option at the upload page to channel selection or channel name input or as you said based on description...

Somehow we should we an solution for auto channel selection this can be so much helpful...

 

Thanks

 

 

[PornLover]

Hi,

 

I also have an new suggestion as the technology grow we also need to adapt to that new tech as far as i know past couple of months we hear a lot about 360

[Tech Support]

 

Hi,

 

Maybe we should add an option at the upload page to channel selection or channel name input or as you said based on description...

Somehow we should we an solution for auto channel selection this can be so much helpful...

 

Thanks

 

 

I still don't understand this, do you want users should be able to specify which channel they are uploading to? But this is standard KVS functionality, you can see it on our demo site. You can create a channel and upload video into it. There is upload video to channel button on channel page:

http://www.kvs-demo.com/channels/top-videos/

 

Template can also be configured to display list of available channels to select if needed.

[Tech Support]

Hi,

 

I also have an new suggestion as the technology grow we also need to adapt to that new tech as far as i know past couple of months we hear a lot about 360

 

We have a lot other tasks for now, we think bringing HLS / DASH support is much more priority than 360 videos. Later on we will think about this. Not possible to add many things at the same time.

[xvids]

Model Avatar generation: Generates avatars for Model from videos / albums

[xvids]

Hi,

 

FEEDBACK for Mark videos deleted [option]

 

send and Email to the copyright holder

 

right after, Reason text:

 

Like" [TABLE=border: 5]

[TR]

[TD=colspan: 2]Videos / Mark videos deleted[/TD]

[/TR]

[TR]

[TD]Videos to delete:[/TD]

[TD=colspan: 3]Title of video[/TD]

[/TR]

[TR]

[TD]Reason:[/TD]

[TD=colspan: 3]This video is no longer available due to a copyright claim.[/TD]

[/TR]

[TR]

[TD]Reason text:[/TD]

[TD=colspan: 3]videos will not be deleted from your database and will remain accessible via direct links, but all their files and screenshots will be deleted[/TD]

[/TR]

[/TABLE]

 

 

Send an email to the reporter:

[TABLE=border: 4, cellpadding: 1, width: 200, height: 30]

[TR]

[TD]Email adrss reporter[/TD]

[/TR]

[/TABLE]

[TABLE=border: 4, cellpadding: 1, width: 300, height: 100]

[TR]

[TD]Massage to the reporter...[/TD]

[/TR]

[/TABLE]

 

 

 

[TABLE=border: 2, cellpadding: 1, width: 135, height: 20]

[TR]

[TD] Mark deleted (Ctrl+S[/TD]

[/TR]

[/TABLE]

 

[Tech Support]

Model Avatar generation: Generates avatars for Model from videos / albums

 

This makes no sense, you cannot autogenerate model images from screenshots / album images because it will need complicated face detection logic to find good images.

Moreover, we will add model grabbers in 3.9.2, we have already started this. So starting from 3.9.2 there will be no need to populate model info manually. And we will add ability to grab it for existing models.

[Simple]

Hello,

 

Can you make the trailer creation feature customizable also for a specific user ?

 

Thanks

 

[Simple]

Hello,

 

It's possible to make the trailer creation customizable for a specific user ?

 

Thanks

[Tech Support]

Hello,

 

It's possible to make the trailer creation customizable for a specific user ?

 

Thanks

 

Please provide more details, I do not understand what you mean by this.

[Simple]

Hi

 

Sorry for the double post.

 

I would like to know if you can implement something like that :