KVS Grabbers

ChrisAngel · February 8

I installed the latest yt-dlp but im not seeing any of the new grabbers in the kvs repository selection. I only see 3 album options and 19 video options. Im on the 6.1.2 now.

How do I use any of the grabbers supported by yt-dlp? Is there any documentation other than "how to create a custom grabber"?

Tech Support · February 8

You need to use Default videos grabber for this.

ChrisAngel · February 8

4 minutes ago, Tech Support said:

You need to use Default videos grabber for this.

Thanks for the quick response. Ok I have enable that option but I am not seeing how I can add "spankbang" as a grabber for instance.

Tech Support · February 8

There will be no specific grabbers added. You need to add default videos grabber and use it for parsing sites that are supported by yt-dlp.

ChrisAngel · February 8

How do I instruct the default video grabber? I dont see any fields to point it to a specific source.

ChrisAngel · February 8

I made a custom grabber for the site I mentioned and Im getting a 'Your admin session has expired, please login once again' But I know thats likely a separate issue.

Just to share and potentially debug for future users looking to use Spankbang for grabbers, hopefully i did it right.:

<?php

// when you change classname, change it at the very bottom as well in this line:
// $grabber = new CustomGrabberYoutube();
class CustomGrabberSpankBang extends KvsGrabberVideoYDL
{
   // ===============================================================================================================
   // infrastructure methods
   // ===============================================================================================================

   public function get_grabber_id()
   {
       //prefix your grabber ID with "custom_"
       return "custom_videos_spankbang";
   }

   public function get_grabber_name()
   {
       // name displayed in admin panel
       return "spankbang.com";
   }

   public function get_grabber_version()
   {
       // this is required for grabbers that are autoupdated from KVS
       return "1";
   }

   public function get_grabber_domain()
   {
       // domain name, KVS will check this to find out if this grabber is suitable for the given URL
       return "spankbang.com";
   }

   public function get_supported_url_patterns()
   {
       // returns list of regexp patterns that describe video URLs, for spankbang this pattern will match
       // https://spankbang.com/94f5b/video/clapping+that+big+ass
       return array('/https?:\/\/(?:[^\/]+\.)?spankbang\.com\/(?:[\da-z]+\/(?:video|play|embed)\b|[\da-z]+-(?:[\da-z]+)\/playlist\/[^\/?#&]+)/i');

   }

   public function can_grab_description()
   {
       // return true if your grabber is going to provide description for each video
       return true;
   }

   public function can_grab_categories()
   {
       // return true if your grabber is going to provide categories for each video
       return false;
   }

   public function can_grab_tags()
   {
       // return true if your grabber is going to provide tags for each video
       return false;
   }

   public function can_grab_models()
   {
       // return true if your grabber is going to provide models for each video
       return false;
   }

   public function can_grab_content_source()
   {
       // return true if your grabber is going to provide content source for each video
       return false;
   }

   public function can_grab_date()
   {
       // return true if your grabber is going to provide date for each video
       return true;
   }

   public function can_grab_rating()
   {
       // return true if your grabber is going to provide rating for each video
       return false;
   }

   public function can_grab_views()
   {
       // return true if your grabber is going to provide views for each video
       return false;
   }


   public function can_grab_video_files()
   {
       // this should be true for youtube-dl
       return true;
   }

   public function get_supported_qualities()
   {
       // list of supported video qualities, should match what youtube-dl returns in its info under formats
       // run this command:
       // youtube-dl --dump-json https://www.youtube.com/watch?v=PhDXRCLsqz4 >> test.json
       // and open test.json in Firefox, find "formats" array and look into the available formats
       // youtube has too many formats, KVS only supports formats with "ext"="mp4"
       // you can list them here and you will be able to select from them in grabber settings
       return array('360p', '480p', '720p', '1080p');
   }

   public function get_downloadable_video_format()
   {
       // for youtube-dl grabber KVS only supports mp4 formats
       return 'mp4';
   }

   public function can_grab_lists()
   {
       // return true if you want to allow this grabber to grab lists and thus be used on autopilot
       // if true, you will also need to implement grab_list() method - see below
       return true;
   }

   // ===============================================================================================================
   // parsing methods - modify if you need to parse lists or add additional info
   // ===============================================================================================================

   public function grab_list($list_url, $limit)
   {
       // this method is used to grab lists of videos from the given list URL
       // $limit parameter means the number of videos to grab (including pagination)
       // if $limit == 0, then you just need to find all videos on the given URL, no need to care about pagination

       $result = new KvsGrabberListResult();

       // $page_content here is the HTML code of the given page
       $page_content = $this->load_page($list_url);

       // parse $page_content and add all video URLs to the result
       // consider pagination if needed
       // you can use $this->load_page($list_url) method to get HTML from any URL
       $result->add_content_page("https://spankbang.com/video1");
       $result->add_content_page("https://spankbang.com/video2");
       $result->add_content_page("https://spankbang.com/video3");

       return $result;
   }

   protected function grab_video_data_impl($page_url, $tmp_dir)
   {
       // by default the base class will populate these fields (if provided by youtube-dl):
       // - title
       // - MP4 video files for the qualities listed in get_supported_qualities() function
       // - description (should be enabled in can_grab_description() function)
       // - date (should be enabled in can_grab_date() function)
       // - tags (should be enabled in can_grab_tags() function)
       // - categories (should be enabled in can_grab_categories() function)

       $result = parent::grab_video_data_impl($page_url, $tmp_dir);
       if ($result->get_error_code() > 0)
       {
           return $result;
       }

       // do any custom grabbing here for additional fields, which are not supported by youtube-dl
       // $page_content here is the HTML code of the given video page
       //$page_content = $this->load_page($page_url);

       // parse HTML code and set additional data into $result, e.g. data which is not provided by youtube-dl
       //$result->set_rating(85);
       //$result->set_votes(10);
       //$result->set_views(123874);
       //$result->set_content_source("Content Source Name");
       //$result->add_model("Model 1");
       //$result->add_model("Model 2");

       return $result;
   }
}

$grabber = new CustomGrabberSpankBang();
KvsGrabberFactory::register_grabber_class(get_class($grabber));
return $grabber;

Spankbangs "test" line in yt-dlp documentation shows the following information for the extractor:

class SpankBangIE(InfoExtractor):
    _VALID_URL = r'''(?x)
                    https?://
                        (?:[^/]+\.)?spankbang\.com/
                        (?:
                            (?P<id>[\da-z]+)/(?:video|play|embed)\b|
                            [\da-z]+-(?P<id_2>[\da-z]+)/playlist/[^/?#&]+
                        )
                    '''
    _TESTS = [{
        'url': 'https://spankbang.com/56b3d/video/the+slut+maker+hmv',
        'md5': '2D13903DE4ECC7895B5D55930741650A',
        'info_dict': {
            'id': '56b3d',
            'ext': 'mp4',
            'title': 'The Slut Maker HMV',
            'description': 'Girls getting converted into cock slaves.',
            'thumbnail': r're:^https?://.*\.jpg$',
            'uploader': 'Mindself',
            'uploader_id': 'mindself',
            'timestamp': 1617109572,
            'upload_date': '20210330',
            'age_limit': 18,
        }
    }, {
        # 480p only
        'url': 'http://spankbang.com/1vt0/video/solvane+gangbang',
        'only_matching': True,
    }, {
        # no uploader
        'url': 'http://spankbang.com/lklg/video/sex+with+anyone+wedding+edition+2',
        'only_matching': True,
    }, {
        # mobile page
        'url': 'http://m.spankbang.com/1o2de/video/can+t+remember+her+name',
        'only_matching': True,
    }, {
        # 4k
        'url': 'https://spankbang.com/1vwqx/video/jade+kush+solo+4k',
        'only_matching': True,
    }, {
        'url': 'https://m.spankbang.com/3vvn/play/fantasy+solo/480p/',
        'only_matching': True,
    }, {
        'url': 'https://m.spankbang.com/3vvn/play',
        'only_matching': True,
    }, {
        'url': 'https://spankbang.com/2y3td/embed/',
        'only_matching': True,
    }, {
        'url': 'https://spankbang.com/2v7ik-7ecbgu/playlist/latina+booty',
        'only_matching': True,
    }]

Tech Support · February 8

14 hours ago, ChrisAngel said:

How do I instruct the default video grabber? I dont see any fields to point it to a specific source.

You don't need to instruct it. You just try to import a video URL from a site domain.com. KVS will check, if there is a grabber designed for domain.com? If yes, KVS would use this grabber. If not, KVS would check if there is a default video grabbers installed? If yes, KVS would try using default videos grabber for this URL. If default video grabber can grab video files from that URL, then it will work. Otherwise it will show error that this site cannot be grabbed.

ChrisAngel · February 8

I was able to add the custom grabber successfully after fixing my cloudflare issue however, I would like to be able to see an example of how the grabber for pornhub or xvideos was constructed so I can add embed mode. Can you supply me with that code?

Tech Support · February 9

Please create support ticket.

Donglong · February 9

I'm hoping they can sounds like your going in the right direction all you need is the code I don't see why they shouldn't be able to help you

desibaba · February 29

On 2/8/2024 at 8:39 AM, ChrisAngel said:

('/https?:\/\/(?:[^\/]+\.)?spankbang\.com\/(?:[\da-z]+\/(?:video|play|embed)\b|[\da-z]+-(?:[\da-z]+)\/playlist\/[^\/?#&]+)/i');

how did you arrive at this URL pattern?

Sign In

KVS Grabbers

Recommended Posts

ChrisAngel

Link to comment

Share on other sites

Tech Support

Link to comment

Share on other sites

ChrisAngel

Link to comment

Share on other sites

Tech Support

Link to comment

Share on other sites

ChrisAngel

Link to comment

Share on other sites

ChrisAngel

Link to comment

Share on other sites

Tech Support

Link to comment

Share on other sites

ChrisAngel

Link to comment

Share on other sites

Tech Support

Link to comment

Share on other sites

Donglong

Link to comment

Share on other sites

desibaba

Link to comment

Share on other sites

Join the conversation

Forum

Activity

Support