Jump to content

KVS Grabbers


Recommended Posts

I installed the latest yt-dlp but im not seeing any of the new grabbers in the kvs repository selection. I only see 3 album options and 19 video options. Im on the 6.1.2 now.

How do I use any of the grabbers supported by yt-dlp? Is there any documentation other than "how to create a custom grabber"?

Link to comment
Share on other sites

I made a custom grabber for the site I mentioned and Im getting a 'Your admin session has expired, please login once again' But I know thats likely a separate issue. 

Just to share and potentially debug for future users looking to use Spankbang for grabbers, hopefully i did it right.:


 

<?php

// when you change classname, change it at the very bottom as well in this line:
// $grabber = new CustomGrabberYoutube();
class CustomGrabberSpankBang extends KvsGrabberVideoYDL
{
   // ===============================================================================================================
   // infrastructure methods
   // ===============================================================================================================

   public function get_grabber_id()
   {
       //prefix your grabber ID with "custom_"
       return "custom_videos_spankbang";
   }

   public function get_grabber_name()
   {
       // name displayed in admin panel
       return "spankbang.com";
   }

   public function get_grabber_version()
   {
       // this is required for grabbers that are autoupdated from KVS
       return "1";
   }

   public function get_grabber_domain()
   {
       // domain name, KVS will check this to find out if this grabber is suitable for the given URL
       return "spankbang.com";
   }

   public function get_supported_url_patterns()
   {
       // returns list of regexp patterns that describe video URLs, for spankbang this pattern will match
       // https://spankbang.com/94f5b/video/clapping+that+big+ass
       return array('/https?:\/\/(?:[^\/]+\.)?spankbang\.com\/(?:[\da-z]+\/(?:video|play|embed)\b|[\da-z]+-(?:[\da-z]+)\/playlist\/[^\/?#&]+)/i');

   }

   public function can_grab_description()
   {
       // return true if your grabber is going to provide description for each video
       return true;
   }

   public function can_grab_categories()
   {
       // return true if your grabber is going to provide categories for each video
       return false;
   }

   public function can_grab_tags()
   {
       // return true if your grabber is going to provide tags for each video
       return false;
   }

   public function can_grab_models()
   {
       // return true if your grabber is going to provide models for each video
       return false;
   }

   public function can_grab_content_source()
   {
       // return true if your grabber is going to provide content source for each video
       return false;
   }

   public function can_grab_date()
   {
       // return true if your grabber is going to provide date for each video
       return true;
   }

   public function can_grab_rating()
   {
       // return true if your grabber is going to provide rating for each video
       return false;
   }

   public function can_grab_views()
   {
       // return true if your grabber is going to provide views for each video
       return false;
   }


   public function can_grab_video_files()
   {
       // this should be true for youtube-dl
       return true;
   }

   public function get_supported_qualities()
   {
       // list of supported video qualities, should match what youtube-dl returns in its info under formats
       // run this command:
       // youtube-dl --dump-json https://www.youtube.com/watch?v=PhDXRCLsqz4 >> test.json
       // and open test.json in Firefox, find "formats" array and look into the available formats
       // youtube has too many formats, KVS only supports formats with "ext"="mp4"
       // you can list them here and you will be able to select from them in grabber settings
       return array('360p', '480p', '720p', '1080p');
   }

   public function get_downloadable_video_format()
   {
       // for youtube-dl grabber KVS only supports mp4 formats
       return 'mp4';
   }

   public function can_grab_lists()
   {
       // return true if you want to allow this grabber to grab lists and thus be used on autopilot
       // if true, you will also need to implement grab_list() method - see below
       return true;
   }

   // ===============================================================================================================
   // parsing methods - modify if you need to parse lists or add additional info
   // ===============================================================================================================

   public function grab_list($list_url, $limit)
   {
       // this method is used to grab lists of videos from the given list URL
       // $limit parameter means the number of videos to grab (including pagination)
       // if $limit == 0, then you just need to find all videos on the given URL, no need to care about pagination

       $result = new KvsGrabberListResult();

       // $page_content here is the HTML code of the given page
       $page_content = $this->load_page($list_url);

       // parse $page_content and add all video URLs to the result
       // consider pagination if needed
       // you can use $this->load_page($list_url) method to get HTML from any URL
       $result->add_content_page("https://spankbang.com/video1");
       $result->add_content_page("https://spankbang.com/video2");
       $result->add_content_page("https://spankbang.com/video3");

       return $result;
   }

   protected function grab_video_data_impl($page_url, $tmp_dir)
   {
       // by default the base class will populate these fields (if provided by youtube-dl):
       // - title
       // - MP4 video files for the qualities listed in get_supported_qualities() function
       // - description (should be enabled in can_grab_description() function)
       // - date (should be enabled in can_grab_date() function)
       // - tags (should be enabled in can_grab_tags() function)
       // - categories (should be enabled in can_grab_categories() function)

       $result = parent::grab_video_data_impl($page_url, $tmp_dir);
       if ($result->get_error_code() > 0)
       {
           return $result;
       }

       // do any custom grabbing here for additional fields, which are not supported by youtube-dl
       // $page_content here is the HTML code of the given video page
       //$page_content = $this->load_page($page_url);

       // parse HTML code and set additional data into $result, e.g. data which is not provided by youtube-dl
       //$result->set_rating(85);
       //$result->set_votes(10);
       //$result->set_views(123874);
       //$result->set_content_source("Content Source Name");
       //$result->add_model("Model 1");
       //$result->add_model("Model 2");

       return $result;
   }
}

$grabber = new CustomGrabberSpankBang();
KvsGrabberFactory::register_grabber_class(get_class($grabber));
return $grabber;

Spankbangs "test" line in yt-dlp documentation shows the following information for the extractor:
 

class SpankBangIE(InfoExtractor):
    _VALID_URL = r'''(?x)
                    https?://
                        (?:[^/]+\.)?spankbang\.com/
                        (?:
                            (?P<id>[\da-z]+)/(?:video|play|embed)\b|
                            [\da-z]+-(?P<id_2>[\da-z]+)/playlist/[^/?#&]+
                        )
                    '''
    _TESTS = [{
        'url': 'https://spankbang.com/56b3d/video/the+slut+maker+hmv',
        'md5': '2D13903DE4ECC7895B5D55930741650A',
        'info_dict': {
            'id': '56b3d',
            'ext': 'mp4',
            'title': 'The Slut Maker HMV',
            'description': 'Girls getting converted into cock slaves.',
            'thumbnail': r're:^https?://.*\.jpg$',
            'uploader': 'Mindself',
            'uploader_id': 'mindself',
            'timestamp': 1617109572,
            'upload_date': '20210330',
            'age_limit': 18,
        }
    }, {
        # 480p only
        'url': 'http://spankbang.com/1vt0/video/solvane+gangbang',
        'only_matching': True,
    }, {
        # no uploader
        'url': 'http://spankbang.com/lklg/video/sex+with+anyone+wedding+edition+2',
        'only_matching': True,
    }, {
        # mobile page
        'url': 'http://m.spankbang.com/1o2de/video/can+t+remember+her+name',
        'only_matching': True,
    }, {
        # 4k
        'url': 'https://spankbang.com/1vwqx/video/jade+kush+solo+4k',
        'only_matching': True,
    }, {
        'url': 'https://m.spankbang.com/3vvn/play/fantasy+solo/480p/',
        'only_matching': True,
    }, {
        'url': 'https://m.spankbang.com/3vvn/play',
        'only_matching': True,
    }, {
        'url': 'https://spankbang.com/2y3td/embed/',
        'only_matching': True,
    }, {
        'url': 'https://spankbang.com/2v7ik-7ecbgu/playlist/latina+booty',
        'only_matching': True,
    }]

 

Link to comment
Share on other sites

14 hours ago, ChrisAngel said:

How do I instruct the default video grabber? I dont see any fields to point it to a specific source.

You don't need to instruct it. You just try to import a video URL from a site domain.com. KVS will check, if there is a grabber designed for domain.com? If yes, KVS would use this grabber. If not, KVS would check if there is a default video grabbers installed? If yes, KVS would try using default videos grabber for this URL. If default video grabber can grab video files from that URL, then it will work. Otherwise it will show error that this site cannot be grabbed.

Link to comment
Share on other sites

I was able to add the custom grabber successfully after fixing my cloudflare issue however, I would like to be able to see an example of how the grabber for pornhub or xvideos was constructed so I can add embed mode. Can you supply me with that code?

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...