Jump to content

Adding Cloudflare R2 Storage

jim99

By jim99
in Technical Support

 Share

Recommended Posts

jim99 Newbie

jim99

Posted
Posted

Hey everyone, 

I saw Cloudflare R2 supports S3 (https://developers.cloudflare.com/r2/api/s3/api/), also it's provide free egress  bandwidth, this thing I want to advantage of by streaming videos. ALos saw some KVS CMS sites using this like tktube. 

Also while adding R2 few things I didn't understand. Like Getting error on CDN Script Control & S3 region name. 

What any other things I'm missing to add there? Can anyone help me in this case? 

Thanks

spacer.png

 

   


spacer.png

Link to comment
Share on other sites
Tech Support Mentor

Tech Support

Posted
Posted

- CDN control script is needed for content protection, this means you have to develop this script yourself using the documentation provided by CDN vendor. If you want to use storage server without link protection, you can switch Streaming type option to Direct URL (no protection), then you don't need CDN control script.

- Put / get / chmod /delete operation failed error means that KVS was not able to run basic operations with the provided details. You can find more info about error in Administration -> Installation info -> Logs -> debug_new_server.txt file. Then depending on error message from S3 API you need to find out what is wrong, could be connection issue (if wrong access point provided), or authorization issue (if wrong access code / secret keys provided).

Based on your details I think you also need to provide some Endpoint URL, because if this is not provided, the API will think you are trying to use Amazon S3 infrastructure.

BTW, in the documentation link you provided it says this:

Quote

The API is available via the https://<ACCOUNT_ID>.r2.cloudflarestorage.com endpoint. Find your account ID in the Cloudflare dashboard.

So this is likely the endpoint URL, you need to check your ACCOUNT_ID and replace in the URL.

  • Like 1
Link to comment
Share on other sites
jim99 Newbie

jim99

Posted
  • Author
Posted
On 11/30/2023 at 6:13 PM, Tech Support said:

So this is likely the endpoint URL, you need to check your ACCOUNT_ID and replace in the URL.

Thanks for the clarification. but now it's showing, this error.

Quote

Put / get / chmod / delete operations failed, insufficient permissions possible

also checked debug_new_server.txt file it's showing this 


  

  • [2023-12-02 05:24:06] [3681748] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
[2023-12-02 05:24:39] [3681748] test_connection_detailed
[2023-12-02 05:24:39] [3681748] put_file: test-storage-.dat
[2023-12-02 05:24:39] [3681748] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
[2023-12-02 05:25:35] [3681748] test_connection_detailed
[2023-12-02 05:25:36] [3681748] put_file: test-storage-.dat
[2023-12-02 05:25:36] [3681748] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
[2023-12-02 05:28:26] [3681748] test_connection_detailed
[2023-12-02 05:28:26] [3681748] put_file: test-storage-.dat
[2023-12-02 05:28:26] [3681748] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI
[2023-12-02 05:46:08] [3814893] test_connection_detailed
[2023-12-02 05:46:08] [3814893] put_file: test-storage-.dat
[2023-12-02 05:46:08] [3814893] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI
[2023-12-02 05:46:22] [3814893] test_connection_detailed
[2023-12-02 05:46:22] [3814893] put_file: test-storage-.dat
[2023-12-02 05:46:22] [3814893] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI
[2023-12-02 05:47:23] [3814925] test_connection_detailed
[2023-12-02 05:47:23] [3814925] put_file: test-storage-.dat
[2023-12-02 05:47:23] [3814925] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI
[2023-12-02 05:47:32] [3814925] test_connection_detailed
[2023-12-02 05:47:32] [3814925] put_file: test-storage-.dat
[2023-12-02 05:47:32] [3814925] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI
[2023-12-02 05:47:50] [3814925] test_connection_detailed
[2023-12-02 05:47:51] [3814925] put_file: test-storage-.dat
[2023-12-02 05:47:51] [3814925] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI
[2023-12-02 05:49:13] [3814925] test_connection_detailed
[2023-12-02 05:49:13] [3814925] put_file: test-storage-.dat
[2023-12-02 05:49:13] [3814925] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI

Can you please help me in this case? 

Link to comment
Share on other sites
jim99 Newbie

jim99

Posted
  • Author
Posted
On 12/1/2023 at 2:10 AM, hbarnetworks said:

You break the ToS if you stream videos from the R2. You need to use cloudflare stream. Doing the other will get you banned from using their service.

Yess, But some big sites use this. & they have 100+PB monthly bandwidth without changing accounts. 

Why not give a try. 

 

Link to comment
Share on other sites
Tech Support Mentor

Tech Support

Posted
Posted

I think the issue is this error:

5 hours ago, jim99 said:

Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI

You need to specify URL in S3 endpoint field, likely this one (add https:// or http://):

https://eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com

  • Like 1
Link to comment
Share on other sites
jim99 Newbie

jim99

Posted
  • Author
Posted
On 12/2/2023 at 5:10 PM, Tech Support said:

You need to specify URL in S3 endpoint field, likely this one (add https:// or http://):

 

But, Now it's getting "Put / get / chmod / delete operations failed, insufficient permissions possible" error. But already given Admin read & Write access bV7jI0.jpg. 

Also, here is server log. Is there something wrong in permission? 

  

[2023-12-03 05:33:20] [1511020] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
[2023-12-03 05:33:29] [1463806] test_connection_detailed
[2023-12-03 05:33:30] [1463806] put_file: test-storage-.dat
[2023-12-03 05:33:30] [1463806] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
[2023-12-03 05:33:47] [1511020] test_connection_detailed
[2023-12-03 05:33:47] [1511020] put_file: test-storage-.dat
[2023-12-03 05:33:47] [1511020] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
[2023-12-03 12:58:16] [3497760] test_connection_detailed
[2023-12-03 12:58:17] [3497760] put_file: test-storage-.dat
[2023-12-03 12:58:17] [3497760] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
Link to comment
Share on other sites
jim99 Newbie

jim99

Posted
  • Author
Posted
1 minute ago, Tech Support said:

wrong region name.

That's the issue, 

Because Cloudflare R2 doesn't have any region, So I just entered eu40 which might be causing the issue. 
But when they don't have any region which should I enter, can't leave it empty. 

So How like tktube using R2 this? 

Link to comment
Share on other sites
  • 2 weeks later...
Tanjiro Enthusiast

Tanjiro

Posted
Posted
On 11/30/2023 at 7:43 PM, Tech Support said:

- CDN control script is needed for content protection, this means you have to develop this script yourself using the documentation provided by CDN vendor. If you want to use storage server without link protection, you can switch Streaming type option to Direct URL (no protection), then you don't need CDN control script.

- Put / get / chmod /delete operation failed error means that KVS was not able to run basic operations with the provided details. You can find more info about error in Administration -> Installation info -> Logs -> debug_new_server.txt file. Then depending on error message from S3 API you need to find out what is wrong, could be connection issue (if wrong access point provided), or authorization issue (if wrong access code / secret keys provided).

Based on your details I think you also need to provide some Endpoint URL, because if this is not provided, the API will think you are trying to use Amazon S3 infrastructure.

BTW, in the documentation link you provided it says this:

So this is likely the endpoint URL, you need to check your ACCOUNT_ID and replace in the URL.

Thanks to this topic I can setup R2, but there is one small issue that it keeps showing warning sign

 WARNING: Content on some storage servers is not protected from direct access.

How can I disable this ? Because R2 is suppose to direct link so it won't be protected right? And I'm fine with that so can I disable that warning sign ?

Link to comment
Share on other sites
Tech Support Mentor

Tech Support

Posted
Posted
36 minutes ago, Tanjiro said:

Because R2 is suppose to direct link so it won't be protected right?

Right, but there may be a way to generate expired signed URLs, at least this documents seems to describe the similar concept:

https://developers.cloudflare.com/r2/api/s3/presigned-urls/

But you will need to hire a PHP dev to do that.

1 hour ago, Tanjiro said:

And I'm fine with that so can I disable that warning sign ?

Missing the content protection may result in others hotlinking your videos and leaching your money.

  • Like 1
Link to comment
Share on other sites
desibaba Rookie

desibaba

Posted
Posted
On 12/16/2023 at 4:55 PM, Tanjiro said:

Thanks to this topic I can setup R2, but there is one small issue that it keeps showing warning sign

 WARNING: Content on some storage servers is not protected from direct access.

How can I disable this ? Because R2 is suppose to direct link so it won't be protected right? And I'm fine with that so can I disable that warning sign ?

From where did you get the CDN control script?

Link to comment
Share on other sites
phpcoder Newbie

phpcoder

Posted
Posted 
<?php
define('CLOUDFLARE_ACCOUNT_ID', '<place your Cloudflare account ID here>');
define('CLOUDFLARE_API_KEY', '<place your Cloudflare API key here>');
define('CLOUDFLARE_API_URL', 'https://<ACCOUNT_ID>.r2.cloudflarestorage.com');

// Utility function to handle Cloudflare R2 authentication and API requests
function cloudflare_api_request($endpoint, $method = 'GET', $headers = [], $body = null) {
    $ch = curl_init();

    // Set the URL and other appropriate options
    curl_setopt($ch, CURLOPT_URL, CLOUDFLARE_API_URL . $endpoint);
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);
    curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

    // Include body if it's a POST request
    if ($method === 'POST' && $body !== null) {
        curl_setopt($ch, CURLOPT_POSTFIELDS, $body);
    }

    // Execute and close the cURL session
    $response = curl_exec($ch);
    curl_close($ch);

    return $response;
}

// Function to generate a presigned URL for Cloudflare R2 buckets
function cloudflare_generate_presigned_url($resourcePath, $expires, $method = 'GET') {
    // Calculate expiry time
    $expiresAt = time() + $expires;

    // Construct the presigned URL
    $presignedUrl = CLOUDFLARE_API_URL . '/' . $resourcePath . '?X-Amz-Date=' . time() . '&X-Amz-Expires=' . $expiresAt;

    // Add signature to the URL (this is a placeholder, actual implementation required)
    // $signature = generate_signature($resourcePath, $expiresAt); // Implement this function based on Cloudflare's signing algorithm
    // $presignedUrl .= '&X-Amz-Signature=' . $signature;

    return $presignedUrl;
}

// Example usage of the API request function
$response = cloudflare_api_request('/your-endpoint', 'GET');

// Example usage of the presigned URL function
$presignedUrl = cloudflare_generate_presigned_url('path/to/resource', 3600); // 3600 seconds = 1 hour

 

Link to comment
Share on other sites
Tanjiro Enthusiast

Tanjiro

Posted
Posted
On 12/26/2023 at 7:11 PM, Tech Support said:

Yes, this is one option, but then there is no support for content protection

My site is very very small so I don't think I need content protection, besides with my knowledge level I can't figure out how to protect video content with R2 cloudflare, nor hire a dev php because I don't have enough money, and outside dev may risky to trust them, if I have to hire someone to do it it would be from KVS, but like I said my site is still so tiny, I hope in future my site can grow bigger so I can hire KVS dev to do it

On 12/26/2023 at 7:11 PM, Tech Support said:

We may need to take a look at this, where we can get an account for storing content?

I'm not sure your question. It's R2 Cloudflare, it's very cheap and efficient, all I need is a cloudflare account, and create bucket R2 and fill in credentials (s3 key, secret), R2 is compatible with s3 so it can connected, user upload image/video and it will store it R2 bucket, unlimited storage, pay as you go, much cheaper than s3 itself

Link to comment
Share on other sites
  • 4 weeks later...
viper Enthusiast

viper

Posted
Posted

Now ok but facing other issue

1. orginal file size - 191.94 mb

but after upload in add video section its 203.73 mb and in r2 bucket showing 213.62 MB

2. Where screenshot store? in r2 storage or in my local server?

Screenshot_2.jpg

Screenshot_3.jpg

Screenshot_4.jpg

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing
×
×
  • Create New...