Adding Cloudflare R2 Storage

jim99 · November 30, 2023

Hey everyone,

I saw Cloudflare R2 supports S3 (https://developers.cloudflare.com/r2/api/s3/api/), also it's provide free egress bandwidth, this thing I want to advantage of by streaming videos. ALos saw some KVS CMS sites using this like tktube.

Also while adding R2 few things I didn't understand. Like Getting error on CDN Script Control & S3 region name.

What any other things I'm missing to add there? Can anyone help me in this case?

Thanks

Tech Support · November 30, 2023

- CDN control script is needed for content protection, this means you have to develop this script yourself using the documentation provided by CDN vendor. If you want to use storage server without link protection, you can switch Streaming type option to Direct URL (no protection), then you don't need CDN control script.

- Put / get / chmod /delete operation failed error means that KVS was not able to run basic operations with the provided details. You can find more info about error in Administration -> Installation info -> Logs -> debug_new_server.txt file. Then depending on error message from S3 API you need to find out what is wrong, could be connection issue (if wrong access point provided), or authorization issue (if wrong access code / secret keys provided).

Based on your details I think you also need to provide some Endpoint URL, because if this is not provided, the API will think you are trying to use Amazon S3 infrastructure.

BTW, in the documentation link you provided it says this:

Quote

The API is available via the https://<ACCOUNT_ID>.r2.cloudflarestorage.com endpoint. Find your account ID in the Cloudflare dashboard.

So this is likely the endpoint URL, you need to check your ACCOUNT_ID and replace in the URL.

hbarnetworks · November 30, 2023

You break the ToS if you stream videos from the R2. You need to use cloudflare stream. Doing the other will get you banned from using their service.

jim99 · December 2, 2023

On 11/30/2023 at 6:13 PM, Tech Support said:

So this is likely the endpoint URL, you need to check your ACCOUNT_ID and replace in the URL.

Thanks for the clarification. but now it's showing, this error.

Quote

Put / get / chmod / delete operations failed, insufficient permissions possible

also checked debug_new_server.txt file it's showing this

[2023-12-02 05:24:06] [3681748] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
[2023-12-02 05:24:39] [3681748] test_connection_detailed
[2023-12-02 05:24:39] [3681748] put_file: test-storage-.dat
[2023-12-02 05:24:39] [3681748] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
[2023-12-02 05:25:35] [3681748] test_connection_detailed
[2023-12-02 05:25:36] [3681748] put_file: test-storage-.dat
[2023-12-02 05:25:36] [3681748] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
[2023-12-02 05:28:26] [3681748] test_connection_detailed
[2023-12-02 05:28:26] [3681748] put_file: test-storage-.dat
[2023-12-02 05:28:26] [3681748] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI
[2023-12-02 05:46:08] [3814893] test_connection_detailed
[2023-12-02 05:46:08] [3814893] put_file: test-storage-.dat
[2023-12-02 05:46:08] [3814893] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI
[2023-12-02 05:46:22] [3814893] test_connection_detailed
[2023-12-02 05:46:22] [3814893] put_file: test-storage-.dat
[2023-12-02 05:46:22] [3814893] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI
[2023-12-02 05:47:23] [3814925] test_connection_detailed
[2023-12-02 05:47:23] [3814925] put_file: test-storage-.dat
[2023-12-02 05:47:23] [3814925] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI
[2023-12-02 05:47:32] [3814925] test_connection_detailed
[2023-12-02 05:47:32] [3814925] put_file: test-storage-.dat
[2023-12-02 05:47:32] [3814925] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI
[2023-12-02 05:47:50] [3814925] test_connection_detailed
[2023-12-02 05:47:51] [3814925] put_file: test-storage-.dat
[2023-12-02 05:47:51] [3814925] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI
[2023-12-02 05:49:13] [3814925] test_connection_detailed
[2023-12-02 05:49:13] [3814925] put_file: test-storage-.dat
[2023-12-02 05:49:13] [3814925] ERROR: exception when using S3 API: Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI

Can you please help me in this case?

jim99 · December 2, 2023

On 12/1/2023 at 2:10 AM, hbarnetworks said:

You break the ToS if you stream videos from the R2. You need to use cloudflare stream. Doing the other will get you banned from using their service.

Yess, But some big sites use this. & they have 100+PB monthly bandwidth without changing accounts.

Why not give a try.

Tech Support · December 2, 2023

I think the issue is this error:

5 hours ago, jim99 said:

Custom endpoint `eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com` was not a valid URI

You need to specify URL in S3 endpoint field, likely this one (add https:// or http://):

https://eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com

jim99 · December 3, 2023

On 12/2/2023 at 5:10 PM, Tech Support said:

You need to specify URL in S3 endpoint field, likely this one (add https:// or http://):

But, Now it's getting "Put / get / chmod / delete operations failed, insufficient permissions possible" error. But already given Admin read & Write access .

Also, here is server log. Is there something wrong in permission?

[2023-12-03 05:33:20] [1511020] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
[2023-12-03 05:33:29] [1463806] test_connection_detailed
[2023-12-03 05:33:30] [1463806] put_file: test-storage-.dat
[2023-12-03 05:33:30] [1463806] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
[2023-12-03 05:33:47] [1511020] test_connection_detailed
[2023-12-03 05:33:47] [1511020] put_file: test-storage-.dat
[2023-12-03 05:33:47] [1511020] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>
[2023-12-03 12:58:16] [3497760] test_connection_detailed
[2023-12-03 12:58:17] [3497760] put_file: test-storage-.dat
[2023-12-03 12:58:17] [3497760] ERROR: exception when using S3 API: Error executing "PutObject" on "https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat"; AWS HTTP error: Client error: `PUT https://videoadd.eb875dee0078ht4bc386586981082f88a7b1.r2.cloudflarestorage.com/test-storage-.dat` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is  (truncated...)
 InvalidRegionName (client): The region name 'eu40' is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto - <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRegionName</Code><Message>The region name &apos;eu40&apos; is not valid. Must be one of: wnam, enam, weur, eeur, apac, auto</Message></Error>

Tech Support · December 3, 2023

The error message is generic, KVS cannot guess what is wrong with S3 details that you've provided, so shows the most probable message. But in the log file you' have attached you can clearly see the error message is saying about wrong region name.

jim99 · December 3, 2023

1 minute ago, Tech Support said:

wrong region name.

That's the issue,

Because Cloudflare R2 doesn't have any region, So I just entered eu40 which might be causing the issue.
But when they don't have any region which should I enter, can't leave it empty.

So How like tktube using R2 this?

Tech Support · December 4, 2023

Have you tried any of these regions?

12 hours ago, jim99 said:

Must be one of: wnam, enam, weur, eeur, apac, auto

Also the documentation is saying this:

Quote

Bucket region

When using the S3 API, the region for an R2 bucket is auto.

Tanjiro · December 16, 2023

On 11/30/2023 at 7:43 PM, Tech Support said:

- CDN control script is needed for content protection, this means you have to develop this script yourself using the documentation provided by CDN vendor. If you want to use storage server without link protection, you can switch Streaming type option to Direct URL (no protection), then you don't need CDN control script.

- Put / get / chmod /delete operation failed error means that KVS was not able to run basic operations with the provided details. You can find more info about error in Administration -> Installation info -> Logs -> debug_new_server.txt file. Then depending on error message from S3 API you need to find out what is wrong, could be connection issue (if wrong access point provided), or authorization issue (if wrong access code / secret keys provided).

Based on your details I think you also need to provide some Endpoint URL, because if this is not provided, the API will think you are trying to use Amazon S3 infrastructure.

BTW, in the documentation link you provided it says this:

So this is likely the endpoint URL, you need to check your ACCOUNT_ID and replace in the URL.

Thanks to this topic I can setup R2, but there is one small issue that it keeps showing warning sign

WARNING: Content on some storage servers is not protected from direct access.

How can I disable this ? Because R2 is suppose to direct link so it won't be protected right? And I'm fine with that so can I disable that warning sign ?

Tech Support · December 16, 2023

36 minutes ago, Tanjiro said:

Because R2 is suppose to direct link so it won't be protected right?

Right, but there may be a way to generate expired signed URLs, at least this documents seems to describe the similar concept:

https://developers.cloudflare.com/r2/api/s3/presigned-urls/

But you will need to hire a PHP dev to do that.

1 hour ago, Tanjiro said:

And I'm fine with that so can I disable that warning sign ?

Missing the content protection may result in others hotlinking your videos and leaching your money.

desibaba · December 23, 2023

On 12/16/2023 at 4:55 PM, Tanjiro said:

Thanks to this topic I can setup R2, but there is one small issue that it keeps showing warning sign

WARNING: Content on some storage servers is not protected from direct access.

How can I disable this ? Because R2 is suppose to direct link so it won't be protected right? And I'm fine with that so can I disable that warning sign ?

From where did you get the CDN control script?

phpcoder · December 25, 2023

<?php
define('CLOUDFLARE_ACCOUNT_ID', '<place your Cloudflare account ID here>');
define('CLOUDFLARE_API_KEY', '<place your Cloudflare API key here>');
define('CLOUDFLARE_API_URL', 'https://<ACCOUNT_ID>.r2.cloudflarestorage.com');

// Utility function to handle Cloudflare R2 authentication and API requests
function cloudflare_api_request($endpoint, $method = 'GET', $headers = [], $body = null) {
    $ch = curl_init();

    // Set the URL and other appropriate options
    curl_setopt($ch, CURLOPT_URL, CLOUDFLARE_API_URL . $endpoint);
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);
    curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

    // Include body if it's a POST request
    if ($method === 'POST' && $body !== null) {
        curl_setopt($ch, CURLOPT_POSTFIELDS, $body);
    }

    // Execute and close the cURL session
    $response = curl_exec($ch);
    curl_close($ch);

    return $response;
}

// Function to generate a presigned URL for Cloudflare R2 buckets
function cloudflare_generate_presigned_url($resourcePath, $expires, $method = 'GET') {
    // Calculate expiry time
    $expiresAt = time() + $expires;

    // Construct the presigned URL
    $presignedUrl = CLOUDFLARE_API_URL . '/' . $resourcePath . '?X-Amz-Date=' . time() . '&X-Amz-Expires=' . $expiresAt;

    // Add signature to the URL (this is a placeholder, actual implementation required)
    // $signature = generate_signature($resourcePath, $expiresAt); // Implement this function based on Cloudflare's signing algorithm
    // $presignedUrl .= '&X-Amz-Signature=' . $signature;

    return $presignedUrl;
}

// Example usage of the API request function
$response = cloudflare_api_request('/your-endpoint', 'GET');

// Example usage of the presigned URL function
$presignedUrl = cloudflare_generate_presigned_url('path/to/resource', 3600); // 3600 seconds = 1 hour

Tech Support · December 26, 2023

Thanks for sharing code, but it doesn't look like full code. It looks like generate_signature() function is missing, that will actually 'sign' the request.

Tanjiro · December 26, 2023

On 12/23/2023 at 11:59 PM, desibaba said:

From where did you get the CDN control script?

There's no need, Streaming type you choose Direct instead CDN option

Tech Support · December 26, 2023

4 hours ago, Tanjiro said:

There's no need, Streaming type you choose Direct instead CDN option

Yes, this is one option, but then there is no support for content protection. We may need to take a look at this, where we can get an account for storing content?

Tanjiro · January 1

On 12/26/2023 at 7:11 PM, Tech Support said:

Yes, this is one option, but then there is no support for content protection

My site is very very small so I don't think I need content protection, besides with my knowledge level I can't figure out how to protect video content with R2 cloudflare, nor hire a dev php because I don't have enough money, and outside dev may risky to trust them, if I have to hire someone to do it it would be from KVS, but like I said my site is still so tiny, I hope in future my site can grow bigger so I can hire KVS dev to do it

On 12/26/2023 at 7:11 PM, Tech Support said:

We may need to take a look at this, where we can get an account for storing content?

I'm not sure your question. It's R2 Cloudflare, it's very cheap and efficient, all I need is a cloudflare account, and create bucket R2 and fill in credentials (s3 key, secret), R2 is compatible with s3 so it can connected, user upload image/video and it will store it R2 bucket, unlimited storage, pay as you go, much cheaper than s3 itself

Tech Support · January 2

OK, added this to our TODO list, will take a look when have time.

viper · January 25

On 1/2/2024 at 7:16 PM, Tech Support said:

OK, added this to our TODO list, will take a look when have time.

Any update? For setup, r2 with kvs, i need help

Tech Support · January 25

Sorry, no time to look into this right now.

viper · January 30

video not playing

is their anything wrong?

viper · January 30

debug_storage_server_5.txt

Tech Support · January 31

Cloudflare returns 400 error when trying to access video file. Looks to be configuration issue at their end.

viper · January 31

Now ok but facing other issue

1. orginal file size - 191.94 mb

but after upload in add video section its 203.73 mb and in r2 bucket showing 213.62 MB

2. Where screenshot store? in r2 storage or in my local server?

Sign In

Adding Cloudflare R2 Storage

Recommended Posts

jim99

Tech Support

hbarnetworks

jim99

jim99

Tech Support

jim99

Tech Support

jim99

Tech Support

Bucket region

Tanjiro

Tech Support

desibaba

phpcoder

Tech Support

Tanjiro

Tech Support

Tanjiro

Tech Support

viper

Tech Support

viper

viper

Tech Support

viper

Join the conversation

Forum

Activity

Support