How to create Google Cloud Storage signed URL with Elixir or Erlang?


Todd Harding

I'm currently trying to get my Elixir web server to generate signed urls for Google Cloud Storage so that expired file urls can be generated. Unfortunately, when I try to use the generated url, I get the following error:

<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.
</Message>

I can generate signed urls via the gsutil tool, albeit slow, but also via the python example given here:

Google Cloud Storage Signed URL Example

My current implementation in Elixir is based on the above Python example and looks like this:

@default_expiration 1000
  def construct_string(http_verb, content_md5, content_type, expiration_timestamp, canonicalized_extension_headers, canonicalized_resource) do
    "#{http_verb}\n
    #{content_md5}\n
    #{content_type}\n
    #{expiration_timestamp}\n
    #{canonicalized_extension_headers}
    #{canonicalized_resource}"
  end

  def load_secret_pem do
    load_local_key_file("/path/to/key")
  end

  def load_local_key_file(path) do
    {ok, pem_bin} = File.read(path)
    [rsa_entry] = :public_key.pem_decode(pem_bin)
    key = :public_key.pem_entry_decode(rsa_entry)
  end

  def base64Sign(plaintext) do
    key = load_secret_pem()
    signature_bytes = :public_key.sign(plaintext, :sha256, key )
    Base.url_encode64(signature_bytes)
    |> String.replace("-", "%2B")
    |> String.replace("_", "%2F")
    |> URI.encode_www_form
  end

  def make_url(verb, path, content_md5 \\ "", content_type \\ "") do
    client_id = GCloud.Credentials.client_email() |> URI.encode_www_form
    expiration =  :os.system_time(:seconds) + @default_expiration
    base_url = GCloud.Storage.base_uri() <> path
    signature_string = construct_string(verb, content_md5, content_type, expiration, "", path )
    url_encoded_signature = base64Sign(signature_string)
    IO.puts "#{base_url}?GoogleAccessId=#{client_id}&Expires=#{expiration}&Signature=#{url_encoded_signature}"
  end

How to properly sign a signed URL with Elixir or Erlang?

end it

Your string construction construct_stringmight be doing something you didn't realize. Remember that Python syntax is different and has other opinions about spaces.

defmodule Test do
  def foo(a,b) do
    "#{a}\n
    #{b}"
  end
end
IO.inspect Test.foo(1,2)
# output:
"1\n\n    2"

If you use heredoc instead """, the leading spaces disappear, but the newlines are still repeated. However, this approach might not be a good idea, since if you save the file from a Windows computer, \r\nthe line might end up in the editor, and getting rid of those lines is still an unnecessary hassle.

Instead, I think you should change your approach here to look like this:

def construct_string(http_verb, content_md5, content_type, expiration_timestamp, canonicalized_extension_headers, canonicalized_resource) do
  headers = Enum.join([http_verb, content_md5, content_type, expiration_timestamp], "\n")
  "#{headers}\n#{canonicalized_extension_headers}#{canonicalized_resource}"
end

I'm not sure if there are other bugs, but this immediately stood out to me.

Related


Create signed URL for Google Cloud Storage using NodeJS

Stuck Kennedy I'm trying to create a signature for a privately stored file in Google Cloud Storage; so I can distribute a limited time link. Currently doing this and the signature is too short...where am I going wrong? var crypto = require("crypto"); var ttl

Create signed URL for Google Cloud Storage using NodeJS

Stuck Kennedy I'm trying to create a signature for a privately stored file in Google Cloud Storage; so I can distribute a limited time link. Currently doing this and the signature is too short...where am I going wrong? var crypto = require("crypto"); var ttl

How to shorten google cloud storage signed download url?

Brandon I have an firebaseapp where I use to Firebase Storageupload pictures. URLWhen I use firebase web sdkupload , si returns reasonable : https://firebasestorage.googleapis.com/v0/b/projectId.appspot.com/o/image.jpg?alt=media&token=51183d4a-551a-41e2-b620-1

How to shorten google cloud storage signed download url?

Brandon I have an firebaseapp where I use to Firebase Storageupload pictures. URLWhen I use firebase web sdkupload , si returns reasonable : https://firebasestorage.googleapis.com/v0/b/projectId.appspot.com/o/image.jpg?alt=media&token=51183d4a-551a-41e2-b620-1

How to shorten google cloud storage signed download url?

Brandon I have an firebaseapp where I use to Firebase Storageupload pictures. URLWhen I use firebase web sdkupload , si returns reasonable : https://firebasestorage.googleapis.com/v0/b/projectId.appspot.com/o/image.jpg?alt=media&token=51183d4a-551a-41e2-b620-1

How to shorten google cloud storage signed download url?

Brandon I have an firebaseapp where I use to Firebase Storageupload pictures. URLWhen I use firebase web sdkupload , si returns reasonable : https://firebasestorage.googleapis.com/v0/b/projectId.appspot.com/o/image.jpg?alt=media&token=51183d4a-551a-41e2-b620-1

How to shorten google cloud storage signed download url?

Brandon I have an firebaseapp where I use to Firebase Storageupload pictures. URLWhen I use firebase web sdkupload , si returns reasonable : https://firebasestorage.googleapis.com/v0/b/projectId.appspot.com/o/image.jpg?alt=media&token=51183d4a-551a-41e2-b620-1

How to create a file size limited signed URL for Google Storage?

jz22: I'm trying to create a signed upload URL so that clients can upload files directly to a google bucket, and I want to prevent users from uploading large files. So I want to define the maximum file size that can be uploaded via signed URL with official Go

How to create a file size limited signed URL for Google Storage?

jz22: I'm trying to create a signed upload URL so that clients can upload files directly to a google bucket, and I want to prevent users from uploading large files. So I want to define the maximum file size that can be uploaded via signed URL with official Go

Provide callback URL in Google Cloud Storage signed URL

username When uploading to GCS (Google Cloud Storage) using BlobStore's createUploadURL function , I can provide a callback along with header data that will be sent to the callback URL. There doesn't seem to be a way to use GCS's signed URLs I know there are "

Provide callback URL in Google Cloud Storage signed URL

username When uploading to GCS (Google Cloud Storage) using BlobStore's createUploadURL function , I can provide a callback along with header data that will be sent to the callback URL. There doesn't seem to be a way to use GCS's signed URLs I know there are "

Provide callback URL in Google Cloud Storage signed URL

username When uploading to GCS (Google Cloud Storage) using BlobStore's createUploadURL function , I can provide a callback along with header data that will be sent to the callback URL. There doesn't seem to be a way to use GCS's signed URLs I know there are "

Provide callback URL in Google Cloud Storage signed URL

username When uploading to GCS (Google Cloud Storage) using BlobStore's createUploadURL function , I can provide a callback along with header data that will be sent to the callback URL. There doesn't seem to be a way to use GCS's signed URLs I know there are "

Provide callback URL in Google Cloud Storage signed URL

username When uploading to GCS (Google Cloud Storage) using BlobStore's createUploadURL function , I can provide a callback along with header data that will be sent to the callback URL. There doesn't seem to be a way to use GCS's signed URLs I know there are "

Provide callback URL in Google Cloud Storage signed URL

username When uploading to GCS (Google Cloud Storage) using BlobStore's createUploadURL function , I can provide a callback along with header data that will be sent to the callback URL. There doesn't seem to be a way to use GCS's signed URLs I know there are "

Google storage with signed URL

Bouwan How to upload files of content type mutlipart/formdata ---boundaryStringusing Google Storage's signed URL ? The problem I'm facing is that when we upload the file using the browser, the browser places the mutlipart/formdata ---boundaryStringwhere bounda

Google storage with signed URL

Bouwan How to upload files of content type mutlipart/formdata ---boundaryStringusing Google Storage's signed URL ? The problem I'm facing is that when we upload the file using the browser, the browser places the mutlipart/formdata ---boundaryStringwhere bounda

Google storage with signed URL

Bouwan How to upload a file of content type mutlipart/formdata ---boundaryStringusing signed URL stored by Google ? The problem I'm facing is that when we upload the file using the browser, the browser places the mutlipart/formdata ---boundaryStringwhere bound

Google storage with signed URL

Bouwan How to upload files of content type mutlipart/formdata ---boundaryStringusing Google Storage's signed URL ? The problem I'm facing is that when we upload the file using the browser, the browser places the mutlipart/formdata ---boundaryStringwhere bounda

Google storage with signed URL

Bouwan How to upload files of content type mutlipart/formdata ---boundaryStringusing Google Storage's signed URL ? The problem I'm facing is that when we upload the file using the browser, the browser places the mutlipart/formdata ---boundaryStringwhere bounda