Unable to access Google Cloud Storage from Google Kubernetes Engine using default service account and Google Cloud Libraries
I have written an application which has the ability to upload images through Google Kubernetes Engine using go lang. Everything else works fine, but when I try to write the image to Google Cloud Storage I'm having issues.
Here is my code to actually use the google storage api in golang:
func putImage(imageURL string, image multipart.File) bool {
fmt.Println("Putting into image location : " + imageURL)
contextBackground := context.Background()
storageClient, err := storage.NewClient(contextBackground)
if err != nil {
fmt.Println("No client.")
return false
}
bucket := storageClient.Bucket("mytestbucketname")
bucketWriter := bucket.Object(imageURL).NewWriter(contextBackground)
bucketWriter.ContentType = "image/jpg"
bucketWriter.CacheControl = "public, max-age=0"
size, err := io.Copy(bucketWriter, image)
if err != nil {
fmt.Println("failed to put image")
return false
}
fmt.Println(size)
fmt.Println("Successfully put image")
bucketWriter.Close()
return true
}
The above function always returns true and the size is always greater than 0. However, when I check the bucket, it actually has nothing. So I researched and realized that the default service account only has read access to Cloud Storage. This is weird because it should return false or size should be 0 or even output a permission denied error.
The official cloud.google.com/go/storage API states that the BucketHandle returned by the Bucket() function doesn't actually perform any network operations, which makes sense as to why I'm not getting a permission denied error (?). So I decided to check if I actually got anything from the client or the bucket, just to see if the read permission worked. I added the following code:
attr, err := bucket.Attrs(contextBackground)
if err != nil {
fmt.Println(err.Error())
}
fmt.Println("bucket name : " + attr.Name)
At this point I started getting fatal errors which shut down my app. The error I'm getting when trying to retrieve the bucket's properties is:
Get https://www.googleapis.com/storage/v1/b/mytestbucketname?alt=json&prettyPrint=false&projection=full: oauth2: cannot fetch token: Post https://oauth2.googleapis.com/token: x509: certificate signed by unknown authority
So I think I need to add the ca certificate to the image. But this doesn't sound logical to me, because if I'm running an image in Google Kubernetes Engine and I'm accessing my Google Cloud Storage, given that it already has a default service account with read permissions, why would I need a certificate? I created a new cluster with version 1.12.8-gke.10 and made sure that issuing client certificates was disabled and that I had read access to the store and still got the same error. I added this line to my docker file and still got the same error:
RUN apk --no-cache add ca-certificates && update-ca-certificates
I've been here for two days, but now I'm running out of ideas. My question is, why do I keep getting the "x.509 certificate signed by an unknown authority" error when I try to access the bucket properties from Kubernetes Engine when I use the default permissions? Technically, getting bucket properties is just a read function, I should be able to use the default permissions right? If anyone has any ideas or encounters the same problem, please give me some help! Thanks!
So I finally figured it out, and if anyone has this problem too, I'll leave it here. It was my fault that I was careless, but after reading this question:
Unable to exchange AccessToken from Google API inside Docker container
I was able to narrow it down to a certificate issue. Actually it was because I didn't install the ca certificate properly. Because I'm using a multi-stage build in the docker file, I misplaced the following line of code:
RUN apk --no-cache add ca-certificates && update-ca-certificates
When placed correctly, it works!