I noticed that my geolocation was not being set. I tried trusting the IP ranges for my load balancers as well, but that did not work. I think an additional IP for my cloud run instance gets added as advised by this article, and that was hard to find documentation for a specific range.
Ended up just taking the first IP address (which in most cases should be the client's original IP) from the X-Forwarded-For header.
map $http_x_forwarded_for $first_x_forwarded_for {
"~^(?P<first>[^,]+)" $first;
default $http_x_forwarded_for;
}
And send that as the Real-IP
proxy_set_header X-Real-IP $first_x_forwarded_for;