I installed datastore emulator in google app engine. After I install the datastore emulator based on the tutorial from https://cloud.google.com/datastore/docs/tools/datastore-emulator. I found
[datastore] API endpoint: http://::1:8862
[datastore] export DATASTORE_EMULATOR_HOST=::1:8862
When I access the endpoint on Safari, it can not be accessed. I think http://::1:8862 is a strange address and I do not know why it happened and how to solve the problem. thx
That address means localhost (in IPv6 format) and port 8862. See What is IPV6 for localhost and 0.0.0.0?
I didn't use the emulator, so I'm not sure if this applies to the API endpoint or not: maybe you can specify your desired one using the --host-port argument. From gcloud beta emulators datastore start:
--host-port=HOST_PORT
The host:port to which the emulator should be bound. Can take the form
of a single address (hostname, IPv4, or IPv6) and port:
ADDRESS[:PORT]
In this format you must enclose IPv6 addresses in square brackets:
e.g.
[2001:db8:0:0:0:ff00:42:8329]:8080
The default value is localhost:8081.
Related
I would like to develop a Google Cloud Function that will subscribe to file changes in a Google Cloud Storage bucket and upload the file to a third party FTP site. This FTP site requires allow-listed IP addresses of clients.
As such, it is possible to get a static IP address for Google Cloud Functions containers?
Update: This feature is now available in GCP https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
First of all this is not an unreasonable request, don't get gaslighted. AWS Lambdas already support this feature and have for awhile now. If you're interested in this feature please star this feature request: https://issuetracker.google.com/issues/112629904
Secondly, we arrived at a work-around which I also posted to that issue as well, maybe this will work for you too:
Setup a VPC Connector
Create a Cloud NAT on the VPC
Create a Proxy host which does not have a public IP, so the egress traffic is routed through Cloud NAT
Configure a Cloud Function which uses the VPC Connector, and which is configured to use the Proxy server for all outbound traffic
A caveat to this approach:
We wanted to put the proxy in a Managed Instance Group and behind a GCP Internal LB so that it would dynamically scale, but GCP Support has confirmed this is not possible because the GCP ILB basically allow-lists the subnet, and the Cloud Function CIDR is outside that subnet
I hope this is helpful.
Update: Just the other day, they announced an early-access beta for this exact feature!!
"Cloud Functions PM here. We actually have an early-access preview of this feature if you'd like to test it out.
Please complete this form so we can add you..."
The form can be found in the Issue linked above.
See answer below -- it took a number of years, but this is now supported.
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
For those wanting to associate cloud functions to a static IP address in order to whitelist the IP for an API or something of the sort I recommend checking out this step by step guide which helped me a lot:
https://dev.to/alvardev/gcp-cloud-functions-with-a-static-ip-3fe9 .
I also want to specify that this solution works for Google Cloud Functions and Firebase Functions (as it is based on GCP).
This functionality is now natively part of Google Cloud Functions (see here)
It's a two-step process according to the GCF docs:
Associating function egress with a static IP address In some cases,
you might want traffic originating from your function to be associated
with a static IP address. For example, this is useful if you are
calling an external service that only allows requests from whitelisted
IP addresses.
Route your function's egress through your VPC network. See the
previous section, Routing function egress through your VPC network.
Set up Cloud NAT and specify a static IP address. Follow the guides at
Specify subnet ranges for NAT and Specify IP addresses for NAT to set
up Cloud NAT for the subnet associated with your function's Serverless
VPC Access connector.
Refer to link below:
https://cloud.google.com/functions/docs/networking/network-settings#associate-static-ip
As per Google, the feature has been released check out the whole thread
https://issuetracker.google.com/issues/112629904
It's not possible to assign a static IP for Google Cloud Functions, as it's pretty much orthogonal to the nature of the architecture being 'serverless' i.e. allocate and deallocate servers on demand.
You can, however, leverage a HTTP proxy to achieve a similar effect. Setup a Google Compute Engine instance, assign it a static IP and install a proxy library such as https://www.npmjs.com/package/http-proxy. You can then route all your external API calls etc through this proxy.
However, this probably reduces scale and flexibility, but it might be a workaround.
I created a Redis instance using https://console.cloud.google.com/launcher/details/bitnami-launchpad/redis-ha
and the network interface is:
I'm trying to connect to this Redis instance from a Firebase trigger.
The question is: what firewall rule do I need to connect from a cloud function to a compute instance?
Please provide as many details as possible, e.g. IP ranges, ingress/egress, etc, and whether I have to connect the Redis client to the instance on the internal IP, or the external IP.
This is the code:
const redis = require('redis');
let redisInstance = redis.createClient({
/* surely external IP needn't be used
here as it's all GCP infra? */
host: '10.1.2.3',
port: 6379
})
redisInstance.on('connect', () => {
console.log(`connected`);
});
redisInstance.on('error', (err) => {
console.log(`Connection error ${err}`);
});
The error in the log is
Connection error Error: Redis connection to 10.1.2.3:6379 failed - connect ETIMEDOUT 10.1.2.3:6379
I've looked at Google Cloud Function cannot connect to Redis but it's not specific enough about the options when setting up a rule.
What I've tried
I tried to set up a firewall rule with these settings:
ingress
network: default
source filter: my firebase service account
protocols/ports: all
targets: all
Just a note about the service account:
created by Firebase
has the Editor role in IAM
is known to work with BigQuery and other Firebase services from my Firebase triggers
This same firewall rule has been in effect for a few hours now, and I've also redeployed the trigger which tests Redis, but still getting ETIMEDOUT
UPDATES
2018-06-25 morning
I phoned GCP Gold support and the problem isn't obvious to the operator, so they'll open a case, investigate, and leave some notes.
2018-06-25 afternoon
Using a permissive firewall rule (source 0.0.0.0/0, destination "all targets") and connecting to the Redis instance's external IP address works (of course!). However, I mentioned many times now on the phone call I don't want the Redis instance to be open to the Internet, and if there's some sort of solution involving a networking bridge/VPN so I can connect to the 10.x.x.x address from the Cloud Function.
The operator said they'll get back to me in 2 days.
2018-06-25 bit later in the afternoon
I've self-answered that it doesn't seem to be possible to connect to a Compute Engine internal IP from a cloud function.
It looks like it's NOT currently possible to connect to Google Compute Engine internal IP from Google Cloud Funtions so all my (and my helpful Gold support operator's) efforts have been in vain.
Here's the (open) issue: https://issuetracker.google.com/issues/36859738
As it is explained in the question you referred to, when you create a new firewall rule you change the Source Filter field from IP ranges to Service Account. In the following step you won't need to specify any IPs, only the name of the service account for Cloud Functions.
Do we have any way to connect document dB emulator from remote system ?
Can we create procedure , triggers , user defined functions etcs in document db emulator ?
The Emulator is meant for local dev scenarios, since it runs exposing a local port, you probably could (never tried, this is purely theoretical) work around the firewall and expose it, then connect from another system using your external IP and the exposed port.
There is also the local SSL certificate that you must solve (that probably is the biggest issue), though you could try with the TCP connection setting, might want to check this thread about which ports need to be opened.
Also, the Emulator does not have the entire feature set that the live service does:
The DocumentDB Emulator supports only a single fixed account and a well-known master key. Key regeneration is not possible in the DocumentDB Emulator.
The DocumentDB Emulator is not a scalable service and will not support a large number of collections.
The DocumentDB Emulator does not simulate different DocumentDB consistency levels.
The DocumentDB Emulator does not simulate multi-region replication.
The DocumentDB Emulator does not support the service quota overrides that are available in the Azure DocumentDB service (e.g. document size limits, increased partitioned collection storage).
As your copy of the DocumentDB Emulator might not be up to date with the most recent changes with the Azure DocumentDB service, please DocumentDB capacity planner to accurately estimate production throughput (RUs) needs of your application.
So, you are probably better off installing the emulator on the other system via the installer or Chocolatey and avoid all the problems.
UPDATE: My following attempted solution doesn't work. Connection Timeout, 192.168.0.101:8881 using the Node.js DocumentDB sdk. I think because of SSL. :/ Sorry. Leaving this "Answer" for documentation on what doesn't work, and if anyone knows how to bypass DocumentDB Emulator SSL.
I am trying to connect DocumentDB Emulator across my local network. (I dev on a virtual machine)
I am trying to do a port forward, to the 8081 local port that DocumentDB Emulator listens on. In Command Prompt (Run as Administrator)
netsh interface portproxy add v4tov4 listenaddress=192.168.0.101 listenport=8080 connectport=8081 connectaddress=127.0.0.1
192.168.0.101 is the network address of the PC.
Now I'm able to navigate to:
https://192.168.0.101:8080/_explorer/index.html and see the data explorer. Optimistic I can get this working for dev, with SSL turned off?
Also tried to use node.js http-proxy couldn't get it working with self-signed certificates. :(
Update, I actually got http-proxy working, but it only works if you start the servers in a specific order...
start api server
start proxy server (on windows box) with secure: true
make a failed connection
change proxy server (on windows box) to secure: false; restart;
now it's working... but useless for dev, because if you restart the API server after code change, the connection fails again.
Sample Node.js Proxy to be run on Windows box:
```
var fs = require('fs'),
httpProxy = require('http-proxy');
//
// Create the proxy server listening on port 443
//
httpProxy.createServer({
ssl: {
key: fs.readFileSync('valid-ssl-key.pem', 'utf8'),
cert: fs.readFileSync('valid-ssl-cert.pem', 'utf8')
},
target: 'https://localhost:8081',
secure: true // Depends on your needs, could be false.
}).listen(8881);
```
You just need to start the documentdb with additional parameters:
start "" "c:\Program Files\Azure Cosmos DB Emulator\CosmosDB.Emulator.exe" /AllowNetworkAccess /NoFirewall /Key=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==
Checkout the documentdb docker file for more details: https://github.com/Azure/azure-cosmos-db-emulator-docker/blob/master/package_scripts/startemu.cmd
I'm working on a project that uses the Google Maps JavaScript API v3 to render directions on an integrated map.
The website it built with Jekyll and I run the server using the following command:
jekyll serve --host 192.168.2.7
The given IP address is the IP Address of the system running the server.
The reason I made it listen to that IP Address is because I want to test the application on my mobile devices, following this answer
In the API Console I whitelisted 192.168.*, so that any local connection can use the API.
However, when I go to that IP Address on my phone or computer, the API says the application is not authorized despite me having whitelisted it.
What am I doing wrong here and how can I solve it?
Jekyll serve
The --hostname flag doesn't exist. Use --host 192.168.2.7 or host: 192.168.2.7 in _config.yml. Target address will then be 192.168.2.7:4000 as 4000 is the default listening port for Jekyll webrick server.
Google API withelist
It's not necessary to white list domain, as with no domain white listed you have Any referer allowed
Google authorization
Verify that you've set your API key to something else than https://maps.googleapis.com/maps/api/js?key=API_KEY you are supposed to have something like this : https://maps.googleapis.com/maps/api/js?key=AIzaSytitittototoX7RhHn-drLRRYututu7eX2hVo
If you still have problems, you can give a repository url to have it fixed quicker.
Happy Jekyll !
192.168.2.7 is a private IP address and is not publicly accessible. To use that you will probably need to get a license.
My website (in php) should send Push notifications to an Android app. It used to work fine for some time, and suddenly stopped.
When I checked the return error code, it was 401 (Unauthorized)
I was using 0.0.0.0/0 as the allowed IPs. Seems it is no longer allowed after a grace period. So I set my DSL spool IP address and my server IP address. I have a dedicated IP to my cloud server (On RackSpace).
Then it worked in my localhost, but not on my cloud server. I also confirmed that my server outgoing IP is same as my server dedicated IP.
I had to set my Servers IPv6 address to Allowed IPs. Then it worked again - perfectly
[On Google Developers Console - API Credentials settings]
I'm documenting this for it would be helpful to someone else too.
Change the allowed IPs in your google console api project..