I am trying to use the Marketo activities.json API endpoint, and I am getting a timeout everytime I try. I have set the cURL timeout to 25 seconds and I am using a valid nextPageToken parameter to filter the results. The timeframe is yesterday and today.
When I try other endpoints (lists.json, activities/pagingtoken.json, leads.json, lists.json, and stats/usage/last7days.json) I get a response and my request does not timeout.
Here is the request I am making to activities.json:
method: "GET"
url: "https://[marketo-id].mktorest.com/rest/v1/activities.json"
parameters: Array
(
[nextPageToken] => [paging-token]
[listId] => [list-id]
[activityTypeIds] => 24
[access_token] => [access-token]
)
Why am I getting a timeout just for the activities.json endpoint? Is this API endpoint broken or down?
The global timeout for Marketo's REST API is 30 seconds, can you first try adjusting your local timeout to match this? If you remove list ID from from the call what happens?
Related
I'm using firebase and would like to update a value no matter what happens using a HttpRequest Cloud function ( connection error, tab or browser closed ).
I thought of a setTimeout but this doesn't solve the issue as if anything happens the request will fail as it is related to client side.
Then my idea was to send the request on a remote server ( a bit like a cron job ) that should perform at a certain delay ( let's say 30 minutes ) but that would cancel the previous request in queue ( based on userID and request path )
How could I achieve that and is this possible?
example:
request number 1 -> www.myrequest with { userUID:1, data:{size:1}}
request time:12:01
request number 2 -> www.myrequest with { userUID:1, data:{size:30}}
request time:12:02
request 2 cancels request 1 and gets executed 30 minutes later
I am using fulcrum to collect data. fulcrum has a webhook feature
I have created a firebase function and linked the firebase function to fulcrums webhook feature with the functions URL. https://us-central1-example.cloudfunctions.net/fulcrumHook
Here is my existing function.
exports.fulcrumHook = functions.https.onRequest((request, response) => {
console.log(response.data.form_id)
response.send(200)
})
Through hours of debugging, in the logs I can see that the data I want is coming through but I am struggling to access it in the function itself.
When I log the request I get IncomingMessage { _readableState: ReadableState { objectMode: false,.....
When I log the response I get ServerResponse { domain: null, _events: [Object: null prototype] { finish: [ [.... as well as the body much further down with the actual data i need in it.
I have search for all the keywords i can think of about how to handle this data but I am completely stumped.
Do I need to handle the response like a promise with response.then(data => ...stuff)
Do I need to establish a connection like a socket with response.on('data', (data) => ...stuff)
Everything you need is in the documentation for HTTP triggers.
The request and response are essentially Express Request and Response objects.
Used as arguments for onRequest(), the Request object gives you access to the properties of the HTTP request sent by the client, and the Response object gives you a way to send a response back to the client.
You can click through to those linked APIs to understand in detail how they work.
Data passed to the function can be found by reading values from the request. If it's a POST request, form values are read like this:
request.body.form_id
The response is sent using response.send(). Just pass it an object that will get automatically serialized as JSON. Or use the linked API for the response object from above to learn more about your options.
Problem:
We are using Https Functions that are taking longer than 60 sec to respond (this is a necessary requirement for our use case and cannot be replaced with async execution). We configured the functions timeout, such that they succeed after e.g. 75 sec, as indicated by direct execution from Chrome or Postman through e.g. https://us-central1-<project-id>.cloudfunctions.net/test.
We have also configured Hosting proxy to rewrite all requests for /api to these functions. However, when submitting identical requests through these "hosting" endpoints (e.g. https://<project-id>.firebaseapp.com/api/test), we receive a 504 (Gateway timeout) error after only 60 sec, both in Chrome and Postman. This suggests that the Hosting proxy itself times out before a function succeeds (which still happens, as indicated in Firebase Console log).
Question:
Is it possible to set Hosting timeout to a higher value (e.g. 120 sec)?
Possible solutions/workarounds:
tried to set Connection: keep-alive and Keep-Alive: timeout=120 headers in client requests, however that didn't seem to have any effect on the Hosting proxy; or maybe we're not doing it right.
the only workaround for now is to not use rewrite rules, instead relying on the "direct" URLs (from cloudfunctions.net). However, that entails dealing with CORS in browser clients and more importantly, changing our URL resolution scheme and organization of functions' code (e.g. we cannot use a long path for each function, such as /api/some/path/to/test, because only the first part of that path, such as api, will be considered as the function name).
Thanks!
It is possible to write your function in such a way as to maintain a persistent connection:
https://firebase.google.com/docs/functions/networking#https_requests
const http = require('http');
const functions = require('firebase-functions');
const agent = new http.Agent({keepAlive: true});
exports.function = functions.https.onRequest((request, response) => {
req = http.request({
host: '',
port: 80,
path: '',
method: 'GET',
agent: agent,
}, res => {
let rawData = '';
res.setEncoding('utf8');
res.on('data', chunk => { rawData += chunk; });
res.on('end', () => {
response.status(200).send(`Data: ${rawData}`);
});
});
req.on('error', e => {
response.status(500).send(`Error: ${e.message}`);
});
req.end();
});
Just ran into this very same problem with a Cloud Scheduler Job that hits my API endpoint (which is connected via Firebase Hosting). It takes around 80 seconds to complete.
But it seems that there is no workaround to this. At least not while using Firebase Hosting. Maybe you can hit the functions' URL directly instead of doing it through Firebase Hosting. But if those request are coming from the browser, you'd have to deal with configuring CORS.
From: https://firebase.google.com/docs/hosting/functions
Text version:
Note: Firebase Hosting is subject to a 60-second request timeout. Even if you configure your HTTPS function with a longer request timeout, you'll still receive an HTTPS status code 504 (request timeout) if your function requires more than 60 seconds to run. To support dynamic content that requires longer compute time, consider using an App Engine flexible environment.
I'm working on an app that makes use of the googlemaps API. I'm not using any API key as I'm performing requests like http://maps.googleapis.com/maps/api/directions/json?origin=Brooklyn&destination=Queens&sensor=false&departure_time=1343641500&mode=transit (shown in one of the Google examples). Unfortunately, lately it keeps giving the following error:
{
"error_message": "You have exceeded your daily request quota for this API.",
"routes": [],
"status": "OVER_QUERY_LIMIT"
}
Even by adding a key I create as a parameter to the GET request (i.e., &key=blahblah), the result is always the same. Does this mean my I.P. is blocked by Google?
What can I do to get it back to work?
Thanks
When you use the key it means that that the limit for the account where the key has been created for has been reached.
When you don't use a key it means that the limit for the IP of your server has been reached.
What you can do:
wait until tomorrow or request the service from clientside
I've desperately tried to figure this out on my own, and did not want to come to SO with this question, but I'm at my wits end (no thanks to the api / oauth docs).
I'm working in PHP and I'm trying to avoid using the Google_Client and AnalyticsService classes, by using REST on the analytics.data.ga.get method.
STEP #1: Create an API Project for a Web Application
I go to the api console and create a project with analytics services and get an OAuth Client ID and Secret.
I'm under the impression that I can create a Client ID for an Installed Application or a Web Application because I'm doing the initial token handshaking manually. Please correct me if I'm wrong.
I create a Client ID for web applications and get my Client ID xxxxxxxxxxxxxx.apps.googleusercontent.com, Client secret yyyyyyyyyyyyyyy, and my Redirect URI is http://localhost:9002
STEP #2: Request initial API access
I enter this link; https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=xxxxxxxxxxxxxx.apps.googleusercontent.com&redirect_uri=http://localhost:9002&scope=https://www.googleapis.com/auth/analytics.readonly&access_type=offline
The access_type=offline is because I'm using REST, and do not expect "the user" (myself) to manually deal with redirects / popups every time I need a refreshed token.
The above request returns http://localhost:9002?code=4/KModH0K_xxxxxxxxxxxxxxxxxxx9Iw.gikOaYRDWywTshQV0ieZDArCOX8XdwI
Code 4/KModH0K_xxxxxxxxxxxxxxxxxxx9Iw.gikOaYRDWywTshQV0ieZDArCOX8XdwI is my permission to request the API Token.
STEP #3: Request First Token
Because of my company’s IT issues, I’m forced to use PHP 5.2.17 and I do not have access to PHP cURL, so I’m using file_get_contents and stream_context_create.
The first token is requested with a PHP file_get_contents();
$opts = array(
'http' => array(
'method' => 'POST',
'header' => 'Content-Type: application/x-www-form-urlencoded',
'content' => 'code=4/KModH0K_xxxxxxxxxxxxxxxxxxx9Iw.gikOaYRDWywTshQV0ieZDArCOX8XdwI&client_id=xxxxxxxxxxxxxx.apps.googleusercontent.com&client_secret=yyyyyyyyyyyyyyy&redirect_uri=http://localhost:9002&grant_type=authorization_code'
)
);
$context = stream_context_create($opts);
$result = file_get_contents('https://accounts.google.com/o/oauth2/token', false, $context);
var_dump($result);
The content parameters must be in a single line.
The above code returns my access_token and refresh_token in json format
string(195) "{ "access_token" : "ya29.AHES6wwwwwwwwwwwwwwwVEBXE6XRbC-Q-pP0wZWdoIm9H804ro", "token_type" : "Bearer", "expires_in" : 3600, "refresh_token" : "1/8tXvdUKcSEcaaxVqqqqqqqqqqqqqoYpj2KSS9qwWI" }"
The refresh token I must store in a safe place, like a DB or protected txt file, which is called upon when my access_token has timed out.
STEP #4: Request Analytics Data
Now from what I understand, I’m ready to roll and should be able to use my access_token to make requests to https://www.googleapis.com/analytics/v3/data/ga.
I do this by sending this request;
$request = 'https://www.googleapis.com/analytics/v3/data/ga' .
'?ids=ga%3Aaaaaaaaa' .
'&start-date=2012-12-07' .
'&end-date=2012-12-09' .
'&metrics=ga%3Avisits';
$opts = array(
'http' => array(
'method' => 'GET',
'header' => 'Content-Type: application/x-www-form-urlencoded\r\n' .
'Authorization: Bearer ya29.AHES6wwwwwwwwwwwwwwwVEBXE6XRbC-Q-pP0wZWdoIm9H804ro \r\n'
)
);
$context = stream_context_create($opts);
$result = file_get_contents($request, FALSE, $context);
var_dump($result);
This request returns a 401 Unauthorized error. I take this as meaning my request is properly formed and making the connection to https://www.googleapis.com/analytics/v3/data/ga.
Also, according to this doc Getting Full Quota, I can make the request with the access_token in the URL like this;
$request = 'https://www.googleapis.com/analytics/v3/data/ga' .
'?ids=ga%3A48564799' .
'&access_token=ya29.AHES6wwwwwwwwwwwwwwwVEBXE6XRbC-Q-pP0wZWdoIm9H804ro' .
'&start-date=2012-12-07' .
'&end-date=2012-12-09' .
'&metrics=ga%3Avisits';
$result = file_get_contents($request, FALSE);
$result = json_decode($result);
var_dump($result);
This time I receive 403 error, in which google includes the response User does not have sufficient permissions for this profile.
QUESTION #1
Am I’m missing something in the API console or a process in the token acquisition? I’m assuming I’m not, because I’m ultimately acquiring the access_token=ya29 and refresh token.
QUESTION #2
Maybe I’m completely off basis in assuming I can do this with simple https reqests? Do I have to use the Google_Client and AnalyticsService classes? I don’t think this is the case, but maybe I’m wrong.
QUESTION #3
Do I need to use a ‘key’ in my request?
&key=bbbbbbbbbbbbbbbb
QUESTION #4
By using PHP 5.2.17 am I missing something? (besides 5.3 or 5.4 themselves)
For example, in some versions of PHP, in stream_context_create, the header should be in an array and not a string, like this;
$opts = array(
'http' => array(
'method' => 'GET',
'header' => array(
'Content-Type: application/x-www-form-urlencoded',
'Authorization: Bearer ya29.AHES6wwwwwwwwwwwwwwwVEBXE6XRbC-Q-pP0wZWdoIm9H804ro '
)
)
);
But I don’t think that it’s an issue in my case. I’m just curious if these HTTP request need to be formed a different way (without using curl).
Any insights and thoughts would be greatly appreciated
Here’s my dim witted mistake that nearly gave me a heart attack.
I typically do my development work in Chrome. And my Chrome browser was signed into my gmail account personal#gmail.com. However, my analytics account, which is under my work#gmail.com was open in FireFox (try not to laugh to hard).
I’m not 100% sure this is correct, but I think this is the general flow. When I did STEP #2: Request initial API access, I did this in my Chrome browser. The endpoint https://accounts.google.com/o/oauth2/auth was authorizing my personal#gmail.com account. And my STEP #4 API request https://www.googleapis.com/analytics/v3/data/ga was looking for an analytics profile under my personal#gmail.com account. Which of course doesn’t exist.
I literally wasted 15 hours on this. This is dumber than trying to troubleshoot an update… and forgetting to flush the cache.
Sorry for wasting your time.
EDIT REFRESH TOKENS
I've once again run into issues with this API and found out the hard way that GA will revoke Refresh Tokens if too many different clients use the token, at least I think that was the problem.
Further reading can be found here.
I got a 403 today and found the cause: in the get function I was using the account ID instead of the profile ID. Switching to the profile ID fixed it for me.
It could be a problem with CURL request. In the GoogleAnalyticsAPI.class.php > class Http > function curl (around line 720) add one more option to stop CURL from verifying the peer's certificate:
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);