Error: Page crashed! at Page._onTargetCrashed PUPPETEER - web-scraping

I'm having page crash error even after handling it via catch and using page.on(error). Please help me fix this error.
// open the headless browser
var browser = await puppeteer.launch({ headless: true, args: ['--disable-gpu','--disable-dev-shm-usage', '--no-sandbox', '--disable-setuid-sandbox'] });
// Open a new page
var page = await browser.newPage();
var hsURL =highschoolURL.concat(hsList[i].trim()).concat("/academics");
await page.setDefaultNavigationTimeout(0);
page.on('error', err=> {
console.log('error happen at the page: ', err);
});
page.on('pageerror', pageerr=> {
console.log('pageerror occurred: ', pageerr);
})
await page.goto(hsURL, {waitUntil : 'networkidle2'}).catch(e => console.error(e));

the default setDefaultNavigationTimeout is 3000(3 seconds). You should not set setDefaultNavigationTimeout to 0 . Just set setDefaultNavigationTimeout to await page.setDefaultNavigationTimeout(3000); or remove that sentences

Error: Page crashed! at Page._onTargetCrashed can occur if there isn't enough memory for puppeteer to run.
Increasing the amount of memory available for puppeteer should help resolve this issue.

Related

Watch console for errors

I have an app using canvas elements, which are difficult to test with javascript but it does throw messages to the console.
How can I watch for errors written to the console?
I tried monkey patching the window console, but it's not working.
const messages = []
window.console.error = (msg) => {
messages.push(msg)
})
// actions causing error
expect(messages.length).to.eq(0)
You can watch console messages with Cypress cy.spy(), ref docs
let spy;
Cypress.on('window:before:load', (win) => {
spy = cy.spy(win.console, "error")
})
// actions causing error
cy.then(() => {
expect(spy).not.to.be.called
})

Apollo Network (Bad Request) error not caught in apollo-link-error

Sending a mutation with a bad set of variables (on purpose in this case) results in errors being thrown in the console, and the apollo-link-error link not picking up the error. Additionally, the mutation loading state is stuck as 'loading' and not error object comes through.
Through a debugging session, I fount that the zen-observable global error handling picks up an error thrown in the error-link's "next" function, because 'result' is not defined
pasted the apollo-link-error code that has the observable wrapped in a try catch, but the catch at the bottom here is not the catch that gets hit when if (result.errors) throws a 'nullpointer' because result in undefined.
try {
sub = forward(operation).subscribe({
next: function (result) {
// result is undefined, throwing an error
if (result.errors) {
retriedResult = errorHandler({
graphQLErrors: result.errors,
response: result,
operation: operation,
forward: forward,
});
if (retriedResult) {
retriedSub = retriedResult.subscribe({
next: observer.next.bind(observer),
error: observer.error.bind(observer),
complete: observer.complete.bind(observer),
});
return;
}
}
observer.next(result);
},
error: function (networkError) {
retriedResult = errorHandler({
operation: operation,
networkError: networkError,
graphQLErrors: networkError &&
networkError.result &&
networkError.result.errors,
forward: forward,
});
if (retriedResult) {
retriedSub = retriedResult.subscribe({
next: observer.next.bind(observer),
error: observer.error.bind(observer),
complete: observer.complete.bind(observer),
});
return;
}
observer.error(networkError);
},
complete: function () {
if (!retriedResult) {
observer.complete.bind(observer)();
}
},
});
} // the error is NOT caught here
catch (e) {
errorHandler({ networkError: e, operation: operation, forward: forward });
observer.error(e);
}
```
Link definition:
```javascript
export const client = new ApolloClient({
link: ApolloLink.from([
onError((errors) => {
console.log('errors in link!', errors);
handleServerError(errors);
}),
new MeteorAccountsLink(),
new HttpLink({
uri: '/graphql',
}),
]),
cache: new InMemoryCache(),
});
Edit:
The request in the browser does show a response with an error object with the graphQlErro of shape {errors: [{..}]} which is strange that it's not coming into the 'result' in the link.
Edit 2:
It looks like Meteor is picking up the error thrown in the http link prior to the error posted above, which might be why "result" is undefined. Writing a custom link to polyfill the missing 'result' so the app will at least work.
It seems like an issue with Meteor swallowing the errors. fixed make making a polyfill link to at least not break the js in the mutation complete function and show a general error.
const stupidMeteorErorrPolyfill = new ApolloLink((operation, forward) => forward(operation).map(data => data || { errors: [{ message: '! Empty response in gql links, see graphql.js and network tab' }] }));

Sequelize delete request with payload works with Postman but not with Vue Frontend implementation

I am trying to send PUT and DELETE requests with my express backend to a sqlite database. Put request works fine but DELETE request always fails.
I already checked the headers within the network tab, it seems to be the right one for both (application/json)
With postman, I can easily delete entries, but with my frontend the body does not seem to be set correctly.
const countryToPush = {title: countryName}
try{
await CountryService.post(countryToPush)
console.log(countryName + ' added!')
} catch(err)
{
console.log(err)
}
},
removeFromDb: async function(countryName){
const countryToDelete = {title: countryName}
try{
await CountryService.delete(countryToDelete)
console.log(countryName + ' deleted!')
} catch(err)
{
console.log(err)
}
}
This is within my vue file where I get the 'countryName' from an on click function.
try {
const country = await Country.create(req.body)
res.send(country)
} catch (err) {
res.status(500).send({
error: 'an error has occurred trying to create the country'
})
}
},
async delete (req, res) {
try {
const toDelete = await Country.findOne({
where: {
title: req.body.title
}
})
await toDelete.destroy()
res.send(req.body.title + ' was deleted')
} catch (err) {
res.status(500).send({
error: 'an error has occurred trying to delete the country'
})
}
}
Whereas this is the example from my sqlite calls
Unfortunately, the DELETE request from my vue frontend always fails and gives me the defined error 500 an error has occurred trying to delete the country.
Any ideas what else I could try to get it working?
Model.delete() is not a Sequelize function, you want to use Model.destroy().
await CountryService.destroy(countryToDelete)
Right now you are swallowing the actual error - add console.log(err) to your catch to see that it is probably saying CountryService.destroy is undefined.

Cypress has started throwing 417 Expectation Failed error on all POST and PUT XMLHttpRequest

All POST and PUT XMLHttpRequest made into Cypress have recently started throwing 417 Expectation Failed. However all these work on the web application when I navigate through it manually.
All my code used to work well in past without any issue.
I read about this error over internet and I'm not sure if this issue exists on the application under test, or on some firewall policy or there is some setting in Cypress which can fix it.
Cypress.Commands.add("Post_Client", () => {
cy.fixture(Cypress.env("ClientInputFile")).then(clientoBJ => {
cy.fixture(clientoBJ.imagePath, "binary").then(imageBin => {
Cypress.Blob.binaryStringToBlob(imageBin, clientoBJ.imageType).then(
blob => {
const xhr = new XMLHttpRequest();
const data = new FormData();
data.set(clientoBJ.nameatr, clientoBJ.nameVal);
data.set(clientoBJ.imageatr, blob);
xhr.open(
"POST",
Cypress.env("APIBaseURL") + Cypress.env("ClientPostURL"),
false
);
xhr.setRequestHeader("accept", "application/json");
xhr.setRequestHeader("access-token", accesstoken);
xhr.setRequestHeader("client", client);
xhr.setRequestHeader("expiry", expiry);
xhr.setRequestHeader("token-type", tokentype);
xhr.setRequestHeader("uid", uid);
xhr.onload = function() {
if (this.status === 201) {
cy.writeFile(
Cypress.env("ClientOutputFile"),
JSON.parse(this.responseText)
);
cy.readFile(Cypress.env("IDStore")).then(obj => {
obj.clientID = JSON.parse(this.responseText).client.id;
cy.writeFile(Cypress.env("IDStore"), obj);
});
}
};
xhr.send(data);
}
);
});
});
});
And then it is called in a Test
it.only("CLIENT API POST TEST", () => {
cy.Post_Client();
});
This stands fixed now.There were two problems causing this and both were at the application layer.
Problem# 1 - Somehow we chose 417 as the error code for any unhandled events.
Fix - We are now using 422 error code for unprocessable entities
Problem# 2 - A formData append method has three params -(name, value, filename) where filename is optional. It is made mandatory in the application code recently.
fix -
data.set(
clientoBJ.imageatr,
blob,
clientoBJ.imagePath.substring(
clientoBJ.imagePath.lastIndexOf("//") + 2
)
);

Mongoose asynchronous .save and callback

This is one of those problem that you can explain but do not know how to fix it. I have a simple store method:
exports.store = async (req, res) => {
const mydata = new MyModel(req.body)
await mydata.save(function(err,user) {
res.location('/mydata/id/' + user._id)
})
res.status(201).json({ data: userdata })
}
When it runs, I get the following error:
events.js:182
throw er; // Unhandled 'error' event
^
Error: Can't set headers after they are sent.
at validateHeader (_http_outgoing.js:489:11)
at ServerResponse.setHeader (_http_outgoing.js:496:3)
at ServerResponse.header (.../node_modules/express/lib/response.js:730:10)
at ServerResponse.location (.../node_modules/express/lib/response.js:847:15)
at .../src/controllers/users-controller.js:26:13
at .../node_modules/mongoose/lib/model.js:3919:16
at .../node_modules/mongoose/lib/services/model/applyHooks.js:162:20
at _combinedTickCallback (internal/process/next_tick.js:131:7)
at process._tickCallback (internal/process/next_tick.js:180:9)
Process finished with exit code 1
I appears that the callback function runs separately and asynchronously because the res.status(201).json({ data: userdata }) seems to be producing the error and does not let me set the location header.
I've looked around for how to set the location header but none of the answers are like what I'm trying to do. This seems like something that should have been done before...? I'm looking for some help on how to do this...
You are mixing up two way of thinking :
Promises (with async/await in your case)
Callback
Use only one
try {
const user = await mydata.save();
res.location(`/mydata/id/${user._id}`);
// Other stuff ...
} catch(err) {
// Handle the errors
}
here you get an article about Promises into mongoose.

Resources