This question already has answers here:
Scraping data to Google Sheets from a website that uses JavaScript
(2 answers)
Closed last month.
starting my journey as a data analyst. I am trying to web scrape a table off of Transferwise (Wise's) price comparison website. I tried to:
Copy and paste URL into cell A1 in Google Sheets
Inspect the table row on the website
Copy and paste full Xpath into cell A2 in Google Sheets
Write formula: =importxml(A1,A2)
Get error
Inspecting table row on Wise price comparison website
Google Sheets error
Does IMPORTXML generally not work if the website uses an API to populate the table? The tables do seem different for those on Wikipedia for example.
It is not possible to retrieve informations by importxml since the page is built by javascript on client side, not server side. What you are lokking for is in fact a script!
<script>
(function (d, s, id) {
var js, fjs = d.getElementsByTagName(s)[0];
if (d.getElementById(id)) return;
js = d.createElement(s);
js.id = id;
js.src = "https://widgets.wise.com/widgets.js";
fjs.parentNode.insertBefore(js, fjs);
}(document, "script", "transferwise-wjs"));
</script>
see other comments here https://webapps.stackexchange.com/questions/115664/how-to-know-if-google-sheets-importdata-importfeed-importhtml-or-importxml-fun
Related
I am trying to scrape data from a website to Google Sheets but because of the double quotes in the xpath_query on "compTable" I keep a formula parse error. When I try do single quotes ie. 'compTable' I get the error imported content is empty. Is there a way I can handle double quotations in xpath inside of an importxml function and get this function to not return an error?
=IMPORTXML("https://www.levels.fyi/comp.html?track=Software%20Engineer&search=sydney&city=1311","//*[#id="compTable"]/tbody/tr[1]/td[2]/span/a")
For context I am trying to use this formula to get the company name from the table in the url e.g. Google, Amazon, Canva. Ultimately I want to scrape this website to create a Google Sheet with each row of the table in this URL so that I have each data point (company name, total compensation, level etc.) on each row of my Google Sheet.
use:
=IMPORTXML("https://www.levels.fyi/comp.html?track=Software%20Engineer&search=sydney&city=1311",
"//*[#id='compTable']/tbody/tr[1]/td[2]/span/a")
I am using contact form 7 on my WordPress site to get data from user and storing user responses on a google sheet using CF7 google sheet connector plugin. I want is that whenever a new entry is added via contact form 7 on the google sheet it triggers a google sheet script's function which sorts the sheet.
Is there any way to do so?
Instead of a script you can use a formula.
(In general it is NOT a good idea to mess with raw data)
To do this just create a new sheet and use the following formula (so as to exclude the if present headers from sorting in row 1)
=SORT('Sheet1'!A2:Z,2,1)
Or even
=QUERY('Copy of Sheet1'!A1:Z,"select * where B is not null order by B",1)
Please adjust ranges to your needs
Functions used:
INDEX
SORT
I'm trying to make spreadsheets to keep track of moods and other things. What I want to do is to have a dashboard with buttons with differnt moods that, when pressed, add a value and a specific color to a cell in a second sheet.
This second sheet will function as a simple database and has the first column with the date (365 rows), and another column with the mood with the colored cells that will be added every time you press the button.
Also, the value when you press the button with the selected mood, must be added next to the row that has the current date.
Summing up:
I want to make buttons in one sheet
When you press thos buttons, a specific value is added into another sheet
That value must be added next to the cell with the current date, in the column with all the dates
I've searched this here, but can't find anything similar. If you can help me directly or posting links to the subject I'll be thankfull.
What you are describing is in fact quite easy to do with Sheets and Google Apps Scripts. These are the steps:
Create a Sheets document.
Create a sheet named database. Set the headers (i.e. A1=Date, B1=Mood).
Go to Tools > Script Editor. Paste the following code and save the project:
var TIMEZONE = "GMT";
function wasInserted(date) {
var lastInsertedDate = PropertiesService.getScriptProperties().getProperty('lastDate');
return date == lastInsertedDate;
}
function getDateNow() {
return Utilities.formatDate(new Date(), TIMEZONE, "M/d/y");
}
function insertMood(mood) {
var date = getDateNow();
if (wasInserted(date)) return; // mood already inserted for the day.
var ss = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('database');
ss.appendRow([date, mood]);
PropertiesService.getScriptProperties().setProperty('lastDate', date);
}
function selectHappy() {
insertMood('happy');
}
function selectConfused() {
insertMood('confused');
}
function selectSad() {
insertMood('sad');
}
Create three images in the main sheet of your Sheets document (important: they must be "Image over cells"). They must map to Happy, Confused, and Sad, which are the moods that i defined for the application. Of course, you may modify this and add as many moods as you please.
Click on each of the pictures, click the three dots and click on "Assign script". For the happy picture, it should be "selectHappy" (see above code - it must map the function names), and so on.
Now, go and test it. click any of the pictures. The first time it will ask you for permissions, which you must accept. Afterwards, you won't be required to accept them. Each time you click them a new entry will be created in the "database" sheet with the selected mood. The buttons can only be clicked once a day, the second time (and so on) will not do anything.
In App Maker, I am displaying a table and want to replace table cell data with different text using a data lookup from another table. Assume two tables, Departments and Employees.
Departments is two fields, DeptID and DeptDescription.
Employees is multiple fields including DeptID.
In the table listing for Employees, I would like to replace the DeptID with the DeptDescription. (The page datasource is Employees. I do not want to set up a relationship between the data models.)
I am guessing I want to do some scripting in the onDataLoad event for the table cell label for DeptID. I have this much so far:
app.datasources.Departments.query.filters.DeptID._equals = widget.datasource.item.DeptID;
app.datasources.Departments.newQuery().run();
widget.text = app.datasources.Departments.item.DeptDescription;
I know this is not correct, but am I close?
This answer is untested, but I wanted to present a possible solution that would not require a lot of DB calls, especially ones that make repeated calls to a server script which might consume a lot of processing time when you do line item calls.
Set up a separate datasource under the Department model. Change the default 'Query Builder' to 'Query Script' and add a parameter of type 'list(number)' or 'list(string)', this should match your Primary Key field type. Uncheck the 'auto load' option.
In your 'Query Script' portion enter the following code:
query.filters.Id._in = query.parameters.YourParameter;
return query.run();
Go to your Employees datasource that is supposed to generate your table and find your 'On Load' client script section. In this section enter the following code:
var departmentsDs = app.datasources.YourDepartmentsDs;
departmentsDs.properties.YourParameter = datasource.items.map(function(deptIds) {return deptIds.DeptID;});
departmentDs.load();
Now go the page that contains your table. If you have not already create a label widget do so now. In this label widget for the text binding enter the following:
#datasources.YourDepartmentsDs.loaded && (#datasources.YourDepartmentsDs.items).map(function(Id){return Id.Id}).indexOf(#widget.datasource.item.DeptID) !== -1 ? #datasources.YourDepartmentDs.items[(#datasources.YourDepartmentsDs.items).map(function(Id){return Id.Id}).indexOf(#widget.datasource.item.DeptID)].DeptDescription : 'Unable to retrieve Dept Description'
As stated this is untested and I wrote the code from memory without App Maker in front of me so it may require some additional tweaking. Going with the first option presented by J.G. would also be a very viable solution though. And I apologize but the code formatter does not seem to be working for me.
1 way) Create an aggregate table that joins your tables if you need to bypass using the relations feature. This way you can use sql to join the two tables in the datasource definition
2) if you don't want to make a new table. Change the text from a value binding to "more options"
=getDescription(#datasource.item.DeptId)
and then the code you wrote in a client side script
function getDescription(id){
google.script.run
.withSuccessHandler(function successHandler(result){ return result;})
.withFailureHandler( function failureHandler(e){ console.log(" Failed" +e);})
.queryValue(id);
}
server side script:
function queryValue(id){
var query = app.models.Departments.newQuery();
query.filters.DeptID._equals = id;
var results = query.run();
return results[0]["DeptDescription"];
}
that last line might be results[0].DeptDescription
I have two different html tables with me.
Now I want to export these two html table to single excel sheet with two different sheets (i.e work sheet). For example, table 1 will be one sheet and table 2 will be in another sheet. But both these sheets will be present in one Excel file only.
I am not using any grid view or data view.
I don't want to save the Excel file on server.
Below method is the simple solution to export all the content in html to excel:
Response.Clear();
Response.Buffer = true;
Response.AppendHeader("Content-Disposition","attachment;filename="+DateTime.
Now.ToString("yyyyMMdd")+".xls");
Response.ContentEncoding = System.Text.Encoding.UTF8;
Response.ContentType = ""application/ms-excel";;
this.EnableViewState = false;
you need to export different html tables, you can first save the html table in two excel files and then, use the merge all the data into one excel with different sheets.
you can see details here:
Export Data to Excel for ASP.NET