IMPORTDATA not grabbing live data from XML - web-scraping

I'm using Google Sheet's IMPORTDATA function to grab information from an XML file that is pulling from an API but the information I pull into the sheet isn't up to date.
How can I modify my sheet to pull in up-to-date data?
Compare the sheet: https://docs.google.com/spreadsheets/d/1W0Bt5z-Tky-tNhG_JtfE4FfjTRgQNRu_eQu2qVhQ-_E/edit?usp=sharing (LiveScores sheet)
To the XML: https://www67.myfantasyleague.com/2019/export?TYPE=liveScoring&L=64741&APIKEY=&W=14&DETAILS=1&JSON=0
Observe franchise id="0015" in both sets of data.
The sheet states <franchise id="0005" score="0.00" gameSecondsRemaining="21600" playersYetToPlay="6" playersCurrentlyPlaying="0" isHome="0">
The XML states <franchise id="0015" score="11.14" gameSecondsRemaining="20004" playersYetToPlay="4" playersCurrentlyPlaying="2"> (This data is for a football game that is currently being played as I'm writing this so the above example may not be exact, but it WON'T be score of 0.00, for example.
Any help would be amazing, thanks!

Have you tried using IMPORTXML? Google Sheets IMPORTXML Page
In IMPORTXML, you can just use the Inspect Element feature to pull the xpath.
Hope this helps. Let me know if I can help further.
Edit: Instructions To Change When Data Is Imported
In the toolbar go to the script editor
Now in the scripts, paste the code listed bellow
/**
* Go through all sheets in a spreadsheet, identify and remove all spreadsheet
* import functions, then replace them a while later. This causes a "refresh"
* of the "import" functions. For periodic refresh of these formulas, set this
* function up as a time-based trigger.
*
* Caution: Formula changes made to the spreadsheet by other scripts or users
* during the refresh period COULD BE OVERWRITTEN.
*
* From: https://stackoverflow.com/a/33875957/1677912
*/
function RefreshImports() {
var lock = LockService.getScriptLock();
if (!lock.tryLock(5000)) return; // Wait up to 5s for previous refresh to end.
// At this point, we are holding the lock.
var id = "YOUR-SHEET-ID";
var ss = SpreadsheetApp.openById(id);
var sheets = ss.getSheets();
for (var sheetNum=0; sheetNum<sheets.length; sheetNum++) {
var sheet = sheets[sheetNum];
var dataRange = sheet.getDataRange();
var formulas = dataRange.getFormulas();
var tempFormulas = [];
for (var row=0; row<formulas.length; row++) {
for (col=0; col<formulas[0].length; col++) {
// Blank all formulas containing any "import" function
// See https://regex101.com/r/bE7fJ6/2
var re = /.*[^a-z0-9]import(?:xml|data|feed|html|range)\(.*/gi;
if (formulas[row][col].search(re) !== -1 ) {
tempFormulas.push({row:row+1,
col:col+1,
formula:formulas[row][col]});
sheet.getRange(row+1, col+1).setFormula("");
}
}
}
// After a pause, replace the import functions
Utilities.sleep(5000);
for (var i=0; i<tempFormulas.length; i++) {
var cell = tempFormulas[i];
sheet.getRange( cell.row, cell.col ).setFormula(cell.formula)
}
// Done refresh; release the lock.
lock.releaseLock();
}
}
This snippet of code came from Periodically refresh IMPORTXML() spreadsheet function
Last and definitely the least, replace the "YOUR-SHEET-ID"
NOTE: I have not personally tested this code and I cannot vouch for it. I recommend making a copy and testing it there first.
Hopefully, this solves the issue of your data not being imported as often as you want. If you want to manually get "fresh" data, you can just delete/cut the import function and paste it back in.

try in A2:
=ARRAYFORMULA(IFNA(VLOOKUP(C2:C, PlayerList!A:F, {2, 6}, 0)))
and in C2:
=ARRAYFORMULA(QUERY(REGEXEXTRACT(QUERY(IMPORTDATA(
"https://www67.myfantasyleague.com/2019/export?TYPE=liveScoring&L=64741&APIKEY=&W=14&DETAILS=1&JSON=0?273"),
"where Col1 contains 'player id'", 0),
"(player id=""(\d+)).+?(score=""(\d+.\d+))"),
"select Col2,Col4"))
spreadsheet demo

Related

AppSpreadsheet (GAS): avoid some problems with sistematic tested data

In my current job with spreadsheet, all inserted data passes through a test, checking if the same value is found on the same index in other sheets. Failing, a caution message is put in the current cell.
//mimimalist algorithm
function safeInsertion(data, row_, col_)
{
let rrow = row_ - 1; //range row
let rcol = col_ - 1; // range col
const active_sheet_name = getActiveSheetName(); // do as the its name suggest
const all_sheets = SpreadsheetApp.getActiveSpreadsheet().getSheets();
//test to evaluate the value to be inserted in the sheet
for (let sh of all_sheets)
{
if (sh.getName() === active_sheet_name)
continue;
//getSheetValues do as its name suggest.
if( getSheetValues(sh)[rrow][rcol] === data )
return "prohibited insertion"
}
return data;
}
// usage (in cell): =safeInsertion("A scarce data", ROW(), COLUMN())
The problems are:
cached values confuse me sometimes. The script or data is changed but not perceived by the sheet itself until renewing manually the cell's content or refreshing all table. Is there any relevant configuration available to this issue?
Sometimes, at loading, a messing result appears. Almost all data are prohibited, for example (originally, all was fine!).
What can I do to obtain a stable sheet using this approach?
PS: The original function does more testing on each data insertion. Those tests consist on counting the frequency in the actual sheet and in all sheets.
EDIT:
In fact, I can't create a stable sheet. For test, a let you a copy of my code with minimal adaptations.
function safelyPut(data, max_onesheet, max_allsheet, row, col)
{
// general initialization
const data_regex = "\^\s*"+data+"\s*$"
const spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
const activesheet = spreadsheet.getActiveSheet();
const active_text_finder = activesheet.createTextFinder(data_regex)
.useRegularExpression(true)
.matchEntireCell(true);
const all_text_finder = spreadsheet.createTextFinder(data_regex)
.useRegularExpression(true)
.matchEntireCell(true);
const all_occurrences = all_text_finder.findAll();
//test general data's environment
const active_freq = active_text_finder.findAll().length;
if (max_onesheet <= active_freq)
return "Too much in a sheet";
const all_freq = all_occurrences.length;
if (max_allsheet <= all_freq)
return "Too much in the work";
//test unicity in a position
const active_sname = activesheet.getName();
for (occurrence of all_occurrences)
{
const sname = occurrence.getSheet().getName();
//if (SYSTEM_SHEETS.includes(sname))
//continue;
if (sname != active_sname)
if (occurrence.getRow() == row && occurrence.getColumn() == col)
if (occurrence.getValue() == data)
{
return `${sname} contains same data with the same indexes.`;
};
}
return data;
}
Create two or three cells and put randomly in a short range short range a value following the usage
=safeInsertion("Scarce Data", 3; 5; ROW(), COLUMN())
Do it, probably you will get a unstable sheet.
About cached values confuse me sometimes. The script is changed but not perceived by the sheet until renewing manually the cell's content or refreshing all table. No relevant configuration available to this issue?, when you want to refresh your custom function of safeInsertion, I thought that this thread might be useful.
About Sometimes, at loading, a messing result appears. Almost all data are prohibited, for example (originally, all was fine!). and What can I do to obtain a stable sheet using this approach?, in this case, for example, how about reducing the process cost of your script? I thought that by reducing the process cost of the script, your situation might be a bit stable.
When your script is modified by reducing the process cost, how about the following modification?
Modified script:
function safeInsertion(data, row_, col_) {
const ss = SpreadsheetApp.getActiveSpreadsheet();
const range = ss.createTextFinder(data).matchEntireCell(true).findNext();
return range && range.getRow() == row_ && range.getColumn() == col_ && range.getSheet().getSheetName() != ss.getActiveSheet().getSheetName() ? "prohibited insertion" : data;
}
The usage of this is the same with your current script like =safeInsertion("A scarce data", ROW(), COLUMN()).
In this modification, TextFinder is used. Because I thought that when the value is searched from all sheets in a Google Spreadsheet, TextFinder is suitable for reducing the process cost.
References:
createTextFinder(findText) of Class Spreadsheet
findNext()

InDesign - Script to find specific words between angle brackets, and copy those to a list

Sorry in advance if I’m not phrasing this question correctly. I know nothing about InDesign scripting, but this would solve a workflow problem I’m having at my company, so I would appreciate any and all help.
I want to find all strings in an InDesign file that are between angle brackets (i.e. <variable>) and export that out into a list. Ideally this list would be a separate document but if it can just be dumped into a text frame or something that’s fine.
Any ideas on how to do this? Thank you in advance for any and all help.
Here is something simple:
app.findGrepPreferences=NothingEnum.NOTHING; //to reset the Grep search
app.findGrepPreferences.findWhat = '<[^<]+?>'; //the word(s) you are searching
var fnd = app.activeDocument.findGrep (); //execute search
var temp_str = ''; //initialize an empty string
for (var i = 0; i < fnd.length; i++) { //loop through results and store the results
temp_str += fnd[i].contents.toString() + '\r'; // add next found item text to string
}
var new_doc = app.documents.add (); //create a new document
app.scriptPreferences.measurementUnit = MeasurementUnits.POINTS; //set measurement units to points
var text_frame = new_doc.pages[0].textFrames.add({geometricBounds:[0,0,100,100]});// create a new text frame on the first page
text_frame.contents = temp_str; //output your text to the text frame in the new document
For more data see here.

More efficient way to add multiple records

So this works but it takes 15 seconds for a spreadsheet with 60 items.
function addToModel(name,birth,age){
var newRecord = app.models.ImportData.newRecord();
newRecord['PRESIDENT'] = name;
newRecord['BIRTH_PLACE'] = birth;
newRecord['AGE_ELECTED'] = age;
app.saveRecords([newRecord]);
}
function getSpreadsheet(){
var sh = SpreadsheetApp.openById("zzz");
var ss = sh.getSheetByName("Sheet1");
var data = ss.getDataRange().getValues();
THIS WAS WAY ONE, TAKES 15 SECONDS
for (var i=1; i<data.length;i++)
{
addToModel(data[i][1],data[i][2],data[i][3].toString());
}//for loop
}
but I noticed that the command is saveRecordS not saveRecord and with anything in google apps script, the fewer calls the better, so I tried this but it doesn't work
//SAME SPREADSHEET INFO
var result = [];
for (var i=0; i<data.length;i++)
{
var newRecord = app.models.ImportData.newRecord();
newRecord['PRESIDENT'] = data[i][1];
newRecord['BIRTH_PLACE'] = data[i][2];
newRecord['AGE_ELECTED'] = data[i][3].toString();
result.push(newRecord);
}//for loop
app.saveRecords([result]);
Expected result: new records in my table, much faster than the first version. Actual result: "Cannot read property "key" from undefined" which is triggered from the last line (saveRecords). I tried both app.saveRecords(result) and ([result]), same problem both times.
Note: this example is from an appmaker university tutorial that no longer works because of the changes for appmaker v2.
I think that it's model.newRecord() takes time for each new item created, while time on app.saveRecords() is ignorable.
Could you please confirm that you are EU user? as we EU user are facing same issue (link) due to the server location, if it is the case, please help star that issue and give more info to help Google solve that issue. Thanks.

Trying to set a trigger

So I'm working on creating my own workout sheet to run on my mobile device b/c I don't like anything the app store offers.
Anyways, I want the sheet to automatically increase the weight being lifted based on a trigger. I've already got the sheet to return the next amount of weight to be lifted using formulas, now I want a trigger to run when the sheet calculates 25 reps to copy that increased value into the weight cell.
I'm trying to use the onEdit function. It runs without giving back an error, but nothing changes on the sheet.
Here's the code I have:
function myFunction() {
function onEdit(e) {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getActiveSheet();
if ("Data!D15" == "False") {
sheet.getrange("Sheet1!C4").setValue("Data!E15+0");
}
if ("Data!G15" == "0") {
sheet.getrange("Sheet1!C4").setValue("Data!D15+5");
}
}
}
Try this:
function onEdit(e) {
var sh1 = e.range.getSheet();
var sh2 = e.source.getSheetByName('Data');
if(!sh2.getRange('D15').getValue()){sh1.getrange("Sheet1!C4").setValue("Data!E15+0");}
if(sh2.getRange('G15').getValue()=="0"){sh1.getrange("Sheet1!C4").setValue("Data!D15+5");}
}

Google Sheets: delete rows containing specified data

I'm new to Java scripting and Google Apps Scripts so i am sorry if this has already been answered. I was not able to find what i was looking for over the last few months of working on this project.
I am working on a variant of the scripts here:
Delete row in Google Sheets if certain "word" is found in cell
AND
Google Sheet Script - Find Value in Col and Delete Row
I want to create a button, or menu, that will allow someone to enter specific data, and have each row in the spreadsheet containing that data deleted.
I have a test sheet here that illustrates the data i'm working with, formulas i'm using, and has the beginning of the script attached to it:
https://docs.google.com/spreadsheets/d/1e2ILQYf8MJD3mrmUeFQyET6lOLYEb-4coDTd52QBWtU/edit?usp=sharing
The first 4 sheets are pulling data from the "Form Responses 1" sheet via a formula in cell A:3 in each sheet so the data would only need to be deleted from the "Form Responses 1" sheet to clear it from the rest of the sheets.
I tried working this in but i do not think i am on the right track.
https://developers.google.com/apps-script/guides/dialogs
I also posted this on Google Docs Help Forum 60 days ago, but have not received any responses.
Any help would be greatly appreciated.
There's a few steps. For usability of UI this takes a little longer code. In concise form:
The user activates a dialog and enters a string.
Rows w/ the string are deleted (with error handling and confirmation)
(Hopefully this gets you started and you can tailor it to your needs)
Function that initiates the menu:
function onOpen(){
SpreadsheetApp.getUi()
.createMenu('My Menu')
.addItem('Delete Data', 'deleteFunction')
.addToUi();
}
The main workhorse:
function deleteFunction(){
//declarations
var sheetName = "Form Responses 1";
var ss = SpreadsheetApp.getActive();
var sheet = ss.getSheetByName(sheetName);
var dataRange = sheet.getDataRange();
var numRows = dataRange.getNumRows();
var values = dataRange.getValues();
var delete_string = getUIstring();//open initial UI, save value
if (delete_string.length < 3) return shortStringError()//UI to protect your document from an accidental entry of a very short string.
//removing the rows (start with i=2, so don't delete header row.)
var rowsDeleted = 0;
for (var i = 2; i <= numRows; i++){
var rowValues = values[i-1].toString();//your sheet has various data types, script can be improved here to allow deleting dates, ect.
if (rowValues.indexOf(delete_string) > -1){
sheet.deleteRow(i - rowsDeleted);//keeps loop and sheet in sync
rowsDeleted++;
}
}
postUIconfirm(rowsDeleted);//Open confirmation UI
}
Isolated UI functions to help make above function more concise:
function getUIstring(){
var ui = SpreadsheetApp.getUi();
var response = ui.prompt("Enter the target data element for deletion")
return response.getResponseText()
}
function postUIconfirm(rowsDeleted){
var ui = SpreadsheetApp.getUi();
ui.alert("Operation complete. There were "+rowsDeleted+" rows deleted.")
}
function shortStringError(){
var ui = SpreadsheetApp.getUi();
ui.alert("The string is too short. Enter a longer string to prevent unexpected deletion")
}
I'll just show a way to delete the cell value if it matches your search criteria. It's up to you to connect it to buttons ,etc.
You'll loop through a Sheet Range. When you find the word match, delete it using clearContent()
function deleteSpecificData() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheets()[0];
var range = sheet.getRange("Sheet1!A1:C4");
var values = range.getValues();
var numArray = [1,2,3,4,5,6,7,8,9];
var deleteItem = "Garen";
Logger.log(range);
for(var i=0; i< values.length; i++){
for(var j=0; j<values[i].length; j++){
if(values[i][j] == deleteItem){
var row = numArray[i];
var col = numArray[j];
var range = sheet.getRange(row,col).clearContent();
}
}
}
}
Before:
After:

Resources