I am trying to scrape some data, stored in a table of a webpage that has observations.
I am using readLines() in order to get the page.
My problem is that, when checking the Lines grabbed, in the position that the table should be located at, there is a message: "No data recorded".
When loading this page in google chrome, this message "No data recorded" appears when the webpage is initially loaded. It stays for 3-4 seconds and then the message disappears and the table with the daily observations gets loaded.
So I guess that when using readLines(), the function does not "wait" to have the table loaded.
the url of the page is:
https://www.wunderground.com/history/daily/gr/mykonos/LGMK/date/2017-11-1
and the codeline I type is:
thepage = readLines('https://www.wunderground.com/history/daily/gr/mykonos/LGMK/date/2017-11-1')
thepage[695:700]
Related
`I am trying to use a loop e.g. asLongAs() in Galing but not getting enough data on google about how to use.
My scenrio is to open a HTML page and that page takes some time to load and for that I have a css selector to check that once the report get loaded we have one css selector to check in the source code.
my code is like:
`exec (http("ABC -${ID} - Id -${ID2}")
.get("web/a/b/c/")
.check(css(.abc).saveAs("URL"))
.exec(session =\> {
val response = session("URL").as\[String\]
println(s"url is: \\n$response")
session
})
exec(http("Open the redirected report - ${ID1} Id-${ID2}")
.get(session =\> session("URL").as\[String\])
Some checks
.check(css(".Image").exists)`
I want to create a loop till this css(.Image) is loading. Because once the URL is hitting at that time this CSS doesn't appear and it takes time to load and i want to calculate that time only.
but not getting enough data on google
Have you tried the official documentation ? It has samples for Java, Scala and Kotlin.
https://gatling.io/docs/gatling/reference/current/core/scenario/#aslongas
I am streaming real time tweets using R script and sending it to Power BI using (Streaming data and REST API) to create a live streaming dashboard.
I have to create a tile which shows tweets over time. But the date/time field does not get reflected in the tile, it is just empty.
I'm not sure if I have to change the datatype or format of the field? Below is how the date/time field looks in R.
The streaming dataset is as below:
Below is the tile I created (Shows blank)
If you don't see anything in the tile, probably your dataset doesn't received anything. Try to post some data manually, e.g. from a PowerShell script (place the correct URL):
$endpoint = "https://api.powerbi.com.com/beta/........."
$Time_Stamp = ($Time_Stamp = Get-Date).Datetime
$payload = #{ "date" = $Time_Stamp; "count" = 98.6; "score" = 98.6 }
Invoke-RestMethod -Method Post -Uri "$endpoint" -Body (ConvertTo-Json #($payload))
Does anything shows up in the tile? As we see from your screenshot, you enabled Historic data analysis. This means that you can create reports with this dataset. Try to make one to show the contents in a table. Is there any rows in this dataset?
UPDATE: Actually, to be sure, copy the sample PowerShell code to post to your dataset and try it. Go to the list of datasets in your workspace, find your one in the list and click on the (i) icon. Then from API Info panel go to PowerShell and copy/paste the code in PowerShell ISE to execute it:
Also, keep in mind that the tile will auto update itself only in a dashboard. You may need to refresh the report to get the up to date data, if you are looking at the tile in a report.
I am trying to scrape data from a web page. Since the page has a dynamic content, I used phantomjs to handle. But, with the codes I am using, I just can download the data seen on the web page. However, I need to input the date range and then submit to get all the data I want.
Here are the codes i used,
library(xml2)
library(rvest)
connection <- "pr.js"
writeLines(sprintf("var page=require('webpage').create();
var fs = require('fs');
page.open('%s',function(){
console.log(page.content);//page source;
fs.write('pr.html', page.content, 'w');
phantom.exit();
});",url),con=connection)
system_input <- paste(path,"phantomjs"," ",connection,sep="")
system(system_input)
Thanks to the codes, I have the html output of the webpage which has been created dynamically.
And as I stated, I also need a date input submit. But I couldn't achieve.
The url is : https://seffaflik.epias.com.tr/transparency/piyasalar/gop/ptf.xhtml
I'm trying to solve an issue where a database column is getting set when navigating to a page in an ASP.NET site. This is a quote management web application, and when opening a quote and navigating to a certain page, the "Expiration Date" is automatically updated to the current date. I have verified that this is occurring prior to triggering the Page_Load function in the ascx.cs file. I have tried to trace what is being run prior to this, but I'm afraid my knowledge of ASP.NET is insufficient, and I don't know exactly where to look. From the pieces I can tell are run prior (for example, the aspx.cs file), I see nothing to indicate any alterations to the record.
When I dig deeper, it seems almost as if the change is taking pace upon leaving the main landing page when editing a quote. If I update the value and travel to any page for the quote except for the main page, it stays the same. And I can travel to the main page, check the record in the database, it stays the same. But it seems like as soon as I navigate from that main page, the Expiration Date will change to the current date. Is there somewhere I can check to see if that's what's happening?
-- EDIT --
Maybe a detail list of actions might help...
View list of quotes in system
Click "Edit" link for quote
View "Quote Details" page, which is first page upon edit access
Look at database query for quote to see exp date is still proper
value
Click "Quote Options" navigation link
See expiration date in "Quote Options" has now changed to current
date
Check database query to see that without performing a known save to
the database the exp date value has updated in the database
Perform update to quote to reset exp date
View any other page in quote edit, return to "Options", see exp
date has not changed again from reset value
Verify proper date with database query
Revisit "Details" page
Again verify correct date with database query
Travel to any other page and then to "Options", or go straight
to "Options", see that exp date has changed to current date
This is the HTML on the link used, which appears to be identical on the other pages...
<a class="tabOff" id="ContentPlaceHolder1_linkButtonAddEditShipping" href='javascript:WebForm_DoPostBackWithOptions(new WebForm_PostBackOptions("ctl00$ctl00$ContentPlaceHolder1$linkButtonAddEditShipping", "", true, "", "", false, true))'>Quote Options</a>
I solved the problem. Sort of. I did discover that the save function was being triggered when leaving the "Details" page, even though no save action was being explicitly activated. The problem was resolved by getting the expiration date from the database at the beginning of the function, and if one was set, pushing that value back in at the very end of the function. While this has in effect prevented the value from being "overwritten", I am still unable to determine why the save function was being triggered, and unable to determine what actions within that function are setting the value to be the current date
Short Version :
If an error is thrown during the execution of an SSRS report after the associated RDP temporary tables have been updated, the next time the report is run for the same parameters, code execution does not insert fresh data into said tables. Any idea why and how to prevent it?
Long Version :
I am trying to modify the run conditions of a standard report (SalesInvoiceReport) so that the original of any invoice can only be printed once. For all reprints, the "Copy preview" option should be used.
For the most part, the customization is working as intended. I used the CustInvoiceJour.PrintedOriginals field and the CustInvoiceJour.updatePrintedOriginals() (both part of Standard Ax) to maintain the number of originals that have been printed and a slight modification to the SalesInvoiceController.outputReport() method throw an error of an original is being re-printed. Below is the code added to achieve this :
...
// <A147> Code to disable reprints if an original is already printed
isOriginalPrintable = CustInvoiceJour::findRecId(custInvoiceJour.RecId).PrintedOriginals == 0;
srsPrintDestinationSettings = formLetterReport.getCurrentPrintSetting().parmPrintJobSettings();
//srsPrintMediumType = srsPrintDestinationSettings.printMediumType();
if ((printCopyOriginal == PrintCopyOriginal::Original || printCopyOriginal == PrintCopyOriginal::OriginalPrint))
{
// If no originals printed, allow prints.
// Need to re-read the CustInvoiceJour record from the table to get updated data
// If the Original is printed but the prin form is not refreshed, the form's recordset does not update to the latest data.
if(isOriginalPrintable)
{
CustInvoiceJour::updatePrinted(custInvoiceJour, (srsPrintDestinationSettings.numberOfCopies() == 0 ? 1 : srsPrintDestinationSettings.numberOfCopies()));
}
// Don't allow the original document to be printed more than once.
else
{
error(strFmt("#GLS223085", custInvoiceJour.InvoiceId, custInvoiceJour.InvoiceDate));
return;
}
}
// </A147>
...
I placed this in the SalesInvoiceContreller.outputReport() because I found similar code present there as part of standard Ax, albeit for use in specific country regions.
When user uses the Original Preview button, the error is thrown and code execution stops but the RDP temporary tables used to hold the report data have already been updated with report data (specifically the field which hold the Report title is set to Invoice).
When the user reruns the report, this time with the Copy Preview button, code execution somehow skips inserting fresh data into the temp tables (which would set the Report Title field to Invoice Copy) and the first copy output still says Invoice, defeating the requirement.