I am trying to scrape some data, stored in a table of a webpage that has observations.
I am using readLines() in order to get the page.
My problem is that, when checking the Lines grabbed, in the position that the table should be located at, there is a message: "No data recorded".
When loading this page in google chrome, this message "No data recorded" appears when the webpage is initially loaded. It stays for 3-4 seconds and then the message disappears and the table with the daily observations gets loaded.
So I guess that when using readLines(), the function does not "wait" to have the table loaded.
the url of the page is:
https://www.wunderground.com/history/daily/gr/mykonos/LGMK/date/2017-11-1
and the codeline I type is:
thepage = readLines('https://www.wunderground.com/history/daily/gr/mykonos/LGMK/date/2017-11-1')
thepage[695:700]
Just this morning when trying to view the Data Explorer UI for an Azure Cosmos DB table the window is totally blank and I see no rows (the table should not be empty). The only connection to this table is a Python script that pushes in simple rows with only a few variables however this has also stopped working just this morning.
I am still able to connect to the table service properly and I've even been able to create a new table through my Python script. However, as soon as I call table_service.insert_or_replace_entity('traps', task) ('traps' is the name of my table and task is the row I'm trying to push up) I receive back an HTTP Error 400. The request URL is invalid.
For reference, my connection in Python is as follows where Account_Name = my personal account name and Account_Key = my personal account key.
table_service = TableService(connection_string="DefaultEndpointsProtocol=https;AccountName=Account_Name;AccountKey=Account_Key;TableEndpoint=https://Account_Name.table.cosmosdb.azure.com:443/;")
for i in list(range(0,len(times))):
print(len(tags))
print(len(times))
print(len(locations))
task = {'PartitionKey': '1', 'RowKey': '{}'.format(tags[i]),'Date_Time' : '{}'.format(times[i]), 'Location' : '{}'.format(locations[i])}
table_service.insert_or_replace_entity('traps', task)
UPDATE
In reference to the HTTP Error 400 I discovered that I was trying to push a \n at the end of each of the tags string (i.e. tags[0] = 'ab123\n'). Stripping out the \n has resolved the HTTP 400 error but I am now receiving The specified resource does not exist. message when I attempt to upload which makes more sense as at why my Data Explorer is blank. I have tried uploading to a new table but its the same thing.
Second Update
Silly mistake on resource not found error was that my table is called "Traps" not "traps". Data appears to be uploading correctly now on the API side. However, the table is still not displaying at all in the data explorer page of the Azure portal. If anyone has insight on this it would be appreciated because the explorer is super helpful while we are still in development.
Third Update
I am able to connect to the table/database through Python and query data effectively. It all seems to be in there and up to date. The only thing I'm left unsure about is why the Data Explorer is not displaying properly. Aside from that, my recommendation is to obviously check your capital letters (my usual mistake haha) and DO NOT try to push up line feeds (\n) in the task/payload.
Want to provide an official update and response to your issue. This issue is being Hotfixed with an ETA rolled out by Monday (09/24/2018).
Firefox has console log limits that can be configured to my understanding, but I'm thinking the IE11 console also has a limit on the number of messages it logs before it fails and stops logging.
I was wondering why my later log messages were not showing up in the log and I would see console warning text of "An error has occurred in the JavaScript Console. Functionality might be affected." when there was no error logged.
So I stuck a simple loop to generate a bunch of log messages to the console.
var li=0;
for (li=0; li<500; ++li) console.log("li %d", li);
console.info("Finished with list of log entries.");
Along with the other informational console log messages I had (6 others) the list ended at 499, so I bumped up the cutoff to 5000:
var li=0;
for (li=0; li<5000; ++li) console.log("li %d", li);
console.info("Finished with list of log entries.");
Now the logging ends at "li 994" as the last line.
The "Finished..." line doesn't even display.
Looking up the console to see how many messages were displayed, I see there are 6 others, not of the "li xxx" type.
So it appears that IE11 has a console limit of 1,000 entries and the stops logging and fails.
Can someone confirm and is there a way to set that limit higher?
My javascript code does some array processing and there is a lot of array dumping of data that goes on. I'm using console groups and they work, but when the entire logging shuts down after the 1000th message, that is a huge problem.
For some reason, when I use the following code in package I'm trying to contribute to, an integral sign (∫) appears in the active document when a button is selected on the dialog.
checkAutoSave: ()->
if atom.workspace.getActiveTextEditor().isModified()
if atom.config.get('build.saveOnBuild') is true
atom.workspace.getActiveTextEditor().save()
return 1
else if atom.config.get('build.promptToSaveOnBuild') is true
diaDirection = atom.confirm
message: 'Do you want to save the current file before building?'
detailedMessage: 'This message can be turned off in the Build settings.'
buttons: ['Yes', 'No', 'Cancel Build']
switch diaDirection
when 0
atom.workspace.getActiveTextEditor().save()
return 1
when 1
return 0
when 2
return -1
else #Current File wasn't changed.
return 0
I have tried narrowing it down and I am 100% it has something to do with the dialog. The problem does not exists without the dialog call. The Integral sign appears in the document regardless of the button pressed.
I've had a similar problem a while ago - see here for more details.
The action I wanted to trigger in the editor was bound to Ctrl+Alt+B, which by default inserts the integral sign on Mac OS X. The integral sign was inserted into the currently open document when I opened a standard alert box from my package's code.
I didn't found out why, but it looks similar to your problem. In the end, I resolved it by removing the alert and using an Atom view instead for showing the message. As soon as I did this, the integral sign was no longer inserted into the document.
It looks like there's an issue with the key binding and preventing event propagation when using some dialogs. In some cases, the key event is handed to the OS and it inserts the character associated with the pressed key.
Maybe you can try using an Atom view instead of the confirm dialog, and it will resolve your issue as well.
I am running a simple query to get data out of my database & display them. I'm getting an error that says Response Buffer Limit Exceeded.
Error is : Response object error 'ASP 0251 : 80004005'
Response Buffer Limit Exceeded
/abc/test_maintenanceDetail.asp, line 0
Execution of the ASP page caused the Response Buffer to exceed its configured limit.
I have also tried Response.flush in my loop and also use response.buffer = false in my top of the page, but still I am not getting any data.
My database contains 5600 records for that, Please give me some steps or code to solve the issue.
I know this is way late, but for anyone else who encounters this problem: If you are using a loop of some kind (in my case, a Do-While) to display the data, make sure that you are moving to the next record (in my case, a rs.MoveNext).
Here is what a Microsoft support page says about this:
https://support.microsoft.com/en-us/help/944886/error-message-when-you-use-the-response-binarywrite-method-in-iis-6-an.
But it’s easier in the GUI:
In Internet Information Services (IIS) Manager, click on ASP.
Change Behavior > Limits Properties > Response Buffering Limit from 4 MB to 64 MB.
Apply and restart.
The reason this is happening is because buffering is turned on by default, and IIS 6 cannot handle the large response.
In Classic ASP, at the top of your page, after <%#Language="VBScript"%> add:
<%Response.Buffer = False%>
In ASP.NET, you would add Buffer="False" to your Page directive.
For example:
<%#Page Language="C#" Buffer="False"%>
I faced the same kind of issue, my IIS version is 8.5. Increased the Response Buffering Limit under the ASP -> Limit Properties solved the issue.
In IIS 8.5, select your project, you can see the options in the right hand side. In that under the IIS, you can see the ASP option.
In the option window increase the Response Buffering Limit to 40194304 (approximately 40 MB) .
Navigate away from the option, in the right hand side top you can see the Actions menu, Select Apply. It solved my problem.
If you are not allowed to change the buffer limit at the server level, you will need to use the <%Response.Buffer = False%> method.
HOWEVER, if you are still getting this error and have a large table on the page, the culprit may be table itself. By design, some versions of Internet Explorer will buffer the entire content between before it is rendered to the page. So even if you are telling the page to not buffer the content, the table element may be buffered and causing this error.
Some alternate solutions may be to paginate the table results, but if you must display the entire table and it has thousands of rows, throw this line of code in the middle of the table generation loop: <% Response.Flush %>. For speed considerations, you may also want to consider adding a basic counter so that the flush only happens every 25 or 100 lines or so.
Drawbacks of not buffering the output:
slowdown of overall page load
tables and columns will adjust their widths as content is populated (table appears to wiggle)
Users will be able to click on links and interact with the page before it is fully loaded. So if you have some javascript at the bottom of the page, you may want to move it to the top to ensure it is loaded before some of your faster moving users click on things.
See this KB article for more information http://support.microsoft.com/kb/925764
Hope that helps.
Thank you so much!
<%Response.Buffer = False%> worked like a charm!
My asp/HTML table that was returning a blank page at about 2700 records. The following debugging lines helped expose the buffering problem: I replace the Do While loop as follows and played with my limit numbers to see what was happening:
Replace
Do While not rs.EOF
'etc .... your block of code that writes the table rows
rs.moveNext
Loop
with
Do While reccount < 2500
if rs.EOF then recount = 2501
'etc .... your block of code that writes the table rows
rs.moveNext
Loop
response.write "recount = " & recount
raise or lower the 2500 and 2501 to see if it is a buffer problem. for my record set, I could see that the blank page return, blank table, was happening at about 2700 records, good luck to all and thank you again for solving this problem! Such a simple great solution!
You can increase the limit as follows:
Stop IIS.
Locate the file %WinDir%\System32\Inetsrv\Metabase.xml
Modify the AspBufferingLimit value. The default value is 4194304, which is about 4 MB.
Changing it to 20MB (20971520).
Restart IIS.
One other answer to the same error message (this just fixed my problem) is that the System drive was low on disk space. Meaning about 700kb free. Deleting a lot of unused stuff on this really old server and then restarting IIS and the website (probably only IIS was necessary) cause the problem to disappear for me.
I'm sure the other answers are more useful for most people, but for a quick fix, just make sure that the System drive has some free space.
I rectified the error 'ASP 0251 : 80004005' Response Buffer Limit as follow:
To increase the buffering limit in IIS 6, follow these steps:
Click Start, click Run, type cmd, and then click OK.
Type the following command, and then press ENTER:
cd /d %systemdrive%\inetpub\adminscripts
Type the following command, and then press ENTER:
cscript.exe adsutil.vbs SET w3svc/aspbufferinglimit LimitSize
Note LimitSize represents the buffering limit size in bytes. For example, the number 67108864 sets the buffering limit size to 64 MB.
To confirm that the buffer limit is set correctly, follow these steps:
Click Start, click Run, type cmd, and then click OK.
Type the following command, and then press ENTER:
cd /d %systemdrive%\inetpub\adminscripts
Type the following command, and then press ENTER:
cscript.exe adsutil.vbs GET w3svc/aspbufferinglimit
refers to https://support.microsoft.com/en-us/kb/944886
If you are looking for the reason and don't want to fight the system settings, these are two major situations I faced:
You may have an infinite loop without next or recordest.movenext
Your text data is very large but you think it is not! The common reason for this situation is to copy-paste an Image from Microsoft word directly into the editor and so the server translates the image to data objects and saves it in your text field. This can easily occupies the database resources and causes buffer problem when you call the data again.
In my case i just have writing this line before rs.Open .....
Response.flush
rs.Open query, conn
It can be due to CursorTypeEnum also. My scenario was the initial value equal to CursorTypeEnum.adOpenStatic 3.
After changed to default, CursorTypeEnum.adOpenForwardOnly 0, it backs to normal.