I am working on creating a asyncio.Future callback for requests whereby I store the body of the response in json format and return this result. I initially wanted to create a callback on the function on_response, however I was not certain how to implement this. Therefore, I decided to include option parameters to add into the callable such as url.
For example:
import asyncio
from functools import partial as func
import requests
import json
url = "https://www.scrapethissite.com/pages/ajax-javascript/?ajax=true&year=2015"
class schedulerLoop(asyncio.Future):
def __init__(self, url):
super(schedulerLoop, self).__init__()
self._url = url
#staticmethod
def unwrapper(funct):
return funct()
def _future(self, *args):
return self.add_done_callback(func(self.unwrapper, *args))
async def on_response(self):
if self.done():
obj = await self._future(func(requests.get, self._url))
body = json.loads(obj)
return body
else:
self.exception()
async def main(loop):
await loop.on_response()
loop = schedulerLoop(url)
if __name__ == '__main__':
asyncio.run(main(loop))
I find that I get:
asyncio.exceptions.InvalidStateError: Exception is not set.
Which supposes that on_response the Future is not done, and so we get a False result but there seems be no no Exception neither.
Related
I want to wrap the upload part in a common function since it is used in multiple API routes, but how do I do it since it is using async def here.
#app.post("/api/od")
async def image_classification(files: typing.List[fastapi.UploadFile] = fastapi.File(...)):
upload_path = pathlib.Path("upload")#.joinpath(token)
upload_path.mkdir(exist_ok=True)
...
return results
Hello i am trying to make a http request in flutter for web, but I am having a lot of problems here.
Response for any request I am doing looks like this:
I am making requests like this:
Ok, I realised that if i decode response like this:
var responseBody = json.decode(utf8.decode(response.bodyBytes));
I get:
Still I don't have full key-value response and the status code
How can I get normal respone?
I think you you need to decode you response body with json.decode from dart:convert.
import 'package:http/http.dart' as http;
import 'dart:convert';
final response = await http.get('https://dog.ceo/api/breeds/image/random');
if (response.statusCode == 200) {
final Map<String, dynamic> responseMap = json.decode(response.body);
}
I'm trying to parse a url image in order to turn it into Uint8 bytes and share across my share provider. When trying to consolidate my http response into bytes I get the following error The method 'consolidateHttpClientResponseBytes' isn't defined by my class. Here is the code I'm trying to use..... Please help!
Future<Null> shareImage() async{
var request = await HttpClient().getUrl(Uri.parse('network_image_url'));
var response = await request.close();
Uint8List bytes = await consolidateHttpClientResponseBytes(response);
}
Import this :
import 'dart:typed_data';
import 'package:flutter/foundation.dart';
import 'package:path_provider/path_provider.dart';
And you should be good
I wanna make a web crawling, currently i am reading a txt file with 12000 urls, i wanna use concurrency in this process, but the requests don't work.
typealias escHandler = ( URLResponse?, Data? ) -> Void
func getRequest(url : URL, _ handler : #escaping escHandler){
let session = URLSession(
configuration: .default,
delegate: nil,
delegateQueue: nil)
var request = URLRequest(url:url)
request.httpMethod = "GET"
let task = session.dataTask(with: request){ (data,response,error) in
handler(response,data)
}
task.resume()
}
for sUrl in textFile.components(separatedBy: "\n"){
let url = URL(string: sUrl)!
getRequest(url: url){ response,data in
print("RESPONSE REACHED")
}
}
If you have your URLSessions working correctly, all you need to go is create separate OperationQueue create a Operation for each of your async tasks you want completed, add it to your operation queue, and set your OperationQueue's maxConcurrentOperationCount to control how many of your tasks can run at one time. Puesdo code:
let operationQueue = OperationQueue()
operationQueue.qualityOfService = .utility
let exOperation = BlockOperation(block: {
//Your URLSessions go here.
})
exOperation.completionBlock = {
// A completionBlock if needed
}
operationQueue.addOperation(exOperation)
exOperation.start()
Using a OperationQueue subclass and Operation subclass will give you additional utilities for dealing with multiple threads.
I have a TemplateView that returns a TemplateResponse object in a process_template_response middle-ware, but the later is never triggered.
When i change the middleware method in process_response and preform render() on the TemplateResponse, the middle-ware method is triggered and the page is render.
Which steps do i need preform to render a view in a process_template_response?
View:
class PageView(TemplateView):
template_name = 'flatpages/default.html'
def get(self, request, *args, **kwargs):
url = kwargs['url']
if not url.endswith('/') and settings.APPEND_SLASH:
return HttpResponseRedirect(url + '/')
if not url.startswith('/'):
url = url + '/'
kwargs.update({'url': url})
context = self.get_context_data(**kwargs)
return self.render_to_response(context)
def get_context_data(self, **kwargs):
url = kwargs.pop('url')
context = super(PageView, self).get_context_data(**kwargs)
page = get_object_or_404(ParentPage, url__exact=url, sites__id__exact=settings.SITE_ID)
context.update({'flatpage': page})
return context
class PageFallbackMiddleware(object):
def process_template_response(self, request, response):
print 'Triggered'
if response.status_code != 404:
return response
try:
return PageView.as_view()(request, url=request.path_info)
except Http404:
return response
except:
if settings.DEBUG:
raise
return response
I think you may have to return a TemplateResponse or similar for the middleware to route the view to another template, returning the response parameter or another HTTPResponse instance is a no-op.