Is there something (I can get the value of) in patch, test_posting, or mock_post that will tell me what values were passed to the original requests.post (or maybe there's some other way)?
The service function looks like this:
def post_to_api_server():
response = requests.post('{}{}'.format(API_SERVER_URL, SEARCH_PATH),
data=payload, headers=header_json, verify=False)
if response.ok:
return response
else:
return None
The test function looks like this:
#patch('requests.post')
def test_posting(mock_post):
mock_post.return_value = Mock(ok=True)
mock_post.return_value.json.return_value = my_test_json
response = post_to_api_server()
With a hint from #mrbean-bremmen, add something like this after the call to response = post_to_api_server():
args_tuple, kwargs_dict = mock_post.call_args
url = args_tuple[0]
if 'data' in kwargs_dict:
data = kwargs_dict['data']
if 'json' in kwargs_dict:
json = kwargs_dict['json']
Related
I would want to test my graphql mutation which should set a HTTP only cookie.
I use graphene-django GraphQTestCase
class CookieTest(GraphQLTestCase):
def test_set_cookie(self):
response = self.query(...) # This should return WSGIRequest
cookie = response.COOKIES.get('myCookie') # This line throws an attrib error
self.assertIsNotNone(cookie)
What is the problem?
For some reason response.cookies works, and returns http.cookies.SimpleCookie object:
So the correct way would be:
myCookie = response.cookies.get('myCookie')
myCookie.key # 'myCookie'
myCookie.value # <cookie-value>
'''
QuestionModel getQuestionModelFromDatasnapshot(
DocumentSnapshot questionSnapshot) {
QuestionModel questionModel = new QuestionModel();
questionModel.question = questionSnapshot.data()'question'];
List<String> options = [
questionSnapshot.data()["option1"],
questionSnapshot.data()["option2"],
questionSnapshot.data()["option3"],
questionSnapshot.data()["option4"]
];
options.shuffle();
questionModel.option1 = options[0];
questionModel.option2 = options[1];
questionModel.option3 = options[2];
questionModel.option4 = options[3];
questionModel.correctOption = questionSnapshot.data()["option1"];
questionModel.answered = false;
print(questionModel.correctOption.toLowerCase());
return questionModel;
}
'''
It looks like you're using dart with null safety on and you are accessing a list or map that might be null. It's hard to tell from your code, but I'm guessing the offending line is
questionModel.correctOption = questionSnapshot.data()["option1"];
It might be one of the other [] calls earlier in your code (such as
questionSnapshot.data()["option1"]), but the solution is the same.
The problem is that questionSnapshot.data() might return null, and the compiler is flagging it.
If you are sure the data is not null, you can use the ! operator, like this:
questionModel.correctOption = questionSnapshot.data()!["option1"];
Another solution is a conditional null-safe operator ?, with a default value. In this case, the code would look like this:
questionModel.correctOption = questionSnapshot.data()?["option1"]??"DefaultValue;
But I would recommend checking if the returned data is null. It's easier for others to read and can be logged more easily:
var data = questionSnapshot.data();
if (data != null) {
questionModel.correctOption = data["option1"];
}
else {
//do something in that case
}
I am scraping from a website that will give every request session a sid, after getting the sid, I perform further search query with this sid and scrape the results.
I want to change the sid every time I've finished scraping all results of a single query, I've tried clearing the cookies but it doesn't work.
However, if I restart my crawler, it wll get a different sid each time, I just don't know how to get a new sid without restart the crawler.
I am wondering if there're something else that let the server know two requests are from the same connection.
Thanks!
Here is my current code:
class MySpider(scrapy.Spider):
name = 'my_spider'
allowed_domains = ['xxx.com']
start_urls = ['http://xxx/']
sid_pattern = r'SID=(\w+)&'
SID = None
query_list = ['aaa', 'bbb', 'ccc']
i = 0
def parse(self, response):
if self.i >= len(self.query_list):
return
pattern = re.compile(self.sid_pattern)
result = re.search(pattern, response.url)
if result is not None:
self.SID = result.group(1)
else:
exit(-1)
search_url = 'http://xxxx/AdvancedSearch.do'
query = self.query_list[i]
self.i += 1
query_form = {
'aaa':'bbb'
}
yield FormRequest(adv_search_url, method='POST', formdata=query_form, dont_filter=True,
callback=self.parse_result_entry)
yield Request(self.start_urls[0], cookies={}, callback=self.parse,dont_filter=True)
def parse_result(self, response):
do something
Setting COOKIES_ENABLED = False can achieve this, but is there another way other than a global settings?
I've created a dictionary from an Uploaded file in Django.
This dictionary has a nested list of dictionaries:
file = {"name": "filename", "sections": [{"section_name": "string", "lines": [{line_number: 0, "line"; "data"}]}], "etc": "etc"}
The model represents the dictionaries depth too.
class Line(EmbeddedDocument):
line_number = IntField()
line = StringField()
definition = ReferenceField(Definition)
class Section(EmbeddedDocument):
section_name = StringField()
lines = EmbeddedDocumentListField(Line))
class File(Document):
name = StringField()
sections = EmbeddedDocumentListField(Section))
created_on = DateTimeField()
created_by = StringField()
modified_on = DateTimeField()
modified_by = StringField()
In the POST I have the following to chop the file up into the above Dict (the file is a simple text file):
file= {}
with open(os.path.join(path, filename + ".txt"), 'r') as temp_file:
filelines = temp_file.readlines()
sections = []
section = {}
lines = []
for i, l in enumerate(filelines):
if i == 0:
section["section_name"] = "Top"
elif '*' in l:
if l.index('*') == 0 and '*' not in lines[len(lines) - 2"line"]:
section["lines"] = lines
lines = []
sections.append(section)
section = dict()
section["section_name"] = filelines[i + 1][1:-2]
line = {"line_number": i + 1, "line": l}
lines.append(line)
section['lines'] = lines
sections.append(section)
file["name"] = filename
file["sections"] = sections
I will tidy this up eventually.
Once the dict has been made how do I serialise it using the serializer?
Is it possible to insert this into a serializer?
If not how can I get it all into the database with validation?
I've tried json.dumps() and JsonRequst() then putting them in data= for the serializer but get Unable to get repr for <class '....'>
I'm pretty new to Django and MongoDB so if you need more info I can provide :)
Thanks!
Update
Change the model's List Fields to EmbeddedDocumentListField as suggest in the answer.
Answered
Thanks to Boris' suggestion below it pointed me to an error I wasn't getting initially. I had a typo and passing the dict directly into FileSerializer(data=file) works like a charm! :)
James!
The easiest way to validate that your incoming JSONs adhere to the Mongoengine Documents schema that you've specified is to use DRF-Mongoengine's DocumentSerializer.
Basically, what you need to do is create a serializer
serializers.py
import rest_framework_mongoengine
class FileSerializer(rest_framework_mongoengine.DocumentSerializer):
class Meta:
fields = '__all__'
model = File
Then you need a view or viewset that makes use of this Serializer to respond to GET/POST/PUT/DELETE requests.
views.py
from rest_framework_mongoengine import viewsets
class FileViewSet(viewsets.ModelViewSet):
lookup_field = 'id'
serializer_class = FileSerializer
def get_queryset(self):
return File.objects.all()
and register this viewset with a router
urls.py
from rest_framework import routers
# this is DRF router for REST API viewsets
router = routers.DefaultRouter()
# register REST API endpoints with DRF router
router.register(r'file', FileViewSet, r"file")
I'd also recommend using EmbeddedDocumentListField instead of ListField(EmbeddedDocumentField(Section)) - it has additional methods.
What if i only want to process image from disk for reading text from it and storing them in text file.
As it is working for both json and data. i want to do work with data only. How to do that?
from __future__ import print_function
import time
import requests
import cv2
import operator
import numpy as np
# Import library to display results
import matplotlib.pyplot as plt
%matplotlib inline
_url = 'https://api.projectoxford.ai/vision/v1/analyses'
_key = 'd784ea882edd4feaa373dc5a80fa87e8'
_maxNumRetries = 10
def processRequest( json, data, headers, params ):
"""
Helper function to process the request to Project Oxford
Parameters:
json: Used when processing images from its URL. See API Documentation
data: Used when processing image read from disk. See API Documentation
headers: Used to pass the key information and the data type request
"""
retries = 0
result = None
while True:
response = requests.request( 'post', _url, json = json, data = data, headers = headers, params = params )
if response.status_code == 429:
print( "Message: %s" % ( response.json()['error']['message'] ) )
if retries <= _maxNumRetries:
time.sleep(1)
retries += 1
continue
else:
print( 'Error: failed after retrying!' )
break
elif response.status_code == 200 or response.status_code == 201:
if 'content-length' in response.headers and int(response.headers['content-length']) == 0:
result = None
elif 'content-type' in response.headers and isinstance(response.headers['content-type'], str):
if 'application/json' in response.headers['content-type'].lower():
result = response.json() if response.content else None
elif 'image' in response.headers['content-type'].lower():
result = response.content
else:
print( "Error code: %d" % ( response.status_code ) )
print( "Message: %s" % ( response.json()['error']['message'] ) )
break
return result
def renderResultOnImage( result, img ):
"""Display the obtained results onto the input image"""
R = int(result['color']['accentColor'][:2],16)
G = int(result['color']['accentColor'][2:4],16)
B = int(result['color']['accentColor'][4:],16)
cv2.rectangle( img,(0,0), (img.shape[1], img.shape[0]), color = (R,G,B), thickness = 25 )
if 'categories' in result:
categoryName = sorted(result['categories'], key=lambda x: x['score'])[0]['name']
cv2.putText( img, categoryName, (30,70), cv2.FONT_HERSHEY_SIMPLEX, 2, (255,0,0), 3 )
pathToFileInDisk = r'test.jpg'
with open( pathToFileInDisk, 'rb' ) as f:
data = f.read()
# Computer Vision parameters
params = { 'visualFeatures' : 'Color,Categories'}
headers = dict()
headers['Ocp-Apim-Subscription-Key'] = _key
headers['Content-Type'] = 'application/octet-stream'
json = None
result = processRequest( json, data, headers, params )
if result is not None:
# Load the original image, fetched from the URL
data8uint = np.fromstring( data, np.uint8 ) # Convert string to an unsigned int array
img = cv2.cvtColor( cv2.imdecode( data8uint, cv2.IMREAD_COLOR ), cv2.COLOR_BGR2RGB )
renderResultOnImage( result, img )
ig, ax = plt.subplots(figsize=(15, 20))
ax.imshow( img )
It's showing sytax error at %matplot inline
I gather you copied your Python code from somewhere, and there are a number of issues with it:
Your syntax error is stemming from the fact that %matplotlib is valid syntax for iPython, not plain Python.
Based on your problem description, IIUC, you've no need for any plotting code, so you might as well drop matplotlib (and cv2 and numpy, for that matter).
Your API URL is wrong: you want https://api.projectoxford.ai/vision/v1.0/ocr.
The code you'll want will be basically like this:
import json
import requests
import urllib
headers = {
# Request headers
'Content-Type': 'application/json',
'Ocp-Apim-Subscription-Key': 'YOUR_KEY_HERE',
}
params = urllib.urlencode({
# Request parameters
'language': 'unk',
'detectOrientation ': 'true',
})
body = {"url":"YOUR_URL_HERE"}
response = requests.post("https://api.projectoxford.ai/vision/v1.0/ocr?%s" % params, json=body, headers=headers)
result = response.json()
for region in result['regions']:
for line in region['lines']:
for word in line['words']:
print word['text']
Get more details about the response JSON on the API page, if you want, for instance, to arrange the text differently.
You forgot to redact your API key, so you'll probably want to generate a new one via the subscriptions page.