My django project has this articles app in it which has the following urls.py
from django.urls import path
from . import views
urlpatterns = [
path('',views.articles_list),
path("<slug:slug>/",views.article_detail),
]
and the following is the views.py
from django.shortcuts import render
from .models import Article
from django.http import HttpResponse
def articles_list(request):
articles=Article.objects.all().order_by('date')
return render(request,'articles\list.html',{'Articles':articles})
def article_detail(request,slug):
HttpResponse(slug)
Now when I request for /articles/hello-world/ my desired output is a webpage with hello-world but that is not the case. It shows value error.
Now when I request for /articles/hello-world/ my desired output is a webpage with hello-world but that is not the case. It shows value error.
My code to used to work. It did:
import iid from '#react-native-firebase/iid';
...
tokenDevice = await iid().getToken();
But now, I get
TypeError: (0, _iid.default) is not a function. (In '(0, _iid.default)()', '(0, _iid.default)' is undefined)
It seems my code cannot import '#react-native-firebase/iid' anymore.
Besides, other firebase packages have versions 12+, package iid is stuck at 11.5, and react-native firebase documentation does not reference iid anymore
https://rnfb-docs.netlify.app/reference
What can I do to get my device token to send messages to a particular phone ?
The iid module was deprecated. The right code to get the token is
import messaging from '#react-native-firebase/messaging'
const deviceToken = await messaging().getToken()
I’m implementing an app that makes uses of EventEmitter. My aim is to use it on both client and server. But strangely, emitting an event on the server does not work:
Emitter.emit('SomeEvent', {
user,
document,
}
);
The above will work on the client, but not on the server.
I’ve tried importing the event like Emitter = new (require('events').EventEmitter);
I’ve also tried to import a file with
import EventEmitter from 'events';
class ClassEmitter extends EventEmitter {}
const Emitter = new ClassEmitter();
export default Emitter;
My aim is to emit such an event after each database query on the server. I am using apollo and graphql.
Any idea on this would be very thankful.
I am trying to get some technical informations about automobiles from this page
Here is my current code:
import scrapy
import re
from arabamcom.items import ArabamcomItem
from scrapy.spiders import CrawlSpider, Rule
from scrapy.linkextractors import LinkExtractor
class BasicSpider(CrawlSpider):
name="arabamcom"
allowed_domains=["arabam.com"]
start_urls=['https://www.arabam.com/ikinci-el/otomobil']
rules=(Rule(LinkExtractor(allow=(r'/ilan')),callback="parse_item",follow=True),)
def parse_item(self,response):
item=ArabamcomItem()
item['fiyat']=response.css('span.color-red.font-huge.bold::text').extract()
item['marka']=response.css('p.color-black.bold.word-break.mb4::text').extract()
item['yil']=response.xpath('//*[#id="js-hook-appendable-technicalPropertiesWrapper"]/div[2]/dl[1]/dd/span/text()').extract()
And this is my items.py file
import scrapy
class ArabamcomItem(scrapy.Item):
fiyat=scrapy.Field()
marka=scrapy.Field()
yil=scrapy.Field()
When i run the code i can get data from 'marka' and 'fiyat' item but spider does not get anything for 'yil' attribute. Also other parts like 'Yakit Tipi','Vites Tipi' etc. How can i solve this problem ?
What's wrong:
//*[#id="js-hook-appendable-technicalPropertiesWrapper"]/......
This id start with js and may be dynamic element appeded by javascript
Scrapy do not have the ability to render javascript by default.
There are 2 solutions you can try
Scrapy-Splash
This is a javascript rendering engine for scrapy
Install Splash as a Docker container
Modify your settings.py file to integrate splash (append following middlewares to your project)
SPLASH_URL = 'http://127.0.0.1:8050'
SPIDER_MIDDLEWARES = {
'scrapy_splash.SplashDeduplicateArgsMiddleware':100,
}
DOWNLOADER_MIDDLEWARES = {
'scrapy_splash.SplashCookiesMiddleware':723,
'scrapy_splash.SplashMiddleware': 725,
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware': 810,
}
Replace your Request Function with SplashRequest
from scrapy_splash import SplashRequest as SP
SP(url=url, callback=parse, endpoint='render.html', args={'wait': 5})
Selenium WebDriver
This is a browser automation-testing framework
Install Selenium from PyPi and install there corresponding driver(e.g. Firefox -> Geckodriver) to PATH folder
Append following middleware class to your project's middleware.py file:
class SeleniumMiddleware(object):
#classmethod
def from_crawler(cls, crawler):
middleware = cls()
crawler.signals.connect(middleware.spider_opened, signals.spider_opened)
crawler.signals.connect(middleware.spider_closed, signals.spider_closed)
return middleware
def process_request(self, request, spider):
request.meta['driver'] = self.driver
self.driver.get(request.url)
self.driver.implicitly_wait(2)
body = to_bytes(self.driver.page_source)
return HtmlResponse(self.driver.current_url, body=body, encoding='utf-8', request=request)
def spider_opened(self, spider):
"""Change your browser mode here"""
self.driver = webdriver.Firefox()
def spider_closed(self, spider):
self.driver.close()
Modify your settings.py file to integrate the Selenium middleware (append following middleware to your project and replace yourproject with your project name)
DOWNLOADER_MIDDLEWARES = {
'yourproject.middlewares.SeleniumMiddleware': 200
}
Comparison
Scrapy-Splash
An official module by Scrapy Company
You can deploy splash instance to cloud, so that you will be able to browse the url in cloud then transfer the render.html back to your spider
It's slow
Splash container will stop if there is a memory leak. (Be sure to deploy splash instance on a high memory cloud instance)
Selenium web driver
You have to have Firefox or Chrome with their corresponding automated-test-driver on your machine, unless you use PhantomJS.
You can't modify request headers directly with Selenium web driver
You could render the webpage using a headless browser but this data can be easily extracted without it, try this:
import re
import ast
...
def parse_item(self,response):
regex = re.compile('dataLayer.push\((\{.*\})\);', re.DOTALL)
html_info = response.xpath('//script[contains(., "dataLayer.push")]').re_first(regex)
data = ast.literal_eval(html_info)
yield {'fiyat': data['CD_Fiyat'],
'marka': data['CD_marka'],
'yil': data['CD_yil']}
# output an item with {'fiyat': '103500', 'marka': 'Renault', 'yil': '2017'}
I executed meteor npm install --save request in the command line
I imported the request library in my code import {request} from 'request'
And tried to use it with
request('http://www.google.com', function (error, response, body) {
if (!error && response.statusCode == 200) {
console.log(body) // Show the HTML for the Google homepage.
}
})
However I keep on getting the following error:
undefined is not a function
How do I use the npm request package with my meteor app?
The default export from the request package is the object you are looking for. Change your import statement to the following:
import request from 'request';
It may be that you need the low-level functionality from request, however your example could also be accomplished with meteor's HTTP pacakge (which is itself a wrapper around request).
Here's an example:
import { Meteor } from 'meteor/meteor';
import { HTTP } from 'meteor/http';
Meteor.startup(() => {
const resp = HTTP.get('http://www.google.com');
console.log(resp.content);
});
Note you'll need to run meteor add http for that to work.