Tuesday 15 September 2015

python - How to use scrapy request class -


Some may help me in requesting access to the scripts request class

I Have tried but it does not work: import scrapy.selector from HtmlXPathSelector from scrapy.http.request import request url = 'http://www.fetise.com' a = request (Url) hxs = HtmlXPathSelector (a)

error is:

  traceback (most recent call final): file "sa.py", line 83, & lt; Module & gt; In the Hxs = HtmlXPathSelector (A) file "/usr/local/lib/python2.7/dist-packages/scrapy/selector/lxmlsel.py"line-31, __init__ _root = LxmlDocument (response, self._parser) file" / Usr / local / lib / python2.7 / dist-packages / scrapy / selector / lxmldocument.py ", / usr / local / lib / python2 in line 27, __new__ cache [parser] = _factory (response, parser) file 7 / dist-packages / scrapy / selector / lxmldocument.py ", line 13, in _factor body = response.in_ass_oneOnod (). Strip () Sign ('utf8') or '  

I know about callback .. I want to scrap urls from the site for the first time, then url Please try it ....

Please try it:

 < Import from the code> Scraper urlib import. HtmlXPathSelector Pprint import pprint url = 'http://www.fetise.com' .read () hxs = HtmlXPathSelector (text = data) from data = urllib.urlopen (url) lista = hxs.select ('// ul [@ Class = "categoryMenu"] / li / ul / li / a / @ href '). Remove () ACB = ["http://www.fetise.com/" + "If http://www.fetise.com/" in the list for Else i] + [u "http: // Www.fetise.com/sale "] pprint (acb)   

and this output is:

  [u'http: //www.fetise Com / apparel / shirts', u 'http: //www.fetise.com/apparel/tees', u'http: //www.fetise.com / apparel / top-end-tees', u' http: / /www.fetise.com/accessories/belts', U 'http: //www.fetise.com/accessories/cufflinks', u'http: //www.fetise.com/accessories/jewellery', u 'http: //www.fetise.com/accessories/lighters ', you' http://www.fetise.com/accessories/others ', u'http: /www.fetise.com/accessories/sunglasses', u'http : //www.fetise.com/accessories/ties-cufflinks', you 'http: //www.fetise.com/accessories/wallets', you' http://www.fetise.com/accessories/watches', U'http: //www.fetise.com/footwear/boots', You'http: //www.fetise.com/footwear/casual ', you' http://www.fetise.com/footwear/flats', you 'http: //www.fetise.com/footwear/heels' , You 'http://www.fetise.com/footwear/loafers', http://www.fetise.com/footwear/shoes ', You' http: //www.fetise.com/footwear/slippers', u'http: //www.fetise.com/footwear/sports', you 'http://www.fetise.com/innerwear/ Boxers', U 'http: //www.fetise.com/innerwear/briefs',' http: //www.fetise.com/personal-care/deos', u'http: //www.fetise Com / personal-care / haircare ', U' http: //www.fetise.com/personal-care/perfumes', you 'http://www.fetise.com/personal-care/personal-care', you 'Http: //www.fetise.com/personal-care/shavers', U' http: // www. Fetise.com/apparel/tees/gifts-for-her ', u'http: //www.fetise.com/footwear/sandals/gifts-for-her', U 'http: //www.fetise.com/ Shoes / shoes / gifts for gifts', 'http://www.fetise.com/footwear/heels/gifts-for-her', u'http: //www.fetise.com/footwear/flats/ gifts For him, 'U' http://www.fetise.com/footwear/ballerinas/gifts-for-her ', U'epi: //www.fetise.com/footwear/loafers/gifts-for-- ', You' http://www.fetise.com/sale ']    

No comments:

Post a Comment