Python爬虫之Scrapy框架安装配置
Linux Ubuntu 平台:
这里只说Linux下的安装,非常简单,只需要执行几条命令几个
1.安装Python
1 2
| 1 sudo apt-get install python2.7 python2.7-dev
|
2.安装 pip
首先下载 get-pip.py
下载好之后,选中该文件所在路径,执行下面的命令
3.直接安装 Scrapy
由于 Linux下已经预装了 lxml 和 OPENSSL
如果想验证 lxml ,可以分别输入
出现下面的提示这证明已经安装成功
1
| Requirement already satisfied (use --upgrade to upgrade): lxml in /usr/lib/python2.7/dist-packages
|
如果想验证 openssl,则直接输入openssl 即可,如果跳转到 OPENSSL 命令行,则安装成功。
接下来直接安装 Scrapy 即可
安装完毕之后,输入 scrapy(下不要输入Scrapy)
如果出现如下提示,这证明安装成功
1 2 3 4 5 6 7 8 9 10 11 12 13 14
| Usage: scrapy <command> [options] [args] Available commands: bench Run quick benchmark test fetch Fetch a URL using the Scrapy downloader runspider Run a self-contained spider (without creating a project) settings Get settings values shell Interactive scraping console startproject Create new project version Print Scrapy version view Open URL in browser, as seen by Scrapy [ more ] More commands available when run from project directory
|