mirror of
https://github.com/binux/pyspider.git
synced 2024-11-25 16:34:30 +08:00
drop support for couchdb
This commit is contained in:
parent
e9cda9aa43
commit
360d1319ee
@ -13,12 +13,13 @@ services:
|
||||
- mysql
|
||||
# - elasticsearch
|
||||
- postgresql
|
||||
- couchdb
|
||||
addons:
|
||||
postgresql: "9.4"
|
||||
apt:
|
||||
packages:
|
||||
- rabbitmq-server
|
||||
env:
|
||||
- IGNORE_COUCHDB=1
|
||||
|
||||
before_install:
|
||||
- sudo apt-get update -qq
|
||||
|
@ -5,7 +5,7 @@ A Powerful Spider(Web Crawler) System in Python.
|
||||
|
||||
- Write script in Python
|
||||
- Powerful WebUI with script editor, task monitor, project manager and result viewer
|
||||
- [MySQL](https://www.mysql.com/), [CouchDB](https://couchdb.apache.org), [MongoDB](https://www.mongodb.org/), [Redis](http://redis.io/), [SQLite](https://www.sqlite.org/), [Elasticsearch](https://www.elastic.co/products/elasticsearch); [PostgreSQL](http://www.postgresql.org/) with [SQLAlchemy](http://www.sqlalchemy.org/) as database backend
|
||||
- [MySQL](https://www.mysql.com/), [MongoDB](https://www.mongodb.org/), [Redis](http://redis.io/), [SQLite](https://www.sqlite.org/), [Elasticsearch](https://www.elastic.co/products/elasticsearch); [PostgreSQL](http://www.postgresql.org/) with [SQLAlchemy](http://www.sqlalchemy.org/) as database backend
|
||||
- [RabbitMQ](http://www.rabbitmq.com/), [Redis](http://redis.io/) and [Kombu](http://kombu.readthedocs.org/) as message queue
|
||||
- Task priority, retry, periodical, recrawl by age, etc...
|
||||
- Distributed architecture, Crawl Javascript pages, Python 2.{6,7}, 3.{3,4,5,6} support, etc...
|
||||
|
@ -31,7 +31,7 @@ class ProjectDB(BaseProjectDB):
|
||||
},
|
||||
'name': self.__collection_name__ + "_" + database
|
||||
}
|
||||
res = self.session.post(self.url+"_index", json=payload).json()
|
||||
res = self.session.post(self.url + "_index", json=payload).json()
|
||||
self.index = res['id']
|
||||
|
||||
def _default_fields(self, each):
|
||||
|
Loading…
Reference in New Issue
Block a user