shen/pyspider
1
0
Fork 0
mirror of https://github.com/binux/pyspider.git synced 2024-11-25 16:34:30 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,181 Commits 5 Branches 13 Tags 30 MiB
master
Commit Graph

1181 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Keith Tunstead
0410c64d19 fixed "connect to scheduler rpc error: error(111, Connection refused)" error 2019-10-24 15:20:33 +02:00
Keith Tunstead
b3dd943bd0 updated travis.yml 2019-10-24 13:25:11 +02:00
Keith Tunstead
4a41f04f44 upgrade python-six 2019-10-24 13:16:35 +02:00
Keith Tunstead
dcbf6dff62 port to python 3.6 2019-10-24 13:10:55 +02:00
Keith Tunstead
8110fd647f updated requirements.txt to fixed package versions 2019-10-24 12:21:14 +02:00
Roy Binux
3fccfabe2b
Merge pull request #868 from vinsec/patch-1 
fix bug: support python 3.7 and solve travis-ci problem
 2019-03-09 15:50:28 -08:00
Roy Binux
4d49e4df29
Merge branch 'master' into patch-1 2019-03-09 15:50:13 -08:00
Roy Binux
8178a97fe4
Merge pull request #869 from binux/farmercode-master 
Farmercode master
 2019-02-23 17:44:19 -08:00
Roy Binux
7ed18c8cdc
Merge pull request #866 from ccl0326/puppeteer 
add puppeteer fetcher
 2019-02-23 17:31:25 -08:00
Roy Binux
c7396bbe61
Merge branch 'master' into puppeteer 2019-02-23 17:30:49 -08:00
Roy Binux
440e7ebd36
Merge pull request #851 from qshine/puppeteer 
feature puppeteer js engine
 2019-02-23 17:25:37 -08:00
Roy Binux
03c9999f0a
Merge branch 'master' into puppeteer 2019-02-23 17:25:30 -08:00
binux
1603785db7 allow fail 2019-02-23 17:19:47 -08:00
binux
b2081ff4cb try again 2019-02-23 17:15:54 -08:00
binux
3fb99bd24a try install librt 2019-02-23 16:59:55 -08:00
binux
398211ddc9 fix for 3.7 2019-02-23 16:58:13 -08:00
binux
44a4dda64b try again 2019-02-23 16:46:26 -08:00
binux
40669065d5 try manually install 2019-02-23 16:32:37 -08:00
binux
0d65272b8e fix setup.py test for py3.3 2019-02-23 16:23:49 -08:00
binux
74874e216a try not to specify a version for apt-get 2019-02-23 16:19:32 -08:00
binux
578664f27c sudo not required any more? 2019-02-23 16:10:23 -08:00
binux
53f9de5cb1 1. python2.7 image is different when using metrix 
2. pip install just works now days
 2019-02-23 16:06:21 -08:00
binux
4a5d243840 fix build for 3.3 2019-02-23 15:50:38 -08:00
binux
e29441724e use suggested python3.7 build 2019-02-23 15:47:16 -08:00
binux
76e33acb66 Merge branch 'master' of https://github.com/farmercode/pyspider into farmercode-master 2019-02-23 15:27:06 -08:00
v1nc3nt
e0b07efd75 fix bugs 
1. some args "async" haven't been replaced completely yet
2. delete Python 3.3 in .travis.yml because the current version of lxml is not supported by Python3.3
 2019-02-18 13:31:31 +08:00
clchen
e8e5b9bcd9 update 2019-02-14 17:29:06 +08:00
clchen
96b5128eb2 add puppeteer fetcher 2019-02-14 17:10:54 +08:00
feiyang
563b5194fe fix: python3.5 install lxml error 2019-01-30 21:33:38 +08:00
feiyang
99c9fb5170 features: add opened pages maximum limit, default 5 2019-01-08 18:03:30 +08:00
feiyang
0bc3c7f238 feature puppeteer js engine 2019-01-05 17:25:29 +08:00
Roy Binux
936aa7ce14
Merge pull request #843 from PhillipPeterson/travis-ci-changes 
Proposed changes to travis ci file to allow passing builds
 2018-11-29 14:07:08 -08:00
Phillip
af629ddad6 removed 2.6 due to lack of support, changed pip install for 3.5 due to pip versioning 2018-11-29 14:28:07 -07:00
farmercode
329cadbfd9 add support py3.7-dev CI test 2018-11-19 17:05:33 +08:00
farmercode
ba30efe3ba add py3.7-dev CI test 2018-11-19 16:15:53 +08:00
farmercode
cbc3e46225 remove python3.7 CI test 2018-11-19 16:02:03 +08:00
farmercode
e379d30c60 add python3.7 CI test 2018-11-19 15:59:41 +08:00
farmercode
6502e1fb9f add python3.7 CI test 2018-11-19 15:50:18 +08:00
farmercode
b51e7455eb add python3.7 CI test 2018-11-19 15:47:42 +08:00
farmercode
501380696c add python3.7 CI test 2018-11-19 15:22:12 +08:00
farmercode
984b8ca215 modify async to async_mode to support python3.7 2018-10-15 17:28:22 +08:00
vibiu
3fb9167983 change async to async_ (#803) 
* change async to async_

* change async to async_ in tests

* change async_ to async_mode
 2018-06-14 12:07:51 -07:00
Lucas
7037a77c6f optimise scheluler dynamic select limit and improve task queue (#796) 
* optimise scheduler select-limit and task queue

* fix test case in python2.6

* fix: time priority queue only compare exetime

* update:add test case for time priority queue

* optimise: add globally auto increasing value for task to keep priority queue in order
 2018-06-09 18:07:17 -07:00
Roy Binux
66347298b1
Merge pull request #800 from binux/binux-patch-1 
fix #799
 2018-06-01 09:46:12 -07:00
Roy Binux
8646179601
fix #799 2018-05-31 23:56:37 -07:00
sdvcrx
124ffef771 Fix mysql return bytes as field name type (#787) 
* use pip version of mysql-connector-python for testing

* fix mysql return bytes as field names type

* fix raise Unread result found error

This error raise on mysql-connector-python with C extension

* fix test

Pure version raise InterfaceError,
  but C extension version raise DatabaseError
 2018-05-11 17:20:57 -07:00
binux
c350f6216b using reserved ip address for testing 
rolling out version 0.3.10
 2018-04-17 20:47:33 -07:00
jxltom
1e457313e0 Fixed db inconsistency (#779) 
* Fixed db creation inconsistency in taskdb, projectdb and resultdb

* Fixed typo
 2018-04-08 14:48:28 -07:00
Roy Binux
a203a77e8f
Merge pull request #778 from jxltom/fixed-typo 
Fixed typo
 2018-04-08 14:47:26 -07:00
Roy Binux
d425b0cc15
fix test 
1.1.1.1 took by cloudflare
 2018-04-08 14:47:11 -07:00