Foreword
This update includes several parts:
- More advanced rights management
- Fun Challenge
- Configurable custom conversion crawler crawler
- The results deduplication
- Task Retry
Update Log
Function / Optimization
- Challenge . The user can perform different fun challenge ..
- More advanced access control . More granular rights management, for example, ordinary users can only view or manage your own project or reptiles, and administrative users can view or manage all reptiles or project.
- Feedback allows users to send feedback and ratings to Crawlab development team.
- Better Home index optimization index display on the home page.
- May be converted to custom configure the crawler crawler A user may own configurable into Scrapy custom crawler crawler.
- Tasks View timed tasks triggered Allows users to view tasks timed tasks triggered. # 648
- The results support de-duplication , allowing users to re-configuration. # 579
- Support task retry Allow task to re-trigger the historic task.
Bug fixes
- CLI is not available on Windows . # 580
- Re-upload error . # 643 # 640
- Upload the missing file directory . # 646
- You can not add regular tasks in reptiles scheduled task tab .
Product Planning
- The results show
- Support for other databases
- Yes placement Reptilia
- Splash crawler can be configured to support
- Configurable crawler support CrawlSpider
- Crawler can be configured to support regular expressions field
- task
- Task retry mechanism
- Regular tasks
- Show calendar
- Overall situation
- Supported version of the update log shows
- server
- Mirror support terminal operation Docker
- SDK
- More command support
- Plug-in system
reference
community
If you feel Crawlab helpful for your daily development or company, please add author micro-channel tikazyq1 and marked "Crawlab", authors will pull you into the group. Welcomes the star, and, if you have any questions, please feel free to put on Github issue on Github. In addition, you are welcome to Crawlab make contribution to the development.