« Changelog History


Pholcus v1.3.0 Release Notes

Release Date: 2018-03-14 // about 6 years ago

  • Spider

    1. Add spider common functions
    2. Fix too many open files bug
    3. Fix spider timer

    Downloader

    ⚡️ 1. Update surfer package ⚡️ 2. Update PhantomJs Downloader Engine

    1. Add DNS cache, disable keepalive

    Output

    1. Simplify data output module.
    2. Mysql: limit 100 per insert
    3. Kafka: compatible with Kafka v0.11.0.0
    4. MongoDB: if mongoDB died or restart, app will get stuck in endless loop, imporve related logic

    Proxy

    👍 1. Support for filling in user name and password

    🌐 Web operation interface

    ⚡️ 1. Optimize websocket log

    1. Single mode: 🌲 3. Fix log blocking bug

    Cmd operation interface

    1. Supports '-c_spider=*'' that meanning all spiders

    Other

    1. Add panic trace info
    2. Fix some bugs ⚡️ 3. Optimize codes