Greasy Fork

Lightweight crawling script, used to download the main content of the webpage, theoretically suitable for any non-Ajax writing novel website, forum, etc. without writing any rules for this

The script will automatically retrieve the main content on the page and download it.

If you are in the novel directory page, it will traverse all the chapters and sort them and save them as TXT files.

Script Github

Donation PayPal.Me

Operation Instructions-Usage

• Open the novel catalog page or forum content page
• Press CTRL+F9 or click the command menu
• About configuration items
  • The following functions need to be entered through the oil monkey command menu
  • Custom download with directory range: such as https://xxx.xxx/book-**[20-99]**.html, https://xxx.xxx/book-**[01- 10]**.html, which means download https://xxx.xxx/book-20.html to https://xxx.xxx/book-99.html, and https://xxx.xxx/book- 01.html to https://xxx.xxx/book-10.html, [1-10] does not add 0
  • Custom download via chapter selector: Just enter the css selector of the chapter link to be downloaded, and then you can connect the url replacement code, such as this site po18, the chapter selector is .l_chaptname>a, but the text cannot be downloaded through the url. The text is downloaded by ajax through articlescontent. At this time, you can use @@articles@@articlescontent to change the chapter. Replace articles in url with articlescontent, .l_chaptname>a@@articles@@articlescontent can be adapted to this site. The first article can use regularity, for example, @@articles\d+@@$1content means to replace "articles1", "articles2", etc. in the link with "1content" "2content"
  • Interference code: fill in the css selector of the interference code, such as .mask, which means to delete the element whose class is mask
  • Reorder by title name: if true, sort all links on the catalog page by title name and save them in txt, otherwise, they will be sorted by page position order

Test webpage-Test case

• http://www.gulongbbs.com/zhentan/bdlr/plje/Index.html
• http://www.jhshe.cn/thread-1837-1-1.html
• http://tieba.baidu.com/p/4871634479