Are created by create-format-examples script and contained in examples directory where they're grouped based on scraper and version. Files are in json

Search code, repositories, users, issues, pull requests...

submited by
Style Pass
2025-01-07 16:30:07

Are created by create-format-examples script and contained in examples directory where they're grouped based on scraper and version. Files are in json format.

Download any kind of supported forums from URLs into DIR, creating json files for threads named by their id's, same for users but beginning with m- e.g. 24 29 m-89 m-125.

Above behaviour is set by default --names id, and can be changed with --names hash which names files by sha256 sum of their source urls.

By default if files to be created are found and are not empty, function exits not overwriting them. This can be changed using --force option.

forumscraper output logging information to stdout (can be changed with --log FILE) and information about failures to stderr (can be changed with --failed FILE)

Download URLs into DIR using 8 threads and log failures into failures.txt (note that this option is specified before the --directory option, otherwise it would create it in the specified directory if relative path is used)

Leave a Comment