Reducing the number of results for wordlists with URI paths with nested directories

[milo2012]

When using long word lists like "exploitdb_all.txt", some websites might return below results (which can include false positives).
We need to find out the actual valid paths and remove the other junk.

Below is a simplified example
Original results
[+] http://127.0.0.1/test/admin/index.php [200] [1027] []
[+] http://127.0.0.1/test/c99php3txt.php3 [200] [1027] []
[+] http://127.0.0.1/test/classifiedscript/admin.php [200] [1027] []
[+] http://127.0.0.1/test/cutenews-utf8/data/category.db.php [200] [1027] []
[+] http://127.0.0.1/test/cutenews/index.php [200] [1027] []
[+] http://127.0.0.1/test/cutenews/register.php [200] [1027] []
[+] http://127.0.0.1/test/cutenews/search.php [200] [1027] []
[+] http://127.0.0.1/test/efront/www/professor.php [200] [1027] []

Updated results with the -i [intelligent mode]
[+] http://127.0.0.1/test [200] [1027] []

The way I have done it is to slice the URI path /test/admin/index.php into parts and compare the status code and page size with one another.

If (1) has the same status code and page size as (2), and also the same as (3), we can effectively determine that (1)=(2)=(3). Therefore, we can remove (1) and (2) from the results and replace it with (3).

1. [+] http://127.0.0.1/test/admin/index.php [200] [1027] []
2. [+] http://127.0.0.1/test/admin [200] [1027] []
3. [+] http://127.0.0.1/test [200] [1027] []

[milo2012]

Fixed in https://github.com/milo2012/pathbrute/releases/tag/v0.0.4.