Permutator output and documentation updates (#1914)

docs/source/command-line-options.rst

@@ -18,7 +18,7 @@ Parsing of account pages and online documents
 
 Maigret will try to extract information about the document/account owner
 (including username and other ids) and will make a search by the
-extracted username and ids. :doc:`Examples <extracting-information-from-pages>`.
+extracted username and ids. See examples :doc:`in the separate section <extracting-information-from-pages>`.
 
 Main options
 ------------
@@ -28,8 +28,8 @@ Options are also configurable through settings files, see
 
 ``--tags`` - Filter sites for searching by tags: sites categories and
 two-letter country codes (**not a language!**). E.g. photo, dating, sport; jp, us, global.
-Multiple tags can be associated with one site. **Warning: tags markup is
+Multiple tags can be associated with one site. **Warning**: tags markup is
-not stable now.**
+not stable now. Read more :doc:`in the separate section <tags>`.
 
 ``-n``, ``--max-connections`` - Allowed number of concurrent connections
 **(default: 100)**.

docs/source/features.rst

@@ -14,13 +14,95 @@ Also, Maigret use found ids and usernames from links to start a recursive search
 
 Enabled by default, can be disabled with ``--no extracting``.
 
+.. code-block:: text
+
+    $ python3 -m maigret soxoj --timeout 5
+        [-] Starting a search on top 500 sites from the Maigret database...
+        [!] You can run search by full list of sites with flag `-a`
+        [*] Checking username soxoj on:
+        ...
+        [+] GitHub: https://github.com/soxoj
+                ├─uid: 31013580
+                ├─image: https://avatars.githubusercontent.com/u/31013580?v=4
+                ├─created_at: 2017-08-14T17:03:07Z
+                ├─location: Amsterdam, Netherlands
+                ├─follower_count: 1304
+                ├─following_count: 54
+                ├─fullname: Soxoj
+                ├─public_gists_count: 3
+                ├─public_repos_count: 88
+                ├─twitter_username: sox0j
+                ├─bio: Head of OSINT Center of Excellence in @SocialLinks-IO
+                ├─is_company: Social Links
+                └─blog_url: soxoj.com
+        ...
+
 Recursive search
 ----------------
 
-Maigret can extract some :ref:`common ids <supported-identifier-types>` and usernames from links on the account page (often people placed links to their other accounts) and immediately start new searches. All the gathered information will be displayed in CLI output and reports.
+Maigret has the ability to scan account pages for :ref:`common identifiers <supported-identifier-types>` and usernames found in links.
+When people include links to their other social media accounts, Maigret can automatically detect and initiate new searches for those profiles.
+Any information discovered through this process will be shown in both the command-line interface output and generated reports.
 
 Enabled by default, can be disabled with ``--no-recursion``.
 
+
+.. code-block:: text
+
+    $ python3 -m maigret soxoj --timeout 5
+        [-] Starting a search on top 500 sites from the Maigret database...
+        [!] You can run search by full list of sites with flag `-a`
+        [*] Checking username soxoj on:
+        ...
+        [+] GitHub: https://github.com/soxoj
+                ├─uid: 31013580
+                ├─image: https://avatars.githubusercontent.com/u/31013580?v=4
+                ├─created_at: 2017-08-14T17:03:07Z
+                ├─location: Amsterdam, Netherlands
+                ├─follower_count: 1304
+                ├─following_count: 54
+                ├─fullname: Soxoj
+                ├─public_gists_count: 3
+                ├─public_repos_count: 88
+                ├─twitter_username: sox0j     <===== another username found here
+                ├─bio: Head of OSINT Center of Excellence in @SocialLinks-IO
+                ├─is_company: Social Links
+                └─blog_url: soxoj.com
+        ...
+        Searching |████████████████████████████████████████| 500/500 [100%] in 9.1s (54.85/s)
+        [-] You can see detailed site check errors with a flag `--print-errors`
+        [*] Checking username sox0j on:
+        [+] Telegram: https://t.me/sox0j
+            ├─fullname: @Sox0j
+            ...
+
+Username permutations
+--------------------
+
+Maigret can generate permutations of usernames. Just pass a few usernames in the CLI and use ``--permute`` flag.
+Thanks to `@balestek <https://github.com/balestek>`_ for the idea and implementation.
+
+.. code-block:: text
+
+    $ python3 -m maigret --permute hope dream --timeout 5
+    [-] 12 permutations from hope dream to check...
+        ├─ hopedream
+        ├─ _hopedream 
+        ├─ hopedream_
+        ├─ hope_dream
+        ├─ hope-dream
+        ├─ hope.dream
+        ├─ dreamhope
+        ├─ _dreamhope
+        ├─ dreamhope_
+        ├─ dream_hope
+        ├─ dream-hope
+        └─ dream.hope
+    [-] Starting a search on top 500 sites from the Maigret database...
+    [!] You can run search by full list of sites with flag `-a`
+    [*] Checking username hopedream on:
+    ...
+
 Reports 
 -------
 
@@ -34,7 +116,8 @@ HTML/PDF reports contain:
 
 Also, there is a short text report in the CLI output after the end of a searching phase.
 
-**Warning**: XMind 8 mindmaps are incompatible with XMind 2022!
+.. warning::
+   XMind 8 mindmaps are incompatible with XMind 2022!
 
 Tags
 ----

docs/source/index.rst

@@ -21,12 +21,12 @@ You may be interested in:
    :caption: Sections
 
    command-line-options
-   extracting-information-from-pages
+   usage-examples
    features
    philosophy
-   roadmap
+   extracting-information-from-pages
    supported-identifier-types
    tags
-   usage-examples
    settings
    development
+   roadmap

docs/source/roadmap.rst

@@ -3,7 +3,8 @@
 Roadmap
 =======
 
-**This roadmap is outdated and needs to be updated.**
+.. warning::
+   This roadmap is outdated and needs to be updated.
 
 .. figure:: https://i.imgur.com/kk8cFdR.png   
    :target: https://i.imgur.com/kk8cFdR.png

docs/source/tags.rst

@@ -5,7 +5,8 @@ Tags
 
 The use of tags allows you to select a subset of the sites from big Maigret DB for search.
 
-**Warning: tags markup is not stable now.**
+.. warning::
+   Tags markup is still not stable.
 
 There are several types of tags:
 

maigret/maigret.py

@@ -615,7 +615,8 @@ async def main():
 
     if len(usernames) > 1 and args.permute  and args.id_type == 'username':
         query_notify.warning(
-            f"{len(usernames)} permutations from {original_usernames} to check..."
+            f"{len(usernames)} permutations from {original_usernames} to check..." +
+            get_dict_ascii_tree(usernames, prepend="\t")
         )
 
     if not site_data:

sites.md

@@ -3130,7 +3130,7 @@ Rank data fetched from Alexa by domains.
 1. ![](https://www.google.com/s2/favicons?domain=https://massagerepublic.com) [massagerepublic.com (https://massagerepublic.com)](https://massagerepublic.com)*: top 100M*
 1. ![](https://www.google.com/s2/favicons?domain=https://mynickname.com) [mynickname.com (https://mynickname.com)](https://mynickname.com)*: top 100M*
 
-The list was updated at (2024-11-26 UTC)
+The list was updated at (2024-11-27 UTC)
 ## Statistics
 
 Enabled/total sites: 2694/3126 = 86.18%