Commit Graph

500 Commits

Author SHA1 Message Date
Soxoj b2283a5b04 Merge pull request #1961 from overcuriousity/main
fix bad linux filename generation
2024-12-12 22:07:21 +01:00
Soxoj f212bc9bc8 Site check fixes 2024-12-12 21:39:35 +01:00
overcuriousity b8c62f95ae fix bad linux filename generation
currently maigret parses urls as usernames related to gravatar. this leads to bad filenames of the output on my linux host, as the slashes cause it to try to write subfolders, causing the script to abort with the error "file does not exist".
Applied a simple fix to replace all "/" with "_" in output file generation.
2024-12-12 15:00:54 +01:00
Soxoj 2653c617f8 Merge pull request #1958 from soxoj/gravatar-pypi-fix
Fixed Gravatar parsing (socid_extractor)
2024-12-12 02:32:35 +01:00
Soxoj 4dd82bf4c9 Fixed Gravatar parsing (socid_extractor) 2024-12-12 02:30:29 +01:00
Ikko Eltociear Ashimine f8ab484cd2 chore: update submit.py
futher -> further
2024-12-11 23:23:45 +09:00
Soxoj 64ae391a4a Updated Vimeo, CNET, DailyMotion 2024-12-11 01:17:20 +01:00
Soxoj 127d9032c3 Fixed Vimeo, activation/probing mechanisms improvements 2024-12-11 00:56:00 +01:00
Soxoj 81a817a39f Improved "submit new site" mode, added tests, fixed top-500 sites (#1952) 2024-12-10 18:02:43 +01:00
Soxoj 51ab988e36 Fixed ProductHunt check (#1951) 2024-12-09 17:06:03 +01:00
Soxoj 5517636850 Updated OP.GG checks (#1950)
* Updated OP.GG checks
* Finalized LoL, added Valorant, disabled Archive.org
2024-12-09 15:59:19 +01:00
Soxoj 4eada16b94 Added a test for submitter (#1944) 2024-12-08 13:35:27 +01:00
Soxoj c66d776f8a Refactoring, test coverage increased to 60% (#1943) 2024-12-08 02:13:28 +01:00
Soxoj 4b1317789d Refactored self-check method, code formatting, small lint fixes (#1942) 2024-12-07 18:05:30 +01:00
Soxoj 8b7d8073d9 Fixed Linktr and discourse.mozilla.org (#1941) 2024-12-07 17:11:39 +01:00
Soxoj 2aa1ea39a0 Site fixes (#1940) 2024-12-06 14:27:38 +01:00
Soxoj cd789ed138 Fixed Ebay and BongaCams checks (#1939) 2024-12-06 13:32:51 +01:00
Soxoj 5641456ba0 Weibo site check fix, activation mechanism added (#1938) 2024-12-06 11:31:20 +01:00
Soxoj f04de78682 Activation mechanism documentation added (#1935)
Few site checks fixed
2024-12-06 01:35:19 +01:00
Soxoj cb9f01c106 Fixed Figma check (#1932)
Fixed cookies bug
Improved self-check mode: don't disable sites because of check errors
2024-12-04 19:21:27 +01:00
Soxoj 1cb25946dd Disabled Figma check (#1928) 2024-12-04 00:27:55 +01:00
Soxoj 2f93963a0a Refactored sites module, updated documentation (#1918) 2024-12-01 11:41:41 +01:00
Soxoj d15e12750b Sites fixes (#1917)
* Some sites fixes

* Sites stats updated
2024-12-01 03:19:36 +01:00
Soxoj e96d09dee7 Permutator output and documentation updates (#1914) 2024-11-29 13:15:03 +01:00
Soxoj 15702bd9f4 Fixed dateutil parsing error for CDT timezone (#1907) 2024-11-29 12:02:41 +01:00
Soxoj 2e2a47a12b Close http connections (#1595) (#1905) 2024-11-27 15:28:10 +01:00
Soxoj 8a98aa9eaa Retries set to 0 by default, refactored code of executor with progress (#1899)
* Retries set to 0 by default, refactored code of executor with progress
2024-11-26 19:07:15 +01:00
Soxoj ee25c61fc2 Maigret bot support (custom progress function fixed) (#1898)
* Fixed progress/close functions
* Fixed tests: execution time increased with alive_progressbar
2024-11-26 15:54:26 +01:00
Soxoj 324c118530 Parallel execution optimization (#1897)
* Connection failure fix: removed futures, added semaphores

* Additional fixes

* Tqdm replace to alive_progress, poetry update

* Self-check mode fix, tests fixes

* Sites checks fixes (#1896)

* Fixed incorrect site names, added method to compare sites
2024-11-26 13:55:12 +01:00
Soxoj b370bc4c44 Sites checks fixes (#1896)
Fixed incorrect site names, added method to compare sites
2024-11-26 13:29:43 +01:00
Soxoj 13c20afe5b Improved self-check mode (#1887) 2024-11-25 18:27:59 +01:00
Soxoj d8a05807ba New sites added (#1888) 2024-11-25 18:24:20 +01:00
Soxoj 86d51bced0 Added 7 sites, implemented integration with Marple, docs update (#1881)
* Added 5 sites, implemented integration with Marple

* Added 2 more sites, updated docs

* Updated sites list
2024-11-25 14:41:34 +01:00
Soxoj 54b864f167 Disabled unavailable sites (#1880) 2024-11-24 17:19:31 +01:00
Soxoj 54fecccbfb Show detailed error statistics for -v (#1879) 2024-11-24 04:21:24 +01:00
Soxoj 24e545b62c Added dev documentation, fixed some sites, removed GitHub issue links from reports (#1869) 2024-11-23 18:45:56 +01:00
synth 05db32f28f Fixed 1 site, PyInstaller workflow, Google Colab example (#1558)
* Updated example colab file (Due to latest update)

* Fix RobertsSpaceIndustries URI

* Fix PyInstaller workflow

* Fix example.ipynb (read desc.)

Currently the version installed via pip3 doesn't appear to contain the latest data.json file, resulting in many false positives..

* Fix non-existant users (read desc.)

Fixed non-existant usernames for the following:
Telegram (t.me)
TikBuddy (tikbuddy.com)
FurAffinity (furaffinity.net)
2024-10-21 15:58:16 +02:00
Paul Pfeister 1cb589eadb chore: remove alik.cz (#1671)
Alik.cz is seeing unusually high traffic on usernames julian and
noonewouldeverusethis due to its presence in both Sherlock and Maigret.
This target is permanently removed and should not be replaced.
2024-10-21 15:57:11 +02:00
jm.balestek 6fb0dc1067 Adding permutator feature for usernames (#1575)
* Adding permutator feature for usernames
("", "_", "-", ".") when id_type == username

File : maigret/permutator.py

Arg : --permute

For now, only permute from 2 elements and doesn't return single elements (element1, _element1, element1_,  element2, _element2, ...). 12 permuts for 2 elements.

To return single elements as well, Permute(usernames).gather(method="all"), but not implemented in maigrat.py. 18 permuts for 2 elements. Should we ? With another argument ?

* Update test_cli.py

permute arg added
2024-07-23 16:19:43 +02:00
ranlo e02a5571b6 Update data.json (#1559)
changed the URL for vidamora.com to www.vidamora.com

any username on https://vidamora.com/profile/{username} returns a redirect, to www.vidamora.com

on https://www.vidamora.com, you get different behavior for existing and non-existing users.
2024-06-24 12:51:54 +02:00
Richard Mwewa 034153791b Fixed 3 sites, disabed 3, added (#1539)
* Fixed/Disabled sites. Update requirements.txt

fixed_sites: AllRecipes, Linktree, CreativeMarket, ImgInn, Shutterstock, Contently

disabled_sites: Forums.ea.com. CrunchyRoll, Windy, MetaCritic, InfosecInstitute, Armchairgm.fandom.com, Bleach.fandom.com

Update requirements to prevent dependency conflicts.

* Update requirements.txt

Update requirements.txt to prevent dependency conflicts

* Update requirements.txt

* Update sites.md

* fixed_sites: Armchairgm.fandom.com, Bleach.fandom.com, Battleraprus. disabled_sites: MicrosoftTechNet, club.cnews.ru, Scorcher

* fixed_sites: Armchairgm.fandom.com, Bleach.fandom.com, Battleraprus. disabled_sites: MicrosoftTechNet, club.cnews.ru, Scorcher

* fixed 2 sites, disabled 22 sites, and added 1 site

* fixed 3 sites, disabled 28, added 4 sites

* update sites.md

* Added 2 more sites

* fixed 3 sites, disabled 3 sites, added 1 site

* fix Twitch. Update snapcraft.yaml. Add pyproject.toml. Remove setup.py, requirements.txt, test-requirements.txt, as they are already specified in pyproject.toml

* fix Twitch. Update snapcraft.yaml. Add pyproject.toml. Remove setup.py, requirements.txt, test-requirements.txt, as they are already specified in pyproject.toml

* fix Twitch. Update snapcraft.yaml. Add pyproject.toml. Remove setup.py, requirements.txt, test-requirements.txt, as they are already specified in pyproject.toml

* fix Twitch. Update snapcraft.yaml. Add pyproject.toml. Remove setup.py, requirements.txt, test-requirements.txt, as they are already specified in pyproject.toml

* Update sites.md

* fix Twitch. Update snapcraft.yaml. Add pyproject.toml. Remove setup.py, requirements.txt, test-requirements.txt, as they are already specified in pyproject.toml

* Update sites.md

* fix forums.drom.ru

* Add EduGeek

* Add EduGeek

* Update python-package.yml

Fix dependency installation

* Update python-package.yml

* Update python-package.yml
2024-05-24 14:51:27 +02:00
Richard Mwewa 9399737ee6 Fixed 4 sites, added 6 sites, disabled 27 sites (#1536)
* Fixed/Disabled sites. Update requirements.txt

fixed_sites: AllRecipes, Linktree, CreativeMarket, ImgInn, Shutterstock, Contently

disabled_sites: Forums.ea.com. CrunchyRoll, Windy, MetaCritic, InfosecInstitute, Armchairgm.fandom.com, Bleach.fandom.com

Update requirements to prevent dependency conflicts.

* Update requirements.txt

Update requirements.txt to prevent dependency conflicts

* Update requirements.txt

* Update sites.md

* fixed_sites: Armchairgm.fandom.com, Bleach.fandom.com, Battleraprus. disabled_sites: MicrosoftTechNet, club.cnews.ru, Scorcher

* fixed_sites: Armchairgm.fandom.com, Bleach.fandom.com, Battleraprus. disabled_sites: MicrosoftTechNet, club.cnews.ru, Scorcher

* fixed 2 sites, disabled 22 sites, and added 1 site

* fixed 3 sites, disabled 28, added 4 sites

* update sites.md

* Added 2 more sites
2024-05-18 01:50:05 +02:00
Richard Mwewa f7f77e587c Fixed/Disabled sites. Update requirements.txt (#1517)
* Fixed/Disabled sites. Update requirements.txt

fixed_sites: AllRecipes, Linktree, CreativeMarket, ImgInn, Shutterstock, Contently

disabled_sites: Forums.ea.com. CrunchyRoll, Windy, MetaCritic, InfosecInstitute, Armchairgm.fandom.com, Bleach.fandom.com

Update requirements to prevent dependency conflicts.

* Update requirements.txt

Update requirements.txt to prevent dependency conflicts

* Update requirements.txt

* Update sites.md

* fixed_sites: Armchairgm.fandom.com, Bleach.fandom.com, Battleraprus. disabled_sites: MicrosoftTechNet, club.cnews.ru, Scorcher

* fixed_sites: Armchairgm.fandom.com, Bleach.fandom.com, Battleraprus. disabled_sites: MicrosoftTechNet, club.cnews.ru, Scorcher
2024-05-14 15:11:17 +02:00
h3x 46b13b4f23 fix reddit (#1296) 2023-11-26 18:38:06 +01:00
weekend sorrow f1ea12d731 Updating site checkers, disabling suspended sites (#1266)
* Fixing checks for broken sites and repairing the ones that were changed

* little tweaks

* little tweaks

---------

Co-authored-by: Weekrow <somewherelse@yandex.ru>
2023-10-27 21:43:45 +02:00
Soxoj 83d5740096 Tests fixes + last updates (#1228)
* Some sites fixed & cloudflare detection

* Fixed issue with tests

* Updates GitHub test workflow and sites data
2023-10-15 21:07:21 +02:00
Soxoj 90599ea3c2 Some sites fixed & cloudflare detection (#1178) 2023-09-09 20:58:01 +02:00
Soxoj c5c78b2a66 Sites fixes 250823 (#1149)
* Additionally fixed sites, win32 build fix

* Fixed and disabled some sites (again)
2023-08-24 10:51:24 +02:00
Soxoj 390f3a49ee Additionally fixed sites, win32 build fix (#1148) 2023-08-24 09:32:24 +02:00
Soxoj b8c035e564 Fixed some sites (again) (#1133) 2023-08-23 21:58:41 +02:00