Commit Graph

72 Commits

Author SHA1 Message Date
Soxoj a5d337b765 Tags and site names improvements (#2427)
- Added social tag to social networks (33 sites)
- Fixed wrong tags (8 sites)
- Filled empty tags for 213 sites in top-1000
- Country tag cleanup (~374 sites)
- Site naming normalization (75 sites)
- New tests (3)
- Documentation updates
2026-03-28 15:42:12 +01:00
Soxoj 656fe1df24 Added Max.ru check; --no-progressbar flag fixed (#2386) 2026-03-25 11:48:12 +01:00
Copilot abd9aa57fe Fix domain substring matching and NoneType crash in submit dialog (#2367)
* Initial plan

* Fix domain matching and NoneType error in submit.py

- Use regex with domain boundary matching instead of substring matching
  to prevent x.com from matching 500px.com, mix.com, etc.
- Handle None old_site gracefully when user enters a site name not in
  the matched list, fixing AttributeError crash.
- Add tests for both fixes.

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/7eabc755-47fd-4b80-a38c-9d6c056c2ce9

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
2026-03-24 22:04:10 +01:00
Copilot 2e430e5039 feat: add tag blacklisting via --exclude-tags (#2352)
* Initial plan

* feat: add tag blacklisting support (--exclude-tags CLI flag, web UI, docs, tests)

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/1a656af2-36bf-494f-9f03-1b5340f0357c

* fix: correct tag cloud label to match click-cycle interaction

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/1a656af2-36bf-494f-9f03-1b5340f0357c

* feat: add all country tags to web interface tag cloud

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/7e184b24-ff26-48fd-8a93-aea12b0a8d7b

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
2026-03-24 22:00:59 +01:00
Soxoj 97cc4b46d9 Improve site-check quality: fix broken site configs, add diagnostic utilities, and make self-check report-only by default with opt-in auto-disable. (#2301)
- Fix VK and TradingView checkType; add Reddit and Microsoft Learn API-style probes where appropriate; adjust or disable entries that are unreliable under anti-bot protection.
- Self-check: stop aggressive auto-disable; default to reporting issues only; add --auto-disable and --diagnose for optional fixes and deeper output.
- Tooling: add utils/site_check.py and utils/check_top_n.py (and related helpers) to inspect and rank site behavior against the top-N list
- Scope: aligns with fixing top-traffic / high-impact sites and making diagnostics repeatable without silently flipping disabled flags
2026-03-22 16:48:35 +01:00
Soxoj 227a25bfa1 Twitter fixed, mirrors mechanism improvement (#2299) 2026-03-22 01:14:17 +01:00
Tang Vu 0e95e2e3cc refactor: missing tests for settings cascade and override logic (#2287)
The `Settings.load()` method iterates through multiple configuration file paths and updates the internal `__dict__`, intending to override earlier default settings with later user-specific ones. This cascading logic is a core configuration feature but lacks explicit tests to guarantee that dictionary merging and overriding behave exactly as documented (e.g., ensuring a setting in `~/.maigret/settings.json` correctly overrides `resources/settings.json` without wiping out other keys).


Affected files: test_settings.py
2026-03-21 18:06:54 +01:00
Soxoj 5e88fd9ba8 Fixed test dialog_adds_site_negative (#2107) 2025-08-10 12:44:05 +02:00
Soxoj 97e5f600d0 Async generator-executor for site checks (#1978) 2024-12-17 22:48:11 +01:00
overcuriousity 88d68490f3 Created web frontend launched via --web flag (#1967)
Author: overcuriousity 
Co-authored-by: Soxoj <soxoj@protonmail.com>
2024-12-16 14:24:03 +01:00
Soxoj c4af0a4df0 Fixed flaky tests to check cookies (#1965) 2024-12-13 12:37:58 +01:00
Soxoj 81a817a39f Improved "submit new site" mode, added tests, fixed top-500 sites (#1952) 2024-12-10 18:02:43 +01:00
Soxoj 4eada16b94 Added a test for submitter (#1944) 2024-12-08 13:35:27 +01:00
Soxoj c66d776f8a Refactoring, test coverage increased to 60% (#1943) 2024-12-08 02:13:28 +01:00
Soxoj 4b1317789d Refactored self-check method, code formatting, small lint fixes (#1942) 2024-12-07 18:05:30 +01:00
Soxoj f04de78682 Activation mechanism documentation added (#1935)
Few site checks fixed
2024-12-06 01:35:19 +01:00
Soxoj 8a98aa9eaa Retries set to 0 by default, refactored code of executor with progress (#1899)
* Retries set to 0 by default, refactored code of executor with progress
2024-11-26 19:07:15 +01:00
Soxoj ee25c61fc2 Maigret bot support (custom progress function fixed) (#1898)
* Fixed progress/close functions
* Fixed tests: execution time increased with alive_progressbar
2024-11-26 15:54:26 +01:00
Soxoj 324c118530 Parallel execution optimization (#1897)
* Connection failure fix: removed futures, added semaphores

* Additional fixes

* Tqdm replace to alive_progress, poetry update

* Self-check mode fix, tests fixes

* Sites checks fixes (#1896)

* Fixed incorrect site names, added method to compare sites
2024-11-26 13:55:12 +01:00
Soxoj b370bc4c44 Sites checks fixes (#1896)
Fixed incorrect site names, added method to compare sites
2024-11-26 13:29:43 +01:00
Soxoj 13c20afe5b Improved self-check mode (#1887) 2024-11-25 18:27:59 +01:00
Soxoj 9b0212d7c7 Fixed test for aiohttp 3.10 (#1876) 2024-11-23 21:42:34 +01:00
jm.balestek 6fb0dc1067 Adding permutator feature for usernames (#1575)
* Adding permutator feature for usernames
("", "_", "-", ".") when id_type == username

File : maigret/permutator.py

Arg : --permute

For now, only permute from 2 elements and doesn't return single elements (element1, _element1, element1_,  element2, _element2, ...). 12 permuts for 2 elements.

To return single elements as well, Permute(usernames).gather(method="all"), but not implemented in maigrat.py. 18 permuts for 2 elements. Should we ? With another argument ?

* Update test_cli.py

permute arg added
2024-07-23 16:19:43 +02:00
dependabot[bot] 7c5995f165 Bump aiohttp from 3.8.3 to 3.8.6 (#1222)
* Bump aiohttp from 3.8.3 to 3.8.6

Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.8.3 to 3.8.6.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.8.3...v3.8.6)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fixed problematic test after aiohttp upgrade

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Soxoj <soxoj@protonmail.com>
2023-10-15 21:34:21 +02:00
Theodore Ni dc9b44bd14 Add compatibility with pytest >= 7.3.0 (#1117)
Starting in this version, marks are no longer ordered. Sorting by their
names still sorts slow marks to the end of the list of tests.
2023-08-24 09:32:03 +02:00
Soxoj a914283a15 Skipped broken tests 2022-03-18 01:51:14 +03:00
dependabot[bot] e8d63ef273 Bump pytest-asyncio from 0.16.0 to 0.17.0 (#314)
* Bump pytest-asyncio from 0.16.0 to 0.17.0

Bumps [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) from 0.16.0 to 0.17.0.
- [Release notes](https://github.com/pytest-dev/pytest-asyncio/releases)
- [Commits](https://github.com/pytest-dev/pytest-asyncio/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: pytest-asyncio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Python version specified for dependency (3.6 support was dropped)

* Pytest config fixes

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Soxoj <soxoj@protonmail.com>
2022-01-15 12:46:33 +03:00
Soxoj 5c05cfa5bc Fixed BongaCams, links parsing improved (#297)
* Fixed BongaCams, links parsing improved

* Fixed tests
2022-01-08 03:45:49 +03:00
Soxoj bda85b290e Fixed failed tests (thx to Meta aka Facebook) (#273) 2021-12-20 23:16:33 +03:00
soxoj 7decbce08d Box drawing logic improved, added new settings (#217) 2021-11-09 02:33:31 +03:00
Soxoj 7a362406d5 Documentation and settings improved 2021-11-06 08:24:12 +03:00
Soxoj f81a500d72 Added cascade settings loading from ~/.maigret/settings.json and ./settings.json 2021-11-05 13:46:17 +03:00
Soxoj 516861e0ae Some security fixes
Added link to Patreon
2021-10-31 22:41:59 +03:00
Soxoj c7977f1cdf Draft of graph report 2021-06-21 22:20:51 +03:00
Soxoj 9b0acc092a Refactoring of submit module, some fixes 2021-06-13 00:43:28 +03:00
Soxoj 02cf330e37 Added I2P sites support 2021-06-02 23:45:11 +03:00
Soxoj 13e1b6f4d1 Added some domains for new DNS checker, fixed reports generation crashes 2021-06-02 23:16:44 +03:00
Soxoj 4aeacef07d Sort by number of data points (#105) 2021-05-22 20:23:53 +03:00
Soxoj ba6169659e Added DNS checker 2021-05-22 03:12:04 +03:00
Soxoj a76f95858f Added checker of Tor sites 2021-05-20 23:26:02 +03:00
Soxoj 99fa58ceed Disabled Twitter activation test 2021-05-18 00:55:18 +03:00
Soxoj 2c04ccce57 Improved tests 2021-05-18 00:43:56 +03:00
Soxoj 28835204f5 Added text report to CLI output 2021-05-15 16:55:05 +03:00
Soxoj c9219d91ec Tags updated, added tests for tags
Added several sites
Updated socid_extractor version to avoid bug #150
2021-05-15 14:51:30 +03:00
Soxoj b696b982f4 Added some photo sites, improved errors detecting 2021-05-08 20:37:34 +03:00
Soxoj b6a207d0e3 Updated sites, improved submit dialog, bump to 0.2.2 2021-05-07 12:27:24 +03:00
Soxoj 0e9655c46a Improve extracting ids from URLs, tests 2021-05-06 22:35:44 +03:00
Soxoj 78e9688ece Test data fix 2021-05-05 23:27:30 +03:00
Soxoj 3cbb9df7b3 Main maigret function refactoring 2021-05-05 18:02:13 +03:00
Soxoj 3b91a9cd31 CLI arguments improvements, tests added 2021-05-05 15:27:56 +03:00