815c8cb2f3
build(deps): bump asgiref from 3.11.0 to 3.11.1 ( #2384 )
...
Bumps [asgiref](https://github.com/django/asgiref ) from 3.11.0 to 3.11.1.
- [Changelog](https://github.com/django/asgiref/blob/main/CHANGELOG.txt )
- [Commits](https://github.com/django/asgiref/compare/3.11.0...3.11.1 )
---
updated-dependencies:
- dependency-name: asgiref
dependency-version: 3.11.1
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-25 15:03:02 +01:00
656fe1df24
Added Max.ru check; --no-progressbar flag fixed ( #2386 )
2026-03-25 11:48:12 +01:00
1c5dc5f152
build(deps): bump pycountry from 24.6.1 to 26.2.16 ( #2382 )
...
Bumps [pycountry](https://github.com/pycountry/pycountry ) from 24.6.1 to 26.2.16.
- [Release notes](https://github.com/pycountry/pycountry/releases )
- [Changelog](https://github.com/pycountry/pycountry/blob/main/HISTORY.txt )
- [Commits](https://github.com/pycountry/pycountry/compare/24.6.1...26.2.16 )
---
updated-dependencies:
- dependency-name: pycountry
dependency-version: 26.2.16
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-25 09:54:11 +01:00
bc3d9faad9
Fix false-positive site checks reported by Maigret Bot ( #2376 )
2026-03-24 23:01:11 +01:00
5aae2ee005
Fix update-site-data workflow race condition on branch push ( #2366 )
...
* Initial plan
* Fix update-site-data workflow race condition on branch push
- Add concurrency control to cancel in-progress runs on new pushes to main
- Delete existing PR branch before creating new one to avoid stale ref conflicts
- Upgrade peter-evans/create-pull-request from v5 to v7 (Node.js 20 deprecation)
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/a095d3d3-0093-43e8-9cc5-82797bd52453
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-24 22:10:03 +01:00
b145e7b26f
feat(core): add POST request support, new sites, migrate to Majestic Million ranking ( #2317 )
...
* feat(core): add POST request support, new sites, migrate to Majestic Million ranking
- Added native POST request support to the Maigret engine (requestMethod, requestPayload) to enable querying modern JSON registration endpoints.
- Replaced the discontinued Alexa rank API with the Majestic Million dataset for global popularity sorting and automated CI updates.
- Fixed multiple false positives among top 500 sites and bypassed standard anti-bot protections using custom User-Agents.
- Updated public documentation and internal playbooks to reflect the new features.
* feat(data): apply all data.json site check updates from main branch
- Added CTFtime and PentesterLab (new sites added in main)
- Removed forums.imore.com (deleted in main as dead site)
- Disabled 5 sites per main branch fixes: Librusec, MirTesen, amateurvoyeurforum.com, forums.stevehoffman.tv, vegalab
- Fixed 5 site checks per main branch: SoundCloud, Taplink, Setlist, RoyalCams, club.cnews.ru (switched from status_code to message checkType with proper markers)
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/a1d194d9-c0ff-4e2b-974c-c5e4b59548bf
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
2026-03-24 22:08:42 +01:00
abd9aa57fe
Fix domain substring matching and NoneType crash in submit dialog ( #2367 )
...
* Initial plan
* Fix domain matching and NoneType error in submit.py
- Use regex with domain boundary matching instead of substring matching
to prevent x.com from matching 500px.com, mix.com, etc.
- Handle None old_site gracefully when user enters a site name not in
the matched list, fixing AttributeError crash.
- Add tests for both fixes.
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/7eabc755-47fd-4b80-a38c-9d6c056c2ce9
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-24 22:04:10 +01:00
2e430e5039
feat: add tag blacklisting via --exclude-tags ( #2352 )
...
* Initial plan
* feat: add tag blacklisting support (--exclude-tags CLI flag, web UI, docs, tests)
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/1a656af2-36bf-494f-9f03-1b5340f0357c
* fix: correct tag cloud label to match click-cycle interaction
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/1a656af2-36bf-494f-9f03-1b5340f0357c
* feat: add all country tags to web interface tag cloud
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/7e184b24-ff26-48fd-8a93-aea12b0a8d7b
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-24 22:00:59 +01:00
f5786f11ce
build(deps): bump certifi from 2025.11.12 to 2026.2.25 ( #2346 )
...
Bumps [certifi](https://github.com/certifi/python-certifi ) from 2025.11.12 to 2026.2.25.
- [Commits](https://github.com/certifi/python-certifi/compare/2025.11.12...2026.02.25 )
---
updated-dependencies:
- dependency-name: certifi
dependency-version: 2026.2.25
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 15:13:10 +01:00
3e56c95e16
Fix SoundCloud false-positive: switch to message-based check ( #2355 )
...
* Initial plan
* Fix SoundCloud false-positive: switch from status_code to message checkType
SoundCloud returns HTTP 200 for non-existent user profiles (soft 404),
causing status_code check to report CLAIMED for random usernames.
Switch to message checkType with:
- presenseStrs: hydratable user marker in server-rendered HTML
- absenceStrs: generic page title for non-existent users
Markers sourced from WhatsMyName project's verified SoundCloud entry.
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/8aa10eef-78bf-4251-bf42-473cd94c7ef4
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-24 15:12:56 +01:00
28f35f9a4f
Fix club.cnews.ru false positive: switch from status_code to message checkType ( #2342 )
...
* Initial plan
* Fix club.cnews.ru false positive: switch from status_code to message checkType with absence strings
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/af131d2f-c7b5-4798-8ad1-86bab2673fe4
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-24 10:52:23 +01:00
79cea49526
feat: add CTFtime and PentesterLab site support ( #2318 )
...
Add two cybersecurity platforms for username enumeration:
- CTFtime (ctftime.org) - CTF competition platform
- PentesterLab (pentesterlab.com) - Security training platform
Both verified working with status_code check type.
Returns 200 for existing users, 404 for non-existent.
Co-authored-by: Julio César Suástegui <juliosuas@users.noreply.github.com >
2026-03-24 10:52:07 +01:00
2d94269656
Automated Sites List Update ( #2341 )
...
* Updated site list and statistics
* Rebase and regenerate sites.md against latest main (#2351 )
* Updated site list and statistics
* Initial plan
* Disable MirTesen site check (false positive) (#2350 )
* Initial plan
* Disable MirTesen site check to fix false-positive probe
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/61c86064-423d-4f1b-8277-2838f747dd89
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
* build(deps): bump attrs from 25.4.0 to 26.1.0 (#2344 )
Bumps [attrs](https://github.com/sponsors/hynek ) from 25.4.0 to 26.1.0.
- [Commits](https://github.com/sponsors/hynek/commits )
---
updated-dependencies:
- dependency-name: attrs
dependency-version: 26.1.0
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Updated site list and statistics
---------
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: soxoj <soxoj@users.noreply.github.com >
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---------
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: soxoj <soxoj@users.noreply.github.com >
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 10:51:21 +01:00
829bda885a
build(deps): bump attrs from 25.4.0 to 26.1.0 ( #2344 )
...
Bumps [attrs](https://github.com/sponsors/hynek ) from 25.4.0 to 26.1.0.
- [Commits](https://github.com/sponsors/hynek/commits )
---
updated-dependencies:
- dependency-name: attrs
dependency-version: 26.1.0
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 09:51:42 +01:00
eb541dcf51
Disable MirTesen site check (false positive) ( #2350 )
...
* Initial plan
* Disable MirTesen site check to fix false-positive probe
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/61c86064-423d-4f1b-8277-2838f747dd89
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-24 09:51:31 +01:00
4c97025a32
Disable Librusec site check (false positive) ( #2349 )
...
* Initial plan
* Disable Librusec site check to fix false-positive probe
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
2026-03-24 09:51:16 +01:00
2775181a6a
build(deps-dev): bump mypy from 1.19.0 to 1.19.1 ( #2347 )
...
Bumps [mypy](https://github.com/python/mypy ) from 1.19.0 to 1.19.1.
- [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md )
- [Commits](https://github.com/python/mypy/compare/v1.19.0...v1.19.1 )
---
updated-dependencies:
- dependency-name: mypy
dependency-version: 1.19.1
dependency-type: direct:development
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 09:35:54 +01:00
b00ef1f5dd
build(deps): bump aiodns from 3.5.0 to 4.0.0 ( #2345 )
...
Bumps [aiodns](https://github.com/saghul/aiodns ) from 3.5.0 to 4.0.0.
- [Release notes](https://github.com/saghul/aiodns/releases )
- [Changelog](https://github.com/aio-libs/aiodns/blob/master/ChangeLog )
- [Commits](https://github.com/saghul/aiodns/compare/v3.5.0...v4.0.0 )
---
updated-dependencies:
- dependency-name: aiodns
dependency-version: 4.0.0
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 09:32:12 +01:00
d3f13ac295
Fix false-positive site probe: Re-enable Taplink with message checkType ( #2326 )
...
* Initial plan
* Disable Taplink site check to fix false-positive detections
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/ef9281f4-ba67-4760-a6e2-57564ac4ea94
* Re-enable Taplink with message checkType and absenceStrs
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/db3e572e-b79b-4cec-ac7f-062e76144660
* Improve Taplink absenceStrs: add Russian variant and presenseStrs
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/28e24317-e8b9-45f6-bad5-0e549b891313
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-23 21:36:36 +01:00
479a614d1d
Automated Sites List Update ( #2339 )
...
* Updated site list and statistics
* Rebase: merge origin/main into auto/update-sites-list (#2340 )
* Updated site list and statistics (#2315 )
Co-authored-by: soxoj <soxoj@users.noreply.github.com >
* Initial plan
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: soxoj <soxoj@users.noreply.github.com >
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
---------
Co-authored-by: soxoj <soxoj@users.noreply.github.com >
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-23 21:36:09 +01:00
e0559e4320
Updated site list and statistics ( #2315 )
...
Co-authored-by: soxoj <soxoj@users.noreply.github.com >
2026-03-23 20:20:53 +01:00
00a9249229
[WIP] Fix invalid link on forums.imore.com ( #2337 )
...
* Initial plan
* Remove dead forums.imore.com site from database
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/c83530d0-d24f-45fc-aca3-ae1e46ece33c
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-23 20:20:28 +01:00
005863c2e0
Fix Setlist site check: switch to message checkType with proper markers ( #2333 )
...
* Initial plan
* Disable Setlist site check due to false positives (soft 404)
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/8c552ca6-51e5-4e79-a791-ddd6f27d2461
* Fix Setlist check: switch to message checkType with proper markers
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/3c387df6-1dfe-451f-96d8-b4b6455f7857
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-23 20:18:33 +01:00
e3aada6aef
Fix RoyalCams site check using BongaCams white-label pattern ( #2334 )
...
* Initial plan
* Disable RoyalCams site check to fix false-positive probe
The Telegram Maigret bot auto-probe reported CLAIMED for three random
usernames. The status_code checkType is unreliable as the site returns
200 for non-existent user profiles (soft 404). Disabling the site check
until a reliable detection method can be established.
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/05b3d513-fe15-477d-a455-0c9ddf0b8b51
* Fix RoyalCams: switch to message checkType using BongaCams white-label pattern
RoyalCams runs on the BongaCams platform. Applied the same fix pattern:
- Switch from status_code to message checkType
- Use Portuguese locale (pt.royalcams.com) as urlProbe
- absenceStrs matches generic title on non-existent profiles
- presenseStrs matches Portuguese profile title for existing users
- Add browser-like headers matching BongaCams config
- Remove disabled flag
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/2f6a9523-278a-4992-ba7c-c320de14bfa4
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-23 20:16:45 +01:00
9b35fc1ab0
[WIP] Fix false-positive probe for vegalab site ( #2336 )
...
* Initial plan
* Disable vegalab site check: domain is dead (DNS does not resolve), causing false positives
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/98430e81-5dcb-4cb3-9aaa-f8c5ce86d026
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-23 20:09:46 +01:00
146bc0481b
Disable forums.stevehoffman.tv due to false positives ( #2331 )
...
* Initial plan
* Disable forums.stevehoffman.tv to fix false-positive site probe
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/39fea4a9-ec6d-4a12-b34b-1a3486d647e4
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-23 20:08:15 +01:00
5930a3022e
Disable false-positive site probe: amateurvoyeurforum.com ( #2332 )
...
* Initial plan
* Disable amateurvoyeurforum.com site check to fix false positives
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/e7fcad2b-4511-4e6d-b186-411951170e0a
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-23 20:07:42 +01:00
b4482e0ba4
build(deps): bump aiohttp-socks from 0.10.1 to 0.11.0 ( #2319 )
...
Bumps [aiohttp-socks](https://github.com/romis2012/aiohttp-socks ) from 0.10.1 to 0.11.0.
- [Release notes](https://github.com/romis2012/aiohttp-socks/releases )
- [Commits](https://github.com/romis2012/aiohttp-socks/compare/v0.10.1...v0.11.0 )
---
updated-dependencies:
- dependency-name: aiohttp-socks
dependency-version: 0.11.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 12:16:47 +01:00
2c55501bc2
build(deps-dev): bump pytest-cov from 7.0.0 to 7.1.0 ( #2320 )
...
Bumps [pytest-cov](https://github.com/pytest-dev/pytest-cov ) from 7.0.0 to 7.1.0.
- [Changelog](https://github.com/pytest-dev/pytest-cov/blob/master/CHANGELOG.rst )
- [Commits](https://github.com/pytest-dev/pytest-cov/compare/v7.0.0...v7.1.0 )
---
updated-dependencies:
- dependency-name: pytest-cov
dependency-version: 7.1.0
dependency-type: direct:development
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 12:15:36 +01:00
3ba07591a1
build(deps-dev): bump coverage from 7.12.0 to 7.13.5 ( #2321 )
...
Bumps [coverage](https://github.com/coveragepy/coveragepy ) from 7.12.0 to 7.13.5.
- [Release notes](https://github.com/coveragepy/coveragepy/releases )
- [Changelog](https://github.com/coveragepy/coveragepy/blob/main/CHANGES.rst )
- [Commits](https://github.com/coveragepy/coveragepy/compare/7.12.0...7.13.5 )
---
updated-dependencies:
- dependency-name: coverage
dependency-version: 7.13.5
dependency-type: direct:development
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 12:14:31 +01:00
a2d4373b68
build(deps): bump reportlab from 4.4.5 to 4.4.10 ( #2323 )
...
Bumps [reportlab](https://www.reportlab.com/ ) from 4.4.5 to 4.4.10.
---
updated-dependencies:
- dependency-name: reportlab
dependency-version: 4.4.10
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 12:14:18 +01:00
b960acec10
Pin requests-toolbelt>=1.0.0 to fix urllib3 v2 incompatibility ( #2316 )
...
* Initial plan
* Add requests-toolbelt ^1.0.0 as explicit dependency to fix urllib3 v2 appengine ImportError
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/458d41b2-c135-4b51-b0b1-b1832490c808
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-22 23:53:20 +01:00
b1a211c3cd
Disable forums.developer.nvidia.com (auth-gated user profiles) ( #2305 )
...
* Initial plan
* disable forums.developer.nvidia.com due to auth-locked user pages
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/b8f41f15-8588-4aac-a443-af5e2aaa1918
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-22 22:43:51 +01:00
56d0c9f2f1
Remove dead site xxxforum.org ( #2310 )
...
* Initial plan
* Remove broken site xxxforum.org from data.json and sites.md
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/bfbd3aa8-bfb1-480a-b2e7-a2c40fc69def
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-22 22:43:21 +01:00
01049b730d
Fix Love.Mail.ru: update to numeric-only identifiers and new profile URL ( #2307 )
...
* Initial plan
* fix: update Love.Mail.ru to use numeric-only identifiers (#1264 )
- Add regexCheck to enforce numeric-only IDs (^\d+$)
- Update usernameClaimed/usernameUnclaimed to numeric values
- Site remains disabled pending live verification
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/6de16097-6bc1-424a-beb1-1d2ec6b99944
* fix: update Love.Mail.ru URL to /profile/ path, enable check with verified ID
Use maintainer-provided working link https://love.mail.ru/profile/1838153357 .
- Change URL pattern from /ru/{username} to /profile/{username}
- Set usernameClaimed to 1838153357
- Remove disabled flag to enable the check
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/ac07d38e-46e2-42d3-9e93-eda3e5cfbcc3
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-22 22:42:59 +01:00
2c2d3409e2
Updated site list and statistics ( #2314 )
...
Co-authored-by: soxoj <soxoj@users.noreply.github.com >
2026-03-22 22:42:50 +01:00
e81b50ef61
Update site data workflow fix: remove ambiguous main tag ( #2313 )
...
* feat(workflow): fix update site data workflow err
* feat(workflow): the final update side data workflow fix (hopefully)
2026-03-22 22:37:48 +01:00
9ac0a65914
feat(workflow): fix update site data workflow err ( #2312 )
2026-03-22 22:31:55 +01:00
4f397fed1c
Re-enable taplink.cc with browser User-Agent to bypass Cloudflare ( #2308 )
...
* Initial plan
* fix(taplink): re-enable taplink.cc with browser User-Agent header to bypass Cloudflare
Remove disabled flag and add a Chrome User-Agent header to help
bypass Cloudflare bot detection for taplink.cc profile checks.
If Cloudflare still blocks requests, maigret's built-in error
detection will gracefully mark results as UNKNOWN.
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/271904b6-e358-4aeb-b503-21c9b91186d9
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-22 22:10:44 +01:00
a17e0c7a13
feat(workflow): fix update site data workflow dependency ( #2306 )
2026-03-22 21:34:30 +01:00
e84e394e6f
Bump svglib from 1.5.1 to 1.6.0 ( #2205 )
...
* Bump svglib from 1.5.1 to 1.6.0
Bumps [svglib](https://github.com/deeplook/svglib ) from 1.5.1 to 1.6.0.
- [Changelog](https://github.com/deeplook/svglib/blob/main/CHANGELOG.rst )
- [Commits](https://github.com/deeplook/svglib/commits )
---
updated-dependencies:
- dependency-name: svglib
dependency-version: 1.6.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
* Add libcairo2-dev to CI workflow for svglib 1.6.0 compatibility (#2304 )
* Initial plan
* Add libcairo2-dev system dependency install step to test workflow
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/3ecab70e-d4a3-4e74-9245-bffc58d6d0a3
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
---------
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com >
2026-03-22 21:06:10 +01:00
b8ada1c818
Update sites list workflow ( #2303 )
2026-03-22 20:59:37 +01:00
959b2be136
feat(sites): fix false positives: disable 74 broken sites, fix 8 with API probes and better markers ( #2302 )
...
- Disable 74 sites: Cloudflare/captcha blocks, identical responses,
dead domains, vBulletin/phpBB engine failures
- Fix Roblox, Salon24.pl, Planetaexcel → status_code (clear 404 signal)
- Fix en.brickimedia.org → message with "noarticletext" absenceStr
- Fix Arduino → narrower title-based presenseStrs/absenceStrs
- Re-enable Fandom (3 wikis) via MediaWiki api.php urlProbe
- Re-enable Substack via /api/v1/user/{}/public_profile urlProbe
- Re-enable hashnode via GraphQL GET urlProbe (URL-encoded query)
- Document lessons: engine template drift, search-by-author fragility,
always-200 sites, TLS degradation, API bypassing Cloudflare,
GraphQL GET support, URL-encoding for template safety
2026-03-22 20:47:51 +01:00
97cc4b46d9
Improve site-check quality: fix broken site configs, add diagnostic utilities, and make self-check report-only by default with opt-in auto-disable. ( #2301 )
...
- Fix VK and TradingView checkType; add Reddit and Microsoft Learn API-style probes where appropriate; adjust or disable entries that are unreliable under anti-bot protection.
- Self-check: stop aggressive auto-disable; default to reporting issues only; add --auto-disable and --diagnose for optional fixes and deeper output.
- Tooling: add utils/site_check.py and utils/check_top_n.py (and related helpers) to inspect and rank site behavior against the top-N list
- Scope: aligns with fixing top-traffic / high-impact sites and making diagnostics repeatable without silently flipping disabled flags
2026-03-22 16:48:35 +01:00
f3b741d283
Update Telegram bot link in README ( #2300 )
2026-03-22 12:23:35 +01:00
33620853a1
Bump certifi from 2025.10.5 to 2025.11.12 ( #2249 )
...
Bumps [certifi](https://github.com/certifi/python-certifi ) from 2025.10.5 to 2025.11.12.
- [Commits](https://github.com/certifi/python-certifi/compare/2025.10.05...2025.11.12 )
---
updated-dependencies:
- dependency-name: certifi
dependency-version: 2025.11.12
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-22 12:21:32 +01:00
19ed03a94d
build(deps): bump werkzeug from 3.1.4 to 3.1.6 ( #2288 )
...
Bumps [werkzeug](https://github.com/pallets/werkzeug ) from 3.1.4 to 3.1.6.
- [Release notes](https://github.com/pallets/werkzeug/releases )
- [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst )
- [Commits](https://github.com/pallets/werkzeug/compare/3.1.4...3.1.6 )
---
updated-dependencies:
- dependency-name: werkzeug
dependency-version: 3.1.6
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-22 12:20:42 +01:00
35372446e0
Bump reportlab from 4.4.4 to 4.4.5 ( #2251 )
...
Bumps [reportlab](https://www.reportlab.com/ ) from 4.4.4 to 4.4.5.
---
updated-dependencies:
- dependency-name: reportlab
dependency-version: 4.4.5
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-22 12:19:43 +01:00
519bb46db6
build(deps): bump flask from 3.1.2 to 3.1.3 ( #2289 )
...
Bumps [flask](https://github.com/pallets/flask ) from 3.1.2 to 3.1.3.
- [Release notes](https://github.com/pallets/flask/releases )
- [Changelog](https://github.com/pallets/flask/blob/main/CHANGES.rst )
- [Commits](https://github.com/pallets/flask/compare/3.1.2...3.1.3 )
---
updated-dependencies:
- dependency-name: flask
dependency-version: 3.1.3
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-22 12:19:21 +01:00
227a25bfa1
Twitter fixed, mirrors mechanism improvement ( #2299 )
2026-03-22 01:14:17 +01:00
dependabot[bot]