Commit Graph

15456 Commits

Author SHA1 Message Date
Derek Land 97f34a48d7
[rtlnl] Extend _VALID_URL (#26549) (closes #25821) 2020-09-13 21:38:16 +07:00
Daniel Peukert ea74e00b3a
[youtube] Fix empty description extraction (#26575) (closes #26006) 2020-09-13 21:23:21 +07:00
Sergey M․ 06cd4cdb25
[srgssr] Extend _VALID_URL (closes #26555, closes #26556, closes #26578) 2020-09-13 21:07:25 +07:00
Sergey M․ da2069fb22
[googledrive] Use redirect URLs for source format (closes #18877, closes #23919, closes #24689, closes #26565) 2020-09-13 20:49:32 +07:00
Sergey M․ 95c9810015
[svtplay] Fix id extraction (closes #26576) 2020-09-13 18:59:37 +07:00
Remita Amine b03eebdb6a [redbulltv] improve support for rebull.com TV localized URLS(#22063) 2020-09-13 11:26:11 +01:00
Remita Amine 1f7675451c [redbulltv] Add support for new redbull.com TV URLs(closes #22037)(closes #22063) 2020-09-12 19:27:58 +01:00
tfvlrue aa27253556
[soundcloud] Reduce pagination limit to fix 502 Bad Gateway errors when listing a user's tracks. (#26557)
Per the documentation here https://developers.soundcloud.com/blog/offset-pagination-deprecated the maximum limit is 200, so let's respect that (even if a higher value sometimes works).

Co-authored-by: tfvlrue <tfvlrue>
2020-09-12 09:35:11 +00:00
Sergey M․ d51e23d9fc
release 2020.09.06 2020-09-06 13:00:41 +07:00
Sergey M․ 50e9fcc1fd
[nrktv:episode] Improve video id extraction (closes #25594, closes #26369, closes #26409) 2020-09-06 12:43:50 +07:00
random-nick 16ee69c1b7
[youtube] Fix age gate content detection (#26100) (closes #26152, closes #26311, closes #26384) 2020-09-06 11:44:53 +07:00
Sergey M․ 67171ed7e9
[youtube:user] Extend _VALID_URL (closes #26443) 2020-09-06 11:31:28 +07:00
Sergey M․ 1d9bf655e6
[utils] Recognize wav mimetype (closes #26463) 2020-09-06 11:19:53 +07:00
TheRealDude2 62ae19ff76
[xhamster] Improve initials regex (#26526) (closes #26353) 2020-09-06 11:10:27 +07:00
Sergey M․ 5ed05f26ad
[svtplay] Fix svt id extraction (closes #26425, closes #26428, closes #26438) 2020-09-06 10:45:57 +07:00
Sergey M․ 841b683804
[twitch] Rework extractors (closes #12297, closes #20414, closes #20604, closes #21811, closes #21812, closes #22979, closes #24263, closes #25010, closes #25553, closes #25606)
* Switch to GraphQL.
+ Add support for collections.
+ Add support for clips and collections playlists.
2020-09-06 10:45:34 +07:00
Remita Amine f5863a3ea0 [biqle] improve video_ext extraction 2020-08-27 19:20:41 +01:00
Sergey M․ 10709fc7c6
[xhamster] Extend _VALID_URL (closes #25927) 2020-08-12 21:51:50 +07:00
TheRealDude2 a7e348556a
[xhamster] Fix extraction (closes #26157) (#26254) 2020-08-12 21:42:17 +07:00
JChris246 6cb30ea5ed
[xhamster] Extend _VALID_URL (closes #25789) (#25804) 2020-08-12 21:37:22 +07:00
Sergey M․ a4ed50bb84
release 2020.07.28 2020-07-28 05:13:03 +07:00
Sergey M․ e450f6cb63
[youtube] Fix sigfunc name extraction (closes #26134, closes #26135, closes #26136, closes #26137) 2020-07-28 05:05:38 +07:00
MRWITEK a115e07594
[youtube] Improve description extraction (closes #25937) (#25980) 2020-07-14 12:01:15 +01:00
Sergey M․ 718393c632
[wistia] Restrict embed regex (closes #25969) 2020-07-11 18:27:19 +07:00
Glenn Slayden 07af16b92e
[youtube] Prevent excess HTTP 301 (#25786) 2020-07-01 02:56:16 +07:00
Sergey M․ e942cfd1a7
[youtube:playlists] Extend _VALID_URL (closes #25810) 2020-06-28 10:30:03 +07:00
Remita Amine 9a7e5cb88a [bellmedia] add support for cp24.com clip URLs(closes #25764) 2020-06-23 15:09:13 +01:00
Sergey M․ 2391941f28
[brightcove] Improve embed detection (closes #25674) 2020-06-16 17:38:25 +07:00
Sergey M․ 9ff6165a81
release 2020.06.16.1 2020-06-16 06:22:01 +07:00
Sergey M․ ee0b726cd7
[youtube] Force old layout (closes #25682, closes #25683, closes #25680, closes #25686) 2020-06-16 06:17:53 +07:00
Sergey M․ dbeafce5d5
[youtube] Fix categories and improve tags extraction 2020-06-16 03:13:39 +07:00
Sergey M․ ed604ce7bc
release 2020.06.16 2020-06-16 02:53:33 +07:00
Sergey M․ a6211d237b
[youtube] Fix uploader id and uploader URL extraction 2020-06-16 02:43:09 +07:00
Sergey M․ 7b16239a49
[youtube] Improve view count extraction 2020-06-16 02:38:45 +07:00
Sergey M․ 37357d21a9
[youtube] Fix upload date extraction 2020-06-16 02:37:19 +07:00
Sergey M․ b477fc1314
[youtube] Fix thumbnails extraction and remove uploader id extraction warning (closes #25676) 2020-06-16 02:29:04 +07:00
Sergey M․ d84b21b427
[youtube] Fix playlist and feed extraction (closes #25675) 2020-06-16 02:01:12 +07:00
Philipp Hagemeister 48bd042ce7 [facebook] Support single-video ID links
I stumbled upon this at https://www.facebook.com/bwfbadminton/posts/10157127020046316 . No idea how prevalent it is yet.
2020-06-14 13:17:51 +02:00
Sergey M․ 84213ea8d4
[youtube] Extract chapters from JSON (closes #24819) 2020-06-06 04:22:10 +07:00
Sergey M․ 562de77f41
[kaltura] Add support for multiple embeds on a webpage (closes #25523) 2020-06-06 02:14:35 +07:00
Sergey M․ e1723c4bac
release 2020.06.06 2020-06-06 01:51:39 +07:00
Sergey M․ a5b6102ea8
[tele5] Bypass geo restriction 2020-06-06 01:45:05 +07:00
Sergey M․ b77888228d
[jwplatform] Add support for bypass geo restriction 2020-06-06 01:44:36 +07:00
Sergey M․ 0b1eaec3bc
[tele5] Prefer jwplatform over nexx (closes #25533) 2020-06-06 01:35:09 +07:00
Sergey M․ b37e47a3f9
[twitch:stream] Expect 400 and 410 HTTP errors from API 2020-06-06 00:57:40 +07:00
Sergey M․ ce3735df02
[twitch:stream] Fix extraction (closes #25528) 2020-06-06 00:55:29 +07:00
Sergey M․ a0455d0ffd
[twitch] Pass v5 accept header and fix thumbnails extraction (closes #25531) 2020-06-06 00:12:47 +07:00
Sergey M․ c8b232cc48
[brightcove] Sort imports 2020-06-05 23:35:57 +07:00
Sergey M․ b4eb0bc7bd
[brightcove] Fix subtitles extraction (closes #25540) 2020-06-05 23:33:14 +07:00
Matej Dujava d5147b65ac
[malltv] Add support for sk.mall.tv (#25445) 2020-06-01 21:11:31 +07:00
Sergey M․ 7b0b53ea69
[twitter:broadcast] Add untitled periscope broadcast test 2020-06-01 20:32:57 +07:00
Sergey M․ 7016e24ebe
[periscope] Fix untitled broadcasts (#25482) 2020-06-01 20:31:51 +07:00
Sergey M․ bef4688c72
[jwplatform] Improve embeds extraction (closes #25467) 2020-05-31 11:10:31 +07:00
Sergey M․ 228c1d685b
release 2020.05.29 2020-05-29 03:33:13 +07:00
Sergey M․ fe515e5c75
[ard:beta] Extend _VALID_URL (closes #25405) 2020-05-29 02:01:51 +07:00
striker.sh 1db5ab6b34
[youtube] Add support for more invidious instances (#25417) 2020-05-27 01:26:45 +07:00
Sergey M․ 2791e80b60
[postprocessor/ffmpeg] Embed series metadata with --add-metadata 2020-05-23 12:28:15 +07:00
JordanWeatherby 8f841fafcd
[giantbomb] Extend _VALID_URL (#25222) 2020-05-21 04:30:50 +07:00
Michael Klein a54c5f83c0
[ard] Improve _VALID_URL (closes #25134) (#25198) 2020-05-20 04:08:08 +07:00
Sergey M․ cd13343ad8
[redtube] Improve formats extraction and extract m3u8 formats (closes #25311, closes #25321) 2020-05-20 03:39:41 +07:00
Rob 9cd5f54e31
[utils] Fix file permissions in write_json_file (closes #12471) (#25122) 2020-05-20 03:21:52 +07:00
tlsssl 9a269547f2
[indavideo] Switch to HTTPS for API request (#25191) 2020-05-20 02:13:06 +07:00
Dave Loyall bf097a5077
[redtube] Improve title extraction (#25208) 2020-05-20 02:11:05 +07:00
Remita Amine 52c50a10af [vimeo] improve format extraction and sorting(closes #25285) 2020-05-15 15:57:06 +01:00
Remita Amine b334732709 [soundcloud] reduce API playlist page limit(closes #25274) 2020-05-15 14:13:02 +01:00
Juan Francisco Cantero Hurtado 384bf91f88
[youtube] Add support for yewtu.be (#25226) 2020-05-14 05:54:42 +07:00
comsomisha adc13b0748
[mailru] Fix extraction (closes #24530) (#25239) 2020-05-14 05:51:40 +07:00
Sergey M․ 327593257c
[bbccouk] PEP8 2020-05-14 05:11:42 +07:00
Remita Amine 9d8f3a12a6 [spike] fix Bellator mgid extraction(closes #25195) 2020-05-12 20:49:08 +01:00
Sergey M․ b002bc433a
release 2020.05.08 2020-05-08 18:10:37 +07:00
Sergey M․ fa3db38333
[youtube] Improve signature cipher extraction (closes #25188) 2020-05-08 17:42:30 +07:00
Sergey M․ 30fa5c6087
[iprima] Improve extraction (closes #25138) 2020-05-06 23:20:14 +07:00
Sergey M․ 6c907eb33f
[downloader/http] Request last data block of exact remaining size
Always request last data block of exact size remaining to download if possible not the current block size.
2020-05-05 21:43:39 +07:00
Sergey M․ f7b42518dc
[downloader/http] Finish downloading once received data length matches expected
Always do this if possible, i.e. if Content-Length or expected length is known, not only in test.
This will save unnecessary last extra loop trying to read 0 bytes.
2020-05-05 21:43:39 +07:00
Remita Amine ce7db64bf1 [uol] fix extraction(closes #22007) 2020-05-05 11:19:40 +01:00
hh0rva1h 1328305851
[orf] Add support for more radio stations (closes #24938) (#24968) 2020-05-05 06:22:50 +07:00
Sergey M․ 6c22cee673
[extractor/common] Use compat_cookiejar_Cookie for _set_cookie (closes #23256, closes #24776)
To always ensure cookie name and value are bytestrings on python 2.
2020-05-05 06:00:37 +07:00
Sergey M․ 6d874fee2a
[compat] Introduce compat_cookiejar_Cookie 2020-05-05 05:54:10 +07:00
Sergey M․ 676723e0da
[dailymotion] Fix typo 2020-05-05 05:09:07 +07:00
Sergey M․ c380cc28c4
[utils] Improve cookie files support
+ Add support for UTF-8 in cookie files
* Skip malformed cookie file entries instead of crashing (invalid entry len, invalid expires at)
2020-05-05 04:21:25 +07:00
Sergey M․ f7f304910d
[puhutv] Remove no longer available HTTP formats (closes #25124) 2020-05-04 21:15:19 +07:00
Sergey M․ 00a41ca4c3
release 2020.05.03 2020-05-03 00:05:05 +07:00
Sergey M․ 6ffc3cf74a
[crunchyroll] Fix and improve extraction (closes #25096, closes #25060) 2020-05-02 23:42:51 +07:00
Sergey M․ 4433bb0245
[extractor/common] Extract multiple JSON-LD entries 2020-05-02 23:40:30 +07:00
Sergey M․ e40c758c2a
[youtube] Improve player id extraction and add tests 2020-05-02 07:18:08 +07:00
Sergey M․ 011e75e641
[youtube] Use redirected video id if any (closes #25063) 2020-05-01 00:40:38 +07:00
Remita Amine 2468a6fa64 [yahoo] fix GYAO Player extraction and relax title URL regex(closes #24178)(closes #24778) 2020-04-29 14:56:32 +01:00
Remita Amine 700265bfcf [tvplay] fix Viafree extraction(closes #15189)(closes #24473)(closes #24789) 2020-04-29 13:38:58 +01:00
Sergey M․ c97f5e934f
[tenplay] Relax _VALID_URL (closes #25001) 2020-04-26 12:41:33 +07:00
Sergey M․ 38db9a405a
[prosiebensat1] Extract series metadata 2020-04-24 02:56:10 +07:00
Philipp Stehle 2cdfe977d7
[prosiebensat1] Improve extraction and remove 7tv.de support (#24948) 2020-04-24 02:44:13 +07:00
willbeaufoy 46d0baf941
[options] Clarify doc on --exec command (closes #19087) (#24883) 2020-04-24 02:31:38 +07:00
Sergey M․ 00eb865b3c
[youtube] Fix DRM videos detection (refs #24736) 2020-04-11 23:05:08 +07:00
Sergey M․ 2f19835726
[thisoldhouse] Improve video id extraction (closes #24549) 2020-04-11 20:07:37 +07:00
AndrewMBL 533f3e3557
[thisoldhouse] Fix video id extraction (closes #24548)
Added support for:
with of without "www."
and either  ".chorus.build" or ".com"

It now validated correctly on older URL's
```
<iframe src="https://thisoldhouse.chorus.build/videos/zype/5e33baec27d2e50001d5f52f
```
and newer ones
```
<iframe src="https://www.thisoldhouse.com/videos/zype/5e2b70e95216cc0001615120
```
2020-04-11 20:07:32 +07:00
Sergey M․ 75294a5ed0
[soundcloud] Improve AAC format extraction (closes #19173, closes #24708) 2020-04-10 17:26:03 +07:00
tom b9e5f87291
[soundcloud] Extract AAC format 2020-04-10 17:25:04 +07:00
Sergey M․ 6b09401b0b
[youtube] Skip broken multifeed videos (closes #24711) 2020-04-09 22:42:43 +07:00
Sergey M․ 5caf88ccb4
[nova:embed] Fix extraction (closes #24700) 2020-04-09 03:52:29 +07:00
Sergey M․ dcc8522fdb
[motherless] Fix extraction (closes #24699) 2020-04-09 02:14:49 +07:00
Felix Stupp c9595ee780
[twitch:clips] Extend _VALID_URL (closes #24290) (#24642) 2020-04-07 23:21:25 +07:00
Sergey M․ 91bd3bd019
[tv4] Fix ISM formats extraction (closes #24667) 2020-04-07 22:56:06 +07:00
Sergey M․ 13b08034b5
[extractor/common] Skip malformed ISM manifest XMLs while extracting ISM formats (#24667) 2020-04-07 22:55:59 +07:00
Sergey M․ 6a6e1a0cd8
[tele5] Fix extraction (closes #24553) 2020-04-06 02:05:06 +07:00
Sergey M․ 4e7b5bba5f
[mofosex] Add support for generic embeds (closes #24633) 2020-04-06 01:29:58 +07:00
Sergey M․ 52c4c51556
[youporn] Add support form generic embeds 2020-04-05 20:56:14 +07:00
Sergey M․ 8fae1a04eb
[spankwire] Add support for generic embeds (refs #24633) 2020-04-05 20:42:56 +07:00
Sergey M․ d44a707fdd
[spankwire] Fix extraction (closes #18924, closes #20648) 2020-04-05 20:42:56 +07:00
Sergey M․ 049c0486bb
release 2020.03.24 2020-03-24 03:14:30 +07:00
Sergey M․ 6e47200b6e
[teachable] Update test 2020-03-24 02:57:53 +07:00
Sergey M․ 38fa761a45
[teachable] Update gns3 domain 2020-03-24 02:57:48 +07:00
Sergey M․ 08a27407c4
[teachable] Update upskillcourses domain
New version does not use teachable platform any longer
2020-03-24 02:57:44 +07:00
Sergey M․ be7dacf9cf
[generic] Look for teachable embeds before wistia 2020-03-24 02:57:38 +07:00
Sergey M․ 4560adc820
[teachable] Extract chapter metadata (closes #24421) 2020-03-24 02:57:32 +07:00
Sergey M․ 63dce3094b
[bilibili] Add support for player.bilibili.com (closes #24402) 2020-03-24 00:24:39 +07:00
Sergey M․ b4eb08bb03
[bilibili] Add support for new URL schema with BV ids (closes #24439, closes #24442) 2020-03-24 00:11:39 +07:00
Remita Amine 2e20cb3636 [limelight] remove disabled API requests(closes #24255) 2020-03-23 12:57:10 +01:00
Remita Amine a6c5859d6b [soundcloud] fix download url extraction(closes #24394) 2020-03-22 09:24:26 +01:00
Sergey M․ c76cdf2382
[cbc:watch] Fix authenticated device token caching (closes #19160) 2020-03-21 01:43:13 +07:00
Devon Meunier 787c360467
[cbc:watch] Add support for authentication 2020-03-21 01:43:08 +07:00
Sergey M․ 73453430c1
[hellporno] Fix extraction (closes #24399) 2020-03-21 00:59:48 +07:00
Sergey M․ 158bc5ac03
[xtube] Fix typo 2020-03-14 22:58:10 +07:00
Sergey M․ 4568a11802
[xtube] Fix formats extraction (closes #24348) 2020-03-14 22:57:10 +07:00
Sergey M․ 4cbce88f8b
[ndr] Fix extraction (closes #24326) 2020-03-14 04:58:24 +07:00
Sergey M․ 541fe3eaff
[nhk] Update m3u8 URL and use native hls (#24329) 2020-03-14 04:42:40 +07:00
Sergey M․ 9bfe088594
[nhk] Remove obsolete rtmp formats (closes #24329) 2020-03-14 04:40:11 +07:00
Sergey M․ fcaf4d7a06
[nhk] Relax _VALID_URL (#24329) 2020-03-14 04:39:21 +07:00
Remita Amine 40b6495d40 Revert "[vimeo] fix showcase password protected video extraction(closes #24224)"
This reverts commit 12ee431676.
2020-03-13 08:59:10 +01:00
Sergey M․ f1a8511f7b
[utils] Add reference to cookie file format 2020-03-10 04:59:02 +07:00
Sergey M․ 042b664933
Revert "[utils] Add support for cookies with spaces used instead of tabs"
According to [1] TABs must be used as separators between fields.
Files produces by some tools with spaces as separators are considered
malformed.

1. https://curl.haxx.se/docs/http-cookies.html

This reverts commit cff99c91d1.
2020-03-10 04:53:51 +07:00
Sergey M․ 68fa15155f
release 2020.03.08 2020-03-08 18:27:20 +07:00
Sergey M․ cff99c91d1
[utils] Add support for cookies with spaces used instead of tabs 2020-03-08 18:01:32 +07:00
Tristan Waddington fa9b8c6628
[pornhub] Add support for pornhubpremium.com (#24288) 2020-03-08 18:00:25 +07:00
Sergey M․ 43ebf77df3
[youtube] Remove outdated code
Additional get_video_info requests don't seem to provide any extra itags any longer
2020-03-08 08:59:58 +07:00
Sergey M․ d332ec725d
[youtube] Improve age-gated videos extraction in 429 error conditions (refs #24283) 2020-03-08 05:41:04 +07:00
Sergey M․ f93abcf1da
[youtube] Improve extraction in 429 error conditions (closes #24283) 2020-03-08 05:09:02 +07:00
Remita Amine 0ec9d4e565 [nhk] update API version(closes #24270) 2020-03-06 20:13:28 +01:00
Sergey M․ 34525a3885
release 2020.03.06 2020-03-06 00:25:43 +07:00
Sergey M․ 5429d6a9cb
[youtube] Fix tests 2020-03-06 00:05:50 +07:00
Sergey M․ dc879c5a37
[youtube] Fix age-gated videos support without login (closes #24248) 2020-03-05 23:48:25 +07:00
Remita Amine 12ee431676 [vimeo] fix showcase password protected video extraction(closes #24224) 2020-03-03 12:33:57 +01:00
Sergey M․ 46cc54ca8f
[pornhub] Improve title extraction (closes #24184) 2020-03-03 06:23:39 +07:00
Sergey M․ 1e1c1960aa
[peertube] Fix issues and improve extraction (closes #23657) 2020-03-03 03:01:47 +07:00
3risian ac379fa236
[peertube] Improve extraction 2020-03-03 03:01:42 +07:00
jxu 0e30a7b973
[youtube:playlist] Fix tests (closes #23872) (#23885) 2020-03-03 01:46:00 +07:00
Sergey M․ 3b5399ce0f
[servus] Add support for new URL schema (closes #23475, closes #23583, closes #24142) 2020-03-03 01:41:53 +07:00
tsia 1c45ff5572
[vimeo] Fix subtitles URLs (#24209) 2020-03-03 01:27:40 +07:00
Sergey M․ 669625a32c
release 2020.03.01 2020-03-01 20:11:32 +07:00
Sergey M․ b274e48d56
[xhamster] Fix extraction (closes #24205) 2020-03-01 20:04:48 +07:00
Sergey M․ 50d19895a1
[franceculture] Fix extraction (closes #24204) 2020-03-01 19:22:09 +07:00