Commit Graph

9406 Commits

Author SHA1 Message Date
Remita Amine 5abcca9060 [sixplay] use raw string for regex 2017-02-10 09:34:59 +01:00
Sergey M․ e01bfc19c3
[extractor/commonmistakes] Restrict _VALID_URL (closes #12050) 2017-02-10 09:39:24 +07:00
Remita Amine 4d32b63851 [tvplayer] Add new extractor 2017-02-09 23:09:21 +01:00
Sergey M․ ff24261ba0
[kaltura] Add explicit port to regexes
They should not match e.g. cdnapi.kaltura.computernetworks.com/...
2017-02-10 01:24:14 +07:00
Sergey M․ fbc6dc525e
[xtube] Fix shortcuts 2017-02-10 01:06:23 +07:00
Sergey M․ 9150d1eb69
[xtube] Fix extraction (closes #12023) 2017-02-10 01:03:35 +07:00
Sergey M․ b7f9843bec
[pornhub] Simplify (closes #12018) 2017-02-10 00:57:44 +07:00
Thomas Christlieb e64b0fca14
[pornhub] Fix extraction (closes #12007) 2017-02-10 00:56:12 +07:00
Sergey M․ 78ef214d2d
[facebook] Improve JS data regex (closes #12042) 2017-02-09 23:42:40 +07:00
Remita Amine 37084f6641 [kaltura] improve embed partner id extraction(fixes #12041) 2017-02-09 16:24:54 +01:00
Remita Amine b04975733c [sprout] Add new extractor 2017-02-09 09:13:29 +01:00
Remita Amine c8b8fb0a99 [sixplay] improve extraction
- skip drm protected formats
- extract more and better formats
- skip duplicate asset urls
2017-02-08 22:56:10 +01:00
Remita Amine 8298018273 [scrippsnetworks:watch] Add new extractor(closes #10765) 2017-02-08 20:44:23 +01:00
Remita Amine ae8d5a5c59 [go] add support for adobe pass auth(closes #11468)(closes #10831) 2017-02-08 18:57:07 +01:00
Sergey M․ b9c9cb5f79
[6play] Fix extraction (closes #12011) 2017-02-08 23:15:39 +07:00
Remita Amine fdf9b959bc [nbc] add support adobe pass auth(closes #12006) 2017-02-08 16:23:42 +01:00
Sergey M․ f28aeff264
[pornhub] Fix extraction (closes #11997) 2017-02-07 01:52:59 +07:00
Sergey M․ 242a14a1f6
[extractor/common] Fix audio only with audio group in m3u8 (closes #11995) 2017-02-07 00:22:16 +07:00
Sergey M․ d5d904ff7d
[canalplus] Add support for cstar.fr (#11990) 2017-02-06 23:53:42 +07:00
Sergey M․ 5620f840f6
[extractor/generic] Add test for #11993 and more metadata for rtmp 2017-02-06 23:31:58 +07:00
Sergey M․ b7a8c1bcfa
[extractor/generic] Improve rtmp support (closes #11993) 2017-02-06 23:23:40 +07:00
Sergey M․ 2aec7256ae
[extractor/common] Speed-up media tags regex (closes #11979) 2017-02-06 00:20:30 +07:00
Yen Chi Hsuan 9c14fe9681
[gaskrank] Minor change and update ChangeLog after #11685 2017-02-06 00:25:28 +08:00
motophil e705755739 [gaskrank] Add new extractor (#11685)
* [gaskrank] Add new extractor

* [gaskrank] Add new extractor - fixes as requested

* [gaskrank] Add new extractor - style fix

* [Gaskrank] Add new extractor - requested fixes

* [Gaskrank] Add new extractor - fix md5 checksum

* [gaskrank] Add new extractor - more requested fixes

* [Gaskrank] Add new extractor - fixed all but one quantified code issues

* [Gaskrank] add new extractor - more fields extracted, added second test

* [Gaskrank] Add new extractor - requested fixes.

* [Gaskrank] Add new extractor - requested changes.

* [Gaskrank] Add new extractor - final(?) fixes.
2017-02-06 00:19:37 +08:00
Yen Chi Hsuan 019f4c0371
[bandcamp] Fix extraction for incomplete albums
Closes #11727
2017-02-05 22:47:04 +08:00
Yen Chi Hsuan 2ab2c0d1f5
[iwara] Add width (closes #11724)
The heuristic is from #11724
2017-02-05 22:30:13 +08:00
Yen Chi Hsuan caf0f5f8b7
[iwara] Fix extraction (closes #11781) 2017-02-05 21:48:13 +08:00
Yen Chi Hsuan e4e50f60b1
[googledrive] Fix extraction on Python 3.6
Since Python 3.6, invalid escape sequences are deprecated. It's likely
that there are invalid escape sequences somewhere on the webpage, so
instead of unescaping the whole webpage, just unescape the URL.

See https://bugs.python.org/issue27364. That change was designed for
string literals, while it affects the 'unicode_escape' encoding as well.
The code path is:

str.decode('unicode_escape')
    codecs.unicode_escape_decode()
        PyUnicode_DecodeUnicodeEscape()
2017-02-05 21:41:08 +08:00
Sergey M․ 6ef3e65a7b
[videopress] Add extractor 2017-02-05 13:37:27 +07:00
Sergey M․ 6fd138bed8
[sportbox] PEP 8 2017-02-05 13:36:52 +07:00
Remita Amine 3d2c2752c5 [afreecatv] extract rtmp formats 2017-02-04 18:18:28 +01:00
Sergey M․ 9db8f6c540
[twitch:stream] Improve _VALID_URL (closes #11971) 2017-02-04 23:21:07 +07:00
Remita Amine 8e4041cf3f [radiocanada] fix extraction for toutv rtmp formats 2017-02-04 17:05:35 +01:00
John Hawkinson c2521c1ac6 [Piksel] Add another app token regex 2017-02-04 23:23:14 +08:00
A Connecticut Princess 643dc0fcfe
[vk] Catch author blocked error message
Example link (video in blocked group):
https://vk.com/search?c%5Bq%5D=%D0%9F%D1%80%D1%8B%D0%B6%D0%BE%D0%BA%20c%20%D0%BA%D1%80%D0%B0%D0%BD%D0%B0%20%D0%B2%20%D1%81%D1%82%D0%B8%D0%BB%D0%B5%20%D0%A7%D0%B5%D0%BB%D0%BE%D0%B2%D0%B5%D0%BA%D0%B0-%D0%BF%D0%B0%D1%83%D0%BA%D0%B0&c%5Bsection%5D=video&c%5Bsort%5D=2&z=video-10639516_456240611
2017-02-04 22:21:09 +07:00
Remita Amine 36fce54816 [turner] fix downloading of secure hls formats using ffmpeg(closes #11358)(closes #11373)(closes #11800) 2017-02-04 15:23:46 +01:00
Remita Amine 2c15db829c [drtv] add support for live and radio sections(closes #1827)(closes #3427) 2017-02-04 08:38:28 +01:00
Remita Amine f65dba7cdb [myspace] fix extraction and extract hls and http formats 2017-02-03 22:25:19 +01:00
Remita Amine 605fd6392f [youtube] add format info for itag 325 and 328 2017-02-03 17:59:48 +01:00
Sergey M․ f962790ee5
[vine] Fix extraction (closes #11955) 2017-02-03 21:56:48 +07:00
Sergey M․ b7cc5f078e
[extractors] Remove remnants of sportbox extractor (#11954) 2017-02-03 21:56:10 +07:00
Sergey M․ f7a10d8cd6
[sportbox] Remove extractor (closes #11954)
Covered by generic extractor
2017-02-03 21:25:44 +07:00
Remita Amine df0588a31f Merge branch 'fstirlitz-filmon' 2017-02-03 10:15:52 +01:00
Remita Amine 4ce3407d08 [filmon] improve extraction 2017-02-03 10:15:03 +01:00
Mattias Wadman 45024183ae [infoq] Add audio only format if available (#11565)
* [infoq] Add audio only format if available

Refactor cookie code into a function.
Renamed formats to http_video, http_audio, rtmp_video
Renamed extract functions to video instead of videos as they return
one or no video.

* [infoq] Rename to _extract_cookies as it more than one

* [infoq] Remove redundant determine_ext

* [infoq] Add comment about hardcoded URL

* [infoq] Use _hidden_inputs instead of messy regex

* [infoq] Probe if audio URL is valid

Make it possible to pass headers to _is_valid_url

* [infoq] Add audio only test
2017-02-03 12:10:13 +08:00
Justsoos 33da98f493
[douyutv] Improve room id regex
http://www.douyu.com/t/lpl  source get extra '\' with "room_id\" (from js coding)
2017-02-03 03:26:41 +07:00
Michal Čihař 0bbcc8a10a
[iprima] Fix extraction (closes #11920, closes #11896) 2017-02-03 03:04:33 +07:00
Yen Chi Hsuan a22b2fd19b
[youtube] Fix ytsearch* when cookies are provided
Closes #11924

The API with `page` is no longer used in browsers, and YouTube always
returns {'reload': 'now'} when cookies are provided.

See http://youtube.github.io/spfjs/documentation/start/ for how SPF
works. Basically appending static link with a `spf` parameter yields the
corresponding dynamic link.
2017-02-03 01:28:24 +08:00
Sergey M․ c54c01f82d
[go] Relax video id regex (closes #11937) 2017-02-02 23:04:46 +07:00
Sergey M․ 5a116e1302
[facebook] Fix title extraction (closes #11941) 2017-02-02 22:45:18 +07:00
Sergey M․ a685751051
[youtube:playlist] Recognize TL playlists (closes #11945) 2017-02-02 22:01:11 +07:00
Yen Chi Hsuan bd8f48c78b
[bilibili] Support new Bangumi URLs (closes #11845)
To reduce complexity, I don't support old Bangumi URLs directly via
_VALID_URL. Instead, I choose to let it go to generic redirection. An
example can be found in #10190:

http://bangumi.bilibili.com/anime/v/40062
2017-02-02 21:51:31 +08:00
Remita Amine 81aeafeb44 [cbc:watch] extract audio codec for audion only formats(fixes #11893) 2017-02-02 08:07:28 +01:00
Jaime Marquínez Ferrándiz 020c5df52d [elpais] Fix extraction for some URLs (closes #11765) 2017-02-01 23:48:34 +01:00
Sergey M․ b83ef507b4
[facebook] Fix extraction (closes #11926) 2017-02-01 23:19:20 +07:00
Yen Chi Hsuan fe5aa197b5
[prosiebensat1] PEP8 and update _TESTS 2017-02-01 23:13:45 +08:00
Thomas Christlieb 7882f1115e Added new Regex for prosiebensat1 Extractor Description. Fixes #11810 (#11929)
* Added new Regex for prosiebensat1 Extractor Description. Fixes #11810

* Using _og_search_description() as a Fallback for Description-Regex

* Using _og_search_description() as a Fallback for Description-Regex - Second try

* Also added fallback regex

* Using _og_search_description() as a Fallback for Description-Regex - Third try

* removed fatal=False from search for description regex. default=None should be preferred only

* Using fatal=false for _og_search_description

* Revert "Using fatal=false for _og_search_description"

This reverts commit 2b7e123f9d0f2bd6ada54fa8e4e6035fece5dbf4.

* Deleted default=None Parameter for _og_search_property
2017-02-01 23:00:41 +08:00
Yen Chi Hsuan 2b2d5d319b
[crunchyroll] Remove ScaledBorderAndShadow settings
See https://github.com/rg3/youtube-dl/pull/9028, especially @lachs0r's
comments for the reason behind this change
2017-02-01 16:39:32 +08:00
Sergey M․ 26c0f09935
[vimeo] PEP 8 2017-02-01 02:15:52 +07:00
Sergey M․ c15cd29640
[vimeo] Extract upload timestamp 2017-02-01 00:58:02 +07:00
Sergey M․ c38a67bcd5
[vimeo] Extract license (closes #11880) 2017-02-01 00:49:28 +07:00
Sergey M․ 7c5329e6f4
[nrk] Improve extraction and update tests (closes #11571) 2017-02-01 00:29:29 +07:00
Odd Stråbø 8fd65faece
[NRKTV] Added NRKTVSeriesIE
[NRKTV] Added season and episode number to metadata.

[NRKTV] Added category to metadata.

[NRKTV] Added tests to NRKTVSeries.

[NRKTV] Fixed whitespace issues (flake8).
2017-01-31 23:10:40 +07:00
Sergey M․ 3c90cc8b6f
[youtube] Fix extraction for domainless player URLs
Closes #11890
Closes #11891
Closes #11892
Closes #11894
Closes #11895
Closes #11897
Closes #11900
Closes #11903
Closes #11904
Closes #11906
Closes #11907
Closes #11909
Closes #11913
Closes #11914
Closes #11915
Closes #11916
Closes #11917
Closes #11918
Closes #11919
2017-01-31 22:22:37 +07:00
Remita Amine ae9a173b64 [vimeo] extract both mixed and separated dash formats 2017-01-31 14:47:56 +01:00
Remita Amine dadb836139 [ruutu] extract dash formats 2017-01-30 09:32:31 +01:00
Remita Amine 4719419951 [itv] fix extraction in python 2.6 2017-01-29 16:04:15 +01:00
Alex Seiler f13da8af28
[azmedien:playlist] Add support for topic and themen playlists 2017-01-29 20:40:19 +07:00
Sergey M․ e228616c6e
[extractor/common] Fix initialization template (closes #11605, closes #11825) 2017-01-29 06:57:39 +07:00
Sergey M․ c58c2d63cb
[extractor/common] Document forgotten fragment base and path interfaces 2017-01-29 05:57:53 +07:00
Sergey M․ d04621daf4
[extractor/common] Fix duration per dash segment (closes #11868) 2017-01-29 05:36:53 +07:00
Andre Walker 56fc078da8 [npo] Update subtitles url
NPO websites changed the domain they used for subtitles, from
e.omroep.nl to tt888.omroep.nl.
2017-01-29 02:29:39 +07:00
Remita Amine 4edeac5bfa [itv] fix subtitle extraction 2017-01-28 17:28:18 +01:00
Remita Amine f592ff9868 [itv] extract subtitles 2017-01-28 17:26:14 +01:00
Remita Amine a71b8d3b3b [itv] Add new extractor(closes #9240) 2017-01-28 15:52:50 +01:00
Sergey M․ 008f247077
[mtv81] Add extractor (closes #7619) 2017-01-28 20:29:22 +07:00
Sergey M․ 661cc229d2
[vlive:channel] Improve 2017-01-28 19:08:01 +07:00
ping b92d3c5343 [vlive] Add support for channels 2017-01-28 18:50:38 +07:00
Sergey M․ ab6f6aee78
[kaltura] Add fallback for fileExt 2017-01-28 18:27:42 +07:00
Sergey M․ 26e40542dd
[kaltura] Improve uploader_id extraction 2017-01-28 17:50:56 +07:00
Alex Seiler 99a0baf370
[konserthusetplay] Add support for rspoplay.se 2017-01-28 01:58:23 +07:00
Sergey M․ e0b6e50ccd
[crunchyroll] Improve series and season metadata extraction (closes #11832) 2017-01-27 23:55:55 +07:00
Sergey M․ 3a194cb4ec
[soundcloud] Improve formats extraction and extract audio bitrate 2017-01-27 23:16:30 +07:00
Sergey M․ 9b73471801
[soundcloud] Extract hls formats 2017-01-27 23:08:32 +07:00
Sergey M․ 489ffc1182
[soundcloud] Fix track URL extraction (closes #11852) 2017-01-27 22:55:42 +07:00
Sergey M․ 0b23c222ba
[twitch:vod] Expand _VALID_URL (closes #11846) 2017-01-27 21:33:05 +07:00
Remita Amine b51a4ebed4 [aenetworks] fix season episodes extraction(fixes #11669) 2017-01-26 19:15:43 +01:00
Remita Amine 9463637887 [tva] Add new extractor(closes #11842) 2017-01-26 18:37:22 +01:00
Sergey M․ 3cbecdd111
[jamendo] Improve and extract more metadata (closes #11836) 2017-01-26 23:26:18 +07:00
Stefan Pöschel c19ef77c31
[jamendo] Extract full title 2017-01-26 22:53:16 +07:00
Remita Amine b3277115a1 [disney] Add new extractor(closes #7409)(closes #11801)(#4975)(#11000) 2017-01-26 16:16:14 +01:00
Remita Amine 9bccdc7004 [vevo] remove request to old api and catch apiv2 errors 2017-01-26 16:16:14 +01:00
Sergey M․ cf0cabbe50
[cmt,mtv,southpark] Add support for episode URLs (closes #11837) 2017-01-26 21:49:34 +07:00
Sergey M․ 556dbe7fe3
[youtube] Add fallback for duration extraction (closes #11841) 2017-01-26 21:43:14 +07:00
Sergey M․ c1fa3f4672
[openload] Fallback video extension to mp4 2017-01-25 23:28:45 +07:00
Sergey M․ 17f8deeb48
[extractor/generic] Add support for openload embeds (closes #11536, closes #11812) 2017-01-25 23:27:22 +07:00
Remita Amine b8a03b6660 [srgssr] fix rts video extraction(closes #11831) 2017-01-25 07:39:41 +01:00
Remita Amine c60089c022 [afreecatv:global] Add new extractor(closes #11807) 2017-01-25 07:39:41 +01:00
Sergey M․ af59bddc4e
[konserthusetplay] Extract subtitles (#11823) 2017-01-24 23:03:09 +07:00
Remita Amine 23b35a634e [crackle] improve extraction
- extract vtt subtitles
- extract multiple resolutions for thumbnails
- pass geo verification proxy headers
- add support for mobile urls
2017-01-24 16:57:22 +01:00
Sergey M․ 74af9c700d
[konserthusetplay] Add support for hls formats (closes #11823) 2017-01-24 22:55:49 +07:00
Sergey M․ d61aa5eb37
[vimeo:review] Fix config URL extraction (closes #11821) 2017-01-24 22:46:40 +07:00
Sergey M․ b494d6856c
[pluralsight] Fix extraction (closes #11820) 2017-01-24 02:51:45 +07:00
Yen Chi Hsuan bc35ed3fb6
[nextmedia] Add support for NextTV (壹電視) 2017-01-23 23:38:31 +08:00
Sergey M․ 6d119c2a6b
[24video] Fix extraction (closes #11811) 2017-01-23 03:50:39 +07:00
Sergey M․ 4201ba13e6
[youtube:playlist] Fix nonexistent/private playlist detection and skip private tests 2017-01-23 02:49:56 +07:00
Grzegorz P 8bc0800d7c [youtube:playlist] Fix nonexistent/private playlist detection (closes #11604) 2017-01-23 02:35:38 +07:00
Alex Seiler a089545e03 [azmedien:show] Improve _VALID_URL 2017-01-23 02:30:29 +07:00
Gaetan Gilbert 30dda24de3 [chirbit] Extract uploader 2017-01-23 02:27:38 +07:00
Sergey M․ 271808b6b2
[pornflip] Improve and extract dash formats (closes #11795) 2017-01-22 03:43:27 +07:00
einstein95 8d1fbe0cb2
[pornflip] Add extractor (closes #11556) 2017-01-22 03:41:59 +07:00
Sergey M․ a243abb80d
[chaturbate] Improve (closes #11797) 2017-01-22 03:02:48 +07:00
einstein95 42697bab3c
[chaturbate] Fix extraction 2017-01-22 02:58:40 +07:00
Sergey M․ 94629e537f
[azmedien] Improve (closes #11784) 2017-01-22 02:17:39 +07:00
Alex Seiler e84495cd8d
[azmedien] Add extractor (closes #11785) 2017-01-22 02:17:39 +07:00
Yen Chi Hsuan 7c20b7484c
[nextmedia] Support redirected URLs 2017-01-22 02:06:34 +08:00
ha shao 04a3d4d234
[vimeo:channel] Extract videos' titles for playlist entries 2017-01-21 23:37:44 +07:00
Sergey M․ 12afdc2ad6
[youtube] Extract episode metadata (closes #9695, closes #11774) 2017-01-21 18:10:32 +07:00
Yen Chi Hsuan f3c21cb7a7
[cspan] Fix _TESTS 2017-01-20 22:27:13 +08:00
Yen Chi Hsuan 972efe60c3
[generic] Remove a dead test
The web page does not contain a video anymore

Ref: #2694, #2696
2017-01-20 22:27:13 +08:00
Yen Chi Hsuan 4447fb2332
[cspan] Support Ustream embedded videos
Closes #11547
2017-01-20 22:27:13 +08:00
Yen Chi Hsuan d77ac73790
[ustream] Add UstreamIE._extract_url()
Ref: #11547
2017-01-20 22:27:13 +08:00
Sergey M․ 1fe84be0f3
[1tv] Add support for hls (closes #11786) 2017-01-20 00:47:04 +07:00
james mike dupont cccd70a275 untie 2017-01-19 04:18:13 -05:00
Sergey M․ eb3f008c9e
[uol] Fix extraction (closes #11770) 2017-01-19 04:49:31 +07:00
Sergey M․ f1e70fc2ff
[mtv] Relax triforce feed regex (closes #11766) 2017-01-18 23:34:11 +07:00
Remita Amine baa3e1845b [bilibili] fix extraction(closes #11077) 2017-01-18 17:00:15 +01:00
Sergey M․ aaf2b7c57a
[canalplus] Add fallback for video id (closes #11764) 2017-01-18 22:20:11 +07:00
Sergey M․ b687c85eab
[extractor/generic] Add support for 20 minuten embeds (closes #11683, closes #11751) 2017-01-18 22:17:47 +07:00
Sergey M․ 538b17a09c
[20min] Improve 2017-01-18 22:17:41 +07:00
Alex Seiler 4e44598547
[20min] Fix extraction 2017-01-18 22:17:34 +07:00
Sergey M․ 136078966b
[imdb] Extend _VALID_URL (closes #11744) 2017-01-17 23:14:07 +07:00
Sergey M․ 8a5f0a6357
[naver] Update tests for #11743 2017-01-17 21:19:57 +07:00
Kagami Hiiragi c0bd51c090 [naver] Support tv.naver.com links 2017-01-17 21:15:50 +07:00
Sergey M․ 79fc8496c6
[xiami] Improve extraction (closes #11699)
* Relax _VALID_URLs
* Improve track metadata extraction
2017-01-16 23:31:50 +07:00
Sergey M․ 906420cae3
[limelight] Improve and make more robust (closes #11737)
+ Add support for direct http for videos hosted on video.llnw.net
* Check handmade http URLs
2017-01-16 21:54:47 +07:00
Yen Chi Hsuan 16e2c8f771
[brightcove] Recognize another player ID
Closes #11688
2017-01-16 00:06:52 +08:00
Yen Chi Hsuan dcae7b3fdc
[niconico] Allow login via cookies
Some codes are borrowed from #7968, which is by @jlhg

Closes #7968
2017-01-15 22:51:54 +08:00
Yen Chi Hsuan 8e4988f1a2
[niconico] Remove codes for downloading anonymously
Apparently Niconico now blocks playing without an account

Closes #11170
2017-01-15 22:10:57 +08:00
Sergey M․ a7acf868a5
[yourupload] Fix extraction (closes #11601) 2017-01-15 10:34:39 +07:00
Sergey M․ af62de104f
[beam:live] Improve and simplify (#10702, closes #11596) 2017-01-15 06:07:35 +07:00
sh!zeeg cd55c6ccd7
[beam:live] Add extractor 2017-01-15 06:06:10 +07:00
Sergey M․ 621a2800ca
[vevo] Improve geo restriction detection 2017-01-15 04:42:05 +07:00
Sergey M․ b80e2ebc8d
[dramafever] Add support for URLs with language code (#11714) 2017-01-14 18:27:22 +07:00
Remita Amine 99d537a5e0 [ooyala] fix typo 2017-01-14 07:12:50 +01:00
Sergey M․ abe8cb763f
[cbc] Improve playlist support (closes #11704) 2017-01-14 08:30:00 +07:00
Jakub Wilk 4f66c16f33 [brightcove:legacy] Fix misplaced backslash in a regexp 2017-01-14 06:26:11 +07:00
Sergey M․ e54fc0524e
[cmt] Add support for video-clips 2017-01-14 06:23:24 +07:00
Sergey M․ adf063dad1
[mtv,cc,cmt,spike] Improve and refactor
- Eliminate _transform_rtmp_url
* Generalize triforce mgid extraction
+ [cmt] Add support for full-episodes (closes #11623)
2017-01-14 06:18:38 +07:00
Remita Amine 5e8eebb600 [mitele] extract dash formats 2017-01-13 23:06:59 +01:00