Commit Graph

10922 Commits

Author SHA1 Message Date
Purdea Andrei dfc8f46e1c
[vimeo:channel] Add video id to url_result
This will allow us to decide much faster that we don't want an already archived video,
and will allow having to download webpages for each video that has already been downloaded,
thus significantly speeding up the archival of channels that have no new content.
2016-06-23 01:26:27 +07:00
Sergey M․ c143ddce5d
[vimeo] Override original URL only when necessary 2016-06-23 00:51:36 +07:00
TRox1972 6ae938b295
[Vine] Extract view count 2016-06-22 23:57:35 +07:00
Sergey M․ cf40fdf5c1
release 2016.06.22 2016-06-22 23:43:24 +07:00
Sergey M․ 23bdae0955
[svt] Various improvements
+ [svt:play] Add fallback path looking for video id and fix extraction for oppetarkiv
* [svt:base] Detect geo restriction
* [svt:base] Extract series related metadata
2016-06-22 23:36:07 +07:00
Shai Coleman ca74c90bf5 Fix issue downloading facebook videos
youtube-dl expects the format items to be returned as a list,
but when there's only one item Facebook returns a dict instead,
this wraps the dict in a list if necessary
2016-06-22 12:52:15 +01:00
Sergey M․ 7cfc1e2a10
[gametrailers] Remove extractor
gametrailers closed (see http://www.polygon.com/2016/2/8/10944452/gametrailers-shuts-down-after-13-year-run)
2016-06-21 22:31:41 +07:00
Remita Amine 1ac5705f62 [gamespot] extract all formats 2016-06-21 13:37:57 +01:00
Yen Chi Hsuan e4f90ea0a7
[svt] Fix extraction for SVTPlay (closes #9809) 2016-06-21 17:55:53 +08:00
Sergey M․ cdfc187cd5
[cbs] Remove unused import 2016-06-20 22:40:33 +07:00
Sergey M․ feef925f49
[streamcloud] Capture error message (#9840) 2016-06-20 22:40:22 +07:00
Sergey M․ 19e2d1cdea
release 2016.06.20 2016-06-20 20:50:01 +07:00
Sergey M․ 8369a4fe76
[downloader/hls] Simplify and carry long lines 2016-06-20 21:55:17 +07:00
Philipp Hagemeister 1f749b6658 Revert "[jsinterp] Avoid double key lookup for setting new key"
This reverts commit 7c05097633.
2016-06-20 13:29:13 +02:00
Remita Amine 819707920a [cbs] fix _VALID_URL 2016-06-19 23:55:19 +01:00
Remita Amine 43518503a6 [cbs,cbsnews,cbssports] reduce requests while extracting all formats 2016-06-19 23:40:00 +01:00
Remita Amine 5839d556e4 [theplatform] reduce requests for theplatform feed info extraction 2016-06-19 23:37:05 +01:00
Yen Chi Hsuan 6c83e583b3
[radiojavan] PEP8
E275 is added in pycodestyle 2.6

See https://github.com/PyCQA/pycodestyle/pull/491
2016-06-19 13:32:08 +08:00
Yen Chi Hsuan 6aeb64b673 Merge pull request #8201 from remitamine/hls-aes
[downloader/hls] Add support for AES-128 encrypted segments in hlsnative downloader
2016-06-19 13:25:08 +08:00
Remita Amine 6cd64b6806 [foxsports] extract http formats 2016-06-19 05:45:48 +01:00
remitamine e154c65128 [downloader/hls] Add support for AES-128 encrypted segments in hlsnative downloader 2016-06-19 01:01:40 +01:00
Sergey M․ a50fd6e026
release 2016.06.19.1 2016-06-19 03:57:14 +07:00
Sergey M․ 6a55bb66ee
[vimeo] Fix rented videos (Closes #9830) 2016-06-19 03:56:01 +07:00
Lucas Moura 7c05097633
[jsinterp] Avoid double key lookup for setting new key
In order to add a new key to both __objects and __functions dicts on jsinterp.py, it is
necessary to first verify if a key was present and if not, create the key and
assign it to a value.

However, this can be done with a single step using dict setdefault method.
2016-06-19 03:29:45 +07:00
Sergey M․ 589568789f
release 2016.06.19 2016-06-19 02:30:29 +07:00
Sergey M․ 7577d849a6
[r7] Fix extraction and add support for articles (Closes #9826) 2016-06-19 02:25:34 +07:00
Sergey M․ cb23192bc4
[closertotruth] Update and improve (Closes #8680) 2016-06-19 00:35:29 +07:00
Steven Gosseling 41c1023300
[closertotruth] Add extractor
Removed print statement from code.

Replaced two regex searches with the corret ones.

Removed some unnecessary semicolumns

fixed title extraction

refactored everything to search_regex

processed comments on commit 5650b0d, fixed feedback from flake8

Improved regexes and returns info dict now.

Added support for closertotruth interview URL

Added support for episodes page
2016-06-18 23:19:56 +07:00
Sergey M․ 90b6288cce
[arte:+7] Simplify _VALID_URL 2016-06-18 22:23:48 +07:00
Sergey M․ d7c6c656c5
[arte:+7] Expand _VALID_URL (Closes #9820) 2016-06-18 21:42:17 +07:00
Yen Chi Hsuan b0b128049a
[extractors] Update references to sportschau (#9799) 2016-06-18 13:43:47 +08:00
Yen Chi Hsuan e8f13f2637
[sportschau.de] Fix extraction and moved to its own file (closes #9799) 2016-06-18 13:42:58 +08:00
Yen Chi Hsuan b5aad37f6b
[ard] Remove SportschauIE, which is now based on WDR (#9799) 2016-06-18 13:42:39 +08:00
Yen Chi Hsuan 6d0d4fc26d
[wdr] Add WDRBaseIE, for Sportschau (#9799) 2016-06-18 13:40:55 +08:00
Yen Chi Hsuan 0278aa443f
[br] Skip invalid tests 2016-06-18 12:53:48 +08:00
Yen Chi Hsuan 1f35745758
[azubu] Don't fail on optional fields 2016-06-18 12:39:08 +08:00
Yen Chi Hsuan 573c35272f
[bbc] Skip a geo-restricted test case 2016-06-18 12:35:55 +08:00
Yen Chi Hsuan 09e3f91e40
[arte] Update _TESTS and fix for pages with multiple YouTube videos
Some tests are from #6895 and #6613
2016-06-18 12:34:58 +08:00
Yen Chi Hsuan 1b6cf16be7
[aftonbladet] Fix extraction 2016-06-18 12:27:39 +08:00
Yen Chi Hsuan 26264cb056
[adobetv] Use embedded data in the webpage
Sometimes the HTML webpage is returned even with '?format=json'
2016-06-18 12:21:40 +08:00
Yen Chi Hsuan a72df5f36f
[mtvservices] Fix ext for RTMP streams 2016-06-18 12:19:06 +08:00
Yen Chi Hsuan c878e635de
[bet] Moved to MTVServices 2016-06-18 12:17:24 +08:00
Sergey M․ 0f47cc2e92
release 2016.06.18.1 2016-06-18 06:20:34 +07:00
Sergey M․ 5fc2757682
release 2016.06.18 2016-06-18 06:00:05 +07:00
Sergey M․ e3944c2621
[pornhd] Add working test 2016-06-18 05:50:17 +07:00
Sergey M․ 667d96480b
[pornhd] Detect removed videos and modernize 2016-06-18 05:42:20 +07:00
Sergey M․ e6fe993c31
[pornhd] Improve formats extraction 2016-06-18 05:37:53 +07:00
Sergey M․ d0d93f76ea
[pornhd] Fix metadata extraction 2016-06-18 05:30:46 +07:00
Sergey M․ 20a6a154fe
[mtv] Use compat_xpath and fix FutureWarning 2016-06-18 04:46:26 +07:00
Sergey M․ f011876076
[nickde] Add extractor (Closes #9778) 2016-06-18 04:40:48 +07:00
Sergey M․ 6929569403
[mitele] Extract series metadata and make title more robust (Closes #9758) 2016-06-18 04:06:19 +07:00
Sergey M․ eb451890da
[carambatv] Add extractor (Closes #9815) 2016-06-18 03:04:14 +07:00
Sergey M․ ded7511a70
[bbccouk] Add support for playlists (Closes #9812) 2016-06-17 23:42:52 +07:00
Sergey M․ d2161cade5
release 2016.06.16 2016-06-16 22:40:55 +07:00
Sergey M․ 27e5fa8198
[cda] Fix extraction (Closes #9803) 2016-06-16 22:33:12 +07:00
Yen Chi Hsuan efbd1eb51a
[wimp] Fix extraction and update _TESTS 2016-06-16 12:27:21 +08:00
Yen Chi Hsuan 369ff75081
[jwplatform] Improved JWPlayer support 2016-06-16 12:26:45 +08:00
Yen Chi Hsuan 47212f7bcb
[utils] Don't transform numbers not starting with a zero
Fix test_Viidea and maybe others
2016-06-16 11:00:54 +08:00
Sergey M․ 4c93ee8d14
[imdb] Improve _VALID_URL (Closes #9788) 2016-06-15 22:34:55 +07:00
Yen Chi Hsuan 8bc4dbb1af
[wrzuta.pl] Detect error and update _TESTS 2016-06-14 11:14:59 +08:00
Sergey M․ 6c3760292c
[pornhub] Improve title extraction (Closes #9777) 2016-06-14 04:57:59 +07:00
Sergey M․ ff4af6ec59
[lynda] Remove superfluous _NETRC_MACHINE 2016-06-14 02:49:33 +07:00
Sergey M․ d01fb21d4c release 2016.06.14 2016-06-14 02:19:42 +07:00
Sergey M․ 1759672eed
[wrzuta:playlist] Improve and simplify (Closes #9341) 2016-06-14 02:13:54 +07:00
venth fea55ef4a9
[wrzuta.pl:playlist] Added playlist extraction from wrzuta.pl 2016-06-14 02:10:48 +07:00
Sergey M․ 16b6bd01d2
[rockstargames] Improve and add Youtube fallback (Closes #9737) 2016-06-14 01:11:24 +07:00
Dracony 14d0f4e0f3
Added extractor for rockstargames.com 2016-06-14 01:09:35 +07:00
Sergey M․ 778f969447
[twitch:clips] Add extractor (Closes #9767) 2016-06-14 00:06:31 +07:00
Sergey M․ 33b72ce64e
[xfileshare] Improve removed videos detection 2016-06-13 01:19:54 +07:00
Sergey M․ cf2bf840ba
[xfileshare] Fix test 2016-06-13 01:11:14 +07:00
Sergey M․ bccdac6874
[xfileshare:xvidstage] Add support for videos with packed codes (Closes #4335) 2016-06-13 01:11:04 +07:00
Sergey M․ e69f9f5d68
[downloader/external] Decode error string before writing to stderr 2016-06-12 16:45:07 +07:00
Sergey M․ 77a9a9c295 release 2016.06.12 2016-06-12 12:06:48 +07:00
Sergey M․ 84dcd1c4e4
[streamcloud] Detect removed videos (Closes #3768) 2016-06-12 11:08:39 +07:00
Sergey M․ 971e3b7520
[nrk:skole] Fix extraction 2016-06-12 07:20:37 +07:00
Sergey M․ 4e79011729
[nrktv] Fix tests 2016-06-12 06:57:04 +07:00
Sergey M․ 98960c911c
[instagram] Extract metadata from JSON 2016-06-12 06:06:04 +07:00
Sergey M․ 329ca3bef6
[utils] Add try_get
To reduce boilerplate when accessing JSON
2016-06-12 06:05:34 +07:00
Sergey M․ 2c3322e36e
[youporn] Fix metadata extraction 2016-06-12 04:49:37 +07:00
Sergey M․ 80ae228b34
[matchtv] Modernize 2016-06-12 01:57:23 +07:00
Yen Chi Hsuan 6d28c408cf
[viki] Do not use a fallback language for title in the first try
In test_Viki_3, 'titles' gives a Hebrew title.
2016-06-11 23:00:44 +08:00
Yen Chi Hsuan c83b35d4aa
[viki] Update _TESTS 2016-06-11 22:39:13 +08:00
Yen Chi Hsuan 94e5d6aedb
[viki] Skip a geo-restricted test 2016-06-11 21:49:01 +08:00
Yen Chi Hsuan 531a74968c
[vimeo] Fix extraction for VimeoReview videos 2016-06-11 21:35:08 +08:00
Yen Chi Hsuan c5edd147d1
[generic] Remove an invalid test
Now handled by telewebion.py
2016-06-11 18:39:58 +08:00
Yen Chi Hsuan 856150d056
[telewebion] Add new extractor (closes #5135) 2016-06-11 18:39:58 +08:00
Paul Henning 15d106787e [utils] Change Firefox 44 to 47
See commit title.
2016-06-11 05:36:31 -04:00
Yen Chi Hsuan 7aab3696dd
[kuwo] Update _TESTS 2016-06-11 15:37:04 +08:00
Yen Chi Hsuan 47787efa2b
[leeco] Recognize Le Sports URLs (fixes #9750) 2016-06-11 13:14:41 +08:00
Sergey M․ 4a420119a6 release 2016.06.11.3 2016-06-11 08:34:30 +07:00
Sergey M․ 33751818d3 release 2016.06.11.2 2016-06-11 08:28:51 +07:00
Sergey M․ fe458b6596
[limelight] Extract ttml subtitles (Closes #9739) 2016-06-11 05:57:27 +07:00
Sergey M․ 21ac1a8ac3
[limelight] Fix typo 2016-06-11 05:52:50 +07:00
Sergey M․ 79027c0ea0
[limelight] Improve _VALID_URLs 2016-06-11 05:40:02 +07:00
Sergey M․ 4cad2929cd
[limelight] Fix _VALID_URLs 2016-06-11 05:30:44 +07:00
Sergey M․ 62666af99f
[indavideo] Fix formats' height (Closes #9744) 2016-06-11 05:13:05 +07:00
Sergey M․ 6626c214e1 release 2016.06.11.1 2016-06-11 03:00:08 +07:00
Sergey M․ d845622b2e release 2016.06.11 2016-06-11 02:41:48 +07:00
Sergey M 1058f56e96 Merge pull request #9747 from TRox1972/lynda
[Lynda] Extract course description
2016-06-11 02:34:58 +07:00
TRox1972 0434358823 [Lynda] Extract course description 2016-06-10 19:17:58 +02:00
Sergey M․ 3841256c2c
[lynda] Skip login if already logged in 2016-06-10 23:01:52 +07:00
Sergey M․ bdf16f8140
[lynda] Add support for new authentication (Closes #9740) 2016-06-10 22:40:18 +07:00
Yen Chi Hsuan 836ab0c554
[compat] Import html5 entities correctly 2016-06-10 18:12:57 +08:00
Yen Chi Hsuan 6c0376fe4f
[dw] Skip an invalid test
DW documentaries only last for one or two weeks. See #9475
2016-06-10 16:53:40 +08:00
Yen Chi Hsuan 1fa309da40
[generic] Update test_Generic_40
The original link now redirects to an YouTube user channel.
2016-06-10 16:39:31 +08:00
Yen Chi Hsuan daa0df9e8b
[youtube:user] Support another URL form
Such an URL comes from http://www.gametrailers.com/. This is originally
a test case in GenericIE, but now seems all GameTrailers videos are on
YouTube.
2016-06-10 16:37:12 +08:00
Yen Chi Hsuan 09728d5fbc
[audiomack:album] Force video_id to be strings
Related: be6217b261
2016-06-10 16:11:28 +08:00
Yen Chi Hsuan c16f8a4659
[voicerepublic] Force video_id to be strings
Related: be6217b261
2016-06-10 16:04:28 +08:00
Yen Chi Hsuan a225238530
[vporn] Improve error detection and update _TESTS 2016-06-10 15:12:53 +08:00
Yen Chi Hsuan 55b2f099c0
[utils] Decode HTML5 entities
Used in test_Vporn_1. Also related to #9270
2016-06-10 15:11:55 +08:00
Yen Chi Hsuan 9631a94fb5
[compat] Add compat_html_entities_html5
Used in tset_Vporn_1. Also Related to #9270
2016-06-10 15:05:24 +08:00
Yen Chi Hsuan cc4444662c [generic] Remove Vulture embed detection
Vulture.com videos now hosts on YouTube, Vimeo, MTV, NBC News or Hulu.
Here's an example of Hulu:
http://www.vulture.com/2016/06/kimmel-interviews-mariah-carey-in-a-bathtub.html
2016-06-10 13:40:57 +08:00
Yen Chi Hsuan de3eb07ed6
[generic] Detect NBC News embeds 2016-06-10 13:32:59 +08:00
Yen Chi Hsuan 5de008e8c3
[nbcnews] Support embed widgets
Used in some Vulture videos
2016-06-10 13:31:55 +08:00
Yen Chi Hsuan 3e74b444e7
[vulture] Remove the extractor
The first 10 URLs in google search "site:http://video.vulture.com/video"
is dead. I guess Vulture does not host videos on their own anymore.
2016-06-10 13:13:59 +08:00
Yen Chi Hsuan e1e0a10c56
[weibo] Remove the extractor
The Weibo weishipin (微視頻, tiny videos) service is dead and now all
videos are hosted on Sina videos, which is covered by sina.py
2016-06-10 13:01:22 +08:00
Yen Chi Hsuan 436214baf7
[xfileshare] Skip an invalid test 2016-06-10 12:31:06 +08:00
Yen Chi Hsuan 506d0e9693
[xuite] Skip the invalid test 2016-06-10 12:29:58 +08:00
Yen Chi Hsuan 55290788d3 [yahoo] Yahoo doesn't like region names in lower cases
Fix test_Yahoo_7
2016-06-10 12:28:56 +08:00
Yen Chi Hsuan bc7e7adf51 [wdr] Subtitles are TTML 2016-06-10 00:22:41 +08:00
Sergey M․ b0aebe702c
[godtv] Relax _VALID_URL 2016-06-09 21:34:47 +07:00
Sergey M․ 416878f41f
[godtv] Add more tests 2016-06-09 21:33:51 +07:00
Sergey M․ c0fed3bda5
[godtv] Improve and add support for playlists (Closes #9608) 2016-06-09 21:29:41 +07:00
TRox1972 bb1e44cc8e
[godtv] Add extractor
[GodTV] Improvements
2016-06-09 21:27:27 +07:00
N1k145 21efee5f8b
[openload] Relax _VALID_URL
[openload] added to _TESTS, removed escape
2016-06-09 20:46:54 +07:00
Yen Chi Hsuan e2713d32f4
[openload] Fix extraction. Thanks @perron375 for the solution
Closes #9706
2016-06-09 19:00:13 +08:00
Yen Chi Hsuan e21c26daf9 Merge pull request #9395 from pmrowla/afreecatv
[afreecatv] Add new extractor for afreecatv.com VODs
2016-06-09 17:20:16 +08:00
Yen Chi Hsuan 1594a4932f
[wdr] Misc changes 2016-06-09 13:49:35 +08:00
Yen Chi Hsuan 6869d634c6
[wdr] Simplify extraction 2016-06-09 13:41:12 +08:00
Yen Chi Hsuan 50918c4ee0
[wdr] Support radio players (closes #6147) 2016-06-09 13:04:30 +08:00
Yen Chi Hsuan 6c33d24b46
[utils] Add audio/mpeg to mimetype2ext()
Used in WDR live radios (#6147)
2016-06-09 12:58:24 +08:00
Sergey M․ be6217b261
[YoutubeDL] Force string conversion on non string video ids 2016-06-09 05:34:19 +07:00
Sergey M․ 9d51a0a9a1
[vessel] Make hls formats non fatal 2016-06-09 04:13:38 +07:00
Sergey M․ 39da509f67
[vessel] Extract DASH formats 2016-06-09 04:12:48 +07:00
Sergey M․ a479b8f687
[vessel] Use native hls by default 2016-06-09 04:09:32 +07:00
Sergey M․ 48a5eabc48
[extractor/generic] Add support vessel embeds (Closes #7083) 2016-06-09 04:02:27 +07:00
Sergey M․ 11380753b5
[vessel] Add support for embed urls and improve extraction 2016-06-09 04:00:47 +07:00
Yen Chi Hsuan 411c590a1f
[youku:show] Add new extractor 2016-06-08 23:45:46 +08:00
Yen Chi Hsuan 6da8d7de69
[twitter] Update _TESTS 2016-06-08 21:48:12 +08:00
Yen Chi Hsuan c6308b3153
[twitter] Fix extraction for videos with HLS streams
Closes #9623
2016-06-08 21:28:10 +08:00
Yen Chi Hsuan fc0a45fa41
[twitter] Detect suspended accounts and update _TESTS 2016-06-08 21:12:14 +08:00
Yen Chi Hsuan e6e90515db
[nbc] Add the test case from #9578
Closes #9578
2016-06-08 20:50:01 +08:00
Yen Chi Hsuan 22a0a95247
[theplatform] Some NBC videos require an additional cookie
Related: #9578
2016-06-08 20:47:39 +08:00
Yen Chi Hsuan 50ce1c331c
[downloader/external] Add another env for proxies in ffmpeg/avconv
Related sources:
https://git.libav.org/?p=libav.git;a=blob;f=libavformat/http.c;h=8fe8d11e1edfdbb04a8726db2c49cfef3f572aac;hb=HEAD#l152
https://git.libav.org/?p=libav.git;a=blob;f=libavformat/tls.c;h=fab243e93e20034e88e619188c13a44a5d8ccdb9;hb=HEAD#l63
https://github.com/FFmpeg/FFmpeg/blob/f8e89d8/libavformat/http.c#L191
https://github.com/FFmpeg/FFmpeg/blob/f8e89d8/libavformat/tls.c#L92
2016-06-08 14:43:52 +08:00
Yen Chi Hsuan 7264e38591
[bilibili] Fix for videos without upload time (closes #9710) 2016-06-08 14:31:40 +08:00
Sergey M․ 33d9f3707c
[thesixtyone] Relax _VALID_URL (Closes #9714) 2016-06-08 02:22:04 +07:00
Sergey M․ a26a9d6239
[livestream:event] Ensure video id is string (Closes #9721) 2016-06-07 23:53:08 +07:00
Yen Chi Hsuan a4a8201c02
[wdr] Update _TESTS 2016-06-08 00:25:51 +08:00
Yen Chi Hsuan a6571f1073
[common] Fix <bootstrapInfo> detection in F4M manifests
Regression since 0a5685b26f
2016-06-08 00:19:33 +08:00
Sergey M․ 57b6e9652e
[canal+] Add support for d17.tv 2016-06-07 22:32:08 +07:00