Commit Graph

832 Commits

Author SHA1 Message Date
Sergey M․ c2d125d99f
[youtube] Improve metadata extraction for age gate content (closes #21943) 2019-07-31 00:14:33 +07:00
Remita Amine 8dbf751aa2 [youtube] improve title and description extraction(closes #21934) 2019-07-30 00:13:33 +01:00
Sergey M․ 13a75688a5
[youtube] Fix some tests 2019-07-21 00:01:46 +07:00
Sergey M․ 2e18adec98
[youtube:playlist] Relax _VIDEO_RE (closes #21844) 2019-07-20 23:46:34 +07:00
geditorit 791d2e8117 [youtube] Add support for invidious.mastodon.host (#21777) 2019-07-15 22:54:22 +07:00
Sergey M․ 898238e9f8
[youtube] Restrict is_live extraction (closes #21782) 2019-07-14 20:30:05 +07:00
geditorit ba036333bf [youtube] Add more invidious instances to _VALID_URL (#21694) 2019-07-14 01:23:22 +07:00
Sergey M․ 27019dbb4b
[youtube] Fix is_live extraction (closes #21734) 2019-07-12 03:45:58 +07:00
Sergey M․ baf67a604d
[youtube] Fix authentication (closes #11270) 2019-07-12 02:26:05 +07:00
Sergey M․ 31ce6e9966
[youtube] Add another signature function pattern 2019-06-22 02:22:41 +07:00
Sergey M․ abefc03f51
[youtube] Update signature function patterns (closes #21469, closes #21476) 2019-06-21 22:58:42 +07:00
Sergey M․ c40714cdee
[youtube] Make --write-annotations non fatal (closes #21452) 2019-06-20 00:57:58 +07:00
Sergey M․ b85eae0f05
[youtube] Hardcode codec metadata for av01 video only formats (closes #21381) 2019-06-14 01:59:05 +07:00
Sergey M․ bf3c932663
[youtube] Fix average rating extraction (closes #21304) 2019-06-07 23:13:33 +07:00
Remita Amine 0d29751890 [youtube] improve DRM protected videos detection(#1774) 2019-05-25 23:15:59 +01:00
Sergey M․ 3089bc748c
Fix W504 and disable W503 (closes #20863) 2019-05-11 03:57:40 +07:00
Sergey M․ 027ffdca0d
[youtube] Use sp field value for signature field name (closes #18841, closes #18927, closes #21028) 2019-05-10 08:38:05 +07:00
pypy b45a9e698e [youtube] Fix channel id extraction (closes #20982) (#21003) 2019-05-09 00:58:47 +07:00
Sergey M․ f8c55c6664
[youtube:entrylistbase] Retry on 5xx HTTP errors (#20965) 2019-05-06 01:25:16 +07:00
Sergey M․ a61ce71468
[youtube] Remove info el for get_video_info request
Since it does not work for quite a long time
2019-04-30 04:49:12 +07:00
Sergey M․ 026fbedc85
[youtube] Improve extraction robustness
Fail on missing token only when no formats found
2019-04-30 04:32:55 +07:00
Remita Amine 92bc97d398 [youtube] extract album from Music in this video section(#20301) 2019-04-28 17:38:20 +01:00
Remita Amine 822b9d9cb0 [youtube] improve Youtube Music Auto-generated description parsing(closes #20742) 2019-04-27 09:16:17 +01:00
quinlander 5caabd3c70 [youtube] Extract additional meta data from video description on youtube music videos
YouTube music videos often have auto-generated video descriptions that can be
utilized to extract additional information about the video. This is desirable
in order to provide the user with as much meta data as possible. This commit
adds extraction methods for the following fields for youtube music videos:
- artist (fallback extraction methods added)
- track (fallback extraction methods added)
- album (new in this commit)
- release_date (new in this commit)
- release_year (new in this commit)

4 tests have been added to test this new functionality:
- YoutubeIE tests 27, 28, 29, and 30

Resolves: #20599
2019-04-27 09:09:54 +01:00
Sergey M․ 56667d622c
[youtube] Fix extraction (closes #20758, closes #20759, closes #20761, closes #20762, closes #20764, closes #20766, closes #20767, closes #20769, closes #20771, closes #20768, closes #20770) 2019-04-24 09:58:00 +07:00
Remita Amine 15be3eb5e5 [youtube] raise ExtractorError when no data available(#20737) 2019-04-22 20:52:43 +01:00
Remita Amine 19041a3877 [youtube] extract srv[1-3] subtitle formats(#20566) 2019-04-05 16:18:57 +01:00
utlasidyo 77d95677b7 [youtube] Add support for invidiou.sh (#20309) 2019-03-17 07:15:15 +07:00
Sergey M․ 067aa17edf
Start moving to ytdl-org 2019-03-11 04:00:54 +07:00
dimqua 8ae113ca9d [youtube] Add more invidious instances
See [Invidious-Instances](https://github.com/omarroth/invidious/wiki/Invidious-Instances) for the reference.
2019-03-03 08:19:36 +07:00
Sergey M․ fa4ac365f6
[youtube] Extend JS player signature function name regexes (closes #18890, closes #18891, closes #18893) 2019-01-17 10:24:44 +07:00
Sergey M․ 4fe54c128a
[youtube] Update tests and add a tests for #18804 2019-01-16 02:18:27 +07:00
Sergey M․ 2f483bc1c3
[youtube] Skip unsupported adaptive stream type (#18804) 2019-01-16 01:28:50 +07:00
Sergey M․ 561b456e2d
[youtube] Extract DASH formats from player response (closes #18804) 2019-01-16 01:12:58 +07:00
Sergey M․ c3e543893b
[youtube] Extract live HLS URL from player response (closes #18799) 2019-01-10 22:46:53 +07:00
Sergey M․ c2dd2dc086
[youtube] Unescape HTML for series (closes #18641) 2018-12-31 19:57:01 +07:00
Remita Amine 140a13f5de [youtube] extract more format metadata 2018-12-26 10:56:10 +01:00
Remita Amine 825cd268a3 [youtube] detect DRM protected videos(#1774) 2018-12-26 10:56:10 +01:00
Sergey M․ 63529e935c
[youtube] Relax html5 player regexes (closes #18465, closes #18466) 2018-12-23 16:57:10 +07:00
Sergey M․ 4273caf5c7
[youtube] Extend html5 player regex (closes #17516) 2018-12-23 16:40:16 +07:00
Sergey M․ 21c340b83f
[youtube] Fix mark watched (closes #18546) 2018-12-16 19:40:49 +07:00
Sergey M․ 8fe104947d
[youtube] Fix multifeed extraction (closes #18531) 2018-12-15 22:25:12 +07:00
Sergey M․ 2511eee215
[youtube] Add another JS signature function name regex (closes #18091, closes #18093, closes #18094) 2018-11-07 09:55:59 +07:00
Sergey M․ c0345b825f
[youtube:playlist] Add support for invidio.us (closes #18077) 2018-11-05 19:08:39 +07:00
Sergey M․ dbdaaa231a
[youtube] Add fallback metadata extraction from videoDetails (closes #18052) 2018-11-03 06:26:16 +07:00
Sergey M․ cd5a74a28e
[youtube] Add support for invidio.us (closes #17613) 2018-09-24 00:14:49 +07:00
Sergey M․ c11485162b
[youtube] Don't pollute default query dict (closes #17593) 2018-09-17 22:13:39 +07:00
Sergey M․ dd4c449219
[youtube] Extract channel meta fields (closes #9676, closes #12939) 2018-09-15 01:24:26 +07:00
Sergey M․ 9a47fa35dd
[youtube] Fix extraction (closes #17457, closes #17464) 2018-09-08 03:36:10 +07:00
Sergey M․ 409b9324da
[youtube:playlist] Add support for music album playlists (OLAK5uy_ prefix) (closes #17361) 2018-08-28 02:20:32 +07:00
Sergey M․ 9a6628aaf9
[youtube] Improve login error handling (closes #13822) 2018-07-05 00:37:32 +07:00
Sergey M․ eb6793ba97
[youtube] Update tests 2018-06-03 02:23:45 +07:00
Sergey M․ 7e72694b5e
[youtube] Move metadata extraction after video availability check 2018-06-03 02:08:38 +07:00
Sergey M․ 936784b272
[youtube] Extract track and artist 2018-06-03 02:05:14 +07:00
Remita Amine 68217024e8 remove unnecessary assignment parenthesis 2018-05-26 16:13:54 +01:00
Remita Amine 54fc90aabf [youtube] fix hd720 format position 2018-05-16 16:24:44 +01:00
Sergey M․ c63ca0eef8
[youtube] Improve format filesize extraction (#16453) 2018-05-14 23:27:56 +07:00
Sergey M․ 84a9fef899
[youtube] Make uploader extraction non fatal (#16444) 2018-05-13 22:49:01 +07:00
Remita Amine 4c76aa0666 [youtube] fix extraction for embed restricted live streams(fixes #16433) 2018-05-13 13:20:16 +01:00
Niklas Haas 30226342ab [youtube] Correctly disable polymer on all requests
Rather than just the one that use the _download_webpage helper. The need
for this was made apparent by 0fe7783e, which refactored
_download_json in a way that completely avoids the use of
_download_webpage, thus breaking youtube.

Fixes #16323
2018-04-29 22:35:16 +07:00
Sergey M․ 70d35d166c
[youtube] Add ability to authenticate with cookies 2018-04-22 06:08:05 +07:00
Sergey M․ 3853309fe2
[youtube:feed] Implement lazy playlist extraction (closes #10184) 2018-04-22 06:07:32 +07:00
Sergey M․ 870f3bfc63
[ytsearch] Fix flat title extraction (closes #11260, closes #15681) 2018-02-23 03:43:42 +07:00
Sergey M․ ea69624992
[youtube] Handle shared URLs with generic extractor (closes #14303) 2018-02-15 22:33:11 +07:00
Sergey M․ e4a60912b8
[youtube] Enforce using chunked HTTP downloading for DASH formats 2018-02-04 07:17:26 +07:00
Yen Chi Hsuan bbb7c3f7e9 [youtube] Extract precise error messages (closes #15284) 2018-01-25 22:30:33 +08:00
Sergey M․ e7f3529f68
[youtube:live] Improve live detection (closes #15365) 2018-01-20 17:57:20 +07:00
Sergey M․ 391dd6f094
[youtube] Fix live streams extraction (closes #15202) 2018-01-14 00:03:22 +07:00
Sergey M․ 07aeced68e
[youtube] Extract uploader, uploader_id and uploader_url for playlists (#11427, #15018) 2017-12-19 03:51:28 +07:00
Sergey M․ f2332f18e6
[youtube] Restrict embed regex (#14600) 2017-10-27 22:26:43 +07:00
Remita Amine fa4bc6e712 [youtube] replace youtube redirect urls in description(fixes #14517) 2017-10-17 10:07:37 +00:00
Sergey M․ 685e87b61f
[youtube] Fix relative URLs in description 2017-10-14 20:26:52 +07:00
Khang Nguyen 04af3aca04 Remove YoutubeSharedVideoIE https://github.com/rg3/youtube-dl/issues/14303 2017-10-11 02:01:18 +08:00
Sergey M․ 197224b7a4
Fix some regexes 2017-10-09 23:50:53 +07:00
Sergey M․ 8b561bfc9d
[youtube] Add support for hooktube.com (closes #14437) 2017-10-07 21:59:04 +07:00
Sergey M․ 66c9fa36c1
[youtube] Separate methods for embeds extraction 2017-09-06 00:48:37 +07:00
Sergey M․ 8d81f3e36d
[youtube] Force old layout for each webpage (closes #14083) 2017-09-02 00:58:19 +07:00
Sergey M․ 7998520933
[youtube] Fix upload date extraction (closes #14065) 2017-08-31 00:47:58 +07:00
Sergey M․ 1c9c8de29e
[youtube] Fix extraction with --youtube-skip-dash-manifest enabled (closes #14037) 2017-08-27 06:06:39 +07:00
Sergey M․ c7121fa7b8
[youtube] Fix controversy videos extraction (closes #14027, closes #14029) 2017-08-26 15:38:38 +07:00
Sergey M․ ea3f20494f
[youtube] PEP 8 2017-07-15 07:02:57 +07:00
Sergey M․ 7d02dcfaa2
[youtube] Don't capture YouTube Red ad for creator meta field (closes #13621) 2017-07-14 22:37:04 +07:00
Sergey M․ ddbb4c5c3e
[youtube] Adapt to new automatic captions rendition (closes #13467) 2017-06-23 02:00:19 +07:00
Sergey M․ 39d4c1be4d
[youtube] Improve chapters extraction (closes #13247) 2017-06-01 23:29:45 +07:00
Yen Chi Hsuan a49eccdfa7
[youtube] Parse player_url if format URLs are encrypted or DASH MPDs are requested
Fixes #13211
2017-05-28 20:20:20 +08:00
Yen Chi Hsuan fd545fc6d1
Revert "[youtube] Don't use the DASH manifest from 'get_video_info' if 'use_cipher_signature' is True (#5118)"
This reverts commit 87dc451108.
2017-05-26 23:22:54 +08:00
Sergey M․ dc4e4f90a2
[youtube] Modernize 2017-05-21 01:18:56 +07:00
Protuhj fdc20f87a6 [youtube] Fix DASH manifest signature decryption (closes #8944) 2017-05-21 01:11:37 +07:00
Sergey M․ 3995d37da5
[youtube] Fix TFA (#12927) 2017-05-07 04:19:11 +07:00
Sergey M․ e00eb564e9
[youtube] Fix authentication (closes #12927) 2017-05-06 23:58:47 +07:00
Sergey M․ 9cafc3fd8b
[youtube] Extract chapters 2017-05-06 02:27:06 +07:00
Yen Chi Hsuan b62985a9a5
[youtube] Recognize another HTML5 player URL (#12885) 2017-04-28 16:25:04 +08:00
Yen Chi Hsuan e31fed95b4
[youtube] Recognize new locale-based player URLs (fixes #12885) 2017-04-28 15:48:30 +08:00
Remita Amine 89beedd31f [youtube] improve m3u8 format extraction 2017-04-11 21:55:59 +01:00
Sergey M․ d23028a8fb
[youtube] Remove explicit preference for audio-only and video-only formats
In order not to break sorting when new formats appear
2017-04-11 22:41:48 +07:00
Sergey M․ d0ba55871e
[youtube] Improve _VALID_URLs (closes #12538) 2017-03-25 01:18:33 +07:00
Yen Chi Hsuan c9612c0487
[youtube] Mark errors about rental videos as expected
Closes #12324
2017-03-02 16:59:53 +08:00
Sergey M․ 87dadd456a
[youtube:playlist] Recognize another playlist pattern (closes #11928, closes #12286) 2017-02-28 23:06:47 +07:00
Sergey M․ 0e879f432a
[youtube:channel] Remove duplicate test 2017-02-27 22:22:43 +07:00
Sergey M․ fd5c4aab59
[youtube] Raise GeoRestrictedError 2017-02-26 16:52:40 +07:00
Sergey M․ 6310acf512
[youtube] Fix parsing codecs (closes #12091) 2017-02-12 18:09:53 +07:00
Remita Amine 605fd6392f [youtube] add format info for itag 325 and 328 2017-02-03 17:59:48 +01:00
Yen Chi Hsuan a22b2fd19b
[youtube] Fix ytsearch* when cookies are provided
Closes #11924

The API with `page` is no longer used in browsers, and YouTube always
returns {'reload': 'now'} when cookies are provided.

See http://youtube.github.io/spfjs/documentation/start/ for how SPF
works. Basically appending static link with a `spf` parameter yields the
corresponding dynamic link.
2017-02-03 01:28:24 +08:00
Sergey M․ a685751051
[youtube:playlist] Recognize TL playlists (closes #11945) 2017-02-02 22:01:11 +07:00
Sergey M․ 3c90cc8b6f
[youtube] Fix extraction for domainless player URLs
Closes #11890
Closes #11891
Closes #11892
Closes #11894
Closes #11895
Closes #11897
Closes #11900
Closes #11903
Closes #11904
Closes #11906
Closes #11907
Closes #11909
Closes #11913
Closes #11914
Closes #11915
Closes #11916
Closes #11917
Closes #11918
Closes #11919
2017-01-31 22:22:37 +07:00
Sergey M․ 556dbe7fe3
[youtube] Add fallback for duration extraction (closes #11841) 2017-01-26 21:43:14 +07:00
Sergey M․ 4201ba13e6
[youtube:playlist] Fix nonexistent/private playlist detection and skip private tests 2017-01-23 02:49:56 +07:00
Grzegorz P 8bc0800d7c [youtube:playlist] Fix nonexistent/private playlist detection (closes #11604) 2017-01-23 02:35:38 +07:00
Sergey M․ 12afdc2ad6
[youtube] Extract episode metadata (closes #9695, closes #11774) 2017-01-21 18:10:32 +07:00
Sergey M․ 022a5d663b
[youtube] Add test for itag 212 (#11575) 2017-01-09 22:30:46 +07:00
Kacper Michajłow 8409b3683c [youtube] Add itag 212
Seen on video with id 1t24XAntNCY
2017-01-09 22:29:03 +07:00
Yen Chi Hsuan ec85ded83c
Fix "invalid escape sequences" error on Python 3.6 2017-01-06 00:58:56 +08:00
Sergey M․ c867adc68c
[youtube:playlist] Pass disable_polymer in query (closes #11193, closes #11270) 2016-11-23 23:28:32 +07:00
Sergey M․ 073d5bf583
[youtube:live] Relax _VALID_URL (closes #11164) 2016-11-16 23:15:19 +07:00
Sergey M․ 481cc7335c
[youtube] Fix --no-playlist behavior for youtu.be/id URLs (closes #10896) 2016-10-19 03:27:18 +07:00
Sergey M․ c1b2a0858c
[youtube:live] Extend _VALID_URL (Closes #10839) 2016-10-04 02:10:23 +07:00
stepshal 2d3d29976b
[youtube] Change test URLs from http to https 2016-09-25 17:45:24 +07:00
stepshal 925194022c
Improve some _VALID_URLs 2016-09-14 22:47:21 +07:00
stepshal 25042f7372 Add missing r prefix for _VALID_URLs 2016-09-08 17:04:57 +07:00
Sergey M․ feaa5ad787
[youtube:playlist] Extend _VALID_URL 2016-09-04 20:12:34 +07:00
Sergey M․ 7e5dc339de
[youtube:watchlater] Fix extraction (Closes #10544) 2016-09-04 00:29:01 +07:00
Sergey M․ 6496ccb413
[youtube] Add support for rental videos' previews (Closes #10532) 2016-09-03 01:17:15 +07:00
Sergey M․ dacb3a864a
[youtube:playlist] Fallback to video extraction for video/playlist URLs when playlist is broken (Closes #10537) 2016-09-02 23:43:20 +07:00
Sergey M․ 05bddcc512
[youtube] Fix authentication (2) (Closes #10392) 2016-08-24 01:29:50 +07:00
Sergey M․ 1212e9972f
[youtube] Fix authentication (#10392) 2016-08-24 00:25:21 +07:00
Sergey M․ e298d3a08c
[youtube] Fix authentication (Closes #10140) 2016-07-22 21:05:39 +07:00
Sergey M․ fd8c8c7dcd
[youtube:shared] Relax _VALID_URL 2016-07-21 22:58:34 +07:00
Sergey M․ 7f29cf545a
[youtube] Add YouTube Red paid video reference test (#10059) 2016-07-12 02:10:35 +07:00
Sergey M․ e9fb6a4bbe
[youtube] Relax TFA regexes 2016-07-11 03:08:38 +07:00
Sergey M․ 9558dcec9c
[youtube:user] Preserve user/c path segment 2016-07-09 08:37:19 +07:00
Sergey M․ 73c4ac2c95
[youtube:channel] Improve channel id extraction and detect unavailable channels (Closes #10009) 2016-07-05 23:30:44 +07:00
Yen Chi Hsuan 4080530624
[youtube:shared] Recognize the new 'shared' URLs
Closes #10007
2016-07-05 13:15:05 +08:00
Sergey M․ be49068d65
[youtube] Fix and skip some tests 2016-06-24 22:47:19 +07:00
Sergey M․ 525cedb971
[youtube] Relax URL expansion in description 2016-06-24 22:37:13 +07:00
Sergey M․ de3c7fe0d4
[youtube] Fix 141 format tests 2016-06-24 22:27:55 +07:00
Yen Chi Hsuan daa0df9e8b
[youtube:user] Support another URL form
Such an URL comes from http://www.gametrailers.com/. This is originally
a test case in GenericIE, but now seems all GameTrailers videos are on
YouTube.
2016-06-10 16:37:12 +08:00
Sergey M․ 2c34735267
[youtube] Add itags 256 and 258 2016-06-05 01:44:13 +07:00
Sergey M․ 25cb7a0eeb
[youtube] Allow empty attribute values in description regex 2016-05-06 22:11:18 +06:00
Sergey M․ 175c2e9ec3
[youtube:search_url] Reimplement in terms of youtube:playlistbase 2016-04-26 22:29:29 +06:00
Sergey M․ 494ab6db73
[youtube] Capture and output login error message 2016-04-20 22:14:32 +06:00
Jaime Marquínez Ferrándiz 1b6182d8f7 [youtube:playlist] Fetch all the videos in a mix (fixes #3837)
Since there doesn't seem to be any indication, it stops when there aren't new videos in the webpage.
2016-04-17 17:07:57 +02:00
remitamine f3a58d46bf [youtube:user] check if the url didn't match only the other youtube extractors 2016-04-15 19:06:13 +01:00
Sergey M․ 466a614537
[youtube:playlist] Recognize popular uploads playlist as mix (Closes #9170) 2016-04-12 21:38:31 +06:00
Sergey M․ 6d4fc66bfc [youtube] Add support for zwearz (Closes #9062) 2016-04-04 02:26:20 +06:00
Sergey M․ 6dee688e6d [youtube:playlistsbase] Restrict playlist regex (Closes #8986) 2016-03-26 20:42:18 +06:00
Sergey M․ 6e6bc8dae5 Use urlencode_postdata across the codebase 2016-03-26 02:19:24 +06:00
Sergey M․ 15707c7e02 [compat] Add compat_urllib_parse_urlencode and eliminate encode_dict
encode_dict functionality has been improved and moved directly into compat_urllib_parse_urlencode
All occurrences of compat_urllib_parse.urlencode throughout the codebase have been replaced by compat_urllib_parse_urlencode

Closes #8974
2016-03-26 01:46:57 +06:00
Sergey M․ f07e276a04 [youtube:live] Add extractor (Closes #8959) 2016-03-25 01:18:14 +06:00
Yen Chi Hsuan 11f12195af [youtube] Added itag 91
Seen in https://www.youtube.com/watch?v=jMN4cxyhJjk
2016-03-17 19:25:37 +08:00
Sergey M․ fd050249af [youtube] Extract uploader_url (Closes #8724) 2016-03-02 23:49:10 +06:00