eeb0a95684
[extractor/common] Add 'preference' to _parse_html5_media_entries
...
Some websites, like NJPWorld, put different qualities on different
player pages.
2017-02-25 18:40:05 +08:00
eea0716cae
[extractor/common] Print origin country for fake IP
2017-02-21 23:14:33 +07:00
336a76551b
[extractor/common] Do not quit _initialize_geo_bypass on empty countries
2017-02-21 23:09:41 +07:00
dc0a869e5e
[extractor/common] Fix typo
2017-02-21 23:05:31 +07:00
e39b5d4ab8
[extractor/common] Allow calling _initialize_geo_bypass from extractors ( #11970 )
2017-02-21 23:00:43 +07:00
3ccdde8cb7
[extractor/common] Emphasize geo bypass APIs are experimental
2017-02-20 23:21:15 +07:00
4248dad92b
Improve geo bypass mechanism
...
* Rename options to preffixly match with --geo-verification-proxy
* Introduce _GEO_COUNTRIES for extractors
* Implement faking IP right away for sites with known geo restriction
2017-02-19 05:10:08 +08:00
0a840f584c
Rename bypass geo restriction options
2017-02-19 05:10:08 +08:00
0016b84e16
Add faked X-Forwarded-For to formats' HTTP headers
2017-02-19 05:10:08 +08:00
773f291dcb
Add experimental geo restriction bypass mechanism
...
Based on faking X-Forwarded-For HTTP header
2017-02-19 05:10:08 +08:00
a4a554a793
[generic] Try parsing JWPlayer embedded videos ( closes #12030 )
2017-02-16 23:44:03 +08:00
08a00eef79
[extractor/common] skip m3u8 manifests protected with Adobe Flash Access
2017-02-10 17:00:09 +01:00
242a14a1f6
[extractor/common] Fix audio only with audio group in m3u8 ( closes #11995 )
2017-02-07 00:22:16 +07:00
2aec7256ae
[extractor/common] Speed-up media tags regex ( closes #11979 )
2017-02-06 00:20:30 +07:00
45024183ae
[infoq] Add audio only format if available ( #11565 )
...
* [infoq] Add audio only format if available
Refactor cookie code into a function.
Renamed formats to http_video, http_audio, rtmp_video
Renamed extract functions to video instead of videos as they return
one or no video.
* [infoq] Rename to _extract_cookies as it more than one
* [infoq] Remove redundant determine_ext
* [infoq] Add comment about hardcoded URL
* [infoq] Use _hidden_inputs instead of messy regex
* [infoq] Probe if audio URL is valid
Make it possible to pass headers to _is_valid_url
* [infoq] Add audio only test
2017-02-03 12:10:13 +08:00
e228616c6e
[extractor/common] Fix initialization template ( closes #11605 , closes #11825 )
2017-01-29 06:57:39 +07:00
c58c2d63cb
[extractor/common] Document forgotten fragment base and path interfaces
2017-01-29 05:57:53 +07:00
d04621daf4
[extractor/common] Fix duration per dash segment ( closes #11868 )
2017-01-29 05:36:53 +07:00
c4251b9aaa
[common] add possibility to customize akamai manifest host
2017-01-13 10:21:36 +01:00
4606c34e19
[extractor/common] Allow non-lang in subtitles' keys
...
See 264e77c406
2016-12-25 01:50:50 +08:00
a07588369f
[common] improve detection for video only formats and m3u8 manifest( fixes #11507 )
2016-12-22 10:02:56 +01:00
7fe1592073
[common] fix dash codec information for mixed videos and fragment url construction( #11490 )
2016-12-20 12:35:03 +01:00
87a449c1ed
[extractor/common] Recognize DASH formats in html5 media entries
2016-12-17 23:03:13 +07:00
8821a718cf
[common] recognize hls manifests that contain video only formats( #11394 )
2016-12-10 17:22:15 +01:00
4ea4c0bb22
[extractor/common] Fix Bandwidth substitution in media template ( closes #11175 )
2016-11-13 05:43:34 +07:00
f076d7972c
[extractor/common] Improve thumbnail extraction from JSON-LD
2016-11-12 23:01:05 +07:00
c4c9b8440c
[extractor/common] Tolerate malformed RESOLUTION attribute in m3u8 manifests ( closes #11113 )
2016-11-04 05:02:31 +07:00
3365ea8929
[extractor/common] Remove unused code
2016-11-02 02:34:23 +07:00
1616f9b452
[extractor/common] Fix typo
2016-11-02 02:30:25 +07:00
02dc0a36b7
[utils] Introduce base_url
2016-11-02 02:30:18 +07:00
b2758123c5
add Basic support for Smooth Streaming protocol( #8118 )
2016-11-02 01:54:45 +07:00
b82c33dd67
[extractor/common] Improve mpd base URL extraction ( closes #10909 , closes #11079 )
2016-11-01 01:15:46 +07:00
0384932e3d
[extractor/common] try to extract non smil wowza mpd manifests
2016-10-19 14:57:12 +01:00
edd6074cea
[extractor/common] detect f4m audio only formats
2016-10-19 14:42:48 +01:00
8d76bdf12b
[extractor/common] Mention podcast in series fields section
2016-10-16 18:37:17 +07:00
cea364f70c
[extractor/common] Support HTML media elements without child nodes
2016-10-12 01:40:28 +08:00
98763ee354
[extractor/common] Add id and title helpers for generic IEs
2016-10-07 19:20:53 +08:00
e71a450956
[common] add hdcore sign to akamai f4m formats
2016-09-24 21:55:53 +01:00
5968d7d2fe
[extractor/common] Improved support for HTML5 subtitles
...
Ref: #10625
In a strict sense, <track>s with kind=captions are not subtitles. [1]
openload misuses this attribute, and I guess there will be more
examples, so I add it to common.py.
Also allow extracting information for subtitles-only <video> or <audio>
tags, which is the case of openload.
[1] https://www.w3.org/TR/html5/embedded-content-0.html#attr-track-kind
2016-09-24 14:20:42 +08:00
30d0b549be
[extractor/common] Add manifest_url for hls and hds formats
2016-09-17 21:33:38 +07:00
86f4d14f81
Refactor fragments interface and dash segments downloader
...
- Eliminate segment_urls and initialization_url
+ Introduce manifest_url (manifest may contain unfragmented data in this case url will be used for direct media URL and manifest_url for manifest itself correspondingly)
* Rewrite dashsegments downloader to use fragments data
* Improve generic mpd extraction
2016-09-17 20:35:22 +07:00
b4c1d6e800
[extractor/common] Expose fragments interface for dashsegments formats
2016-09-17 18:31:18 +07:00
a0d5077c8d
[extractor/common] Introduce fragments interface
2016-09-17 18:31:09 +07:00
6ad0219556
[common] add helper method for Wowza Streaming Engine format extraction
2016-09-16 19:30:38 +01:00
dcce092e0a
[extractor/common] Simplify _get_netrc_login_info and carry long lines
2016-09-15 22:35:12 +07:00
32443dd346
[extractor/common] Update _get_login_info's comment
2016-09-15 22:34:29 +07:00
2133565cec
[extractor/common] Simplify _get_login_info
2016-09-15 22:26:37 +07:00
c849836854
[utils] Improve _hidden_inputs
2016-09-15 21:54:48 +07:00
1b6712ab23
[adobepass] add specific options for adobe pass authentication
...
- add --ap-username and --ap-password option to specify
TV provider username and password in the cmd line
- add --ap-retries option to limit the number of retries
- add --list-ap-msi-ids to list the supported TV Providers
2016-09-13 22:16:01 +01:00
0def758782
[internetvideoarchive] extract all formats
2016-09-04 11:45:29 +01:00
dedb177029
Fix parsing of HTML5 media elements
...
This fixes an error in _parse_html5_media_entries in case
an audio or video tag directly uses a src attribute insted
of <source> elements in it's body.
2016-09-03 16:09:35 +07:00
e816c9d158
[extractor/common] Simplify _extract_m3u8_formats
2016-09-01 22:18:16 +07:00
9250181f37
[extractor/common] Restore NAME usage from EXT-X-MEDIA tag for formats codes in _extract_m3u8_formats ( Closes #10522 )
2016-09-01 21:37:25 +07:00
f39ffc5877
[common] extract formats from #EXT-X-MEDIA tags
2016-08-25 08:49:44 +01:00
c7c43a93ba
[common] add helper method to extract akamai m3u8 and f4m formats
2016-08-22 07:49:34 +01:00
ad120ae1c5
[extractor/common] Change the default m3u8 protocol in HTML5
...
Helper functions should have consistent default values
2016-08-22 02:26:07 +08:00
520251c093
[extractor/common] Recognize m3u8 manifests in HTML5 multimedia tags
2016-08-19 23:53:47 +08:00
2118fdd1a9
[common] add separate method for getting netrc ligin info
2016-08-14 11:55:28 +01:00
321b5e082a
[extractor/common] Respect default in _search_json_ld
2016-08-08 22:36:18 +07:00
37768f9242
[common] correctly lower the preference of m3u8 master manifest format
2016-08-07 10:59:09 +01:00
d16b3c6677
[common] extract partOfTVSeries info in json-ld
2016-08-06 18:58:38 +01:00
c57244cdb1
[common] lower the preference of m3u8 master manifest format
2016-08-06 18:55:05 +01:00
46933a15d6
[extractor/common] Support root JSON-LD lists ( Closes #10203 )
2016-08-05 23:14:32 +07:00
8b40854529
[common] lower proto_preference of rtsp formats
...
Most of the time the RtspFD fail to download videos but it report
success of the download with this output:
[mpv] 0 bytes
[download] 100% of 0.00B
2016-08-04 09:39:06 +01:00
b070564efb
[extractor/common] Support multiple properties in _og_search_property
2016-08-02 22:55:14 +07:00
40090e8d51
[extractor/common] Improve is_suitable
...
In order to fix breakage introduced by a3aa814b77
2016-07-26 23:54:06 +07:00
f094834857
[extractor/common] Add support for $ in SegmentTemplate in MPD manifests
2016-07-24 10:27:16 +07:00
00f4764cb7
[common] extract vbr, abr and fps for Unified Streaming Platform m3u8 manifests
2016-07-13 15:58:24 +01:00
0b68de3cc1
Merge pull request #8876 from remitamine/html5_media
...
[extractor/common] add helper method to extract html5 media entries
2016-07-10 23:40:45 +08:00
95b31e266b
[extractor/common] Add expected_type in json ld routines
2016-07-09 03:28:04 +07:00
6b3a3098b5
[extractor/common] Extract more metadata for VideoObject in _json_ld
2016-07-09 03:27:11 +07:00
2de624fdd5
[extractor/common] Introduce filesize metafield for thumbnails
2016-07-09 03:24:36 +07:00
3fee7f636c
[flipagram] Add extractor
2016-07-09 03:23:32 +07:00
38cce791c7
Rename --cn-verfication-proxy to --geo-verification-proxy
...
And deprecate the former one
Since commit f138873900
2016-07-03 23:29:56 +08:00
bf8dd79045
[extractor/common] Fix sorting with custom field preference
2016-06-26 21:09:07 +07:00
59bbe4911a
[extractor/common] add helper method to extract html5 media entries
2016-06-26 14:04:08 +01:00
88d9f6c0c4
[utils] Add support for name list in _html_search_meta
2016-06-26 16:57:14 +07:00
e154c65128
[downloader/hls] Add support for AES-128 encrypted segments in hlsnative downloader
2016-06-19 01:01:40 +01:00
a6571f1073
[common] Fix <bootstrapInfo> detection in F4M manifests
...
Regression since 0a5685b26f
2016-06-08 00:19:33 +08:00
77b8b4e696
[extractor/common] Borrow quality metadata from parent set-level manifest for f4m
2016-05-27 01:47:44 +06:00
fac2af3c51
[common] Fix m3u8 extraction in f4m manifests
2016-05-27 01:41:27 +08:00
448bb5f333
[common] Fix non-bootstrapped support in f4m
2016-05-27 00:03:48 +08:00
240b60453e
[common] Support m3u8 in f4m manifests
...
Related: #9531
2016-05-26 21:55:43 +08:00
0a5685b26f
[common] Support non-bootstraped streams in f4m manifests
...
Related: #9531
2016-05-26 21:41:47 +08:00
7b2fcbfd4e
[common] Skip TYPE=CLOSED-CAPTIONS lines in m3u8 manifests
...
According to [1], valid values for TYPE are AUDIO, VIDEO, SUBTITLES
and CLOSED-CAPTIONS. Such a value is found in Anvato master playlists,
though I don't use _extract_m3u8_formats() in the end.
Part of #9522 .
[1] https://tools.ietf.org/html/draft-pantos-http-live-streaming-19#section-4.3.4.1
2016-05-21 13:16:28 +08:00
16da9bbc29
[common] Add _m3u8_meta_format() template
...
For extractors who handle m3u8 manifests by themselves. (eg., AnvatoIE)
Part of #9522
2016-05-21 13:15:28 +08:00
ad96b4c8f5
[common] Extract audio formats in SMIL
...
Found in http://www.cbc.ca/player/play/2657631896
Closes #5156
2016-05-20 19:02:53 +08:00
ed56f26039
[extractor/common] Improve name extraction for m3u8 formats
2016-05-15 03:34:35 +06:00
8a92e51c60
[extractor/common] Relax wording for creator metafield
2016-05-02 21:31:35 +06:00
e9c6cdf4a1
[common] Fix format_id construction for HLS
2016-04-29 22:50:16 +08:00
b24d6336a7
[vlive] Add support for live videos
2016-04-29 14:22:50 +03:00
d6712378e7
Merge branch 'akamai_pv' of https://github.com/remitamine/youtube-dl into remitamine-akamai_pv
2016-04-25 21:02:02 +08:00
fb72ec58ae
[extractor/common] do not process f4m manifest that contain akamai playerVerificationChallenge
2016-04-25 13:37:03 +01:00
2c0d9c6217
[extractor/common] Allow empty post data
2016-04-21 13:06:06 +08:00
49caf3307f
[extractor/common] Remove irrelevant comment
2016-04-10 17:10:27 +06:00
bacec0397f
[extractor/common] Relax _hidden_inputs
2016-04-08 23:33:45 +06:00
fb38aa8b53
[extractor/common] Support arbitrary format strings for template based identifiers in mpd manifests ( Closes #9119 , closes #9120 )
2016-04-08 22:48:08 +06:00
7a93ab5f3f
[extractor/common] Introduce music album metafields
2016-04-07 02:53:53 +06:00
b507cc925b
[extractor/common] Carry long line
2016-04-02 18:49:58 +06:00
db8ee7ec05
[extractor/common] Fix numeric identifiers conversion in DASH URL templates
2016-04-02 18:48:05 +06:00
df634be2ed
[common] prefer using mime type over ext for smil subtitle extraction
...
the subtitle ext for http://www.cnet.com/videos/download-amazon-prime-movies-and-tv/
is adb_xml while using the mime type it get tt(application/smptett+xml)
2016-04-01 19:47:49 +01:00
41d06b0424
[extractor/common] Improve _request_webpage
...
* Do not ignore data, headers and query for Requests
* Default values for headers and query switched to dicts since these are used by urllib itself
2016-03-31 22:58:38 +06:00
b22ca76204
[extractor/common] Filter out unsupported encrypted media for f4m formats ( Closes #8573 )
2016-03-27 07:42:38 +06:00
19dbaeece3
Remove _sort_formats from _extract_*_formats methods
...
Now _sort_formats should be called explicitly.
_sort_formats has been added to all the necessary places in code.
Closes #8051
2016-03-27 07:03:08 +06:00
15707c7e02
[compat] Add compat_urllib_parse_urlencode and eliminate encode_dict
...
encode_dict functionality has been improved and moved directly into compat_urllib_parse_urlencode
All occurrences of compat_urllib_parse.urlencode throughout the codebase have been replaced by compat_urllib_parse_urlencode
Closes #8974
2016-03-26 01:46:57 +06:00
49dea4913b
Merge pull request #8513 from remitamine/dash-sort
...
[extractor/common] fix dash formats sorting
2016-03-15 18:39:50 +01:00
0fdbb3322b
[extractor/common] Add _parse_f4m_formats routine
2016-03-13 03:16:08 +06:00
09f572fbc0
[extractor/common] add transform_source to _download_smil and _extract_smil_formats
2016-03-11 22:37:07 +01:00
15bf934de5
Merge pull request #8819 from remitamine/simple-webpage-requests
...
[extractor/common] simplify using data, headers and query params with _download_* methods
2016-03-11 18:19:43 +01:00
cdfee16818
[extractor/common] add data, headers and query params to _request_webpage
2016-03-11 18:12:50 +01:00
a6c8b75904
[common] Use mimeType to determine file extensions ( #8766 )
2016-03-11 23:51:42 +08:00
64f08d4ff2
Merge pull request #8766 from yan12125/dash-detect-ext
...
Detect file extensions of DASH formats from their codecs
2016-03-11 21:40:07 +08:00
af7d5a63b2
[common] Document protocol http_dash_segments
2016-03-06 17:47:07 +08:00
2def60c5f3
[common] Use codec2ext for DASH formats ( #8764 )
2016-03-05 18:18:39 +08:00
e9c0cdd389
[jython] Introduce compat_os_name
...
os.name is always 'java' on Jython
2016-03-03 19:24:24 +08:00
7bcd2830dd
[extractor/common] Document uploader_url
2016-03-02 23:31:24 +06:00
2bc0c46f98
[extractor/common] Document license metafield
2016-03-02 23:06:39 +06:00
d77ab8e255
Add --mark-watched feature ( Closes #5054 )
2016-03-01 01:01:33 +06:00
9cdffeeb3f
[extractor/common] Clarify rationale on media playlist detection
2016-02-27 07:01:11 +06:00
fbb6edd298
[extractor/common] Properly extract audio only formats in master m3u8 playlists
2016-02-27 06:48:13 +06:00
f5bdb44443
[extractor/common] Add _remove_duplicate_formats
2016-02-22 01:19:39 +06:00
cafcf657a4
add more subtitles mime types to mimetype2ext and fix the platform subtitle extraction
2016-02-20 22:02:03 +01:00
611c1dd96e
[refactor] Single quotes consistency
2016-02-14 15:37:17 +06:00
d800609c62
[refactor] Do not specify redundant None as second argument in dict.get()
2016-02-14 14:25:04 +06:00
bb20526b64
[extractor/common] Improve base url construction
2016-02-13 00:13:56 +06:00
c349456ef6
[extractor/common] strip http urls in smil manifest
2016-02-12 17:38:48 +01:00
81e1c4e2fc
[extractor/common] remove duplicate rtmp formats in smil manifest
2016-02-11 17:58:48 +01:00
dd86780596
[extractor/common] fix dash formats sorting
2016-02-11 10:55:50 +01:00
154c209e2d
[extractor/common] improve dash format ids
2016-02-11 10:33:26 +01:00
51e9094f4a
[extractor/common] extract youtube dash formats filesize( fixes #8480 )
2016-02-09 20:05:39 +01:00
d413095f7e
[extractor/common] remove duplicated formats and subtiles in smil manifests
2016-02-09 17:15:41 +01:00
6a3828fddd
[common] use float conversion instead of using division from __future__
2016-02-06 14:27:04 +01:00
91cb6b5065
rename _parse_mpd to _parse_mpd_formats and add default value for mpd namespace
2016-02-06 14:03:48 +01:00
0826a0b555
[common] sort dash formats
2016-02-06 06:52:48 +01:00
255732f0d3
[common] fix segment duration calculation
2016-02-03 23:57:08 +01:00
53c269c6fd
[common] fix media_template string formating
2016-02-03 23:54:34 +01:00
675d001633
[common] skip drm protected dash formats
2016-02-03 18:44:43 +01:00
d577c79632
[common] ignore ISO 639-2 generic codes
2016-02-03 13:24:07 +01:00
f14be22816
[common] remove duplicate reference to namespace
2016-02-02 22:02:08 +01:00
9c74423510
[common] fix media template regex
2016-02-02 18:30:31 +01:00
1bac34556f
[common] add a generic support for mpd manifests
2016-02-02 18:09:25 +01:00
2d2fa82d17
[common] Add _extract_dash_manifest_formats
2016-01-30 22:52:23 +08:00
c94678957f
[common] Remove unused arguments
2016-01-30 22:45:16 +08:00
16f38a699f
[common] Rename to namespace
...
For consistency with _parse_smil_*
2016-01-30 22:40:56 +08:00
df374b5222
[common] Prefer the manifest than formats_dict in determining codecs
2016-01-30 21:42:27 +08:00
5ea1eb78f5
[common] Fix for youtube
2016-01-30 21:36:01 +08:00
b323e1707d
[common] Modify _parse_dash_manifest for use in Facebook
2016-01-30 21:27:43 +08:00
17b598d30c
[common] _parse_dash_manifest() from youtube.py
2016-01-30 21:05:55 +08:00
350cf045d8
[extractor/common] Restrict checks when auto calculating tbr
2016-01-30 01:47:46 +06:00
a9d5f12fec
Merge pull request #8328 from remitamine/hls-master-detect
...
[extractor/common] detect media playlist in _extract_m3u8_formats
2016-01-27 18:07:30 +01:00
Yen Chi Hsuan
Sergey M․
Remita Amine
Mattias Wadman
Christian Pointner
Déstin Reed
Kagami Hiiragi