This PR adds two new config option, to pass a PO token (config 'po_token') and
a visitor ID (config 'visitor_data') to Youtube. These two strings are required
to play videos using the WEB client.
Warning: These strings gives much more identifiable information to Google!
If the po_token setting is filled in, then the WEB client is used. If not, the
Android client is used. TvHtml5ScreenEmbed will still be used as a fallback.
Script for generating po_token and visitor_data:
https://github.com/iv-org/youtube-trusted-session-generator
Helps with issue 4734
This PR adds support for inv_sig_helper, which offloads the player fetching,
function extraction and signature parsing, which in turn allows to use the
web client to watch videos.
When the new config option "signature_server" is not set, the logic for the
external signature server is not enabled and invidious behaves like before.
This PR also updates the crystal overrides because the stdlib changed quite
a while ago (See issue 11049 at crystal-lang/crystal) and those were required
to properly use TCP/unix sockets.
Closes issue 4649
Note: Does not add rel="noreferrer noopener" to:
* links in channel description
* links in video descriptions
* links in video comments
Related to issue 4267
This PR adds support for parsing the newer channel header format
(banner + subscription parsing)
Before this change:
* 0 subscribers
* No banner image
After this change:
* Example with Mr Breast channel: 299M
* Image banner is visible
Closes issue 4783
Before this PR, Invidious assumed that every playlist had at least one video.
When a playlist had no videos, Invidious was throwing an "Index out of bounds"
exception.
The following API endpoints were impacted:
* api/v1/playlists/:plid
* api/v1/auth/playlists/:plid
Fixes issue 4679
Videos of a playlist cataloged as podcast are called "episodes" therefore
Invidious was not able to find video in the text value inside the stats array.
Test case: "/playlist?list=PLDu-Eh5lUs1a4irCbnxMIB6FrUMaTXgVF"
Fixes issue 4688
This pull request fixes that bug that was causing the query parameters to get
doubled in the streaming URLs when '?local=true' is passed to the
'/api/v1/videos/{id}' API endpoint.
Before: host/path?parameters?parameters
After: host/path?parameters
No associated open issue
At the moment Invidious will return hardcoded data for the 'size',
'qualityLabel' and 'fps' fields for streams, when such hardcoded data is
available, otherwise it just omits those fields from the response (e.g. with
the AV1 formats). Those issues are especially noticable when Invidious claims
that 50fps streams have 60fps and when it claims that the dimensions for a
vertical video are landscape. The DASH manifests that Invidious generates
already use the correct information.
This pull request corrects that issue by returning the information that
YouTube provides instead of hardcoded values and also fixes the long
standing bug of Invidious claiming that audio streams have 30 fps.
Here are two test cases:
50/25/13fps: https://youtu.be/GbXYZwUigCM (/api/v1/videos/GbXYZwUigCM)
vertical video: https://youtu.be/hxQwWEOOyU8 (/api/v1/videos/hxQwWEOOyU8)
Originally these problems were going to be solved by the complete refactor
of stream handling in 3620, but as that pull request got closed by the stale
bot over a month ago and has such a massive scope that it would require a
massive amount of work to complete it, I decided to open this pull request
that takes a less radical approach of just fixing bugs instead of a full
on refactoring.
FreeTube generates it's own DASH manifests instead of using Invidious' one,
so that it can support multiple audio tracks and HDR. Unfortunately due to
the missing and inaccurate information in the API responses, FreeTube has
to request the DASH manifest from Invidious to extract the height, width and
fps. With this pull request FreeTube could rely just on the API response,
saving that extra request to the Invidious instance. It would also make it
possible for FreeTube to use the vp9 streams with Invidious, which would
reduce the load on the video proxies.
Closes issue 4131
Before this PR, setting the modified code repo URL through the preferences
page in Invidious was broken:
* the HTML input tag for this field had invalid type "input"
(though browser falls back on text input)
* the URL was used to set the "checked" property and not as a plain value,
which makes no sense for a text-based input (and resulted in a blank field)
* when the submitted field is empty, the retrieved value was an empty 'String'
instead of 'nil', causing the "modified source code URL" to be an empty
'href' link which just pointed to the current page
No associated open issue
When visiting /api/v1/popular and popular endpoint is disabled
Before:
500 {"error":"Closed stream"}
After
403 {"error":"Administrator has disabled this endpoint."}
The current Content Security Policy does not allow to embed videos
inside local HTML files which are viewed in the browser via the file
protocol. This commit adds the file protocol to the allowed frame
ancestors, so that the embedded videos load correctly in local HTML
files.
This behaviour is consistent which how the official YouTube website
allows to embed videos from itself.
Closes issue 4448
Some opengraph implementations don't support a URL without the domain
therefore failing to fetch the video thumbnail and channel image.
This pull request basically fixes that.
The transcript logic in Invidious was written specifically
as a workaround for captions, and not transcripts as a feature.
This commit genericises the logic a bit as so it can be used for
implementing transcripts within Invidious' API and UI as well.
The most notable change is the added parsing of section headings
when it was previously skipped over in favor of regular lines.
The HTTP::Client created via `make_client` is affected by the
force_resolve configuration option. However, api.invidious.io
does not support ipv6 and as such any request with ipv6 to
api.invidious.io will instead raise.
Directly calling the HTTP::Client will ignore the force_resolve option
allowing requests to go through ipv4 when needed.
Invidious assumes that every playlist will have at least one video
because it needs to check for the `index` key. So if there is no videos
on a playlist, there is no `index` key and Invidious throws
`Index out of bounds`
Videos of a playlist cataloged as podcast are called episodes therefore
Invidious was not able to find `video` in the `text` value inside the
stats array.
Trending music was returning less than 24 videos which was getting
filtered out. Since multiple categories only appears on the default
trending page, we don't need to do that filtering for Music, Gaming or movies.
Closes issue 4596
The new comment format is similar to the description's commandRuns.
This should fix the issues with most comments but there are still
some more changes that would need to be made like adding support for
formatting (bold, italic, underline) and channel emojis.
Fixes issue 4566
Returns the 'isPostLiveDvr' field in the videos API when the video
is a post-live DVR (= ended livestream that hasn't been reprocessed
into VOD yet).
Example taken 10 minutes after that livestream ended:
/api/v1/videos/euqnWk-uP6M
{
...
"isPostLiveDvr": true,
...
}
Partially fixes 4421
YouTube added a new format for like data in December 2023.
This PR adds support for parsing saod new format.
Related: LuanRT/YouTube.js issue 557
Closes 4419
Before this PR, Invidious assumed that fetching the DASH manifest from
YouTube will always be successful and didn't check the status code.
That meant that if YouTube gave a rate-limiting page, invidious would
return an HTTP 200 response with the 'application/dash+xml' Content-Type
header and the YouTube ratelimiting page as the body.
No associated issue
The current Content Security Policy does not allow to embed videos
inside local HTML files which are viewed in the browser via the file
protocol. This commit adds the file protocol to the allowed frame
ancestors, so that the embedded videos load correctly in local HTML
files.
This behaviour is consistent which how the official YouTube website
allows to embed videos from itself.
Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
This handler should no have been removed in 4276, as it adds the required CORS
header (Access-Control-Allow-Origin) for public acces to the API.
Thanks to iBicha for noticing this!
This handler should no have been removed in 4276, as it adds the required CORS
header (Access-Control-Allow-Origin) for public acces to the API.
Thanks to iBicha for noticing this!
The function names `count_users_active_6m` and `count_users_active_1m` were
swapped. As the names were swapped on both sides (declaration and use), this
had no functional impact.
No related isse was tracked.
This PR fixes an issue where the `scopes` parameter would see its wildmark
characters (*) removed during the login page redirection, after that a call
to `/authorize_token` was made while the user was not logged in.
Closes issue 4200
Multiple users have reported that the fields parameter is slowing down API
response times significantly. As most API endpoints are already optimized to
make as few requests as possible to Youtube, there is no point in limiting the
output. Furthermore, the added processing might be part of the broader memory
leak problem (See 1438).
In addition, the small increase in data output is not much of an issue compared
to the huge video proxy that lies next to this API.
No related issue tracked
This PR parses the start and end time for clips.
It also adds a new, dedicated API endpoint (`/api/v1/clips/{id}`) for
retrieving the start and end time of a clip.
Here is a sample response from that new endpoint (`video` is a video object,
as described in https://docs.invidious.io/api/common_types/#videoobject):
GET `/api/v1/clips/UgkxxPM3BRphCAPLP88YoUGuj79KXPfpNNO_?pretty=1`
Response:
```
{
"startTime": 8842.645,
"endTime": 8855.856,
"clipTitle": "✂️ Kirby is pink!",
"video": {}
}
```
Closes issue 3921
Some playlist author's don't have a YouTube channel, so does movies.
This caused various extractors (related videos, search) to fail.
Closes the following issues:
2530, 3349, 3766, 3812, 4133
* Add option to disable force_resolve in make_client
Some websites such as archive.org and textcaptcha.com
does not support IPv6 and as such requests fail when Invidious requests
with IPv6 to those services.
* Reenable force_resolve on pubsub subcribe request
* Make force_resolve false by default in make_client
* Remove missed explicit force_resolve=false
