The current option to export subscriptions for NewPipe (`Export subscriptions as OPML (for NewPipe & FreeTube)`) isn't accepted by NewPipe anymore. As far as I can tell, NewPipe only accepts importing from its own SQLite database, from Google Account CSV export and from its own JSON export format. This is mentioned in https://github.com/iv-org/invidious/issues/5438.
This PR adds support to export subscriptions in the NewPipe compatible format. I'm looking for input on a few points:
- I'm not a huge fan of hardcoding the NewPipe version in the export, but it is required by NewPipe to recognize it as a valid file. If someone has a better idea I'm all ears but I think it's the only way. In any case, NewPipe can import formats in a backwards compatible way in the future so it shouldn't be a huge deal.
- I haven't checked if the OPML export option actually works for FreeTube. If it doesn't, it should probably be removed altogether, for now I just removed the `(for NewPipe)` part.
- The translations would need to be updated, but I don't know how that is handled in this project so I need some help there.
The new format looks like this:
```
{
"subscriptions": [
{
"service_id": 0,
"url": "https://www.youtube.com/channel/UCHnyfMqiRRG1u-2MsSQLbXA",
"name": "Veritasium"
},
{
"service_id": 0,
"url": "https://www.youtube.com/channel/UC6nSFpj9HTCZ5t-N3Rm3-HA",
"name": "Vsauce"
}
],
"app_version": "0.29.0",
"app_version_int": 1014
}
```
* add subscription json export for NewPipe
* remove old newpipe export format
* make newpipe export values constant
* remove duplicate export option, add i18n keys
* Convert to_newpipe to a serializable struct
* Use same naming format as NewPipe for the exported subscriptions
---------
Co-authored-by: Fijxu <fijxu@nadeko.net>
* chore: Encapsulate more functions into their own modules and improve specs
* update path for required .cr file in search/hastag_spec.cr
* make the linter happy
* Align PR changes with the current source code
* feat(channel): add support for shows tab
* feat: add support for /show/ urls
* chore(docs): add comment for GridShowRenderer and fix format
* fix: check that shows tabs exists before trying to fetch/parse
* fix: Remove `thumbnaiext`
This property doesn't exist.
Not exactly sure how it happened but I'm guessing I combined thumbnailOverlays and text somehow when I was doing a find and replace (Ctrl + H) and then formatting json with prettier
* chore: Completely remove Reddit from Invidious
* chore: remove reddit from config.example.yml
* fix comment preferences for nojs
* Add fallback for people that used reddit in the comments API and their preferences
* add missing TODO
* chore: remove unused swap_comments JS function from watch.js
* remove blank line
* chore: update to ameba 1.7.0 and adapt most of the code to it
* forgot to add this to build ameba into the ./bin directory for use, as ameba does not longer offer a postinstall script
* forgot to update ameba in the shards files
* remove leftover : in rescue
* Use ameba linter action instead of installing it with shards
Show a nice message for unsupported versions of the Crystal compiler, encouraging the user compiling Invidious, to update it's Crystal compiler version.
* feat: sanity check video IDs with regex
* Return 400 instead
* fix wrong function
* add coderabbit suggestion about PCRE2 regex and length of the string with a new line
* Use validate_video_id on more places
* Add video id validation to /watch route.
* Revert "Fix disappearing end of the comments with emoji (#5587)"
This reverts commit 86c425b43f.
* fix: fix rendered links and timestamps in video descriptions
* Add error message when comments are disabled
* Add btn to try reddit comments when yt is disabled
* Escape warning messages for disabled comments
* Add missing semicolons
* Update comment detection to not use msg renderer
* Use short syntax for comment entry-point detection
Co-authored-by: Samantaz Fox <coding@samantaz.fr>
* Condense try-reddit-comments-link HTML in player.js
Co-authored-by: Samantaz Fox <coding@samantaz.fr>
* Improve locale keys for comments disabled messages
Co-authored-by: Samantaz Fox <coding@samantaz.fr>
* Remove extra whitespace
* Use button instead of anchor for try reddit link
* Request Reddit comm when yt disabled during nojs=1
* Change order of commentsEnabled in parse results
* Lint
* Rebase error
* update the naming of the comments section in Innertube
Downstreamed from 36ac67610d
---------
Co-authored-by: Samantaz Fox <coding@samantaz.fr>
Co-authored-by: Fijxu <fijxu@nadeko.net>
Community posts comments still preserve the old author thumbnails
format that has an Array of thumbnails. I forgot to check that as I had
no idea `comments_youtube.cr` was also used for community posts comments
```json
"authorThumbnail": {
"thumbnails": [
{
"url": "//yt3.googleusercontent.com/ytc/AIdro_m9CJFVl3bEWvGnNnN4G9ErBO2lTpKePWCjx_FQtLWaDww=s32-c-k-c0x00ffffff-no-rj-mo",
"width": 32,
"height": 32
},
{
"url": "//yt3.googleusercontent.com/ytc/AIdro_m9CJFVl3bEWvGnNnN4G9ErBO2lTpKePWCjx_FQtLWaDww=s48-c-k-c0x00ffffff-no-rj-mo",
"width": 48,
"height": 48
},
{
"url": "//yt3.googleusercontent.com/ytc/AIdro_m9CJFVl3bEWvGnNnN4G9ErBO2lTpKePWCjx_FQtLWaDww=s76-c-k-c0x00ffffff-no-rj-mo",
"width": 76,
"height": 76
}
],
"accessibility": {
"accessibilityData": {
"label": "SomeOrdinaryGamers"
}
}
},
```
Some YouTube videos like https://www.youtube.com/watch?v=Ajkr_Wg-k8g
had an non comment key that was inside the Array that contained some
comment information that Invidious parses (like comment_key and
toolbar_key), that being the `commentFilterContextViewModel` key, which
is just an indicator on how comments are sorted:
```
{
"commentThreadRenderer": { ... } <- comment information that is
handled by line 134 of youtube.cr
},
{
"commentFilterContextViewModel": {
"text": {
"content": "Top is selected, so you'll see featured comments"
},
"accessibility": {
"accessibilityData": {
"label": "Top is selected, so you'll see featured comments"
}
}
} <- Useless information, this is not a comment
}
```
* Add option to disable easy to abuse API endpoints
The API endpoints that will be disabled when this option are:
- /api/v1/videos
- /api/v1/clips
- /api/v1/transcripts
There is still API endponts that need some sort of validation or
connection/proxying to Invidious companion like
`/api/v1/captions` and `/api/v1/storyboards` since they also do a video
request to Invidious companion. I'm not sure if the `/next` API endpoint
could be used to gather that type of information, if so, that would be
better.
Closes#5599
* Rename configuration variable, add additional comment for the option
Saves an unnecessary redirect by using `www.youtube.com` on some parts of Invidious where `youtube.com` was used instead. youtube.com will redirect to www.youtube.com anyways.
Co-authored-by: janmoesen <@>
* Add support for alternative domains for Invidious cookies
* check if @@secure is already true before changing it's value
* Add alternative_domains config option example
* fix: Do not append query params `quality=medium` to videos that are about to premiere
Video premieres do not have `audio_streams` and they aren't `video.live_now == true` either,
therefore we skip the checks that are made for older videos (`if audio_streams.empty? && !video.live_now` closure)
if the video is a premiere.
This commit also adds the `microformat` field to the `info` instance
variable of the `Video` struct, since it's needed for the function
`Video#premiere_timestamp : Time?`. This is more like a quick fix than a
proper fix because in `Invidious::Videos::Parser#parse_video_info`,
`premiere_timestamp` is gathered from the `microformat` JSON Object but
is used to set the `published` hash key for the `params` variable.
The parsers and structs really need a rework :/
* Revert "fix: Do not append query params `quality=medium` to videos that are about to premiere"
This reverts commit 5cfc8dace8.
* chore: build premiere_timestamp using video_type and published time
Previous code use UTF-8 to count characters however Emojis are UTF-16 units.
This difference leads to misalignment of index offsets.
Co-authored-by: shiny-comic <shiny-comic@winPC>
* fix author verification in channels
Fixes https://github.com/iv-org/invidious/issues/5730
Author verification badge is not longer located in
`c4TabbedHeaderRenderer`. It may be deprecated.
* Also detect AUDIO_BADGE for verified author
* feat: add support for /pl_c/ images
This path can be found on Podcast images.
* add support for /pl_c and /tvfilm_banner paths
* add support for /pl_c and /tvfilm_banner paths 2
* remove leftover comment
* fix: fix channel videos and playlists on searches
Channel videos are now encapsulated in a `lockupViewModel`.
There is 3 types of content that can be inside a `lockupViewModel`:
- LOCKUP_CONTENT_TYPE_VIDEO
- LOCKUP_CONTENT_TYPE_PLAYLIST
- LOCKUP_CONTENT_TYPE_PODCAST
This commit parses `LOCKUP_CONTENT_TYPE_VIDEO`, `LOCKUP_CONTENT_TYPE_PLAYLIST`, `LOCKUP_CONTENT_TYPE_PODCAST` types
to fix videos in channels, playlists in channels, podcast in channels, and other parts of Invidious were playlists and videos are displayed.
* remove unused variable `author_verified`
* fix parsing for podcasts
For some reason, Podcasts contains an empty JSON Object that we have to
skip, therefore we just iterate metadataRows until finding metadataParts
since metadataRows will not always contain a single Object.
* fix length_seconds for channel videos
* fix playlists parsing for playlists without metadataParts
On some channels like MrBeast, metadataParts is absent, missing the
author information, this is intended behaviour by Youtube since there is
no author information attached to them.
Example URL: https://www.youtube.com/channel/UCX6OQ3DkcsbYNE6H8uQQuVA/playlists
* restore author_verified functionality
* more robust metadata_parts parsing
Videos that have two or more authors (in collaborations), have their
author information in JSON Objects inside metadataParts, alongside with
their view count and their published date, therefore, we need to iterate
the metadataParts array and do some filtering based on the content of
each JSON Object to find the view count and published date for some
videos.
Example:
```json
"metadataParts": [
{
"text": {
"content": "Veritasium"
},
"icon": {
"name": "CHECK_CIRCLE_FILLED",
"height": 14,
"width": 14,
"accessibilityLabel": "Verified"
}
},
{
"text": {
"content": "and Linus Tech Tips"
},
"icon": {
"name": "CHECK_CIRCLE_FILLED",
"height": 14,
"width": 14,
"accessibilityLabel": "Verified"
}
},
{
"text": {
"content": "10M"
},
"accessibilityLabel": "10 million views"
},
{
"text": {
"content": "1y ago"
}
}
]
```
* improve playlist metadataRows and metadataParts parsing for channel playlists
* apply ameba suggestion
* Also parse lessons for playlists that are a course
