Mirrored_Repos/invidious
1
0
Fork 0
mirror of https://github.com/iv-org/invidious.git synced 2024-11-14 12:18:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
5005212bec
invidious/src/invidious.cr

3365 lines
100 KiB
Crystal
Raw Normal View History

Update project synopsis
2018-09-04 14:22:10 +00:00
# "Invidious" (which is an alternative front-end to YouTube)
Update copyright notice
2019-03-15 16:44:53 +00:00
# Copyright (C) 2019 Omar Roth
Change license
2018-01-28 17:32:40 +00:00
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published
# by the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
Add text CAPTCHA
2018-11-22 19:26:08 +00:00
require "digest/md5"
Add -o option for redirecting output
2019-01-23 20:15:19 +00:00
require "file_utils"
Initial commit
2017-11-23 07:48:55 +00:00
require "kemal"
Use `athena-negotiation` to detect language through Accept-Language header (#2324) Detect language through Accept-Language header
2021-08-24 19:59:27 +00:00
require "athena-negotiation"
Add separate user accounts
2018-07-18 19:26:02 +00:00
require "openssl/hmac"
Add ability to specify number of threads
2018-02-03 22:13:14 +00:00
require "option_parser"
Add view counter, likes/dislikes, rating, and HTML parser
2017-11-24 04:06:43 +00:00
require "pg"
Properly parse NewPipe imports
2018-11-21 23:12:13 +00:00
require "sqlite3"
Fix audio-only for video.js
2018-01-16 20:02:35 +00:00
require "xml"
Add config file
2018-03-09 18:42:23 +00:00
require "yaml"
Support Crystal 0.35.0
2020-06-15 22:57:20 +00:00
require "compress/zip"
Refactor continuation token handling
2019-10-27 17:50:42 +00:00
require "protodec/utils"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
require "./invidious/helpers/*"
Remove patch and update file structure
2018-07-06 12:59:56 +00:00
require "./invidious/*"
Debloat channels.cr into multiple files (#2225) Cherry picked from ui overhaul branch with a few modifications: - channel folder is renamed to channels - parsing for channel home and featured channels are removed due to lack of infrastructure from other commits (cherry picked from commit 44d18b8e147b47ad06a54cc6fd08423d9f39074d)
2021-07-14 15:46:12 +00:00
require "./invidious/channels/*"
Provide rough draft of better project organization
2020-10-06 04:41:18 +00:00
require "./invidious/routes/**"
require "./invidious/jobs/**"
Move views to src/ directory
2017-11-29 21:33:46 +00:00
Add config environment variables The config file can now be specified with `INVIDIOUS_CONFIG_FILE`. A YAML formatted string can still be passed with `INVIDIOUS_CONFIG`, replacing the config file. Additionally all options can now be specified as environment variables. The syntax for variable names is `INVIDIOUS_` followed by the option name in upper case. The values are parsed as YAML. These new env vars only update the provided main configuration, but it is possible to point the config file at the example config and then use env vars for all config options: ``` INVIDIOUS_CONFIG_FILE=./config/config.example.yml \ INVIDIOUS_CHANNEL_THREADS=10 \ ./invidious ```
2021-01-23 17:58:13 +00:00
CONFIG = Config.load
HMAC_KEY = CONFIG.hmac_key || Random::Secure.hex(32)
Add config file
2018-03-09 18:42:23 +00:00
Allow providing 12-Factor-style Database URL in config
2021-01-30 14:52:48 +00:00
PG_DB = DB.open CONFIG.database_url
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
ARCHIVE_URL = URI.parse("https://archive.org")
LOGIN_URL = URI.parse("https://accounts.google.com")
PUBSUB_URL = URI.parse("https://pubsubhubbub.appspot.com")
REDDIT_URL = URI.parse("https://www.reddit.com")
Switch textcaptcha to HTTPS
2020-03-10 15:12:11 +00:00
TEXTCAPTCHA_URL = URI.parse("https://textcaptcha.com")
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
YT_URL = URI.parse("https://www.youtube.com")
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
HOST_URL = make_host_url(Kemal.config)
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
Add '/api/v1/auth/feeds'
2019-06-07 17:39:12 +00:00
CHARS_SAFE = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_"
TEST_IDS = {"AgbeGFYluEA", "BaW_jenozKc", "a9LDPn-MO4I", "ddFvjfvPnqk", "iqKdEhx-dD4"}
Set max preference size
2019-06-08 21:04:55 +00:00
MAX_ITEMS_PER_PAGE = 1500
Fix regression for youtube pool
2018-03-05 04:25:03 +00:00
Update QUICPool
2019-11-24 18:41:47 +00:00
REQUEST_HEADERS_WHITELIST = {"accept", "accept-encoding", "cache-control", "content-length", "if-none-match", "range"}
RESPONSE_HEADERS_BLACKLIST = {"access-control-allow-origin", "alt-svc", "server"}
Chunk video files to bypass throttling
2019-07-04 20:30:00 +00:00
HTTP_CHUNK_SIZE = 10485760 # ~10MB
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
Update code formatting for Crystal 0.33.0 Crystal 0.33.0 introduced some changes to to the code formatter. Run "crystal tool format" so CI doesn't fail anymore.
2020-02-15 18:52:28 +00:00
CURRENT_BRANCH = {{ "#{`git branch | sed -n '/* /s///p'`.strip}" }}
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
CURRENT_COMMIT = {{ "#{`git rev-list HEAD --max-count=1 --abbrev-commit`.strip}" }}
Fix invidious version for docker image As Alpine-Linux per default uses the busybox date command, the -f option is not available and the build will fail.
2020-12-05 19:06:24 +00:00
CURRENT_VERSION = {{ "#{`git log -1 --format=%ci | awk '{print $1}' | sed s/-/./g`.strip}" }}
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
Use separate asset version for cache busting
2019-05-09 16:52:37 +00:00
# This is used to determine the `?v=` on the end of file URLs (for cache busting). We
# only need to expire modified assets, so we can use this to find the last commit that changes
# any assets
ASSET_COMMIT = {{ "#{`git rev-list HEAD --max-count=1 --abbrev-commit -- assets`.strip}" }}
Add '--version' to command line
2019-04-06 13:28:53 +00:00
SOFTWARE = {
"name" => "invidious",
"version" => "#{CURRENT_VERSION}-#{CURRENT_COMMIT}",
"branch" => "#{CURRENT_BRANCH}",
}
add option to disable quic
2021-04-03 20:11:35 +00:00
YT_POOL = YoutubeConnectionPool.new(YT_URL, capacity: CONFIG.pool_size, timeout: 2.0, use_quic: CONFIG.use_quic)
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
# CLI
Add '--version' to command line
2019-04-06 13:28:53 +00:00
Kemal.config.extra_options do |parser|
parser.banner = "Usage: invidious [arguments]"
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
parser.on("-c THREADS", "--channel-threads=THREADS", "Number of threads for refreshing channels (default: #{CONFIG.channel_threads})") do |number|
Add '--version' to command line
2019-04-06 13:28:53 +00:00
begin
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
CONFIG.channel_threads = number.to_i
Add '--version' to command line
2019-04-06 13:28:53 +00:00
rescue ex
puts "THREADS must be integer"
exit
end
end
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
parser.on("-f THREADS", "--feed-threads=THREADS", "Number of threads for refreshing feeds (default: #{CONFIG.feed_threads})") do |number|
Add '--version' to command line
2019-04-06 13:28:53 +00:00
begin
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
CONFIG.feed_threads = number.to_i
Add '--version' to command line
2019-04-06 13:28:53 +00:00
rescue ex
puts "THREADS must be integer"
exit
end
end
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
parser.on("-o OUTPUT", "--output=OUTPUT", "Redirect output (default: #{CONFIG.output})") do |output|
CONFIG.output = output
Add '--version' to command line
2019-04-06 13:28:53 +00:00
end
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
parser.on("-l LEVEL", "--log-level=LEVEL", "Log level, one of #{LogLevel.values} (default: #{CONFIG.log_level})") do |log_level|
CONFIG.log_level = LogLevel.parse(log_level)
Improve logging Everything that gets logged now has a log level associated with it. The log level can be set with the new `-l` or `--log-level` arguments. The defaul log level is `debug` for now. There aren't many things that get logged but if the logs get spammed in the future it can be set down to `info`.
2020-12-21 15:05:35 +00:00
end
parser.on("-v", "--version", "Print version") do
Add '--version' to command line
2019-04-06 13:28:53 +00:00
puts SOFTWARE.to_pretty_json
exit
end
end
Kemal::CLI.new ARGV
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
if CONFIG.output.upcase != "STDOUT"
FileUtils.mkdir_p(File.dirname(CONFIG.output))
Add CLI arguments to config file The log level can now be set with `log_level` (accepts ints and strings). The log file can now be set with `output` (also accepts `STDOUT`).
2021-01-04 15:05:15 +00:00
end
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
OUTPUT = CONFIG.output.upcase == "STDOUT" ? STDOUT : File.open(CONFIG.output, mode: "a")
LOGGER = Invidious::LogHandler.new(OUTPUT, CONFIG.log_level)
Add CLI arguments to config file The log level can now be set with `log_level` (accepts ints and strings). The log file can now be set with `output` (also accepts `STDOUT`).
2021-01-04 15:05:15 +00:00
Add config option to cache annotations from IA
2019-04-15 16:13:09 +00:00
# Check table integrity
Make check_tables config option
2019-04-11 17:13:25 +00:00
if CONFIG.check_tables
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
check_enum(PG_DB, "privacy", PlaylistPrivacy)
Add support for custom playlists
2019-08-05 23:49:13 +00:00
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
check_table(PG_DB, "channels", InvidiousChannel)
check_table(PG_DB, "channel_videos", ChannelVideo)
check_table(PG_DB, "playlists", InvidiousPlaylist)
check_table(PG_DB, "playlist_videos", PlaylistVideo)
check_table(PG_DB, "nonces", Nonce)
check_table(PG_DB, "session_ids", SessionId)
check_table(PG_DB, "users", User)
check_table(PG_DB, "videos", Video)
Add config option to cache annotations from IA
2019-04-15 16:13:09 +00:00
if CONFIG.cache_annotations
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
check_table(PG_DB, "annotations", Annotation)
Add config option to cache annotations from IA
2019-04-15 16:13:09 +00:00
end
Make check_tables config option
2019-04-11 17:13:25 +00:00
end
Add channel refresh
2018-03-26 03:18:29 +00:00
Automatically migrate database
2019-04-10 21:23:37 +00:00
# Start jobs
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
Only start refresh jobs when necessary If `channel_threads` or `feed_threads` is set to zero the corresponding job is now not started.
2021-01-23 18:41:50 +00:00
if CONFIG.channel_threads > 0
Invidious::Jobs.register Invidious::Jobs::RefreshChannelsJob.new(PG_DB)
end
if CONFIG.feed_threads > 0
Invidious::Jobs.register Invidious::Jobs::RefreshFeedsJob.new(PG_DB)
end
add config to decrypt on demand instead of polling
2020-09-27 17:19:44 +00:00
DECRYPT_FUNCTION = DecryptFunction.new(CONFIG.decrypt_polling)
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if CONFIG.decrypt_polling
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
Invidious::Jobs.register Invidious::Jobs::UpdateDecryptFunctionJob.new
add config to decrypt on demand instead of polling
2020-09-27 17:19:44 +00:00
end
Add support for subscribing to channels via PubSubHubbub
2019-03-04 01:18:23 +00:00
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if CONFIG.statistics_enabled
Invidious::Jobs.register Invidious::Jobs::StatisticsRefreshJob.new(PG_DB, SOFTWARE)
[refactor] Finish converting jobs to new job setup (#1420) [refactor] Finish converting jobs to new job setup
2020-10-17 12:25:57 +00:00
end
Minor refactor
2019-08-27 13:08:26 +00:00
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if (CONFIG.use_pubsub_feeds.is_a?(Bool) && CONFIG.use_pubsub_feeds.as(Bool)) || (CONFIG.use_pubsub_feeds.is_a?(Int32) && CONFIG.use_pubsub_feeds.as(Int32) > 0)
Invidious::Jobs.register Invidious::Jobs::SubscribeToFeedsJob.new(PG_DB, HMAC_KEY)
Respect `use_pubsub_feeds` config Setting `use_pubsub_feeds: false` now properly disables it.
2021-01-07 19:15:26 +00:00
end
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if CONFIG.popular_enabled
Add `popular-enabled` option This is similar to the removed `top-enabled` option but for the Popular feed. The instance needs to be restarted if the feed was enabled. Editing admin options on the preferences page is also fixed. The handling of the feed pages now only happens in a single place. Instead of redirecting: - The Top feed now displays a message that it was removed from Invidious. - The Popular feed now displays a message that it was disabled if it was.
2020-12-27 05:12:43 +00:00
Invidious::Jobs.register Invidious::Jobs::PullPopularVideosJob.new(PG_DB)
end
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if CONFIG.captcha_key
Invidious::Jobs.register Invidious::Jobs::BypassCaptchaJob.new
Add /api/v1/stats
2019-03-02 01:25:16 +00:00
end
Clean up footer and add version
2019-03-01 22:47:06 +00:00
[refactor] Finish converting jobs to new job setup (#1420) [refactor] Finish converting jobs to new job setup
2020-10-17 12:25:57 +00:00
connection_channel = Channel({Bool, Channel(PQ::Notification)}).new(32)
Allow providing 12-Factor-style Database URL in config
2021-01-30 14:52:48 +00:00
Invidious::Jobs.register Invidious::Jobs::NotificationJob.new(connection_channel, CONFIG.database_url)
[refactor] Finish converting jobs to new job setup (#1420) [refactor] Finish converting jobs to new job setup
2020-10-17 12:25:57 +00:00
Provide rough draft of better project organization
2020-10-06 04:41:18 +00:00
Invidious::Jobs.start_all
def popular_videos
Invidious::Jobs::PullPopularVideosJob::POPULAR_VIDEOS.get
Add job for pulling popular videos
2018-11-09 02:08:03 +00:00
end
Add fix for testing authentication
2018-03-25 03:56:41 +00:00
before_all do |env|
Use `athena-negotiation` to detect language through Accept-Language header (#2324) Detect language through Accept-Language header
2021-08-24 19:59:27 +00:00
preferences = Preferences.from_json("{}")
begin
if prefs_cookie = env.request.cookies["PREFS"]?
preferences = Preferences.from_json(URI.decode_www_form(prefs_cookie.value))
else
if language_header = env.request.headers["Accept-Language"]?
if language = ANG.language_negotiator.best(language_header, LOCALES.keys)
preferences.locale = language.header
end
end
end
Migrate to a good Content Security Policy (#1023) So attacks such as XSS (see [0]) will no longer be of an issue. [0]: https://github.com/omarroth/invidious/issues/1022
2020-03-15 21:46:08 +00:00
rescue
Use `athena-negotiation` to detect language through Accept-Language header (#2324) Detect language through Accept-Language header
2021-08-24 19:59:27 +00:00
preferences = Preferences.from_json("{}")
Migrate to a good Content Security Policy (#1023) So attacks such as XSS (see [0]) will no longer be of an issue. [0]: https://github.com/omarroth/invidious/issues/1022
2020-03-15 21:46:08 +00:00
end
Set preferences before conditional breaks in before_all (#1412)
2020-10-16 10:23:18 +00:00
env.set "preferences", preferences
Fix typo in X-XSS-Protection
2019-05-10 21:48:38 +00:00
env.response.headers["X-XSS-Protection"] = "1; mode=block"
Add X-XSS-Protection and X-Content-Type-Options
2018-09-06 02:51:40 +00:00
env.response.headers["X-Content-Type-Options"] = "nosniff"
Rewrite response headers Fixes #2018 and #2153
2021-06-17 17:45:20 +00:00
Typo
2021-06-19 07:40:33 +00:00
# Allow media resources to be loaded from google servers
Rewrite response headers Fixes #2018 and #2153
2021-06-17 17:45:20 +00:00
# TODO: check if *.youtube.com can be removed
Migrate to a good Content Security Policy (#1023) So attacks such as XSS (see [0]) will no longer be of an issue. [0]: https://github.com/omarroth/invidious/issues/1022
2020-03-15 21:46:08 +00:00
if CONFIG.disabled?("local") || !preferences.local
Rewrite response headers Fixes #2018 and #2153
2021-06-17 17:45:20 +00:00
extra_media_csp = " https://*.googlevideo.com:443 https://*.youtube.com:443"
else
extra_media_csp = ""
end
# Only allow the pages at /embed/* to be embedded
if env.request.resource.starts_with?("/embed")
frame_ancestors = "'self' http: https:"
else
Fix quoting of 'none' in CSP header The keyword 'none' must be surrounded by single quotes. Regression introduced by #2168.
2021-06-20 16:43:00 +00:00
frame_ancestors = "'none'"
Migrate to a good Content Security Policy (#1023) So attacks such as XSS (see [0]) will no longer be of an issue. [0]: https://github.com/omarroth/invidious/issues/1022
2020-03-15 21:46:08 +00:00
end
Rewrite response headers Fixes #2018 and #2153
2021-06-17 17:45:20 +00:00
# TODO: Remove style-src's 'unsafe-inline', requires to remove all
# inline styles (<style> [..] </style>, style=" [..] ")
env.response.headers["Content-Security-Policy"] = {
"default-src 'none'",
"script-src 'self'",
"style-src 'self' 'unsafe-inline'",
"img-src 'self' data:",
"font-src 'self' data:",
"connect-src 'self'",
"manifest-src 'self'",
"media-src 'self' blob:" + extra_media_csp,
"child-src 'self' blob:",
"frame-src 'self'",
"frame-ancestors " + frame_ancestors,
}.join("; ")
Add CSP, STS, and Referrer-Policy
2019-04-07 19:01:08 +00:00
env.response.headers["Referrer-Policy"] = "same-origin"
Rewrite response headers Fixes #2018 and #2153
2021-06-17 17:45:20 +00:00
# Ask the chrom*-based browsers to disable FLoC
# See: https://blog.runcloud.io/google-floc/
env.response.headers["Permissions-Policy"] = "interest-cohort=()"
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if (Kemal.config.ssl || CONFIG.https_only) && CONFIG.hsts
Bump max-age for HSTS
2019-05-01 01:53:56 +00:00
env.response.headers["Strict-Transport-Security"] = "max-age=31536000; includeSubDomains; preload"
Add CSP, STS, and Referrer-Policy
2019-04-07 19:01:08 +00:00
end
Update handling for preferences
2019-03-28 18:43:40 +00:00
Skip preferences for assets
2019-11-20 17:03:52 +00:00
next if {
"/sb/",
"/vi/",
"/s_p/",
"/yts/",
"/ggpht/",
"/api/manifest/",
"/videoplayback",
"/latest_version",
}.any? { |r| env.request.resource.starts_with? r }
Add user preferences
2018-07-16 16:24:24 +00:00
if env.request.cookies.has_key? "SID"
Add notifications
2018-04-01 00:09:27 +00:00
sid = env.request.cookies["SID"].value
Move user state into before_all
2018-07-05 23:43:26 +00:00
Add authentication API
2019-04-18 21:23:50 +00:00
if sid.starts_with? "v1:"
raise "Cannot use token as SID"
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
# Invidious users only have SID
if !env.request.cookies.has_key? "SSID"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
if email = PG_DB.query_one?("SELECT email FROM session_ids WHERE id = $1", sid, as: String)
Run 'crystal tool format'
2019-02-12 02:52:47 +00:00
user = PG_DB.query_one("SELECT * FROM users WHERE email = $1", email, as: User)
Add support for custom playlists
2019-08-05 23:49:13 +00:00
csrf_token = generate_response(sid, {
":authorize_token",
":playlist_ajax",
":signout",
":subscription_ajax",
":token_ajax",
":watch_ajax",
}, HMAC_KEY, PG_DB, 1.week)
Add CSRF prevention for /signout
2018-11-08 23:42:25 +00:00
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
preferences = user.preferences
Set preferences before conditional breaks in before_all (#1412)
2020-10-16 10:23:18 +00:00
env.set "preferences", preferences
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
Add support for multiple sessions
2018-08-15 17:40:42 +00:00
env.set "sid", sid
Add authentication API
2019-04-18 21:23:50 +00:00
env.set "csrf_token", csrf_token
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
env.set "user", user
Add separate user accounts
2018-07-18 19:26:02 +00:00
end
else
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
headers = HTTP::Headers.new
headers["Cookie"] = env.request.headers["Cookie"]
Add separate user accounts
2018-07-18 19:26:02 +00:00
begin
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user, sid = get_user(sid, headers, PG_DB, false)
Add support for custom playlists
2019-08-05 23:49:13 +00:00
csrf_token = generate_response(sid, {
":authorize_token",
":playlist_ajax",
":signout",
":subscription_ajax",
":token_ajax",
":watch_ajax",
}, HMAC_KEY, PG_DB, 1.week)
Add CSRF token for Google accounts
2018-11-16 02:23:17 +00:00
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
preferences = user.preferences
Set preferences before conditional breaks in before_all (#1412)
2020-10-16 10:23:18 +00:00
env.set "preferences", preferences
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
Add support for multiple sessions
2018-08-15 17:40:42 +00:00
env.set "sid", sid
Add authentication API
2019-04-18 21:23:50 +00:00
env.set "csrf_token", csrf_token
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
env.set "user", user
Add separate user accounts
2018-07-18 19:26:02 +00:00
rescue ex
end
Add handling for invalid login
2018-07-16 17:50:41 +00:00
end
Add dark theme
2018-04-14 02:32:14 +00:00
end
Fix referers
2018-08-17 15:19:20 +00:00
Add prefers-color-scheme support (#601) * Add prefers-color-scheme support This should fix <https://github.com/omarroth/invidious/issues/559>. The cookie storage format has been changed from boolean ("true"/"false") to tri-state ("dark"/"light"/""), so that users without a cookie set will get dark mode if they have enabled the dark theme in their operating system. The code for handling the cookie state, along with the user's operating system theme, has been factored out into a new function `update_mode`, which is called both at window load and at the "storage" event listener, because the "storage" event listener is only trigerred when a change is made to the localStorage from another tab/window (for more info - see <https://stackoverflow.com/a/4679754>).
2019-08-15 16:29:55 +00:00
dark_mode = convert_theme(env.params.query["dark_mode"]?) || preferences.dark_mode.to_s
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
thin_mode = env.params.query["thin_mode"]? || preferences.thin_mode.to_s
thin_mode = thin_mode == "true"
locale = env.params.query["hl"]? || preferences.locale
preferences.dark_mode = dark_mode
preferences.thin_mode = thin_mode
preferences.locale = locale
Fix: functional hl / dark_mode / thin_mode params
2021-03-17 19:07:38 +00:00
env.set "preferences", preferences
Add support for translations
2018-12-20 21:32:09 +00:00
Fix referers
2018-08-17 15:19:20 +00:00
current_page = env.request.path
if env.request.query
query = HTTP::Params.parse(env.request.query.not_nil!)
if query["referer"]?
query["referer"] = get_referer(env, "/")
end
current_page += "?#{query}"
end
Update to Crystal 0.31.0, resolve compiler deprecation warnings, update dependencies (#764) * shard: update to crystal 0.31.0 Additionally, no longer use the Crystal "markdown" library which has been removed from the Crystal stdlib in version 0.31.0. See https://github.com/crystal-lang/crystal/pull/8115. Also fix some deprecation warnings using the following commands: find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.escape/URI\.encode_www_form/g' "{}" \; find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.unescape/URI\.decode_www_form/g' "{}" \; sed -i 's/while \%pull\.kind \!\= \:end_object/until \%pull\.kind\.end_object\?/g' src/invidious/helpers/patch_mapping.cr
2019-09-24 17:31:33 +00:00
env.set "current_page", URI.encode_www_form(current_page)
Add logout
2018-03-22 17:44:36 +00:00
end
Adjust routes Simple routes have been moved into a single `Misc` file. Embed routes have been moved into a single `Embed` file. The preferences route has been renamed to be more consistent with other parts of the codebase.
2021-02-02 05:18:16 +00:00
Invidious::Routing.get "/", Invidious::Routes::Misc, :home
Invidious::Routing.get "/privacy", Invidious::Routes::Misc, :privacy
Invidious::Routing.get "/licenses", Invidious::Routes::Misc, :licenses
Add aliases for watch route
2021-04-07 03:07:19 +00:00
Extract channel routes (#2227) * Extract primary channel routes from invidious.cr Also removes timedtext_video stub since all it does is redirect to the homepage. However, Invidious's 404 handler already does this. -- As the template for the channel about page doesn't exist yet, the behavior for the /channel/:ucid/about endpoint has been changed to be the same as what's currently present on Invidious (cherry picked from commit 8fad19d8057d7d22e3de27ebbc88a9978c1df27b) * Manually extract brand_redirect from 1b569bbc99207cae7c20aa285f42477ae361dd30 This commit manually extracts the brand_redirect function from the commit mentioned. However, the redirect to the `.../about` endpoint is removed due to the fact that it doesn't exist yet. This commit is also mainly just a bridge for the next few cherry picks from \#2215 * Update brand_redirect to use youtubei resolve_url (cherry picked from commit 53335fe7cfdfac392365b7cac447bc7cc6478134) * Add additional channel endpoints to brand_redirect (cherry picked from commit 8fc6f3add637dabb09b2034f4d82fc3d039ba15c) * Add separate handler for /profile endpoint * Add /channel/:ucid/home route * Document all channel brand_urls
2021-08-03 21:46:15 +00:00
Invidious::Routing.get "/channel/:ucid", Invidious::Routes::Channels, :home
Invidious::Routing.get "/channel/:ucid/home", Invidious::Routes::Channels, :home
Invidious::Routing.get "/channel/:ucid/videos", Invidious::Routes::Channels, :videos
Invidious::Routing.get "/channel/:ucid/playlists", Invidious::Routes::Channels, :playlists
Invidious::Routing.get "/channel/:ucid/community", Invidious::Routes::Channels, :community
Invidious::Routing.get "/channel/:ucid/about", Invidious::Routes::Channels, :about
["", "/videos", "/playlists", "/community", "/about"].each do |path|
# /c/LinusTechTips
Invidious::Routing.get "/c/:user#{path}", Invidious::Routes::Channels, :brand_redirect
# /user/linustechtips | Not always the same as /c/
Invidious::Routing.get "/user/:user#{path}", Invidious::Routes::Channels, :brand_redirect
# /attribution_link?a=anything&u=/channel/UCZYTClx2T1of7BRZ86-8fow
Invidious::Routing.get "/attribution_link#{path}", Invidious::Routes::Channels, :brand_redirect
# /profile?user=linustechtips
Invidious::Routing.get "/profile/#{path}", Invidious::Routes::Channels, :profile
end
Add aliases for watch route
2021-04-07 03:07:19 +00:00
Invidious::Routing.get "/watch", Invidious::Routes::Watch, :handle
Invidious::Routing.get "/watch/:id", Invidious::Routes::Watch, :redirect
Invidious::Routing.get "/shorts/:id", Invidious::Routes::Watch, :redirect
Invidious::Routing.get "/w/:id", Invidious::Routes::Watch, :redirect
Invidious::Routing.get "/v/:id", Invidious::Routes::Watch, :redirect
Invidious::Routing.get "/e/:id", Invidious::Routes::Watch, :redirect
Add route to redirect to another instance
2021-03-27 01:36:40 +00:00
Invidious::Routing.get "/redirect", Invidious::Routes::Misc, :cross_instance_redirect
Add aliases for watch route
2021-04-07 03:07:19 +00:00
Adjust routes Simple routes have been moved into a single `Misc` file. Embed routes have been moved into a single `Embed` file. The preferences route has been renamed to be more consistent with other parts of the codebase.
2021-02-02 05:18:16 +00:00
Invidious::Routing.get "/embed/", Invidious::Routes::Embed, :redirect
Invidious::Routing.get "/embed/:id", Invidious::Routes::Embed, :show
Add aliases for watch route
2021-04-07 03:07:19 +00:00
Extract playlist routes from global file
2020-11-20 01:21:48 +00:00
Invidious::Routing.get "/create_playlist", Invidious::Routes::Playlists, :new
Invidious::Routing.post "/create_playlist", Invidious::Routes::Playlists, :create
Invidious::Routing.get "/subscribe_playlist", Invidious::Routes::Playlists, :subscribe
Invidious::Routing.get "/delete_playlist", Invidious::Routes::Playlists, :delete_page
Invidious::Routing.post "/delete_playlist", Invidious::Routes::Playlists, :delete
Invidious::Routing.get "/edit_playlist", Invidious::Routes::Playlists, :edit
Invidious::Routing.post "/edit_playlist", Invidious::Routes::Playlists, :update
Invidious::Routing.get "/add_playlist_items", Invidious::Routes::Playlists, :add_playlist_items_page
Invidious::Routing.post "/playlist_ajax", Invidious::Routes::Playlists, :playlist_ajax
Invidious::Routing.get "/playlist", Invidious::Routes::Playlists, :show
Invidious::Routing.get "/mix", Invidious::Routes::Playlists, :mix
Add aliases for watch route
2021-04-07 03:07:19 +00:00
Extracting search endpoints
2020-12-02 00:02:39 +00:00
Invidious::Routing.get "/opensearch.xml", Invidious::Routes::Search, :opensearch
Invidious::Routing.get "/results", Invidious::Routes::Search, :results
Invidious::Routing.get "/search", Invidious::Routes::Search, :search
Add aliases for watch route
2021-04-07 03:07:19 +00:00
Extract login/signout routes from global file
2020-12-09 02:50:39 +00:00
Invidious::Routing.get "/login", Invidious::Routes::Login, :login_page
Invidious::Routing.post "/login", Invidious::Routes::Login, :login
Invidious::Routing.post "/signout", Invidious::Routes::Login, :signout
Add aliases for watch route
2021-04-07 03:07:19 +00:00
Adjust routes Simple routes have been moved into a single `Misc` file. Embed routes have been moved into a single `Embed` file. The preferences route has been renamed to be more consistent with other parts of the codebase.
2021-02-02 05:18:16 +00:00
Invidious::Routing.get "/preferences", Invidious::Routes::PreferencesRoute, :show
Invidious::Routing.post "/preferences", Invidious::Routes::PreferencesRoute, :update
Invidious::Routing.get "/toggle_theme", Invidious::Routes::PreferencesRoute, :toggle_theme
Add '/comments/:id'
2018-07-22 01:56:11 +00:00
Extract feed routes (#2269) * Extract feed routes from invidious.cr * Removes the deprecated route for /feed/top * Deprecate /view_all_playlist & use /feed/playlists * Move feed views into their own directory * Add haltf method to halt current route context * Change status_code + return blocks to use haltf * Set appropriate response headers for RSS routes
2021-08-30 14:58:24 +00:00
# Feeds
Invidious::Routing.get "/view_all_playlists", Invidious::Routes::Feeds, :view_all_playlists_redirect
Invidious::Routing.get "/feed/playlists", Invidious::Routes::Feeds, :playlists
Invidious::Routing.get "/feed/popular", Invidious::Routes::Feeds, :popular
Invidious::Routing.get "/feed/trending", Invidious::Routes::Feeds, :trending
Invidious::Routing.get "/feed/subscriptions", Invidious::Routes::Feeds, :subscriptions
Invidious::Routing.get "/feed/history", Invidious::Routes::Feeds, :history
# RSS Feeds
Invidious::Routing.get "/feed/channel/:ucid", Invidious::Routes::Feeds, :rss_channel
Invidious::Routing.get "/feed/private", Invidious::Routes::Feeds, :rss_private
Invidious::Routing.get "/feed/playlist/:plid", Invidious::Routes::Feeds, :rss_playlist
Invidious::Routing.get "/feeds/videos.xml", Invidious::Routes::Feeds, :rss_videos
# Support push notifications via PubSubHubbub
Invidious::Routing.get "/feed/webhook/:token", Invidious::Routes::Feeds, :push_notifications_get
Invidious::Routing.post "/feed/webhook/:token", Invidious::Routes::Feeds, :push_notifications_post
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
# Users
Add '/comments/:id'
2018-07-22 01:56:11 +00:00
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
post "/watch_ajax" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
user = env.get? "user"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
sid = env.get? "sid"
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
referer = get_referer(env, "/feed/subscriptions")
redirect = env.params.query["redirect"]?
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
redirect ||= "true"
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
redirect = redirect == "true"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
if !user
Add authentication API
2019-04-18 21:23:50 +00:00
if redirect
next env.redirect referer
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "No such user")
Add authentication API
2019-04-18 21:23:50 +00:00
end
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
end
Add support for translations
2018-12-20 21:32:09 +00:00
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
user = user.as(User)
sid = sid.as(String)
Add authentication API
2019-04-18 21:23:50 +00:00
token = env.params.body["csrf_token"]?
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
id = env.params.query["id"]?
if !id
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
env.response.status_code = 400
next
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
end
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
begin
Add authentication API
2019-04-18 21:23:50 +00:00
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
rescue ex
if redirect
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, ex)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
end
end
if env.params.query["action_mark_watched"]?
action = "action_mark_watched"
elsif env.params.query["action_mark_unwatched"]?
action = "action_mark_unwatched"
else
next env.redirect referer
end
case action
when "action_mark_watched"
if !user.watched.includes? id
Remove DB array concatenation
2020-02-28 16:46:24 +00:00
PG_DB.exec("UPDATE users SET watched = array_append(watched, $1) WHERE email = $2", id, user.email)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
end
when "action_mark_unwatched"
Properly parse NewPipe imports
2018-11-21 23:12:13 +00:00
PG_DB.exec("UPDATE users SET watched = array_remove(watched, $1) WHERE email = $2", id, user.email)
Fix warnings in latest version of Crystal
2020-04-09 17:18:09 +00:00
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "Unsupported action #{action}")
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
end
if redirect
env.redirect referer
else
env.response.content_type = "application/json"
"{}"
end
end
Major cleanup
2018-08-05 04:07:38 +00:00
# /modify_notifications
# will "ding" all subscriptions.
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
# /modify_notifications?receive_all_updates=false&receive_no_updates=false
# will "unding" all subscriptions.
get "/modify_notifications" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
user = env.get? "user"
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
sid = env.get? "sid"
referer = get_referer(env, "/")
Add /embed/:id
2018-07-14 13:36:31 +00:00
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
redirect = env.params.query["redirect"]?
redirect ||= "false"
redirect = redirect == "true"
Add 'results' route redirect
2018-07-30 02:05:40 +00:00
Add authentication API
2019-04-18 21:23:50 +00:00
if !user
if redirect
next env.redirect referer
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "No such user")
Add authentication API
2019-04-18 21:23:50 +00:00
end
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
end
user = user.as(User)
if !user.password
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
channel_req = {} of String => String
Minor fixes
2018-02-27 00:59:02 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
channel_req["receive_all_updates"] = env.params.query["receive_all_updates"]? || "true"
channel_req["receive_no_updates"] = env.params.query["receive_no_updates"]? || ""
channel_req["receive_post_updates"] = env.params.query["receive_post_updates"]? || "true"
Speed up get_video
2018-01-07 17:42:24 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
channel_req.reject! { |k, v| v != "true" && v != "false" }
Add related videos and clean up video class
2018-01-07 02:39:24 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
headers = HTTP::Headers.new
headers["Cookie"] = env.request.headers["Cookie"]
Add search
2017-12-30 21:21:43 +00:00
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
html = YT_POOL.client &.get("/subscription_manager?disable_polymer=1", headers)
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
Upgrade to crystal 1.0.0
2021-05-24 13:45:50 +00:00
cookies = HTTP::Cookies.from_client_headers(headers)
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
html.cookies.each do |cookie|
if {"VISITOR_INFO1_LIVE", "YSC", "SIDCC"}.includes? cookie.name
if cookies[cookie.name]?
cookies[cookie.name] = cookie
else
cookies << cookie
end
end
end
headers = cookies.add_request_headers(headers)
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
if match = html.body.match(/'XSRF_TOKEN': "(?<session_token>[^"]+)"/)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
session_token = match["session_token"]
else
next env.redirect referer
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
headers["content-type"] = "application/x-www-form-urlencoded"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
channel_req["session_token"] = session_token
Add referer redirect
2018-04-08 02:36:09 +00:00
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
subs = XML.parse_html(html.body)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
subs.xpath_nodes(%q(//a[@class="subscription-title yt-uix-sessionlink"]/@href)).each do |channel|
channel_id = channel.content.lstrip("/channel/").not_nil!
channel_req["channel_id"] = channel_id
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
YT_POOL.client &.post("/subscription_ajax?action_update_subscription_preferences=1", headers, form: channel_req)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
end
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
if redirect
env.redirect referer
else
env.response.content_type = "application/json"
"{}"
end
end
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
post "/subscription_ajax" do |env|
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env, "/")
redirect = env.params.query["redirect"]?
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
redirect ||= "true"
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
redirect = redirect == "true"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
if !user
Add authentication API
2019-04-18 21:23:50 +00:00
if redirect
next env.redirect referer
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "No such user")
Add authentication API
2019-04-18 21:23:50 +00:00
end
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
end
user = user.as(User)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
sid = sid.as(String)
Add authentication API
2019-04-18 21:23:50 +00:00
token = env.params.body["csrf_token"]?
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
begin
Add authentication API
2019-04-18 21:23:50 +00:00
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
rescue ex
if redirect
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, ex)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
end
end
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
Minor refactor
2019-06-08 00:56:41 +00:00
if env.params.query["action_create_subscription_to_channel"]?.try &.to_i?.try &.== 1
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
action = "action_create_subscription_to_channel"
Minor refactor
2019-06-08 00:56:41 +00:00
elsif env.params.query["action_remove_subscriptions"]?.try &.to_i?.try &.== 1
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
action = "action_remove_subscriptions"
else
next env.redirect referer
end
channel_id = env.params.query["c"]?
channel_id ||= ""
if !user.password
Support 'sort_by' in reddit /api/v1/comments
2019-04-14 23:08:00 +00:00
# Sync subscriptions with YouTube
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
subscribe_ajax(channel_id, action, env.request.headers)
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
end
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
email = user.email
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
case action
Minor refactor
2019-06-08 00:56:41 +00:00
when "action_create_subscription_to_channel"
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
if !user.subscriptions.includes? channel_id
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
get_channel(channel_id, PG_DB, false, false)
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = array_append(subscriptions, $1) WHERE email = $2", channel_id, email)
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
end
Minor refactor
2019-06-08 00:56:41 +00:00
when "action_remove_subscriptions"
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = array_remove(subscriptions, $1) WHERE email = $2", channel_id, email)
Fix warnings in latest version of Crystal
2020-04-09 17:18:09 +00:00
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "Unsupported action #{action}")
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
end
if redirect
env.redirect referer
else
env.response.content_type = "application/json"
"{}"
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add TFA
2018-04-29 14:40:33 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/subscription_manager" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
user = env.get? "user"
Store session_ids in separate table
2019-02-10 18:33:29 +00:00
sid = env.get? "sid"
Add authentication API
2019-04-18 21:23:50 +00:00
referer = get_referer(env)
Don't redirect to current page
2018-08-09 01:26:02 +00:00
Add authentication API
2019-04-18 21:23:50 +00:00
if !user
Don't redirect to current page
2018-08-09 01:26:02 +00:00
next env.redirect referer
General cleanup
2018-04-28 14:27:05 +00:00
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
user = user.as(User)
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
if !user.password
# Refresh account
headers = HTTP::Headers.new
headers["Cookie"] = env.request.headers["Cookie"]
Add referer redirect
2018-04-08 02:36:09 +00:00
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user, sid = get_user(sid, headers, PG_DB)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
action_takeout = env.params.query["action_takeout"]?.try &.to_i?
action_takeout ||= 0
action_takeout = action_takeout == 1
Add separate user accounts
2018-07-18 19:26:02 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
format = env.params.query["format"]?
format ||= "rss"
Add separate user accounts
2018-07-18 19:26:02 +00:00
Add '/api/v1/auth/subscriptions'
2019-04-22 15:40:29 +00:00
if user.subscriptions.empty?
values = "'{}'"
else
values = "VALUES #{user.subscriptions.map { |id| %(('#{id}')) }.join(",")}"
end
subscriptions = PG_DB.query_all("SELECT * FROM channels WHERE id = ANY(#{values})", as: InvidiousChannel)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
subscriptions.sort_by! { |channel| channel.author.downcase }
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
if action_takeout
if format == "json"
env.response.content_type = "application/json"
env.response.headers["content-disposition"] = "attachment"
Add playlist import/export
2020-07-25 17:30:28 +00:00
playlists = PG_DB.query_all("SELECT * FROM playlists WHERE author = $1 AND id LIKE 'IV%' ORDER BY created", user.email, as: InvidiousPlaylist)
next JSON.build do |json|
json.object do
json.field "subscriptions", user.subscriptions
json.field "watch_history", user.watched
json.field "preferences", user.preferences
json.field "playlists" do
json.array do
playlists.each do |playlist|
json.object do
json.field "title", playlist.title
json.field "description", html_to_content(playlist.description_html)
json.field "privacy", playlist.privacy.to_s
json.field "videos" do
json.array do
Fix playlist export for playlists with more than 100 videos
2020-07-28 21:21:39 +00:00
PG_DB.query_all("SELECT id FROM playlist_videos WHERE plid = $1 ORDER BY array_position($2, index) LIMIT 500", playlist.id, playlist.index, as: String).each do |video_id|
json.string video_id
Add playlist import/export
2020-07-25 17:30:28 +00:00
end
end
end
end
end
end
end
end
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
else
env.response.content_type = "application/xml"
env.response.headers["content-disposition"] = "attachment"
export = XML.build do |xml|
xml.element("opml", version: "1.1") do
xml.element("body") do
if format == "newpipe"
title = "YouTube Subscriptions"
else
title = "Invidious Subscriptions"
end
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
xml.element("outline", text: title, title: title) do
subscriptions.each do |channel|
if format == "newpipe"
xmlUrl = "https://www.youtube.com/feeds/videos.xml?channel_id=#{channel.id}"
else
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
xmlUrl = "#{HOST_URL}/feed/channel/#{channel.id}"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
xml.element("outline", text: channel.author, title: channel.author,
"type": "rss", xmlUrl: xmlUrl)
end
end
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
end
Add login page
2018-03-16 16:40:29 +00:00
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
next export.gsub(%(<?xml version="1.0"?>\n), "")
end
end
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
templated "subscription_manager"
end
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/data_control" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
user = env.get? "user"
Don't redirect to current page
2018-08-09 01:26:02 +00:00
referer = get_referer(env)
Add login page
2018-03-16 16:40:29 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
templated "data_control"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
post "/data_control" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
user = env.get? "user"
Don't redirect to current page
2018-08-09 01:26:02 +00:00
referer = get_referer(env)
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
if user
user = user.as(User)
Add TFA
2018-04-29 14:40:33 +00:00
Fix warnings with latest version of Crystal
2020-07-26 14:58:50 +00:00
# TODO: Find a way to prevent browser timeout
Try to prevent timeout in /data_control
2019-04-25 01:18:35 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
HTTP::FormData.parse(env.request) do |part|
body = part.body.gets_to_end
Add playlist import/export
2020-07-25 17:30:28 +00:00
next if body.empty?
Add support for incorrect password on /login
2018-04-17 22:54:33 +00:00
Fix warnings in latest version of Crystal
2020-04-09 17:18:09 +00:00
# TODO: Unify into single import based on content-type
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
case part.name
when "import_invidious"
body = JSON.parse(body)
Add fix to switch to supported authentication schemes
2018-07-26 15:20:15 +00:00
Clean up data import/export
2018-11-09 23:25:24 +00:00
if body["subscriptions"]?
user.subscriptions += body["subscriptions"].as_a.map { |a| a.as_s }
user.subscriptions.uniq!
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user.subscriptions = get_batch_channels(user.subscriptions, PG_DB, false, false)
Clean up data import/export
2018-11-09 23:25:24 +00:00
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = $1 WHERE email = $2", user.subscriptions, user.email)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add fix to switch to supported authentication schemes
2018-07-26 15:20:15 +00:00
Speed up importing watch history
2018-11-08 22:43:28 +00:00
if body["watch_history"]?
Clean up data import/export
2018-11-09 23:25:24 +00:00
user.watched += body["watch_history"].as_a.map { |a| a.as_s }
user.watched.uniq!
PG_DB.exec("UPDATE users SET watched = $1 WHERE email = $2", user.watched, user.email)
Add fix to switch to supported authentication schemes
2018-07-26 15:20:15 +00:00
end
Add TFA
2018-04-29 14:40:33 +00:00
Add support for partial data restore
2018-11-08 22:35:26 +00:00
if body["preferences"]?
Fix warnings with latest version of Crystal
2020-07-26 14:58:50 +00:00
user.preferences = Preferences.from_json(body["preferences"].to_json)
Clean up data import/export
2018-11-09 23:25:24 +00:00
PG_DB.exec("UPDATE users SET preferences = $1 WHERE email = $2", user.preferences.to_json, user.email)
Add support for partial data restore
2018-11-08 22:35:26 +00:00
end
Add playlist import/export
2020-07-25 17:30:28 +00:00
if playlists = body["playlists"]?.try &.as_a?
playlists.each do |item|
title = item["title"]?.try &.as_s?.try &.delete("<>")
description = item["description"]?.try &.as_s?.try &.delete("\r")
privacy = item["privacy"]?.try &.as_s?.try { |privacy| PlaylistPrivacy.parse? privacy }
next if !title
next if !description
next if !privacy
playlist = create_playlist(PG_DB, title, privacy, user)
PG_DB.exec("UPDATE playlists SET description = $1 WHERE id = $2", description, playlist.id)
Fix playlist export for playlists with more than 100 videos
2020-07-28 21:21:39 +00:00
videos = item["videos"]?.try &.as_a?.try &.each_with_index do |video_id, idx|
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
raise InfoException.new("Playlist cannot have more than 500 videos") if idx > 500
Fix playlist export for playlists with more than 100 videos
2020-07-28 21:21:39 +00:00
Add playlist import/export
2020-07-25 17:30:28 +00:00
video_id = video_id.try &.as_s?
next if !video_id
begin
video = get_video(video_id, PG_DB)
rescue ex
next
end
Fix warnings with latest version of Crystal
2020-07-26 14:58:50 +00:00
playlist_video = PlaylistVideo.new({
title: video.title,
id: video.id,
author: video.author,
ucid: video.ucid,
Add playlist import/export
2020-07-25 17:30:28 +00:00
length_seconds: video.length_seconds,
Fix warnings with latest version of Crystal
2020-07-26 14:58:50 +00:00
published: video.published,
plid: playlist.id,
live_now: video.live_now,
index: Random::Secure.rand(0_i64..Int64::MAX),
})
Add playlist import/export
2020-07-25 17:30:28 +00:00
video_array = playlist_video.to_a
args = arg_array(video_array)
PG_DB.exec("INSERT INTO playlist_videos VALUES (#{args})", args: video_array)
Fix stale playlist video counting on add/drop (https://github.com/iv-org/invidious/issues/1448) The return of `cardinality()` is not affected by the `array_append()` and `array_remove()` of the same statement.
2020-11-01 04:14:46 +00:00
PG_DB.exec("UPDATE playlists SET index = array_append(index, $1), video_count = cardinality(index) + 1, updated = $2 WHERE id = $3", playlist_video.index, Time.utc, playlist.id)
Add playlist import/export
2020-07-25 17:30:28 +00:00
end
end
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
when "import_youtube"
format invidious.cr using crystal cli
2021-02-27 20:59:09 +00:00
if body[0..4] == "<opml"
integrate legacy yt xml subscription import with new json sub import
2021-02-27 17:58:55 +00:00
subscriptions = XML.parse(body)
user.subscriptions += subscriptions.xpath_nodes(%q(//outline[@type="rss"])).map do |channel|
channel["xmlUrl"].match(/UC[a-zA-Z0-9_-]{22}/).not_nil![0]
end
else
subscriptions = JSON.parse(body)
user.subscriptions += subscriptions.as_a.compact_map do |entry|
entry["snippet"]["resourceId"]["channelId"].as_s
fixed line indent typo
2021-02-27 18:12:01 +00:00
end
Clean up data import/export
2018-11-09 23:25:24 +00:00
end
user.subscriptions.uniq!
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user.subscriptions = get_batch_channels(user.subscriptions, PG_DB, false, false)
Prevent duplicate subscriptions when importing user data
2018-10-06 23:19:47 +00:00
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = $1 WHERE email = $2", user.subscriptions, user.email)
Clean up data import/export
2018-11-09 23:25:24 +00:00
when "import_freetube"
user.subscriptions += body.scan(/"channelId":"(?<channel_id>[a-zA-Z0-9_-]{24})"/).map do |md|
md["channel_id"]
end
user.subscriptions.uniq!
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user.subscriptions = get_batch_channels(user.subscriptions, PG_DB, false, false)
Clean up data import/export
2018-11-09 23:25:24 +00:00
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = $1 WHERE email = $2", user.subscriptions, user.email)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
when "import_newpipe_subscriptions"
body = JSON.parse(body)
Add support for 'user' URLs in NewPipe import
2019-04-22 20:39:57 +00:00
user.subscriptions += body["subscriptions"].as_a.compact_map do |channel|
if match = channel["url"].as_s.match(/\/channel\/(?<channel>UC[a-zA-Z0-9_-]{22})/)
next match["channel"]
elsif match = channel["url"].as_s.match(/\/user\/(?<user>.+)/)
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
response = YT_POOL.client &.get("/user/#{match["user"]}?disable_polymer=1&hl=en&gl=US")
Fix channel redirect
2020-01-14 13:21:17 +00:00
html = XML.parse_html(response.body)
ucid = html.xpath_node(%q(//link[@rel="canonical"])).try &.["href"].split("/")[-1]
next ucid if ucid
Add support for 'user' URLs in NewPipe import
2019-04-22 20:39:57 +00:00
end
nil
Clean up data import/export
2018-11-09 23:25:24 +00:00
end
user.subscriptions.uniq!
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user.subscriptions = get_batch_channels(user.subscriptions, PG_DB, false, false)
Clean up data import/export
2018-11-09 23:25:24 +00:00
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = $1 WHERE email = $2", user.subscriptions, user.email)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
when "import_newpipe"
Support Crystal 0.35.0
2020-06-15 22:57:20 +00:00
Compress::Zip::Reader.open(IO::Memory.new(body)) do |file|
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
file.each_entry do |entry|
if entry.filename == "newpipe.db"
Properly parse NewPipe imports
2018-11-21 23:12:13 +00:00
tempfile = File.tempfile(".db")
File.write(tempfile.path, entry.io.gets_to_end)
db = DB.open("sqlite3://" + tempfile.path)
Add TFA
2018-04-29 14:40:33 +00:00
Properly parse NewPipe imports
2018-11-21 23:12:13 +00:00
user.watched += db.query_all("SELECT url FROM streams", as: String).map { |url| url.lchop("https://www.youtube.com/watch?v=") }
Clean up data import/export
2018-11-09 23:25:24 +00:00
user.watched.uniq!
Add separate user accounts
2018-07-18 19:26:02 +00:00
Clean up data import/export
2018-11-09 23:25:24 +00:00
PG_DB.exec("UPDATE users SET watched = $1 WHERE email = $2", user.watched, user.email)
Prevent duplicate subscriptions when importing user data
2018-10-06 23:19:47 +00:00
Properly parse NewPipe imports
2018-11-21 23:12:13 +00:00
user.subscriptions += db.query_all("SELECT url FROM subscriptions", as: String).map { |url| url.lchop("https://www.youtube.com/channel/") }
Clean up data import/export
2018-11-09 23:25:24 +00:00
user.subscriptions.uniq!
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user.subscriptions = get_batch_channels(user.subscriptions, PG_DB, false, false)
Clean up data import/export
2018-11-09 23:25:24 +00:00
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = $1 WHERE email = $2", user.subscriptions, user.email)
Properly parse NewPipe imports
2018-11-21 23:12:13 +00:00
db.close
tempfile.delete
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
end
Add support for Google Authenticator codes
2018-07-08 13:57:06 +00:00
end
Fix warnings in latest version of Crystal
2020-04-09 17:18:09 +00:00
else nil # Ignore
Add separate user accounts
2018-07-18 19:26:02 +00:00
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
env.redirect referer
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
Add option to change passwords
2019-04-22 15:18:17 +00:00
get "/change_password" do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env)
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
Add option to change passwords
2019-04-22 15:18:17 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
csrf_token = generate_response(sid, {":change_password"}, HMAC_KEY, PG_DB)
templated "change_password"
Add option to change passwords
2019-04-22 15:18:17 +00:00
end
post "/change_password" do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env)
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
token = env.params.body["csrf_token"]?
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
# We don't store passwords for Google accounts
if !user.password
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, "Cannot change password for Google accounts")
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
begin
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
password = env.params.body["password"]?
if !password
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(401, "Password is a required field")
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
new_passwords = env.params.body.select { |k, v| k.match(/^new_password\[\d+\]$/) }.map { |k, v| v }
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if new_passwords.size <= 1 || new_passwords.uniq.size != 1
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, "New passwords must match")
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
new_password = new_passwords.uniq[0]
if new_password.empty?
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(401, "Password cannot be empty")
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if new_password.bytesize > 55
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, "Password cannot be longer than 55 characters")
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if !Crypto::Bcrypt::Password.new(user.password.not_nil!).verify(password.byte_slice(0, 55))
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(401, "Incorrect password")
Add option to change passwords
2019-04-22 15:18:17 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
new_password = Crypto::Bcrypt::Password.create(new_password, cost: 10)
PG_DB.exec("UPDATE users SET password = $1 WHERE email = $2", new_password.to_s, user.email)
Add option to change passwords
2019-04-22 15:18:17 +00:00
env.redirect referer
end
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
get "/delete_account" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
user = env.get? "user"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
sid = env.get? "sid"
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
referer = get_referer(env)
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
csrf_token = generate_response(sid, {":delete_account"}, HMAC_KEY, PG_DB)
templated "delete_account"
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
end
post "/delete_account" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
user = env.get? "user"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
sid = env.get? "sid"
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
referer = get_referer(env)
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
end
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
token = env.params.body["csrf_token"]?
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
begin
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
view_name = "subscriptions_#{sha256(user.email)}"
PG_DB.exec("DELETE FROM users * WHERE email = $1", user.email)
PG_DB.exec("DELETE FROM session_ids * WHERE email = $1", user.email)
PG_DB.exec("DROP MATERIALIZED VIEW #{view_name}")
env.request.cookies.each do |cookie|
cookie.expires = Time.utc(1990, 1, 1)
env.response.cookies << cookie
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
end
env.redirect referer
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/clear_watch_history" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
user = env.get? "user"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
sid = env.get? "sid"
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
referer = get_referer(env)
Don't redirect to current page
2018-08-09 01:26:02 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
csrf_token = generate_response(sid, {":clear_watch_history"}, HMAC_KEY, PG_DB)
templated "clear_watch_history"
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
end
post "/clear_watch_history" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
user = env.get? "user"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
sid = env.get? "sid"
Don't redirect to current page
2018-08-09 01:26:02 +00:00
referer = get_referer(env)
Add login page
2018-03-16 16:40:29 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
end
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
token = env.params.body["csrf_token"]?
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
begin
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
PG_DB.exec("UPDATE users SET watched = '{}' WHERE email = $1", user.email)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
env.redirect referer
end
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
get "/authorize_token" do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env)
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
end
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
csrf_token = generate_response(sid, {":authorize_token"}, HMAC_KEY, PG_DB)
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
scopes = env.params.query["scopes"]?.try &.split(",")
scopes ||= [] of String
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
callback_url = env.params.query["callback_url"]?
if callback_url
callback_url = URI.parse(callback_url)
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
expire = env.params.query["expire"]?.try &.to_i?
templated "authorize_token"
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
end
Add authentication API
2019-04-18 21:23:50 +00:00
post "/authorize_token" do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env)
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
end
Add authentication API
2019-04-18 21:23:50 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
user = env.get("user").as(User)
sid = sid.as(String)
token = env.params.body["csrf_token"]?
Add authentication API
2019-04-18 21:23:50 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
begin
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add authentication API
2019-04-18 21:23:50 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
scopes = env.params.body.select { |k, v| k.match(/^scopes\[\d+\]$/) }.map { |k, v| v }
callback_url = env.params.body["callbackUrl"]?
expire = env.params.body["expire"]?.try &.to_i?
Add authentication API
2019-04-18 21:23:50 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
access_token = generate_token(user.email, scopes, expire, HMAC_KEY, PG_DB)
Add authentication API
2019-04-18 21:23:50 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if callback_url
Update to Crystal 0.31.0, resolve compiler deprecation warnings, update dependencies (#764) * shard: update to crystal 0.31.0 Additionally, no longer use the Crystal "markdown" library which has been removed from the Crystal stdlib in version 0.31.0. See https://github.com/crystal-lang/crystal/pull/8115. Also fix some deprecation warnings using the following commands: find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.escape/URI\.encode_www_form/g' "{}" \; find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.unescape/URI\.decode_www_form/g' "{}" \; sed -i 's/while \%pull\.kind \!\= \:end_object/until \%pull\.kind\.end_object\?/g' src/invidious/helpers/patch_mapping.cr
2019-09-24 17:31:33 +00:00
access_token = URI.encode_www_form(access_token)
Minor refactor
2019-07-13 02:00:50 +00:00
url = URI.parse(callback_url)
Add authentication API
2019-04-18 21:23:50 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if url.query
query = HTTP::Params.parse(url.query.not_nil!)
Add authentication API
2019-04-18 21:23:50 +00:00
else
Minor refactor
2019-07-13 02:00:50 +00:00
query = HTTP::Params.new
Add authentication API
2019-04-18 21:23:50 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
query["token"] = access_token
url.query = query.to_s
env.redirect url.to_s
else
csrf_token = ""
env.set "access_token", access_token
templated "authorize_token"
Add authentication API
2019-04-18 21:23:50 +00:00
end
end
get "/token_manager" do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env, "/subscription_manager")
if !user
next env.redirect referer
end
user = user.as(User)
tokens = PG_DB.query_all("SELECT id, issued FROM session_ids WHERE email = $1 ORDER BY issued DESC", user.email, as: {session: String, issued: Time})
templated "token_manager"
end
post "/token_ajax" do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env)
redirect = env.params.query["redirect"]?
redirect ||= "true"
redirect = redirect == "true"
if !user
if redirect
next env.redirect referer
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "No such user")
Add authentication API
2019-04-18 21:23:50 +00:00
end
end
user = user.as(User)
sid = sid.as(String)
token = env.params.body["csrf_token"]?
begin
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
rescue ex
if redirect
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Add authentication API
2019-04-18 21:23:50 +00:00
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, ex)
Add authentication API
2019-04-18 21:23:50 +00:00
end
end
if env.params.query["action_revoke_token"]?
action = "action_revoke_token"
else
next env.redirect referer
end
session = env.params.query["session"]?
session ||= ""
case action
when .starts_with? "action_revoke_token"
PG_DB.exec("DELETE FROM session_ids * WHERE id = $1 AND email = $2", session, user.email)
Fix warnings in latest version of Crystal
2020-04-09 17:18:09 +00:00
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "Unsupported action #{action}")
Add authentication API
2019-04-18 21:23:50 +00:00
end
if redirect
env.redirect referer
else
env.response.content_type = "application/json"
"{}"
end
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
# Channels
Add support for '/c/:user/live'
2019-04-28 16:47:16 +00:00
{"/channel/:ucid/live", "/user/:user/live", "/c/:user/live"}.each do |route|
get route do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
# Appears to be a bug in routing, having several routes configured
# as `/a/:a`, `/b/:a`, `/c/:a` results in 404
value = env.request.resource.split("/")[2]
body = ""
{"channel", "user", "c"}.each do |type|
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
response = YT_POOL.client &.get("/#{type}/#{value}/live?disable_polymer=1")
Add support for '/c/:user/live'
2019-04-28 16:47:16 +00:00
if response.status_code == 200
body = response.body
end
end
video_id = body.match(/'VIDEO_ID': "(?<id>[a-zA-Z0-9_-]{11})"/).try &.["id"]?
if video_id
params = [] of String
env.params.query.each do |k, v|
params << "#{k}=#{v}"
end
params = params.join("&")
url = "/watch?v=#{video_id}"
if !params.empty?
url += "&#{params}"
end
env.redirect url
else
env.redirect "/channel/#{value}"
end
end
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
# API Endpoints
Add channel RSS
2018-07-28 13:24:53 +00:00
Add /api/v1/stats
2019-03-02 01:25:16 +00:00
get "/api/v1/stats" do |env|
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add /api/v1/stats
2019-03-02 01:25:16 +00:00
env.response.content_type = "application/json"
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if !CONFIG.statistics_enabled
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "Statistics are not enabled.")
Add /api/v1/stats
2019-03-02 01:25:16 +00:00
end
[refactor] Finish converting jobs to new job setup (#1420) [refactor] Finish converting jobs to new job setup
2020-10-17 12:25:57 +00:00
Invidious::Jobs::StatisticsRefreshJob::STATISTICS.to_json
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
end
Add /api/v1/stats
2019-03-02 01:25:16 +00:00
Fix content-type for captions
2019-05-21 01:22:01 +00:00
# YouTube provides "storyboards", which are sprites containing x * y
Add video previews
2019-05-02 19:20:19 +00:00
# preview thumbnails for individual scenes in a video.
# See https://support.jwplayer.com/articles/how-to-add-preview-thumbnails
get "/api/v1/storyboards/:id" do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
env.response.content_type = "application/json"
id = env.params.url["id"]
region = env.params.query["region"]?
begin
Refactor proxy_list into global
2019-06-29 02:17:56 +00:00
video = get_video(id, PG_DB, region: region)
Add video previews
2019-05-02 19:20:19 +00:00
rescue ex : VideoRedirect
Add support for channel redirects
2019-09-08 16:08:59 +00:00
env.response.headers["Location"] = env.request.resource.gsub(id, ex.video_id)
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(302, "Video is unavailable", {"videoId" => ex.video_id})
Add video previews
2019-05-02 19:20:19 +00:00
rescue ex
env.response.status_code = 500
next
end
storyboards = video.storyboards
width = env.params.query["width"]?
height = env.params.query["height"]?
if !width && !height
response = JSON.build do |json|
json.object do
json.field "storyboards" do
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
generate_storyboards(json, id, storyboards)
Add video previews
2019-05-02 19:20:19 +00:00
end
end
end
next response
end
env.response.content_type = "text/vtt"
storyboard = storyboards.select { |storyboard| width == "#{storyboard[:width]}" || height == "#{storyboard[:height]}" }
if storyboard.empty?
env.response.status_code = 404
next
else
storyboard = storyboard[0]
end
Minor refactor
2019-08-27 13:08:26 +00:00
String.build do |str|
str << <<-END_VTT
WEBVTT
Add video previews
2019-05-02 19:20:19 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
END_VTT
Add video previews
2019-05-02 19:20:19 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
start_time = 0.milliseconds
end_time = storyboard[:interval].milliseconds
Add video previews
2019-05-02 19:20:19 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
storyboard[:storyboard_count].times do |i|
Fix storyboards
2020-10-25 08:35:16 +00:00
url = storyboard[:url]
authority = /(i\d?).ytimg.com/.match(url).not_nil![1]?
Improve storyboards Instead of limiting the width of storyboards through CSS it is now done in the VTT directly.
2020-12-21 16:02:36 +00:00
url = url.gsub("$M", i).gsub(%r(https://i\d?.ytimg.com/sb/), "")
Fix storyboards
2020-10-25 08:35:16 +00:00
url = "#{HOST_URL}/sb/#{authority}/#{url}"
Add video previews
2019-05-02 19:20:19 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
storyboard[:storyboard_height].times do |j|
storyboard[:storyboard_width].times do |k|
str << <<-END_CUE
#{start_time}.000 --> #{end_time}.000
Improve storyboards Instead of limiting the width of storyboards through CSS it is now done in the VTT directly.
2020-12-21 16:02:36 +00:00
#{url}#xywh=#{storyboard[:width] * k},#{storyboard[:height] * j},#{storyboard[:width] - 2},#{storyboard[:height]}
Add video previews
2019-05-02 19:20:19 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
END_CUE
Add video previews
2019-05-02 19:20:19 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
start_time += storyboard[:interval].milliseconds
end_time += storyboard[:interval].milliseconds
end
Add video previews
2019-05-02 19:20:19 +00:00
end
end
end
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/api/v1/captions/:id" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add support for proxying comments
2018-09-26 23:44:37 +00:00
env.response.content_type = "application/json"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
id = env.params.url["id"]
Add 'region' parameter to captions and manifest endpoints
2018-11-17 23:37:57 +00:00
region = env.params.query["region"]?
Add fix for nonexistent channels
2018-07-30 02:01:28 +00:00
Fix content-type for captions
2019-05-21 01:22:01 +00:00
# See https://github.com/ytdl-org/youtube-dl/blob/6ab30ff50bf6bd0585927cb73c7421bef184f87a/youtube_dl/extractor/youtube.py#L1354
# It is possible to use `/api/timedtext?type=list&v=#{id}` and
# `/api/timedtext?type=track&v=#{id}&lang=#{lang_code}` directly,
# but this does not provide links for auto-generated captions.
#
# In future this should be investigated as an alternative, since it does not require
# getting video info.
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
begin
Refactor proxy_list into global
2019-06-29 02:17:56 +00:00
video = get_video(id, PG_DB, region: region)
Add basic handling for (almost) valid video URLs
2018-10-07 03:22:22 +00:00
rescue ex : VideoRedirect
Add support for channel redirects
2019-09-08 16:08:59 +00:00
env.response.headers["Location"] = env.request.resource.gsub(id, ex.video_id)
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(302, "Video is unavailable", {"videoId" => ex.video_id})
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
rescue ex
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
env.response.status_code = 500
next
Add channel RSS
2018-07-28 13:24:53 +00:00
end
Major cleanup
2018-08-05 04:07:38 +00:00
captions = video.captions
Add channel RSS
2018-07-28 13:24:53 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
label = env.params.query["label"]?
Add 'lang' and 'tlang' to '/api/v1/captions'
2018-09-30 15:13:07 +00:00
lang = env.params.query["lang"]?
tlang = env.params.query["tlang"]?
if !label && !lang
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
response = JSON.build do |json|
json.object do
json.field "captions" do
json.array do
Major cleanup
2018-08-05 04:07:38 +00:00
captions.each do |caption|
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
json.object do
Fix caption parsing on age restricted videos
2021-06-27 14:18:16 +00:00
json.field "label", caption.name
Add JSON mapping for captions
2018-08-06 23:25:25 +00:00
json.field "languageCode", caption.languageCode
Fix caption parsing on age restricted videos
2021-06-27 14:18:16 +00:00
json.field "url", "/api/v1/captions/#{id}?label=#{URI.encode_www_form(caption.name)}"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
end
end
end
Add channel RSS
2018-07-28 13:24:53 +00:00
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add channel RSS
2018-07-28 13:24:53 +00:00
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
next response
end
Add channel RSS
2018-07-28 13:24:53 +00:00
Fix content-type for captions
2019-05-21 01:22:01 +00:00
env.response.content_type = "text/vtt; charset=UTF-8"
Add 'lang' and 'tlang' to '/api/v1/captions'
2018-09-30 15:13:07 +00:00
if lang
caption = captions.select { |caption| caption.languageCode == lang }
Minor refactor
2020-01-09 01:27:21 +00:00
else
Fix caption parsing on age restricted videos
2021-06-27 14:18:16 +00:00
caption = captions.select { |caption| caption.name == label }
Add 'lang' and 'tlang' to '/api/v1/captions'
2018-09-30 15:13:07 +00:00
end
Major cleanup
2018-08-05 04:07:38 +00:00
if caption.empty?
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
env.response.status_code = 404
next
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
else
Major cleanup
2018-08-05 04:07:38 +00:00
caption = caption[0]
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add channel RSS
2018-07-28 13:24:53 +00:00
Update code for Crystal 0.36.0 Rename `HTTPClient@socket` to `HTTPClient@io`, see https://github.com/crystal-lang/crystal/pull/9543. Rename `URI#full_path` to `URI#request_target`, see https://github.com/crystal-lang/crystal/pull/10099.
2021-01-31 18:52:32 +00:00
url = URI.parse("#{caption.baseUrl}&tlang=#{tlang}").request_target
Add channel RSS
2018-07-28 13:24:53 +00:00
Only modify cues for auto-generated captions
2019-05-19 01:27:19 +00:00
# Auto-generated captions often have cues that aren't aligned properly with the video,
# as well as some other markup that makes it cumbersome, so we try to fix that here
Fix caption parsing on age restricted videos
2021-06-27 14:18:16 +00:00
if caption.name.includes? "auto-generated"
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
caption_xml = YT_POOL.client &.get(url).body
Only modify cues for auto-generated captions
2019-05-19 01:27:19 +00:00
caption_xml = XML.parse(caption_xml)
Add channel RSS
2018-07-28 13:24:53 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
webvtt = String.build do |str|
str << <<-END_VTT
WEBVTT
Kind: captions
Language: #{tlang || caption.languageCode}
Add channel RSS
2018-07-28 13:24:53 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
END_VTT
Fix duration for auto-generated captions
2018-08-07 12:36:51 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
caption_nodes = caption_xml.xpath_nodes("//transcript/text")
caption_nodes.each_with_index do |node, i|
start_time = node["start"].to_f.seconds
duration = node["dur"]?.try &.to_f.seconds
duration ||= start_time
Only modify cues for auto-generated captions
2019-05-19 01:27:19 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
if caption_nodes.size > i + 1
end_time = caption_nodes[i + 1]["start"].to_f.seconds
else
end_time = start_time + duration
end
Add channel RSS
2018-07-28 13:24:53 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
start_time = "#{start_time.hours.to_s.rjust(2, '0')}:#{start_time.minutes.to_s.rjust(2, '0')}:#{start_time.seconds.to_s.rjust(2, '0')}.#{start_time.milliseconds.to_s.rjust(3, '0')}"
end_time = "#{end_time.hours.to_s.rjust(2, '0')}:#{end_time.minutes.to_s.rjust(2, '0')}:#{end_time.seconds.to_s.rjust(2, '0')}.#{end_time.milliseconds.to_s.rjust(3, '0')}"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
text = HTML.unescape(node.content)
text = text.gsub(/<font color="#[a-fA-F0-9]{6}">/, "")
text = text.gsub(/<\/font>/, "")
if md = text.match(/(?<name>.*) : (?<text>.*)/)
text = "<v #{md["name"]}>#{md["text"]}</v>"
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
str << <<-END_CUE
#{start_time} --> #{end_time}
#{text}
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
Minor refactor
2019-08-27 13:08:26 +00:00
END_CUE
end
Only modify cues for auto-generated captions
2019-05-19 01:27:19 +00:00
end
else
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
webvtt = YT_POOL.client &.get("#{url}&format=vtt").body
Add channel RSS
2018-07-28 13:24:53 +00:00
end
Add subtitles to download widget
2019-04-11 17:08:43 +00:00
if title = env.params.query["title"]?
# https://blog.fastmail.com/2011/06/24/download-non-english-filenames/
Update to Crystal 0.31.0, resolve compiler deprecation warnings, update dependencies (#764) * shard: update to crystal 0.31.0 Additionally, no longer use the Crystal "markdown" library which has been removed from the Crystal stdlib in version 0.31.0. See https://github.com/crystal-lang/crystal/pull/8115. Also fix some deprecation warnings using the following commands: find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.escape/URI\.encode_www_form/g' "{}" \; find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.unescape/URI\.decode_www_form/g' "{}" \; sed -i 's/while \%pull\.kind \!\= \:end_object/until \%pull\.kind\.end_object\?/g' src/invidious/helpers/patch_mapping.cr
2019-09-24 17:31:33 +00:00
env.response.headers["Content-Disposition"] = "attachment; filename=\"#{URI.encode_www_form(title)}\"; filename*=UTF-8''#{URI.encode_www_form(title)}"
Add subtitles to download widget
2019-04-11 17:08:43 +00:00
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
webvtt
Add channel RSS
2018-07-28 13:24:53 +00:00
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/api/v1/comments/:id" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Use get_video for pulling comment token
2019-02-06 23:55:22 +00:00
region = env.params.query["region"]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add support for proxying comments
2018-09-26 23:44:37 +00:00
env.response.content_type = "application/json"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
id = env.params.url["id"]
Add Atom feed
2018-07-20 16:19:49 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
source = env.params.query["source"]?
source ||= "youtube"
Add Atom feed
2018-07-20 16:19:49 +00:00
Fix thin mode
2019-03-27 16:31:05 +00:00
thin_mode = env.params.query["thin_mode"]?
thin_mode = thin_mode == "true"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
format = env.params.query["format"]?
format ||= "json"
Add Atom feed
2018-07-20 16:19:49 +00:00
Add option to view comments with JS disabled
2018-10-31 21:47:53 +00:00
continuation = env.params.query["continuation"]?
Support 'sort_by' in reddit /api/v1/comments
2019-04-14 23:08:00 +00:00
sort_by = env.params.query["sort_by"]?.try &.downcase
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
Add option to view comments with JS disabled
2018-10-31 21:47:53 +00:00
if source == "youtube"
Support 'sort_by' in reddit /api/v1/comments
2019-04-14 23:08:00 +00:00
sort_by ||= "top"
Add option to view comments with JS disabled
2018-10-31 21:47:53 +00:00
begin
Use the new youtube api for comments (#2217) * use the new youtube api for comments * remove PG_DB & action parameter + allow force region * support new comments data with onResponseReceivedEndpoints
2021-08-12 17:14:30 +00:00
comments = fetch_youtube_comments(id, continuation, format, locale, thin_mode, region, sort_by: sort_by)
Add option to view comments with JS disabled
2018-10-31 21:47:53 +00:00
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Add Atom feed
2018-07-20 16:19:49 +00:00
end
Fix templating for videos with 0 comments
2018-11-10 15:05:26 +00:00
next comments
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
elsif source == "reddit"
Support 'sort_by' in reddit /api/v1/comments
2019-04-14 23:08:00 +00:00
sort_by ||= "confidence"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
begin
Support 'sort_by' in reddit /api/v1/comments
2019-04-14 23:08:00 +00:00
comments, reddit_thread = fetch_reddit_comments(id, sort_by: sort_by)
Add support for translations
2018-12-20 21:32:09 +00:00
content_html = template_reddit_comments(comments, locale)
Add enable and disable notifications (bell dinging feature)
2018-03-31 01:52:10 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
content_html = fill_links(content_html, "https", "www.reddit.com")
Replace YouTube links
2018-09-04 03:15:47 +00:00
content_html = replace_links(content_html)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
rescue ex
Add format=json for reddit comments
2018-09-06 15:19:28 +00:00
comments = nil
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
reddit_thread = nil
content_html = ""
end
Add user preferences
2018-07-16 16:24:24 +00:00
Add format=json for reddit comments
2018-09-06 15:19:28 +00:00
if !reddit_thread || !comments
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
env.response.status_code = 404
next
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add enable and disable notifications (bell dinging feature)
2018-03-31 01:52:10 +00:00
Add format=json for reddit comments
2018-09-06 15:19:28 +00:00
if format == "json"
reddit_thread = JSON.parse(reddit_thread.to_json).as_h
reddit_thread["comments"] = JSON.parse(comments.to_json)
Add 'pretty=1' option to API endpoints
2019-01-25 16:50:18 +00:00
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
next reddit_thread.to_json
Add format=json for reddit comments
2018-09-06 15:19:28 +00:00
else
Add 'pretty=1' option to API endpoints
2019-01-25 16:50:18 +00:00
response = {
Add format=json for reddit comments
2018-09-06 15:19:28 +00:00
"title" => reddit_thread.title,
Fix missing 'end'
2018-09-06 23:18:36 +00:00
"permalink" => reddit_thread.permalink,
"contentHtml" => content_html,
Add 'pretty=1' option to API endpoints
2019-01-25 16:50:18 +00:00
}
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
next response.to_json
Fix missing 'end'
2018-09-06 23:18:36 +00:00
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
end
Add enable and disable notifications (bell dinging feature)
2018-03-31 01:52:10 +00:00
Add /api/v1/insights
2018-09-18 01:08:26 +00:00
get "/api/v1/insights/:id" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(410, "YouTube has removed publicly available analytics.")
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
end
Add /api/v1/insights
2018-09-18 01:08:26 +00:00
Add '/api/v1/annotations/:id'
2019-04-01 03:07:06 +00:00
get "/api/v1/annotations/:id" do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
env.response.content_type = "text/xml"
id = env.params.url["id"]
source = env.params.query["source"]?
source ||= "archive"
if !id.match(/[a-zA-Z0-9_-]{11}/)
env.response.status_code = 400
next
end
annotations = ""
case source
when "archive"
Add config option to cache annotations from IA
2019-04-15 16:13:09 +00:00
if CONFIG.cache_annotations && (cached_annotation = PG_DB.query_one?("SELECT * FROM annotations WHERE id = $1", id, as: Annotation))
annotations = cached_annotation.annotations
else
index = CHARS_SAFE.index(id[0]).not_nil!.to_s.rjust(2, '0')
Add '/api/v1/annotations/:id'
2019-04-01 03:07:06 +00:00
Add config option to cache annotations from IA
2019-04-15 16:13:09 +00:00
# IA doesn't handle leading hyphens,
# so we use https://archive.org/details/youtubeannotations_64
if index == "62"
index = "64"
id = id.sub(/^-/, 'A')
end
Add '/api/v1/annotations/:id'
2019-04-01 03:07:06 +00:00
Update to Crystal 0.31.0, resolve compiler deprecation warnings, update dependencies (#764) * shard: update to crystal 0.31.0 Additionally, no longer use the Crystal "markdown" library which has been removed from the Crystal stdlib in version 0.31.0. See https://github.com/crystal-lang/crystal/pull/8115. Also fix some deprecation warnings using the following commands: find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.escape/URI\.encode_www_form/g' "{}" \; find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.unescape/URI\.decode_www_form/g' "{}" \; sed -i 's/while \%pull\.kind \!\= \:end_object/until \%pull\.kind\.end_object\?/g' src/invidious/helpers/patch_mapping.cr
2019-09-24 17:31:33 +00:00
file = URI.encode_www_form("#{id[0, 3]}/#{id}.xml")
Add '/api/v1/annotations/:id'
2019-04-01 03:07:06 +00:00
Close http clients after using The crystal http client maintains a keepalive connection to the other server which stays alive for some time. This should be closed if the client instance is not used again to avoid hogging resources
2020-12-23 05:52:23 +00:00
location = make_client(ARCHIVE_URL, &.get("/download/youtubeannotations_#{index}/#{id[0, 2]}.tar/#{file}"))
Add '/api/v1/annotations/:id'
2019-04-01 03:07:06 +00:00
Add config option to cache annotations from IA
2019-04-15 16:13:09 +00:00
if !location.headers["Location"]?
env.response.status_code = location.status_code
end
Add '/api/v1/annotations/:id'
2019-04-01 03:07:06 +00:00
Close http clients after using The crystal http client maintains a keepalive connection to the other server which stays alive for some time. This should be closed if the client instance is not used again to avoid hogging resources
2020-12-23 05:52:23 +00:00
response = make_client(URI.parse(location.headers["Location"]), &.get(location.headers["Location"]))
Add '/api/v1/annotations/:id'
2019-04-01 03:07:06 +00:00
Add config option to cache annotations from IA
2019-04-15 16:13:09 +00:00
if response.body.empty?
env.response.status_code = 404
next
end
Fix 404 for annotations endpoint
2019-04-13 13:28:59 +00:00
Add config option to cache annotations from IA
2019-04-15 16:13:09 +00:00
if response.status_code != 200
env.response.status_code = response.status_code
next
end
Add '/api/v1/annotations/:id'
2019-04-01 03:07:06 +00:00
Add config option to cache annotations from IA
2019-04-15 16:13:09 +00:00
annotations = response.body
cache_annotation(PG_DB, id, annotations)
end
Fix warnings in latest version of Crystal
2020-04-09 17:18:09 +00:00
else # "youtube"
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
response = YT_POOL.client &.get("/annotations_invideo?video_id=#{id}")
Add '/api/v1/annotations/:id'
2019-04-01 03:07:06 +00:00
if response.status_code != 200
env.response.status_code = response.status_code
next
end
annotations = response.body
end
Add etag to /api/v1/annotations
2019-11-10 03:05:17 +00:00
etag = sha256(annotations)[0, 16]
if env.request.headers["If-None-Match"]?.try &.== etag
env.response.status_code = 304
else
env.response.headers["ETag"] = etag
annotations
end
Add '/api/v1/annotations/:id'
2019-04-01 03:07:06 +00:00
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/api/v1/videos/:id" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add mixes
2018-09-29 04:12:35 +00:00
env.response.content_type = "application/json"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
id = env.params.url["id"]
Add 'region' parameter for bypassing region locks
2018-11-17 23:33:30 +00:00
region = env.params.query["region"]?
Add enable and disable notifications (bell dinging feature)
2018-03-31 01:52:10 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
begin
Refactor proxy_list into global
2019-06-29 02:17:56 +00:00
video = get_video(id, PG_DB, region: region)
Add basic handling for (almost) valid video URLs
2018-10-07 03:22:22 +00:00
rescue ex : VideoRedirect
Add support for channel redirects
2019-09-08 16:08:59 +00:00
env.response.headers["Location"] = env.request.resource.gsub(id, ex.video_id)
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(302, "Video is unavailable", {"videoId" => ex.video_id})
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add enable and disable notifications (bell dinging feature)
2018-03-31 01:52:10 +00:00
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
video.to_json(locale)
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
end
Add options to import and export user data
2018-07-30 17:34:57 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/api/v1/trending" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add 'pretty=1' option to API endpoints
2019-01-25 16:50:18 +00:00
env.response.content_type = "application/json"
Add /feed/trending
2018-11-20 17:18:12 +00:00
region = env.params.query["region"]?
trending_type = env.params.query["type"]?
begin
Refactor proxy_list into global
2019-06-29 02:17:56 +00:00
trending, plid = fetch_trending(trending_type, region, locale)
Add /feed/trending
2018-11-20 17:18:12 +00:00
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Add /feed/trending
2018-11-20 17:18:12 +00:00
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
videos = JSON.build do |json|
json.array do
Add /feed/trending
2018-11-20 17:18:12 +00:00
trending.each do |video|
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
video.to_json(locale, json)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Patch StaticFileHandler to serve files from memory
2019-06-23 17:54:46 +00:00
end
end
Add options to import and export user data
2018-07-30 17:34:57 +00:00
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
videos
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
Add '/api/v1/popular'
2018-11-26 00:13:11 +00:00
get "/api/v1/popular" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add 'pretty=1' option to API endpoints
2019-01-25 16:50:18 +00:00
env.response.content_type = "application/json"
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if !CONFIG.popular_enabled
Add `popular-enabled` option This is similar to the removed `top-enabled` option but for the Popular feed. The instance needs to be restarted if the feed was enabled. Editing admin options on the preferences page is also fixed. The handling of the feed pages now only happens in a single place. Instead of redirecting: - The Top feed now displays a message that it was removed from Invidious. - The Popular feed now displays a message that it was disabled if it was.
2020-12-27 05:12:43 +00:00
error_message = {"error" => "Administrator has disabled this endpoint."}.to_json
env.response.status_code = 400
next error_message
end
Add 'since' to '/api/v1/auth/notifications'
2019-04-28 23:14:16 +00:00
JSON.build do |json|
Add '/api/v1/popular'
2018-11-26 00:13:11 +00:00
json.array do
popular_videos.each do |video|
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
video.to_json(locale, json)
Add '/api/v1/popular'
2018-11-26 00:13:11 +00:00
end
end
end
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
end
Add '/api/v1/popular'
2018-11-26 00:13:11 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/api/v1/top" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add 'pretty=1' option to API endpoints
2019-01-25 16:50:18 +00:00
env.response.content_type = "application/json"
Add `popular-enabled` option This is similar to the removed `top-enabled` option but for the Popular feed. The instance needs to be restarted if the feed was enabled. Editing admin options on the preferences page is also fixed. The handling of the feed pages now only happens in a single place. Instead of redirecting: - The Top feed now displays a message that it was removed from Invidious. - The Popular feed now displays a message that it was disabled if it was.
2020-12-27 05:12:43 +00:00
env.response.status_code = 400
{"error" => "The Top feed has been removed from Invidious."}.to_json
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
end
Add options to import and export user data
2018-07-30 17:34:57 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/api/v1/channels/:ucid" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Refactor name to ucid conversion
2018-09-21 14:40:04 +00:00
env.response.content_type = "application/json"
Pull 'extract_videos' out into seperate function
2018-08-10 14:44:19 +00:00
Refactor name to ucid conversion
2018-09-21 14:40:04 +00:00
ucid = env.params.url["ucid"]
Add channel sort options
2018-11-14 01:04:25 +00:00
sort_by = env.params.query["sort_by"]?.try &.downcase
sort_by ||= "newest"
Add support for genre channels
2018-09-05 02:04:40 +00:00
Refactor name to ucid conversion
2018-09-21 14:40:04 +00:00
begin
Add thumbnail and banners to channel page
2019-06-29 01:48:24 +00:00
channel = get_about_info(ucid, locale)
Add support for channel redirects
2019-09-08 16:08:59 +00:00
rescue ex : ChannelRedirect
env.response.headers["Location"] = env.request.resource.gsub(ucid, ex.channel_id)
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(302, "Channel is unavailable", {"authorId" => ex.channel_id})
Refactor name to ucid conversion
2018-09-21 14:40:04 +00:00
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Add support for genre channels
2018-09-05 02:04:40 +00:00
end
Bump number of videos in channel resources to 60
2018-09-06 13:43:22 +00:00
page = 1
Add thumbnail and banners to channel page
2019-06-29 01:48:24 +00:00
if channel.auto_generated
Show playlists for auto-generated channels
2019-02-24 22:39:44 +00:00
videos = [] of SearchVideo
count = 0
else
begin
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
count, videos = get_60_videos(channel.ucid, channel.author, page, channel.auto_generated, sort_by)
Show playlists for auto-generated channels
2019-02-24 22:39:44 +00:00
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Show playlists for auto-generated channels
2019-02-24 22:39:44 +00:00
end
Add error message for fetching channel videos
2018-11-08 23:10:14 +00:00
end
Use '/video' page for channel endpoint
2018-08-29 01:29:08 +00:00
Add thumbnail and banners to channel page
2019-06-29 01:48:24 +00:00
JSON.build do |json|
Add 'to_json' into respective structs
2019-06-08 18:31:41 +00:00
# TODO: Refactor into `to_json` for InvidiousChannel
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
json.object do
Add thumbnail and banners to channel page
2019-06-29 01:48:24 +00:00
json.field "author", channel.author
json.field "authorId", channel.ucid
json.field "authorUrl", channel.author_url
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
json.field "authorBanners" do
json.array do
Add support for default channel banners
2019-06-30 17:59:38 +00:00
if channel.banner
qualities = {
{width: 2560, height: 424},
{width: 2120, height: 351},
{width: 1060, height: 175},
}
qualities.each do |quality|
json.object do
Add '/api/v1/channels/comments'
2019-07-02 23:53:19 +00:00
json.field "url", channel.banner.not_nil!.gsub("=w1060-", "=w#{quality[:width]}-")
Add support for default channel banners
2019-06-30 17:59:38 +00:00
json.field "width", quality[:width]
json.field "height", quality[:height]
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add support for default channel banners
2019-06-30 17:59:38 +00:00
json.object do
Add '/api/v1/channels/comments'
2019-07-02 23:53:19 +00:00
json.field "url", channel.banner.not_nil!.split("=w1060-")[0]
Add support for default channel banners
2019-06-30 17:59:38 +00:00
json.field "width", 512
json.field "height", 288
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
end
end
Add subscribe and unsubscribe button
2018-03-31 14:51:14 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
json.field "authorThumbnails" do
json.array do
Use tuples for "qualities" in API endpoints
2019-03-25 15:00:18 +00:00
qualities = {32, 48, 76, 100, 176, 512}
Add separate user accounts
2018-07-18 19:26:02 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
qualities.each do |quality|
json.object do
Fix authorThumbnails in /api/v1/channels
2020-04-10 16:49:51 +00:00
json.field "url", channel.author_thumbnail.gsub(/=s\d+/, "=s#{quality}")
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
json.field "width", quality
json.field "height", quality
end
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
end
Add subscribe and unsubscribe button
2018-03-31 14:51:14 +00:00
end
Add thumbnail and banners to channel page
2019-06-29 01:48:24 +00:00
json.field "subCount", channel.sub_count
json.field "totalViews", channel.total_views
json.field "joined", channel.joined.to_unix
Add subscribe and unsubscribe button
2018-03-31 14:51:14 +00:00
Add thumbnail and banners to channel page
2019-06-29 01:48:24 +00:00
json.field "autoGenerated", channel.auto_generated
json.field "isFamilyFriendly", channel.is_family_friendly
json.field "description", html_to_content(channel.description_html)
json.field "descriptionHtml", channel.description_html
Add descriptionHtml to playlists
2018-09-05 00:27:10 +00:00
Add thumbnail and banners to channel page
2019-06-29 01:48:24 +00:00
json.field "allowedRegions", channel.allowed_regions
Add watch history
2018-07-29 01:40:59 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
json.field "latestVideos" do
json.array do
Use '/video' page for channel endpoint
2018-08-29 01:29:08 +00:00
videos.each do |video|
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
video.to_json(locale, json)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
end
end
Add related_channels to /api/v1/channels
2018-11-28 04:07:45 +00:00
json.field "relatedChannels" do
json.array do
Add thumbnail and banners to channel page
2019-06-29 01:48:24 +00:00
channel.related_channels.each do |related_channel|
Add related_channels to /api/v1/channels
2018-11-28 04:07:45 +00:00
json.object do
Add thumbnail and banners to channel page
2019-06-29 01:48:24 +00:00
json.field "author", related_channel.author
json.field "authorId", related_channel.ucid
json.field "authorUrl", related_channel.author_url
Add related_channels to /api/v1/channels
2018-11-28 04:07:45 +00:00
json.field "authorThumbnails" do
json.array do
Use tuples for "qualities" in API endpoints
2019-03-25 15:00:18 +00:00
qualities = {32, 48, 76, 100, 176, 512}
Add related_channels to /api/v1/channels
2018-11-28 04:07:45 +00:00
qualities.each do |quality|
json.object do
Fix image url extractor
2019-08-01 00:16:09 +00:00
json.field "url", related_channel.author_thumbnail.gsub(/=\d+/, "=s#{quality}")
Add related_channels to /api/v1/channels
2018-11-28 04:07:45 +00:00
json.field "width", quality
json.field "height", quality
end
end
end
end
end
end
end
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add watch history
2018-07-29 01:40:59 +00:00
end
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
end
Organize routes
2018-07-16 13:18:59 +00:00
Add support for '/c/:user/live'
2019-04-28 16:47:16 +00:00
{"/api/v1/channels/:ucid/videos", "/api/v1/channels/videos/:ucid"}.each do |route|
Add alias '/api/v1/channels/videos/:ucid'
2018-09-18 15:47:22 +00:00
get route do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Refactor name to ucid conversion
2018-09-21 14:40:04 +00:00
env.response.content_type = "application/json"
Add support for playlists and channels in search
2018-09-20 14:36:09 +00:00
ucid = env.params.url["ucid"]
page = env.params.query["page"]?.try &.to_i?
page ||= 1
Add minor API fixes
2019-02-15 23:28:54 +00:00
sort_by = env.params.query["sort"]?.try &.downcase
sort_by ||= env.params.query["sort_by"]?.try &.downcase
Add channel sort to '/api/v1/channels/videos'
2018-11-14 01:11:16 +00:00
sort_by ||= "newest"
Organize routes
2018-07-16 13:18:59 +00:00
Refactor name to ucid conversion
2018-09-21 14:40:04 +00:00
begin
Add thumbnail and banners to channel page
2019-06-29 01:48:24 +00:00
channel = get_about_info(ucid, locale)
Add support for channel redirects
2019-09-08 16:08:59 +00:00
rescue ex : ChannelRedirect
env.response.headers["Location"] = env.request.resource.gsub(ucid, ex.channel_id)
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(302, "Channel is unavailable", {"authorId" => ex.channel_id})
Refactor name to ucid conversion
2018-09-21 14:40:04 +00:00
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Add support for playlists and channels in search
2018-09-20 14:36:09 +00:00
end
Organize routes
2018-07-16 13:18:59 +00:00
Add error message for fetching channel videos
2018-11-08 23:10:14 +00:00
begin
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
count, videos = get_60_videos(channel.ucid, channel.author, page, channel.auto_generated, sort_by)
Add error message for fetching channel videos
2018-11-08 23:10:14 +00:00
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Add error message for fetching channel videos
2018-11-08 23:10:14 +00:00
end
Add fix for nonexistent channels
2018-07-30 02:01:28 +00:00
Add 'to_json' into respective structs
2019-06-08 18:31:41 +00:00
JSON.build do |json|
Add support for playlists and channels in search
2018-09-20 14:36:09 +00:00
json.array do
videos.each do |video|
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
video.to_json(locale, json)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
end
Patch StaticFileHandler to serve files from memory
2019-06-23 17:54:46 +00:00
end
end
end
Organize routes
2018-07-16 13:18:59 +00:00
Add support for '/c/:user/live'
2019-04-28 16:47:16 +00:00
{"/api/v1/channels/:ucid/latest", "/api/v1/channels/latest/:ucid"}.each do |route|
Add `/api/v1/channels/latest/:ucid`
2019-02-19 23:00:06 +00:00
get route do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translating time intervals
2019-02-20 14:49:39 +00:00
Add `/api/v1/channels/latest/:ucid`
2019-02-19 23:00:06 +00:00
env.response.content_type = "application/json"
ucid = env.params.url["ucid"]
begin
videos = get_latest_videos(ucid)
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Add `/api/v1/channels/latest/:ucid`
2019-02-19 23:00:06 +00:00
end
Add 'since' to '/api/v1/auth/notifications'
2019-04-28 23:14:16 +00:00
JSON.build do |json|
Add `/api/v1/channels/latest/:ucid`
2019-02-19 23:00:06 +00:00
json.array do
videos.each do |video|
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
video.to_json(locale, json)
Add `/api/v1/channels/latest/:ucid`
2019-02-19 23:00:06 +00:00
end
end
end
Patch StaticFileHandler to serve files from memory
2019-06-23 17:54:46 +00:00
end
end
Add `/api/v1/channels/latest/:ucid`
2019-02-19 23:00:06 +00:00
Add support for '/c/:user/live'
2019-04-28 16:47:16 +00:00
{"/api/v1/channels/:ucid/playlists", "/api/v1/channels/playlists/:ucid"}.each do |route|
Add /api/v1/channels/playlists/:ucid
2019-02-19 23:05:27 +00:00
get route do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add /api/v1/channels/playlists/:ucid
2019-02-19 23:05:27 +00:00
env.response.content_type = "application/json"
ucid = env.params.url["ucid"]
continuation = env.params.query["continuation"]?
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
sort_by = env.params.query["sort"]?.try &.downcase ||
env.params.query["sort_by"]?.try &.downcase ||
"last"
Add /api/v1/channels/playlists/:ucid
2019-02-19 23:05:27 +00:00
begin
Add thumbnail and banners to channel page
2019-06-29 01:48:24 +00:00
channel = get_about_info(ucid, locale)
Add support for channel redirects
2019-09-08 16:08:59 +00:00
rescue ex : ChannelRedirect
env.response.headers["Location"] = env.request.resource.gsub(ucid, ex.channel_id)
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(302, "Channel is unavailable", {"authorId" => ex.channel_id})
Add /api/v1/channels/playlists/:ucid
2019-02-19 23:05:27 +00:00
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Add /api/v1/channels/playlists/:ucid
2019-02-19 23:05:27 +00:00
end
Use the youtubei API over the legacy one
2021-03-24 05:33:45 +00:00
items, continuation = fetch_channel_playlists(channel.ucid, channel.author, continuation, sort_by)
Add /api/v1/channels/playlists/:ucid
2019-02-19 23:05:27 +00:00
Add thumbnail and banners to channel page
2019-06-29 01:48:24 +00:00
JSON.build do |json|
Add /api/v1/channels/playlists/:ucid
2019-02-19 23:05:27 +00:00
json.object do
json.field "playlists" do
json.array do
items.each do |item|
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
item.to_json(locale, json) if item.is_a?(SearchPlaylist)
Add /api/v1/channels/playlists/:ucid
2019-02-19 23:05:27 +00:00
end
end
end
json.field "continuation", continuation
end
end
end
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
end
Add /api/v1/channels/playlists/:ucid
2019-02-19 23:05:27 +00:00
Add '/api/v1/channels/comments'
2019-07-02 23:53:19 +00:00
{"/api/v1/channels/:ucid/comments", "/api/v1/channels/comments/:ucid"}.each do |route|
get route do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
env.response.content_type = "application/json"
ucid = env.params.url["ucid"]
Add community page
2019-07-09 14:31:04 +00:00
thin_mode = env.params.query["thin_mode"]?
thin_mode = thin_mode == "true"
Add '/api/v1/channels/comments'
2019-07-02 23:53:19 +00:00
Add community page
2019-07-09 14:31:04 +00:00
format = env.params.query["format"]?
format ||= "json"
continuation = env.params.query["continuation"]?
Add '/api/v1/channels/comments'
2019-07-02 23:53:19 +00:00
# sort_by = env.params.query["sort_by"]?.try &.downcase
begin
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
fetch_channel_community(ucid, continuation, locale, format, thin_mode)
Add '/api/v1/channels/comments'
2019-07-02 23:53:19 +00:00
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Add '/api/v1/channels/comments'
2019-07-02 23:53:19 +00:00
end
end
end
Add '/api/v1/channels/search'
2018-09-22 15:49:42 +00:00
get "/api/v1/channels/search/:ucid" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add '/api/v1/channels/search'
2018-09-22 15:49:42 +00:00
env.response.content_type = "application/json"
ucid = env.params.url["ucid"]
query = env.params.query["q"]?
query ||= ""
page = env.params.query["page"]?.try &.to_i?
page ||= 1
count, search_results = channel_search(query, page, ucid)
Add 'to_json' into respective structs
2019-06-08 18:31:41 +00:00
JSON.build do |json|
Add '/api/v1/channels/search'
2018-09-22 15:49:42 +00:00
json.array do
search_results.each do |item|
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
item.to_json(locale, json)
Add '/api/v1/channels/search'
2018-09-22 15:49:42 +00:00
end
end
end
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
end
Add '/api/v1/channels/search'
2018-09-22 15:49:42 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/api/v1/search" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for 'region' in search
2019-02-07 00:21:40 +00:00
region = env.params.query["region"]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add '/api/v1/channels/search'
2018-09-22 15:49:42 +00:00
env.response.content_type = "application/json"
Major cleanup
2018-08-05 04:07:38 +00:00
query = env.params.query["q"]?
query ||= ""
Add redirect for '/channels/:ucid/videos'
2018-08-02 22:18:57 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
page = env.params.query["page"]?.try &.to_i?
page ||= 1
Add filters to '/api/v1/search' endpoint
2018-08-04 22:12:58 +00:00
sort_by = env.params.query["sort_by"]?.try &.downcase
sort_by ||= "relevance"
date = env.params.query["date"]?.try &.downcase
date ||= ""
Fix duration in /api/v1/search
2019-02-26 20:31:37 +00:00
duration = env.params.query["duration"]?.try &.downcase
Add filters to '/api/v1/search' endpoint
2018-08-04 22:12:58 +00:00
duration ||= ""
features = env.params.query["features"]?.try &.split(",").map { |feature| feature.downcase }
features ||= [] of String
Add support for playlists and channels in search
2018-09-20 14:36:09 +00:00
content_type = env.params.query["type"]?.try &.downcase
content_type ||= "video"
Add filters to '/api/v1/search' endpoint
2018-08-04 22:12:58 +00:00
begin
fix search Put search page in the super secret proto field
2021-02-26 02:50:40 +00:00
search_params = produce_search_params(page, sort_by, date, content_type, duration, features)
Add filters to '/api/v1/search' endpoint
2018-08-04 22:12:58 +00:00
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, ex)
Add filters to '/api/v1/search' endpoint
2018-08-04 22:12:58 +00:00
end
fix search Put search page in the super secret proto field
2021-02-26 02:50:40 +00:00
count, search_results = search(query, search_params, region).as(Tuple)
Add 'to_json' into respective structs
2019-06-08 18:31:41 +00:00
JSON.build do |json|
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
json.array do
Add support for playlists and channels in search
2018-09-20 14:36:09 +00:00
search_results.each do |item|
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
item.to_json(locale, json)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
end
end
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
end
Add redirect for '/channels/:ucid/videos'
2018-08-02 22:18:57 +00:00
Add /api/v1/search/suggestions
2019-05-21 12:15:15 +00:00
get "/api/v1/search/suggestions" do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
region = env.params.query["region"]?
env.response.content_type = "application/json"
query = env.params.query["q"]?
query ||= ""
begin
Cleanup request headers
2020-05-08 14:00:53 +00:00
headers = HTTP::Headers{":authority" => "suggestqueries.google.com"}
response = YT_POOL.client &.get("/complete/search?hl=en&gl=#{region}&client=youtube&ds=yt&q=#{URI.encode_www_form(query)}&callback=suggestCallback", headers).body
Run 'crystal tool format'
2019-11-28 14:20:44 +00:00
Add /api/v1/search/suggestions
2019-05-21 12:15:15 +00:00
body = response[35..-2]
body = JSON.parse(body).as_a
suggestions = body[1].as_a[0..-2]
JSON.build do |json|
json.object do
json.field "query", body[0].as_s
json.field "suggestions" do
json.array do
suggestions.each do |suggestion|
json.string suggestion[0].as_s
end
end
end
end
end
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Add /api/v1/search/suggestions
2019-05-21 12:15:15 +00:00
end
end
Add support for custom playlists
2019-08-05 23:49:13 +00:00
{"/api/v1/playlists/:plid", "/api/v1/auth/playlists/:plid"}.each do |route|
get route do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add playlist playback support
2018-10-08 02:11:33 +00:00
Add support for custom playlists
2019-08-05 23:49:13 +00:00
env.response.content_type = "application/json"
plid = env.params.url["plid"]
Redirect mixes
2018-10-07 03:18:50 +00:00
Add support for custom playlists
2019-08-05 23:49:13 +00:00
offset = env.params.query["index"]?.try &.to_i?
offset ||= env.params.query["page"]?.try &.to_i?.try { |page| (page - 1) * 100 }
offset ||= 0
Add playlist page and endpoint
2018-08-15 15:22:36 +00:00
Add support for custom playlists
2019-08-05 23:49:13 +00:00
continuation = env.params.query["continuation"]?
Show info instead of empty playlist when possible
2018-09-28 14:54:45 +00:00
Add support for custom playlists
2019-08-05 23:49:13 +00:00
format = env.params.query["format"]?
format ||= "json"
Add playlist page and endpoint
2018-08-15 15:22:36 +00:00
Add support for custom playlists
2019-08-05 23:49:13 +00:00
if plid.starts_with? "RD"
next env.redirect "/api/v1/mixes/#{plid}"
end
Add playlist page and endpoint
2018-08-15 15:22:36 +00:00
Add support for custom playlists
2019-08-05 23:49:13 +00:00
begin
playlist = get_playlist(PG_DB, plid, locale)
Fix `watch_videos` endpoint Playlists created by `watch_videos` do not have an author which caused a crash previously.
2021-01-04 04:35:59 +00:00
rescue ex : InfoException
next error_json(404, ex)
Add support for custom playlists
2019-08-05 23:49:13 +00:00
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(404, "Playlist does not exist.")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
Add author thumbnails to playlist endpoint
2018-09-25 15:28:40 +00:00
Add support for custom playlists
2019-08-05 23:49:13 +00:00
user = env.get?("user").try &.as(User)
Allow unlisted playlists to be viewed from /api/v1/playlists/
2019-10-16 12:21:26 +00:00
if !playlist || playlist.privacy.private? && playlist.author != user.try &.email
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(404, "Playlist does not exist.")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
Add author thumbnails to playlist endpoint
2018-09-25 15:28:40 +00:00
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
response = playlist.to_json(offset, locale, continuation: continuation)
Add playlist page and endpoint
2018-08-15 15:22:36 +00:00
Add support for custom playlists
2019-08-05 23:49:13 +00:00
if format == "html"
response = JSON.parse(response)
playlist_html = template_playlist(response)
Format source
2019-10-22 01:40:03 +00:00
index, next_video = response["videos"].as_a.skip(1).select { |video| !video["author"].as_s.empty? }[0]?.try { |v| {v["index"], v["videoId"]} } || {nil, nil}
Add playlist page and endpoint
2018-08-15 15:22:36 +00:00
Add support for custom playlists
2019-08-05 23:49:13 +00:00
response = {
"playlistHtml" => playlist_html,
"index" => index,
"nextVideo" => next_video,
}.to_json
Add playlist page and endpoint
2018-08-15 15:22:36 +00:00
end
Add playlist playback support
2018-10-08 02:11:33 +00:00
Add support for custom playlists
2019-08-05 23:49:13 +00:00
response
Add playlist playback support
2018-10-08 02:11:33 +00:00
end
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
end
Add playlist page and endpoint
2018-08-15 15:22:36 +00:00
Add mixes
2018-09-29 04:12:35 +00:00
get "/api/v1/mixes/:rdid" do |env|
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for translations
2018-12-20 21:32:09 +00:00
Add mixes
2018-09-29 04:12:35 +00:00
env.response.content_type = "application/json"
rdid = env.params.url["rdid"]
continuation = env.params.query["continuation"]?
Add minor API fixes
2019-02-15 23:28:54 +00:00
continuation ||= rdid.lchop("RD")[0, 11]
Add mixes
2018-09-29 04:12:35 +00:00
Add playlist playback support
2018-10-08 02:11:33 +00:00
format = env.params.query["format"]?
format ||= "json"
Add mixes
2018-09-29 04:12:35 +00:00
begin
Add support for translations
2018-12-20 21:32:09 +00:00
mix = fetch_mix(rdid, continuation, locale: locale)
Fix mix continuation
2018-10-31 14:15:17 +00:00
if !rdid.ends_with? continuation
mix = fetch_mix(rdid, mix.videos[1].id)
index = mix.videos.index(mix.videos.select { |video| video.id == continuation }[0]?)
end
mix.videos = mix.videos[index..-1]
Add mixes
2018-09-29 04:12:35 +00:00
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Add mixes
2018-09-29 04:12:35 +00:00
end
response = JSON.build do |json|
json.object do
json.field "title", mix.title
json.field "mixId", mix.id
json.field "videos" do
json.array do
mix.videos.each do |video|
json.object do
json.field "title", video.title
json.field "videoId", video.id
json.field "author", video.author
json.field "authorId", video.ucid
json.field "authorUrl", "/channel/#{video.ucid}"
json.field "videoThumbnails" do
json.array do
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
generate_thumbnails(json, video.id)
Add mixes
2018-09-29 04:12:35 +00:00
end
end
json.field "index", video.index
json.field "lengthSeconds", video.length_seconds
end
end
end
end
end
end
Add playlist playback support
2018-10-08 02:11:33 +00:00
if format == "html"
response = JSON.parse(response)
playlist_html = template_mix(response)
Skip deleted videos in playlist
2019-10-21 23:00:56 +00:00
next_video = response["videos"].as_a.select { |video| !video["author"].as_s.empty? }[0]?.try &.["videoId"]
Add playlist playback support
2018-10-08 02:11:33 +00:00
response = {
"playlistHtml" => playlist_html,
"nextVideo" => next_video,
}.to_json
end
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
response
Add mixes
2018-09-29 04:12:35 +00:00
end
Add '/api/v1/auth/feeds'
2019-06-07 17:39:12 +00:00
# Authenticated endpoints
Add '/api/v1/notifications'
2019-04-10 22:58:42 +00:00
get "/api/v1/auth/notifications" do |env|
Refactor connect_listen for notifications
2019-06-02 12:41:53 +00:00
env.response.content_type = "text/event-stream"
Add '/api/v1/notifications'
2019-04-10 22:58:42 +00:00
topics = env.params.query["topics"]?.try &.split(",").uniq.first(1000)
topics ||= [] of String
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
create_notification_stream(env, topics, connection_channel)
Add POST /api/v1/auth/subscriptions
2019-05-21 14:01:17 +00:00
end
Add '/api/v1/notifications'
2019-04-10 22:58:42 +00:00
Add POST /api/v1/auth/subscriptions
2019-05-21 14:01:17 +00:00
post "/api/v1/auth/notifications" do |env|
Refactor connect_listen for notifications
2019-06-02 12:41:53 +00:00
env.response.content_type = "text/event-stream"
Add POST /api/v1/auth/subscriptions
2019-05-21 14:01:17 +00:00
topics = env.params.body["topics"]?.try &.split(",").uniq.first(1000)
topics ||= [] of String
Add '/api/v1/notifications'
2019-04-10 22:58:42 +00:00
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
create_notification_stream(env, topics, connection_channel)
Add '/api/v1/notifications'
2019-04-10 22:58:42 +00:00
end
Add '/api/v1/auth/preferences'
2019-05-01 02:01:57 +00:00
get "/api/v1/auth/preferences" do |env|
env.response.content_type = "application/json"
user = env.get("user").as(User)
user.preferences.to_json
end
Add authentication API
2019-04-18 21:23:50 +00:00
Add '/api/v1/auth/preferences'
2019-05-01 02:01:57 +00:00
post "/api/v1/auth/preferences" do |env|
env.response.content_type = "application/json"
user = env.get("user").as(User)
begin
Fix warnings with latest version of Crystal
2020-07-26 14:58:50 +00:00
preferences = Preferences.from_json(env.request.body || "{}")
Add '/api/v1/auth/preferences'
2019-05-01 02:01:57 +00:00
rescue
preferences = user.preferences
end
PG_DB.exec("UPDATE users SET preferences = $1 WHERE email = $2", preferences.to_json, user.email)
env.response.status_code = 204
end
Add authentication API
2019-04-18 21:23:50 +00:00
Add '/api/v1/auth/feeds'
2019-06-07 17:39:12 +00:00
get "/api/v1/auth/feed" do |env|
env.response.content_type = "application/json"
user = env.get("user").as(User)
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Set max preference size
2019-06-08 21:04:55 +00:00
max_results = env.params.query["max_results"]?.try &.to_i?
max_results ||= user.preferences.max_results
max_results ||= CONFIG.default_user_preferences.max_results
Add '/api/v1/auth/feeds'
2019-06-07 17:39:12 +00:00
page = env.params.query["page"]?.try &.to_i?
page ||= 1
videos, notifications = get_subscription_feed(PG_DB, user, max_results, page)
JSON.build do |json|
json.object do
json.field "notifications" do
json.array do
notifications.each do |video|
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
video.to_json(locale, json)
Add '/api/v1/auth/feeds'
2019-06-07 17:39:12 +00:00
end
end
end
json.field "videos" do
json.array do
videos.each do |video|
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
video.to_json(locale, json)
Add '/api/v1/auth/feeds'
2019-06-07 17:39:12 +00:00
end
end
end
end
end
end
Add '/api/v1/auth/subscriptions'
2019-04-22 15:40:29 +00:00
get "/api/v1/auth/subscriptions" do |env|
env.response.content_type = "application/json"
user = env.get("user").as(User)
Add authentication API
2019-04-18 21:23:50 +00:00
Add '/api/v1/auth/subscriptions'
2019-04-22 15:40:29 +00:00
if user.subscriptions.empty?
values = "'{}'"
else
values = "VALUES #{user.subscriptions.map { |id| %(('#{id}')) }.join(",")}"
end
Add authentication API
2019-04-18 21:23:50 +00:00
Add '/api/v1/auth/subscriptions'
2019-04-22 15:40:29 +00:00
subscriptions = PG_DB.query_all("SELECT * FROM channels WHERE id = ANY(#{values})", as: InvidiousChannel)
JSON.build do |json|
json.array do
subscriptions.each do |subscription|
json.object do
json.field "author", subscription.author
json.field "authorId", subscription.id
end
end
end
end
end
post "/api/v1/auth/subscriptions/:ucid" do |env|
env.response.content_type = "application/json"
user = env.get("user").as(User)
ucid = env.params.url["ucid"]
if !user.subscriptions.includes? ucid
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
get_channel(ucid, PG_DB, false, false)
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = array_append(subscriptions,$1) WHERE email = $2", ucid, user.email)
Add '/api/v1/auth/subscriptions'
2019-04-22 15:40:29 +00:00
end
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
# For Google accounts, access tokens don't have enough information to
# make a request on the user's behalf, which is why we don't sync with
# YouTube.
Add '/api/v1/auth/subscriptions'
2019-04-22 15:40:29 +00:00
env.response.status_code = 204
end
delete "/api/v1/auth/subscriptions/:ucid" do |env|
env.response.content_type = "application/json"
user = env.get("user").as(User)
ucid = env.params.url["ucid"]
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = array_remove(subscriptions, $1) WHERE email = $2", ucid, user.email)
Add '/api/v1/auth/subscriptions'
2019-04-22 15:40:29 +00:00
env.response.status_code = 204
end
Add authentication API
2019-04-18 21:23:50 +00:00
Add support for custom playlists
2019-08-05 23:49:13 +00:00
get "/api/v1/auth/playlists" do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
env.response.content_type = "application/json"
user = env.get("user").as(User)
playlists = PG_DB.query_all("SELECT * FROM playlists WHERE author = $1", user.email, as: InvidiousPlaylist)
JSON.build do |json|
json.array do
playlists.each do |playlist|
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
playlist.to_json(0, locale, json)
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
end
end
end
post "/api/v1/auth/playlists" do |env|
env.response.content_type = "application/json"
user = env.get("user").as(User)
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
title = env.params.json["title"]?.try &.as(String).delete("<>").byte_slice(0, 150)
if !title
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "Invalid title.")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
privacy = env.params.json["privacy"]?.try { |privacy| PlaylistPrivacy.parse(privacy.as(String).downcase) }
if !privacy
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "Invalid privacy setting.")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
if PG_DB.query_one("SELECT count(*) FROM playlists WHERE author = $1", user.email, as: Int64) >= 100
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "User cannot have more than 100 playlists.")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
playlist = create_playlist(PG_DB, title, privacy, user)
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
env.response.headers["Location"] = "#{HOST_URL}/api/v1/auth/playlists/#{playlist.id}"
Add support for custom playlists
2019-08-05 23:49:13 +00:00
env.response.status_code = 201
{
"title" => title,
"playlistId" => playlist.id,
}.to_json
end
patch "/api/v1/auth/playlists/:plid" do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
env.response.content_type = "application/json"
user = env.get("user").as(User)
plid = env.params.url["plid"]
playlist = PG_DB.query_one?("SELECT * FROM playlists WHERE id = $1", plid, as: InvidiousPlaylist)
Allow unlisted playlists to be viewed from /api/v1/playlists/
2019-10-16 12:21:26 +00:00
if !playlist || playlist.author != user.email && playlist.privacy.private?
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(404, "Playlist does not exist.")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
if playlist.author != user.email
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "Invalid user")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
title = env.params.json["title"].try &.as(String).delete("<>").byte_slice(0, 150) || playlist.title
privacy = env.params.json["privacy"]?.try { |privacy| PlaylistPrivacy.parse(privacy.as(String).downcase) } || playlist.privacy
description = env.params.json["description"]?.try &.as(String).delete("\r") || playlist.description
if title != playlist.title ||
privacy != playlist.privacy ||
description != playlist.description
updated = Time.utc
else
updated = playlist.updated
end
PG_DB.exec("UPDATE playlists SET title = $1, privacy = $2, description = $3, updated = $4 WHERE id = $5", title, privacy, description, updated, plid)
env.response.status_code = 204
end
delete "/api/v1/auth/playlists/:plid" do |env|
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for custom playlists
2019-08-05 23:49:13 +00:00
env.response.content_type = "application/json"
user = env.get("user").as(User)
plid = env.params.url["plid"]
playlist = PG_DB.query_one?("SELECT * FROM playlists WHERE id = $1", plid, as: InvidiousPlaylist)
Allow unlisted playlists to be viewed from /api/v1/playlists/
2019-10-16 12:21:26 +00:00
if !playlist || playlist.author != user.email && playlist.privacy.private?
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(404, "Playlist does not exist.")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
if playlist.author != user.email
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "Invalid user")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
PG_DB.exec("DELETE FROM playlist_videos * WHERE plid = $1", plid)
PG_DB.exec("DELETE FROM playlists * WHERE id = $1", plid)
env.response.status_code = 204
end
post "/api/v1/auth/playlists/:plid/videos" do |env|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
env.response.content_type = "application/json"
user = env.get("user").as(User)
plid = env.params.url["plid"]
playlist = PG_DB.query_one?("SELECT * FROM playlists WHERE id = $1", plid, as: InvidiousPlaylist)
Allow unlisted playlists to be viewed from /api/v1/playlists/
2019-10-16 12:21:26 +00:00
if !playlist || playlist.author != user.email && playlist.privacy.private?
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(404, "Playlist does not exist.")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
if playlist.author != user.email
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "Invalid user")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
if playlist.index.size >= 500
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "Playlist cannot have more than 500 videos")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
video_id = env.params.json["videoId"].try &.as(String)
if !video_id
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "Invalid videoId")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
begin
video = get_video(video_id, PG_DB)
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(500, ex)
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
Fix warnings with latest version of Crystal
2020-07-26 14:58:50 +00:00
playlist_video = PlaylistVideo.new({
title: video.title,
id: video.id,
author: video.author,
ucid: video.ucid,
Add support for custom playlists
2019-08-05 23:49:13 +00:00
length_seconds: video.length_seconds,
Fix warnings with latest version of Crystal
2020-07-26 14:58:50 +00:00
published: video.published,
plid: plid,
live_now: video.live_now,
index: Random::Secure.rand(0_i64..Int64::MAX),
})
Add support for custom playlists
2019-08-05 23:49:13 +00:00
video_array = playlist_video.to_a
args = arg_array(video_array)
PG_DB.exec("INSERT INTO playlist_videos VALUES (#{args})", args: video_array)
Fix stale playlist video counting on add/drop (https://github.com/iv-org/invidious/issues/1448) The return of `cardinality()` is not affected by the `array_append()` and `array_remove()` of the same statement.
2020-11-01 04:14:46 +00:00
PG_DB.exec("UPDATE playlists SET index = array_append(index, $1), video_count = cardinality(index) + 1, updated = $2 WHERE id = $3", playlist_video.index, Time.utc, plid)
Add support for custom playlists
2019-08-05 23:49:13 +00:00
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
env.response.headers["Location"] = "#{HOST_URL}/api/v1/auth/playlists/#{plid}/videos/#{playlist_video.index.to_u64.to_s(16).upcase}"
Add support for custom playlists
2019-08-05 23:49:13 +00:00
env.response.status_code = 201
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
playlist_video.to_json(locale, index: playlist.index.size)
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
delete "/api/v1/auth/playlists/:plid/videos/:index" do |env|
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add support for custom playlists
2019-08-05 23:49:13 +00:00
env.response.content_type = "application/json"
user = env.get("user").as(User)
plid = env.params.url["plid"]
index = env.params.url["index"].to_i64(16)
playlist = PG_DB.query_one?("SELECT * FROM playlists WHERE id = $1", plid, as: InvidiousPlaylist)
Allow unlisted playlists to be viewed from /api/v1/playlists/
2019-10-16 12:21:26 +00:00
if !playlist || playlist.author != user.email && playlist.privacy.private?
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(404, "Playlist does not exist.")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
if playlist.author != user.email
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "Invalid user")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
if !playlist.index.includes? index
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(404, "Playlist does not contain index")
Add support for custom playlists
2019-08-05 23:49:13 +00:00
end
PG_DB.exec("DELETE FROM playlist_videos * WHERE index = $1", index)
Fix stale playlist video counting on add/drop (https://github.com/iv-org/invidious/issues/1448) The return of `cardinality()` is not affected by the `array_append()` and `array_remove()` of the same statement.
2020-11-01 04:14:46 +00:00
PG_DB.exec("UPDATE playlists SET index = array_remove(index, $1), video_count = cardinality(index) - 1, updated = $2 WHERE id = $3", index, Time.utc, plid)
Add support for custom playlists
2019-08-05 23:49:13 +00:00
env.response.status_code = 204
end
# patch "/api/v1/auth/playlists/:plid/videos/:index" do |env|
# TODO: Playlist stub
# end
Add authentication API
2019-04-18 21:23:50 +00:00
get "/api/v1/auth/tokens" do |env|
env.response.content_type = "application/json"
user = env.get("user").as(User)
scopes = env.get("scopes").as(Array(String))
tokens = PG_DB.query_all("SELECT id, issued FROM session_ids WHERE email = $1", user.email, as: {session: String, issued: Time})
JSON.build do |json|
json.array do
tokens.each do |token|
json.object do
json.field "session", token[:session]
json.field "issued", token[:issued].to_unix
end
end
end
end
end
post "/api/v1/auth/tokens/register" do |env|
user = env.get("user").as(User)
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
case env.request.headers["Content-Type"]?
when "application/x-www-form-urlencoded"
scopes = env.params.body.select { |k, v| k.match(/^scopes\[\d+\]$/) }.map { |k, v| v }
callback_url = env.params.body["callbackUrl"]?
expire = env.params.body["expire"]?.try &.to_i?
when "application/json"
scopes = env.params.json["scopes"].as(Array).map { |v| v.as_s }
callback_url = env.params.json["callbackUrl"]?.try &.as(String)
expire = env.params.json["expire"]?.try &.as(Int64)
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "Invalid or missing header 'Content-Type'")
Add authentication API
2019-04-18 21:23:50 +00:00
end
if callback_url && callback_url.empty?
callback_url = nil
end
if callback_url
callback_url = URI.parse(callback_url)
end
if sid = env.get?("sid").try &.as(String)
env.response.content_type = "text/html"
csrf_token = generate_response(sid, {":authorize_token"}, HMAC_KEY, PG_DB, use_nonce: true)
next templated "authorize_token"
else
env.response.content_type = "application/json"
superset_scopes = env.get("scopes").as(Array(String))
authorized_scopes = [] of String
scopes.each do |scope|
if scopes_include_scope(superset_scopes, scope)
authorized_scopes << scope
end
end
access_token = generate_token(user.email, authorized_scopes, expire, HMAC_KEY, PG_DB)
if callback_url
Update to Crystal 0.31.0, resolve compiler deprecation warnings, update dependencies (#764) * shard: update to crystal 0.31.0 Additionally, no longer use the Crystal "markdown" library which has been removed from the Crystal stdlib in version 0.31.0. See https://github.com/crystal-lang/crystal/pull/8115. Also fix some deprecation warnings using the following commands: find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.escape/URI\.encode_www_form/g' "{}" \; find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.unescape/URI\.decode_www_form/g' "{}" \; sed -i 's/while \%pull\.kind \!\= \:end_object/until \%pull\.kind\.end_object\?/g' src/invidious/helpers/patch_mapping.cr
2019-09-24 17:31:33 +00:00
access_token = URI.encode_www_form(access_token)
Add authentication API
2019-04-18 21:23:50 +00:00
if query = callback_url.query
query = HTTP::Params.parse(query.not_nil!)
else
query = HTTP::Params.new
end
query["token"] = access_token
callback_url.query = query.to_s
env.redirect callback_url.to_s
else
access_token
end
end
end
post "/api/v1/auth/tokens/unregister" do |env|
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add authentication API
2019-04-18 21:23:50 +00:00
env.response.content_type = "application/json"
user = env.get("user").as(User)
scopes = env.get("scopes").as(Array(String))
session = env.params.json["session"]?.try &.as(String)
session ||= env.get("session").as(String)
# Allow tokens to revoke other tokens with correct scope
if session == env.get("session").as(String)
PG_DB.exec("DELETE FROM session_ids * WHERE id = $1", session)
elsif scopes_include_scope(scopes, "GET:tokens")
PG_DB.exec("DELETE FROM session_ids * WHERE id = $1", session)
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "Cannot revoke session #{session}")
Add authentication API
2019-04-18 21:23:50 +00:00
end
env.response.status_code = 204
end
Add redirect for local DASH urls
2018-08-07 18:10:52 +00:00
get "/api/manifest/dash/id/videoplayback" do |env|
Remove content type from videoplayback redirects
2019-04-12 16:08:33 +00:00
env.response.headers.delete("Content-Type")
Add CORS headers to /api/manifest/dash/id/videoplayback
2018-08-07 18:23:27 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
Add redirect for local DASH urls
2018-08-07 18:10:52 +00:00
env.redirect "/videoplayback?#{env.params.query}"
end
get "/api/manifest/dash/id/videoplayback/*" do |env|
Remove content type from videoplayback redirects
2019-04-12 16:08:33 +00:00
env.response.headers.delete("Content-Type")
Add CORS headers to /api/manifest/dash/id/videoplayback
2018-08-07 18:23:27 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
Add redirect for local DASH urls
2018-08-07 18:10:52 +00:00
env.redirect env.request.path.lchop("/api/manifest/dash/id")
end
Organize routes
2018-07-16 13:18:59 +00:00
get "/api/manifest/dash/id/:id" do |env|
env.response.headers.add("Access-Control-Allow-Origin", "*")
env.response.content_type = "application/dash+xml"
local = env.params.query["local"]?.try &.== "true"
id = env.params.url["id"]
Add 'region' parameter to captions and manifest endpoints
2018-11-17 23:37:57 +00:00
region = env.params.query["region"]?
Organize routes
2018-07-16 13:18:59 +00:00
Add 'unique_res' option to '/api/manifest/dash/id'
2019-06-05 01:54:38 +00:00
# Since some implementations create playlists based on resolution regardless of different codecs,
Fix 'unique_res' to keep resolutions unique within a representation
2019-06-07 02:32:39 +00:00
# we can opt to only add a source to a representation if it has a unique height within that representation
Minor refactor
2020-01-09 01:27:21 +00:00
unique_res = env.params.query["unique_res"]?.try { |q| (q == "true" || q == "1").to_unsafe }
Add 'unique_res' option to '/api/manifest/dash/id'
2019-06-05 01:54:38 +00:00
Organize routes
2018-07-16 13:18:59 +00:00
begin
Refactor proxy_list into global
2019-06-29 02:17:56 +00:00
video = get_video(id, PG_DB, region: region)
Add basic handling for (almost) valid video URLs
2018-10-07 03:22:22 +00:00
rescue ex : VideoRedirect
Add support for channel redirects
2019-09-08 16:08:59 +00:00
next env.redirect env.request.resource.gsub(id, ex.video_id)
Organize routes
2018-07-16 13:18:59 +00:00
rescue ex
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
env.response.status_code = 403
next
Organize routes
2018-07-16 13:18:59 +00:00
end
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
if dashmpd = video.dash_manifest_url
Update code for Crystal 0.36.0 Rename `HTTPClient@socket` to `HTTPClient@io`, see https://github.com/crystal-lang/crystal/pull/9543. Rename `URI#full_path` to `URI#request_target`, see https://github.com/crystal-lang/crystal/pull/10099.
2021-01-31 18:52:32 +00:00
manifest = YT_POOL.client &.get(URI.parse(dashmpd).request_target).body
Organize routes
2018-07-16 13:18:59 +00:00
manifest = manifest.gsub(/<BaseURL>[^<]+<\/BaseURL>/) do |baseurl|
url = baseurl.lchop("<BaseURL>")
url = url.rchop("</BaseURL>")
if local
save host when using dash manifest
2021-01-10 23:00:45 +00:00
uri = URI.parse(url)
Update code for Crystal 0.36.0 Rename `HTTPClient@socket` to `HTTPClient@io`, see https://github.com/crystal-lang/crystal/pull/9543. Rename `URI#full_path` to `URI#request_target`, see https://github.com/crystal-lang/crystal/pull/10099.
2021-01-31 18:52:32 +00:00
url = "#{uri.request_target}host/#{uri.host}/"
Organize routes
2018-07-16 13:18:59 +00:00
end
"<BaseURL>#{url}</BaseURL>"
end
next manifest
end
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
adaptive_fmts = video.adaptive_fmts
Organize routes
2018-07-16 13:18:59 +00:00
if local
adaptive_fmts.each do |fmt|
Update code for Crystal 0.36.0 Rename `HTTPClient@socket` to `HTTPClient@io`, see https://github.com/crystal-lang/crystal/pull/9543. Rename `URI#full_path` to `URI#request_target`, see https://github.com/crystal-lang/crystal/pull/10099.
2021-01-31 18:52:32 +00:00
fmt["url"] = JSON::Any.new(URI.parse(fmt["url"].as_s).request_target)
Organize routes
2018-07-16 13:18:59 +00:00
end
end
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
audio_streams = video.audio_streams
video_streams = video.video_streams.sort_by { |stream| {stream["width"].as_i, stream["fps"].as_i} }.reverse
Major cleanup
2018-08-05 04:07:38 +00:00
Remove content type from videoplayback redirects
2019-04-12 16:08:33 +00:00
XML.build(indent: " ", encoding: "UTF-8") do |xml|
Clean up DASH manifest
2018-08-11 20:01:22 +00:00
xml.element("MPD", "xmlns": "urn:mpeg:dash:schema:mpd:2011",
Add 'unique_res' option to '/api/manifest/dash/id'
2019-06-05 01:54:38 +00:00
"profiles": "urn:mpeg:dash:profile:full:2011", minBufferTime: "PT1.5S", type: "static",
Update video extractor
2019-07-30 00:41:45 +00:00
mediaPresentationDuration: "PT#{video.length_seconds}S") do
Organize routes
2018-07-16 13:18:59 +00:00
xml.element("Period") do
Add id to AdaptationSets
2019-04-12 16:19:54 +00:00
i = 0
Add webm to dash manifests
2019-04-12 13:04:59 +00:00
{"audio/mp4", "audio/webm"}.each do |mime_type|
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
mime_streams = audio_streams.select { |stream| stream["mimeType"].as_s.starts_with? mime_type }
next if mime_streams.empty?
Remove empty representations from dash manifests
2019-07-09 15:08:00 +00:00
Add id to AdaptationSets
2019-04-12 16:19:54 +00:00
xml.element("AdaptationSet", id: i, mimeType: mime_type, startWithSAP: 1, subsegmentAlignment: true) do
Remove empty representations from dash manifests
2019-07-09 15:08:00 +00:00
mime_streams.each do |fmt|
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
codecs = fmt["mimeType"].as_s.split("codecs=")[1].strip('"')
bandwidth = fmt["bitrate"].as_i
itag = fmt["itag"].as_i
url = fmt["url"].as_s
Add webm to dash manifests
2019-04-12 13:04:59 +00:00
xml.element("Representation", id: fmt["itag"], codecs: codecs, bandwidth: bandwidth) do
xml.element("AudioChannelConfiguration", schemeIdUri: "urn:mpeg:dash:23003:3:audio_channel_configuration:2011",
value: "2")
xml.element("BaseURL") { xml.text url }
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
xml.element("SegmentBase", indexRange: "#{fmt["indexRange"]["start"]}-#{fmt["indexRange"]["end"]}") do
xml.element("Initialization", range: "#{fmt["initRange"]["start"]}-#{fmt["initRange"]["end"]}")
Add webm to dash manifests
2019-04-12 13:04:59 +00:00
end
Organize routes
2018-07-16 13:18:59 +00:00
end
end
end
Add id to AdaptationSets
2019-04-12 16:19:54 +00:00
i += 1
Organize routes
2018-07-16 13:18:59 +00:00
end
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
potential_heights = {4320, 2160, 1440, 1080, 720, 480, 360, 240, 144}
Add webm to dash manifests
2019-04-12 13:04:59 +00:00
{"video/mp4", "video/webm"}.each do |mime_type|
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
mime_streams = video_streams.select { |stream| stream["mimeType"].as_s.starts_with? mime_type }
Minor refactor
2020-01-09 01:27:21 +00:00
next if mime_streams.empty?
Remove empty representations from dash manifests
2019-07-09 15:08:00 +00:00
Fix 'unique_res' to keep resolutions unique within a representation
2019-06-07 02:32:39 +00:00
heights = [] of Int32
Add id to AdaptationSets
2019-04-12 16:19:54 +00:00
xml.element("AdaptationSet", id: i, mimeType: mime_type, startWithSAP: 1, subsegmentAlignment: true, scanType: "progressive") do
Remove empty representations from dash manifests
2019-07-09 15:08:00 +00:00
mime_streams.each do |fmt|
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
codecs = fmt["mimeType"].as_s.split("codecs=")[1].strip('"')
bandwidth = fmt["bitrate"].as_i
itag = fmt["itag"].as_i
url = fmt["url"].as_s
width = fmt["width"].as_i
height = fmt["height"].as_i
Provide dash qualities as reported by YouTube player
2019-05-30 20:39:02 +00:00
# Resolutions reported by YouTube player (may not accurately reflect source)
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
height = potential_heights.min_by { |i| (height - i).abs }
Add 'unique_res' option to '/api/manifest/dash/id'
2019-06-05 01:54:38 +00:00
next if unique_res && heights.includes? height
heights << height
Add webm to dash manifests
2019-04-12 13:04:59 +00:00
xml.element("Representation", id: itag, codecs: codecs, width: width, height: height,
startWithSAP: "1", maxPlayoutRate: "1",
bandwidth: bandwidth, frameRate: fmt["fps"]) do
xml.element("BaseURL") { xml.text url }
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
xml.element("SegmentBase", indexRange: "#{fmt["indexRange"]["start"]}-#{fmt["indexRange"]["end"]}") do
xml.element("Initialization", range: "#{fmt["initRange"]["start"]}-#{fmt["initRange"]["end"]}")
Add webm to dash manifests
2019-04-12 13:04:59 +00:00
end
Organize routes
2018-07-16 13:18:59 +00:00
end
end
end
Add id to AdaptationSets
2019-04-12 16:19:54 +00:00
i += 1
Organize routes
2018-07-16 13:18:59 +00:00
end
end
end
end
end
Add livestream support
2018-07-27 23:25:58 +00:00
get "/api/manifest/hls_variant/*" do |env|
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
response = YT_POOL.client &.get(env.request.path)
Add livestream support
2018-07-27 23:25:58 +00:00
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
if response.status_code != 200
env.response.status_code = response.status_code
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
next
Add livestream support
2018-07-27 23:25:58 +00:00
end
Add 'local=true' to hlsUrl
2019-04-25 17:41:35 +00:00
local = env.params.query["local"]?.try &.== "true"
Add livestream support
2018-07-27 23:25:58 +00:00
env.response.content_type = "application/x-mpegURL"
env.response.headers.add("Access-Control-Allow-Origin", "*")
Major cleanup
2018-08-05 04:07:38 +00:00
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
manifest = response.body
Add 'local=true' to hlsUrl
2019-04-25 17:41:35 +00:00
if local
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
manifest = manifest.gsub("https://www.youtube.com", HOST_URL)
Add 'local=true' to hlsUrl
2019-04-25 17:41:35 +00:00
manifest = manifest.gsub("index.m3u8", "index.m3u8?local=true")
end
manifest
Add livestream support
2018-07-27 23:25:58 +00:00
end
get "/api/manifest/hls_playlist/*" do |env|
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
response = YT_POOL.client &.get(env.request.path)
Add livestream support
2018-07-27 23:25:58 +00:00
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
if response.status_code != 200
env.response.status_code = response.status_code
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
next
Add livestream support
2018-07-27 23:25:58 +00:00
end
Add 'local=true' to hlsUrl
2019-04-25 17:41:35 +00:00
local = env.params.query["local"]?.try &.== "true"
env.response.content_type = "application/x-mpegURL"
env.response.headers.add("Access-Control-Allow-Origin", "*")
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
manifest = response.body
Add 'local=true' to hlsUrl
2019-04-25 17:41:35 +00:00
if local
Fix livestream parsing URLs (#2356)
2021-08-26 21:02:26 +00:00
manifest = manifest.gsub(/^https:\/\/\w+---.{11}\.c\.youtube\.com[^\n]*/m) do |match|
Fix redirect for livestream segments
2019-07-05 17:08:39 +00:00
path = URI.parse(match).path
path = path.lchop("/videoplayback/")
path = path.rchop("/")
path = path.gsub(/mime\/\w+\/\w+/) do |mimetype|
mimetype = mimetype.split("/")
mimetype[0] + "/" + mimetype[1] + "%2F" + mimetype[2]
end
path = path.split("/")
raw_params = {} of String => Array(String)
path.each_slice(2) do |pair|
key, value = pair
Update to Crystal 0.31.0, resolve compiler deprecation warnings, update dependencies (#764) * shard: update to crystal 0.31.0 Additionally, no longer use the Crystal "markdown" library which has been removed from the Crystal stdlib in version 0.31.0. See https://github.com/crystal-lang/crystal/pull/8115. Also fix some deprecation warnings using the following commands: find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.escape/URI\.encode_www_form/g' "{}" \; find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.unescape/URI\.decode_www_form/g' "{}" \; sed -i 's/while \%pull\.kind \!\= \:end_object/until \%pull\.kind\.end_object\?/g' src/invidious/helpers/patch_mapping.cr
2019-09-24 17:31:33 +00:00
value = URI.decode_www_form(value)
Fix redirect for livestream segments
2019-07-05 17:08:39 +00:00
if raw_params[key]?
raw_params[key] << value
else
raw_params[key] = [value]
end
end
raw_params = HTTP::Params.new(raw_params)
if fvip = raw_params["hls_chunk_host"].match(/r(?<fvip>\d+)---/)
raw_params["fvip"] = fvip["fvip"]
end
raw_params["local"] = "true"
Add 'local=true' to hlsUrl
2019-04-25 17:41:35 +00:00
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
"#{HOST_URL}/videoplayback?#{raw_params}"
Fix redirect for livestream segments
2019-07-05 17:08:39 +00:00
end
end
Add livestream support
2018-07-27 23:25:58 +00:00
manifest
end
Add internal redirect for video URLs
2019-01-28 02:35:32 +00:00
# YouTube /videoplayback links expire after 6 hours,
# so we have a mechanism here to redirect to the latest version
get "/latest_version" do |env|
Remove JS from download widget
2019-02-24 17:04:46 +00:00
if env.params.query["download_widget"]?
download_widget = JSON.parse(env.params.query["download_widget"])
Add subtitles to download widget
2019-04-11 17:08:43 +00:00
Remove JS from download widget
2019-02-24 17:04:46 +00:00
id = download_widget["id"].as_s
title = download_widget["title"].as_s
Add subtitles to download widget
2019-04-11 17:08:43 +00:00
if label = download_widget["label"]?
env.redirect "/api/v1/captions/#{id}?label=#{label}&title=#{title}"
next
else
Fix downloads The `itag` is now converted to a number, matching the `itag` of `Video.adaptive_fmts` and `Video.fmt_stream`.
2021-01-09 19:40:01 +00:00
itag = download_widget["itag"].as_s.to_i
Fix url in storyboards
2019-04-12 12:29:47 +00:00
local = "true"
end
Add subtitles to download widget
2019-04-11 17:08:43 +00:00
end
Remove JS from download widget
2019-02-24 17:04:46 +00:00
id ||= env.params.query["id"]?
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
itag ||= env.params.query["itag"]?.try &.to_i
Add internal redirect for video URLs
2019-01-28 02:35:32 +00:00
Add `region` to `latest_version`
2019-02-09 18:28:43 +00:00
region = env.params.query["region"]?
Remove JS from download widget
2019-02-24 17:04:46 +00:00
local ||= env.params.query["local"]?
Fix colors and data-url in download widget
2019-01-28 03:20:52 +00:00
local ||= "false"
local = local == "true"
Add internal redirect for video URLs
2019-01-28 02:35:32 +00:00
if !id || !itag
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
env.response.status_code = 400
next
Add internal redirect for video URLs
2019-01-28 02:35:32 +00:00
end
Refactor proxy_list into global
2019-06-29 02:17:56 +00:00
video = get_video(id, PG_DB, region: region)
Add internal redirect for video URLs
2019-01-28 02:35:32 +00:00
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
fmt = video.fmt_stream.find(nil) { |f| f["itag"].as_i == itag } || video.adaptive_fmts.find(nil) { |f| f["itag"].as_i == itag }
url = fmt.try &.["url"]?.try &.as_s
Add internal redirect for video URLs
2019-01-28 02:35:32 +00:00
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
if !url
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
env.response.status_code = 404
next
Fix colors and data-url in download widget
2019-01-28 03:20:52 +00:00
end
Update code for Crystal 0.36.0 Rename `HTTPClient@socket` to `HTTPClient@io`, see https://github.com/crystal-lang/crystal/pull/9543. Rename `URI#full_path` to `URI#request_target`, see https://github.com/crystal-lang/crystal/pull/10099.
2021-01-31 18:52:32 +00:00
url = URI.parse(url).request_target.not_nil! if local
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
url = "#{url}&title=#{title}" if title
Remove JS from download widget
2019-02-24 17:04:46 +00:00
Fix colors and data-url in download widget
2019-01-28 03:20:52 +00:00
env.redirect url
Add internal redirect for video URLs
2019-01-28 02:35:32 +00:00
end
Fix options for '/videoplayback'
2018-08-07 18:25:22 +00:00
options "/videoplayback" do |env|
Remove content type from videoplayback redirects
2019-04-12 16:08:33 +00:00
env.response.headers.delete("Content-Type")
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
Add OPTIONS to allowed CORS methods
2018-08-09 14:43:47 +00:00
env.response.headers["Access-Control-Allow-Methods"] = "GET, OPTIONS"
env.response.headers["Access-Control-Allow-Headers"] = "Content-Type, Range"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add '/videoplayback' redirect
2018-08-07 16:39:56 +00:00
options "/videoplayback/*" do |env|
Remove content type from videoplayback redirects
2019-04-12 16:08:33 +00:00
env.response.headers.delete("Content-Type")
Add '/videoplayback' redirect
2018-08-07 16:39:56 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
Add OPTIONS to allowed CORS methods
2018-08-09 14:43:47 +00:00
env.response.headers["Access-Control-Allow-Methods"] = "GET, OPTIONS"
env.response.headers["Access-Control-Allow-Headers"] = "Content-Type, Range"
Add '/videoplayback' redirect
2018-08-07 16:39:56 +00:00
end
Add CORS headers to local URLs
2018-08-07 18:18:38 +00:00
options "/api/manifest/dash/id/videoplayback" do |env|
Remove content type from videoplayback redirects
2019-04-12 16:08:33 +00:00
env.response.headers.delete("Content-Type")
Add CORS headers to local URLs
2018-08-07 18:18:38 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
Add OPTIONS to allowed CORS methods
2018-08-09 14:43:47 +00:00
env.response.headers["Access-Control-Allow-Methods"] = "GET, OPTIONS"
env.response.headers["Access-Control-Allow-Headers"] = "Content-Type, Range"
Add CORS headers to local URLs
2018-08-07 18:18:38 +00:00
end
options "/api/manifest/dash/id/videoplayback/*" do |env|
Remove content type from videoplayback redirects
2019-04-12 16:08:33 +00:00
env.response.headers.delete("Content-Type")
Add CORS headers to local URLs
2018-08-07 18:18:38 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
Add OPTIONS to allowed CORS methods
2018-08-09 14:43:47 +00:00
env.response.headers["Access-Control-Allow-Methods"] = "GET, OPTIONS"
env.response.headers["Access-Control-Allow-Headers"] = "Content-Type, Range"
Add CORS headers to local URLs
2018-08-07 18:18:38 +00:00
end
Add '/videoplayback' redirect
2018-08-07 16:39:56 +00:00
get "/videoplayback/*" do |env|
Fix dash endpoint
2018-06-06 22:55:51 +00:00
path = env.request.path
Add '/videoplayback' redirect
2018-08-07 16:39:56 +00:00
Add option to toggle light and dark to navbar
2018-08-07 16:49:14 +00:00
path = path.lchop("/videoplayback/")
path = path.rchop("/")
Add fix for mimetype in /videoplayback
2018-07-16 02:53:24 +00:00
Add option to toggle light and dark to navbar
2018-08-07 16:49:14 +00:00
path = path.gsub(/mime\/\w+\/\w+/) do |mimetype|
mimetype = mimetype.split("/")
mimetype[0] + "/" + mimetype[1] + "%2F" + mimetype[2]
end
Add fix for mimetype in /videoplayback
2018-07-16 02:53:24 +00:00
Add option to toggle light and dark to navbar
2018-08-07 16:49:14 +00:00
path = path.split("/")
Fix dash endpoint
2018-06-06 22:55:51 +00:00
Add option to toggle light and dark to navbar
2018-08-07 16:49:14 +00:00
raw_params = {} of String => Array(String)
path.each_slice(2) do |pair|
key, value = pair
Update to Crystal 0.31.0, resolve compiler deprecation warnings, update dependencies (#764) * shard: update to crystal 0.31.0 Additionally, no longer use the Crystal "markdown" library which has been removed from the Crystal stdlib in version 0.31.0. See https://github.com/crystal-lang/crystal/pull/8115. Also fix some deprecation warnings using the following commands: find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.escape/URI\.encode_www_form/g' "{}" \; find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.unescape/URI\.decode_www_form/g' "{}" \; sed -i 's/while \%pull\.kind \!\= \:end_object/until \%pull\.kind\.end_object\?/g' src/invidious/helpers/patch_mapping.cr
2019-09-24 17:31:33 +00:00
value = URI.decode_www_form(value)
Fix dash endpoint
2018-06-06 22:55:51 +00:00
Add option to toggle light and dark to navbar
2018-08-07 16:49:14 +00:00
if raw_params[key]?
raw_params[key] << value
else
raw_params[key] = [value]
Fix dash endpoint
2018-06-06 22:55:51 +00:00
end
Add option to toggle light and dark to navbar
2018-08-07 16:49:14 +00:00
end
Fix dash endpoint
2018-06-06 22:55:51 +00:00
Add option to toggle light and dark to navbar
2018-08-07 16:49:14 +00:00
query_params = HTTP::Params.new(raw_params)
Add '/videoplayback' redirect
2018-08-07 16:39:56 +00:00
Use http-streaming for livestreams
2018-08-11 19:29:51 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
Add '/videoplayback' redirect
2018-08-07 16:39:56 +00:00
env.redirect "/videoplayback?#{query_params}"
end
get "/videoplayback" do |env|
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
Add option to toggle light and dark to navbar
2018-08-07 16:49:14 +00:00
query_params = env.params.query
Add /videoplayback endpoint
2018-04-16 01:47:37 +00:00
Add better fallback for '/videoplayback'
2019-03-11 19:07:55 +00:00
fvip = query_params["fvip"]? || "3"
Fix videoplayback when encountering redirector URLs
2019-05-31 01:47:04 +00:00
mns = query_params["mn"]?.try &.split(",")
mns ||= [] of String
Add better fallback for '/videoplayback'
2019-03-11 19:07:55 +00:00
Fix local redirects in /videoplayback
2019-03-27 19:59:53 +00:00
if query_params["region"]?
region = query_params["region"]
query_params.delete("region")
end
Add 'host' to '/videoplayback'
2019-03-11 18:14:30 +00:00
if query_params["host"]? && !query_params["host"].empty?
host = "https://#{query_params["host"]}"
Remove 'host' from query params
2019-03-11 18:32:46 +00:00
query_params.delete("host")
Add 'host' to '/videoplayback'
2019-03-11 18:14:30 +00:00
else
Add better fallback for '/videoplayback'
2019-03-11 19:07:55 +00:00
host = "https://r#{fvip}---#{mns.pop}.googlevideo.com"
Add 'host' to '/videoplayback'
2019-03-11 18:14:30 +00:00
end
Add /videoplayback endpoint
2018-04-16 01:47:37 +00:00
url = "/videoplayback?#{query_params.to_s}"
Add '&local=true' to watch and embed pages
2019-03-11 16:43:48 +00:00
headers = HTTP::Headers.new
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
REQUEST_HEADERS_WHITELIST.each do |header|
Add '&local=true' to watch and embed pages
2019-03-11 16:43:48 +00:00
if env.request.headers[header]?
headers[header] = env.request.headers[header]
end
end
Add retry for /videoplaybacl
2019-01-24 19:52:33 +00:00
Handle redirects in /videoplayback
2019-08-27 14:53:44 +00:00
client = make_client(URI.parse(host), region)
Fix monkeypatch for HTTP::Client
2019-10-26 15:43:28 +00:00
response = HTTP::Client::Response.new(500)
Add debug info to videoplayback
2020-03-06 18:50:00 +00:00
error = ""
Handle redirects in /videoplayback
2019-08-27 14:53:44 +00:00
5.times do
begin
response = client.head(url, headers)
if response.headers["Location"]?
location = URI.parse(response.headers["Location"])
env.response.headers["Access-Control-Allow-Origin"] = "*"
Close http clients after using The crystal http client maintains a keepalive connection to the other server which stays alive for some time. This should be closed if the client instance is not used again to avoid hogging resources
2020-12-23 05:52:23 +00:00
new_host = "#{location.scheme}://#{location.host}"
if new_host != host
host = new_host
client.close
client = make_client(URI.parse(new_host), region)
end
Handle redirects in /videoplayback
2019-08-27 14:53:44 +00:00
Update code for Crystal 0.36.0 Rename `HTTPClient@socket` to `HTTPClient@io`, see https://github.com/crystal-lang/crystal/pull/9543. Rename `URI#full_path` to `URI#request_target`, see https://github.com/crystal-lang/crystal/pull/10099.
2021-01-31 18:52:32 +00:00
url = "#{location.request_target}&host=#{location.host}#{region ? "&region=#{region}" : ""}"
Handle redirects in /videoplayback
2019-08-27 14:53:44 +00:00
else
break
end
rescue Socket::Addrinfo::Error
if !mns.empty?
mn = mns.pop
end
fvip = "3"
host = "https://r#{fvip}---#{mn}.googlevideo.com"
client = make_client(URI.parse(host), region)
rescue ex
Add debug info to videoplayback
2020-03-06 18:50:00 +00:00
error = ex.message
Handle redirects in /videoplayback
2019-08-27 14:53:44 +00:00
end
end
if response.status_code >= 400
env.response.status_code = response.status_code
Add debug info to videoplayback
2020-03-06 18:50:00 +00:00
env.response.content_type = "text/plain"
next error
Handle redirects in /videoplayback
2019-08-27 14:53:44 +00:00
end
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
if url.includes? "&file=seg.ts"
Add administrator option to disable proxying
2019-07-07 14:07:53 +00:00
if CONFIG.disabled?("livestreams")
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(403, "Administrator has disabled this endpoint.")
Add administrator option to disable proxying
2019-07-07 14:07:53 +00:00
end
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
begin
client.get(url, headers) do |response|
response.headers.each do |key, value|
Update QUICPool
2019-11-24 18:41:47 +00:00
if !RESPONSE_HEADERS_BLACKLIST.includes?(key.downcase)
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
env.response.headers[key] = value
end
end
Add /videoplayback endpoint
2018-04-16 01:47:37 +00:00
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
Chunk video files to bypass throttling
2019-07-04 20:30:00 +00:00
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
if location = response.headers["Location"]?
location = URI.parse(location)
Update code for Crystal 0.36.0 Rename `HTTPClient@socket` to `HTTPClient@io`, see https://github.com/crystal-lang/crystal/pull/9543. Rename `URI#full_path` to `URI#request_target`, see https://github.com/crystal-lang/crystal/pull/10099.
2021-01-31 18:52:32 +00:00
location = "#{location.request_target}&host=#{location.host}"
Chunk video files to bypass throttling
2019-07-04 20:30:00 +00:00
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
if region
location += "&region=#{region}"
end
Fix formatting
2019-05-26 15:53:56 +00:00
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
next env.redirect location
end
IO.copy(response.body_io, env.response)
end
rescue ex
end
else
Add administrator option to disable proxying
2019-07-07 14:07:53 +00:00
if query_params["title"]? && CONFIG.disabled?("downloads") ||
CONFIG.disabled?("dash")
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(403, "Administrator has disabled this endpoint.")
Add administrator option to disable proxying
2019-07-07 14:07:53 +00:00
end
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
content_length = nil
first_chunk = true
range_start, range_end = parse_range(env.request.headers["Range"]?)
chunk_start = range_start
chunk_end = range_end
if !chunk_end || chunk_end - chunk_start > HTTP_CHUNK_SIZE
chunk_end = chunk_start + HTTP_CHUNK_SIZE - 1
Chunk video files to bypass throttling
2019-07-04 20:30:00 +00:00
end
Fix CORS headers for proxied assets
2019-05-19 12:12:45 +00:00
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
# TODO: Record bytes written so we can restart after a chunk fails
while true
if !range_end && content_length
range_end = content_length
end
Fix local redirects in /videoplayback
2019-03-27 19:59:53 +00:00
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
if range_end && chunk_start > range_end
break
end
if range_end && chunk_end > range_end
chunk_end = range_end
end
Fix local redirects in /videoplayback
2019-03-27 19:59:53 +00:00
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
headers["Range"] = "bytes=#{chunk_start}-#{chunk_end}"
Chunk video files to bypass throttling
2019-07-04 20:30:00 +00:00
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
begin
client.get(url, headers) do |response|
if first_chunk
if !env.request.headers["Range"]? && response.status_code == 206
env.response.status_code = 200
else
env.response.status_code = response.status_code
end
Chunk video files to bypass throttling
2019-07-04 20:30:00 +00:00
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
response.headers.each do |key, value|
Update QUICPool
2019-11-24 18:41:47 +00:00
if !RESPONSE_HEADERS_BLACKLIST.includes?(key.downcase) && key.downcase != "content-range"
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
env.response.headers[key] = value
end
Chunk video files to bypass throttling
2019-07-04 20:30:00 +00:00
end
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
Chunk video files to bypass throttling
2019-07-04 20:30:00 +00:00
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
if location = response.headers["Location"]?
location = URI.parse(location)
Update code for Crystal 0.36.0 Rename `HTTPClient@socket` to `HTTPClient@io`, see https://github.com/crystal-lang/crystal/pull/9543. Rename `URI#full_path` to `URI#request_target`, see https://github.com/crystal-lang/crystal/pull/10099.
2021-01-31 18:52:32 +00:00
location = "#{location.request_target}&host=#{location.host}#{region ? "&region=#{region}" : ""}"
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
env.redirect location
break
end
if title = query_params["title"]?
# https://blog.fastmail.com/2011/06/24/download-non-english-filenames/
Update to Crystal 0.31.0, resolve compiler deprecation warnings, update dependencies (#764) * shard: update to crystal 0.31.0 Additionally, no longer use the Crystal "markdown" library which has been removed from the Crystal stdlib in version 0.31.0. See https://github.com/crystal-lang/crystal/pull/8115. Also fix some deprecation warnings using the following commands: find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.escape/URI\.encode_www_form/g' "{}" \; find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.unescape/URI\.decode_www_form/g' "{}" \; sed -i 's/while \%pull\.kind \!\= \:end_object/until \%pull\.kind\.end_object\?/g' src/invidious/helpers/patch_mapping.cr
2019-09-24 17:31:33 +00:00
env.response.headers["Content-Disposition"] = "attachment; filename=\"#{URI.encode_www_form(title)}\"; filename*=UTF-8''#{URI.encode_www_form(title)}"
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
end
if !response.headers.includes_word?("Transfer-Encoding", "chunked")
content_length = response.headers["Content-Range"].split("/")[-1].to_i64
if env.request.headers["Range"]?
env.response.headers["Content-Range"] = "bytes #{range_start}-#{range_end || (content_length - 1)}/#{content_length}"
env.response.content_length = ((range_end.try &.+ 1) || content_length) - range_start
else
env.response.content_length = content_length
end
Fix chunking for livestream segments
2019-07-05 16:02:12 +00:00
end
Chunk video files to bypass throttling
2019-07-04 20:30:00 +00:00
end
Revert "Chunk videoplayback response to avoid throttling" This reverts commit 818cd2454de4d6ea20d4ac2765a7d4e4083eecd9.
2019-07-01 15:45:09 +00:00
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
proxy_file(response, env)
end
rescue ex
if ex.message != "Error reading socket: Connection reset by peer"
break
Handle redirects in /videoplayback
2019-08-27 14:53:44 +00:00
else
Close http clients after using The crystal http client maintains a keepalive connection to the other server which stays alive for some time. This should be closed if the client instance is not used again to avoid hogging resources
2020-12-23 05:52:23 +00:00
client.close
Handle redirects in /videoplayback
2019-08-27 14:53:44 +00:00
client = make_client(URI.parse(host), region)
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
end
Revert "Chunk videoplayback response to avoid throttling" This reverts commit 818cd2454de4d6ea20d4ac2765a7d4e4083eecd9.
2019-07-01 15:45:09 +00:00
end
Fix chunking for livestream segments
2019-07-05 16:02:12 +00:00
Fix chunk size for livestreams
2019-07-05 16:34:22 +00:00
chunk_start = chunk_end + 1
chunk_end += HTTP_CHUNK_SIZE
first_chunk = false
Fix duplicate file extension
2019-03-25 21:32:11 +00:00
end
Fix formatting
2019-05-26 15:53:56 +00:00
end
Close http clients after using The crystal http client maintains a keepalive connection to the other server which stays alive for some time. This should be closed if the client instance is not used again to avoid hogging resources
2020-12-23 05:52:23 +00:00
client.close
Proxy thumbnails
2018-09-15 02:24:28 +00:00
end
Proxy profile pictures
2018-09-17 23:39:28 +00:00
get "/ggpht/*" do |env|
url = env.request.path.lchop("/ggpht")
Cleanup request headers
2020-05-08 14:00:53 +00:00
headers = HTTP::Headers{":authority" => "yt3.ggpht.com"}
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
REQUEST_HEADERS_WHITELIST.each do |header|
Add storyboards and fix image caching
2019-04-11 22:00:00 +00:00
if env.request.headers[header]?
headers[header] = env.request.headers[header]
end
end
Catch connection reset when proxying files
2019-05-26 14:41:12 +00:00
begin
Replace static asset requests with QUIC
2020-03-06 18:53:35 +00:00
YT_POOL.client &.get(url, headers) do |response|
Fix body when proxying assets with status code > 300
2019-07-03 18:13:40 +00:00
env.response.status_code = response.status_code
Fix formatting
2019-05-26 15:53:56 +00:00
response.headers.each do |key, value|
Update QUICPool
2019-11-24 18:41:47 +00:00
if !RESPONSE_HEADERS_BLACKLIST.includes?(key.downcase)
Fix formatting
2019-05-26 15:53:56 +00:00
env.response.headers[key] = value
end
Fix CORS headers for proxied assets
2019-05-19 12:12:45 +00:00
end
Add storyboards and fix image caching
2019-04-11 22:00:00 +00:00
Fix body when proxying assets with status code > 300
2019-07-03 18:13:40 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
if response.status_code >= 300
Fix 404 for video thumbnails
2019-07-03 18:54:15 +00:00
env.response.headers.delete("Transfer-Encoding")
Fix formatting
2019-05-26 15:53:56 +00:00
break
end
Add storyboards and fix image caching
2019-04-11 22:00:00 +00:00
Fix formatting
2019-05-26 15:53:56 +00:00
proxy_file(response, env)
end
Catch connection reset when proxying files
2019-05-26 14:41:12 +00:00
rescue ex
end
Add storyboards and fix image caching
2019-04-11 22:00:00 +00:00
end
Fix storyboards
2020-10-25 08:35:16 +00:00
options "/sb/:authority/:id/:storyboard/:index" do |env|
Add video previews
2019-05-02 19:20:19 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
env.response.headers["Access-Control-Allow-Methods"] = "GET, OPTIONS"
env.response.headers["Access-Control-Allow-Headers"] = "Content-Type, Range"
end
Fix storyboards
2020-10-25 08:35:16 +00:00
get "/sb/:authority/:id/:storyboard/:index" do |env|
authority = env.params.url["authority"]
Add storyboards and fix image caching
2019-04-11 22:00:00 +00:00
id = env.params.url["id"]
storyboard = env.params.url["storyboard"]
index = env.params.url["index"]
Replace static asset requests with QUIC
2020-03-06 18:53:35 +00:00
url = "/sb/#{id}/#{storyboard}/#{index}?#{env.params.query}"
headers = HTTP::Headers.new
Fix storyboards
2020-10-25 08:35:16 +00:00
headers[":authority"] = "#{authority}.ytimg.com"
Add storyboards and fix image caching
2019-04-11 22:00:00 +00:00
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
REQUEST_HEADERS_WHITELIST.each do |header|
Add '&local=true' to watch and embed pages
2019-03-11 16:43:48 +00:00
if env.request.headers[header]?
headers[header] = env.request.headers[header]
end
end
Proxy profile pictures
2018-09-17 23:39:28 +00:00
Catch connection reset when proxying files
2019-05-26 14:41:12 +00:00
begin
Replace static asset requests with QUIC
2020-03-06 18:53:35 +00:00
YT_POOL.client &.get(url, headers) do |response|
Fix formatting
2019-05-26 15:53:56 +00:00
env.response.status_code = response.status_code
response.headers.each do |key, value|
Update QUICPool
2019-11-24 18:41:47 +00:00
if !RESPONSE_HEADERS_BLACKLIST.includes?(key.downcase)
Fix formatting
2019-05-26 15:53:56 +00:00
env.response.headers[key] = value
end
Fix CORS headers for proxied assets
2019-05-19 12:12:45 +00:00
end
Proxy profile pictures
2018-09-17 23:39:28 +00:00
Prevent storyboards from hanging
2020-03-04 20:36:39 +00:00
env.response.headers["Connection"] = "close"
Fix body when proxying assets with status code > 300
2019-07-03 18:13:40 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
if response.status_code >= 300
Fix 404 for video thumbnails
2019-07-03 18:54:15 +00:00
env.response.headers.delete("Transfer-Encoding")
Fix formatting
2019-05-26 15:53:56 +00:00
break
end
Proxy profile pictures
2018-09-17 23:39:28 +00:00
Fix formatting
2019-05-26 15:53:56 +00:00
proxy_file(response, env)
end
Catch connection reset when proxying files
2019-05-26 14:41:12 +00:00
rescue ex
end
Proxy profile pictures
2018-09-17 23:39:28 +00:00
end
Fix season playlists
2019-08-16 20:46:37 +00:00
get "/s_p/:id/:name" do |env|
id = env.params.url["id"]
name = env.params.url["name"]
url = env.request.resource
Cleanup request headers
2020-05-08 14:00:53 +00:00
headers = HTTP::Headers{":authority" => "i9.ytimg.com"}
Fix season playlists
2019-08-16 20:46:37 +00:00
REQUEST_HEADERS_WHITELIST.each do |header|
if env.request.headers[header]?
headers[header] = env.request.headers[header]
end
end
begin
Replace static asset requests with QUIC
2020-03-06 18:53:35 +00:00
YT_POOL.client &.get(url, headers) do |response|
Fix season playlists
2019-08-16 20:46:37 +00:00
env.response.status_code = response.status_code
response.headers.each do |key, value|
Update QUICPool
2019-11-24 18:41:47 +00:00
if !RESPONSE_HEADERS_BLACKLIST.includes?(key.downcase)
Fix season playlists
2019-08-16 20:46:37 +00:00
env.response.headers[key] = value
end
Add support for '/yts/img' endpoint
2019-11-01 16:02:38 +00:00
end
env.response.headers["Access-Control-Allow-Origin"] = "*"
if response.status_code >= 300 && response.status_code != 404
env.response.headers.delete("Transfer-Encoding")
break
end
proxy_file(response, env)
end
rescue ex
end
end
get "/yts/img/:name" do |env|
headers = HTTP::Headers.new
REQUEST_HEADERS_WHITELIST.each do |header|
if env.request.headers[header]?
headers[header] = env.request.headers[header]
end
end
begin
YT_POOL.client &.get(env.request.resource, headers) do |response|
env.response.status_code = response.status_code
response.headers.each do |key, value|
Update QUICPool
2019-11-24 18:41:47 +00:00
if !RESPONSE_HEADERS_BLACKLIST.includes?(key.downcase)
Add support for '/yts/img' endpoint
2019-11-01 16:02:38 +00:00
env.response.headers[key] = value
end
Fix season playlists
2019-08-16 20:46:37 +00:00
end
env.response.headers["Access-Control-Allow-Origin"] = "*"
if response.status_code >= 300 && response.status_code != 404
env.response.headers.delete("Transfer-Encoding")
break
end
proxy_file(response, env)
end
rescue ex
end
end
Proxy thumbnails
2018-09-15 02:24:28 +00:00
get "/vi/:id/:name" do |env|
id = env.params.url["id"]
name = env.params.url["name"]
Cleanup request headers
2020-05-08 14:00:53 +00:00
headers = HTTP::Headers{":authority" => "i.ytimg.com"}
Replace static asset requests with QUIC
2020-03-06 18:53:35 +00:00
Proxy thumbnails
2018-09-15 02:24:28 +00:00
if name == "maxres.jpg"
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
build_thumbnails(id).each do |thumb|
Replace static asset requests with QUIC
2020-03-06 18:53:35 +00:00
if YT_POOL.client &.head("/vi/#{id}/#{thumb[:url]}.jpg", headers).status_code == 200
Proxy thumbnails
2018-09-15 02:24:28 +00:00
name = thumb[:url] + ".jpg"
break
end
end
end
url = "/vi/#{id}/#{name}"
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
REQUEST_HEADERS_WHITELIST.each do |header|
Add '&local=true' to watch and embed pages
2019-03-11 16:43:48 +00:00
if env.request.headers[header]?
headers[header] = env.request.headers[header]
end
end
Proxy thumbnails
2018-09-15 02:24:28 +00:00
Catch connection reset when proxying files
2019-05-26 14:41:12 +00:00
begin
Replace static asset requests with QUIC
2020-03-06 18:53:35 +00:00
YT_POOL.client &.get(url, headers) do |response|
Fix formatting
2019-05-26 15:53:56 +00:00
env.response.status_code = response.status_code
response.headers.each do |key, value|
Update QUICPool
2019-11-24 18:41:47 +00:00
if !RESPONSE_HEADERS_BLACKLIST.includes?(key.downcase)
Fix formatting
2019-05-26 15:53:56 +00:00
env.response.headers[key] = value
end
Fix CORS headers for proxied assets
2019-05-19 12:12:45 +00:00
end
Proxy thumbnails
2018-09-15 02:24:28 +00:00
Fix body when proxying assets with status code > 300
2019-07-03 18:13:40 +00:00
env.response.headers["Access-Control-Allow-Origin"] = "*"
Fix 404 for video thumbnails
2019-07-03 18:54:15 +00:00
if response.status_code >= 300 && response.status_code != 404
env.response.headers.delete("Transfer-Encoding")
Fix formatting
2019-05-26 15:53:56 +00:00
break
end
Proxy thumbnails
2018-09-15 02:24:28 +00:00
Fix formatting
2019-05-26 15:53:56 +00:00
proxy_file(response, env)
end
Catch connection reset when proxying files
2019-05-26 14:41:12 +00:00
rescue ex
end
Add /videoplayback endpoint
2018-04-16 01:47:37 +00:00
end
Add support for image captcha in Google login
2019-10-27 04:19:05 +00:00
get "/Captcha" do |env|
Cleanup request headers
2020-05-08 14:00:53 +00:00
headers = HTTP::Headers{":authority" => "accounts.google.com"}
response = YT_POOL.client &.get(env.request.resource, headers)
Add support for image captcha in Google login
2019-10-27 04:19:05 +00:00
env.response.headers["Content-Type"] = response.headers["Content-Type"]
response.body
end
Refactor search extractor
2019-08-21 23:23:20 +00:00
# Undocumented, creates anonymous playlist with specified 'video_ids', max 50 videos
Add '/watch_videos' endpoint
2019-05-03 14:11:27 +00:00
get "/watch_videos" do |env|
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
response = YT_POOL.client &.get(env.request.resource)
Add '/watch_videos' endpoint
2019-05-03 14:11:27 +00:00
if url = response.headers["Location"]?
Update code for Crystal 0.36.0 Rename `HTTPClient@socket` to `HTTPClient@io`, see https://github.com/crystal-lang/crystal/pull/9543. Rename `URI#full_path` to `URI#request_target`, see https://github.com/crystal-lang/crystal/pull/10099.
2021-01-31 18:52:32 +00:00
url = URI.parse(url).request_target
Add '/watch_videos' endpoint
2019-05-03 14:11:27 +00:00
next env.redirect url
end
env.response.status_code = response.status_code
end
Use error template for 404 and 500
2018-02-10 15:15:23 +00:00
error 404 do |env|
Fix redirect for shortened video urls
2019-03-27 10:28:53 +00:00
if md = env.request.path.match(/^\/(?<id>([a-zA-Z0-9_-]{11})|(\w+))$/)
Fix typo in 404 handler
2019-04-17 19:46:00 +00:00
item = md["id"]
Add extra handling for shortened video URLs
2018-10-07 03:19:36 +00:00
Fix typo in 404 handler
2019-04-17 19:46:00 +00:00
# Check if item is branding URL e.g. https://youtube.com/gaming
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
response = YT_POOL.client &.get("/#{item}")
Fix redirect for shortened video urls
2019-03-27 10:28:53 +00:00
if response.status_code == 301
Update code for Crystal 0.36.0 Rename `HTTPClient@socket` to `HTTPClient@io`, see https://github.com/crystal-lang/crystal/pull/9543. Rename `URI#full_path` to `URI#request_target`, see https://github.com/crystal-lang/crystal/pull/10099.
2021-01-31 18:52:32 +00:00
response = YT_POOL.client &.get(URI.parse(response.headers["Location"]).request_target)
Fix redirect for shortened video urls
2019-03-27 10:28:53 +00:00
end
Fix 404 handling for endpoints matching short URLs
2019-06-07 17:42:07 +00:00
if response.body.empty?
env.response.headers["Location"] = "/"
halt env, status_code: 302
end
Fix redirect for shortened video urls
2019-03-27 10:28:53 +00:00
html = XML.parse_html(response.body)
Fix channel redirect
2020-01-14 13:21:17 +00:00
ucid = html.xpath_node(%q(//link[@rel="canonical"])).try &.["href"].split("/")[-1]
Fix redirect for shortened video urls
2019-03-27 10:28:53 +00:00
if ucid
Fix channel redirect
2020-01-14 13:21:17 +00:00
env.response.headers["Location"] = "/channel/#{ucid}"
Fix redirect for shortened video urls
2019-03-27 10:28:53 +00:00
halt env, status_code: 302
end
Add extra handling for shortened video URLs
2018-10-07 03:19:36 +00:00
params = [] of String
env.params.query.each do |k, v|
params << "#{k}=#{v}"
end
params = params.join("&")
Fix typo in 404 handler
2019-04-17 19:46:00 +00:00
url = "/watch?v=#{item}"
Add extra handling for shortened video URLs
2018-10-07 03:19:36 +00:00
if !params.empty?
url += "&#{params}"
end
Fix typo in 404 handler
2019-04-17 19:46:00 +00:00
# Check if item is video ID
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
if item.match(/^[a-zA-Z0-9_-]{11}$/) && YT_POOL.client &.head("/watch?v=#{item}").status_code != 404
Add support for custom channel URLs
2019-02-21 21:07:22 +00:00
env.response.headers["Location"] = url
halt env, status_code: 302
end
end
Improve error message for 500 and add redirect for 404
2019-01-12 19:18:08 +00:00
env.response.headers["Location"] = "/"
halt env, status_code: 302
Add search
2017-12-30 21:21:43 +00:00
end
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
error 500 do |env, ex|
locale = LOCALES[env.get("preferences").as(Preferences).locale]?
error_template(500, ex)
Add search
2017-12-30 21:21:43 +00:00
end
Add max-age for static assets
2018-03-09 17:28:57 +00:00
static_headers do |response, filepath, filestat|
Add simple form of cache busting
2019-05-08 13:58:10 +00:00
response.headers.add("Cache-Control", "max-age=2629800")
Add max-age for static assets
2018-03-09 17:28:57 +00:00
end
Initial commit
2017-11-23 07:48:55 +00:00
public_folder "assets"
Add filtered handler for videoplayback
2018-04-16 03:56:58 +00:00
Disable Kemal header
2018-07-30 23:42:45 +00:00
Kemal.config.powered_by_header = false
Add filtered handler for videoplayback
2018-04-16 03:56:58 +00:00
add_handler FilteredCompressHandler.new
Add CORS to API endpoints
2019-02-03 04:48:47 +00:00
add_handler APIHandler.new
Add authentication API
2019-04-18 21:23:50 +00:00
add_handler AuthHandler.new
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
add_handler DenyFrame.new
Add authentication API
2019-04-18 21:23:50 +00:00
add_context_storage_type(Array(String))
Allow user to save preferences without creating an account
2019-02-24 15:49:48 +00:00
add_context_storage_type(Preferences)
Add authentication API
2019-04-18 21:23:50 +00:00
add_context_storage_type(User)
Initial commit
2017-11-23 07:48:55 +00:00
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
Kemal.config.logger = LOGGER
Add config options for host binding and port
2019-09-23 17:05:29 +00:00
Kemal.config.host_binding = Kemal.config.host_binding != "0.0.0.0" ? Kemal.config.host_binding : CONFIG.host_binding
Kemal.config.port = Kemal.config.port != 3000 ? Kemal.config.port : CONFIG.port
Initial commit
2017-11-23 07:48:55 +00:00
Kemal.run
Reference in New Issue Copy Permalink