Mirrored_Repos/invidious
1
0
Fork 0
mirror of https://github.com/iv-org/invidious.git synced 2024-11-14 12:18:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
094f835642
invidious/src/invidious.cr

1372 lines
42 KiB
Crystal
Raw Normal View History

Update project synopsis
2018-09-04 14:22:10 +00:00
# "Invidious" (which is an alternative front-end to YouTube)
Update copyright notice
2019-03-15 16:44:53 +00:00
# Copyright (C) 2019 Omar Roth
Change license
2018-01-28 17:32:40 +00:00
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published
# by the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
Add text CAPTCHA
2018-11-22 19:26:08 +00:00
require "digest/md5"
Add -o option for redirecting output
2019-01-23 20:15:19 +00:00
require "file_utils"
Initial commit
2017-11-23 07:48:55 +00:00
require "kemal"
Use `athena-negotiation` to detect language through Accept-Language header (#2324) Detect language through Accept-Language header
2021-08-24 19:59:27 +00:00
require "athena-negotiation"
Add separate user accounts
2018-07-18 19:26:02 +00:00
require "openssl/hmac"
Add ability to specify number of threads
2018-02-03 22:13:14 +00:00
require "option_parser"
Properly parse NewPipe imports
2018-11-21 23:12:13 +00:00
require "sqlite3"
Fix audio-only for video.js
2018-01-16 20:02:35 +00:00
require "xml"
Add config file
2018-03-09 18:42:23 +00:00
require "yaml"
Support Crystal 0.35.0
2020-06-15 22:57:20 +00:00
require "compress/zip"
Refactor continuation token handling
2019-10-27 17:50:42 +00:00
require "protodec/utils"
Move DB queries related to 'videos' in a separate module
2021-11-26 18:36:31 +00:00
require "./invidious/database/*"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
require "./invidious/helpers/*"
Add 'require' statement to 'invidious.cr'
2021-10-07 20:32:04 +00:00
require "./invidious/yt_backend/*"
Remove patch and update file structure
2018-07-06 12:59:56 +00:00
require "./invidious/*"
Debloat channels.cr into multiple files (#2225) Cherry picked from ui overhaul branch with a few modifications: - channel folder is renamed to channels - parsing for channel home and featured channels are removed due to lack of infrastructure from other commits (cherry picked from commit 44d18b8e147b47ad06a54cc6fd08423d9f39074d)
2021-07-14 15:46:12 +00:00
require "./invidious/channels/*"
Add mising 'require' statement to 'invidious.cr'
2021-10-07 20:00:50 +00:00
require "./invidious/user/*"
Provide rough draft of better project organization
2020-10-06 04:41:18 +00:00
require "./invidious/routes/**"
require "./invidious/jobs/**"
Move views to src/ directory
2017-11-29 21:33:46 +00:00
Add config environment variables The config file can now be specified with `INVIDIOUS_CONFIG_FILE`. A YAML formatted string can still be passed with `INVIDIOUS_CONFIG`, replacing the config file. Additionally all options can now be specified as environment variables. The syntax for variable names is `INVIDIOUS_` followed by the option name in upper case. The values are parsed as YAML. These new env vars only update the provided main configuration, but it is possible to point the config file at the example config and then use env vars for all config options: ``` INVIDIOUS_CONFIG_FILE=./config/config.example.yml \ INVIDIOUS_CHANNEL_THREADS=10 \ ./invidious ```
2021-01-23 17:58:13 +00:00
CONFIG = Config.load
HMAC_KEY = CONFIG.hmac_key || Random::Secure.hex(32)
Add config file
2018-03-09 18:42:23 +00:00
Allow providing 12-Factor-style Database URL in config
2021-01-30 14:52:48 +00:00
PG_DB = DB.open CONFIG.database_url
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
ARCHIVE_URL = URI.parse("https://archive.org")
LOGIN_URL = URI.parse("https://accounts.google.com")
PUBSUB_URL = URI.parse("https://pubsubhubbub.appspot.com")
REDDIT_URL = URI.parse("https://www.reddit.com")
Switch textcaptcha to HTTPS
2020-03-10 15:12:11 +00:00
TEXTCAPTCHA_URL = URI.parse("https://textcaptcha.com")
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
YT_URL = URI.parse("https://www.youtube.com")
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
HOST_URL = make_host_url(Kemal.config)
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
Add '/api/v1/auth/feeds'
2019-06-07 17:39:12 +00:00
CHARS_SAFE = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_"
TEST_IDS = {"AgbeGFYluEA", "BaW_jenozKc", "a9LDPn-MO4I", "ddFvjfvPnqk", "iqKdEhx-dD4"}
Set max preference size
2019-06-08 21:04:55 +00:00
MAX_ITEMS_PER_PAGE = 1500
Fix regression for youtube pool
2018-03-05 04:25:03 +00:00
Update QUICPool
2019-11-24 18:41:47 +00:00
REQUEST_HEADERS_WHITELIST = {"accept", "accept-encoding", "cache-control", "content-length", "if-none-match", "range"}
RESPONSE_HEADERS_BLACKLIST = {"access-control-allow-origin", "alt-svc", "server"}
Chunk video files to bypass throttling
2019-07-04 20:30:00 +00:00
HTTP_CHUNK_SIZE = 10485760 # ~10MB
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
Update code formatting for Crystal 0.33.0 Crystal 0.33.0 introduced some changes to to the code formatter. Run "crystal tool format" so CI doesn't fail anymore.
2020-02-15 18:52:28 +00:00
CURRENT_BRANCH = {{ "#{`git branch | sed -n '/* /s///p'`.strip}" }}
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
CURRENT_COMMIT = {{ "#{`git rev-list HEAD --max-count=1 --abbrev-commit`.strip}" }}
Fix invidious version for docker image As Alpine-Linux per default uses the busybox date command, the -f option is not available and the build will fail.
2020-12-05 19:06:24 +00:00
CURRENT_VERSION = {{ "#{`git log -1 --format=%ci | awk '{print $1}' | sed s/-/./g`.strip}" }}
Refactor commonly used request and response headers
2019-06-23 13:39:14 +00:00
Use separate asset version for cache busting
2019-05-09 16:52:37 +00:00
# This is used to determine the `?v=` on the end of file URLs (for cache busting). We
# only need to expire modified assets, so we can use this to find the last commit that changes
# any assets
ASSET_COMMIT = {{ "#{`git rev-list HEAD --max-count=1 --abbrev-commit -- assets`.strip}" }}
Add '--version' to command line
2019-04-06 13:28:53 +00:00
SOFTWARE = {
"name" => "invidious",
"version" => "#{CURRENT_VERSION}-#{CURRENT_COMMIT}",
"branch" => "#{CURRENT_BRANCH}",
}
Use default timeout (5 seconds) for YT pool (#2430)
2021-09-26 21:03:45 +00:00
YT_POOL = YoutubeConnectionPool.new(YT_URL, capacity: CONFIG.pool_size, use_quic: CONFIG.use_quic)
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
# CLI
Add '--version' to command line
2019-04-06 13:28:53 +00:00
Kemal.config.extra_options do |parser|
parser.banner = "Usage: invidious [arguments]"
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
parser.on("-c THREADS", "--channel-threads=THREADS", "Number of threads for refreshing channels (default: #{CONFIG.channel_threads})") do |number|
Add '--version' to command line
2019-04-06 13:28:53 +00:00
begin
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
CONFIG.channel_threads = number.to_i
Add '--version' to command line
2019-04-06 13:28:53 +00:00
rescue ex
puts "THREADS must be integer"
exit
end
end
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
parser.on("-f THREADS", "--feed-threads=THREADS", "Number of threads for refreshing feeds (default: #{CONFIG.feed_threads})") do |number|
Add '--version' to command line
2019-04-06 13:28:53 +00:00
begin
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
CONFIG.feed_threads = number.to_i
Add '--version' to command line
2019-04-06 13:28:53 +00:00
rescue ex
puts "THREADS must be integer"
exit
end
end
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
parser.on("-o OUTPUT", "--output=OUTPUT", "Redirect output (default: #{CONFIG.output})") do |output|
CONFIG.output = output
Add '--version' to command line
2019-04-06 13:28:53 +00:00
end
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
parser.on("-l LEVEL", "--log-level=LEVEL", "Log level, one of #{LogLevel.values} (default: #{CONFIG.log_level})") do |log_level|
CONFIG.log_level = LogLevel.parse(log_level)
Improve logging Everything that gets logged now has a log level associated with it. The log level can be set with the new `-l` or `--log-level` arguments. The defaul log level is `debug` for now. There aren't many things that get logged but if the logs get spammed in the future it can be set down to `info`.
2020-12-21 15:05:35 +00:00
end
parser.on("-v", "--version", "Print version") do
Add '--version' to command line
2019-04-06 13:28:53 +00:00
puts SOFTWARE.to_pretty_json
exit
end
end
Kemal::CLI.new ARGV
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
if CONFIG.output.upcase != "STDOUT"
FileUtils.mkdir_p(File.dirname(CONFIG.output))
Add CLI arguments to config file The log level can now be set with `log_level` (accepts ints and strings). The log file can now be set with `output` (also accepts `STDOUT`).
2021-01-04 15:05:15 +00:00
end
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
OUTPUT = CONFIG.output.upcase == "STDOUT" ? STDOUT : File.open(CONFIG.output, mode: "a")
LOGGER = Invidious::LogHandler.new(OUTPUT, CONFIG.log_level)
Add CLI arguments to config file The log level can now be set with `log_level` (accepts ints and strings). The log file can now be set with `output` (also accepts `STDOUT`).
2021-01-04 15:05:15 +00:00
Add config option to cache annotations from IA
2019-04-15 16:13:09 +00:00
# Check table integrity
Make check_tables config option
2019-04-11 17:13:25 +00:00
if CONFIG.check_tables
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
check_enum(PG_DB, "privacy", PlaylistPrivacy)
Add support for custom playlists
2019-08-05 23:49:13 +00:00
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
check_table(PG_DB, "channels", InvidiousChannel)
check_table(PG_DB, "channel_videos", ChannelVideo)
check_table(PG_DB, "playlists", InvidiousPlaylist)
check_table(PG_DB, "playlist_videos", PlaylistVideo)
check_table(PG_DB, "nonces", Nonce)
check_table(PG_DB, "session_ids", SessionId)
check_table(PG_DB, "users", User)
check_table(PG_DB, "videos", Video)
Add config option to cache annotations from IA
2019-04-15 16:13:09 +00:00
if CONFIG.cache_annotations
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
check_table(PG_DB, "annotations", Annotation)
Add config option to cache annotations from IA
2019-04-15 16:13:09 +00:00
end
Make check_tables config option
2019-04-11 17:13:25 +00:00
end
Add channel refresh
2018-03-26 03:18:29 +00:00
Automatically migrate database
2019-04-10 21:23:37 +00:00
# Start jobs
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
Only start refresh jobs when necessary If `channel_threads` or `feed_threads` is set to zero the corresponding job is now not started.
2021-01-23 18:41:50 +00:00
if CONFIG.channel_threads > 0
Invidious::Jobs.register Invidious::Jobs::RefreshChannelsJob.new(PG_DB)
end
if CONFIG.feed_threads > 0
Invidious::Jobs.register Invidious::Jobs::RefreshFeedsJob.new(PG_DB)
end
add config to decrypt on demand instead of polling
2020-09-27 17:19:44 +00:00
DECRYPT_FUNCTION = DecryptFunction.new(CONFIG.decrypt_polling)
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if CONFIG.decrypt_polling
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
Invidious::Jobs.register Invidious::Jobs::UpdateDecryptFunctionJob.new
add config to decrypt on demand instead of polling
2020-09-27 17:19:44 +00:00
end
Add support for subscribing to channels via PubSubHubbub
2019-03-04 01:18:23 +00:00
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if CONFIG.statistics_enabled
Invidious::Jobs.register Invidious::Jobs::StatisticsRefreshJob.new(PG_DB, SOFTWARE)
[refactor] Finish converting jobs to new job setup (#1420) [refactor] Finish converting jobs to new job setup
2020-10-17 12:25:57 +00:00
end
Minor refactor
2019-08-27 13:08:26 +00:00
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if (CONFIG.use_pubsub_feeds.is_a?(Bool) && CONFIG.use_pubsub_feeds.as(Bool)) || (CONFIG.use_pubsub_feeds.is_a?(Int32) && CONFIG.use_pubsub_feeds.as(Int32) > 0)
Invidious::Jobs.register Invidious::Jobs::SubscribeToFeedsJob.new(PG_DB, HMAC_KEY)
Respect `use_pubsub_feeds` config Setting `use_pubsub_feeds: false` now properly disables it.
2021-01-07 19:15:26 +00:00
end
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if CONFIG.popular_enabled
Add `popular-enabled` option This is similar to the removed `top-enabled` option but for the Popular feed. The instance needs to be restarted if the feed was enabled. Editing admin options on the preferences page is also fixed. The handling of the feed pages now only happens in a single place. Instead of redirecting: - The Top feed now displays a message that it was removed from Invidious. - The Popular feed now displays a message that it was disabled if it was.
2020-12-27 05:12:43 +00:00
Invidious::Jobs.register Invidious::Jobs::PullPopularVideosJob.new(PG_DB)
end
[refactor] Finish converting jobs to new job setup (#1420) [refactor] Finish converting jobs to new job setup
2020-10-17 12:25:57 +00:00
connection_channel = Channel({Bool, Channel(PQ::Notification)}).new(32)
Allow providing 12-Factor-style Database URL in config
2021-01-30 14:52:48 +00:00
Invidious::Jobs.register Invidious::Jobs::NotificationJob.new(connection_channel, CONFIG.database_url)
[refactor] Finish converting jobs to new job setup (#1420) [refactor] Finish converting jobs to new job setup
2020-10-17 12:25:57 +00:00
Provide rough draft of better project organization
2020-10-06 04:41:18 +00:00
Invidious::Jobs.start_all
def popular_videos
Invidious::Jobs::PullPopularVideosJob::POPULAR_VIDEOS.get
Add job for pulling popular videos
2018-11-09 02:08:03 +00:00
end
Add fix for testing authentication
2018-03-25 03:56:41 +00:00
before_all do |env|
Use `athena-negotiation` to detect language through Accept-Language header (#2324) Detect language through Accept-Language header
2021-08-24 19:59:27 +00:00
preferences = Preferences.from_json("{}")
begin
if prefs_cookie = env.request.cookies["PREFS"]?
preferences = Preferences.from_json(URI.decode_www_form(prefs_cookie.value))
else
if language_header = env.request.headers["Accept-Language"]?
if language = ANG.language_negotiator.best(language_header, LOCALES.keys)
preferences.locale = language.header
end
end
end
Migrate to a good Content Security Policy (#1023) So attacks such as XSS (see [0]) will no longer be of an issue. [0]: https://github.com/omarroth/invidious/issues/1022
2020-03-15 21:46:08 +00:00
rescue
Use `athena-negotiation` to detect language through Accept-Language header (#2324) Detect language through Accept-Language header
2021-08-24 19:59:27 +00:00
preferences = Preferences.from_json("{}")
Migrate to a good Content Security Policy (#1023) So attacks such as XSS (see [0]) will no longer be of an issue. [0]: https://github.com/omarroth/invidious/issues/1022
2020-03-15 21:46:08 +00:00
end
Set preferences before conditional breaks in before_all (#1412)
2020-10-16 10:23:18 +00:00
env.set "preferences", preferences
Fix typo in X-XSS-Protection
2019-05-10 21:48:38 +00:00
env.response.headers["X-XSS-Protection"] = "1; mode=block"
Add X-XSS-Protection and X-Content-Type-Options
2018-09-06 02:51:40 +00:00
env.response.headers["X-Content-Type-Options"] = "nosniff"
Rewrite response headers Fixes #2018 and #2153
2021-06-17 17:45:20 +00:00
Typo
2021-06-19 07:40:33 +00:00
# Allow media resources to be loaded from google servers
Rewrite response headers Fixes #2018 and #2153
2021-06-17 17:45:20 +00:00
# TODO: check if *.youtube.com can be removed
Migrate to a good Content Security Policy (#1023) So attacks such as XSS (see [0]) will no longer be of an issue. [0]: https://github.com/omarroth/invidious/issues/1022
2020-03-15 21:46:08 +00:00
if CONFIG.disabled?("local") || !preferences.local
Rewrite response headers Fixes #2018 and #2153
2021-06-17 17:45:20 +00:00
extra_media_csp = " https://*.googlevideo.com:443 https://*.youtube.com:443"
else
extra_media_csp = ""
end
# Only allow the pages at /embed/* to be embedded
if env.request.resource.starts_with?("/embed")
frame_ancestors = "'self' http: https:"
else
Fix quoting of 'none' in CSP header The keyword 'none' must be surrounded by single quotes. Regression introduced by #2168.
2021-06-20 16:43:00 +00:00
frame_ancestors = "'none'"
Migrate to a good Content Security Policy (#1023) So attacks such as XSS (see [0]) will no longer be of an issue. [0]: https://github.com/omarroth/invidious/issues/1022
2020-03-15 21:46:08 +00:00
end
Rewrite response headers Fixes #2018 and #2153
2021-06-17 17:45:20 +00:00
# TODO: Remove style-src's 'unsafe-inline', requires to remove all
# inline styles (<style> [..] </style>, style=" [..] ")
env.response.headers["Content-Security-Policy"] = {
"default-src 'none'",
"script-src 'self'",
"style-src 'self' 'unsafe-inline'",
"img-src 'self' data:",
"font-src 'self' data:",
"connect-src 'self'",
"manifest-src 'self'",
"media-src 'self' blob:" + extra_media_csp,
"child-src 'self' blob:",
"frame-src 'self'",
"frame-ancestors " + frame_ancestors,
}.join("; ")
Add CSP, STS, and Referrer-Policy
2019-04-07 19:01:08 +00:00
env.response.headers["Referrer-Policy"] = "same-origin"
Rewrite response headers Fixes #2018 and #2153
2021-06-17 17:45:20 +00:00
# Ask the chrom*-based browsers to disable FLoC
# See: https://blog.runcloud.io/google-floc/
env.response.headers["Permissions-Policy"] = "interest-cohort=()"
Make config a constant Instead of passing around `config` there is now the global `CONFIG`.
2021-01-23 18:39:04 +00:00
if (Kemal.config.ssl || CONFIG.https_only) && CONFIG.hsts
Bump max-age for HSTS
2019-05-01 01:53:56 +00:00
env.response.headers["Strict-Transport-Security"] = "max-age=31536000; includeSubDomains; preload"
Add CSP, STS, and Referrer-Policy
2019-04-07 19:01:08 +00:00
end
Update handling for preferences
2019-03-28 18:43:40 +00:00
Skip preferences for assets
2019-11-20 17:03:52 +00:00
next if {
"/sb/",
"/vi/",
"/s_p/",
"/yts/",
"/ggpht/",
"/api/manifest/",
"/videoplayback",
"/latest_version",
}.any? { |r| env.request.resource.starts_with? r }
Add user preferences
2018-07-16 16:24:24 +00:00
if env.request.cookies.has_key? "SID"
Add notifications
2018-04-01 00:09:27 +00:00
sid = env.request.cookies["SID"].value
Move user state into before_all
2018-07-05 23:43:26 +00:00
Add authentication API
2019-04-18 21:23:50 +00:00
if sid.starts_with? "v1:"
raise "Cannot use token as SID"
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
# Invidious users only have SID
if !env.request.cookies.has_key? "SSID"
Move DB queries related to session tokens in a separate module
2021-12-02 22:57:13 +00:00
if email = Invidious::Database::SessionIDs.select_email(sid)
Move DB queries related to 'users' in a separate module (1/2)
2021-12-03 01:27:51 +00:00
user = Invidious::Database::Users.select!(email: email)
Add support for custom playlists
2019-08-05 23:49:13 +00:00
csrf_token = generate_response(sid, {
":authorize_token",
":playlist_ajax",
":signout",
":subscription_ajax",
":token_ajax",
":watch_ajax",
}, HMAC_KEY, PG_DB, 1.week)
Add CSRF prevention for /signout
2018-11-08 23:42:25 +00:00
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
preferences = user.preferences
Set preferences before conditional breaks in before_all (#1412)
2020-10-16 10:23:18 +00:00
env.set "preferences", preferences
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
Add support for multiple sessions
2018-08-15 17:40:42 +00:00
env.set "sid", sid
Add authentication API
2019-04-18 21:23:50 +00:00
env.set "csrf_token", csrf_token
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
env.set "user", user
Add separate user accounts
2018-07-18 19:26:02 +00:00
end
else
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
headers = HTTP::Headers.new
headers["Cookie"] = env.request.headers["Cookie"]
Add separate user accounts
2018-07-18 19:26:02 +00:00
begin
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user, sid = get_user(sid, headers, PG_DB, false)
Add support for custom playlists
2019-08-05 23:49:13 +00:00
csrf_token = generate_response(sid, {
":authorize_token",
":playlist_ajax",
":signout",
":subscription_ajax",
":token_ajax",
":watch_ajax",
}, HMAC_KEY, PG_DB, 1.week)
Add CSRF token for Google accounts
2018-11-16 02:23:17 +00:00
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
preferences = user.preferences
Set preferences before conditional breaks in before_all (#1412)
2020-10-16 10:23:18 +00:00
env.set "preferences", preferences
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
Add support for multiple sessions
2018-08-15 17:40:42 +00:00
env.set "sid", sid
Add authentication API
2019-04-18 21:23:50 +00:00
env.set "csrf_token", csrf_token
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
env.set "user", user
Add separate user accounts
2018-07-18 19:26:02 +00:00
rescue ex
end
Add handling for invalid login
2018-07-16 17:50:41 +00:00
end
Add dark theme
2018-04-14 02:32:14 +00:00
end
Fix referers
2018-08-17 15:19:20 +00:00
Add prefers-color-scheme support (#601) * Add prefers-color-scheme support This should fix <https://github.com/omarroth/invidious/issues/559>. The cookie storage format has been changed from boolean ("true"/"false") to tri-state ("dark"/"light"/""), so that users without a cookie set will get dark mode if they have enabled the dark theme in their operating system. The code for handling the cookie state, along with the user's operating system theme, has been factored out into a new function `update_mode`, which is called both at window load and at the "storage" event listener, because the "storage" event listener is only trigerred when a change is made to the localStorage from another tab/window (for more info - see <https://stackoverflow.com/a/4679754>).
2019-08-15 16:29:55 +00:00
dark_mode = convert_theme(env.params.query["dark_mode"]?) || preferences.dark_mode.to_s
Add 'dark_mode', 'thin_mode' as query parameters
2019-03-11 17:44:25 +00:00
thin_mode = env.params.query["thin_mode"]? || preferences.thin_mode.to_s
thin_mode = thin_mode == "true"
locale = env.params.query["hl"]? || preferences.locale
preferences.dark_mode = dark_mode
preferences.thin_mode = thin_mode
preferences.locale = locale
Fix: functional hl / dark_mode / thin_mode params
2021-03-17 19:07:38 +00:00
env.set "preferences", preferences
Add support for translations
2018-12-20 21:32:09 +00:00
Fix referers
2018-08-17 15:19:20 +00:00
current_page = env.request.path
if env.request.query
query = HTTP::Params.parse(env.request.query.not_nil!)
if query["referer"]?
query["referer"] = get_referer(env, "/")
end
current_page += "?#{query}"
end
Update to Crystal 0.31.0, resolve compiler deprecation warnings, update dependencies (#764) * shard: update to crystal 0.31.0 Additionally, no longer use the Crystal "markdown" library which has been removed from the Crystal stdlib in version 0.31.0. See https://github.com/crystal-lang/crystal/pull/8115. Also fix some deprecation warnings using the following commands: find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.escape/URI\.encode_www_form/g' "{}" \; find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.unescape/URI\.decode_www_form/g' "{}" \; sed -i 's/while \%pull\.kind \!\= \:end_object/until \%pull\.kind\.end_object\?/g' src/invidious/helpers/patch_mapping.cr
2019-09-24 17:31:33 +00:00
env.set "current_page", URI.encode_www_form(current_page)
Add logout
2018-03-22 17:44:36 +00:00
end
Add support to make invidious API-only via flag (#2363)
2021-10-02 20:04:02 +00:00
{% unless flag?(:api_only) %}
Invidious::Routing.get "/", Invidious::Routes::Misc, :home
Invidious::Routing.get "/privacy", Invidious::Routes::Misc, :privacy
Invidious::Routing.get "/licenses", Invidious::Routes::Misc, :licenses
Invidious::Routing.get "/channel/:ucid", Invidious::Routes::Channels, :home
Invidious::Routing.get "/channel/:ucid/home", Invidious::Routes::Channels, :home
Invidious::Routing.get "/channel/:ucid/videos", Invidious::Routes::Channels, :videos
Invidious::Routing.get "/channel/:ucid/playlists", Invidious::Routes::Channels, :playlists
Invidious::Routing.get "/channel/:ucid/community", Invidious::Routes::Channels, :community
Invidious::Routing.get "/channel/:ucid/about", Invidious::Routes::Channels, :about
["", "/videos", "/playlists", "/community", "/about"].each do |path|
# /c/LinusTechTips
Invidious::Routing.get "/c/:user#{path}", Invidious::Routes::Channels, :brand_redirect
# /user/linustechtips | Not always the same as /c/
Invidious::Routing.get "/user/:user#{path}", Invidious::Routes::Channels, :brand_redirect
# /attribution_link?a=anything&u=/channel/UCZYTClx2T1of7BRZ86-8fow
Invidious::Routing.get "/attribution_link#{path}", Invidious::Routes::Channels, :brand_redirect
# /profile?user=linustechtips
Invidious::Routing.get "/profile/#{path}", Invidious::Routes::Channels, :profile
end
Extract channel routes (#2227) * Extract primary channel routes from invidious.cr Also removes timedtext_video stub since all it does is redirect to the homepage. However, Invidious's 404 handler already does this. -- As the template for the channel about page doesn't exist yet, the behavior for the /channel/:ucid/about endpoint has been changed to be the same as what's currently present on Invidious (cherry picked from commit 8fad19d8057d7d22e3de27ebbc88a9978c1df27b) * Manually extract brand_redirect from 1b569bbc99207cae7c20aa285f42477ae361dd30 This commit manually extracts the brand_redirect function from the commit mentioned. However, the redirect to the `.../about` endpoint is removed due to the fact that it doesn't exist yet. This commit is also mainly just a bridge for the next few cherry picks from \#2215 * Update brand_redirect to use youtubei resolve_url (cherry picked from commit 53335fe7cfdfac392365b7cac447bc7cc6478134) * Add additional channel endpoints to brand_redirect (cherry picked from commit 8fc6f3add637dabb09b2034f4d82fc3d039ba15c) * Add separate handler for /profile endpoint * Add /channel/:ucid/home route * Document all channel brand_urls
2021-08-03 21:46:15 +00:00
Add support to make invidious API-only via flag (#2363)
2021-10-02 20:04:02 +00:00
Invidious::Routing.get "/watch", Invidious::Routes::Watch, :handle
Invidious::Routing.get "/watch/:id", Invidious::Routes::Watch, :redirect
Invidious::Routing.get "/shorts/:id", Invidious::Routes::Watch, :redirect
Invidious::Routing.get "/w/:id", Invidious::Routes::Watch, :redirect
Invidious::Routing.get "/v/:id", Invidious::Routes::Watch, :redirect
Invidious::Routing.get "/e/:id", Invidious::Routes::Watch, :redirect
Invidious::Routing.get "/redirect", Invidious::Routes::Misc, :cross_instance_redirect
Invidious::Routing.get "/embed/", Invidious::Routes::Embed, :redirect
Invidious::Routing.get "/embed/:id", Invidious::Routes::Embed, :show
Invidious::Routing.get "/create_playlist", Invidious::Routes::Playlists, :new
Invidious::Routing.post "/create_playlist", Invidious::Routes::Playlists, :create
Invidious::Routing.get "/subscribe_playlist", Invidious::Routes::Playlists, :subscribe
Invidious::Routing.get "/delete_playlist", Invidious::Routes::Playlists, :delete_page
Invidious::Routing.post "/delete_playlist", Invidious::Routes::Playlists, :delete
Invidious::Routing.get "/edit_playlist", Invidious::Routes::Playlists, :edit
Invidious::Routing.post "/edit_playlist", Invidious::Routes::Playlists, :update
Invidious::Routing.get "/add_playlist_items", Invidious::Routes::Playlists, :add_playlist_items_page
Invidious::Routing.post "/playlist_ajax", Invidious::Routes::Playlists, :playlist_ajax
Invidious::Routing.get "/playlist", Invidious::Routes::Playlists, :show
Invidious::Routing.get "/mix", Invidious::Routes::Playlists, :mix
Invidious::Routing.get "/opensearch.xml", Invidious::Routes::Search, :opensearch
Invidious::Routing.get "/results", Invidious::Routes::Search, :results
Invidious::Routing.get "/search", Invidious::Routes::Search, :search
Invidious::Routing.get "/login", Invidious::Routes::Login, :login_page
Invidious::Routing.post "/login", Invidious::Routes::Login, :login
Invidious::Routing.post "/signout", Invidious::Routes::Login, :signout
Invidious::Routing.get "/preferences", Invidious::Routes::PreferencesRoute, :show
Invidious::Routing.post "/preferences", Invidious::Routes::PreferencesRoute, :update
Invidious::Routing.get "/toggle_theme", Invidious::Routes::PreferencesRoute, :toggle_theme
# Feeds
Invidious::Routing.get "/view_all_playlists", Invidious::Routes::Feeds, :view_all_playlists_redirect
Invidious::Routing.get "/feed/playlists", Invidious::Routes::Feeds, :playlists
Invidious::Routing.get "/feed/popular", Invidious::Routes::Feeds, :popular
Invidious::Routing.get "/feed/trending", Invidious::Routes::Feeds, :trending
Invidious::Routing.get "/feed/subscriptions", Invidious::Routes::Feeds, :subscriptions
Invidious::Routing.get "/feed/history", Invidious::Routes::Feeds, :history
# RSS Feeds
Invidious::Routing.get "/feed/channel/:ucid", Invidious::Routes::Feeds, :rss_channel
Invidious::Routing.get "/feed/private", Invidious::Routes::Feeds, :rss_private
Invidious::Routing.get "/feed/playlist/:plid", Invidious::Routes::Feeds, :rss_playlist
Invidious::Routing.get "/feeds/videos.xml", Invidious::Routes::Feeds, :rss_videos
# Support push notifications via PubSubHubbub
Invidious::Routing.get "/feed/webhook/:token", Invidious::Routes::Feeds, :push_notifications_get
Invidious::Routing.post "/feed/webhook/:token", Invidious::Routes::Feeds, :push_notifications_post
{% end %}
Extract feed routes (#2269) * Extract feed routes from invidious.cr * Removes the deprecated route for /feed/top * Deprecate /view_all_playlist & use /feed/playlists * Move feed views into their own directory * Add haltf method to halt current route context * Change status_code + return blocks to use haltf * Set appropriate response headers for RSS routes
2021-08-30 14:58:24 +00:00
Extract image routes
2021-10-11 20:22:11 +00:00
Invidious::Routing.get "/ggpht/*", Invidious::Routes::Images, :ggpht
Invidious::Routing.options "/sb/:authority/:id/:storyboard/:index", Invidious::Routes::Images, :options_storyboard
Invidious::Routing.get "/sb/:authority/:id/:storyboard/:index", Invidious::Routes::Images, :get_storyboard
Invidious::Routing.get "/s_p/:id/:name", Invidious::Routes::Images, :s_p_image
Invidious::Routing.get "/yts/img/:name", Invidious::Routes::Images, :yts_image
Invidious::Routing.get "/vi/:id/:name", Invidious::Routes::Images, :thumbnails
Extract API routes (#2271) * Extract API routes from invidious.cr * Remove deprecated APIs - insights - top feed
2021-08-30 16:17:20 +00:00
# API routes (macro)
Extract API routes from invidious.cr (1/?)
2021-07-22 04:34:16 +00:00
define_v1_api_routes()
Extract API routes (#2271) * Extract API routes from invidious.cr * Remove deprecated APIs - insights - top feed
2021-08-30 16:17:20 +00:00
# Video playback (macros)
Extract API routes from invidious.cr (2/?) - Video playback endpoints - Search feed api - Video info api
2021-08-13 06:31:12 +00:00
define_api_manifest_routes()
define_video_playback_routes()
Extract API routes from invidious.cr (1/?)
2021-07-22 04:34:16 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
# Users
Add '/comments/:id'
2018-07-22 01:56:11 +00:00
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
post "/watch_ajax" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add support for translations
2018-12-20 21:32:09 +00:00
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
user = env.get? "user"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
sid = env.get? "sid"
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
referer = get_referer(env, "/feed/subscriptions")
redirect = env.params.query["redirect"]?
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
redirect ||= "true"
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
redirect = redirect == "true"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
if !user
Add authentication API
2019-04-18 21:23:50 +00:00
if redirect
next env.redirect referer
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "No such user")
Add authentication API
2019-04-18 21:23:50 +00:00
end
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
end
Add support for translations
2018-12-20 21:32:09 +00:00
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
user = user.as(User)
sid = sid.as(String)
Add authentication API
2019-04-18 21:23:50 +00:00
token = env.params.body["csrf_token"]?
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
id = env.params.query["id"]?
if !id
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
env.response.status_code = 400
next
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
end
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
begin
Add authentication API
2019-04-18 21:23:50 +00:00
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
rescue ex
if redirect
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, ex)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
end
end
if env.params.query["action_mark_watched"]?
action = "action_mark_watched"
elsif env.params.query["action_mark_unwatched"]?
action = "action_mark_unwatched"
else
next env.redirect referer
end
case action
when "action_mark_watched"
if !user.watched.includes? id
Move DB queries related to 'users' in a separate module (1/2)
2021-12-03 01:27:51 +00:00
Invidious::Database::Users.mark_watched(user, id)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
end
when "action_mark_unwatched"
Move DB queries related to 'users' in a separate module (1/2)
2021-12-03 01:27:51 +00:00
Invidious::Database::Users.mark_unwatched(user, id)
Fix warnings in latest version of Crystal
2020-04-09 17:18:09 +00:00
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "Unsupported action #{action}")
Add ability to mark videos as watched in subscription feed
2018-11-20 04:06:59 +00:00
end
if redirect
env.redirect referer
else
env.response.content_type = "application/json"
"{}"
end
end
Major cleanup
2018-08-05 04:07:38 +00:00
# /modify_notifications
# will "ding" all subscriptions.
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
# /modify_notifications?receive_all_updates=false&receive_no_updates=false
# will "unding" all subscriptions.
get "/modify_notifications" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add support for translations
2018-12-20 21:32:09 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
user = env.get? "user"
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
sid = env.get? "sid"
referer = get_referer(env, "/")
Add /embed/:id
2018-07-14 13:36:31 +00:00
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
redirect = env.params.query["redirect"]?
redirect ||= "false"
redirect = redirect == "true"
Add 'results' route redirect
2018-07-30 02:05:40 +00:00
Add authentication API
2019-04-18 21:23:50 +00:00
if !user
if redirect
next env.redirect referer
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "No such user")
Add authentication API
2019-04-18 21:23:50 +00:00
end
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
end
user = user.as(User)
if !user.password
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
channel_req = {} of String => String
Minor fixes
2018-02-27 00:59:02 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
channel_req["receive_all_updates"] = env.params.query["receive_all_updates"]? || "true"
channel_req["receive_no_updates"] = env.params.query["receive_no_updates"]? || ""
channel_req["receive_post_updates"] = env.params.query["receive_post_updates"]? || "true"
Speed up get_video
2018-01-07 17:42:24 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
channel_req.reject! { |k, v| v != "true" && v != "false" }
Add related videos and clean up video class
2018-01-07 02:39:24 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
headers = HTTP::Headers.new
headers["Cookie"] = env.request.headers["Cookie"]
Add search
2017-12-30 21:21:43 +00:00
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
html = YT_POOL.client &.get("/subscription_manager?disable_polymer=1", headers)
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
Upgrade to crystal 1.0.0
2021-05-24 13:45:50 +00:00
cookies = HTTP::Cookies.from_client_headers(headers)
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
html.cookies.each do |cookie|
if {"VISITOR_INFO1_LIVE", "YSC", "SIDCC"}.includes? cookie.name
if cookies[cookie.name]?
cookies[cookie.name] = cookie
else
cookies << cookie
end
end
end
headers = cookies.add_request_headers(headers)
Add support for polymer redesign
2020-06-15 22:33:23 +00:00
if match = html.body.match(/'XSRF_TOKEN': "(?<session_token>[^"]+)"/)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
session_token = match["session_token"]
else
next env.redirect referer
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
headers["content-type"] = "application/x-www-form-urlencoded"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
channel_req["session_token"] = session_token
Add referer redirect
2018-04-08 02:36:09 +00:00
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
subs = XML.parse_html(html.body)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
subs.xpath_nodes(%q(//a[@class="subscription-title yt-uix-sessionlink"]/@href)).each do |channel|
channel_id = channel.content.lstrip("/channel/").not_nil!
channel_req["channel_id"] = channel_id
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
YT_POOL.client &.post("/subscription_ajax?action_update_subscription_preferences=1", headers, form: channel_req)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
end
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
if redirect
env.redirect referer
else
env.response.content_type = "application/json"
"{}"
end
end
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
post "/subscription_ajax" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env, "/")
redirect = env.params.query["redirect"]?
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
redirect ||= "true"
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
redirect = redirect == "true"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
if !user
Add authentication API
2019-04-18 21:23:50 +00:00
if redirect
next env.redirect referer
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "No such user")
Add authentication API
2019-04-18 21:23:50 +00:00
end
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
end
user = user.as(User)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
sid = sid.as(String)
Add authentication API
2019-04-18 21:23:50 +00:00
token = env.params.body["csrf_token"]?
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
begin
Add authentication API
2019-04-18 21:23:50 +00:00
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
rescue ex
if redirect
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, ex)
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
end
end
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
Minor refactor
2019-06-08 00:56:41 +00:00
if env.params.query["action_create_subscription_to_channel"]?.try &.to_i?.try &.== 1
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
action = "action_create_subscription_to_channel"
Minor refactor
2019-06-08 00:56:41 +00:00
elsif env.params.query["action_remove_subscriptions"]?.try &.to_i?.try &.== 1
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
action = "action_remove_subscriptions"
else
next env.redirect referer
end
channel_id = env.params.query["c"]?
channel_id ||= ""
if !user.password
Support 'sort_by' in reddit /api/v1/comments
2019-04-14 23:08:00 +00:00
# Sync subscriptions with YouTube
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
subscribe_ajax(channel_id, action, env.request.headers)
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
end
case action
Minor refactor
2019-06-08 00:56:41 +00:00
when "action_create_subscription_to_channel"
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
if !user.subscriptions.includes? channel_id
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
get_channel(channel_id, PG_DB, false, false)
Move DB queries related to 'users' in a separate module (1/2)
2021-12-03 01:27:51 +00:00
Invidious::Database::Users.subscribe_channel(user, channel_id)
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
end
Minor refactor
2019-06-08 00:56:41 +00:00
when "action_remove_subscriptions"
Move DB queries related to 'users' in a separate module (1/2)
2021-12-03 01:27:51 +00:00
Invidious::Database::Users.unsubscribe_channel(user, channel_id)
Fix warnings in latest version of Crystal
2020-04-09 17:18:09 +00:00
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "Unsupported action #{action}")
Fix subscription_ajax for Google accounts
2019-04-07 17:59:12 +00:00
end
if redirect
env.redirect referer
else
env.response.content_type = "application/json"
"{}"
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add TFA
2018-04-29 14:40:33 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/subscription_manager" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add support for translations
2018-12-20 21:32:09 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
user = env.get? "user"
Store session_ids in separate table
2019-02-10 18:33:29 +00:00
sid = env.get? "sid"
Add authentication API
2019-04-18 21:23:50 +00:00
referer = get_referer(env)
Don't redirect to current page
2018-08-09 01:26:02 +00:00
Add authentication API
2019-04-18 21:23:50 +00:00
if !user
Don't redirect to current page
2018-08-09 01:26:02 +00:00
next env.redirect referer
General cleanup
2018-04-28 14:27:05 +00:00
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
user = user.as(User)
Move DB queries related to session tokens in a separate module
2021-12-02 22:57:13 +00:00
sid = sid.as(String)
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
if !user.password
# Refresh account
headers = HTTP::Headers.new
headers["Cookie"] = env.request.headers["Cookie"]
Add referer redirect
2018-04-08 02:36:09 +00:00
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user, sid = get_user(sid, headers, PG_DB)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
action_takeout = env.params.query["action_takeout"]?.try &.to_i?
action_takeout ||= 0
action_takeout = action_takeout == 1
Add separate user accounts
2018-07-18 19:26:02 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
format = env.params.query["format"]?
format ||= "rss"
Add separate user accounts
2018-07-18 19:26:02 +00:00
Move DB queries related to channels in a separate module
2021-12-02 18:16:41 +00:00
subscriptions = Invidious::Database::Channels.select(user.subscriptions)
Fix Style/VerboseBlock issues
2021-09-25 02:42:43 +00:00
subscriptions.sort_by!(&.author.downcase)
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
if action_takeout
if format == "json"
env.response.content_type = "application/json"
env.response.headers["content-disposition"] = "attachment"
Add playlist import/export
2020-07-25 17:30:28 +00:00
playlists = PG_DB.query_all("SELECT * FROM playlists WHERE author = $1 AND id LIKE 'IV%' ORDER BY created", user.email, as: InvidiousPlaylist)
next JSON.build do |json|
json.object do
json.field "subscriptions", user.subscriptions
json.field "watch_history", user.watched
json.field "preferences", user.preferences
json.field "playlists" do
json.array do
playlists.each do |playlist|
json.object do
json.field "title", playlist.title
json.field "description", html_to_content(playlist.description_html)
json.field "privacy", playlist.privacy.to_s
json.field "videos" do
json.array do
Fix playlist export for playlists with more than 100 videos
2020-07-28 21:21:39 +00:00
PG_DB.query_all("SELECT id FROM playlist_videos WHERE plid = $1 ORDER BY array_position($2, index) LIMIT 500", playlist.id, playlist.index, as: String).each do |video_id|
json.string video_id
Add playlist import/export
2020-07-25 17:30:28 +00:00
end
end
end
end
end
end
end
end
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
else
env.response.content_type = "application/xml"
env.response.headers["content-disposition"] = "attachment"
export = XML.build do |xml|
xml.element("opml", version: "1.1") do
xml.element("body") do
if format == "newpipe"
title = "YouTube Subscriptions"
else
title = "Invidious Subscriptions"
end
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
xml.element("outline", text: title, title: title) do
subscriptions.each do |channel|
if format == "newpipe"
Fix Style/VariableNames issues
2021-09-25 02:15:23 +00:00
xml_url = "https://www.youtube.com/feeds/videos.xml?channel_id=#{channel.id}"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
else
Fix Style/VariableNames issues
2021-09-25 02:15:23 +00:00
xml_url = "#{HOST_URL}/feed/channel/#{channel.id}"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
xml.element("outline", text: channel.author, title: channel.author,
Fix Style/VariableNames issues
2021-09-25 02:15:23 +00:00
"type": "rss", xmlUrl: xml_url)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
end
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
end
Add login page
2018-03-16 16:40:29 +00:00
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
next export.gsub(%(<?xml version="1.0"?>\n), "")
end
end
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
templated "subscription_manager"
end
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/data_control" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add support for translations
2018-12-20 21:32:09 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
user = env.get? "user"
Don't redirect to current page
2018-08-09 01:26:02 +00:00
referer = get_referer(env)
Add login page
2018-03-16 16:40:29 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
templated "data_control"
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
post "/data_control" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add support for translations
2018-12-20 21:32:09 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
user = env.get? "user"
Don't redirect to current page
2018-08-09 01:26:02 +00:00
referer = get_referer(env)
Add login page
2018-03-16 16:40:29 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
if user
user = user.as(User)
Add TFA
2018-04-29 14:40:33 +00:00
Fix warnings with latest version of Crystal
2020-07-26 14:58:50 +00:00
# TODO: Find a way to prevent browser timeout
Try to prevent timeout in /data_control
2019-04-25 01:18:35 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
HTTP::FormData.parse(env.request) do |part|
body = part.body.gets_to_end
Add playlist import/export
2020-07-25 17:30:28 +00:00
next if body.empty?
Add support for incorrect password on /login
2018-04-17 22:54:33 +00:00
Fix warnings in latest version of Crystal
2020-04-09 17:18:09 +00:00
# TODO: Unify into single import based on content-type
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
case part.name
when "import_invidious"
body = JSON.parse(body)
Add fix to switch to supported authentication schemes
2018-07-26 15:20:15 +00:00
Clean up data import/export
2018-11-09 23:25:24 +00:00
if body["subscriptions"]?
Fix Style/VerboseBlock issues
2021-09-25 02:42:43 +00:00
user.subscriptions += body["subscriptions"].as_a.map(&.as_s)
Clean up data import/export
2018-11-09 23:25:24 +00:00
user.subscriptions.uniq!
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user.subscriptions = get_batch_channels(user.subscriptions, PG_DB, false, false)
Clean up data import/export
2018-11-09 23:25:24 +00:00
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = $1 WHERE email = $2", user.subscriptions, user.email)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add fix to switch to supported authentication schemes
2018-07-26 15:20:15 +00:00
Speed up importing watch history
2018-11-08 22:43:28 +00:00
if body["watch_history"]?
Fix Style/VerboseBlock issues
2021-09-25 02:42:43 +00:00
user.watched += body["watch_history"].as_a.map(&.as_s)
Clean up data import/export
2018-11-09 23:25:24 +00:00
user.watched.uniq!
PG_DB.exec("UPDATE users SET watched = $1 WHERE email = $2", user.watched, user.email)
Add fix to switch to supported authentication schemes
2018-07-26 15:20:15 +00:00
end
Add TFA
2018-04-29 14:40:33 +00:00
Add support for partial data restore
2018-11-08 22:35:26 +00:00
if body["preferences"]?
Fix warnings with latest version of Crystal
2020-07-26 14:58:50 +00:00
user.preferences = Preferences.from_json(body["preferences"].to_json)
Clean up data import/export
2018-11-09 23:25:24 +00:00
PG_DB.exec("UPDATE users SET preferences = $1 WHERE email = $2", user.preferences.to_json, user.email)
Add support for partial data restore
2018-11-08 22:35:26 +00:00
end
Add playlist import/export
2020-07-25 17:30:28 +00:00
if playlists = body["playlists"]?.try &.as_a?
playlists.each do |item|
title = item["title"]?.try &.as_s?.try &.delete("<>")
description = item["description"]?.try &.as_s?.try &.delete("\r")
privacy = item["privacy"]?.try &.as_s?.try { |privacy| PlaylistPrivacy.parse? privacy }
next if !title
next if !description
next if !privacy
playlist = create_playlist(PG_DB, title, privacy, user)
Move DB queries related to playlists in a separate module (2/3)
2021-11-30 02:11:21 +00:00
Invidious::Database::Playlists.update_description(playlist.id, description)
Add playlist import/export
2020-07-25 17:30:28 +00:00
Fix playlist export for playlists with more than 100 videos
2020-07-28 21:21:39 +00:00
videos = item["videos"]?.try &.as_a?.try &.each_with_index do |video_id, idx|
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
raise InfoException.new("Playlist cannot have more than 500 videos") if idx > 500
Fix playlist export for playlists with more than 100 videos
2020-07-28 21:21:39 +00:00
Add playlist import/export
2020-07-25 17:30:28 +00:00
video_id = video_id.try &.as_s?
next if !video_id
begin
video = get_video(video_id, PG_DB)
rescue ex
next
end
Fix warnings with latest version of Crystal
2020-07-26 14:58:50 +00:00
playlist_video = PlaylistVideo.new({
title: video.title,
id: video.id,
author: video.author,
ucid: video.ucid,
Add playlist import/export
2020-07-25 17:30:28 +00:00
length_seconds: video.length_seconds,
Fix warnings with latest version of Crystal
2020-07-26 14:58:50 +00:00
published: video.published,
plid: playlist.id,
live_now: video.live_now,
index: Random::Secure.rand(0_i64..Int64::MAX),
})
Add playlist import/export
2020-07-25 17:30:28 +00:00
Move DB queries related to playlists in a separate module (1/3)
2021-11-30 01:24:24 +00:00
Invidious::Database::PlaylistVideos.insert(playlist_video)
Invidious::Database::Playlists.update_video_added(playlist.id, playlist_video.index)
Add playlist import/export
2020-07-25 17:30:28 +00:00
end
end
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
when "import_youtube"
format invidious.cr using crystal cli
2021-02-27 20:59:09 +00:00
if body[0..4] == "<opml"
integrate legacy yt xml subscription import with new json sub import
2021-02-27 17:58:55 +00:00
subscriptions = XML.parse(body)
user.subscriptions += subscriptions.xpath_nodes(%q(//outline[@type="rss"])).map do |channel|
channel["xmlUrl"].match(/UC[a-zA-Z0-9_-]{22}/).not_nil![0]
end
else
subscriptions = JSON.parse(body)
user.subscriptions += subscriptions.as_a.compact_map do |entry|
entry["snippet"]["resourceId"]["channelId"].as_s
fixed line indent typo
2021-02-27 18:12:01 +00:00
end
Clean up data import/export
2018-11-09 23:25:24 +00:00
end
user.subscriptions.uniq!
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user.subscriptions = get_batch_channels(user.subscriptions, PG_DB, false, false)
Prevent duplicate subscriptions when importing user data
2018-10-06 23:19:47 +00:00
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = $1 WHERE email = $2", user.subscriptions, user.email)
Clean up data import/export
2018-11-09 23:25:24 +00:00
when "import_freetube"
user.subscriptions += body.scan(/"channelId":"(?<channel_id>[a-zA-Z0-9_-]{24})"/).map do |md|
md["channel_id"]
end
user.subscriptions.uniq!
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user.subscriptions = get_batch_channels(user.subscriptions, PG_DB, false, false)
Clean up data import/export
2018-11-09 23:25:24 +00:00
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = $1 WHERE email = $2", user.subscriptions, user.email)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
when "import_newpipe_subscriptions"
body = JSON.parse(body)
Add support for 'user' URLs in NewPipe import
2019-04-22 20:39:57 +00:00
user.subscriptions += body["subscriptions"].as_a.compact_map do |channel|
if match = channel["url"].as_s.match(/\/channel\/(?<channel>UC[a-zA-Z0-9_-]{22})/)
next match["channel"]
elsif match = channel["url"].as_s.match(/\/user\/(?<user>.+)/)
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
response = YT_POOL.client &.get("/user/#{match["user"]}?disable_polymer=1&hl=en&gl=US")
Fix channel redirect
2020-01-14 13:21:17 +00:00
html = XML.parse_html(response.body)
ucid = html.xpath_node(%q(//link[@rel="canonical"])).try &.["href"].split("/")[-1]
next ucid if ucid
Add support for 'user' URLs in NewPipe import
2019-04-22 20:39:57 +00:00
end
nil
Clean up data import/export
2018-11-09 23:25:24 +00:00
end
user.subscriptions.uniq!
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user.subscriptions = get_batch_channels(user.subscriptions, PG_DB, false, false)
Clean up data import/export
2018-11-09 23:25:24 +00:00
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = $1 WHERE email = $2", user.subscriptions, user.email)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
when "import_newpipe"
Support Crystal 0.35.0
2020-06-15 22:57:20 +00:00
Compress::Zip::Reader.open(IO::Memory.new(body)) do |file|
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
file.each_entry do |entry|
if entry.filename == "newpipe.db"
Properly parse NewPipe imports
2018-11-21 23:12:13 +00:00
tempfile = File.tempfile(".db")
File.write(tempfile.path, entry.io.gets_to_end)
db = DB.open("sqlite3://" + tempfile.path)
Add TFA
2018-04-29 14:40:33 +00:00
Fix Style/VerboseBlock issues
2021-09-25 02:42:43 +00:00
user.watched += db.query_all("SELECT url FROM streams", as: String).map(&.lchop("https://www.youtube.com/watch?v="))
Clean up data import/export
2018-11-09 23:25:24 +00:00
user.watched.uniq!
Add separate user accounts
2018-07-18 19:26:02 +00:00
Clean up data import/export
2018-11-09 23:25:24 +00:00
PG_DB.exec("UPDATE users SET watched = $1 WHERE email = $2", user.watched, user.email)
Prevent duplicate subscriptions when importing user data
2018-10-06 23:19:47 +00:00
Fix Style/VerboseBlock issues
2021-09-25 02:42:43 +00:00
user.subscriptions += db.query_all("SELECT url FROM subscriptions", as: String).map(&.lchop("https://www.youtube.com/channel/"))
Clean up data import/export
2018-11-09 23:25:24 +00:00
user.subscriptions.uniq!
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
user.subscriptions = get_batch_channels(user.subscriptions, PG_DB, false, false)
Clean up data import/export
2018-11-09 23:25:24 +00:00
Add 'needs_update' column for scheduling feed refresh
2019-06-01 15:19:18 +00:00
PG_DB.exec("UPDATE users SET feed_needs_update = true, subscriptions = $1 WHERE email = $2", user.subscriptions, user.email)
Properly parse NewPipe imports
2018-11-21 23:12:13 +00:00
db.close
tempfile.delete
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
end
Add support for Google Authenticator codes
2018-07-08 13:57:06 +00:00
end
Fix warnings in latest version of Crystal
2020-04-09 17:18:09 +00:00
else nil # Ignore
Add separate user accounts
2018-07-18 19:26:02 +00:00
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
env.redirect referer
end
Add separate user accounts
2018-07-18 19:26:02 +00:00
Add option to change passwords
2019-04-22 15:18:17 +00:00
get "/change_password" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add option to change passwords
2019-04-22 15:18:17 +00:00
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env)
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
Add option to change passwords
2019-04-22 15:18:17 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
csrf_token = generate_response(sid, {":change_password"}, HMAC_KEY, PG_DB)
templated "change_password"
Add option to change passwords
2019-04-22 15:18:17 +00:00
end
post "/change_password" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add option to change passwords
2019-04-22 15:18:17 +00:00
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env)
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
token = env.params.body["csrf_token"]?
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
# We don't store passwords for Google accounts
if !user.password
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, "Cannot change password for Google accounts")
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
begin
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
password = env.params.body["password"]?
if !password
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(401, "Password is a required field")
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
new_passwords = env.params.body.select { |k, v| k.match(/^new_password\[\d+\]$/) }.map { |k, v| v }
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if new_passwords.size <= 1 || new_passwords.uniq.size != 1
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, "New passwords must match")
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
new_password = new_passwords.uniq[0]
if new_password.empty?
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(401, "Password cannot be empty")
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if new_password.bytesize > 55
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, "Password cannot be longer than 55 characters")
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option to change passwords
2019-04-22 15:18:17 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if !Crypto::Bcrypt::Password.new(user.password.not_nil!).verify(password.byte_slice(0, 55))
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(401, "Incorrect password")
Add option to change passwords
2019-04-22 15:18:17 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
new_password = Crypto::Bcrypt::Password.create(new_password, cost: 10)
PG_DB.exec("UPDATE users SET password = $1 WHERE email = $2", new_password.to_s, user.email)
Add option to change passwords
2019-04-22 15:18:17 +00:00
env.redirect referer
end
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
get "/delete_account" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add support for translations
2018-12-20 21:32:09 +00:00
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
user = env.get? "user"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
sid = env.get? "sid"
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
referer = get_referer(env)
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
csrf_token = generate_response(sid, {":delete_account"}, HMAC_KEY, PG_DB)
templated "delete_account"
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
end
post "/delete_account" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add support for translations
2018-12-20 21:32:09 +00:00
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
user = env.get? "user"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
sid = env.get? "sid"
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
referer = get_referer(env)
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
end
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
token = env.params.body["csrf_token"]?
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
begin
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
view_name = "subscriptions_#{sha256(user.email)}"
Move DB queries related to 'users' in a separate module (1/2)
2021-12-03 01:27:51 +00:00
Invidious::Database::Users.delete(user)
Move DB queries related to session tokens in a separate module
2021-12-02 22:57:13 +00:00
Invidious::Database::SessionIDs.delete(email: user.email)
Minor refactor
2019-07-13 02:00:50 +00:00
PG_DB.exec("DROP MATERIALIZED VIEW #{view_name}")
env.request.cookies.each do |cookie|
cookie.expires = Time.utc(1990, 1, 1)
env.response.cookies << cookie
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
end
env.redirect referer
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
get "/clear_watch_history" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add support for translations
2018-12-20 21:32:09 +00:00
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
user = env.get? "user"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
sid = env.get? "sid"
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
referer = get_referer(env)
Don't redirect to current page
2018-08-09 01:26:02 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
csrf_token = generate_response(sid, {":clear_watch_history"}, HMAC_KEY, PG_DB)
templated "clear_watch_history"
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
end
post "/clear_watch_history" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add support for translations
2018-12-20 21:32:09 +00:00
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
user = env.get? "user"
Refactor CSRF tokens (using format in #473)
2019-04-16 04:23:40 +00:00
sid = env.get? "sid"
Don't redirect to current page
2018-08-09 01:26:02 +00:00
referer = get_referer(env)
Add login page
2018-03-16 16:40:29 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
end
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
token = env.params.body["csrf_token"]?
Add option for user to delete their account
2018-11-08 06:12:14 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
begin
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
end
Move DB queries related to 'users' in a separate module (1/2)
2021-12-03 01:27:51 +00:00
Invidious::Database::Users.clear_watch_history(user)
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
env.redirect referer
end
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
get "/authorize_token" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env)
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
end
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
user = user.as(User)
sid = sid.as(String)
csrf_token = generate_response(sid, {":authorize_token"}, HMAC_KEY, PG_DB)
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
scopes = env.params.query["scopes"]?.try &.split(",")
scopes ||= [] of String
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
callback_url = env.params.query["callback_url"]?
if callback_url
callback_url = URI.parse(callback_url)
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
expire = env.params.query["expire"]?.try &.to_i?
templated "authorize_token"
Add GET '/authorize_token'
2019-05-15 17:26:29 +00:00
end
Add authentication API
2019-04-18 21:23:50 +00:00
post "/authorize_token" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add authentication API
2019-04-18 21:23:50 +00:00
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env)
Minor refactor
2019-07-13 02:00:50 +00:00
if !user
next env.redirect referer
end
Add authentication API
2019-04-18 21:23:50 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
user = env.get("user").as(User)
sid = sid.as(String)
token = env.params.body["csrf_token"]?
Add authentication API
2019-04-18 21:23:50 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
begin
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
rescue ex
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Minor refactor
2019-07-13 02:00:50 +00:00
end
Add authentication API
2019-04-18 21:23:50 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
scopes = env.params.body.select { |k, v| k.match(/^scopes\[\d+\]$/) }.map { |k, v| v }
callback_url = env.params.body["callbackUrl"]?
expire = env.params.body["expire"]?.try &.to_i?
Add authentication API
2019-04-18 21:23:50 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
access_token = generate_token(user.email, scopes, expire, HMAC_KEY, PG_DB)
Add authentication API
2019-04-18 21:23:50 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if callback_url
Update to Crystal 0.31.0, resolve compiler deprecation warnings, update dependencies (#764) * shard: update to crystal 0.31.0 Additionally, no longer use the Crystal "markdown" library which has been removed from the Crystal stdlib in version 0.31.0. See https://github.com/crystal-lang/crystal/pull/8115. Also fix some deprecation warnings using the following commands: find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.escape/URI\.encode_www_form/g' "{}" \; find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.unescape/URI\.decode_www_form/g' "{}" \; sed -i 's/while \%pull\.kind \!\= \:end_object/until \%pull\.kind\.end_object\?/g' src/invidious/helpers/patch_mapping.cr
2019-09-24 17:31:33 +00:00
access_token = URI.encode_www_form(access_token)
Minor refactor
2019-07-13 02:00:50 +00:00
url = URI.parse(callback_url)
Add authentication API
2019-04-18 21:23:50 +00:00
Minor refactor
2019-07-13 02:00:50 +00:00
if url.query
query = HTTP::Params.parse(url.query.not_nil!)
Add authentication API
2019-04-18 21:23:50 +00:00
else
Minor refactor
2019-07-13 02:00:50 +00:00
query = HTTP::Params.new
Add authentication API
2019-04-18 21:23:50 +00:00
end
Minor refactor
2019-07-13 02:00:50 +00:00
query["token"] = access_token
url.query = query.to_s
env.redirect url.to_s
else
csrf_token = ""
env.set "access_token", access_token
templated "authorize_token"
Add authentication API
2019-04-18 21:23:50 +00:00
end
end
get "/token_manager" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add authentication API
2019-04-18 21:23:50 +00:00
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env, "/subscription_manager")
if !user
next env.redirect referer
end
user = user.as(User)
Move DB queries related to session tokens in a separate module
2021-12-02 22:57:13 +00:00
tokens = Invidious::Database::SessionIDs.select_all(user.email)
Add authentication API
2019-04-18 21:23:50 +00:00
templated "token_manager"
end
post "/token_ajax" do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add authentication API
2019-04-18 21:23:50 +00:00
user = env.get? "user"
sid = env.get? "sid"
referer = get_referer(env)
redirect = env.params.query["redirect"]?
redirect ||= "true"
redirect = redirect == "true"
if !user
if redirect
next env.redirect referer
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(403, "No such user")
Add authentication API
2019-04-18 21:23:50 +00:00
end
end
user = user.as(User)
sid = sid.as(String)
token = env.params.body["csrf_token"]?
begin
validate_request(token, sid, env.request, HMAC_KEY, PG_DB, locale)
rescue ex
if redirect
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_template(400, ex)
Add authentication API
2019-04-18 21:23:50 +00:00
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, ex)
Add authentication API
2019-04-18 21:23:50 +00:00
end
end
if env.params.query["action_revoke_token"]?
action = "action_revoke_token"
else
next env.redirect referer
end
session = env.params.query["session"]?
session ||= ""
case action
when .starts_with? "action_revoke_token"
Move DB queries related to session tokens in a separate module
2021-12-02 22:57:13 +00:00
Invidious::Database::SessionIDs.delete(sid: session, email: user.email)
Fix warnings in latest version of Crystal
2020-04-09 17:18:09 +00:00
else
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
next error_json(400, "Unsupported action #{action}")
Add authentication API
2019-04-18 21:23:50 +00:00
end
if redirect
env.redirect referer
else
env.response.content_type = "application/json"
"{}"
end
end
Split helpers.cr into multiple files
2018-08-04 20:30:44 +00:00
# Channels
Add support for '/c/:user/live'
2019-04-28 16:47:16 +00:00
{"/channel/:ucid/live", "/user/:user/live", "/c/:user/live"}.each do |route|
get route do |env|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add support for '/c/:user/live'
2019-04-28 16:47:16 +00:00
# Appears to be a bug in routing, having several routes configured
# as `/a/:a`, `/b/:a`, `/c/:a` results in 404
value = env.request.resource.split("/")[2]
body = ""
{"channel", "user", "c"}.each do |type|
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
response = YT_POOL.client &.get("/#{type}/#{value}/live?disable_polymer=1")
Add support for '/c/:user/live'
2019-04-28 16:47:16 +00:00
if response.status_code == 200
body = response.body
end
end
video_id = body.match(/'VIDEO_ID': "(?<id>[a-zA-Z0-9_-]{11})"/).try &.["id"]?
if video_id
params = [] of String
env.params.query.each do |k, v|
params << "#{k}=#{v}"
end
params = params.join("&")
url = "/watch?v=#{video_id}"
if !params.empty?
url += "&#{params}"
end
env.redirect url
else
env.redirect "/channel/#{value}"
end
end
end
Add '/api/v1/auth/feeds'
2019-06-07 17:39:12 +00:00
# Authenticated endpoints
Add subscribe and unsubscribe button
2018-03-31 14:51:14 +00:00
Extract API routes from invidious.cr (3/3) - Auth (excluding notifications*) APIs - Mixes *Notifications currently require the "connection_channel" channel for talking with the notifications job. Unfortunately, we cannot access that within the route modules yet.
2021-08-14 07:08:46 +00:00
# The notification APIs can't be extracted yet
# due to the requirement of the `connection_channel`
# used by the `NotificationJob`
Add descriptionHtml to playlists
2018-09-05 00:27:10 +00:00
Add '/api/v1/notifications'
2019-04-10 22:58:42 +00:00
get "/api/v1/auth/notifications" do |env|
Refactor connect_listen for notifications
2019-06-02 12:41:53 +00:00
env.response.content_type = "text/event-stream"
Add watch history
2018-07-29 01:40:59 +00:00
Add '/api/v1/notifications'
2019-04-10 22:58:42 +00:00
topics = env.params.query["topics"]?.try &.split(",").uniq.first(1000)
topics ||= [] of String
Add related_channels to /api/v1/channels
2018-11-28 04:07:45 +00:00
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
create_notification_stream(env, topics, connection_channel)
Format files and trim trailing whitespace
2019-03-23 19:05:13 +00:00
end
Organize routes
2018-07-16 13:18:59 +00:00
Add POST /api/v1/auth/subscriptions
2019-05-21 14:01:17 +00:00
post "/api/v1/auth/notifications" do |env|
Refactor connect_listen for notifications
2019-06-02 12:41:53 +00:00
env.response.content_type = "text/event-stream"
Add support for translations
2018-12-20 21:32:09 +00:00
Add POST /api/v1/auth/subscriptions
2019-05-21 14:01:17 +00:00
topics = env.params.body["topics"]?.try &.split(",").uniq.first(1000)
topics ||= [] of String
Add '/api/v1/notifications'
2019-04-10 22:58:42 +00:00
Make HOST_URL constant
2020-06-15 22:10:30 +00:00
create_notification_stream(env, topics, connection_channel)
Add '/api/v1/notifications'
2019-04-10 22:58:42 +00:00
end
Add support for image captcha in Google login
2019-10-27 04:19:05 +00:00
get "/Captcha" do |env|
Cleanup request headers
2020-05-08 14:00:53 +00:00
headers = HTTP::Headers{":authority" => "accounts.google.com"}
response = YT_POOL.client &.get(env.request.resource, headers)
Add support for image captcha in Google login
2019-10-27 04:19:05 +00:00
env.response.headers["Content-Type"] = response.headers["Content-Type"]
response.body
end
Refactor search extractor
2019-08-21 23:23:20 +00:00
# Undocumented, creates anonymous playlist with specified 'video_ids', max 50 videos
Add '/watch_videos' endpoint
2019-05-03 14:11:27 +00:00
get "/watch_videos" do |env|
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
response = YT_POOL.client &.get(env.request.resource)
Add '/watch_videos' endpoint
2019-05-03 14:11:27 +00:00
if url = response.headers["Location"]?
Update code for Crystal 0.36.0 Rename `HTTPClient@socket` to `HTTPClient@io`, see https://github.com/crystal-lang/crystal/pull/9543. Rename `URI#full_path` to `URI#request_target`, see https://github.com/crystal-lang/crystal/pull/10099.
2021-01-31 18:52:32 +00:00
url = URI.parse(url).request_target
Add '/watch_videos' endpoint
2019-05-03 14:11:27 +00:00
next env.redirect url
end
env.response.status_code = response.status_code
end
Use error template for 404 and 500
2018-02-10 15:15:23 +00:00
error 404 do |env|
Fix redirect for shortened video urls
2019-03-27 10:28:53 +00:00
if md = env.request.path.match(/^\/(?<id>([a-zA-Z0-9_-]{11})|(\w+))$/)
Fix typo in 404 handler
2019-04-17 19:46:00 +00:00
item = md["id"]
Add extra handling for shortened video URLs
2018-10-07 03:19:36 +00:00
Fix typo in 404 handler
2019-04-17 19:46:00 +00:00
# Check if item is branding URL e.g. https://youtube.com/gaming
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
response = YT_POOL.client &.get("/#{item}")
Fix redirect for shortened video urls
2019-03-27 10:28:53 +00:00
if response.status_code == 301
Update code for Crystal 0.36.0 Rename `HTTPClient@socket` to `HTTPClient@io`, see https://github.com/crystal-lang/crystal/pull/9543. Rename `URI#full_path` to `URI#request_target`, see https://github.com/crystal-lang/crystal/pull/10099.
2021-01-31 18:52:32 +00:00
response = YT_POOL.client &.get(URI.parse(response.headers["Location"]).request_target)
Fix redirect for shortened video urls
2019-03-27 10:28:53 +00:00
end
Fix 404 handling for endpoints matching short URLs
2019-06-07 17:42:07 +00:00
if response.body.empty?
env.response.headers["Location"] = "/"
halt env, status_code: 302
end
Fix redirect for shortened video urls
2019-03-27 10:28:53 +00:00
html = XML.parse_html(response.body)
Fix channel redirect
2020-01-14 13:21:17 +00:00
ucid = html.xpath_node(%q(//link[@rel="canonical"])).try &.["href"].split("/")[-1]
Fix redirect for shortened video urls
2019-03-27 10:28:53 +00:00
if ucid
Fix channel redirect
2020-01-14 13:21:17 +00:00
env.response.headers["Location"] = "/channel/#{ucid}"
Fix redirect for shortened video urls
2019-03-27 10:28:53 +00:00
halt env, status_code: 302
end
Add extra handling for shortened video URLs
2018-10-07 03:19:36 +00:00
params = [] of String
env.params.query.each do |k, v|
params << "#{k}=#{v}"
end
params = params.join("&")
Fix typo in 404 handler
2019-04-17 19:46:00 +00:00
url = "/watch?v=#{item}"
Add extra handling for shortened video URLs
2018-10-07 03:19:36 +00:00
if !params.empty?
url += "&#{params}"
end
Fix typo in 404 handler
2019-04-17 19:46:00 +00:00
# Check if item is video ID
Add HTTPClient pool
2019-10-25 16:58:16 +00:00
if item.match(/^[a-zA-Z0-9_-]{11}$/) && YT_POOL.client &.head("/watch?v=#{item}").status_code != 404
Add support for custom channel URLs
2019-02-21 21:07:22 +00:00
env.response.headers["Location"] = url
halt env, status_code: 302
end
end
Improve error message for 500 and add redirect for 404
2019-01-12 19:18:08 +00:00
env.response.headers["Location"] = "/"
halt env, status_code: 302
Add search
2017-12-30 21:21:43 +00:00
end
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
error 500 do |env, ex|
i18n: pass only the ISO code string to 'translate()' Don't use the whole Hash everywhere. Also fall back nicely to english string if no translation exists.
2021-11-08 22:52:55 +00:00
locale = env.get("preferences").as(Preferences).locale
Add backtraces to errors (#1498) Error handling has been reworked to always go through the new `error_template`, `error_json` and `error_atom` macros. They all accept a status code followed by a string message or an exception object. `error_json` accepts a hash with additional fields as third argument. If the second argument is an exception a backtrace will be printed, if it is a string only the string is printed. Since up till now only the exception message was printed a new `InfoException` class was added for situations where no backtrace is intended but a string cannot be used. `error_template` with a string message automatically localizes the message. Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497 `error_json` with a string message does not localize the message. This is the same as previous behavior. If translations are desired for `error_json` they can be added easily but those error messages have not been collected yet. Uncaught exceptions previously only printed a generic message ("Looks like you've found a bug in Invidious. [...]"). They still print that message but now also include a backtrace.
2020-11-30 09:59:21 +00:00
error_template(500, ex)
Add search
2017-12-30 21:21:43 +00:00
end
Fix Style/VariableNames issues
2021-09-25 02:15:23 +00:00
static_headers do |response|
Add simple form of cache busting
2019-05-08 13:58:10 +00:00
response.headers.add("Cache-Control", "max-age=2629800")
Add max-age for static assets
2018-03-09 17:28:57 +00:00
end
Initial commit
2017-11-23 07:48:55 +00:00
public_folder "assets"
Add filtered handler for videoplayback
2018-04-16 03:56:58 +00:00
Disable Kemal header
2018-07-30 23:42:45 +00:00
Kemal.config.powered_by_header = false
Add filtered handler for videoplayback
2018-04-16 03:56:58 +00:00
add_handler FilteredCompressHandler.new
Add CORS to API endpoints
2019-02-03 04:48:47 +00:00
add_handler APIHandler.new
Add authentication API
2019-04-18 21:23:50 +00:00
add_handler AuthHandler.new
Move 'pretty=1' into middleware
2019-03-23 15:24:30 +00:00
add_handler DenyFrame.new
Add authentication API
2019-04-18 21:23:50 +00:00
add_context_storage_type(Array(String))
Allow user to save preferences without creating an account
2019-02-24 15:49:48 +00:00
add_context_storage_type(Preferences)
Add authentication API
2019-04-18 21:23:50 +00:00
add_context_storage_type(User)
Initial commit
2017-11-23 07:48:55 +00:00
Make logger a constant Instead of passing around `logger` there is now the global `LOGGER`.
2021-01-04 15:51:06 +00:00
Kemal.config.logger = LOGGER
Add config options for host binding and port
2019-09-23 17:05:29 +00:00
Kemal.config.host_binding = Kemal.config.host_binding != "0.0.0.0" ? Kemal.config.host_binding : CONFIG.host_binding
Kemal.config.port = Kemal.config.port != 3000 ? Kemal.config.port : CONFIG.port
Bump dependencies (#2378) * Upgrade to Kemal v1.1.0 * Bump postgres driver
2021-09-10 07:42:15 +00:00
Kemal.config.app_name = "Invidious"
Use kemal in production mode (#2455)
2021-10-11 12:42:22 +00:00
# Use in kemal's production mode.
# Users can also set the KEMAL_ENV environmental variable for this to be set automatically.
{% if flag?(:release) || flag?(:production) %}
Kemal.config.env = "production" if !ENV.has_key?("KEMAL_ENV")
{% end %}
Initial commit
2017-11-23 07:48:55 +00:00
Kemal.run
Reference in New Issue Copy Permalink