7a8620a570
Add CLI arguments to config file
...
The log level can now be set with `log_level` (accepts ints and strings).
The log file can now be set with `output` (also accepts `STDOUT`).
2021-01-05 20:35:59 +01:00
2c24bf3222
Merge pull request #1389 from vhuynh3000/decrypt_on_demand
...
add config to decrypt on demand instead of polling
2021-01-05 19:32:22 +01:00
8b56a038b7
Set content type for HTML error helpers
...
This fixes `Unexpected char '<' at 1:1` errors caused by content type mismatch.
2021-01-04 05:24:08 +01:00
3a2bd4e928
add config to decrypt on demand instead of polling
2021-01-03 20:50:52 +01:00
82c8f3b556
Merge pull request #1600 from jksladjflkjsadflkjsadf/closeclients
...
Close http clients after use
2020-12-30 01:14:31 +00:00
b19524d56a
Merge pull request #1609 from saltycrys/add-popular-enabled-option
...
Add `popular-enabled` option
2020-12-28 11:30:55 +01:00
e0d25ff887
Close http clients after using
...
The crystal http client maintains a keepalive connection to the other
server which stays alive for some time. This should be closed if the
client instance is not used again to avoid hogging resources
2020-12-27 19:40:58 -05:00
198dfffaeb
Add `popular-enabled` option
...
This is similar to the removed `top-enabled` option but for the Popular feed.
The instance needs to be restarted if the feed was enabled.
Editing admin options on the preferences page is also fixed.
The handling of the feed pages now only happens in a single place.
Instead of redirecting:
- The Top feed now displays a message that it was removed from Invidious.
- The Popular feed now displays a message that it was disabled if it was.
2020-12-27 06:12:43 +01:00
eeeecf9763
Merge pull request #1572 from saltycrys/add-dash-quality-preference
...
Add DASH quality preference
2020-12-23 20:36:13 +01:00
b39f01dcdf
Improve logging
...
Everything that gets logged now has a log level associated with it.
The log level can be set with the new `-l` or `--log-level` arguments.
The defaul log level is `debug` for now. There aren't many things that get
logged but if the logs get spammed in the future it can be set down to `info`.
2020-12-21 16:32:34 +01:00
eea7ca9b72
Add DASH quality preference
...
The options are `auto` (the current and default behavior), `best` and `worst`.
The UI is only updated once playback starts.
2020-12-19 18:37:30 +01:00
2de206cb81
Fix comments
...
The YouTube headers are now always added for requests to YouTube.
Previously they were only added for requests going through QUIC.
The session token is now JSON decoded to unescape escaped Unicode characters.
The comment continuation protobuf has been updated and the request now goes
through the YouTube `pbj` JSON API.
2020-12-06 04:11:41 +01:00
b409cdece5
Improve error message 2
...
Electric Boogaloo
The long backtrace has been moved into a `<details>` HTML element, as suggested
by @B0pol. To make the error still visible it has been added to the top under
`Title:`. This also encourages informative issue titles.
2020-12-01 13:41:19 +01:00
df3f9a2ae8
fix: channel info parsing
2020-11-30 19:35:45 +01:00
ca781651cb
Fix formatting of #1504
2020-11-30 15:56:01 +01:00
92bb477f68
Improve error message
...
The error message has been reworded and the issue template now includes the
date, route, version and backtrace.
2020-11-30 13:02:35 +01:00
3dac33ffba
Add backtraces to errors ( #1498 )
...
Error handling has been reworked to always go through the new `error_template`,
`error_json` and `error_atom` macros.
They all accept a status code followed by a string message or an exception
object. `error_json` accepts a hash with additional fields as third argument.
If the second argument is an exception a backtrace will be printed, if it is a
string only the string is printed. Since up till now only the exception message
was printed a new `InfoException` class was added for situations where no
backtrace is intended but a string cannot be used.
`error_template` with a string message automatically localizes the message.
Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497
`error_json` with a string message does not localize the message. This is the
same as previous behavior. If translations are desired for `error_json` they
can be added easily but those error messages have not been collected yet.
Uncaught exceptions previously only printed a generic message ("Looks like
you've found a bug in Invidious. [...]"). They still print that message
but now also include a backtrace.
2020-11-30 10:59:21 +01:00
b41ca72d2b
revert: remove 'JSON.parse("'
2020-11-26 18:22:31 +01:00
1ba17a0e14
feat: centralize ytInitialData parsing
2020-11-26 13:43:53 +01:00
480d31eb5e
fix: ytInitialData parsing with regex
2020-11-26 07:22:47 +01:00
3a2f4d4def
allow to set a custom anti-captcha api url ( #1473 )
2020-11-20 22:21:26 +01:00
9a8b81fcbe
Fix storyboards
2020-10-25 09:35:16 +01:00
22d9d16a7a
Update parsing of ytInitialData ( #1427 )
...
Update parsing of ytInitialData
2020-10-22 10:02:15 +00:00
0520e67303
[refactor] Finish converting jobs to new job setup ( #1420 )
...
[refactor] Finish converting jobs to new job setup
2020-10-17 12:25:57 +00:00
989317e5d3
Extract RefreshChannelsJob ( #1403 )
2020-10-15 14:22:41 +00:00
812a21bce6
Merge pull request #1372 from tleydxdy/patch-1
...
Add exponential backoff to refresh_channels
2020-10-10 14:01:47 +02:00
f65e57ee82
Fix formatting
2020-10-03 15:19:12 +02:00
9995a8d53f
Allow cookies from anti-captcha to be optional ( #1371 )
2020-09-09 23:04:08 +00:00
f02ac01f7e
Add exponential backoff to refresh_channels
...
In practice with the patch I usually see backoff to 2 hours when blocked, so it should improve recovery time. The lim_thread is to work with multi-threading, not sure if it's the best way to do it.
2020-09-09 11:47:11 -04:00
4a6e920d0e
Use new youtube API to fetch channel videos ( #1355 )
...
* Use new API to fetch videos from channels
This mirrors the process used by subscriptions.gir.st. The old API is
tried first, and if it fails then the new one is used.
* Use the new API whenever getting videos from a channel
I created the get_channel_videos_response function because now instead
of just getting a single url, there are extra steps involved in getting
the API response for channel videos, and these steps don't need to be
repeated throughout the code.
The only remaining exception is the bypass_captcha function, which still
only makes a request to the old API. I don't know whether this code
needs to be updated to use the new API for captcha bypassing to work
correctly.
* Correctly determine video length with new api
* Remove unnecessary line
2020-09-02 20:28:57 +00:00
db83ede73c
Add missing default for ConfigPreferences
2020-07-26 11:09:45 -04:00
452d1e8307
Fix warnings with latest version of Crystal
2020-07-26 10:59:31 -04:00
b5f79444f9
Fix optional value in data extractor
2020-07-21 20:11:13 -04:00
8c65b8c16f
Update playlists to support polymer redesign
2020-06-24 22:19:36 -04:00
83b1d80a5b
Fix signature URL regex
2020-06-16 19:21:29 -05:00
056e7432bd
Update channel playlists to use polymer
2020-06-16 17:51:35 -05:00
1eca969cf6
Add support for polymer redesign
2020-06-15 18:18:04 -05:00
c1cbdae5ee
Make HOST_URL constant
2020-06-15 18:13:14 -05:00
4d4b6a2fa0
Remove top page
2020-06-15 18:11:36 -05:00
d30a972a90
Support Crystal 0.35.0
2020-06-15 18:11:06 -05:00
6435c7b921
Fix reCaptcha
2020-05-28 12:21:47 -05:00
ceb252986e
Update captcha job
2020-05-25 12:52:15 -05:00
750ef296c6
Update captcha handler
2020-05-13 16:09:39 -05:00
ca1185d0be
Fix warnings in latest version of Crystal
2020-04-09 12:18:09 -05:00
02d4186b11
Fix player matching
2020-04-09 10:55:50 -05:00
3f97bebd69
Support adding video to playlist from watch page
2020-04-09 10:55:32 -05:00
59a15ceef6
Remove VarInt class
2020-03-15 17:47:16 -04:00
4011a113cc
Strip invalid characters from referer URLs
2020-03-15 17:47:16 -04:00
a117d87f33
Skip validation checks for videoplayback, ggpht
2020-03-04 13:06:17 -05:00
0d536d11e3
Verify token signature in constant time, Run cheap checks first in token validation process ( #1032 )
...
* Verify token signature in constant time
To prevent timing side channel attacks
* Run cheap checks first in token validation process
Expensive checks such as the nonce lookup on the database or the
signature check can be run after cheap/fast checks.
2020-03-02 10:04:36 -06:00
43da06a354
Remove temp fix for crystal/crystal-lang#7383
2020-02-20 18:30:46 -05:00
bc9dc3bf1e
Update code formatting for Crystal 0.33.0
...
Crystal 0.33.0 introduced some changes to to the code formatter.
Run "crystal tool format" so CI doesn't fail anymore.
2020-02-15 19:52:28 +01:00
a2c5211b20
Check /browse_ajax for channel blocks
2020-02-01 11:23:12 -05:00
648cc0f006
Refactor signature extraction
2020-01-24 17:02:28 -05:00
823603650f
Add support for /sorry/index CAPTCHA
2019-12-03 19:14:11 -05:00
a017574f74
Add support for force_resolve to QUIC client
2019-11-28 08:19:28 -06:00
0e3a48ff76
Update QUICPool
2019-11-24 13:41:47 -05:00
d46b26e3bc
Use QUIC for connections to YouTube
2019-11-18 17:28:32 -05:00
71bc9eea28
Add support for Anti-Captcha
2019-11-09 14:22:39 -05:00
142d974641
Use force_resolve for search suggestions
2019-11-07 12:25:34 -05:00
bcdb8cd770
Fix default fo dark_mode
2019-11-04 17:08:13 -05:00
833a60f29c
Update pubsub to use client pool
2019-11-01 07:34:36 -04:00
13e7cca1a4
Bump read timeout
2019-10-28 12:34:50 -04:00
03e24cccd0
Add support for configurable administrator email
2019-10-27 14:18:07 -04:00
0cf187dee7
Add support for image captcha in Google login
2019-10-27 00:19:05 -04:00
bdeb325bad
Fix monkeypatch for HTTP::Client
2019-10-26 11:51:23 -04:00
a1225b6d0d
Sanitize input to decode_length_seconds
2019-10-26 10:17:25 -04:00
202de1436d
Fix broken connections in pool
2019-10-25 23:06:08 -04:00
7f8746fcd4
Remove invalid connections from pool
2019-10-25 22:40:53 -04:00
e05a25d701
Vary user-agent
2019-10-25 18:02:33 -04:00
6930570fa2
Add HTTPClient pool
2019-10-25 12:58:16 -04:00
7524b5e349
Move feed_menu and default_home into user preferences
2019-10-20 20:43:33 -04:00
2a4b252a9d
Only force resolve for www.youtube.com
2019-10-18 12:41:03 -04:00
be055d9dcb
Add support for custom playlists
2019-10-15 21:17:14 -04:00
7aada3f328
Avoid override for X-Client headers
2019-10-10 23:45:46 -04:00
dad885c051
Add YouTube-Client headers to HTTP requests
2019-10-10 22:03:39 -04:00
da07f99d3d
Bump supported Crystal version
2019-09-30 15:36:54 -04:00
f69f0b97f5
Add fix for index out of bounds during high load
2019-09-24 13:38:50 -04:00
4361ea9686
Update DB calls for 0.31.0
2019-09-24 13:38:50 -04:00
8c2ddb0255
Add config options for host binding and port
2019-09-24 13:38:50 -04:00
1aefc5b540
Update to Crystal 0.31.0, resolve compiler deprecation warnings, update dependencies ( #764 )
...
* shard: update to crystal 0.31.0
Additionally, no longer use the Crystal "markdown" library which has
been removed from the Crystal stdlib in version 0.31.0.
See https://github.com/crystal-lang/crystal/pull/8115 .
Also fix some deprecation warnings using the following commands:
find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.escape/URI\.encode_www_form/g' "{}" \;
find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.unescape/URI\.decode_www_form/g' "{}" \;
sed -i 's/while \%pull\.kind \!\= \:end_object/until \%pull\.kind\.end_object\?/g' src/invidious/helpers/patch_mapping.cr
2019-09-24 13:31:33 -04:00
b1fc80b79a
Update sub_count extractor
2019-09-12 21:09:23 -04:00
50d793e49b
Hide video count for auto-generated channels
2019-09-12 13:11:21 -04:00
34c43b8349
Add support for abbreviated sub count in search
2019-09-12 13:06:27 -04:00
7c75111c41
Refactor error handling for API endpoints
2019-09-05 14:12:14 -04:00
7b53b6bfef
Shrink continuation cursor for YouTube comments
2019-09-04 15:47:27 -04:00
26107bd6c3
Minor refactor
2019-08-27 08:08:26 -05:00
059f50dad4
Add 'playlistThumbnail' to playlist objects
2019-08-21 19:08:11 -05:00
9f9cc1ffb5
Refactor search extractor
2019-08-21 18:23:20 -05:00
2b94975345
Fix playlist_thumbnail extractor
2019-08-16 20:06:21 -05:00
a19cdb5e72
Fix season playlists
2019-08-16 15:46:59 -05:00
f54fbd057e
Add prefers-color-scheme support ( #601 )
...
* Add prefers-color-scheme support
This should fix <https://github.com/omarroth/invidious/issues/559 >.
The cookie storage format has been changed from boolean
("true"/"false") to tri-state ("dark"/"light"/""), so that users
without a cookie set will get dark mode if they have enabled the dark
theme in their operating system. The code for handling the cookie
state, along with the user's operating system theme, has been factored
out into a new function `update_mode`, which is called both at window
load and at the "storage" event listener, because the "storage" event
listener is only trigerred when a change is made to the localStorage
from another tab/window (for more info - see
<https://stackoverflow.com/a/4679754 >).
2019-08-15 11:29:55 -05:00
2a9a348164
Format Crystal files
...
Crystal 0.30.1 apparently introduced some breaking changes to their
code formatter which made CI fail.
The code was automatically formatted by running
crystal tool format
2019-08-14 23:31:07 +02:00
b63f469110
Fix typo in ConfigPreferences
2019-08-09 14:09:24 -05:00
46577fb128
Add support for player styles
...
This currently includes the following styles:
- Invidious, the default
- YouTube, using a centered play button and always visible video control bar
Implements https://github.com/omarroth/invidious/issues/670 .
Supersedes https://github.com/omarroth/invidious/pull/661 .
2019-08-09 02:04:36 +02:00
f18d8229c0
Refactor continuation protocol buffers
2019-07-20 20:18:08 -05:00
1b74a04efd
Add 'force_resolve' to fix issues with rate limiting
2019-07-18 18:51:10 -05:00
1a7b341745
Update Google login
2019-07-12 12:04:39 -05:00
af592ea8c1
Fix extraction for ytInitialData
2019-07-11 07:27:54 -05:00
2cc25b1e6e
Add administrator option to disable proxying
2019-07-08 12:15:18 -05:00
5fd3ed782f
Add fix for #600
2019-07-08 10:00:08 -05:00
c34a24b633
Attempt to optimize query for subscription feed
2019-07-07 14:00:42 -05:00
e5fa5df7be
Chunk video files to bypass throttling
2019-07-04 23:29:28 -05:00
f7dbf2bdd4
Add 'pipe' for proxying assets
2019-07-04 23:29:28 -05:00
041debcd93
Revert "Chunk videoplayback response to avoid throttling"
...
This reverts commit 818cd2454d
2019-07-01 10:45:09 -05:00
e30d70b6d4
Refactor proxy_list into global
2019-06-28 21:17:56 -05:00
dd0be7c522
Revert "Push potential fix for #578"
...
This reverts commit ebfd7d2153
2019-06-28 11:05:08 -05:00
ebfd7d2153
Push potential fix for #578
2019-06-26 14:44:06 -05:00
818cd2454d
Chunk videoplayback response to avoid throttling
2019-06-26 14:43:33 -05:00
b31d1c06f5
Fix typo in StaticFileHandler
2019-06-23 15:41:44 -05:00
6cd884555c
Patch StaticFileHandler to serve files from memory
2019-06-23 12:54:46 -05:00
3be1c9261f
Fix sleep in pull_top_videos
2019-06-15 19:18:36 -05:00
fa6bf21cd1
Update Google login
2019-06-09 13:48:31 -05:00
b43e9ed7e7
Refactor 'description_html'
2019-06-08 15:08:27 -05:00
801dffd571
Fix RSS content-type
2019-06-07 21:39:32 -05:00
2febc268f7
Fix warnings in Crystal 0.29
2019-06-07 21:13:50 -05:00
58995bb3a2
Add support for log levels
2019-06-07 21:13:50 -05:00
8c944815bc
Minor refactor
2019-06-07 21:13:50 -05:00
317d8703ca
Optimize query for pulling popular videos
2019-06-06 21:33:30 -05:00
352e409a6e
Fix toggle_theme when visiting preferences with JS disabled
2019-06-04 20:13:58 -05:00
d6ec441c8e
Add buffer for notification channels
2019-06-03 13:36:49 -05:00
d892ba6aa5
Refactor connection channel for delivering notifications
2019-06-03 13:12:06 -05:00
108648b427
Optimize query for creating subscription feeds
2019-06-02 11:48:18 -05:00
71bf8b6b4d
Refactor connect_listen for notifications
2019-06-02 07:41:53 -05:00
18d66ddded
Add 'needs_update' column for scheduling feed refresh
2019-06-01 10:19:18 -05:00
da48bbf312
Add support for partial POST to '/api/v1/auth/preferences'
2019-05-30 19:00:38 -05:00
ad09e734da
Refactor refresh_feeds
2019-05-27 14:48:57 -05:00
c07ad0941c
Fix typo in refresh_feeds
2019-05-27 12:51:18 -05:00
3ac766530d
Add proper queuing for feed events
2019-05-27 12:23:15 -05:00
51c5f28443
Add config option for updating feeds on event
2019-05-26 12:06:01 -05:00
b016a60a75
Add triggers for updating feeds
2019-05-26 11:28:54 -05:00
208bb2d72f
Catch connection reset when proxying files
2019-05-26 09:41:12 -05:00
267bf289c4
Exclude /api/v1/auth/notifications from middleware
2019-05-21 10:08:49 -05:00
b3e083d866
Add POST /api/v1/auth/subscriptions
2019-05-21 09:01:17 -05:00
a675c64c2d
Refactor DBConfig
2019-05-21 09:00:35 -05:00
06bf0c2622
Copy proxy_file in chunks
2019-05-20 12:06:44 -05:00
3ac8de0a64
Fix proxy_file when response body is empty
2019-05-19 07:13:13 -05:00
ab4df7e078
Fix response for proxied assets
2019-05-18 19:15:47 -05:00
97ef2191fd
Add 'hsts' as config option
2019-05-14 08:21:01 -05:00
bdfe170c3b
Fix length seconds for videos with longer duration
2019-05-11 10:59:47 -05:00
53fbb257b9
Update fix for HTTP Client
2019-05-05 19:03:56 -05:00
65a32d6e20
Update fix for crystal-lang/crystal#7383
2019-05-05 17:47:45 -05:00
bba80bc80f
Fix content-type for HEAD '.jpg'
2019-05-03 08:23:11 -05:00
56fe3ede5b
Add annotation preferences
2019-04-30 23:39:04 -05:00
21190a240f
Add support for adding banner to site header
2019-04-30 21:17:34 -05:00
ef309bd8d0
Translate value for 'familyFriendly'
2019-04-28 14:56:06 -05:00
6cdb6ec711
Add support for plurlization to locales
2019-04-28 14:50:17 -05:00
f15b7cebac
Try to prevent timeout in /data_control
2019-04-24 20:18:35 -05:00
fb7068d415
Add '/api/v1/notifications'
2019-04-20 09:33:45 -05:00
aa10a9d899
Language fixes ( #366 )
...
* Language fixes
2019-04-19 11:14:11 -05:00
bb5a1ad513
Add 'continue_autoplay' preference
2019-04-19 09:38:27 -05:00
2a6c81a89d
Add authentication API
2019-04-18 16:23:50 -05:00
b6fff53b21
Refactor HTTP::Client calls into make_client
2019-04-17 09:06:31 -05:00
3bcb98e644
Add config option to cache annotations from IA
2019-04-15 11:13:09 -05:00
b51fd7fc13
Add view count to video items
2019-04-14 17:43:44 -05:00
80a567bf1e
Fix video count in playlist extractor
2019-04-12 16:37:35 -05:00
ce2a3361eb
Fix missing author name for channel_videos
2019-04-12 16:29:23 -05:00
4e6a931de3
Make check_tables config option
2019-04-11 12:13:25 -05:00
e1c78fcbd3
Update view names to avoid collisions
2019-04-10 19:56:38 -05:00
28d5bedcc7
Speed up table creation
2019-04-10 17:16:18 -05:00
373b890e1d
Log command before execution
2019-04-10 17:09:36 -05:00
5dc45c35e6
Automatically migrate database
2019-04-10 16:23:37 -05:00
305d636217
Add multithreading to pubsub job
2019-04-04 07:49:53 -05:00
31312747e9
Fix from_yaml in ConfigPreferences
2019-04-03 19:04:33 -05:00
f6615a490d
Allow disabling download widget for specific videos (in compliance with DMCA)
2019-04-03 14:54:38 -05:00
bd4f5ebcdf
Add option to configure default user preferences
2019-04-03 11:38:41 -05:00
1fd7ff5655
Add scheme to author thumbnail
2019-04-02 08:51:28 -05:00
a7723e6ded
Implement "fields" parameter from the YouTube Data API ( #429 )
...
* Implement fields handling
2019-03-30 20:18:34 -05:00
1b78001201
Use struct for allocations
2019-03-29 16:30:02 -05:00
388e58bf1e
Update handling for preferences
2019-03-28 13:43:40 -05:00
591a6b330a
Remove 'crawl_threads', fix sleep in fibers
2019-03-25 09:23:42 -05:00
847ee61bf4
Fix typo in APIHandler
2019-03-24 09:01:18 -05:00
0c6cede287
Format files and trim trailing whitespace
2019-03-23 14:05:13 -05:00
57c05354c2
Move 'pretty=1' into middleware
2019-03-23 10:24:30 -05:00
fd26f9f34e
Add support for premieres to search and feed
2019-03-22 14:54:35 -05:00
2ebc773863
Add mixes to genre channels
2019-03-17 18:31:11 -05:00
99aa214859
Add 'thumbnail_id' to playlists
2019-03-17 12:21:47 -05:00
e86eb16d91
Add temporary fix for crystal-lang/crystal#7383
2019-03-11 16:17:40 -05:00
bc1e62ce51
Add 'external_port'
2019-03-08 11:37:52 -06:00
1435516a9c
Add port number to host URL
2019-03-05 12:56:59 -06:00
aa63c3f70e
Update formatting and default feed menu
2019-03-04 10:46:58 -06:00
64cfd2296c
Add support for subscribing to channels via PubSubHubbub
2019-03-03 19:18:23 -06:00
17cf0772fb
Set domain to be nil by default
2019-03-03 12:02:15 -06:00
66605196ad
Remove "detect_language" from dependencies
2019-03-03 11:51:28 -06:00
4be82c5ca6
Add /api/v1/stats
2019-03-01 19:25:16 -06:00
a39b1583da
Add administrator preferences
2019-03-01 16:06:45 -06:00
9a1f4de323
Convert intervals to integers
2019-02-20 09:37:33 -06:00
83493237a5
Add support for translating time intervals
2019-02-20 08:49:54 -06:00
27663b10a2
Add minor API fixes
2019-02-15 17:28:54 -06:00
7a9ef0d664
Add produce_channel_playlists_url
2019-02-09 10:15:14 -06:00
29af5fc4a6
Prune proxy list
2019-02-06 21:29:31 -06:00
276662a147
Use IO::Memory for creating continuation tokens
2019-02-04 15:17:10 -06:00
d0a690c303
Add CORS to API endpoints
2019-02-02 22:48:47 -06:00
7a6a0f364c
Run 'crystal tool format'
2019-01-23 14:37:04 -06:00
e1540390a8
Fix typo in config documentation
2019-01-23 14:30:45 -06:00
71ba071160
Add documentation to config
2019-01-23 14:28:31 -06:00
af449161ff
Add -o option for redirecting output
2019-01-23 14:15:19 -06:00
5cb57fb176
Move 'domain' into config.yml
2019-01-20 22:19:14 -06:00
20c4d213d9
Use config.domain in place of hardcoded value
2019-01-19 09:10:52 -06:00
0832fa9bdb
Filter paid videos from RSS and channel list
2019-01-02 20:09:00 -06:00
saltycrys