* Move Crystal stdlib classes overrides to a separate file
* Document known crystal overrides
* Update crystal overrides for HTTP::Client socket
* Update shard.yml to restrict crystal versions
* Fix compilation error in Crystal 1.1.x (See
https://github.com/crystal-lang/crystal/issues/10965
for more details about this issue).
The private `_post_json` method of the YoutubeAPI requires a ClientConfig
as the third parameter. This was passed in all Youtube API methods except the
`#resolve_url` method.
* Put youtube API functions under the YoutubeAPI namespace
* Implement the following endpoints:
- `next`
- `player`
- `resolve_url`
* Allow a ClientConfig to be passed to YoutubeAPI endpoint handlers.
* Add constants for many new clients
* Fix documentation of YoutubeAPI.browse(): Comments and search
result aren't returned by the browse() endpoint but by the next()
and search() endpoints, respectively.
* Accept gzip compressed data, to help save on bandwidth
* Add debug/trace logging
* Other minor fixes
Related to #1416, it doesn't really fix the real error, but instead mutes the exception message.
Like explained in #1416, this "exception Error" while flushing the client data doesn't harm the client-server connection. However, this exception message continuously spams the logs and makes debugging and error finding really difficult.
* Remove percent-encoding of the search query when calling youtube API, as it
breaks UTF-8
* Empty search redirects to /search, not /
* Show the fullscreen search "home page" (from #1977) at /search
* Allow 'region=' parameter to be passed to /search
* Other minor fixes
Add documentation
Bump web client version string
Add charset=UTF-8 to the 'content-type' header
Parse JSON and return it as a Hash
Handle API error messages
The config file can now be specified with `INVIDIOUS_CONFIG_FILE`.
A YAML formatted string can still be passed with `INVIDIOUS_CONFIG`, replacing
the config file.
Additionally all options can now be specified as environment variables.
The syntax for variable names is `INVIDIOUS_` followed by the option name in
upper case. The values are parsed as YAML.
These new env vars only update the provided main configuration, but it is
possible to point the config file at the example config and then use env vars
for all config options:
```
INVIDIOUS_CONFIG_FILE=./config/config.example.yml \
INVIDIOUS_CHANNEL_THREADS=10 \
./invidious
```
The default log level has been changed from `debug` to `info`.
The `debug` log level is now more verbose. `debug` now gives a general overview
of what is happening (where implemented) while `trace` gives all available
details.
The crystal http client maintains a keepalive connection to the other
server which stays alive for some time. This should be closed if the
client instance is not used again to avoid hogging resources
This is similar to the removed `top-enabled` option but for the Popular feed.
The instance needs to be restarted if the feed was enabled.
Editing admin options on the preferences page is also fixed.
The handling of the feed pages now only happens in a single place.
Instead of redirecting:
- The Top feed now displays a message that it was removed from Invidious.
- The Popular feed now displays a message that it was disabled if it was.
Everything that gets logged now has a log level associated with it.
The log level can be set with the new `-l` or `--log-level` arguments.
The defaul log level is `debug` for now. There aren't many things that get
logged but if the logs get spammed in the future it can be set down to `info`.
The YouTube headers are now always added for requests to YouTube.
Previously they were only added for requests going through QUIC.
The session token is now JSON decoded to unescape escaped Unicode characters.
The comment continuation protobuf has been updated and the request now goes
through the YouTube `pbj` JSON API.
Electric Boogaloo
The long backtrace has been moved into a `<details>` HTML element, as suggested
by @B0pol. To make the error still visible it has been added to the top under
`Title:`. This also encourages informative issue titles.
Error handling has been reworked to always go through the new `error_template`,
`error_json` and `error_atom` macros.
They all accept a status code followed by a string message or an exception
object. `error_json` accepts a hash with additional fields as third argument.
If the second argument is an exception a backtrace will be printed, if it is a
string only the string is printed. Since up till now only the exception message
was printed a new `InfoException` class was added for situations where no
backtrace is intended but a string cannot be used.
`error_template` with a string message automatically localizes the message.
Missing error translations have been collected in https://github.com/iv-org/invidious/issues/1497
`error_json` with a string message does not localize the message. This is the
same as previous behavior. If translations are desired for `error_json` they
can be added easily but those error messages have not been collected yet.
Uncaught exceptions previously only printed a generic message ("Looks like
you've found a bug in Invidious. [...]"). They still print that message
but now also include a backtrace.
In practice with the patch I usually see backoff to 2 hours when blocked, so it should improve recovery time. The lim_thread is to work with multi-threading, not sure if it's the best way to do it.
* Use new API to fetch videos from channels
This mirrors the process used by subscriptions.gir.st. The old API is
tried first, and if it fails then the new one is used.
* Use the new API whenever getting videos from a channel
I created the get_channel_videos_response function because now instead
of just getting a single url, there are extra steps involved in getting
the API response for channel videos, and these steps don't need to be
repeated throughout the code.
The only remaining exception is the bypass_captcha function, which still
only makes a request to the old API. I don't know whether this code
needs to be updated to use the new API for captcha bypassing to work
correctly.
* Correctly determine video length with new api
* Remove unnecessary line
* Verify token signature in constant time
To prevent timing side channel attacks
* Run cheap checks first in token validation process
Expensive checks such as the nonce lookup on the database or the
signature check can be run after cheap/fast checks.
* shard: update to crystal 0.31.0
Additionally, no longer use the Crystal "markdown" library which has
been removed from the Crystal stdlib in version 0.31.0.
See https://github.com/crystal-lang/crystal/pull/8115.
Also fix some deprecation warnings using the following commands:
find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.escape/URI\.encode_www_form/g' "{}" \;
find . \( -type d -name .git -prune \) -o -type f -exec sed -i 's/URI\.unescape/URI\.decode_www_form/g' "{}" \;
sed -i 's/while \%pull\.kind \!\= \:end_object/until \%pull\.kind\.end_object\?/g' src/invidious/helpers/patch_mapping.cr
* Add prefers-color-scheme support
This should fix <https://github.com/omarroth/invidious/issues/559>.
The cookie storage format has been changed from boolean
("true"/"false") to tri-state ("dark"/"light"/""), so that users
without a cookie set will get dark mode if they have enabled the dark
theme in their operating system. The code for handling the cookie
state, along with the user's operating system theme, has been factored
out into a new function `update_mode`, which is called both at window
load and at the "storage" event listener, because the "storage" event
listener is only trigerred when a change is made to the localStorage
from another tab/window (for more info - see
<https://stackoverflow.com/a/4679754>).
Crystal 0.30.1 apparently introduced some breaking changes to their
code formatter which made CI fail.
The code was automatically formatted by running
crystal tool format
Émilien Devos