Merge branch 'dev'

このコミットが含まれているのは:
n9k 2022-06-12 04:53:31 +00:00
コミット 536f25444a
23個のファイルの変更598行の追加133行の削除

ファイルの表示

@ -28,7 +28,8 @@ source venv/bin/activate
python -m pip install -r requirements.txt
```
Before you run it might want to edit [/config.toml][config]:
Before you run it you should edit [/config.toml][config], e.g. these
options:
* `secret_key`:
used for cryptography, make it any long random string
@ -36,11 +37,11 @@ Before you run it might want to edit [/config.toml][config]:
* `segments/directory`:
directory containing stream segments, the default is `stream/` in
the project root
the cloned repository
* `title/file`:
location of the stream title, the default is `title.txt` in the
project root
cloned repository
* `captcha/fonts`:
locations of fonts for the captcha, leaving it blank will use the
@ -53,16 +54,16 @@ python -m uvicorn app:app --port 5051
This will start a webserver listening on localhost port 5051.
If you go to `http://localhost:5051` in a regular web browser now
you should see the interface. When you started the webserver some
credentials were printed in the terminal; you can log in with those at
If you go to `http://localhost:5051` in a web browser now you should see
the site. When you started the webserver some credentials were
printed in the terminal; you can log in with those at
`http://localhost:5051/login` (requires cookies).
The only things left are (1) streaming, and (2) letting other people
access your stream. [/STREAMING.md][streaming] has instructions for
setting up OBS Studio and a Tor onion service. The instructions will
be useful even if you want to use different streaming software and put
your stream on the Internet some other way.
access your stream. [/STREAMING.md][streaming] has instructions for
setting up OBS Studio and a Tor onion service. If you want to use
different streaming software and put your stream on the Internet some
other way, still read those instructions and copy the gist.
## Copying
@ -102,10 +103,10 @@ anonstream is AGPL 3.0 or later, see
* werkzeug <https://github.com/pallets/werkzeug>
([BSD 3-Clause][werkzeug])
[config]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/config.toml
[licence]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/LICENSES/AGPL-3.0-or-later.md
[config]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/config.toml
[licence]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/LICENSES/AGPL-3.0-or-later.md
[settings.svg]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/anonstream/static/settings.svg
[streaming]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/STREAMING.md
[streaming]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/STREAMING.md
[aiofiles]: https://github.com/Tinche/aiofiles/blob/master/LICENSE
[captcha]: https://github.com/lepture/captcha/blob/master/LICENSE

ファイルの表示

@ -1,56 +1,199 @@
### Tor
Install tor and include these lines in your [torrc][torrc]:
Install tor. On Linux you can probably install a package called `tor` and
be done, otherwise [compile it][tor]. On Windows download this binary:
<https://www.torproject.org/download/tor/>.
Find your [torrc][torrc]. On Linux it is probably at `/etc/tor/torrc`.
On Windows it might be somewhere in `%appdata%\tor` or something.
#### Background
A Tor hidden service is a regular TCP service that you talk to via a
6-hop circuit created inside the Tor network. You initiate the creation
of this circuit by providing tor with the service's hostname, which is a
long base32-encoded string ending in ".onion". This hostname is derived
from a pair of cryptographic keys generated by the hidden service
operator.
A TCP service is a computer program you interact with over the Internet
using TCP. TCP is a low-level networking protocol that sits above IP
and creates a reliable so-called "connection" between two computers. It
handles the reordering and resending of packets that are shuffled or
lost in transit on the Internet, such that the bytes sent from one
computer will match exactly the bytes that arrive at the other computer
(barring active interference (MITM), TCP is not secure). Getting
reliability for free greatly simplifies the creation of network
applications, and for this reason and other historical reasons TCP is
ubiquitous on the Internet to this day. Many applications use TCP, for
example IRC, SSH, RTMP, Minecraft, and HTTP (like us here).
#### Configuration
We are now going to create a hidden service. We need to give tor a
directory to store the keys it generates, the location of our existing
TCP service, and a virtual TCP port to listen on. There are two
directives we have to add to our torrc: `HiddenServiceDir` and
`HiddenServicePort`. (There is a commented-out section in the default
torrc for hidden services, you may wish to make these changes there.)
##### `HiddenServiceDir`
`HiddenServiceDir` sets the directory for the hidden service's keys and
other data. You could choose any directory, but you should make sure
it's owned by the user the tor daemon runs as, and the directory's
permissions are `0700/drwx------` (`rwx` for user, `---` for group and
everyone else).
If you configure this in a way tor doesn't like, tor will kill itself
and complain in one of these two ways:
```
Jun 11 23:21:17.000 [warn] Directory /home/n9k/projects/anonstream/hidden_service cannot be read: Permission denied
```
```
Jun 12 02:37:51.036 [warn] Permissions on directory /var/lib/tor/anonstream are too permissive.
```
The simplest option is to copy the examples provided in the torrc, on
Linux that would probably be a directory inside `/var/lib/tor`, e.g.
`HiddenServiceDir /var/lib/tor/anonstream`. tor will create this
directory itself with the uid, gid, and permissions that it likes, which
for me are these:
```
Access: (0700/drwx------) Uid: ( 42/ tor) Gid: ( 42/ tor)
```
###### `HiddenServiceDir` troubleshooting
If you created the directory yourself and gave it the wrong permissions
or uid or gid, delete the directory and let tor create it itself, or do
this:
```
# chown -R tor:tor /var/lib/tor/anonstream
# chmod 0700 /var/lib/tor/anonstream
# chmod 0600 /var/lib/tor/anonstream/*
# chmod 0700 /var/lib/tor/anonstream/*/
```
If the user and group `tor` do not exist, your tor daemon runs as some
other user. There may be a `User` directive in your torrc or in a file
included by your torrc, for example on Debian it's `User debian-tor`.
This means that a tor process running as root will immediately drop
privileges by switching to the user `debian-tor`. The user's primary
group should have the same name, but you can check as root like this:
`# id debian-tor`.
On Linux, if tor is already running you can see what user and group it is
running as like this:
```
$ ps -C tor -o uid,gid,cmd
UID GID CMD
42 42 tor --quiet --runasdaemon 0
$ cat /etc/passwd | grep :42: | cut -f 1 -d : # 42 is the UID here
tor
$ cat /etc/group | grep :42: | cut -f 1 -d : # 42 is the GID here
tor
```
Alternatively you could specify a directory inside the cloned
repository, e.g. `/home/delphine/Documents/anonstream/hidden_service`
or something like that. This will only work if the tor daemon has `rwx`
permissions on the directory and at least `r-x` permissions on all the
directories above it. This is probably not the case for you since your
home folder might have `0700/drwx------` permissions. If you
installed tor as a package, the daemon probably runs as its own user
(e.g. `debian-tor` on Debian, `tor` on Arch/Gentoo). If you want to
figure this out yourself go ahead. I would advise just using
`/var/lib/tor/anonstream` though.
##### `HiddenServicePort`
Include this line verbatim directly below the `HiddenServiceDir` line:
```
HiddenServiceDir $PROJECT_ROOT/hidden_service
HiddenServicePort 80 127.0.0.1:5051
```
but replace `$PROJECT_ROOT` with the folder you cloned the git repo
into.
Then reload tor. If everything went well, the directory will have been
created and your onion address will be in
`$PROJECT_ROOT/hidden_service/hostname`.
tor will listen for connections to our onion address at virtual port
80 (this is the conventional HTTP port), and it will forward that
traffic to our TCP service at 127.0.0.1:5051, which is our webserver.
##### Finish
Example configuration:
```
HiddenServiceDir /var/lib/tor/anonstream
HiddenServicePort 80 127.0.0.1:5051
```
Reload tor to make it reread its torrc: `# pkill -HUP tor`. With
systemd you can alternatively do `# systemctl reload tor`. If
everything went well, the directory will have been created and your
onion address will be in `$HIDDEN_SERVICE_DIR/hostname`.
### OBS Studio
Install OBS Studio. If the autoconfiguration wizard prompts you to
choose a third-party service, ignore it since we're not gonna be doing
that.
Install OBS Studio. If the autoconfiguration wizard prompts you to
choose a third-party service, ignore it since we're not going to be
using a third-party service.
Click `Settings` and set these:
* Advanced
* Recording
* Filename Formatting: `stream`
* Overwrite if file exists: yes
* Video
* Output (Scaled) Resolution: `960x540` or lower
* Output (Scaled) Resolution: `960x540` or lower, or whatever you want
* Common FPS Values: any integer framerate (e.g. 30 or 60)
* Output
* Output Mode: `Advanced`
* Recording:
| | |
|----------------------------|------------------------------------------------------------------------------------------------|
| Type | `Custom Output (FFmpeg)` |
| FFmpeg Output Type | `Output to File` |
| File path or URL | same as config.toml: `segments/directory` (but should be an absolute path) |
| Container Format | `hls` |
| Muxer Settings (if any) | `hls_init_time=0 hls_time=2 hls_list_size=120 hls_flags=delete_segments hls_segment_type=fmp4` |
| Video bitrate | `420 Kbps` or lower |
| Keyframe interval (frames) | `30` (same as the framerate, or exactly half) |
| Video Encoder | libx264, or an H.264 hardware encoder (e.g. `h264_nvenc` for Nvidia, [see here][ffmpeg]) |
| Audio Bitrate | `96 Kbps` |
| Audio Encoder | `aac` |
```
+----------------------------+-------------------------------------+
| Field | Value |
+============================+=====================================+
| Type | `Custom Output (FFmpeg)` |
+----------------------------+-------------------------------------+
| FFmpeg Output Type | `Output to File` |
+----------------------------+-------------------------------------+
| File path or URL | same as the `segments/directory` |
| | option in config.toml, but make it |
| | an absolute path |
+----------------------------+-------------------------------------+
| Container Format | `hls` |
+----------------------------+-------------------------------------+
| Muxer Settings (if any) | `hls_init_time=0 hls_time=2 ` |
| | `hls_list_size=120 ` |
| | `hls_flags=delete_segments ` |
| | `hls_segment_type=fmp4` |
+----------------------------+-------------------------------------+
| Video bitrate | `420 Kbps` or lower, or whatever |
| | you want |
+----------------------------+-------------------------------------+
| Keyframe interval (frames) | `framerate` * `hls_time`, e.g. for |
| | 60fps and an `hls_time` of 2 |
| | seconds, set this to 120 |
+----------------------------+-------------------------------------+
| Video Encoder | libx264, or an H.264 hardware |
| | encoder (e.g. `h264_nvenc` for |
| | Nvidia, [see here][ffmpeg]) |
+----------------------------+-------------------------------------+
| Audio Bitrate | `96 Kbps`, or whatever you want |
+----------------------------+-------------------------------------+
| Audio Encoder | `aac` |
+----------------------------+-------------------------------------+
```
Then click `OK`.
To start streaming click `Start Recording`.
That's it. To start streaming click `Start Recording`.
Because of the muxer settings we used, segments older than four
minutes will be constantly deleted. When you stop streaming, the last
four minutes worth of segments will remain the segments directory.
You can delete them if you want. When you're not streaming you can
delete everything in the segments directory and it'll be fine.
When it is recording, segments older than four minutes will be regularly
deleted, and when it stops recording the last four minutes worth of
segments will remain the segments directory. (You can change the number
of kept segments by modifying the `hls_list_size` option in the muxer
settings.) When it is not recording, you can delete the files in the
segments directory without consequence. Old segments will never be sent
over the network even if they are not deleted.
[tor]: https://gitlab.torproject.org/tpo/core/tor
[torrc]: https://support.torproject.org/#tbb-editing-torrc
[ffmpeg]: https://trac.ffmpeg.org/wiki/HWAccelIntro

ファイルの表示

@ -6,13 +6,16 @@ import secrets
import toml
from collections import OrderedDict
from quart import Quart
from quart_compress import Compress
from werkzeug.security import generate_password_hash
from anonstream.quart import Quart
from anonstream.utils.captcha import create_captcha_factory, create_captcha_signer
from anonstream.utils.colour import color_to_colour
from anonstream.utils.user import generate_token
compress = Compress()
def create_app(config_file):
with open(config_file) as fp:
config = toml.load(fp)
@ -50,6 +53,8 @@ def create_app(config_file):
'MAX_CHAT_SCROLLBACK': config['memory']['chat_scrollback'],
'TASK_PERIOD_ROTATE_USERS': config['tasks']['rotate_users'],
'TASK_PERIOD_ROTATE_CAPTCHAS': config['tasks']['rotate_captchas'],
'TASK_PERIOD_ROTATE_WEBSOCKETS': config['tasks']['rotate_websockets'],
'TASK_PERIOD_BROADCAST_PING': config['tasks']['broadcast_ping'],
'TASK_PERIOD_BROADCAST_USERS_UPDATE': config['tasks']['broadcast_users_update'],
'TASK_PERIOD_BROADCAST_STREAM_INFO_UPDATE': config['tasks']['broadcast_stream_info_update'],
'THRESHOLD_USER_NOTWATCHING': config['thresholds']['user_notwatching'],
@ -61,8 +66,10 @@ def create_app(config_file):
'CHAT_NAME_MIN_CONTRAST': config['chat']['min_name_contrast'],
'CHAT_BACKGROUND_COLOUR': color_to_colour(config['chat']['background_color']),
'CHAT_LEGACY_TRIPCODE_ALGORITHM': config['chat']['legacy_tripcode_algorithm'],
'FLOOD_DURATION': config['flood']['duration'],
'FLOOD_THRESHOLD': config['flood']['threshold'],
'FLOOD_MESSAGE_DURATION': config['flood']['messages']['duration'],
'FLOOD_MESSAGE_THRESHOLD': config['flood']['messages']['threshold'],
'FLOOD_LINE_DURATION': config['flood']['lines']['duration'],
'FLOOD_LINE_THRESHOLD': config['flood']['lines']['threshold'],
'CAPTCHA_LIFETIME': config['captcha']['lifetime'],
'CAPTCHA_FONTS': config['captcha']['fonts'],
'CAPTCHA_ALPHABET': config['captcha']['alphabet'],
@ -112,4 +119,11 @@ def create_app(config_file):
import anonstream.routes
import anonstream.tasks
# Compress some responses
compress.init_app(app)
app.config.update({
"COMPRESS_MIN_SIZE": 2048,
"COMPRESS_LEVEL": 9,
})
return app

ファイルの表示

@ -8,7 +8,7 @@ from quart import current_app, escape
from anonstream.broadcast import broadcast, broadcast_users_update
from anonstream.helpers.chat import generate_nonce_hash, get_scrollback
from anonstream.utils.chat import get_message_for_websocket
from anonstream.utils.chat import get_message_for_websocket, get_approx_linespan
CONFIG = current_app.config
MESSAGES_BY_ID = current_app.messages_by_id
@ -33,18 +33,48 @@ def add_chat_message(user, nonce, comment, ignore_empty=False):
if ignore_empty and len(comment) == 0:
return False
timestamp_ms = time.time_ns() // 1_000_000
timestamp = timestamp_ms // 1000
# Check user
while user['linespan']:
linespan_timestamp, _ = user['linespan'][0]
if timestamp - linespan_timestamp >= CONFIG['FLOOD_LINE_DURATION']:
user['linespan'].popleft()
else:
break
total_recent_linespan = sum(map(
lambda linespan_tuple: linespan_tuple[1],
user['linespan'],
))
if total_recent_linespan > CONFIG['FLOOD_LINE_THRESHOLD']:
raise Rejected(
f'Chat overuse in the last '
f'{CONFIG["FLOOD_LINE_DURATION"]:.0f} seconds'
)
# Check message
message_id = generate_nonce_hash(nonce)
if message_id in MESSAGES_BY_ID:
raise Rejected('Discarded suspected duplicate message')
if len(comment) == 0:
raise Rejected('Message was empty')
if len(comment.strip()) == 0:
raise Rejected('Message was practically empty')
if len(comment) > 512:
raise Rejected('Message exceeded 512 chars')
if comment.count('\n') + 1 > 12:
raise Rejected('Message exceeded 12 lines')
linespan = get_approx_linespan(comment)
if linespan > 12:
raise Rejected('Message would span too many lines')
# Record linespan
linespan_tuple = (timestamp, linespan)
user['linespan'].append(linespan_tuple)
# Create and add message
timestamp_ms = time.time_ns() // 1_000_000
timestamp = timestamp_ms // 1000
try:
last_message = next(reversed(MESSAGES))
except StopIteration:

ファイルの表示

@ -3,7 +3,7 @@
import hashlib
import base64
from collections import OrderedDict
from collections import deque, OrderedDict
from math import inf
from quart import current_app
@ -35,7 +35,7 @@ def generate_user(timestamp, token, broadcaster, presence):
'tag': tag,
'broadcaster': broadcaster,
'verified': broadcaster,
'websockets': set(),
'websockets': {},
'name': None,
'color': colour_to_color(colour),
'tripcode': None,
@ -45,6 +45,7 @@ def generate_user(timestamp, token, broadcaster, presence):
'watching': -inf,
},
'presence': presence,
'linespan': deque(),
}
def get_default_name(user):

51
anonstream/quart.py ノーマルファイル
ファイルの表示

@ -0,0 +1,51 @@
import asyncio
from werkzeug.wrappers import Response as WerkzeugResponse
from quart.app import Quart as Quart_
from quart.asgi import ASGIHTTPConnection as ASGIHTTPConnection_
from quart.utils import encode_headers
RESPONSE_ITERATOR_TIMEOUT = 10
class ASGIHTTPConnection(ASGIHTTPConnection_):
async def _send_response(self, send, response):
await send({
"type": "http.response.start",
"status": response.status_code,
"headers": encode_headers(response.headers),
})
if isinstance(response, WerkzeugResponse):
for data in response.response:
body = data.encode(response.charset) if isinstance(data, str) else data
await asyncio.wait_for(
send({
"type": "http.response.body",
"body": body,
"more_body": True,
}),
timeout=RESPONSE_ITERATOR_TIMEOUT,
)
else:
async with response.response as response_body:
async for data in response_body:
body = data.encode(response.charset) if isinstance(data, str) else data
await asyncio.wait_for(
send({
"type": "http.response.body",
"body": body,
"more_body": True,
}),
timeout=RESPONSE_ITERATOR_TIMEOUT,
)
await send({
"type": "http.response.body",
"body": b"",
"more_body": False,
})
class Quart(Quart_):
asgi_http_class = ASGIHTTPConnection

ファイルの表示

@ -5,7 +5,7 @@ from quart import current_app, request, render_template, redirect, url_for, esca
from anonstream.captcha import get_random_captcha_digest_for
from anonstream.chat import add_chat_message, Rejected
from anonstream.stream import get_stream_title, get_stream_uptime_and_viewership
from anonstream.stream import is_online, get_stream_title, get_stream_uptime_and_viewership
from anonstream.user import add_state, pop_state, try_change_appearance, update_presence, get_users_by_presence, Presence, verify, deverify, BadCaptcha
from anonstream.routes.wrappers import with_user_from, render_template_with_etag
from anonstream.helpers.chat import get_scrollback
@ -24,6 +24,7 @@ async def nojs_stream(user):
'nojs_stream.html',
csp=generate_csp(),
user=user,
online=is_online(),
)
@current_app.route('/info.html')

ファイルの表示

@ -3,8 +3,11 @@
import asyncio
from math import inf
from quart import current_app, websocket
from anonstream.user import see
from anonstream.websocket import websocket_outbound, websocket_inbound
from anonstream.routes.wrappers import with_user_from
@ -12,11 +15,12 @@ from anonstream.routes.wrappers import with_user_from
@with_user_from(websocket)
async def live(user):
queue = asyncio.Queue(maxsize=0)
user['websockets'].add(queue)
user['websockets'][queue] = -inf
producer = websocket_outbound(queue, user)
consumer = websocket_inbound(queue, user)
try:
await asyncio.gather(producer, consumer)
finally:
user['websockets'].remove(queue)
see(user)
user['websockets'].pop(queue)

ファイルの表示

@ -33,14 +33,21 @@ def auth_required(f):
async def wrapper(*args, **kwargs):
if check_auth(request):
return await f(*args, **kwargs)
hint = 'The broadcaster should log in with the credentials printed ' \
'in their terminal.'
body = (
f'<p>{hint}</p>'
if request.authorization is None else
'<p>Wrong username or password. Refresh the page to try again.</p>'
f'<p>{hint}</p>'
hint = (
'The broadcaster should log in with the credentials printed in '
'their terminal.'
)
if request.authorization is None:
body = (
f'<!doctype html>\n'
f'<p>{hint}</p>\n'
)
else:
body = (
f'<!doctype html>\n'
f'<p>Wrong username or password. Refresh the page to try again.</p>\n'
f'<p>{hint}</p>\n'
)
return body, 401, {'WWW-Authenticate': 'Basic'}
return wrapper

ファイルの表示

@ -25,11 +25,12 @@ class UnsafePath(Exception):
def get_mtime():
try:
mtime = os.path.getmtime(CONFIG['SEGMENT_PLAYLIST'])
except FileNotFoundError as e:
raise Stale from e
except OSError as e:
raise Stale(f"couldn't stat playlist: {e}") from e
else:
if time.time() - mtime >= CONFIG['SEGMENT_PLAYLIST_STALE_THRESHOLD']:
raise Stale
mtime_ago = time.time() - mtime
if mtime_ago >= CONFIG['SEGMENT_PLAYLIST_STALE_THRESHOLD']:
raise Stale(f'last modified {mtime_ago:.1f}s ago')
return mtime
@ttl_cache(CONFIG['SEGMENT_PLAYLIST_CACHE_LIFETIME'])
@ -38,13 +39,18 @@ def get_playlist():
try:
mtime = get_mtime()
except Stale as e:
raise Offline from e
reason, *_ = e.args
raise Offline(f'stale playlist: {reason}') from e
else:
playlist = m3u8._load_from_file(CONFIG['SEGMENT_PLAYLIST'])
if playlist.is_endlist:
raise Offline
if len(playlist.segments) == 0:
raise Offline
try:
playlist = m3u8._load_from_file(CONFIG['SEGMENT_PLAYLIST'])
except OSError:
raise Offline(f"couldn't read playlist: {e}") from e
else:
if playlist.is_endlist:
raise Offline('playlist ended')
if len(playlist.segments) == 0:
raise Offline('empty playlist')
return playlist, mtime
@ -76,12 +82,18 @@ def get_next_segment(uri):
segment = None
return segment
async def get_segment_uris():
async def get_segment_uris(token):
try:
segment = get_starting_segment()
except Offline:
except Offline as e:
reason, *_ = e.args
print(
f'[debug @ {time.time():.3f}: {token=}] '
f'stream went offline before we could find any segments ({reason})'
)
return
else:
if segment.init_section is not None:
yield segment.init_section.uri
while True:
@ -91,13 +103,25 @@ async def get_segment_uris():
while True:
try:
next_segment = get_next_segment(segment.uri)
except Offline:
except Offline as e:
reason, *_ = e.args
print(
f'[debug @ {time.time():.3f}: {token=}] '
f'stream went offline while looking for the '
f'segment following {segment.uri!r} ({reason})'
)
return
else:
if next_segment is not None:
segment = next_segment
break
elif time.monotonic() - t0 >= CONFIG['SEGMENT_SEARCH_TIMEOUT']:
print(
f'[debug @ {time.time():.3f}: {token=}] '
f'timed out looking for the segment following '
f'{segment.uri!r} '
f'(timeout={CONFIG["SEGMENT_SEARCH_TIMEOUT"]}s)'
)
return
else:
await asyncio.sleep(CONFIG['SEGMENT_SEARCH_COOLDOWN'])
@ -112,8 +136,7 @@ def path_for(uri):
async def segments(segment_read_hook=lambda uri: None, token=None):
print(f'[debug @ {time.time():.3f}: {token=}] entering segment generator')
uri = None
async for uri in get_segment_uris():
async for uri in get_segment_uris(token):
#print(f'[debug @ {time.time():.3f}: {token=}] {uri=}')
try:
path = path_for(uri)
@ -121,7 +144,7 @@ async def segments(segment_read_hook=lambda uri: None, token=None):
unsafe_path, *_ = e.args
print(
f'[debug @ {time.time():.3f}: {token=}] '
f'segment {uri=} has unsafe {path=}'
f'segment {uri=} has {unsafe_path=}'
)
break
@ -136,10 +159,10 @@ async def segments(segment_read_hook=lambda uri: None, token=None):
f'segment {uri=} at {path=} unexpectedly does not exist'
)
break
else:
print(
f'[debug @ {time.time():.3f}: {token=}] '
f'could not find segment following {uri=} after at least '
f'{CONFIG["SEGMENT_SEARCH_TIMEOUT"]} seconds'
)
except OSError as e:
print(
f'[debug @ {time.time():.3f}: {token=}] '
f'segment {uri=} at {path=} cannot be read: {e}'
)
break
print(f'[debug @ {time.time():.3f}: {token=}] exiting segment generator')

ファイルの表示

@ -11,14 +11,17 @@ const TOKEN_HASH = document.body.dataset.tokenHash;
const CSP = document.body.dataset.csp;
/* insert js-only markup */
const jsmarkup_stream = `<video id="stream_js" src="/stream.mp4?token=${encodeURIComponent(TOKEN)}" autoplay controls></video>`
const jsmarkup_stream_video = '<video id="stream__video" autoplay controls></video>'
const jsmarkup_stream_offline = '<header id="stream__offline"><h1>[offline]</h1></header>'
const jsmarkup_info = '<div id="info_js" data-js="true"></div>';
const jsmarkup_info_float = '<aside id="info_js__float"></aside>';
const jsmarkup_info_float_button = '<button id="info_js__float__button">Reload stream</button>';
const jsmarkup_info_float_button = '<button id="info_js__float__button" accesskey="r">Reload stream</button>';
const jsmarkup_info_float_viewership = '<div id="info_js__float__viewership"></div>';
const jsmarkup_info_float_uptime = '<div id="info_js__float__uptime"></div>';
const jsmarkup_info_title = '<header id="info_js__title"></header>';
const jsmarkup_chat_messages = '<ol id="chat-messages_js" data-js="true"></ol>';
const jsmarkup_chat_messages = `\
<ol id="chat-messages_js" data-js="true"></ol>
<button id="chat-messages-unlock">Chat scroll paused. Click to resume.</button>`;
const jsmarkup_chat_users = `\
<article id="chat-users_js">
<h5 id="chat-users_js__watching-header"></h5>
@ -38,11 +41,11 @@ const jsmarkup_chat_form = `\
<span data-verbose="false">&times;</span>
</span>
</div>
<input id="chat-form_js__submit" type="submit" value="Chat" accesskey="p" disabled>
<input id="chat-form_js__captcha-digest" type="hidden" name="captcha-digest" disabled>
<input id="chat-form_js__captcha-image" type="image" width="72" height="30">
<input id="chat-form_js__captcha-answer" name="captcha-answer" placeholder="Captcha" disabled>
<input id="chat-form_js__settings" type="image" src="/static/settings.svg" width="28" height="28" alt="Settings">
<input id="chat-form_js__submit" type="submit" value="Chat" accesskey="p" disabled>
<article id="chat-form_js__notice">
<button id="chat-form_js__notice__button" type="button">
<header id="chat-form_js__notice__button__header"></header>
@ -81,9 +84,13 @@ const insert_jsmarkup = () => {
style_tripcode_colors.nonce = CSP;
document.head.insertAdjacentElement("beforeend", style_tripcode_colors);
}
if (document.getElementById("stream_js") === null) {
if (document.getElementById("stream__video") === null) {
const parent = document.getElementById("stream");
parent.insertAdjacentHTML("beforeend", jsmarkup_stream);
parent.insertAdjacentHTML("beforeend", jsmarkup_stream_video);
}
if (document.getElementById("stream__offline") === null) {
const parent = document.getElementById("stream");
parent.insertAdjacentHTML("beforeend", jsmarkup_stream_offline);
}
if (document.getElementById("info_js") === null) {
const parent = document.getElementById("info");
@ -262,6 +269,9 @@ let stats = null;
let stats_received = null;
let default_name = {true: "Broadcaster", false: "Anonymous"};
let max_chat_scrollback = 256;
let pingpong_period = 8.0;
let ping = null;
const pingpong_timeout = () => pingpong_period * 1.5 + 4.0;
const tidy_stylesheet = ({stylesheet, selector_regex, ignore_condition}) => {
const to_delete = [];
const to_ignore = new Set();
@ -566,6 +576,12 @@ const update_users_list = () => {
chat_users_notwatching_header.innerText = `Not watching (${notwatching})`;
}
const show_offline_screen = () => {
video.removeAttribute("src");
video.load();
stream.dataset.offline = "";
}
const on_websocket_message = (event) => {
//console.log("websocket message", event);
const receipt = JSON.parse(event.data);
@ -579,7 +595,7 @@ const on_websocket_message = (event) => {
case "init":
console.log("ws init", receipt);
// set title
pingpong_period = receipt.pingpong;
set_title(receipt.title);
// update stats (uptime/viewership)
@ -662,7 +678,7 @@ const on_websocket_message = (event) => {
}
// stream reload button
if (stats === null || stream.networkState === stream.NETWORK_LOADING) {
if (stats === null || video.networkState === video.NETWORK_LOADING) {
info_button.removeAttribute("data-visible");
} else {
info_button.dataset.visible = "";
@ -692,11 +708,13 @@ const on_websocket_message = (event) => {
case "message":
console.log("ws message", receipt);
create_and_add_chat_message(receipt.message);
chat_messages.scrollTo({
left: 0,
top: chat_messages.scrollTopMax,
behavior: "smooth",
});
if (chat_messages.dataset.scrollLock === undefined) {
chat_messages.scrollTo({
left: 0,
top: chat_messages.scrollTopMax,
behavior: "smooth",
});
}
break;
case "set-users":
@ -760,6 +778,13 @@ const on_websocket_message = (event) => {
break;
case "ping":
console.log("ws ping");
ping = new Date();
const payload = {type: "pong"};
ws.send(JSON.stringify(payload));
break;
default:
console.log("incomprehensible websocket message", receipt);
}
@ -815,18 +840,26 @@ const connect_websocket = () => {
connect_websocket();
/* stream reload button */
const stream = document.getElementById("stream_js");
const video = document.getElementById("stream__video");
const info_button = document.getElementById("info_js__float__button");
info_button.addEventListener("click", (event) => {
stream.load();
stream.removeAttribute("data-offline");
video.src = `/stream.mp4?token=${encodeURIComponent(TOKEN)}`;
video.load();
info_button.removeAttribute("data-visible");
});
stream.addEventListener("error", (event) => {
video.addEventListener("error", (event) => {
if (video.error !== null && video.error.message === "404: Not Found") {
show_offline_screen();
}
if (stats !== null) {
info_button.dataset.visible = "";
}
});
/* load stream */
video.src = `/stream.mp4?token=${encodeURIComponent(TOKEN)}`;
/* override js-only chat form */
const chat_form_nonce = document.getElementById("chat-form_js__nonce");
const chat_form_comment = document.getElementById("chat-form_js__comment");
@ -866,7 +899,31 @@ const peg_bottom = (entries) => {
}
const resize = new ResizeObserver(peg_bottom);
resize.observe(chat_messages);
track_scroll(chat_messages);
/* chat scroll lock */
chat_messages.addEventListener("scroll", (event) => {
track_scroll(chat_messages);
const scroll = chat_messages.scrollTopMax - chat_messages.scrollTop;
const locked = chat_messages.dataset.scrollLock !== undefined
if (scroll >= 160 && !locked) {
chat_messages.dataset.scrollLock = "";
} else if (scroll == 0 && locked) {
chat_messages.removeAttribute("data-scroll-lock");
}
});
track_scroll(chat_messages);
const chat_messages_unlock = document.getElementById("chat-messages-unlock");
chat_messages_unlock.addEventListener("click", (event) => {
chat_messages.scrollTop = chat_messages.scrollTopMax;
});
/* close websocket after prolonged absence of pings */
const rotate_websocket = () => {
const this_pingpong_timeout = pingpong_timeout();
if (ping === null || (new Date() - ping) / 1000 > this_pingpong_timeout) {
console.log(`no pings heard in ${this_pingpong_timeout} seconds, closing websocket...`);
ws.close();
}
setTimeout(rotate_websocket, this_pingpong_timeout * 1000);
};
setTimeout(rotate_websocket, pingpong_timeout() * 1000);

ファイルの表示

@ -44,6 +44,7 @@ a {
color: #42a5d7;
}
iframe {
display: grid;
width: 100%;
border: none;
box-sizing: border-box;
@ -56,11 +57,30 @@ noscript {
#stream {
background: black;
grid-area: stream;
position: relative;
}
#stream_js {
#stream__video {
width: 100%;
height: 100%;
}
#stream__offline {
position: absolute;
top: 0;
width: 100%;
height: 100%;
text-align: center;
display: grid;
align-content: center;
font-size: 20pt;
background-color: black;
user-select: none;
}
#stream__offline > h1 {
margin: 0;
}
#stream:not([data-offline]) > #stream__offline {
display: none;
}
#stream_nojs {
height: 100%;
}
@ -100,13 +120,11 @@ noscript {
#chat {
display: grid;
grid-auto-rows: auto 1fr auto;
grid-auto-rows: auto minmax(150px, 1fr) auto;
background-color: var(--chat-bg-color);
border-top: var(--chat-border);
border-bottom: var(--chat-border);
grid-area: chat;
height: 50vh;
min-height: 24ch;
position: relative;
}
#chat__toggle {
@ -180,6 +198,28 @@ noscript {
bottom: 0;
font-size: 11pt;
}
#chat-messages_js:not([data-scroll-lock]) + #chat-messages-unlock {
display: none;
}
#chat-messages-unlock {
position: absolute;
bottom: 1rem;
color: inherit;
font-size: 10pt;
text-align: center;
width: calc(100% - 4rem);
margin: 0 2rem;
padding: 0.5rem 0;
box-sizing: border-box;
background-color: #316aaf;
border-radius: 4px;
border: 1px outset #4c91e6;
box-shadow: 0 0 3px #4c91e6;
cursor: pointer;
}
#chat-messages-unlock:hover {
background-color: #3674bf;
}
#chat-messages_nojs {
height: 100%;
}
@ -199,7 +239,12 @@ noscript {
cursor: default;
}
.chat-insignia {
text-shadow: 0 0 2px orangered;
color: var(--chat-bg);
border: 1px outset #0000007f;
text-shadow: 0 0 1px var(--chat-bg-color);
background-color: orangered;
padding: 0 2px;
border-radius: 3px;
cursor: help;
}
.chat-name {
@ -319,6 +364,7 @@ noscript {
}
#chat-form_js__submit {
grid-column: 5;
grid-row: 2;
}
#chat-form_js:not([data-captcha]) > #chat-form_js__captcha-image,
#chat-form_js:not([data-captcha]) > #chat-form_js__captcha-answer {
@ -470,7 +516,7 @@ footer {
border-style: inset;
}
#both:target #info_nojs {
height: 9ch;
height: 11ch;
}
#info:target {
grid-row-end: chat-end;

ファイルの表示

@ -5,7 +5,7 @@ import asyncio
import itertools
from functools import wraps
from quart import current_app
from quart import current_app, websocket
from anonstream.broadcast import broadcast, broadcast_users_update
from anonstream.stream import is_online, get_stream_title, get_stream_uptime_and_viewership
@ -86,6 +86,27 @@ async def t_expire_captchas(iteration):
for digest in to_delete:
CAPTCHAS.pop(digest)
@with_period(CONFIG['TASK_PERIOD_ROTATE_WEBSOCKETS'])
@with_timestamp
async def t_close_websockets(timestamp, iteration):
THRESHOLD = CONFIG['TASK_PERIOD_BROADCAST_PING'] * 1.5 + 4.0
if iteration == 0:
return
else:
for user in USERS:
for queue in user['websockets']:
last_pong = user['websockets'][queue]
last_pong_ago = timestamp - last_pong
if last_pong_ago > THRESHOLD:
queue.put_nowait({'type': 'close'})
@with_period(CONFIG['TASK_PERIOD_BROADCAST_PING'])
async def t_broadcast_ping(iteration):
if iteration == 0:
return
else:
broadcast(USERS, payload={'type': 'ping'})
@with_period(CONFIG['TASK_PERIOD_BROADCAST_USERS_UPDATE'])
async def t_broadcast_users_update(iteration):
if iteration == 0:
@ -147,5 +168,7 @@ async def t_broadcast_stream_info_update(iteration):
current_app.add_background_task(t_sunset_users)
current_app.add_background_task(t_expire_captchas)
current_app.add_background_task(t_close_websockets)
current_app.add_background_task(t_broadcast_ping)
current_app.add_background_task(t_broadcast_users_update)
current_app.add_background_task(t_broadcast_stream_info_update)

ファイルの表示

@ -242,13 +242,13 @@
<label for="password-toggle" class="show-password pseudolink">set</label>
{% else %}
<label id="tripcode" for="password-toggle" class="show-password tripcode">{{ user.tripcode.digest }}</label>
<label id="show-cleared" for="cleared-toggle" class="pseudolink x">&cross;</label>
<label id="show-cleared" for="cleared-toggle" class="pseudolink x">&times;</label>
<div id="cleared" class="tripcode">(cleared)</div>
<label id="hide-cleared" for="cleared-toggle" class="pseudolink">undo</label>
{% endif %}
</div>
<input id="appearance-form__password" name="password" type="password" placeholder="(tripcode password)" maxlength="1024">
<div id="hide-password"><label for="password-toggle" class="pseudolink x">&cross;</label></div>
<div id="hide-password"><label for="password-toggle" class="pseudolink x">&times;</label></div>
<div id="appearance-form__buttons">
<div id="appearance-form__buttons__exit"><label for="toggle" class="pseudolink">Return to chat</label></div>
<input type="submit" value="Update">

ファイルの表示

@ -109,7 +109,12 @@
cursor: default;
}
.chat-message__insignia {
text-shadow: 0 0 2px orangered;
color: var(--chat-bg);
border: 1px outset #0000007f;
text-shadow: 0 0 1px var(--chat-bg-color);
background-color: orangered;
padding: 0 2px;
border-radius: 3px;
cursor: help;
}
.chat-message__name {

ファイルの表示

@ -83,11 +83,14 @@
#h1 {
animation-duration: {{ 36000 - uptime }}s;
}
#uptime-dynamic {
animation: disappear step-end {{ 360000 - uptime }}s forwards;
}
#uptime-dynamic-overflow {
animation: appear step-end {{ 360000 - uptime }}s backwards;
}
#uptime-dynamic {
animation: disappear step-end {{ 360000 - uptime }}s forwards;
#uptime-dynamic-overflow::after {
content: "100+ hours";
}
@keyframes appear {
from {
@ -137,7 +140,7 @@
{% if user.presence != Presence.WATCHING %}
<form id="float__form" action="{{ url_for('nojs_stream') }}" target="stream_nojs">
<input type="hidden" name="token" value="{{ user.token }}">
<input type="submit" value="Reload stream">
<input type="submit" value="Reload stream" accesskey="r">
</form>
{% endif %}
<div id="float__viewership">{{ viewership }} viewers</div>
@ -146,11 +149,11 @@
<span id="uptime-static__label">Uptime:</span>
<span>
{%- if uptime >= 3600 -%}
{{- (uptime // 3600) | int -}}
{{- uptime | int // 3600 -}}
{{- ':' -}}
{{- '%02.0f' | format(uptime % 3600 // 60) -}}
{%- else -%}
{{- uptime % 3600 // 60 | int -}}
{{- uptime | int % 3600 // 60 -}}
{%- endif -%}
{{- ':' -}}
{{- '%02.0f' | format(uptime % 60) -}}
@ -170,7 +173,7 @@
{{- '' -}}
<span id="s0"></span>
</div>
<div id="uptime-dynamic-overflow">100+ hours</div>
<div id="uptime-dynamic-overflow"></div>
{% endif %}
</div>
</aside>

ファイルの表示

@ -16,14 +16,35 @@
height: 100%;
margin: 0;
overflow: hidden;
color: #ddd;
font-family: sans-serif;
}
#stream {
width: 100%;
height: 100%;
}
{% if online %}
#video {
width: 100%;
height: 100%;
}
{% else %}
#offline {
width: 100%;
height: 100%;
text-align: center;
display: grid;
align-content: center;
font-size: 20pt;
user-select: none;
}
#offline > h1 {
margin: 0;
}
{% endif %}
</style>
</head>
<body>
<video id="stream" src="{{ url_for('stream', token=user.token) }}" autoplay controls></video>
{% if online %}
<video id="video" src="{{ url_for('stream', token=user.token) }}" autoplay controls></video>
{% else %}
<header id="offline"><h1>[offline]</h1></header>
{% endif %}
</body>
</html>

ファイルの表示

@ -152,12 +152,12 @@ def deverify(timestamp, user):
n_user_messages = 0
for message in reversed(MESSAGES):
message_sent_ago = timestamp - message['timestamp']
if message_sent_ago >= CONFIG['FLOOD_DURATION']:
if message_sent_ago >= CONFIG['FLOOD_MESSAGE_DURATION']:
break
elif message['token'] == user['token']:
n_user_messages += 1
if n_user_messages >= CONFIG['FLOOD_THRESHOLD']:
if n_user_messages >= CONFIG['FLOOD_MESSAGE_THRESHOLD']:
user['verified'] = False
def _update_presence(timestamp, user):

ファイルの表示

@ -3,6 +3,7 @@
import base64
import hashlib
import math
import secrets
class NonceReuse(Exception):
@ -18,3 +19,10 @@ def get_message_for_websocket(user, message):
**{key: message[key] for key in message_keys},
**{key: user[key] for key in user_keys},
}
def get_approx_linespan(text):
def height(line):
return math.ceil(len(line) / 48)
linespan = sum(map(height, text.splitlines()))
linespan = linespan if linespan > 0 else 1
return linespan

ファイルの表示

@ -3,7 +3,7 @@
from enum import Enum
WS = Enum('WS', names=('MESSAGE, CAPTCHA, APPEARANCE'))
WS = Enum('WS', names=('PONG', 'MESSAGE', 'CAPTCHA', 'APPEARANCE'))
class Malformed(Exception):
pass
@ -48,5 +48,8 @@ def parse_websocket_data(receipt):
case 'captcha':
return WS.CAPTCHA, ()
case 'pong':
return WS.PONG, ()
case _:
raise Malformed('malformed type')

ファイルの表示

@ -10,6 +10,7 @@ from anonstream.stream import get_stream_title, get_stream_uptime_and_viewership
from anonstream.captcha import get_random_captcha_digest_for
from anonstream.chat import get_all_messages_for_websocket, add_chat_message, Rejected
from anonstream.user import get_all_users_for_websocket, see, verify, deverify, BadCaptcha, try_change_appearance
from anonstream.wrappers import with_timestamp
from anonstream.utils.chat import generate_nonce
from anonstream.utils.websocket import parse_websocket_data, Malformed, WS
@ -29,11 +30,17 @@ async def websocket_outbound(queue, user):
},
'scrollback': CONFIG['MAX_CHAT_SCROLLBACK'],
'digest': get_random_captcha_digest_for(user),
'pingpong': CONFIG['TASK_PERIOD_BROADCAST_PING'],
}
await websocket.send_json(payload)
await websocket.send_json({'type': 'ping'})
while True:
payload = await queue.get()
await websocket.send_json(payload)
if payload['type'] == 'close':
await websocket.close(1011)
break
else:
await websocket.send_json(payload)
async def websocket_inbound(queue, user):
while True:
@ -59,17 +66,26 @@ async def websocket_inbound(queue, user):
handle = handle_inbound_appearance
case WS.CAPTCHA:
handle = handle_inbound_captcha
payload = handle(user, *parsed)
case WS.PONG:
handle = handle_inbound_pong
payload = handle(queue, user, *parsed)
queue.put_nowait(payload)
if payload is not None:
queue.put_nowait(payload)
def handle_inbound_captcha(user):
@with_timestamp
def handle_inbound_pong(timestamp, queue, user):
print(f'[pong] {user["token"]}')
user['websockets'][queue] = timestamp
return None
def handle_inbound_captcha(queue, user):
return {
'type': 'captcha',
'digest': get_random_captcha_digest_for(user),
}
def handle_inbound_appearance(user, name, color, password, want_tripcode):
def handle_inbound_appearance(queue, user, name, color, password, want_tripcode):
errors = try_change_appearance(user, name, color, password, want_tripcode)
if errors:
return {
@ -85,7 +101,7 @@ def handle_inbound_appearance(user, name, color, password, want_tripcode):
#'tripcode': user['tripcode'],
}
def handle_inbound_message(user, nonce, comment, digest, answer):
def handle_inbound_message(queue, user, nonce, comment, digest, answer):
try:
verification_happened = verify(user, digest, answer)
except BadCaptcha as e:

ファイルの表示

@ -33,6 +33,8 @@ chat_scrollback = 256
[tasks]
rotate_users = 60.0
rotate_captchas = 60.0
rotate_websockets = 2.0
broadcast_ping = 8.0
broadcast_users_update = 4.0
broadcast_stream_info_update = 3.0
@ -47,10 +49,14 @@ min_name_contrast = 3.0
background_color = "#232327"
legacy_tripcode_algorithm = false
[flood]
[flood.messages]
duration = 20.0
threshold = 4
[flood.lines]
duration = 20.0
threshold = 20
[thresholds]
user_notwatching = 8.0
user_tentative = 20.0

ファイルの表示

@ -1,6 +1,7 @@
aiofiles==0.8.0
asgiref==3.5.0
blinker==1.4
Brotli==1.0.9
git+https://github.com/lepture/captcha@27920681b86c27c990da484984f673dba1dd47e5#egg=captcha
click==8.0.4
h11==0.13.0
@ -16,6 +17,7 @@ MarkupSafe==2.1.0
Pillow==9.0.1
priority==2.0.0
quart==0.16.3
quart-compress==0.2.1
toml==0.10.2
uvicorn==0.17.5
Werkzeug==2.0.3