v1.6.9

It was too high on mobile screens.

README.md

@@ -5,7 +5,7 @@ Recipe for livestreaming over Tor
 ## Repo
 
 The canonical location of this repo is
-<https://git.076.ne.jp/ninya9k/anonstream>.
+<https://gitler.moe/ninya9k/anonstream>.
 
 These mirrors also exist:
 * <https://gitlab.com/ninya9k/anonstream>
@@ -13,12 +13,12 @@ These mirrors also exist:
 
 ## Setup
 
-You must have Python 3.10 at a minimum.  You can check your version of Python
-with `python --version`.
+You must have Python 3.10 at a minimum.  You can check your version of
+Python with `python --version`.
 
 Clone the repo:
 ```sh
-git clone https://git.076.ne.jp/ninya9k/anonstream.git
+git clone https://gitler.moe/ninya9k/anonstream.git
 cd anonstream
 ```
 
@@ -29,14 +29,15 @@ source venv/bin/activate
 python -m pip install -r requirements.txt
 ```
 
-Before you run it you may want to edit the config ([/config.toml][config]).
-Most of the defaults are probably okay, but here are some that you might want
-to know what they do:
+This is all the setup needed to run the application, but before you do
+you may want to edit [the config](config.toml).  Most of the defaults
+are probably okay, but here are some that you might want to know what
+they do:
 
 * `secret_key`:
   used for cryptography, make it any long random string (e.g.
   `$ dd if=/dev/urandom bs=16 count=1 | base64`), definitely set this
-  yourself before running in "production" (whatever that is for you)
+  yourself before running in "production"
 
 * `segments/directory`:
   directory containing stream segments, the default is `stream/` in
@@ -59,8 +60,8 @@ Run it:
 python -m anonstream
 ```
 
-This will start a webserver listening on the local host at port 5051 (use
-`--port PORT` to override).
+This will start a webserver listening on the local host at port 5051
+(use `--port PORT` to override).
 
 If you go to `http://localhost:5051` in a web browser now you should see
 the site.  When you started the webserver some credentials were printed
@@ -68,8 +69,9 @@ in the terminal; you can log in with those at
 `http://localhost:5051/login`.
 
 The only things left are (1) streaming, and (2) letting other people
-access your stream.  [/STREAMING.md][streaming] has instructions for
-setting up OBS Studio and a Tor onion service.  If you want to use
+access your stream.  [OBS.md](doc/guide/OBS.md) has instructions for
+setting up OBS Studio and [ONIONSITE.md](doc/guide/ONIONSITE.md) has
+instructions for creating a Tor onion service.  If you want to use
 different streaming software and put your stream on the Internet some
 other way, read those instructions and copy the gist.
 
@@ -90,21 +92,20 @@ python -m uvicorn asgi:create_app --factory --port 5051
 In either case you can explicitly set the location of the config file
 using the `ANONSTREAM_CONFIG` environment variable.
 
-
 ## Hacking
 
 anonstream has APIs for accessing internal state and hooking into
 internal events. They can be used by humans and other programs.  See
-[/HACKING.md][hacking].
+[HACKING.md](/doc/HACKING.md).
 
 ## Copying
 
 anonstream is AGPL 3.0 or later, see
-[/LICENSES/AGPL-3.0-or-later.md][licence].
+[LICENSES/AGPL-3.0-or-later.md][licence].
 
 ### Assets
 
-* [/anonstream/static/settings.svg][settings.svg]:
+* [anonstream/static/settings.svg][settings.svg]:
   [setting](https://thenounproject.com/icon/setting-685325/) by
   [ulimicon](https://thenounproject.com/unlimicon/) is licensed under
   [CC BY 3.0](https://creativecommons.org/licenses/by/3.0/).
@@ -135,11 +136,8 @@ anonstream is AGPL 3.0 or later, see
 * werkzeug <https://github.com/pallets/werkzeug>
   ([BSD 3-Clause][werkzeug])
 
-[config]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/config.toml
-[hacking]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/HACKING.md
-[licence]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/LICENSES/AGPL-3.0-or-later.md
-[settings.svg]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/anonstream/static/settings.svg
-[streaming]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/STREAMING.md
+[licence]: https://gitler.moe/ninya9k/anonstream/src/branch/master/LICENSES/AGPL-3.0-or-later.md
+[settings.svg]: https://gitler.moe/ninya9k/anonstream/src/branch/master/anonstream/static/settings.svg
 
 [aiofiles]: https://github.com/Tinche/aiofiles/blob/master/LICENSE
 [captcha]: https://github.com/lepture/captcha/blob/master/LICENSE

STREAMING.md

@@ -1,202 +0,0 @@
-### Tor
-
-Install tor.  On Linux you can probably install a package called `tor` and
-be done, otherwise [compile it][tor].  On Windows download this binary:
-<https://www.torproject.org/download/tor/>.
-
-Find your [torrc][torrc].  On Linux it is probably at `/etc/tor/torrc`.
-On Windows it might be somewhere in `%appdata%\tor` or something.
-
-#### Background
-
-A Tor hidden service is a regular TCP service that you talk to via a
-6-hop circuit created inside the Tor network.  You initiate the creation
-of this circuit by providing tor with the service's hostname, a long
-base32-encoded string ending in ".onion".  This hostname is derived from
-a pair of cryptographic keys generated by the hidden service operator.
-
-A TCP service is a computer program you interact with over the Internet
-using TCP.  TCP is a low-level networking protocol that sits above IP
-and creates a reliable "connection" between two computers.  It handles
-the reordering and resending of packets that are shuffled or lost in
-transit on the Internet, such that the bytes sent from one computer will
-match exactly the bytes that arrive at the other (barring active
-interference (MITM), TCP is not secure).  Getting reliability for free
-greatly simplifies the creation of network applications, and for this
-reason and other historical reasons TCP is ubiquitous on the Internet to
-this day.  Many applications use TCP, for example IRC, SSH, RTMP,
-Minecraft, and HTTP (like us here).
-
-#### Configuration
-
-We are now going to create a hidden service. We need to give tor a
-directory to store the keys it generates, the location of our existing
-TCP service, and a virtual TCP port to listen on.  There are two
-directives we have to add to our torrc: `HiddenServiceDir` and
-`HiddenServicePort`.  (There is a commented-out section in the default
-torrc for hidden services, you may wish to make these changes there.)
-
-##### `HiddenServiceDir`
-
-`HiddenServiceDir` sets the directory for the hidden service's keys and
-other data. You could choose any directory, but you should make sure
-it's owned by the user the tor daemon runs as, and the directory's
-permissions are `0700/drwx------` (`rwx` for user, `---` for group and
-everyone else).
-
-If you configure this in a way tor doesn't like, tor will kill itself
-and complain in one of these two ways:
-```
-Jun 11 23:21:17.000 [warn] Directory /home/n9k/projects/anonstream/hidden_service cannot be read: Permission denied
-```
-```
-Jun 12 02:37:51.036 [warn] Permissions on directory /var/lib/tor/anonstream are too permissive.
-```
-
-The simplest option is to copy the examples provided in the torrc, on
-Linux that would probably be a directory inside `/var/lib/tor`, e.g.
-`HiddenServiceDir /var/lib/tor/anonstream`.  tor will create this
-directory itself with the uid, gid, and permissions that it likes, which
-for me are these:
-```
-Access: (0700/drwx------)  Uid: (   42/     tor)   Gid: (   42/     tor)
-```
-
-###### `HiddenServiceDir` troubleshooting
-
-If you created the directory yourself and gave it the wrong permissions
-or uid or gid, delete the directory and let tor create it itself, or do
-this:
-```
-# chown -R tor:tor /var/lib/tor/anonstream
-# chmod 0700 /var/lib/tor/anonstream
-# chmod 0600 /var/lib/tor/anonstream/*
-# chmod 0700 /var/lib/tor/anonstream/*/
-```
-
-If the user and group `tor` do not exist, your tor daemon runs as some
-other user.  There may be a `User` directive in your torrc or in a file
-included by your torrc, for example on Debian it's `User debian-tor`.
-This means that a tor process running as root will immediately drop
-privileges by switching to the user `debian-tor`. The user's primary
-group should have the same name, check like this as root:
-`# id debian-tor`.
-
-On Linux, if tor is already running you can see what user and group it is
-running as like this:
-```
-$ ps -C tor -o uid,gid,cmd
-UID GID CMD
- 42  42 tor --quiet --runasdaemon 0
-$ cat /etc/passwd | grep :42: | cut -f 1 -d :  # 42 is the UID here
-tor
-$ cat /etc/group | grep :42: | cut -f 1 -d :  # 42 is the GID here
-tor
-```
-
-Alternatively you could specify a directory inside the cloned
-repository, e.g. `/home/delphine/Documents/anonstream/hidden_service`
-or something like that.  This will only work if the tor daemon has `rwx`
-permissions on the directory and at least `r-x` permissions on all the
-directories above it.  This is probably not the case for you since your
-home folder might have `0700/drwx------` permissions.  If you
-installed tor as a package, the daemon probably runs as its own user
-(e.g. `debian-tor` on Debian, `tor` on Arch/Gentoo).  If you want to
-figure this out yourself go ahead.  I would advise just using
-`/var/lib/tor/anonstream` though.
-
-##### `HiddenServicePort`
-
-Include this line verbatim directly below the `HiddenServiceDir` line:
-```
-HiddenServicePort 80 127.0.0.1:5051
-```
-
-tor will listen for connections to our onion address at virtual port 80
-(the conventional HTTP port), and it will forward traffic to the TCP
-service at 127.0.0.1:5051, which is our webserver.
-
-##### Finish
-
-Example configuration:
-```
-HiddenServiceDir /var/lib/tor/anonstream
-HiddenServicePort 80 127.0.0.1:5051
-```
-
-Reload tor to make it reread its torrc: `# pkill -HUP tor`.  With
-systemd you can alternatively do `# systemctl reload tor`.  If
-everything went well, the directory will have been created and your
-onion address will be in `$HIDDEN_SERVICE_DIR/hostname`.
-
-### OBS Studio
-
-Install OBS Studio.  If the autoconfiguration wizard prompts you to
-choose a third-party service, ignore it since we're not going to be
-using a third-party service.
-
-Click `Settings` and set these:
-
-* Advanced
-  * Recording
-    * Filename Formatting: `stream`
-    * Overwrite if file exists: yes
-* Video
-  * Output (Scaled) Resolution: `960x540` or lower, or whatever you want
-  * Common FPS Values: any integer framerate (e.g. 30 or 60)
-* Output
-  * Output Mode: `Advanced`
-  * Recording:
-    ```
-    +----------------------------+-------------------------------------+
-    | Field                      | Value                               |
-    +============================+=====================================+
-    | Type                       | `Custom Output (FFmpeg)`            |
-    +----------------------------+-------------------------------------+
-    | FFmpeg Output Type         | `Output to File`                    |
-    +----------------------------+-------------------------------------+
-    | File path or URL           | same as the `segments/directory`    |
-    |                            | option in config.toml, but make it  |
-    |                            | an absolute path                    |
-    +----------------------------+-------------------------------------+
-    | Container Format           | `hls`                               |
-    +----------------------------+-------------------------------------+
-    | Muxer Settings (if any)    | `hls_init_time=0 hls_time=2 `       |
-    |                            | `hls_list_size=120 `                |
-    |                            | `hls_flags=delete_segments `        |
-    |                            | `hls_segment_type=fmp4`             |
-    +----------------------------+-------------------------------------+
-    | Video bitrate              | `420 Kbps` or lower, or whatever    |
-    |                            | you want                            |
-    +----------------------------+-------------------------------------+
-    | Keyframe interval (frames) | `framerate` * `hls_time`, e.g. for  |
-    |                            | 60fps and an `hls_time` of 2        |
-    |                            | seconds, set this to 120            |
-    +----------------------------+-------------------------------------+
-    | Video Encoder              | libx264, or an H.264 hardware       |
-    |                            | encoder (e.g. `h264_nvenc` for      |
-    |                            | Nvidia, [see here][ffmpeg])         |
-    +----------------------------+-------------------------------------+
-    | Audio Bitrate              | `96 Kbps`, or whatever you want     |
-    +----------------------------+-------------------------------------+
-    | Audio Encoder              | `aac`                               |
-    +----------------------------+-------------------------------------+
-    ```
-
-> *If this table looks garbled, read this file as plaintext or [click
-> here][plaintext] and scroll to the bottom.*
-
-To start streaming click `Start Recording`.
-
-When it is recording, segments older than four minutes will be regularly
-deleted, and when it stops recording the last four minutes worth of
-segments will remain the segments directory.  (You can change the number
-of kept segments by modifying the `hls_list_size` option in the muxer
-settings.)  When it is not recording, you can delete the files in the
-segments directory without consequence.  Old segments will never be sent
-over the network even if they are not deleted.
-
-[tor]: https://gitlab.torproject.org/tpo/core/tor
-[torrc]: https://support.torproject.org/#tbb-editing-torrc
-[ffmpeg]: https://trac.ffmpeg.org/wiki/HWAccelIntro
-[plaintext]: https://git.076.ne.jp/ninya9k/anonstream/raw/branch/master/STREAMING.md

anonstream/__init__.py

@@ -1,18 +1,19 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
+import asyncio
 import json
 from collections import OrderedDict
 
 from quart_compress import Compress
 
 from anonstream.config import update_flask_from_toml
+from anonstream.emote import load_emote_schema
 from anonstream.quart import Quart
 from anonstream.utils.captcha import create_captcha_factory, create_captcha_signer
-from anonstream.utils.chat import precompute_emote_regex
 from anonstream.utils.user import generate_blank_allowedness
 
-__version__ = '1.6.1'
+__version__ = '1.6.9'
 
 def create_app(toml_config):
     app = Quart('anonstream', static_folder=None)
@@ -48,10 +49,10 @@ def create_app(toml_config):
     app.allowedness = generate_blank_allowedness()
 
     # Read emote schema
-    with open(app.config['EMOTE_SCHEMA']) as fp:
-        emotes = json.load(fp)
-        precompute_emote_regex(emotes)
-        app.emotes = emotes
+    try:
+        app.emotes = load_emote_schema(app.config['EMOTE_SCHEMA'])
+    except (OSError, json.JSONDecodeError) as e:
+        raise AssertionError(f'couldn\'t load emote schema: {e!r}') from e
 
     # State for tasks
     app.users_update_buffer = set()
@@ -60,20 +61,26 @@ def create_app(toml_config):
     app.stream_viewership = None
     app.last_info_task = None
 
-    # Background tasks' asyncio.sleep tasks, cancelled on shutdown
-    app.background_sleep = set()
+    # asyncio tasks to be cancelled on shutdown
+    app.tasks = set()
 
     # Queues for event socket clients
     app.event_queues = set()
 
     @app.after_serving
     async def shutdown():
-        # Force all background tasks to finish
-        for task in app.background_sleep:
+        # Cancel started asyncio tasks that would otherwise block shutdown
+        # The asyncio tasks we create are:
+        #   * quart background tasks awaiting asyncio.sleep()
+        for task in app.tasks:
             task.cancel()
 
     @app.before_serving
     async def startup():
+        # Create routes and background tasks
+        import anonstream.routes
+        import anonstream.tasks
+
         # Start control server
         if app.config['SOCKET_CONTROL_ENABLED']:
             from anonstream.control.server import start_control_server_at
@@ -92,8 +99,4 @@ def create_app(toml_config):
                 )
             app.add_background_task(start_event_server)
 
-        # Create routes and background tasks
-        import anonstream.routes
-        import anonstream.tasks
-
     return app

anonstream/__main__.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import argparse

anonstream/access.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import time

anonstream/broadcast.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 from quart import current_app

anonstream/captcha.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import secrets

anonstream/chat.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import time
@@ -8,7 +8,8 @@ from quart import current_app, escape
 
 from anonstream.broadcast import broadcast, broadcast_users_update
 from anonstream.events import notify_event_sockets
-from anonstream.helpers.chat import generate_nonce_hash, get_scrollback, insert_emotes
+from anonstream.helpers.chat import generate_nonce_hash, get_scrollback
+from anonstream.helpers.emote import insert_emotes
 from anonstream.utils.chat import get_message_for_websocket, get_approx_linespan
 
 CONFIG = current_app.config
@@ -30,9 +31,9 @@ def get_all_messages_for_websocket():
     ))
 
 def add_chat_message(user, nonce, comment, ignore_empty=False):
-    # Special case: if the comment is empty, do nothing and return
+    # Special case: if the comment is empty, do nothing and return None
     if ignore_empty and len(comment) == 0:
-        return False
+        return None
 
     timestamp_ms = time.time_ns() // 1_000_000
     timestamp = timestamp_ms // 1000
@@ -136,7 +137,7 @@ def add_chat_message(user, nonce, comment, ignore_empty=False):
         },
     )
 
-    return True
+    return seq
 
 def delete_chat_messages(seqs):
     seq_set = set(seqs)

anonstream/config.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import os

anonstream/control/exceptions.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 class ControlSocketExit(Exception):

anonstream/control/parse.py

@@ -1,22 +1,28 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 from anonstream.control.spec import ParseException, Parsed
 from anonstream.control.spec.common import Str
 from anonstream.control.spec.methods.allowedness import SPEC as SPEC_ALLOWEDNESS
 from anonstream.control.spec.methods.chat import SPEC as SPEC_CHAT
-from anonstream.control.spec.methods.exit import SPEC as SPEC_EXIT
+from anonstream.control.spec.methods.config import SPEC as SPEC_CONFIG
+from anonstream.control.spec.methods.emote import SPEC as SPEC_EMOTE
 from anonstream.control.spec.methods.help import SPEC as SPEC_HELP
+from anonstream.control.spec.methods.quit import SPEC as SPEC_QUIT
 from anonstream.control.spec.methods.title import SPEC as SPEC_TITLE
+from anonstream.control.spec.methods.tripcode import SPEC as SPEC_TRIPCODE
 from anonstream.control.spec.methods.user import SPEC as SPEC_USER
 
 SPEC = Str({
     'help': SPEC_HELP,
-    'exit': SPEC_EXIT,
+    'quit': SPEC_QUIT,
     'title': SPEC_TITLE,
     'chat': SPEC_CHAT,
     'user': SPEC_USER,
     'allowednesss': SPEC_ALLOWEDNESS,
+    'emote': SPEC_EMOTE,
+    'config': SPEC_CONFIG,
+    'tripcode': SPEC_TRIPCODE,
 })
 
 async def parse(request):

anonstream/control/server.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import asyncio

anonstream/control/spec/__init__.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 class ParseException(Exception):

anonstream/control/spec/common.py

@@ -1,11 +1,16 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import json
 
-from anonstream.control.spec import Spec, NoParse, Ambiguous, Parsed
+from quart import current_app
+
+from anonstream.control.spec import Spec, NoParse, Ambiguous, BadArgument, Parsed
 from anonstream.control.spec.utils import get_item, startswith
 
+USERS_BY_TOKEN = current_app.users_by_token
+USERS = current_app.users
+
 class Str(Spec):
     AS_ARG = False
 
@@ -146,3 +151,26 @@ class ArgsJsonStringArray(ArgsJson):
                 f'bad argument at position {index} {obj_json!r}: '
                 f'could not decode json array of strings'
             )
+
+class ArgsJsonTokenUser(ArgsJsonString):
+    def transform_obj(self, token):
+        try:
+            user = USERS_BY_TOKEN[token]
+        except KeyError:
+            raise BadArgument(f'no user with token {token!r}')
+        return user
+
+class ArgsJsonHashUser(ArgsString):
+    def transform_string(self, token_hash):
+        for user in USERS:
+            if user['token_hash'] == token_hash:
+                break
+        else:
+            raise BadArgument(f'no user with token_hash {token_hash!r}')
+        return user
+
+def ArgsUser(spec):
+    return Str({
+        'token': ArgsJsonTokenUser(spec),
+        'hash': ArgsJsonHashUser(spec),
+    })

anonstream/control/spec/methods/allowedness.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import json

anonstream/control/spec/methods/chat.py

@@ -1,12 +1,19 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import itertools
+import json
+
+from quart import current_app
 
 from anonstream.chat import delete_chat_messages
+from anonstream.control.exceptions import CommandFailed
 from anonstream.control.spec import NoParse
-from anonstream.control.spec.common import Str, End, Args
+from anonstream.control.spec.common import Str, End, Args, ArgsJsonString, ArgsUser
 from anonstream.control.spec.utils import get_item, json_dumps_contiguous
+from anonstream.chat import add_chat_message, Rejected
+
+MESSAGES = current_app.messages
 
 class ArgsSeqs(Args):
     def consume(self, words, index):
@@ -31,25 +38,52 @@ class ArgsSeqs(Args):
 async def cmd_chat_help():
     normal = ['chat', 'help']
     response = (
-        'Usage: chat delete SEQS\n'
+        'Usage: chat {show | delete SEQS | add USER NONCE COMMENT}\n'
         'Commands:\n'
-        #' chat show [MESSAGES]......show chat messages\n'
-        ' chat delete SEQS..........delete chat messages\n'
+        ' chat show......................show all chat messages\n'
+        ' chat delete SEQS...............delete chat messages\n'
+        ' chat add USER NONCE COMMENT....add chat message\n'
         'Definitions:\n'
-        #' MESSAGES..................undefined\n'
-        ' SEQS......................=SEQ [SEQ...]\n'
-        ' SEQ.......................a chat message\'s seq, base-10 integer\n'
+        ' SEQS.......................=SEQ [SEQ...]\n'
+        ' SEQ........................a chat message\'s seq, base-10 integer\n'
+        ' USER.......................={token TOKEN | hash HASH}\n'
+        ' TOKEN......................a user\'s token, json string\n'
+        ' HASH.......................a user\'s token hash\n'
+        ' NONCE......................a chat message\'s nonce, json string\n'
+        ' COMMENT....................json string\n'
     )
     return normal, response
 
+async def cmd_chat_show():
+    normal = ['chat', 'show']
+    response = json.dumps(tuple(MESSAGES), separators=(',', ':')) + '\n'
+    return normal, response
+
 async def cmd_chat_delete(*seqs):
     delete_chat_messages(seqs)
     normal = ['chat', 'delete', *map(str, seqs)]
     response = ''
     return normal, response
 
+async def cmd_chat_add(user, nonce, comment):
+    try:
+        seq = add_chat_message(user, nonce, comment)
+    except Rejected as e:
+        raise CommandFailed(f'rejected: {e}') from e
+    else:
+        assert seq is not None
+    normal = [
+        'chat', 'add',
+        'token', json_dumps_contiguous(user['token']),
+        json_dumps_contiguous(nonce), json_dumps_contiguous(comment),
+    ]
+    response = str(seq) + '\n'
+    return normal, response
+
 SPEC = Str({
     None: End(cmd_chat_help),
     'help': End(cmd_chat_help),
+    'show': End(cmd_chat_show),
     'delete': ArgsSeqs(End(cmd_chat_delete)),
+    'add': ArgsUser(ArgsJsonString(ArgsJsonString(End(cmd_chat_add)))),
 })

anonstream/control/spec/methods/config.py

@@ -0,0 +1,43 @@
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
+# SPDX-License-Identifier: AGPL-3.0-or-later
+
+import json
+
+from quart import current_app
+
+from anonstream.control.exceptions import CommandFailed
+from anonstream.control.spec.common import Str, End, ArgsString
+
+CONFIG = current_app.config
+
+async def cmd_config_help():
+    normal = ['config', 'help']
+    response = (
+        'Usage: config show OPTION\n'
+        'Commands:\n'
+        ' config show OPTION....show entry in app.config\n'
+        'Definitions:\n'
+        ' OPTION................app.config key, re:[A-Z0-9_]+\n'
+    )
+    return normal, response
+
+async def cmd_config_show(option):
+    if option in {'SECRET_KEY', 'SECRET_KEY_STRING'}:
+        raise CommandFailed('not going to show our secret key')
+    try:
+        value = CONFIG[option]
+    except KeyError:
+        raise CommandFailed(f'no config option with key {option!r}')
+    try:
+        value_json = json.dumps(value)
+    except (TypeError, ValueError):
+        raise CommandFailed(f'value is not json serializable')
+    normal = ['config', 'show']
+    response = value_json + '\n'
+    return normal, response
+
+SPEC = Str({
+    None: End(cmd_config_help),
+    'help': End(cmd_config_help),
+    'show': ArgsString(End(cmd_config_show)),
+})

anonstream/control/spec/methods/emote.py

@@ -0,0 +1,61 @@
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
+# SPDX-License-Identifier: AGPL-3.0-or-later
+
+import json
+
+from quart import current_app
+
+from anonstream.emote import load_emote_schema_async, BadEmote
+from anonstream.helpers.emote import get_emote_markup
+from anonstream.control.spec.common import Str, End
+from anonstream.control.exceptions import CommandFailed
+
+CONFIG = current_app.config
+EMOTES = current_app.emotes
+
+async def cmd_emote_help():
+    normal = ['emote', 'help']
+    response = (
+        'Usage: emote {show | reload}\n'
+        'Commands:\n'
+        ' emote show........show all current emotes\n'
+        ' emote reload......try to reload the emote schema (existing messages are not modified)\n'
+    )
+    return normal, response
+
+async def cmd_emote_show():
+    emotes_for_json = [emote.copy() for emote in EMOTES]
+    for emote in emotes_for_json:
+        emote['regex'] = emote['regex'].pattern
+    normal = ['emote', 'show']
+    response = json.dumps(emotes_for_json) + '\n'
+    return normal, response
+
+async def cmd_emote_reload():
+    try:
+        emotes = await load_emote_schema_async(CONFIG['EMOTE_SCHEMA'])
+    except OSError as e:
+        raise CommandFailed(f'could not read emote schema: {e}') from e
+    except json.JSONDecodeError as e:
+        raise CommandFailed('could not decode emote schema as json') from e
+    except BadEmote as e:
+        error, *_ = e.args
+        raise CommandFailed(error) from e
+    else:
+        # Mutate current_app.emotes in place
+        EMOTES.clear()
+        for emote in emotes:
+            EMOTES.append(emote)
+        # Clear emote markup cache -- emotes by the same name may have changed
+        get_emote_markup.cache_clear()
+    normal = ['emote', 'reload']
+    response = ''
+    return normal, response
+
+SPEC = Str({
+    None: End(cmd_emote_help),
+    'help': End(cmd_emote_help),
+    'show': End(cmd_emote_show),
+    'reload': End(cmd_emote_reload),
+})
+

anonstream/control/spec/methods/exit.py

@@ -1,21 +0,0 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
-# SPDX-License-Identifier: AGPL-3.0-or-later
-
-from anonstream.control.spec.common import Str, End
-from anonstream.control.exceptions import ControlSocketExit
-
-async def cmd_exit():
-    raise ControlSocketExit
-
-async def cmd_exit_help():
-    normal = ['exit', 'help']
-    response = (
-        'Usage: exit\n'
-        'close the connection\n'
-    )
-    return normal, response
-
-SPEC = Str({
-    None: End(cmd_exit),
-    'help': End(cmd_exit_help),
-})

anonstream/control/spec/methods/help.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 from anonstream.control.spec.common import Str, End
@@ -9,7 +9,7 @@ async def cmd_help():
         'Usage: METHOD [COMMAND | help]\n'
         'Examples:\n'
         ' help...........................show this help message\n'
-        ' exit...........................close the control connection\n'
+        ' quit...........................close the control connection\n'
         ' title [show]...................show the stream title\n'
         ' title set TITLE................set the stream title\n'
         ' user [show]....................show a list of users\n'
@@ -18,12 +18,18 @@ async def cmd_help():
         ' user set USER ATTR VALUE.......set an attribute of a user\n'
         ' user eyes USER [show]..........show a list of active video responses\n'
         ' user eyes USER delete EYES_ID..end a video response\n'
-        #' chat show MESSAGES.............show a list of messages\n'
-        ' chat delete SEQS...............delete a set of messages\n'
+        ' user add VERIFIED TOKEN........add new user\n'
+        ' chat show......................show a list of all chat messages\n'
+        ' chat delete SEQS...............delete a set of chat messages\n'
+        ' chat add USER NONCE COMMENT....add a chat message\n'
         ' allowedness [show].............show the current allowedness\n'
         ' allowedness setdefault BOOLEAN.set the default allowedness\n'
         ' allowedness add SET STRING.....add to the blacklist/whitelist\n'
         ' allowedness remove SET STRING..remove from the blacklist/whitelist\n'
+        ' emote show.....................show all current emotes\n'
+        ' emote reload...................try reloading the emote schema\n'
+        ' config show OPTION.............show app config option\n'
+        ' tripcode generate PASSWORD.....show tripcode for given password\n'
     )
     return normal, response
 

anonstream/control/spec/methods/quit.py

@@ -0,0 +1,22 @@
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
+# SPDX-License-Identifier: AGPL-3.0-or-later
+
+from anonstream.control.spec.common import Str, End
+from anonstream.control.exceptions import ControlSocketExit
+
+async def cmd_quit():
+    raise ControlSocketExit
+
+async def cmd_quit_help():
+    normal = ['quit', 'help']
+    response = (
+        'Usage: quit\n'
+        'Commands:\n'
+        ' quit......close the connection\n'
+    )
+    return normal, response
+
+SPEC = Str({
+    None: End(cmd_quit),
+    'help': End(cmd_quit_help),
+})

anonstream/control/spec/methods/title.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import json
@@ -12,7 +12,7 @@ from anonstream.stream import get_stream_title, set_stream_title
 async def cmd_title_help():
     normal = ['title', 'help']
     response = (
-        'Usage: title [show | set TITLE]\n'
+        'Usage: title {show | set TITLE}\n'
         'Commands:\n'
         ' title [show].......show the stream title\n'
         ' title set TITLE....set the stream title to TITLE\n'

anonstream/control/spec/methods/tripcode.py

@@ -0,0 +1,42 @@
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
+# SPDX-License-Identifier: AGPL-3.0-or-later
+
+import json
+
+from quart import current_app
+
+from anonstream.control.exceptions import CommandFailed
+from anonstream.control.spec.common import Str, End, ArgsJsonString
+from anonstream.control.spec.utils import json_dumps_contiguous
+from anonstream.helpers.tripcode import generate_tripcode
+
+CONFIG = current_app.config
+
+async def cmd_tripcode_help():
+    normal = ['tripcode', 'help']
+    response = (
+        'Usage: tripcode generate PASSWORD\n'
+        'Commands:\n'
+        ' tripcode generate PASSWORD....show tripcode for given password\n'
+        'Definitions:\n'
+        ' PASSWORD................json string, max length in config.toml (`chat.max_tripcode_password_length`)\n'
+    )
+    return normal, response
+
+async def cmd_tripcode_generate(password):
+    if len(password) > CONFIG['CHAT_TRIPCODE_PASSWORD_MAX_LENGTH']:
+        raise CommandFailed(
+            f'password exceeded maximum configured length of '
+            f'{CONFIG["CHAT_TRIPCODE_PASSWORD_MAX_LENGTH"]} '
+            f'characters'
+        )
+    tripcode = generate_tripcode(password)
+    normal = ['tripcode', 'generate', json_dumps_contiguous(password)]
+    response = json.dumps(tripcode) + '\n'
+    return normal, response
+
+SPEC = Str({
+    None: End(cmd_tripcode_help),
+    'help': End(cmd_tripcode_help),
+    'generate': ArgsJsonString(End(cmd_tripcode_generate)),
+})

anonstream/control/spec/methods/user.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import json
@@ -6,38 +6,15 @@ import json
 from quart import current_app
 
 from anonstream.control.exceptions import CommandFailed
-from anonstream.control.spec import BadArgument
-from anonstream.control.spec.common import Str, End, ArgsInt, ArgsString, ArgsJson, ArgsJsonString
-from anonstream.control.spec.utils import get_item, json_dumps_contiguous
+from anonstream.control.spec.common import Str, End, ArgsInt, ArgsString, ArgsJson, ArgsJsonBoolean, ArgsJsonString, ArgsUser
+from anonstream.control.spec.utils import json_dumps_contiguous
 from anonstream.utils.user import USER_WEBSOCKET_ATTRS
+from anonstream.routes.wrappers import generate_and_add_user
+from anonstream.wrappers import get_timestamp
 
 USERS_BY_TOKEN = current_app.users_by_token
-USERS = current_app.users
 USERS_UPDATE_BUFFER = current_app.users_update_buffer
 
-class ArgsJsonTokenUser(ArgsJsonString):
-    def transform_obj(self, token):
-        try:
-            user = USERS_BY_TOKEN[token]
-        except KeyError:
-            raise BadArgument(f'no user with token {token!r}')
-        return user
-
-class ArgsJsonHashUser(ArgsString):
-    def transform_string(self, token_hash):
-        for user in USERS:
-            if user['token_hash'] == token_hash:
-                break
-        else:
-            raise BadArgument(f'no user with token_hash {token_hash!r}')
-        return user
-
-def ArgsUser(spec):
-    return Str({
-        'token': ArgsJsonTokenUser(spec),
-        'hash': ArgsJsonHashUser(spec),
-    })
-
 async def cmd_user_help():
     normal = ['user', 'help']
     response = (
@@ -49,6 +26,7 @@ async def cmd_user_help():
             ' user set USER ATTR VALUE......set an attribute of a user\n'
             ' user eyes USER [show].........show a user\'s active video responses\n'
             ' user eyes USER delete EYES_ID.end a video response to a user\n'
+            ' user add VERIFIED TOKEN.......add new user\n'
             'Definitions:\n'
             ' USER..........................={token TOKEN | hash HASH}\n'
             ' TOKEN.........................a token, json string\n'
@@ -56,6 +34,7 @@ async def cmd_user_help():
             ' ATTR..........................a user attribute, re:[a-z0-9_]+\n'
             ' VALUE.........................json value\n'
             ' EYES_ID.......................a user\'s eyes_id, base 10 integer\n'
+            ' VERIFIED......................user\'s verified state: true = normal, false = can\'t chat, null = will be kicked to access captcha\n'
     )
     return normal, response
 
@@ -79,10 +58,8 @@ async def cmd_user_get(user, attr):
     except (TypeError, ValueError) as e:
         raise CommandFailed('value is not representable in json') from e
     normal = [
-        'user',
-        'get',
-        'token',
-        json_dumps_contiguous(user['token']),
+        'user', 'get',
+        'token', json_dumps_contiguous(user['token']),
         attr,
     ]
     response = value_json + '\n'
@@ -95,10 +72,8 @@ async def cmd_user_set(user, attr, value):
     if attr in USER_WEBSOCKET_ATTRS:
         USERS_UPDATE_BUFFER.add(user['token'])
     normal = [
-        'user',
-        'set',
-        'token',
-        json_dumps_contiguous(user['token']),
+        'user', 'set',
+        'token', json_dumps_contiguous(user['token']),
         attr,
         json_dumps_contiguous(value),
     ]
@@ -107,11 +82,9 @@ async def cmd_user_set(user, attr, value):
 
 async def cmd_user_eyes_show(user):
     normal = [
-        'user',
-        'eyes',
-        'token',
-        json_dumps_contiguous(user['token']),
-        'show'
+        'user', 'eyes',
+        'token', json_dumps_contiguous(user['token']),
+        'show',
     ]
     response = json.dumps(user['eyes']['current']) + '\n'
     return normal, response
@@ -122,12 +95,24 @@ async def cmd_user_eyes_delete(user, eyes_id):
     except KeyError:
         pass
     normal = [
-        'user',
-        'eyes',
-        'token',
-        json_dumps_contiguous(user['token']),
-        'delete',
-        str(eyes_id),
+        'user', 'eyes',
+        'token', json_dumps_contiguous(user['token']),
+        'delete', str(eyes_id),
+    ]
+    response = ''
+    return normal, response
+
+async def cmd_user_add(verified, token):
+    if token in USERS_BY_TOKEN:
+        raise CommandFailed(f'user with token {token!r} already exists')
+    _user = generate_and_add_user(
+        timestamp=get_timestamp(),
+        token=token,
+        verified=verified,
+    )
+    normal = [
+        'user', 'add',
+        json_dumps_contiguous(verified), json_dumps_contiguous(token),
     ]
     response = ''
     return normal, response
@@ -144,4 +129,5 @@ SPEC = Str({
         'show': End(cmd_user_eyes_show),
         'delete': ArgsInt(End(cmd_user_eyes_delete)),
     })),
+    'add': ArgsJsonBoolean(ArgsJsonString(End(cmd_user_add))),
 })

anonstream/control/spec/utils.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import json

anonstream/emote.py

@@ -0,0 +1,55 @@
+import json
+import re
+
+import aiofiles
+from quart import escape
+
+class BadEmote(Exception):
+    pass
+
+class BadEmoteName(BadEmote):
+    pass
+
+def _load_emote_schema(emotes):
+    for key in ('name', 'file', 'width', 'height'):
+        for emote in emotes:
+            if key not in emote:
+                raise BadEmote(f'emotes must have a `{key}`: {emote}')
+    precompute_emote_regex(emotes)
+    return emotes
+
+def load_emote_schema(filepath):
+    with open(filepath) as fp:
+        emotes = json.load(fp)
+    return _load_emote_schema(emotes)
+
+async def load_emote_schema_async(filepath):
+    async with aiofiles.open(filepath) as fp:
+        data = await fp.read(8192)
+    return _load_emote_schema(json.loads(data))
+
+def precompute_emote_regex(schema):
+    for emote in schema:
+        if not emote['name']:
+            raise BadEmoteName(f'emote names cannot be empty: {emote}')
+        if re.search(r'\s', emote['name']):
+            raise BadEmoteName(
+                f'whitespace is not allowed in emote names: {emote["name"]!r}'
+            )
+        for length in (emote['width'], emote['height']):
+            if length is not None and (not isinstance(length, int) or length < 0):
+                raise BadEmoteName(
+                    f'emote dimensions must be null or non-negative integers: '
+                    f'{emote}'
+                )
+        # If the emote name begins with a word character [a-zA-Z0-9_],
+        # match only if preceded by a non-word character or the empty
+        # string.  Similarly for the end of the emote name.
+        # Examples:
+        #  * ":joy:" matches "abc :joy:~xyz"   and   "abc:joy:xyz"
+        #  * "JoySi" matches "abc JoySi~xyz" but NOT "abcJoySiabc"
+        onset = r'(?:^|(?<=\W))' if re.fullmatch(r'\w', emote['name'][0]) else r''
+        finish = r'(?:$|(?=\W))' if re.fullmatch(r'\w', emote['name'][-1]) else r''
+        emote['regex'] = re.compile(''.join(
+            (onset, re.escape(escape(emote['name'])), finish)
+        ))

anonstream/events.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import asyncio
@@ -6,8 +6,8 @@ import json
 
 from quart import current_app
 
-async def start_event_server_at(address):
-    return await asyncio.start_unix_server(serve_event_client, address)
+def start_event_server_at(address):
+    return asyncio.start_unix_server(serve_event_client, address)
 
 async def serve_event_client(reader, writer):
     reader.feed_eof()

anonstream/helpers/captcha.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import base64

anonstream/helpers/chat.py

@@ -1,14 +1,11 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import hashlib
-from functools import lru_cache
 
-import markupsafe
-from quart import current_app, escape, url_for, Markup
+from quart import current_app
 
 CONFIG = current_app.config
-EMOTES = current_app.emotes
 
 def generate_nonce_hash(nonce):
     parts = CONFIG['SECRET_KEY'] + b'nonce-hash\0' + nonce.encode()
@@ -19,22 +16,3 @@ def get_scrollback(messages):
     if len(messages) < n:
         return messages
     return list(messages)[-n:]
-
-@lru_cache
-def get_emote_markup(emote_name, emote_file, emote_width, emote_height):
-    emote_name_markup = escape(emote_name)
-    return Markup(
-        f'''<img class="emote" '''
-        f'''src="{url_for('static', filename=emote_file)}" '''
-        f'''width="{escape(emote_width)}" height="{escape(emote_height)}" '''
-        f'''alt="{emote_name_markup}" title="{emote_name_markup}">'''
-    )
-
-def insert_emotes(markup):
-    assert isinstance(markup, markupsafe.Markup)
-    for emote in EMOTES:
-        emote_markup = get_emote_markup(
-            emote['name'], emote['file'], emote['width'], emote['height'],
-        )
-        markup = emote['regex'].sub(emote_markup, markup)
-    return Markup(markup)

anonstream/helpers/emote.py

@@ -0,0 +1,27 @@
+import markupsafe
+from functools import lru_cache
+
+from quart import current_app, escape, url_for, Markup
+
+EMOTES = current_app.emotes
+
+@lru_cache
+def get_emote_markup(emote_name, emote_file, emote_width, emote_height):
+    emote_name_markup = escape(emote_name)
+    width = '' if emote_width is None else f'width="{escape(emote_width)}" '
+    height = '' if emote_height is None else f'height="{escape(emote_height)}" '
+    return Markup(
+        f'''<img class="emote" '''
+        f'''src="{escape(url_for('static', filename=emote_file))}" '''
+        f'''{width}{height}'''
+        f'''alt="{emote_name_markup}" title="{emote_name_markup}">'''
+    )
+
+def insert_emotes(markup):
+    assert isinstance(markup, markupsafe.Markup)
+    for emote in EMOTES:
+        emote_markup = get_emote_markup(
+            emote['name'], emote['file'], emote['width'], emote['height'],
+        )
+        markup = emote['regex'].sub(emote_markup, markup)
+    return Markup(markup)

anonstream/helpers/tripcode.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import base64

anonstream/helpers/user.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import hashlib

anonstream/quart.py

@@ -1,7 +1,7 @@
 # This file is pretty much entirely based on a snippet from asgi.py in
 # the Quart repository (MIT, see README.md). That means it takes on the
 # MIT licence I guess(???) If not then it's the same as every other file
-# by me: 2022 n9k <https://git.076.ne.jp/ninya9k>, AGPL 3.0 or any later
+# by me: 2022 n9k <https://gitler.moe/ninya9k>, AGPL 3.0 or any later
 # version.
 
 import asyncio

anonstream/routes/__init__.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import anonstream.routes.error

anonstream/routes/core.py

@@ -1,8 +1,9 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import math
 import re
+from urllib.parse import quote
 
 from quart import current_app, request, render_template, abort, make_response, redirect, url_for, send_from_directory
 from werkzeug.exceptions import Forbidden, NotFound, TooManyRequests
@@ -22,7 +23,12 @@ CAPTCHA_SIGNER = current_app.captcha_signer
 STATIC_DIRECTORY = current_app.root_path / 'static'
 
 @current_app.route('/')
-@with_user_from(request, fallback_to_token=True, ignore_allowedness=True)
+@with_user_from(
+    request,
+    fallback_to_token=True,
+    ignore_allowedness=True,
+    redundant_token_redirect=True,
+)
 async def home(timestamp, user_or_token):
     match user_or_token:
         case str() | None as token:
@@ -128,12 +134,13 @@ async def access(timestamp, user_or_token):
                     failure_id = None
                     user = generate_and_add_user(timestamp, token, verified=True)
             if failure_id is not None:
-                url = url_for('home', token=token, failure=failure_id)
-                raise abort(redirect(url, 303))
+                response = redirect(url_for('home', token=token, failure=failure_id), 303)
+            else:
+                response = redirect(url_for('home', token=user['token']), 303)
+                response.headers['Set-Cookie'] = f'token={quote(user["token"])}; path=/'
         case dict() as user:
-            pass
-    url = url_for('home', token=user['token'])
-    return redirect(url, 303)
+            response = redirect(url_for('home', token=user['token']), 303)
+    return response
 
 @current_app.route('/static/<path:filename>')
 @with_user_from(request)

anonstream/routes/nojs.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 from quart import current_app, request, render_template, redirect, url_for, escape, Markup
@@ -10,11 +10,12 @@ from anonstream.user import add_state, pop_state, try_change_appearance, update_
 from anonstream.routes.wrappers import with_user_from, render_template_with_etag
 from anonstream.helpers.chat import get_scrollback
 from anonstream.helpers.user import get_default_name
-from anonstream.utils.chat import generate_nonce
+from anonstream.utils.chat import generate_nonce, should_show_initial_date
 from anonstream.utils.security import generate_csp
 from anonstream.utils.user import concatenate_for_notice
 
 CONFIG = current_app.config
+MESSAGES = current_app.messages
 USERS_BY_TOKEN = current_app.users_by_token
 
 @current_app.route('/stream.html')
@@ -32,9 +33,9 @@ async def nojs_stream(timestamp, user):
 async def nojs_info(timestamp, user):
     update_presence(user)
     uptime, viewership = get_stream_uptime_and_viewership()
-    return await render_template(
+    return await render_template_with_etag(
         'nojs_info.html',
-        csp=generate_csp(),
+        {'csp': generate_csp()},
         refresh=CONFIG['NOJS_REFRESH_INFO'],
         user=user,
         viewership=viewership,
@@ -47,15 +48,17 @@ async def nojs_info(timestamp, user):
 @with_user_from(request)
 async def nojs_chat_messages(timestamp, user):
     reading(user)
+    messages = get_scrollback(MESSAGES)
     return await render_template_with_etag(
         'nojs_chat_messages.html',
         {'csp': generate_csp()},
         refresh=CONFIG['NOJS_REFRESH_MESSAGES'],
         user=user,
         users_by_token=USERS_BY_TOKEN,
-        messages=get_scrollback(current_app.messages),
+        messages=messages,
         timeout=CONFIG['NOJS_TIMEOUT_CHAT'],
         get_default_name=get_default_name,
+        show_initial_date=should_show_initial_date(timestamp, messages),
     )
 
 @current_app.route('/chat/messages')
@@ -135,12 +138,13 @@ async def nojs_submit_message(timestamp, user):
         try:
             # If the comment is empty but the captcha was just solved,
             # be lenient: don't raise an exception and don't create a notice
-            message_was_added = add_chat_message(
+            seq = add_chat_message(
                 user,
                 nonce,
                 comment,
                 ignore_empty=verification_happened,
             )
+            message_was_added = seq is not None
         except Rejected as e:
             notice, *_ = e.args
             state_id = add_state(

anonstream/routes/websocket.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import asyncio

anonstream/routes/wrappers.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import hashlib
@@ -8,7 +8,7 @@ import string
 from functools import wraps
 from urllib.parse import quote, unquote
 
-from quart import current_app, request, make_response, render_template, request, url_for, Markup
+from quart import current_app, request, make_response, render_template, redirect, url_for, Markup
 from werkzeug.exceptions import BadRequest, Unauthorized, Forbidden
 from werkzeug.security import check_password_hash
 
@@ -87,7 +87,12 @@ def generate_and_add_user(
     USERS_UPDATE_BUFFER.add(token)
     return user
 
-def with_user_from(context, fallback_to_token=False, ignore_allowedness=False):
+def with_user_from(
+    context,
+    fallback_to_token=False,
+    ignore_allowedness=False,
+    redundant_token_redirect=False,
+):
     def with_user_from_context(f):
         @wraps(f)
         async def wrapper(*args, **kwargs):
@@ -129,6 +134,18 @@ def with_user_from(context, fallback_to_token=False, ignore_allowedness=False):
                         f"terminal when they started anonstream."
                     ))
 
+            # If token from the client's cookie is same as the token in the URL
+            # query string, the client supports cookies.  If we want, we can
+            # redirect the client to this same URL path but with the token
+            # parameter removed, since we'll pick up their token from their
+            # cookie anyway.
+            if (
+                redundant_token_redirect
+                and token_from_context is not None
+                and token_from_args == token_from_cookie
+            ):
+                return redirect(context.path, 303)
+
             # Create response
             user = USERS_BY_TOKEN.get(token)
             if CONFIG['ACCESS_CAPTCHA'] and not broadcaster:

anonstream/segments.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import asyncio

anonstream/static/anonstream.js

@@ -189,6 +189,7 @@ const create_chat_message = (object) => {
   chat_message.classList.add("chat-message");
   chat_message.dataset.seq = object.seq;
   chat_message.dataset.tokenHash = object.token_hash;
+  chat_message.dataset.date = object.date;
 
   const chat_message_time = document.createElement("time");
   chat_message_time.classList.add("chat-message__time");
@@ -256,47 +257,65 @@ const create_chat_user_components = (user) => {
   result.push(...[chat_user_name, chat_user_tripcode_nbsp, chat_user_tripcode]);
   return result;
 }
+const zeropad = (n) => ("0" + n).slice(-2);
+const datestamp = () => {
+  const date = new Date();
+  return `${date.getUTCFullYear()}-${zeropad(date.getUTCMonth() + 1)}-${zeropad(date.getUTCDate())}`;
+}
 const create_and_add_chat_message = (object) => {
+  // date
+  last_chat_message = chat_messages.querySelector(".chat-message:last-of-type");
+  if (last_chat_message === null || last_chat_message.dataset.date !== object.date) {
+    const chat_date = document.createElement("li");
+    chat_date.classList.add("chat-date");
+    chat_date.dataset.date = object.date;
+
+    const chat_date_hr = document.createElement("hr");
+    const chat_date_div = document.createElement("div");
+
+    const chat_date_div_time = document.createElement("time");
+    chat_date_div_time.datetime = object.date;
+    chat_date_div_time.innerText = object.date;
+
+    chat_date_div.insertAdjacentElement("beforeend", chat_date_div_time);
+    chat_date.insertAdjacentElement("beforeend", chat_date_hr);
+    chat_date.insertAdjacentElement("beforeend", chat_date_div);
+    if (last_chat_message === null && object.date === datestamp())
+      chat_date.dataset.hidden = "";
+    chat_messages.insertAdjacentElement("beforeend", chat_date);
+  }
+
+  // message
   const chat_message = create_chat_message(object);
   chat_messages.insertAdjacentElement("beforeend", chat_message);
-  while (chat_messages.children.length > max_chat_scrollback) {
-    chat_messages.children[0].remove();
+  const first_chat_message = chat_messages.querySelector(".chat-message");
+  if (first_chat_message !== null) {
+    const first_chat_date = chat_messages.querySelector(".chat-date");
+    if (first_chat_date !== null && first_chat_date.hasAttribute("data-hidden") && (object.date !== first_chat_message.dataset.date || object.date !== datestamp()))
+      first_chat_date.removeAttribute("data-hidden");
   }
+  const string_seqs = new Set();
+  for (const this_chat_message of chat_messages.querySelectorAll(".chat-message")) {
+    if (chat_messages.querySelectorAll(".chat-message").length - string_seqs.size > max_chat_scrollback)
+      string_seqs.add(this_chat_message.dataset.seq);
+    else
+      break;
+  }
+  delete_chat_messages({string_seqs});
 }
-const delete_chat_messages = (seqs) => {
-  string_seqs = new Set(seqs.map(n => n.toString()));
-  to_delete = [];
-  for (const chat_message of chat_messages.children) {
-    if (string_seqs.has(chat_message.dataset.seq))
-      to_delete.push(chat_message);
+const delete_chat_messages = ({string_seqs, keep=false}) => {
+  const keep_dates = new Set();
+  for (const chat_message of chat_messages.querySelectorAll(".chat-message")) {
+    if (string_seqs.has(chat_message.dataset.seq) === keep)
+      keep_dates.add(chat_message.dataset.date);
   }
-  for (const chat_message of to_delete) {
-    chat_message.remove();
+  const to_delete = [];
+  for (const child of chat_messages.children) {
+    if (child.classList.contains("chat-date") && !keep_dates.has(child.dataset.date) || child.classList.contains("chat-message") && string_seqs.has(child.dataset.seq) !== keep)
+      to_delete.push(child);
   }
-}
-
-const hexdigest = async (string, bytelength) => {
-  uint8array = new TextEncoder().encode(string);
-  arraybuffer = await crypto.subtle.digest("sha-256", uint8array);
-  array = Array.from(new Uint8Array(arraybuffer).slice(0, bytelength));
-  hex = array.map(b => b.toString(16).padStart(2, "0")).join("");
-  return hex
-}
-const escape_css_string = (string) => {
-  /* https://drafts.csswg.org/cssom/#common-serializing-idioms */
-  const result = [];
-  for (const char of string) {
-    if (char === "\0") {
-      result.push("\ufffd");
-    } else if (char < "\u0020" || char == "\u007f") {
-      result.push(`\\${char.charCodeAt().toString(16)}`);
-    } else if (char == '"' || char == "\\") {
-      result.push(`\\${char}`);
-    } else {
-      result.push(char);
-    }
-  }
-  return result.join("");
+  for (const element of to_delete)
+    element.remove();
 }
 
 let users = {};
@@ -364,7 +383,7 @@ const get_user_name = ({user=null, token_hash}) => {
 }
 const update_user_names = (token_hash=null) => {
   const token_hashes = token_hash === null ? Object.keys(users) : [token_hash];
-  for (const chat_message of chat_messages.children) {
+  for (const chat_message of chat_messages.querySelectorAll(".chat-message")) {
     const this_token_hash = chat_message.dataset.tokenHash;
     if (token_hashes.includes(this_token_hash)) {
       const user = users[this_token_hash];
@@ -446,7 +465,7 @@ const update_user_tripcodes = (token_hash=null) => {
   }
 
   // update inner texts
-  for (const chat_message of chat_messages.children) {
+  for (const chat_message of chat_messages.querySelectorAll(".chat-message")) {
     const this_token_hash = chat_message.dataset.tokenHash;
     const tripcode = users[this_token_hash].tripcode;
     if (token_hashes.includes(this_token_hash)) {
@@ -646,6 +665,11 @@ const on_websocket_message = async (event) => {
         info_button.dataset.visible = "";
       }
 
+      // form input maxlengths
+      chat_form_comment.maxLength = receipt.maxlength.comment;
+      chat_appearance_form_name.maxLength = receipt.maxlength.name;
+      chat_appearance_form_password.maxLength = receipt.maxlength.password;
+
       // chat form nonce
       chat_form_nonce.value = receipt.nonce;
 
@@ -656,17 +680,8 @@ const on_websocket_message = async (event) => {
       chat_form_submit.disabled = false;
 
       // remove messages the server isn't acknowledging the existence of
-      const seqs = new Set(receipt.messages.map((message) => {return message.seq;}));
-      const to_delete = [];
-      for (const chat_message of chat_messages.children) {
-        const chat_message_seq = parseInt(chat_message.dataset.seq);
-        if (!seqs.has(chat_message_seq)) {
-          to_delete.push(chat_message);
-        }
-      }
-      for (const chat_message of to_delete) {
-        chat_message.remove();
-      }
+      const string_seqs = new Set(receipt.messages.map(message => message.seq.toString()));
+      delete_chat_messages({string_seqs, keep: true});
 
       // settings
       default_name = receipt.default;
@@ -689,7 +704,7 @@ const on_websocket_message = async (event) => {
           left: 0,
           top: chat_messages.scrollTopMax,
           behavior: "instant",
-		});
+		    });
       }
 
       // appearance form default values
@@ -701,7 +716,8 @@ const on_websocket_message = async (event) => {
       chat_appearance_form_color.setAttribute("value", user.color);
 
       // insert new messages
-      const last = chat_messages.children.length == 0 ? null : chat_messages.children[chat_messages.children.length - 1];
+      const chat_messages_messages = chat_messages.querySelectorAll(".chat-message");
+      const last = chat_messages_messages.length == 0 ? null : chat_messages_messages[chat_messages_messages.length - 1];
       const last_seq = last === null ? null : parseInt(last.dataset.seq);
       for (const message of receipt.messages) {
         if (message.seq > last_seq) {
@@ -768,7 +784,7 @@ const on_websocket_message = async (event) => {
 
     case "delete":
       console.log("ws delete", receipt);
-      delete_chat_messages(receipt.seqs);
+      delete_chat_messages({string_seqs: new Set(receipt.seqs.map(n => n.toString()))});
       break;
 
     case "set-users":
@@ -925,7 +941,7 @@ info_button.addEventListener("click", (event) => {
   info_button.removeAttribute("data-visible");
 });
 video.addEventListener("error", (event) => {
-  if (video.error !== null && video.error.message === "404: Not Found") {
+  if (video.error !== null && video.networkState === video.NETWORK_NO_SOURCE) {
     show_offline_screen();
   }
   if (stats !== null) {
@@ -993,6 +1009,14 @@ chat_messages_unlock.addEventListener("click", (event) => {
   chat_messages.scrollTop = chat_messages.scrollTopMax;
 });
 
+/* show initial chat date if a day has passed */
+const show_initial_date = () => {
+  const chat_date = chat_messages.querySelector(".chat-date:first-child");
+  if (chat_date !== null && chat_date.hasAttribute("data-hidden") && chat_date.dataset.date !== datestamp())
+    chat_date.removeAttribute("data-hidden");
+}
+setInterval(show_initial_date, 30000);
+
 /* close websocket after prolonged absence of pings */
 
 const rotate_websocket = () => {

anonstream/static/style.css

@@ -273,6 +273,29 @@ noscript {
   font-size: 9pt;
   cursor: default;
 }
+.chat-date {
+  text-align: center;
+  position: relative;
+  display: grid;
+  align-items: center;
+  margin: 8px 0;
+  color: #b2b2b3;
+  cursor: default;
+}
+.chat-date[data-hidden] {
+  display: none;
+}
+.chat-date > hr {
+  margin: 0;
+  position: absolute;
+  width: 100%;
+  box-sizing: border-box;
+}
+.chat-date > :not(hr) > time {
+  padding: 0 1ch;
+  background-color: #232327;
+  position: relative;
+}
 #chat__body__users {
   background-color: #121214;
   mask-image: linear-gradient(black calc(100% - 0.625rem), transparent calc(100% - 0.125rem));

anonstream/stream.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import itertools

anonstream/tasks.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import asyncio
@@ -19,14 +19,13 @@ USERS = current_app.users
 CAPTCHAS = current_app.captchas
 CAPTCHA_SIGNER = current_app.captcha_signer
 
-async def sleep_and_collect_task(delay):
-    coro = asyncio.sleep(delay)
+async def cancel_on_shutdown(coro):
     task = asyncio.create_task(coro)
-    current_app.background_sleep.add(task)
+    current_app.tasks.add(task)
     try:
         await task
     finally:
-        current_app.background_sleep.remove(task)
+        current_app.tasks.remove(task)
 
 def with_period(period):
     def periodically(f):
@@ -35,7 +34,7 @@ def with_period(period):
             for iteration in itertools.count():
                 await f(iteration, *args, **kwargs)
                 try:
-                    await sleep_and_collect_task(period)
+                    await cancel_on_shutdown(asyncio.sleep(period))
                 except asyncio.CancelledError:
                     break
 
@@ -64,8 +63,11 @@ async def t_sunset_users(timestamp, iteration):
     if iteration == 0:
         return
 
-    # De-access absent users
-    absent_users = tuple(get_absent_users(timestamp))
+    # Revoke access for absent users (except the broadcaster)
+    absent_users = tuple(filter(
+        lambda user: not user['broadcaster'],
+        get_absent_users(timestamp)
+    ))
     for user in absent_users:
         user['verified'] = None
     # Absent users should have no connected websockets,

anonstream/templates/captcha.html

@@ -1,3 +1,7 @@
+{##
+ # SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+ # SPDX-License-Identifier: AGPL-3.0-or-later
+ ##}
 <!doctype html>
 <html>
 <head>
@@ -11,7 +15,7 @@
       font-family: sans-serif;
       font-size: 14pt;
       display: grid;
-      grid-template-rows: calc(50% - 4rem) 1fr;
+      grid-template-rows: calc(50% - 10vh + 2rem) 1fr;
       height: 100vh;
       margin: 0;
       padding: 1rem;
@@ -33,20 +37,23 @@
       border-radius: 2px;
       color: #ddd;
       font-size: 14pt;
-      padding: 4px 5px;
+      padding: 5px 6px;
       width: 10ch;
-    }
-    input[name="answer"]:hover {
-      background-color: #37373a;
       transition: 0.25s;
-	}
-	input[type="submit"] {
+    }
+    input[name="answer"]:focus {
+      background-color: black;
+      border-color: #3584e4;
+    }
+    input[type="submit"] {
       font-size: 14pt;
-	}
-	p {
+      padding-left: 8px;
+      padding-right: 8px;
+    }
+    p {
       grid-column: 1 / span 2;
       text-align: center;
-	}
+    }
   </style>
 </head>
 <body>

anonstream/templates/error.html

@@ -1,3 +1,7 @@
+{##
+ # SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+ # SPDX-License-Identifier: AGPL-3.0-or-later
+ ##}
 <!doctype html>
 <html>
 <head>

anonstream/templates/home.html

@@ -1,5 +1,5 @@
 {##
- # SPDX-FileCopyrightText: 2022 n9k [https://git.076.ne.jp/ninya9k]
+ # SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
  # SPDX-License-Identifier: AGPL-3.0-or-later
  ##}
 <!doctype html>
@@ -41,7 +41,7 @@
     <a href="#chat">chat</a>
     <a href="#both">both</a>
   </nav>
-  <footer>anonstream {{ version }} &mdash; <a href="https://git.076.ne.jp/ninya9k/anonstream" target="_blank">source</a></footer>
+  <footer>anonstream {{ version }} &mdash; <a href="https://gitler.moe/ninya9k/anonstream" target="_blank">source</a></footer>
   <script src="{{ url_for('static', filename='anonstream.js') }}" type="text/javascript"></script>
 </body>
 </html>

anonstream/templates/macros/user.html

@@ -1,5 +1,5 @@
 {##
- # SPDX-FileCopyrightText: 2022 n9k [https://git.076.ne.jp/ninya9k]
+ # SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
  # SPDX-License-Identifier: AGPL-3.0-or-later
  ##}
 {%

anonstream/templates/nojs_chat_form.html

@@ -1,5 +1,5 @@
 {##
- # SPDX-FileCopyrightText: 2022 n9k [https://git.076.ne.jp/ninya9k]
+ # SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
  # SPDX-License-Identifier: AGPL-3.0-or-later
  ##}
 <!doctype html>
@@ -21,6 +21,7 @@
       height: 100%;
       color: #ddd;
       font-family: sans-serif;
+      background-color: #232327;
     }
     a {
       color: var(--link-color);

anonstream/templates/nojs_chat_messages.html

@@ -1,5 +1,5 @@
 {##
- # SPDX-FileCopyrightText: 2022 n9k [https://git.076.ne.jp/ninya9k]
+ # SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
  # SPDX-License-Identifier: AGPL-3.0-or-later
  ##}
 {% from 'macros/user.html' import appearance with context %}
@@ -24,6 +24,7 @@
       font-family: sans-serif;
       font-size: 11pt;
       transform: rotate(180deg);
+      background-color: #232327;
     }
 
     .button {
@@ -143,6 +144,27 @@
       font-size: 9pt;
       cursor: default;
     }
+    .chat-date {
+      transform: rotate(-180deg);
+      text-align: center;
+      position: relative;
+      display: grid;
+      align-items: center;
+      margin: 8px 0;
+      color: #b2b2b3;
+      cursor: default;
+    }
+    .chat-date > hr {
+      margin: 0;
+      position: absolute;
+      width: 100%;
+      box-sizing: border-box;
+    }
+    .chat-date > :not(hr) > time {
+      padding: 0 1ch;
+      background-color: #232327;
+      position: relative;
+    }
 
     {% for token in messages | map(attribute='token') | list | unique %}
       {% with this_user = users_by_token[token] %}
@@ -171,7 +193,7 @@
   <ol id="chat-messages">
     {% for message in messages | reverse %}
       {% with this_user = users_by_token[message.token] %}
-        <li class="chat-message" data-seq="{{ message.seq }}" data-token-hash="{{ this_user.token_hash }}">
+        <li class="chat-message" data-seq="{{ message.seq }}" data-token-hash="{{ this_user.token_hash }}" data-date="{{ message.date }}">
           <time class="chat-message__time" datetime="{{ message.date }}T{{ message.time_seconds }}Z" title="{{ message.date }} {{ message.time_seconds }}">{{ message.time_minutes }}</time>
           {{- '&nbsp;' | safe -}}
           {{ appearance(this_user, insignia_class='chat-message__insignia', name_class='chat-message__name', tag_class='chat-message__name__tag') }}
@@ -179,6 +201,15 @@
           <span class="chat-message__markup">{{ message.markup }}</span>
         </li>
       {% endwith %}
+      {%
+        if loop.nextitem is defined and loop.nextitem.date != message.date
+        or loop.nextitem is not defined and show_initial_date
+      %}
+        <li class="chat-date" data-date="{{ message.date }}">
+          <hr>
+          <div><time datetime="{{ message.date }}">{{ message.date }}</time></div>
+        </li>
+      {% endif %}
     {% endfor %}
   </ol>
   <aside id="timeout-dismiss">

anonstream/templates/nojs_chat_users.html

@@ -1,5 +1,5 @@
 {##
- # SPDX-FileCopyrightText: 2022 n9k [https://git.076.ne.jp/ninya9k]
+ # SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
  # SPDX-License-Identifier: AGPL-3.0-or-later
  ##}
 {% from 'macros/user.html' import appearance with context %}
@@ -18,6 +18,7 @@
       margin: 0;
       color: #ddd;
       font-family: sans-serif;
+      background-color: #121214;
     }
     #timeout {
       height: 0;

anonstream/templates/nojs_info.html

@@ -1,5 +1,5 @@
 {##
- # SPDX-FileCopyrightText: 2022 n9k [https://git.076.ne.jp/ninya9k]
+ # SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
  # SPDX-License-Identifier: AGPL-3.0-or-later
  ##}
 <!doctype html>
@@ -15,6 +15,7 @@
       margin: 0.75ch 1.25ch;
       font-family: sans-serif;
       color: #ddd;
+      background-color: #18181a;
     }
     #float {
       float: right;

anonstream/templates/nojs_stream.html

@@ -1,5 +1,5 @@
 {##
- # SPDX-FileCopyrightText: 2022 n9k [https://git.076.ne.jp/ninya9k]
+ # SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
  # SPDX-License-Identifier: AGPL-3.0-or-later
  ##}
 <!doctype html>
@@ -18,6 +18,7 @@
       overflow: hidden;
       color: #ddd;
       font-family: sans-serif;
+      background-color: black;
     }
     {% if online %}
       #video {

anonstream/user.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import operator
@@ -8,6 +8,7 @@ from math import inf
 
 from quart import current_app
 
+from anonstream.events import notify_event_sockets
 from anonstream.wrappers import try_except_log, with_timestamp, get_timestamp
 from anonstream.helpers.user import get_default_name, get_presence, Presence
 from anonstream.helpers.captcha import check_captcha_digest, Answer
@@ -98,6 +99,21 @@ def try_change_appearance(user, name, color, password, want_tripcode):
         # Add to the users update buffer
         USERS_UPDATE_BUFFER.add(user['token'])
 
+        # Notify event sockets that a user's appearance was set
+        # NOTE: Changing appearance is currently NOT ratelimited.
+        #       Applications using the event socket API should buffer these
+        #       events or do something else to a prevent a potential denial of
+        #       service.
+        notify_event_sockets({
+            'type': 'appearance',
+            'event': {
+                'token': user['token'],
+                'name': user['name'],
+                'color': user['color'],
+                'tripcode': user['tripcode'],
+            }
+        })
+
     return errors
 
 def change_name(user, name, dry_run=False):

anonstream/utils/captcha.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import hashlib

anonstream/utils/chat.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import base64
@@ -6,6 +6,7 @@ import hashlib
 import math
 import re
 import secrets
+from datetime import datetime
 from functools import lru_cache
 
 from quart import escape
@@ -31,19 +32,14 @@ def get_approx_linespan(text):
     linespan = linespan if linespan > 0 else 1
     return linespan
 
-def precompute_emote_regex(schema):
-    for emote in schema:
-        assert emote['name'], 'emote names cannot be empty'
-        assert not re.search(r'\s', emote['name']), \
-            'whitespace is not allowed in emote names'
-        # If the emote name begins with a word character [a-zA-Z0-9_],
-        # match only if preceded by a non-word character or the empty
-        # string. Similarly for the end of the emote name.
-        # Examples:
-        #  * ":joy:" matches "abc :joy:~xyz"   and   "abc:joy:xyz"
-        #  * "JoySi" matches "abc JoySi~xyz" but NOT "abcJoySiabc"
-        onset = r'(?:^|(?<=\W))' if re.fullmatch(r'\w', emote['name'][0]) else r''
-        finish = r'(?:$|(?=\W))' if re.fullmatch(r'\w', emote['name'][-1]) else r''
-        emote['regex'] = re.compile(''.join(
-            (onset, re.escape(escape(emote['name'])), finish)
-        ))
+def should_show_initial_date(timestamp, messages):
+    try:
+        first_message = next(iter(messages))
+    except StopIteration:
+        return False
+    if any(message['date'] != first_message['date'] for message in messages):
+        return True
+    else:
+        latest_date = max(map(lambda message: message['date'], messages))
+        date = datetime.utcfromtimestamp(timestamp).strftime('%Y-%m-%d')
+        return date != latest_date

anonstream/utils/colour.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import re

anonstream/utils/security.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import secrets

anonstream/utils/user.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import base64

anonstream/utils/websocket.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 from enum import Enum

anonstream/websocket.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import asyncio
@@ -36,6 +36,11 @@ async def websocket_outbound(queue, user):
         'scrollback': CONFIG['MAX_CHAT_SCROLLBACK'],
         'digest': get_random_captcha_digest_for(user),
         'pingpong': CONFIG['TASK_BROADCAST_PING'],
+        'maxlength': {
+            'comment': CONFIG['CHAT_COMMENT_MAX_LENGTH'],
+            'name': CONFIG['CHAT_NAME_MAX_LENGTH'],
+            'password': CONFIG['CHAT_TRIPCODE_PASSWORD_MAX_LENGTH'],
+        },
     })
     while True:
         payload = await queue.get()
@@ -140,12 +145,13 @@ def handle_inbound_message(timestamp, queue, user, nonce, comment, digest, answe
         message_was_added = False
     else:
         try:
-            message_was_added = add_chat_message(
+            seq = add_chat_message(
                 user,
                 nonce,
                 comment,
                 ignore_empty=verification_happened,
             )
+            message_was_added = seq is not None
         except Rejected as e:
             notice, *_ = e.args
             message_was_added = False

anonstream/wrappers.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import time

asgi.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2022 n9k <https://git.076.ne.jp/ninya9k>
+# SPDX-FileCopyrightText: 2022 n9k <https://gitler.moe/ninya9k>
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 if __name__ == '__main__':

config.toml

@@ -57,7 +57,7 @@ anonymous = "Anonymous"
 
 [chat]
 max_comment_length = 512
-max_comment_lines = 12
+max_comment_lines = 20
 max_name_length = 24
 min_name_contrast = 3.0
 background_color = "#232327"

doc/EMOTES.md

@@ -0,0 +1,32 @@
+## Emotes
+
+Emotes are small images that are inserted into chat messages in place of
+given strings.  To add emotes, add entries to `emotes.json` that look
+like below, then restart the server.
+
+```json
+{
+  "name": ":joy:",
+  "file": "joy.png",
+  "width": 20,
+  "height": 24
+}
+```
+```json
+{
+  "name": "JoySpin",
+  "file": "emote/joyspin.gif",
+  "width": null,
+  "height": 24
+}
+```
+
+* `name` is the string that will be replaced in chat messages.
+
+* `file` is the location of the emote image relative to the static
+  directory `anonstream/static`.  The file must actually be in the
+  static directory or a subdirectory of it.
+
+* `width` and `height` are the dimensions the inserted <img> element
+  will have.  Each can be either a non-negative integer or `null`
+  (automatic width/height).

doc/HACKING.md

@@ -1,26 +1,26 @@
 ## Hacking
 
-By default anonstream has two APIs it exposes through two UNIX sockets:
-the control socket `control.sock` and the event socket `event.sock`.  If
-the platform you are using does not support UNIX sockets, they can be
-disabled in the config.
+By default anonstream has two private APIs it exposes through two UNIX
+sockets: the control socket `control.sock` and the event socket
+`event.sock`.  If the platform you are on does not support UNIX sockets,
+they can be disabled in the config.
 
 ### Control socket
 
 The control socket allows reading and modifying internal state, e.g.
 setting the title or changing a user's name.  Currently the control
-socket has checks to see if what you're doing is sane, but they're not
+socket has checks to see if what you're doing is sane, but they're non-
 comprehensive; you could craft commands that lead to undefined
 behaviour.  If you have `socat`, you can use the control socket
 interactively like this:
 ```sh
-rlwrap socat STDIN UNIX-CONNECT:control.sock
-```
-`rlwrap` only adds line editing and is optional. If you don't have it
-you can still get (inferior) line editing by doing:
-```sh
 socat READLINE UNIX-CONNECT:control.sock
 ```
+If you have it, you can use `rlwrap` to get line editing that's a bit
+nicer:
+```sh
+rlwrap socat STDIN UNIX-CONNECT:control.sock
+```
 Once connected, type "help" and press enter to get a list of commands.
 
 ### Event socket
@@ -28,7 +28,7 @@ Once connected, type "help" and press enter to get a list of commands.
 The event socket is a read-only socket that sends out internal events as
 they happen.  Currently the only supported event is a chat message being
 added.  The intended use is to hook into other applications that depend
-on chat, e.g.  text-to-speech or Twitch Plays Pokémon.
+on chat, e.g. text-to-speech or Twitch Plays Pokémon.
 
 View events like this:
 ```sh

doc/guide/OBS.md

@@ -0,0 +1,94 @@
+### OBS Studio
+
+anonstream considers the stream online when new video segments are
+actively being created.  It knows which segments are new by examining
+the segment playlist: an HLS playlist located by default at
+`stream/stream.m3u8`.  Software like FFmpeg and OBS can write to this
+file for you.  This guide is for OBS Studio.  (An example that uses
+FFmpeg instead is [at the bottom](#ffmpeg-example).)
+
+Install OBS Studio.  If the autoconfiguration wizard prompts you to
+choose a third-party service, ignore it since we're not going to be
+using a third-party service.
+
+Click `Settings` and set these:
+
+* Advanced
+  * Recording
+    * Filename Formatting: `stream`
+    * Overwrite if file exists: yes
+* Video
+  * Output (Scaled) Resolution: `960x540` or lower, or whatever you want
+  * Common FPS Values: any integer framerate (e.g. 30 or 60)
+* Output
+  * Output Mode: `Advanced`
+  * Recording:
+    ```
+    +----------------------------+-------------------------------------+
+    | Field                      | Value                               |
+    +============================+=====================================+
+    | Type                       | `Custom Output (FFmpeg)`            |
+    +----------------------------+-------------------------------------+
+    | FFmpeg Output Type         | `Output to File`                    |
+    +----------------------------+-------------------------------------+
+    | File path or URL           | same as the `segments/directory`    |
+    |                            | option in config.toml, but make it  |
+    |                            | an absolute path                    |
+    +----------------------------+-------------------------------------+
+    | Container Format           | `hls`                               |
+    +----------------------------+-------------------------------------+
+    | Muxer Settings (if any)    | `hls_init_time=0 hls_time=2 `       |
+    |                            | `hls_list_size=120 `                |
+    |                            | `hls_flags=delete_segments `        |
+    |                            | `hls_segment_type=fmp4`             |
+    +----------------------------+-------------------------------------+
+    | Video bitrate              | `420 Kbps` or lower, or whatever    |
+    |                            | you want                            |
+    +----------------------------+-------------------------------------+
+    | Keyframe interval (frames) | `framerate` * `hls_time`, e.g. for  |
+    |                            | 60fps and an `hls_time` of 2        |
+    |                            | seconds, set this to 120            |
+    +----------------------------+-------------------------------------+
+    | Video Encoder              | libx264, or an H.264 hardware       |
+    |                            | encoder (e.g. `h264_nvenc` for      |
+    |                            | Nvidia, [see here][hwaccel])        |
+    +----------------------------+-------------------------------------+
+    | Audio Bitrate              | `96 Kbps`, or whatever you want     |
+    +----------------------------+-------------------------------------+
+    | Audio Encoder              | `aac`                               |
+    +----------------------------+-------------------------------------+
+    ```
+
+> *If this table looks garbled, read this file as plaintext or [click
+> here][plaintext] and scroll to the bottom.*
+
+To start streaming click `Start Recording`.
+
+When OBS is recording, segments older than four minutes will be
+regularly deleted.  When OBS stops recording, the last four minutes
+worth of segments will remain the segments directory.  (You can change
+how many segments are kept by modifying the `hls_list_size` option in
+the muxer settings.)  When OBS is not recording, you can delete the
+files in the segments directory without consequence.  Old segments will
+never be sent over the network even if they are not deleted.
+
+### FFmpeg example
+
+This FFmpeg command is basically equivalent to the OBS settings above.
+The input (`-i ...`) can be anything, e.g. to screen record see
+<https://trac.ffmpeg.org/wiki/Capture/Desktop>.
+
+```sh
+ffmpeg \
+-re -i somevideo.mp4 \
+-c:v h264 -b:v 300k -vf scale=-2:360 -g 50 \
+-c:a aac -b:a 80k \
+-f hls \
+-hls_init_time 0 -hls_time 2 \
+-hls_list_size 120 -hls_flags delete_segments \
+-hls_segment_type fmp4 \
+stream/stream.m3u8
+```
+
+[hwaccel]: https://trac.ffmpeg.org/wiki/HWAccelIntro
+[plaintext]: https://gitler.moe/ninya9k/anonstream/raw/branch/master/doc/guide/OBS.md

doc/guide/ONIONSITE.md

@@ -0,0 +1,133 @@
+### Onionsite setup
+
+You probably want to put your livestream on the Internet somehow.  A
+simple way of doing that is to create an onion address.  Follow the
+setup in [the readme][readme] if you haven't already.  You
+should be to access your site locally at http://127.0.0.1:5051.
+
+Install tor.  On Linux you can probably install a package called `tor` and
+be done, otherwise [compile it][tor].  On Windows download this binary:
+<https://www.torproject.org/download/tor/>.
+
+Find your [torrc][torrc].  On Linux it is probably at `/etc/tor/torrc`.
+On Windows it might be somewhere in `%appdata%\tor` or something.
+
+#### Background
+
+In Tor, a hidden service is a regular TCP service that you talk to via a
+6-hop circuit created within Tor network.  You initiate the creation of
+this circuit by providing tor with the service's hostname (a long
+base32-encoded string ending in ".onion").  This hostname is derived
+from cryptographic keys generated by the hidden service operator.
+
+A TCP service is a computer program you interact with over the Internet
+using TCP, which is a low-level networking protocol sitting above IP
+that creates a reliable connection between two computers.  TCP is
+ubiquitous on the Internet and a lot of applications are built on top
+of it, e.g. IRC, SSH, RTMP, Minecraft, and HTTP (which we're using).
+
+#### Configuration
+
+We are now going to create a hidden service.  We need to give tor a
+directory to store the keys it generates, the location of our existing
+TCP service, and a virtual TCP port to listen on.  There are two
+directives we have to add to our torrc: `HiddenServiceDir` and
+`HiddenServicePort`.  (There is a commented-out section for hidden
+services in the default torrc, you probably want to make changes there.)
+
+##### `HiddenServiceDir`
+
+`HiddenServiceDir` sets the directory for the hidden service's keys and
+other data.  You could choose any directory, but it should be owned by
+the user the tor daemon runs as, and its permissions should be
+`0700/drwx------` (`rwx` for user, `---` for group and everyone else).
+
+If you configure this in a way tor doesn't like, tor will kill itself
+and complain in one of these two ways:
+```
+Jun 11 23:21:17.000 [warn] Directory /home/n9k/projects/anonstream/hidden_service cannot be read: Permission denied
+```
+```
+Jun 12 02:37:51.036 [warn] Permissions on directory /var/lib/tor/anonstream are too permissive.
+```
+
+The simplest option is to go by the examples provided in the torrc.  On
+Linux that would probably be a directory inside `/var/lib/tor`, e.g.
+```
+HiddenServiceDir /var/lib/tor/anonstream
+```
+tor will create this directory itself with the uid, gid, and permissions
+that it likes, which for me are these:
+```
+Access: (0700/drwx------)  Uid: (   42/     tor)   Gid: (   42/     tor)
+```
+
+###### `HiddenServiceDir` troubleshooting
+
+If you created the directory yourself and gave it the wrong permissions
+or uid or gid, delete the directory and let tor create it itself, or do
+this as root:
+```sh
+# chown -R tor:tor /var/lib/tor/anonstream
+# chmod 0700 /var/lib/tor/anonstream
+# chmod 0600 /var/lib/tor/anonstream/*
+# chmod 0700 /var/lib/tor/anonstream/*/
+```
+
+If the user and group `tor` do not exist, your tor daemon runs as some
+other user.  There may be a `User` directive in your torrc or in a file
+included by your torrc, for example on Debian it's `User debian-tor`.
+This means that a tor process running as root will immediately drop
+privileges by switching to the user `debian-tor`.  The user's primary
+group should have the same name, check like this as root:
+`# id debian-tor`.
+
+On Linux, if tor is already running you can see what user and group it
+is running as like this:
+```
+$ ps -C tor -o uid,gid,cmd
+UID GID CMD
+ 42  42 tor --quiet --runasdaemon 0
+$ cat /etc/passwd | grep :42: | cut -f 1 -d :  # 42 is the UID here
+tor
+$ cat /etc/group | grep :42: | cut -f 1 -d :  # 42 is the GID here
+tor
+```
+
+Alternatively you could specify a directory inside the cloned
+repository, e.g. `/home/delphine/Documents/anonstream/hidden_service`
+or something like that.  This will only work if the tor daemon has `rwx`
+permissions on the directory and at least `r-x` permissions on all the
+directories above it.  This is probably not the case for you since your
+home folder might have `0700/drwx------` permissions.  If you
+installed tor as a package, the daemon probably runs as its own user
+(e.g. `debian-tor` on Debian, `tor` on Arch/Gentoo).  I would advise not
+going this route and instead just using `/var/lib/tor/anonstream`.
+
+##### `HiddenServicePort`
+
+Include this line verbatim directly below the `HiddenServiceDir` line:
+```
+HiddenServicePort 80 127.0.0.1:5051
+```
+
+tor will listen for connections to our onion address at virtual port 80
+(the conventional HTTP port), and it will forward traffic to the TCP
+service at 127.0.0.1:5051 (our webserver).
+
+##### Finish
+
+Example configuration:
+```
+HiddenServiceDir /var/lib/tor/anonstream
+HiddenServicePort 80 127.0.0.1:5051
+```
+
+Reload tor to have it reread the torrc: `# pkill -HUP tor`.  With
+systemd you can alternatively do `# systemctl reload tor`.  If
+everything went well, the directory will have been created and your
+onion address will be in `$HIDDEN_SERVICE_DIR/hostname`.
+
+[readme]: https://gitler.moe/ninya9k/anonstream/src/branch/master/README.md#setup
+[tor]: https://gitlab.torproject.org/tpo/core/tor
+[torrc]: https://support.torproject.org/#tbb-editing-torrc

emotes.json

@@ -1 +1 @@
-{}
+[]