From ffca068c86a94b96a4962a7ad76f56b2ac747cef Mon Sep 17 00:00:00 2001 From: n9k Date: Sun, 24 Jul 2022 10:18:19 +0000 Subject: [PATCH] Big readme rejiggering --- README.md | 32 +++--- STREAMING.md | 202 ----------------------------------- HACKING.md => doc/HACKING.md | 22 ++-- doc/guide/OBS.md | 94 ++++++++++++++++ doc/guide/ONIONSITE.md | 133 +++++++++++++++++++++++ 5 files changed, 253 insertions(+), 230 deletions(-) delete mode 100644 STREAMING.md rename HACKING.md => doc/HACKING.md (79%) create mode 100644 doc/guide/OBS.md create mode 100644 doc/guide/ONIONSITE.md diff --git a/README.md b/README.md index 6f07ea0..466ff14 100644 --- a/README.md +++ b/README.md @@ -13,8 +13,8 @@ These mirrors also exist: ## Setup -You must have Python 3.10 at a minimum. You can check your version of Python -with `python --version`. +You must have Python 3.10 at a minimum. You can check your version of +Python with `python --version`. Clone the repo: ```sh @@ -29,14 +29,15 @@ source venv/bin/activate python -m pip install -r requirements.txt ``` -Before you run it you may want to edit the config ([/config.toml][config]). -Most of the defaults are probably okay, but here are some that you might want -to know what they do: +This is all the setup needed to run the application, but before you do +you may want to edit [the config](config.toml). Most of the defaults +are probably okay, but here are some that you might want to know what +they do: * `secret_key`: used for cryptography, make it any long random string (e.g. `$ dd if=/dev/urandom bs=16 count=1 | base64`), definitely set this - yourself before running in "production" (whatever that is for you) + yourself before running in "production" * `segments/directory`: directory containing stream segments, the default is `stream/` in @@ -59,8 +60,8 @@ Run it: python -m anonstream ``` -This will start a webserver listening on the local host at port 5051 (use -`--port PORT` to override). +This will start a webserver listening on the local host at port 5051 +(use `--port PORT` to override). If you go to `http://localhost:5051` in a web browser now you should see the site. When you started the webserver some credentials were printed @@ -68,8 +69,9 @@ in the terminal; you can log in with those at `http://localhost:5051/login`. The only things left are (1) streaming, and (2) letting other people -access your stream. [/STREAMING.md][streaming] has instructions for -setting up OBS Studio and a Tor onion service. If you want to use +access your stream. [OBS.md](doc/guide/OBS.md) has instructions for +setting up OBS Studio and [ONIONSITE.md](doc/guide/ONIONSITE.md) has +instructions for creating a Tor onion service. If you want to use different streaming software and put your stream on the Internet some other way, read those instructions and copy the gist. @@ -90,21 +92,20 @@ python -m uvicorn asgi:create_app --factory --port 5051 In either case you can explicitly set the location of the config file using the `ANONSTREAM_CONFIG` environment variable. - ## Hacking anonstream has APIs for accessing internal state and hooking into internal events. They can be used by humans and other programs. See -[/HACKING.md][hacking]. +[HACKING.md][/doc/HACKING.md]. ## Copying anonstream is AGPL 3.0 or later, see -[/LICENSES/AGPL-3.0-or-later.md][licence]. +[LICENSES/AGPL-3.0-or-later.md][licence]. ### Assets -* [/anonstream/static/settings.svg][settings.svg]: +* [anonstream/static/settings.svg][settings.svg]: [setting](https://thenounproject.com/icon/setting-685325/) by [ulimicon](https://thenounproject.com/unlimicon/) is licensed under [CC BY 3.0](https://creativecommons.org/licenses/by/3.0/). @@ -135,11 +136,8 @@ anonstream is AGPL 3.0 or later, see * werkzeug ([BSD 3-Clause][werkzeug]) -[config]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/config.toml -[hacking]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/HACKING.md [licence]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/LICENSES/AGPL-3.0-or-later.md [settings.svg]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/anonstream/static/settings.svg -[streaming]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/STREAMING.md [aiofiles]: https://github.com/Tinche/aiofiles/blob/master/LICENSE [captcha]: https://github.com/lepture/captcha/blob/master/LICENSE diff --git a/STREAMING.md b/STREAMING.md deleted file mode 100644 index 0ba4070..0000000 --- a/STREAMING.md +++ /dev/null @@ -1,202 +0,0 @@ -### Tor - -Install tor. On Linux you can probably install a package called `tor` and -be done, otherwise [compile it][tor]. On Windows download this binary: -. - -Find your [torrc][torrc]. On Linux it is probably at `/etc/tor/torrc`. -On Windows it might be somewhere in `%appdata%\tor` or something. - -#### Background - -A Tor hidden service is a regular TCP service that you talk to via a -6-hop circuit created inside the Tor network. You initiate the creation -of this circuit by providing tor with the service's hostname, a long -base32-encoded string ending in ".onion". This hostname is derived from -a pair of cryptographic keys generated by the hidden service operator. - -A TCP service is a computer program you interact with over the Internet -using TCP. TCP is a low-level networking protocol that sits above IP -and creates a reliable "connection" between two computers. It handles -the reordering and resending of packets that are shuffled or lost in -transit on the Internet, such that the bytes sent from one computer will -match exactly the bytes that arrive at the other (barring active -interference (MITM), TCP is not secure). Getting reliability for free -greatly simplifies the creation of network applications, and for this -reason and other historical reasons TCP is ubiquitous on the Internet to -this day. Many applications use TCP, for example IRC, SSH, RTMP, -Minecraft, and HTTP (like us here). - -#### Configuration - -We are now going to create a hidden service. We need to give tor a -directory to store the keys it generates, the location of our existing -TCP service, and a virtual TCP port to listen on. There are two -directives we have to add to our torrc: `HiddenServiceDir` and -`HiddenServicePort`. (There is a commented-out section in the default -torrc for hidden services, you may wish to make these changes there.) - -##### `HiddenServiceDir` - -`HiddenServiceDir` sets the directory for the hidden service's keys and -other data. You could choose any directory, but you should make sure -it's owned by the user the tor daemon runs as, and the directory's -permissions are `0700/drwx------` (`rwx` for user, `---` for group and -everyone else). - -If you configure this in a way tor doesn't like, tor will kill itself -and complain in one of these two ways: -``` -Jun 11 23:21:17.000 [warn] Directory /home/n9k/projects/anonstream/hidden_service cannot be read: Permission denied -``` -``` -Jun 12 02:37:51.036 [warn] Permissions on directory /var/lib/tor/anonstream are too permissive. -``` - -The simplest option is to copy the examples provided in the torrc, on -Linux that would probably be a directory inside `/var/lib/tor`, e.g. -`HiddenServiceDir /var/lib/tor/anonstream`. tor will create this -directory itself with the uid, gid, and permissions that it likes, which -for me are these: -``` -Access: (0700/drwx------) Uid: ( 42/ tor) Gid: ( 42/ tor) -``` - -###### `HiddenServiceDir` troubleshooting - -If you created the directory yourself and gave it the wrong permissions -or uid or gid, delete the directory and let tor create it itself, or do -this: -``` -# chown -R tor:tor /var/lib/tor/anonstream -# chmod 0700 /var/lib/tor/anonstream -# chmod 0600 /var/lib/tor/anonstream/* -# chmod 0700 /var/lib/tor/anonstream/*/ -``` - -If the user and group `tor` do not exist, your tor daemon runs as some -other user. There may be a `User` directive in your torrc or in a file -included by your torrc, for example on Debian it's `User debian-tor`. -This means that a tor process running as root will immediately drop -privileges by switching to the user `debian-tor`. The user's primary -group should have the same name, check like this as root: -`# id debian-tor`. - -On Linux, if tor is already running you can see what user and group it is -running as like this: -``` -$ ps -C tor -o uid,gid,cmd -UID GID CMD - 42 42 tor --quiet --runasdaemon 0 -$ cat /etc/passwd | grep :42: | cut -f 1 -d : # 42 is the UID here -tor -$ cat /etc/group | grep :42: | cut -f 1 -d : # 42 is the GID here -tor -``` - -Alternatively you could specify a directory inside the cloned -repository, e.g. `/home/delphine/Documents/anonstream/hidden_service` -or something like that. This will only work if the tor daemon has `rwx` -permissions on the directory and at least `r-x` permissions on all the -directories above it. This is probably not the case for you since your -home folder might have `0700/drwx------` permissions. If you -installed tor as a package, the daemon probably runs as its own user -(e.g. `debian-tor` on Debian, `tor` on Arch/Gentoo). If you want to -figure this out yourself go ahead. I would advise just using -`/var/lib/tor/anonstream` though. - -##### `HiddenServicePort` - -Include this line verbatim directly below the `HiddenServiceDir` line: -``` -HiddenServicePort 80 127.0.0.1:5051 -``` - -tor will listen for connections to our onion address at virtual port 80 -(the conventional HTTP port), and it will forward traffic to the TCP -service at 127.0.0.1:5051, which is our webserver. - -##### Finish - -Example configuration: -``` -HiddenServiceDir /var/lib/tor/anonstream -HiddenServicePort 80 127.0.0.1:5051 -``` - -Reload tor to make it reread its torrc: `# pkill -HUP tor`. With -systemd you can alternatively do `# systemctl reload tor`. If -everything went well, the directory will have been created and your -onion address will be in `$HIDDEN_SERVICE_DIR/hostname`. - -### OBS Studio - -Install OBS Studio. If the autoconfiguration wizard prompts you to -choose a third-party service, ignore it since we're not going to be -using a third-party service. - -Click `Settings` and set these: - -* Advanced - * Recording - * Filename Formatting: `stream` - * Overwrite if file exists: yes -* Video - * Output (Scaled) Resolution: `960x540` or lower, or whatever you want - * Common FPS Values: any integer framerate (e.g. 30 or 60) -* Output - * Output Mode: `Advanced` - * Recording: - ``` - +----------------------------+-------------------------------------+ - | Field | Value | - +============================+=====================================+ - | Type | `Custom Output (FFmpeg)` | - +----------------------------+-------------------------------------+ - | FFmpeg Output Type | `Output to File` | - +----------------------------+-------------------------------------+ - | File path or URL | same as the `segments/directory` | - | | option in config.toml, but make it | - | | an absolute path | - +----------------------------+-------------------------------------+ - | Container Format | `hls` | - +----------------------------+-------------------------------------+ - | Muxer Settings (if any) | `hls_init_time=0 hls_time=2 ` | - | | `hls_list_size=120 ` | - | | `hls_flags=delete_segments ` | - | | `hls_segment_type=fmp4` | - +----------------------------+-------------------------------------+ - | Video bitrate | `420 Kbps` or lower, or whatever | - | | you want | - +----------------------------+-------------------------------------+ - | Keyframe interval (frames) | `framerate` * `hls_time`, e.g. for | - | | 60fps and an `hls_time` of 2 | - | | seconds, set this to 120 | - +----------------------------+-------------------------------------+ - | Video Encoder | libx264, or an H.264 hardware | - | | encoder (e.g. `h264_nvenc` for | - | | Nvidia, [see here][ffmpeg]) | - +----------------------------+-------------------------------------+ - | Audio Bitrate | `96 Kbps`, or whatever you want | - +----------------------------+-------------------------------------+ - | Audio Encoder | `aac` | - +----------------------------+-------------------------------------+ - ``` - -> *If this table looks garbled, read this file as plaintext or [click -> here][plaintext] and scroll to the bottom.* - -To start streaming click `Start Recording`. - -When it is recording, segments older than four minutes will be regularly -deleted, and when it stops recording the last four minutes worth of -segments will remain the segments directory. (You can change the number -of kept segments by modifying the `hls_list_size` option in the muxer -settings.) When it is not recording, you can delete the files in the -segments directory without consequence. Old segments will never be sent -over the network even if they are not deleted. - -[tor]: https://gitlab.torproject.org/tpo/core/tor -[torrc]: https://support.torproject.org/#tbb-editing-torrc -[ffmpeg]: https://trac.ffmpeg.org/wiki/HWAccelIntro -[plaintext]: https://git.076.ne.jp/ninya9k/anonstream/raw/branch/master/STREAMING.md diff --git a/HACKING.md b/doc/HACKING.md similarity index 79% rename from HACKING.md rename to doc/HACKING.md index fd88c66..8bedc2f 100644 --- a/HACKING.md +++ b/doc/HACKING.md @@ -1,26 +1,26 @@ ## Hacking -By default anonstream has two APIs it exposes through two UNIX sockets: -the control socket `control.sock` and the event socket `event.sock`. If -the platform you are using does not support UNIX sockets, they can be -disabled in the config. +By default anonstream has two private APIs it exposes through two UNIX +sockets: the control socket `control.sock` and the event socket +`event.sock`. If the platform you are on does not support UNIX sockets, +they can be disabled in the config. ### Control socket The control socket allows reading and modifying internal state, e.g. setting the title or changing a user's name. Currently the control -socket has checks to see if what you're doing is sane, but they're not +socket has checks to see if what you're doing is sane, but they're non- comprehensive; you could craft commands that lead to undefined behaviour. If you have `socat`, you can use the control socket interactively like this: ```sh -rlwrap socat STDIN UNIX-CONNECT:control.sock -``` -`rlwrap` only adds line editing and is optional. If you don't have it -you can still get (inferior) line editing by doing: -```sh socat READLINE UNIX-CONNECT:control.sock ``` +If you have it, you can use `rlwrap` to get line editing that's a bit +nicer: +```sh +rlwrap socat STDIN UNIX-CONNECT:control.sock +``` Once connected, type "help" and press enter to get a list of commands. ### Event socket @@ -28,7 +28,7 @@ Once connected, type "help" and press enter to get a list of commands. The event socket is a read-only socket that sends out internal events as they happen. Currently the only supported event is a chat message being added. The intended use is to hook into other applications that depend -on chat, e.g. text-to-speech or Twitch Plays Pokémon. +on chat, e.g. text-to-speech or Twitch Plays Pokémon. View events like this: ```sh diff --git a/doc/guide/OBS.md b/doc/guide/OBS.md new file mode 100644 index 0000000..d927792 --- /dev/null +++ b/doc/guide/OBS.md @@ -0,0 +1,94 @@ +### OBS Studio + +anonstream considers the stream online when new video segments are +actively being created. It knows which segments are new by examining +the segment playlist: an HLS playlist located by default at +`stream/stream.m3u8`. Software like FFmpeg and OBS can write to this +file for you. This guide is for OBS Studio. (An example that uses +FFmpeg instead is [at the bottom](#ffmpeg-example).) + +Install OBS Studio. If the autoconfiguration wizard prompts you to +choose a third-party service, ignore it since we're not going to be +using a third-party service. + +Click `Settings` and set these: + +* Advanced + * Recording + * Filename Formatting: `stream` + * Overwrite if file exists: yes +* Video + * Output (Scaled) Resolution: `960x540` or lower, or whatever you want + * Common FPS Values: any integer framerate (e.g. 30 or 60) +* Output + * Output Mode: `Advanced` + * Recording: + ``` + +----------------------------+-------------------------------------+ + | Field | Value | + +============================+=====================================+ + | Type | `Custom Output (FFmpeg)` | + +----------------------------+-------------------------------------+ + | FFmpeg Output Type | `Output to File` | + +----------------------------+-------------------------------------+ + | File path or URL | same as the `segments/directory` | + | | option in config.toml, but make it | + | | an absolute path | + +----------------------------+-------------------------------------+ + | Container Format | `hls` | + +----------------------------+-------------------------------------+ + | Muxer Settings (if any) | `hls_init_time=0 hls_time=2 ` | + | | `hls_list_size=120 ` | + | | `hls_flags=delete_segments ` | + | | `hls_segment_type=fmp4` | + +----------------------------+-------------------------------------+ + | Video bitrate | `420 Kbps` or lower, or whatever | + | | you want | + +----------------------------+-------------------------------------+ + | Keyframe interval (frames) | `framerate` * `hls_time`, e.g. for | + | | 60fps and an `hls_time` of 2 | + | | seconds, set this to 120 | + +----------------------------+-------------------------------------+ + | Video Encoder | libx264, or an H.264 hardware | + | | encoder (e.g. `h264_nvenc` for | + | | Nvidia, [see here][hwaccel]) | + +----------------------------+-------------------------------------+ + | Audio Bitrate | `96 Kbps`, or whatever you want | + +----------------------------+-------------------------------------+ + | Audio Encoder | `aac` | + +----------------------------+-------------------------------------+ + ``` + +> *If this table looks garbled, read this file as plaintext or [click +> here][plaintext] and scroll to the bottom.* + +To start streaming click `Start Recording`. + +When OBS is recording, segments older than four minutes will be +regularly deleted. When OBS stops recording, the last four minutes +worth of segments will remain the segments directory. (You can change +how many segments are kept by modifying the `hls_list_size` option in +the muxer settings.) When OBS is not recording, you can delete the +files in the segments directory without consequence. Old segments will +never be sent over the network even if they are not deleted. + +### FFmpeg example + +This FFmpeg command is basically equivalent to the OBS settings above. +The input (`-i ...`) can be anything, e.g. to screen record see +. + +```sh +ffmpeg \ +-re -i somevideo.mp4 \ +-c:v h264 -b:v 300k -vf scale=-2:360 -g 50 \ +-c:a aac -b:a 80k \ +-f hls \ +-hls_init_time 0 -hls_time 2 \ +-hls_list_size 120 -hls_flags delete_segments \ +-hls_segment_type fmp4 \ +stream/stream.m3u8 +``` + +[hwaccel]: https://trac.ffmpeg.org/wiki/HWAccelIntro +[plaintext]: https://git.076.ne.jp/ninya9k/anonstream/raw/branch/master/doc/guide/OBS.md diff --git a/doc/guide/ONIONSITE.md b/doc/guide/ONIONSITE.md new file mode 100644 index 0000000..537cecb --- /dev/null +++ b/doc/guide/ONIONSITE.md @@ -0,0 +1,133 @@ +### Onionsite setup + +You probably want to put your livestream on the Internet somehow. A +simple way of doing that is to create an onion address. Follow the +setup in [the readme][readme] if you haven't already. You +should be to access your site locally at http://127.0.0.1:5051. + +Install tor. On Linux you can probably install a package called `tor` and +be done, otherwise [compile it][tor]. On Windows download this binary: +. + +Find your [torrc][torrc]. On Linux it is probably at `/etc/tor/torrc`. +On Windows it might be somewhere in `%appdata%\tor` or something. + +#### Background + +In Tor, a hidden service is a regular TCP service that you talk to via a +6-hop circuit created within Tor network. You initiate the creation of +this circuit by providing tor with the service's hostname (a long +base32-encoded string ending in ".onion"). This hostname is derived +from cryptographic keys generated by the hidden service operator. + +A TCP service is a computer program you interact with over the Internet +using TCP, which is a low-level networking protocol sitting above IP +that creates a reliable connection between two computers. TCP is +ubiquitous on the Internet and a lot of applications are built on top +of it, e.g. IRC, SSH, RTMP, Minecraft, and HTTP (which we're using). + +#### Configuration + +We are now going to create a hidden service. We need to give tor a +directory to store the keys it generates, the location of our existing +TCP service, and a virtual TCP port to listen on. There are two +directives we have to add to our torrc: `HiddenServiceDir` and +`HiddenServicePort`. (There is a commented-out section for hidden +services in the default torrc, you probably want to make changes there.) + +##### `HiddenServiceDir` + +`HiddenServiceDir` sets the directory for the hidden service's keys and +other data. You could choose any directory, but it should be owned by +the user the tor daemon runs as, and its permissions should be +`0700/drwx------` (`rwx` for user, `---` for group and everyone else). + +If you configure this in a way tor doesn't like, tor will kill itself +and complain in one of these two ways: +``` +Jun 11 23:21:17.000 [warn] Directory /home/n9k/projects/anonstream/hidden_service cannot be read: Permission denied +``` +``` +Jun 12 02:37:51.036 [warn] Permissions on directory /var/lib/tor/anonstream are too permissive. +``` + +The simplest option is to go by the examples provided in the torrc. On +Linux that would probably be a directory inside `/var/lib/tor`, e.g. +``` +HiddenServiceDir /var/lib/tor/anonstream +``` +tor will create this directory itself with the uid, gid, and permissions +that it likes, which for me are these: +``` +Access: (0700/drwx------) Uid: ( 42/ tor) Gid: ( 42/ tor) +``` + +###### `HiddenServiceDir` troubleshooting + +If you created the directory yourself and gave it the wrong permissions +or uid or gid, delete the directory and let tor create it itself, or do +this as root: +```sh +# chown -R tor:tor /var/lib/tor/anonstream +# chmod 0700 /var/lib/tor/anonstream +# chmod 0600 /var/lib/tor/anonstream/* +# chmod 0700 /var/lib/tor/anonstream/*/ +``` + +If the user and group `tor` do not exist, your tor daemon runs as some +other user. There may be a `User` directive in your torrc or in a file +included by your torrc, for example on Debian it's `User debian-tor`. +This means that a tor process running as root will immediately drop +privileges by switching to the user `debian-tor`. The user's primary +group should have the same name, check like this as root: +`# id debian-tor`. + +On Linux, if tor is already running you can see what user and group it +is running as like this: +``` +$ ps -C tor -o uid,gid,cmd +UID GID CMD + 42 42 tor --quiet --runasdaemon 0 +$ cat /etc/passwd | grep :42: | cut -f 1 -d : # 42 is the UID here +tor +$ cat /etc/group | grep :42: | cut -f 1 -d : # 42 is the GID here +tor +``` + +Alternatively you could specify a directory inside the cloned +repository, e.g. `/home/delphine/Documents/anonstream/hidden_service` +or something like that. This will only work if the tor daemon has `rwx` +permissions on the directory and at least `r-x` permissions on all the +directories above it. This is probably not the case for you since your +home folder might have `0700/drwx------` permissions. If you +installed tor as a package, the daemon probably runs as its own user +(e.g. `debian-tor` on Debian, `tor` on Arch/Gentoo). I would advise not +going this route and instead just using `/var/lib/tor/anonstream`. + +##### `HiddenServicePort` + +Include this line verbatim directly below the `HiddenServiceDir` line: +``` +HiddenServicePort 80 127.0.0.1:5051 +``` + +tor will listen for connections to our onion address at virtual port 80 +(the conventional HTTP port), and it will forward traffic to the TCP +service at 127.0.0.1:5051 (our webserver). + +##### Finish + +Example configuration: +``` +HiddenServiceDir /var/lib/tor/anonstream +HiddenServicePort 80 127.0.0.1:5051 +``` + +Reload tor to have it reread the torrc: `# pkill -HUP tor`. With +systemd you can alternatively do `# systemctl reload tor`. If +everything went well, the directory will have been created and your +onion address will be in `$HIDDEN_SERVICE_DIR/hostname`. + +[readme]: https://git.076.ne.jp/ninya9k/anonstream/src/branch/master/README.md#setup +[tor]: https://gitlab.torproject.org/tpo/core/tor +[torrc]: https://support.torproject.org/#tbb-editing-torrc