From 5a262053c14d55b186e71105c9f5a6ae70870c87 Mon Sep 17 00:00:00 2001 From: Arya Kiran Date: Tue, 7 Mar 2023 18:05:57 +0530 Subject: [PATCH] update scraping section of readme --- README.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 886e136..b8a68a3 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,8 @@ The ``master`` branch runs on the master branch of GotHub. This means that the i Have an instance you want on the list? Make an issue or PR! +You can find a json-list of the same in [gothub/gothub-instances](https://codeberg.org/gothub/gothub-instances). + [^1]: Has some modifications: https://git.vern.cc/vern/modifications/src/branch/master/gothub ## Setup @@ -64,7 +66,13 @@ gothub.example.com { } ``` ## Scraping -Variables: 1) GOTHUB_USER_AGENT - Sets the User Agent to use for making the requests. Defaults to Chrome 110 on Windows 10. +Other than for the explore page, gothub uses scraping in order to retrieve the information. + +This makes it hard for gothub to get ratelimited. + +You can also set the GOTHUB_USER_AGENT envvar in order to set the user agent with which GotHub makes requests to Github. + +By default, we use the user agent of Chrome 110 on Windows 10. ## DMCA/Legal notice