13 files changed, 0 insertions(+), 598 deletions(-)

D docs/book.toml
D docs/src/SUMMARY.md
D docs/src/coding-conventions.md
D docs/src/commit-messages.md
D docs/src/cronjobs.md
D docs/src/database-backup.md
D docs/src/dependencies.md
D docs/src/deployment.md
D docs/src/file-structure-walkthrough.md
D docs/src/introduction.md
D docs/src/main-repository-readme.md
D docs/src/runbook.md
D docs/src/server-migration.md

@@ 1,6 0,0 @@
-[book]
-authors = ["Theodore Keloglou"]
-language = "en"
-multilingual = false
-src = "src"
-title = "Mataroa Documentation"

@@ 1,13 0,0 @@
-# Summary
-
-- [Introduction](./introduction.md)
-- [Main Repository README](./main-repository-readme.md)
-- [Coding Conventions](./coding-conventions.md)
-- [Git Commit Message Guidelines](./commit-messages.md)
-- [File Structure Walkthrough](./file-structure-walkthrough.md)
-- [Dependencies](./dependencies.md)
-- [Deployment](./deployment.md)
-- [Cronjobs](./cronjobs.md)
-- [Database Backup](./database-backup.md)
-- [Server Migration](./server-migration.md)
-- [Runbook](./runbook.md)

@@ 1,4 0,0 @@
-# Coding Conventions
-
-1. All files should end with a new line character.
-1. Python code should be formatted with [ruff](https://github.com/astral-sh/ruff).

@@ 1,15 0,0 @@
-# Git Commit Message Guidelines
-
-We follow some simple non-austere git commit message guidelines.
-
-* Start with a verb
-    * `add`
-    * `change`
-    * `delete`
-    * `fix`
-    * `refactor`
-    * `tweak`
-    * et al.
-* Start with a lowercase letter
-    * eg. `change analytic page path to the same of page slug`
-* Do not end with a fullstop

@@ 1,31 0,0 @@
-# Cronjobs
-
-We don't use cron but systemd timers for jobs that need to run recurringly.
-
-## Process email notifications
-
-```sh
-python manage.py processnotifications
-```
-
-Sends notification emails for new blog posts.
-
-Triggers daily at 10AM server time.
-
-## Email blog exports
-
-```sh
-python manage.py mailexports
-```
-
-Emails users their blog exports.
-
-Triggers monthly, first day of the month, 6AM server time.
-
-## Database backup
-
-```
-./backup-database.sh
-```
-
-Triggers every 6 hours.

@@ 1,45 0,0 @@
-# Database Backup
-
-## Shell Script
-
-We use the script [`backup-database.sh`](backup-database.sh) to dump the
-database and upload it into an S3-compatible object storage cloud using
-[rclone](https://rclone.org/). This script needs the database password
-as an environment variable. The key must be `PGPASSWORD`. The variable can live
-in `.envrc` as such:
-
-```sh
-export PGPASSWORD=db-password
-```
-
-## Commands
-
-To create a database dump run:
-
-```sh
-pg_dump -Fc --no-acl mataroa -h localhost -U mataroa -f /home/deploy/mataroa.dump -W
-```
-
-To restore a database dump run:
-
-```sh
-pg_restore --disable-triggers -j 4 -v -h localhost -cO --if-exists -d mataroa -U mataroa -W mataroa.dump
-```
-
-## Initialise and configure backup script
-
-```sh
-cp /var/www/mataroa/backup-database.sh /home/deploy/
-```
-
-## Setup rclone
-
-1. Create bucket on Scaleway or any other S3-compatible object storage.
-1. Find bucket URL.
-    * On Scaleway: it's on Bucket Settings.
-1. Acquire IAM Access Key ID and Secret Key.
-    * On Scaleway: IAM -> Applications -> Project default -> API Keys
-
-```sh
-rclone config
-```

@@ 1,47 0,0 @@
-# Dependencies
-
-## Dependency Policy
-
-The mataroa project has an unusually strict yet usually unclear dependency policy.
-
-Vague rules include:
-
-* No third-party Django apps.
-* All Python / PyPI packages should be individually vetted.
-    * Packages should be published from community-trusted organisations or developers.
-    * Packages should be actively maintained (though not necessarily actively developed).
-    * Packages should hold a high quality of coding practices.
-* No JavaScript libraries / dependencies.
-
-Current list of top-level PyPI dependencies (source at [`pyproject.toml`](/pyproject.toml)):
-
-* [Django](https://pypi.org/project/Django/)
-* [psycopg](https://pypi.org/project/psycopg/)
-* [gunicorn](https://pypi.org/project/gunicorn/)
-* [Markdown](https://pypi.org/project/Markdown/)
-* [Pygments](https://pypi.org/project/Pygments/)
-* [bleach](https://pypi.org/project/bleach/)
-* [stripe](https://pypi.org/project/stripe/)
-
-## Adding a new dependency
-
-After approving a dependency, add it using `uv`:
-
-1. Ensure `uv` is installed and a virtualenv exists (managed by `uv`).
-1. Add the dependency to `pyproject.toml` and lockfile with:
-   - Runtime: `uv add PACKAGE`
-   - Dev-only: `uv add --dev PACKAGE`
-1. Install/sync dependencies: `uv sync`
-
-## Upgrading dependencies
-
-When a new Django version is out it’s a good idea to upgrade everything.
-
-Steps:
-
-1. Update the lockfile: `uv lock --upgrade`
-1. Review changes: `git diff uv.lock` and spot non-patch level version bumps.
-1. Examine release notes of each one.
-1. Install updated deps: `uv sync`
-1. Unless something comes up, make sure tests and smoke tests pass.
-1. Deploy new dependency versions.

@@ 1,66 0,0 @@
-# Deployment
-
-## Step 1: Ansible
-
-We use ansible to provision a Debian 12 Linux server.
-
-(1a) First, set up configuration files:
-
-```sh
-cd ansible/
-# Make a copy of the example file
-cp .envrc.example .envrc
-
-# Edit parameters as required
-vim .envrc
-
-# Load variables into environment
-source .envrc
-```
-
-(1b) Then, provision:
-
-```sh
-ansible-playbook playbook.yaml -v
-```
-
-## Step 2: Wildcard certificates
-
-We use Automatic DNS API integration with DNSimple:
-
-* https://github.com/acmesh-official/acme.sh?tab=readme-ov-file#1-how-to-install
-* https://github.com/acmesh-official/acme.sh/wiki/dnsapi#dns_dnsimple
-
-Note: acme.sh's default SSL provider is ZeroSSL which does not accept email with
-plus-subaddressing. It will not error gracefully, just fail with a cryptic
-message (tested with acmesh v3.0.7).
-
-```sh
-curl https://get.acme.sh | sh -s email=person@example.com
-# Note: Installation inserts a cronjob for auto-renewal
-
-# Setup DNSimple API
-echo 'export DNSimple_OAUTH_TOKEN="token-here"' >> /root/.acme.sh/acme.sh.env
-
-# Issue cert
-acme.sh --issue --dns dns_dnsimple -d mataroa.blog -d *.mataroa.blog
-
-# We "install" (copy) the cert because we should not use the cert from acme.sh's internal store
-acme.sh --install-cert -d mataroa.blog -d *.mataroa.blog --key-file /etc/caddy/mataroa-blog-key.pem --fullchain-file /etc/caddy/mataroa-blog-cert.pem --reloadcmd "chown caddy:www-data /etc/caddy/mataroa-blog-{cert,key}.pem && systemctl restart caddy"
-```
-
-## Step 3: Cronjobs and Automated backups
-
-There are a few cronjobs that need setting up and, of course, backups are essential:
-
-* (3a) [Cronjobs](./cronjobs.md)
-* (3b) [Database Backup](./database-backup.md)
-
-## Step 4: Deploy changes
-
-```sh
-git push origin main
-source .venv/bin/activate
-cd ansible/
-ansible-playbook -v deploy.yaml
-```

@@ 1,191 0,0 @@
-# File Structure Walkthrough
-
-Here, an overview of the project's code sources is presented. The purpose is
-for the reader to understand what kind of functionality is located where in
-the sources.
-
-All business logic of the application is in one Django app: [`main`](/main).
-
-Condensed and commented sources file tree:
-
-```
-.
-├── .build.yml # SourceHut CI build config
-├── .envrc.example # example direnv file
-├── .github/ # GitHub Actions config files
-├── Caddyfile # configuration for Caddy webserver
-├── Dockerfile
-├── LICENSE
-├── Makefile # make-defined tasks
-├── README.md
-├── backup-database.sh
-├── default.nix # nix profile
-├── deploy.sh
-├── docker-compose.yml
-├── docs/
-├── export_base_epub/ # base sources for epub export functionality
-├── export_base_hugo/ # base sources for hugo export functionality
-├── export_base_zola/ # base sources for zola export functionality
-├── main/
-│   ├── admin.py
-│   ├── apps.py
-│   ├── denylist.py # list of various keywords allowed and denied
-│   ├── feeds.py # django rss functionality
-│   ├── fixtures/
-│   │   └── dev-data.json # sample development data
-│   ├── forms.py
-│   ├── management/ # commands under `python manage.py`
-│   │   └── commands/
-│   │       └── processnotifications.py
-│   │       └── mailexports.py
-│   ├── middleware.py # mostly subdomain routing
-│   ├── migrations/
-│   ├── models.py
-│   ├── static/
-│   ├── templates
-│   │   ├── main/ # HTML templates for most pages
-│   │   ├── assets/
-│   │   │   ├── drag-and-drop-upload.js
-│   │   │   └── style.css
-│   │   ├── partials/
-│   │   │   ├── footer.html
-│   │   │   ├── footer_blog.html
-│   │   │   └── webring.html
-│   │   └── registration/
-│   ├── tests/
-│   │   ├── test_billing.py
-│   │   ├── test_blog.py
-│   │   ├── test_comments.py
-│   │   ├── test_images.py
-│   │   ├── test_management.py
-│   │   ├── test_pages.py
-│   │   ├── test_posts.py
-│   │   ├── test_users.py
-│   │   └── testdata/
-│   ├── urls.py
-│   ├── util.py
-│   ├── validators.py # custom form and field validators
-│   ├── views.py
-│   ├── views_api.py
-│   ├── views_billing.py
-│   └── views_export.py
-├── manage.py
-└── mataroa
-    ├── asgi.py
-    ├── settings.py # django configuration file
-    ├── urls.py
-    └── wsgi.py
-```
-
-## [`main/urls.py`](/main/urls.py)
-
-All urls are in this module. They are visually divided into several sections:
-
-* general, includes index, dashboard, static pages
-* user system, includes signup, settings, logout
-* blog posts, the CRUD opertions of
-* blog extras, includes rss and newsletter features
-* comments, related to the blog post comments
-* billing, subscription and card related
-* blog import, export, webring
-* images CRUD
-* analytics list and details
-* pages CRUD
-
-## [`main/views.py`](/main/views.py)
-
-The majority of business logic is in the `views.py` module.
-
-It includes:
-
-* indexes, dashboard, static pages
-* user CRUD and login/logout
-* posts CRUD
-* comments CRUD
-* images CRUD
-* pages CRUD
-* webring
-* analytics
-* notifications subscribe/unsubscribe
-* moderation dashboard
-* sitemaps
-
-Generally,
-[Django class-based generic views](https://docs.djangoproject.com/en/3.2/topics/class-based-views/generic-display/)
-are used most of the time as they provide useful functionality abstracted away.
-
-The Django source code [for generic views](https://github.com/django/django/tree/main/django/views/generic)
-is also extremely readable:
-
-* [base.py](https://github.com/django/django/blob/main/django/views/generic/base.py): base `View` and `TemplateView`
-* [list.py](https://github.com/django/django/blob/main/django/views/generic/list.py): `ListView`
-* [edit.py](https://github.com/django/django/blob/main/django/views/generic/edit.py): `UpdateView`, `DeleteView`, `FormView`
-* [detail.py](https://github.com/django/django/blob/main/django/views/generic/detail.py): `DetailView`
-
-[Function-based views](https://docs.djangoproject.com/en/3.2/intro/tutorial01/#write-your-first-view)
-are used in cases where the CRUD/RESTful design pattern is not clear such as
-`notification_unsubscribe_key` where we unsubscribe an email via a GET operation.
-
-## [`main/views_api.py`](/main/views_api.py)
-
-This module contains all API related views. These views have their own
-api key based authentication.
-
-## [`main/views_export.py`](/main/views_export.py)
-
-This module contains all views related to the export capabilities of mataroa.
-
-The way the exports work is by reading the base files from the repository root:
-[export_base_hugo](export_base_hugo/), [export_base_zola](export_base_zola/),
-[export_base_epub](export_base_epub/) for Hugo, Zola, and epub respectively.
-After reading, we replace some strings on the configurations, generate posts
-as markdown strings, and zip-archive everything in-memory. Finally, we respond
-using the appropriate content type (`application/zip` or `application/epub`) and
-[Content-Disposition](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition)
-`attachment`.
-
-## [`main/views_billing.py`](/main/views_billing.py)
-
-This module contains all billing and subscription related views. It’s designed to
-support one payment processor, Stripe.
-
-## [`main/tests/`](/main/tests/)
-
-All tests are under this directory. They are divided into several modules,
-based on the functionality and the views they test.
-
-Everything uses the built-in Python `unittest` module along with standard
-Django testing facilities.
-
-## [`main/models.py`](/main/models.py) and [`main/migrations/`](/main/migrations/)
-
-`main/models.py` is where the database schema is defined, translated into
-Django ORM-speak. This always displays the latest schema.
-
-`main/migrations/` includes all incremental migrations required to reach
-the schema defined in `main/models.py` starting from an empty database.
-
-We use the built-in Django commands to generate and execute migrations, namely
-`makemigrations` and `migrate`. For example, the steps to make a schema change
-would be something like:
-
-1. Make the change in `main/models.py`. See
-[Django Model field reference](https://docs.djangoproject.com/en/3.2/ref/models/fields/).
-1. Run `python manage.py makemigrations` to auto-generate the migrations.
-1. Potentially refactor the auto-generated migration file (located at `main/migrations/XXXX_auto_XXXXXXXX.py`)
-1. Run `python manage.py migrate` to execute migrations.
-1. Also `make format` before committing.
-
-## [`main/forms.py`](/main/forms.py)
-
-Here a collection of Django-based forms resides, mostly in regards to user creation,
-upload functionalities (for post import or image upload), and card details
-submission.
-
-See [Django Form fields reference](https://docs.djangoproject.com/en/3.2/ref/forms/fields/).
-
-## [`main/templates/assets/style.css`](main/templates/assets/style.css)
-
-On Mataroa, a user can enable an option, Theme Zia Lucia, and get a higher font
-size by default. Because we need to change the body font-size value, we render
-the CSS. It is not static. This is why it lives inside the templates directory.

@@ 1,29 0,0 @@
-# Introduction
-
-Welcome to the documentation site of the
-[mataroa blog](https://github.com/mataroablog)
-project!
-
-**Main repository on GitHub**  
-[github.com/mataroablog/mataroa](https://github.com/mataroablog/mataroa)
-
-**Mirror repository on sr.ht**  
-[git.sr.ht/~sirodoht/mataroa](https://git.sr.ht/~sirodoht/mataroa)
-
-**Report bugs on GitHub**  
-[github.com/mataroablog/mataroa/issues](https://github.com/mataroablog/mataroa/issues)
-
-**Contribute on GitHub with Pull Requests**  
-[github.com/mataroablog/mataroa/pulls](https://github.com/mataroablog/mataroa/pulls)
-
-**Contribute (platform independent) with email patches**  
-[~sirodoht/public-inbox@lists.sr.ht](mailto:~sirodoht/public-inbox@lists.sr.ht)
-
-**Community mailing list**  
-[lists.sr.ht/~sirodoht/mataroa-community](https://lists.sr.ht/~sirodoht/mataroa-community)
-
-## Start
-
-Start learning about mataroa with reading the
-<a href="https://github.com/mataroablog/mataroa">main repository</a>
-README:

@@ 1,1 0,0 @@
-../../README.md>
\ No newline at end of file

@@ 1,114 0,0 @@
-# Runbook
-
-So, mataroa is down. What do we do?
-
-Firstly, panic. Run around in circles with your hands up in despair. It's important to
-do this, don't think this is a joke! Ok, once that's done:
-
-## 1. Check Caddy
-
-Caddy is the first point of contact inside the server from the outside world.
-
-First ssh into server:
-
-```sh
-ssh root@mataroa.blog
-```
-
-Caddy runs as a systemd service. Check status with:
-
-```sh
-systemctl status caddy
-```
-
-Exit with `q`. If the service is not running and is errored restart with:
-
-```sh
-systemctl restart caddy
-```
-
-If restart does not work, check logs:
-
-```sh
-journalctl -u caddy -r
-```
-
-`-r` is for reverse. Use `-f` to follow logs real time:
-
-```sh
-journalctl -u caddy -f
-```
-
-To search within all logs do slash and then the keyword itself, eg: `/keyword-here`,
-then hit enter.
-
-The config for Caddy is:
-
-```sh
-cat /etc/caddy/Caddyfile
-```
-
-One entry is to serve anything with *.mataroa.blog host, and the second is for anything
-not in that domain, which is exclusively all the blogs custom domains.
-
-The systemd config for Caddy is:
-
-```sh
-cat /etc/systemd/system/multi-user.target.wants/caddy.service
-```
-
-## 2. Check gunicorn
-
-After caddy receives the request, it forwards it to gunicorn. Gunicorn is what runs the
-mataroa Django instances, so it's named `mataroa`. It also runs as a systemd service.
-
-To see status:
-
-```sh
-systemctl status mataroa
-```
-
-To restart:
-
-```sh
-systemctl restart mataroa
-```
-
-To see logs:
-
-```sh
-journalctl -u mataroa -r
-```
-
-and to follow them:
-
-```sh
-journalctl -u mataroa -f
-```
-
-The systemd config for mataroa/gunicorn is:
-
-```sh
-cat /etc/systemd/system/multi-user.target.wants/mataroa.service
-```
-
-Note that the env variables for production live inside the systemd service file.
-
-## 3. How to hotfix code
-
-Here's where the code lives and how to access it:
-
-```sh
-sudo -i -u deploy
-cd /var/www/mataroa/
-source .envrc  # load env variables for manual runs
-source .venv/bin/activate  # activate venv
-python manage.py
-```
-
-If you make a change in the source code files (inside `/var/www/mataroa`) you need to
-restart the service for the changes to take effect:
-
-```sh
-systemctl restart mataroa
-```

@@ 1,36 0,0 @@
-# Server Migration
-
-Sadly or not, nothing lasts forever. One day you might do a server migration.
-Among many, mataroa is doing something naughty. We store everything, images
-including, in the Postgres database. Naughty indeed, yet makes it much easier to
-backup but also migrate.
-
-To start with, one a migrator has setup their new server (see
-[Deployment](./deployment.md)) we recommend testing everything in another
-domain, other than the main (existing) one.
-
-Once everything works:
-
-1. Verify all production variables and canonical server names exist in settings et al.
-1. Disconnect production server from public IP. This is not a zero-downtime migration — to be clear.
-1. Run backup-database.sh one last time.
-1. Assign elastic/floating IP to new server.
-1. Run TLS certificate (naked and wildcard) generations.
-1. `scp` database dump into new server.
-1. Restore database dump in new server.
-1. Start mataroa and caddy systemd services
-
-Later:
-
-1. Setup cronjobs / systemd timers
-1. Setup healthcheks for recurring jobs.
-1. Verify DEBUG is 0.
-
-The above assume the migrator has a floating IP that they can move around. If
-not, there are two problems. The migrator needs to coordinate DNS but much
-more problematically all custom domains stop working :/ For this reason we
-should implement CNAME custom domains. However, CNAME custom domains do not
-support root domains, so what's the point anyway you ask. Good question. I don't
-know. I only hope I never decide to switch away from Hetzner.
-
-Peace.