Trash: delete, restore and delete for good

README.md

@@ -60,8 +60,9 @@ Standard PostgreSQL environment variables (`PGHOST`, `PGPORT`, `PGUSER`,
 
 fmsg-webapi assumes [fmsgd](https://github.com/markmnl/fmsgd) is the fmsg host
 implementation and uses the same PostgreSQL database. The fmsgd schema remains
-the base schema; this repository's [dd.sql](dd.sql) adds the API-key and
-delegation tables used by fmsg-webapi.
+the base schema; this repository's [dd.sql](dd.sql) adds the API-key,
+delegation and [trash](#trash) tables used by fmsg-webapi. Apply it after the
+fmsgd schema, which it extends.
 
 A `.env` file placed in the working directory is loaded automatically at startup
 (values in the environment take precedence).
@@ -347,9 +348,10 @@ the application.
 | `POST`   | `/fmsg`                          | Create a draft message   |
 | `GET`    | `/fmsg/:id`                      | Retrieve a message       |
 | `PUT`    | `/fmsg/:id`                      | Update a draft message   |
-| `DELETE` | `/fmsg/:id`                      | Delete a draft message   |
+| `DELETE` | `/fmsg/:id`                      | Delete a draft; move a sent message to the trash, or delete it for good |
 | `POST`   | `/fmsg/:id/send`                 | Send a message           |
 | `POST`   | `/fmsg/:id/read`                 | Mark a message as read   |
+| `POST`   | `/fmsg/:id/restore`              | Take a message or thread out of the trash |
 | `POST`   | `/fmsg/:id/add-to`               | Add recipients           |
 | `POST`   | `/fmsg/:id/react`                | Set or clear an emoji reaction (FMSG-005) |
 | `GET`    | `/fmsg/:id/data`                 | Download message data    |
@@ -554,6 +556,10 @@ event types can be added without breaking clients:
 | `recipients_added` | The refreshed message object. | An add-to batch was recorded on a message the user participates in. |
 | `read` | The message object, with `read` and `time_read` set. | The user marked a message read (`POST /fmsg/:id/read`), on this or any other connection, so every tab and device can stop showing it as unread. Sent once, when the message first becomes read. |
 | `reaction` | The refreshed **subject** message object, with its `reactions` up to date. | A reaction (FMSG-005) on a message the user participates in arrives. The reaction message itself is not pushed as `new_msg` and triggers no Web Push. |
+| `trash` | The message object with its `trash` state, or `{"id": <int>, "trash": "purged"}` for a message deleted for good. | The user moved a message into or out of their [trash](#trash), or deleted it for good, on this or any other connection. |
+
+No event about a message the user deleted for good is ever pushed again,
+except that `trash` event.
 
 A client only ever receives events for messages it is a participant on. The
 server sends periodic WebSocket pings; clients should respond with pongs (most
@@ -589,6 +595,7 @@ has read the message.
 | --------- | ------- | ----------- |
 | `limit`   | `20`    | Max messages to return (at least 1; values above 100 return 100) |
 | `offset`  | `0`     | Number of messages to skip for pagination |
+| `trash`   | `exclude` | Messages in the caller's [trash](#trash): `exclude` leaves them out, `include` lists them too, `only` lists only them. Messages deleted for good are never listed. |
 
 **Response:** JSON array of message objects. Each object has the same shape as the single-message response from `GET /fmsg/:id` (with an additional `id` field), including the `read`/`time_read` fields reflecting the caller's per-recipient read state. Message body data and attachment contents are not included — use the dedicated download endpoints instead.
 
@@ -604,6 +611,7 @@ This includes both sent messages and drafts (`time_sent` may be `NULL`).
 | --------- | ------- | ----------- |
 | `limit`   | `20`    | Max messages to return (at least 1; values above 100 return 100) |
 | `offset`  | `0`     | Number of messages to skip for pagination |
+| `trash`   | `exclude` | As for `GET /fmsg`. |
 
 **Response:** JSON array of message objects with the same shape as `GET /fmsg`.
 
@@ -760,16 +768,59 @@ Updates a draft message. Only the owner (`from`) may update, and the message mus
 
 ### DELETE `/fmsg/:id`
 
-Deletes a draft message and all its attachments from the database and disk. Only the owner may delete, and the message must not have been sent.
+**A draft:** deletes it and all its attachments from the database and disk. Only
+the owner may delete a draft. **Response:** `204 No Content`.
 
-**Response:** `204 No Content`.
+**A sent message:** moves it into the caller's [trash](#trash). Its sender and
+any of its recipients may do this; it changes nothing for anyone else.
+
+| Parameter | Description |
+| --------- | ----------- |
+| `thread=true` | Delete the whole thread below this message, including replies that arrive later. |
+| `permanent=true` | Delete for good: this message and everything below it (with `thread=true`, the whole thread). |
+
+**Response:** `200 OK` with `{"id": <int>, "trash": "message" | "thread"}`, or
+for `permanent=true` `{"id": <int>, "trash": "purged", "purged": <int>, "removed": <int>}`:
+how many messages were newly deleted for good, and how many of those were
+removed from the host.
 
 **Errors:**
 
 | Status | Condition |
 | ------ | --------- |
-| `403`  | Not the owner, or message already sent |
-| `404`  | Message not found |
+| `403`  | A draft and not its owner |
+| `404`  | Message not found, the caller is not a participant, or it is already deleted for good |
+
+### POST `/fmsg/:id/restore`
+
+Takes a message out of the caller's [trash](#trash). With `thread=true`, takes
+the whole thread below it out. Messages deleted for good can't be restored.
+
+**Response:** `200 OK` with `{"id": <int>, "trash": null}`; `trash` is still
+`"thread"` when the message is in a deleted thread that wasn't restored.
+
+**Errors:** `404` when the message isn't found, the caller is not a
+participant, or it is deleted for good.
+
+### Trash
+
+Each identity has its own trash; nothing in it changes what anyone else sees.
+
+- **Deleted** messages (`"trash": "message"`) and messages in a deleted thread
+  (`"trash": "thread"`) are left out of `GET /fmsg` and `GET /fmsg/sent` unless
+  `trash=include` or `trash=only` is given, and carry their `trash` state on
+  every route. Replies keep arriving: a reply in a deleted thread is in the
+  trash too, and triggers no [Web Push](#web-push).
+- **Deleted for good** (`permanent=true`) is final. The message and everything
+  that replies to it are never listed again for that identity, every route
+  answers `404` for them (or shows them as not visible in a thread), and a
+  reply to them is refused (`400 parent message not found`). What they counted
+  against that identity's storage quota is given back to fmsgid. Once no local
+  participant holds a message any more (none sent or accepted it without
+  deleting it for good) and none of its deliveries is pending, it is removed
+  from the host: its rows and files are deleted, and replies to it from
+  anywhere are then rejected by fmsgd as parent not found. This needs the
+  fmsgd schema that lets a transaction set `fmsg.purge`.
 
 ### Message hashes and references
 
@@ -1130,7 +1181,8 @@ The push carries a JSON payload:
 ```
 
 `threadId`, `url` and `tag` reference the thread's **root** message id, so
-replies group with their thread. If a push service responds `404` or `410` the
+replies group with their thread. A message in the recipient's
+[trash](#trash), such as a reply in a thread they deleted, is not pushed. If a push service responds `404` or `410` the
 subscription is dead and is deleted automatically.
 
 **Enabling.** Web Push is active only when `FMSG_VAPID_PUBLIC_KEY`,

cmd/fmsg-webapi/apikey_cli_integration_test.go

@@ -51,6 +51,18 @@ func newAPIKeyCLIDB(t *testing.T) *db.DB {
 		t.Fatal(err)
 	}
 	t.Cleanup(pool.Close)
+	// This service's dd.sql extends the fmsgd schema.
+	fmsgdDD := os.Getenv("FMSG_TEST_DD")
+	if fmsgdDD == "" {
+		fmsgdDD = "../../../fmsgd/dd.sql"
+	}
+	base, err := os.ReadFile(fmsgdDD)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if _, err := pool.Exec(ctx, string(base)); err != nil {
+		t.Fatal(err)
+	}
 	dd, err := os.ReadFile("../../dd.sql")
 	if err != nil {
 		t.Fatal(err)

cmd/fmsg-webapi/main.go

@@ -284,6 +284,7 @@ func registerRoutes(router *gin.Engine, jwtMiddleware gin.HandlerFunc, msgHandle
 		fmsg.DELETE("/:id", msgHandler.Atomic((*handlers.MessageHandler).Delete))
 		fmsg.POST("/:id/send", msgHandler.Atomic((*handlers.MessageHandler).Send))
 		fmsg.POST("/:id/read", msgHandler.MarkRead)
+		fmsg.POST("/:id/restore", msgHandler.Atomic((*handlers.MessageHandler).Restore))
 		fmsg.POST("/:id/add-to", msgHandler.Atomic((*handlers.MessageHandler).AddRecipients))
 		fmsg.POST("/:id/react", msgHandler.Atomic((*handlers.MessageHandler).React))
 		fmsg.GET("/:id/data", msgHandler.DownloadData)

dd.sql

@@ -72,3 +72,37 @@ CREATE INDEX IF NOT EXISTS fmsg_api_sub_account_sub_idx
 CREATE UNIQUE INDEX IF NOT EXISTS fmsg_api_sub_account_owner_sub_unique
     ON fmsg_api_sub_account ((lower(owner_addr)), (lower(sub_addr)))
     WHERE agent <> '';
+
+-- Trash: what each address has deleted. A row marks one message for one
+-- address (lower-cased). thread marks the whole thread below that message;
+-- time_purged means deleted for good: never shown to that address again,
+-- nor anything that replies to it. Rows go with their message when it is
+-- removed from the host.
+CREATE TABLE IF NOT EXISTS fmsg_api_trash (
+    addr text NOT NULL,
+    msg_id bigint NOT NULL REFERENCES msg (id) ON DELETE CASCADE,
+    thread boolean NOT NULL DEFAULT false,
+    time_deleted double precision NOT NULL,
+    time_purged double precision,
+    PRIMARY KEY (addr, msg_id)
+);
+
+-- fmsg_api_trash_state is where a message stands for an address: 'purged'
+-- when it or an ancestor is deleted for good, 'thread' when it or an ancestor
+-- carries a thread mark, 'message' when it is deleted itself, NULL when it
+-- isn't deleted. addr must be lower-cased.
+CREATE OR REPLACE FUNCTION fmsg_api_trash_state(p_addr text, p_msg bigint) RETURNS text
+LANGUAGE sql STABLE AS $$
+    WITH RECURSIVE up(id, depth) AS (
+        SELECT p_msg, 0
+        UNION ALL
+        SELECT m.pid, up.depth + 1 FROM msg m JOIN up ON m.id = up.id
+        WHERE m.pid IS NOT NULL AND up.depth < 1000
+    )
+    SELECT CASE
+        WHEN bool_or(t.time_purged IS NOT NULL) THEN 'purged'
+        WHEN bool_or(t.thread) THEN 'thread'
+        WHEN bool_or(up.depth = 0) THEN 'message'
+    END
+    FROM up JOIN fmsg_api_trash t ON t.addr = p_addr AND t.msg_id = up.id
+$$;

go.mod

@@ -10,7 +10,7 @@ require (
 	github.com/gorilla/websocket v1.5.3
 	github.com/jackc/pgx/v5 v5.8.0
 	github.com/joho/godotenv v1.5.1
-	github.com/markmnl/fmsgd v0.6.1-0.20261010085601-c99adfef703e
+	github.com/markmnl/fmsgd v0.6.1-0.20261022024806-f5515e09b914
 	golang.org/x/sync v0.20.0
 )
 

go.sum

@@ -60,8 +60,8 @@ github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
 github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
 github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
 github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
-github.com/markmnl/fmsgd v0.6.1-0.20261010085601-c99adfef703e h1:VQf6zdJ1Hqd4+GhWnHgOwIGd64Z0j3OMJ9Z3046Bm0Y=
-github.com/markmnl/fmsgd v0.6.1-0.20261010085601-c99adfef703e/go.mod h1:CiTt3R2Fw3zhEV8SDB7Ml1efdmYom5C/QOBT7GafEAM=
+github.com/markmnl/fmsgd v0.6.1-0.20261022024806-f5515e09b914 h1:qSAvLtudh4oEwCOcgl89DWqm1loARku7ZMW9DjicKc0=
+github.com/markmnl/fmsgd v0.6.1-0.20261022024806-f5515e09b914/go.mod h1:CiTt3R2Fw3zhEV8SDB7Ml1efdmYom5C/QOBT7GafEAM=
 github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
 github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
 github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=

internal/handlers/attachments.go

@@ -198,7 +198,7 @@ func (h *AttachmentHandler) Download(c *gin.Context) {
 		}
 		return
 	}
-	if draftHidden([]string{identity}, fromAddr, timeSent) {
+	if draftHidden([]string{identity}, fromAddr, timeSent) || purgedFor(ctx, h.q(), identity, msgID) {
 		c.JSON(http.StatusNotFound, gin.H{"error": "message not found"})
 		return
 	}

internal/handlers/finalization_integration_test.go

@@ -75,6 +75,14 @@ func newFinalizationAPIWithID(t *testing.T, fmsgID http.Handler) *finalizationAP
 	if _, err = pool.Exec(ctx, string(sql)); err != nil {
 		t.Fatal(err)
 	}
+	// This service's own tables (dd.sql at the repository root).
+	apiSQL, err := os.ReadFile("../../dd.sql")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if _, err = pool.Exec(ctx, string(apiSQL)); err != nil {
+		t.Fatal(err)
+	}
 	id := httptest.NewServer(fmsgID)
 	t.Cleanup(id.Close)
 	h := NewMessageHandler(&db.DB{Pool: pool}, t.TempDir(), 1<<20, 2<<20, 256, nil, id.URL, "example.com")
@@ -91,6 +99,7 @@ func newFinalizationAPIWithID(t *testing.T, fmsgID http.Handler) *finalizationAP
 	r.GET("/fmsg/:id/data", h.DownloadData)
 	r.POST("/fmsg/:id/send", h.Atomic((*MessageHandler).Send))
 	r.POST("/fmsg/:id/read", h.MarkRead)
+	r.POST("/fmsg/:id/restore", h.Atomic((*MessageHandler).Restore))
 	r.POST("/fmsg/:id/add-to", h.Atomic((*MessageHandler).AddRecipients))
 	r.POST("/fmsg/:id/react", h.Atomic((*MessageHandler).React))
 	r.GET("/fmsg/:id/thread", h.ThreadText)

internal/handlers/hub.go

@@ -20,6 +20,7 @@ const (
 	eventRecipientsAdded = "recipients_added"
 	eventReaction        = "reaction"
 	eventRead            = "read"
+	eventTrash           = "trash"
 )
 
 // wsEnvelope is the JSON shape of every frame pushed over a WebSocket. The
@@ -144,7 +145,10 @@ func (h *Hub) listen(ctx context.Context, onConnected func()) error {
 	if _, err := conn.Exec(ctx, "LISTEN msg_read"); err != nil {
 		return err
 	}
-	log.Println("ws hub: listening on new_msg, delivered, recipients_added, msg_read")
+	if _, err := conn.Exec(ctx, "LISTEN msg_trash"); err != nil {
+		return err
+	}
+	log.Println("ws hub: listening on new_msg, delivered, recipients_added, msg_read, msg_trash")
 	onConnected()
 
 	for {
@@ -191,6 +195,11 @@ func (h *Hub) route(ctx context.Context, channel string, msgID int64, addr strin
 		// recipient. Pushes the message with its read state so the reader's
 		// other tabs and devices stop showing it as unread.
 		h.dispatch(ctx, msgID, addr, eventRead)
+	case "msg_trash":
+		// The address moved the message into or out of its trash, or
+		// deleted it for good (see trash.go). Its other tabs and devices
+		// get the message with its trash state.
+		h.dispatchTrash(ctx, msgID, addr)
 	case "recipients_added":
 		// An add-to batch was recorded against the message; addr is one
 		// of its participants (see notify_recipients_added in fmsgd's
@@ -244,7 +253,49 @@ func (h *Hub) dispatch(ctx context.Context, msgID int64, addr string, eventType
 		log.Printf("ws hub: build message %d for %s: %v", msgID, addr, err)
 		return
 	}
-	payload, err := json.Marshal(wsEnvelope{Type: eventType, Data: item})
+	if item.purged {
+		return // deleted for good by addr: never shown to it again
+	}
+	h.send(clients, msgID, wsEnvelope{Type: eventType, Data: item})
+}
+
+// trashGone is the trash event data for a message deleted for good.
+type trashGone struct {
+	ID    int64  `json:"id"`
+	Trash string `json:"trash"`
+}
+
+// dispatchTrash pushes a trash event to addr's clients: the message with its
+// trash state, or only its id when addr deleted it for good (it may no longer
+// exist).
+func (h *Hub) dispatchTrash(ctx context.Context, msgID int64, addr string) {
+	clients := h.clientsOf(addr)
+	if len(clients) == 0 {
+		return
+	}
+	item, err := h.buildItem(ctx, msgID, addr)
+	if err != nil || item.purged {
+		h.send(clients, msgID, wsEnvelope{Type: eventTrash, Data: trashGone{ID: msgID, Trash: trashPurged}})
+		return
+	}
+	h.send(clients, msgID, wsEnvelope{Type: eventTrash, Data: item})
+}
+
+// clientsOf lists the clients connected as addr.
+func (h *Hub) clientsOf(addr string) []*wsClient {
+	h.mu.RLock()
+	defer h.mu.RUnlock()
+	set := h.registry[strings.ToLower(addr)]
+	clients := make([]*wsClient, 0, len(set))
+	for c := range set {
+		clients = append(clients, c)
+	}
+	return clients
+}
+
+// send marshals env and queues it on each client.
+func (h *Hub) send(clients []*wsClient, msgID int64, env wsEnvelope) {
+	payload, err := json.Marshal(env)
 	if err != nil {
 		log.Printf("ws hub: marshal message %d: %v", msgID, err)
 		return