# Nuntius, the `notify` module (plan NS)

The send path: one transactional SMS, its receipt and its count, and (plan NU) what a text costs, bulk sends, sender names, products' e-mail and who isn't written to. It lives in `app/Modules/Notify/`. WhatsApp, inbound replies and STOP by SMS wait for Desk (NU-10).

| Table | What a row says | Platform or scoped |
|---|---|---|
| `sms_provider` | A connected provider. Credentials are encrypted and hidden, and a partial unique index keeps one sending at a time | Platform |
| `sms_sender` (`snd_…`) | A sender name with one provider: an organisation's, or the company's with no organisation. Per provider, because each approves its own. An organisation's walks `to_register`, `submitted`, `registered` or `refused`, and `replaced` once a newer name is registered there (plan NU, NU4); it keeps who asked (a person or a product's server) and when it was submitted and registered. Audited | Platform, naming its organisation |
| `sms_template` | A text's wording, with its secret tokens, and whether it's exempt from a client's limit. Starts with `sign_in_code` and `password_reset_code` | Platform |
| `message` (`msg_…`) | One text, however many reports arrive. A trigger freezes what it said and to whom, and its outcome once it has one; it leaves only through the purge (`log.message_days`, 180 by default) | Platform, naming its organisation and server |
| `sms_report` | What a provider reported, once per delivery ID. Kept 30 days (`OldNotifyRecords`) | Platform |
| `product_email` (`pem_…`) | One e-mail a product sent (plan NU, NU5): who it was from, its kind (`transactional` or `announcement`), its words as drawn, and its recipients on the one row (`[{e, n, state, mailer, reason, at}]`, never a row each), with the status they add up to (`queued`, `sent`, `partly_sent`, `failed`). A trigger freezes what it said and who it was from; an erasure empties it through the logs' second door; it leaves through the purge (`log.product_email_days`, 180) | Platform, naming its organisation and server |
| `send_block` (`blk_…`) | An address or number not to be written to, now (plan NU, NU6): `email` or `sms`, why (`bounced`, `complained`, `soft_bounces`, `by_hand`: every organisation; `unsubscribed`: one organisation's announcements, the database holding that shape), the provider's words and the product e-mail it came from. One a channel, address and organisation | Platform, an unsubscribe naming its organisation |
| `send_block_line` | Every change to it (`blocked`, `lifted` with a reason from Input Fields `block_lift_reason`, `soft_bounce`, `resubscribed`), who made it; append-only and kept for good, since a suppression list must outlive what it suppresses | Platform |
| `notify_event` | Each SES event once, by SNS's message ID; swept after 30 days with `sms_report` | Platform |
| `message_batch` (`bat_…`) | One bulk send (plan NU, NU3): its template, how many it was for, what it reserved, its state (`sending`, `finished`, `cancelled`, `refused`), and `sent_through`, the cursor of the last text handed over. Each recipient is a `message` with `batch_id`, tied to the same organisation by a composite key | Platform, naming its organisation and server |

## States

| State | Means | Charged? |
|---|---|---|
| `queued` | Waiting for a worker | Not yet |
| `sent` | The provider accepted it | Yes |
| `delivered`, `undelivered` | The provider's report, or its answer when asked | Yes |
| `failed` | Never sent: refused, over the limit, no sender name, or no provider | No |
| `unknown` | The provider may have it and didn't say. Never retried, never counted as failed | Not known |

## The pieces

| Piece | What it does |
|---|---|
| `Nuntius::template` | The call for Atrium's own modules and `/v1`. Fills the template, blanks secret tokens in the row, reserves against the limit, queues the send. Refuses an exempt template from a server with no product. From a product's server an exempt (company-paid) template takes only digits for `code` (4 to 8) and `minutes`, a `host` that is one of that product's own live hosts for the organisation, and Atrium's own `name` (the organisation's); and no more than `SignInCodes::PER_NUMBER_HOUR` to one number in an hour from one server (plan SEC, SEC4 F1). A retry key is the server's within one organisation (`message_once`, F8) |
| `Nuntius::finishedText` | A text its caller wrote, only for a server with `sends_finished_text` (plan NS-4), set by `notify:finished-text` and audited |
| `Nuntius::sendAgain` | A new message pointing at the first (`resend_of`), naming who asked. Refused for a code, and for a text that was sent and hasn't failed |
| `SendSms` | The encrypted job for one text: the only place a code's real words travel. It retries only if nothing reached the provider |
| `Handover` (NU7) | Before the provider hears of a text it's claimed (`message.handed_at`, a conditional update): a run that finds the claim moves the text to `unknown` and never sends it twice; an answer that nothing reached the provider takes the claim back. Every request passes one count a second per provider (`per_second`), bulk, single and codes together; a provider's 429 is `unreachable`, a wait, not a refusal |
| `Handover` | One text handed to the provider, the same for `SendSms` and `SendBatch`: the sender name and the company fallback, the answer moved onto the message. A provider out of reach leaves the text queued for the caller to try again |
| `SmsDriver`, `SailupDriver`, `ArkeselDriver` | Lifted from TuteCampus. `SendResult::fromConnection` separates a read timeout (`unknown`) from never connecting (`unreachable`), the fault plan NS §2 found in Campus. A 5xx is also `unknown` |
| `MessageReports` | Settles a text. Once only by delivery ID, and only from `sent` or `unknown`. A report matching no ID is matched to an `unknown` text by number within `UNKNOWN_MINUTES`, oldest first, and that gives it its ID |
| `MessageChanges` | The one writer of a state. It also raises `MessageChanged`, a webhook to the sending server alone (`NotificationEvent::webhookServer`) |
| `SenderNames` | Asking for a name (`ask`: with every connected provider; the same again changes nothing, a different one deletes what was still waiting, never what is registered), the Service's record of each provider's answer (`move`, by `SenderNames::MOVES`; a refusal needs the provider's reason; registering one leaves the older registered name with that provider `replaced`), the company's own (`company`), and `summary`: the latest name asked for, each provider's state and the name its texts go under today (plan NU, NU4) |
| `Senders` | Which name a text goes under. The client's newest registered with this provider. Otherwise the company's: always for a code or for Atrium's own, by the client's `notify.sender_fallback` otherwise, and never for a server with no product |
| `ChannelReport` | The Channels page: balance, days left, delivery rate, and whether reports have stopped |

## Limits

A text is ten segments at most (`Message::MOST_SEGMENTS`), alone or in a bulk send (NU7). A server may send `notify.codes_per_hour` company-paid codes in an hour (600), whatever the numbers, beside `SignInCodes::PER_NUMBER_HOUR` to one number.


`usage_limit` with `notify` and `message` is enforced inside `Nuntius::queue`: the limit's row is locked while this month's client-paid, not-failed messages are counted, so two sends can't both take the last one. Over it, the row is written as `failed` and `POST /v1/notify/sms` answers 422 with that row. A code is exempt, and doesn't count. `usage_event` is the record of what was used, not the brake: it counts on `sent`, or when a report settles an `unknown`, with the message's ID as its reference.

## What a text costs (plan NU, NU1)

While `notify.charge_texts` is on (off by default, a client able to differ; the Architect switches it on with the price), a client's text
costs one credit a segment (`Credit::segments`: GSM-7 160, or 153 a part; otherwise UCS-2 70, or 67 a part; extension characters count
twice). Inside `Nuntius::queue`'s transaction, under the organisation's `sms_credit` row lock (`Credit::payer`), it's paid for out of its
product's allowance this calendar month (the offering's `texts_a_month` allowance, `Allowance::TEXTS`, whole texts only; none named is
none, never unlimited), else out of the organisation's credit, shared by its products (`Credit::spend`, a `spent` line); with neither,
it's written `failed` with `notify.reason.no_credit` and `POST /v1/notify/sms` answers 422. `message.segments` and `message.paid_from`
(`allowance` or `credit`) record it. A text that ends `failed` (never left: `MessageChanges::finish`, or a report saying so) is given back
once (`Credit::giveBack`, a `given_back` line; `sms_credit_line_once` holds it); `unknown`, `undelivered` and `expired` keep their charge
(NS-5). Codes are never charged (NS-7); the Service's sending again is on the company.

`sms_credit` (one row an organisation: `balance`, `low_told_at`, `out_told_at`) and `sms_credit_line` (`bought`, `spent`, `given_back`,
`adjusted`; signed `amount`, `balance_after`, the message or payment it came from, a reason from Input Fields `credit_reason`, a note,
who) are changed only by `Credit`. A line is never edited; the balance never goes below nothing (a CHECK), and taking more than is there
takes what is there. Falling below `notify.credit_low` (200 texts) tells the owners and billing people and the organisation's products
once (`notify.credit_low`), and running out once (`notify.credit_out`); buying or adding credit over the mark starts them again.

Credit is bought at `id` (NU2: `/billing/texts/buy`, `billing.manage`) in packs the Service prices in the console; see `subscription.md`, "SMS credit bought". The Service adds or takes credit by hand on a client's page in the console (`POST /console/client/{tenant}/credit`, `workspace.create`
and the password; reason and note required and kept). The organisation's owners and billing people see it at `id` under Texts
(`/billing/texts`, `billing.view`).

## Bulk sends (plan NU, NU3; NU-5)

`POST /v1/notify/batch` takes `template`, shared `values`, and `recipients` (1 to 10,000, each `to` and its own `values`, over the shared
ones). `Batches::start` checks every recipient before anything is kept: a bad number or a missing value refuses the whole send with the
first twenty problems named by place (`recipients.3.to`); the same words to the same number twice go once (`duplicates`). A company-paid or
secret template (a code) is refused: a code is one person's, and a send's words are kept on each row. Then, in one transaction: the month's
`usage_limit` for all the texts at once (`Nuntius::overLimit`), and while texts are charged, `Credit::payers` under the credit lock (the
product's allowance takes each text it still covers, the credit the rest). Short of either, the send is kept as `refused` with its reason
and no message, and the API answers 422; it never sends half. Otherwise one `message` row a recipient is written in thousands, the credit
lines with them (`Credit::spendBatch`), and `SendBatch` goes on the `bulk` pool.

`SendBatch` hands texts over for a minute a run, each at least 1/`per_second` of a second after the last (`sms_provider.per_second`, 20 by
default, set on the provider's connect page), so no second ever holds more and a slow answer is never made up with a burst; then it queues
itself again, so another send gets its turn. One run at a time per provider (a cache lock), however many workers. Before each text the
cursor moves to it, only while the send is still `sending`, so a cancel either came first or waits. A run that dies resumes after the
cursor; the text it was handing over may have gone, so it becomes `unknown` and is never sent twice (NS-5). A provider out of reach took
nothing: the cursor goes back and the same text goes again after `SendSms::BACKOFF`; after the last wait, what's left fails with the reason
and is given back. `notify:check` picks up a send quiet for 20 minutes (`NotifyCheckCommand::QUIET`, longer than the longest wait).

`DELETE /v1/notify/batch/{batch}` (or the console) cancels: every text after the cursor fails as `notify.reason.cancelled`, its cost goes
back at once (`Credit::giveBackBatch`, a `given_back` line each), and `cancelled` counts them; the one in the provider's hands finishes as it
would have. When nothing is left waiting, the send is `finished` and its server hears `notify.batch_finished` once, with the counts
(`total`, `queued`, `sent`, `delivered`, `undelivered`, `failed`, `unknown`, `cancelled`). A bulk send's texts raise no `message.changed`
each: `GET /v1/notify/batch/{batch}` reads the counts, and `GET /v1/notify/batch/{batch}/message?state=failed` lists recipients a hundred to
a page by cursor, to learn who didn't get it. A server reads and cancels only its own, in the same organisation.

Sending is one text at a time per provider, so the provider's answer time is the real ceiling: at 200 ms an answer, about five a second
whatever the pace. Sending several at once (an HTTP pool in the driver) waits for a measurement against the real provider.

## Products' e-mail (plan NU, NU5; NU-7)

`POST /v1/notify/email` takes `to` (1 to 50, each `email` and an optional `name`), `subject`, `body` (the house renderer's words: a
paragraph a line, `**bold**`, `*italic*`, `` `code` ``, links; everything else escaped), an optional `action` (`label`, an https `url`) and
`kind` (`transactional` or `announcement`, which an unsubscribe stops; required since NU7, so leaving it out can't skip an unsubscribe). Only a server allowed its own wording sends
(`sends_finished_text`, NS-4; a 403 otherwise): no product's templates live in Atrium yet. `ProductMail::send` draws the words once, keeps
them, and queues `SendProductEmail`; an `Idempotency-Key` is kept on the row for good. Each recipient gets an e-mail of their own, so nobody
sees who else it went to, from the product's own address (`no-reply@notify.<product domain>`, `config/notify.php`), named for the
organisation, with `X-SES-CONFIGURATION-SET` set to the product's code (so one product's bounces never harm another's) and `X-Atrium-Email`
naming the row. A product with no domain can't send (422, `from`).

The chain (`notify.mail.chain`, from `NOTIFY_MAIL_CHAIN`) is the mailers tried in order, each a mailer in `config/mail.php`: `ses_smtp` is
SES through its SMTP interface, so no SDK is installed. `ProductMail::chain` puts first those with quota left this month
(`notify.mail.limits`, counted from the recipients each mailer took since its month began, `notify.mail.reset_days`); with every one spent,
e-mail goes anyway. For each recipient not yet reached the first mailer that accepts takes it, and it's saved at once, so a run that dies
never sends to that recipient again and the next one's quota counts it; a mailer that throws hands it to the next. With none taking it, the
job waits (1, 5, 15 minutes), only for those not reached, then marks them `failed` with the last reason. Each recipient reached is counted
(`usage_event`, `notify`/`email`); e-mail is never charged. When every recipient has an outcome the server hears `notify.email_finished`
(the e-mail, its status, how many sent and failed; never the words or the addresses). `GET /v1/notify/email/{email}` gives each
recipient's state, for the server that sent it.

## Who isn't written to (plan NU, NU6; NU-8, NU-9, NU-12)

`Blocks` keeps the list, keyed by the address or number (most parents have no account). A permanent bounce, a complaint, three soft
bounces in a week (`Blocks::SOFT_LIMIT`) or the Service's hand stops every product and organisation; an unsubscribe stops one
organisation's announcements. `Blocks::stops` is asked for each recipient at the moment of sending, not when queued: by
`SendProductEmail` (an announcement also stops for its organisation's unsubscribe; a transactional e-mail only for the rest; the recipient
becomes `blocked` with why) and by `Handover` for every text, a code included (failed with why, its cost given back). Numbers are blocked by
the Service's hand for now; a provider's "not a phone" will feed it once a real provider's reports are known.

SES reports products' bounces and complaints through SNS to `POST /v1/notify/report/ses` (`SesReports`): only a topic on
`notify.ses.topics` (`NOTIFY_SES_TOPICS`) is believed, and only with Amazon's signature (versions 1 and 2) checked against the certificate
at its `SigningCertURL`, which must be an `sns.<region>.amazonaws.com` address; a 403 otherwise (Campus believed anyone's topic). A
subscription confirmation is answered at its `SubscribeURL`, an SNS host only. Each event is applied once by its message ID
(`notify_event`), kept and applied in one transaction; a message older than an hour (`SesReports::FRESH_SECONDS`) is refused as replayed (NU7). A permanent bounce blocks with SES's diagnostic; a transient one counts; a complaint blocks. The e-mail named by its
`X-Atrium-Email` header learns it: a `sent` recipient becomes `bounced` or `complained`, never back.

An announcement carries `List-Unsubscribe` with one-click (`List-Unsubscribe-Post`, RFC 8058) and a footer link, each to
`/unsubscribe/{token}` at `id`: the token is the address and organisation encrypted (`Blocks::token`), so no address sits in a URL. The page
needs no sign-in (`UnsubscribeController`, `notify/Unsubscribe.vue` in `QuietShell`): it asks before acting, since mail scanners open links,
says what still comes, and undoes in one press (`resubscribe`); a mail client's one-click POST acts at once and gets a 200. CSRF is skipped
for `unsubscribe/*` (the token is the proof) and the named limiter `unsubscribe` counts per address.

## The API

`POST /v1/notify/sms` takes `to` and either `template` with `values`, or `text` with `title`. It needs a key with the `notify` scope and `X-Organisation`. An `Idempotency-Key` is kept on the message's row for good, so a retry after the middleware's 24 hours still finds the first. It answers 201 with `public_id` and `state`.

`GET /v1/notify/message/{message}` is for the server that sent it, naming the same organisation; anyone else gets a 404.

`POST /v1/notify/batch`, `GET` and `DELETE /v1/notify/batch/{batch}`, and `GET /v1/notify/batch/{batch}/message` are the bulk sends above.

`POST /v1/notify/email` and `GET /v1/notify/email/{email}` are products' e-mail above.

`POST /v1/notify/sender` (NU4) asks for a sender name for the organisation (`name`, 3 to 11 letters, digits and spaces starting with a
letter; `SenderNames::PATTERN`) and answers 202; `GET /v1/notify/sender` says where it stands: `name`, `state` (`none`, `pending`,
`registered` with every provider, `refused` by any) and `sends_as`, the name a text goes under today through the sending provider. Neither
names a provider or the person who asked. `notify.sender_changed` says the same by webhook when a provider registers or refuses it.

`GET /v1/notify/balance` (NU1) answers whether texts are charged, this product's allowance this month (`included`, `used`, `left`) and the organisation's `credit`, so a product can show "top up SMS"; `notify.credit_low` and `notify.credit_out` say so by webhook when it crosses.

To hear what becomes of a text, subscribe an address to the `message.changed` webhook. Its payload is `message`, `state` and `reason`, never the words or the number.

## Provider reports

| Address | Believed because |
|---|---|
| `POST /v1/notify/report/sailup` | `X-Sailup-Signature` is an HMAC of `timestamp.body` under the webhook secret. A 401 otherwise |
| `POST /v1/notify/report/ses` (NU6) | An allow-listed SNS topic and Amazon's signature on the certificate at an SNS address. A 403 otherwise |
| `GET`, `POST /v1/notify/report/arkesel` | It isn't: the callback only prompts Atrium to ask Arkesel, so a forged one can't set an outcome |

`notify:check`, every 10 minutes, asks about texts accepted over 5 minutes ago and within 3 days, and notes each provider's balance. Only a report that arrived sets `last_report_at`, so asking doesn't hide reports having stopped.

## Commands

| Command | What it does |
|---|---|
| `notify:sender {driver} {name} [--client=org_…]` | Records an approved sender name, for a host with no console at hand: the console's Sender names page does the same (NU4) |
| `notify:finished-text {server} [--off]` | Lets a server send its own wording, or takes that back |
| `notify:check` | The sweep above; bulk sends gone quiet; single texts queued over an hour (never handed over: failed and given back; handed over: `unknown`); give-backs a crash lost (`Credit::sweepGiveBacks`) (NU7) |

## Settings

`notify.sender_fallback` (a client's own, on by default), `notify.short_links` (off, with a client able to differ: a text's links shortened through Tutelink, `links.md`), `notify.reports_per_minute`, `notify.charge_texts` (off; the Service's alone, one client differing only by its exception: NU1, NU7), `notify.codes_per_hour` (600, NU7), `notify.credit_low` (200 texts), `log.message_days`, `log.product_email_days` (NU5). Products' e-mail's chain, limits and sending address are `config/notify.php`, from the environment.

## Not done, and why

- **No real provider has been called.** Every test uses a fake, and Sailup's answers are as Campus's code reads them. The first real send needs the company's key in the console.
- **Nothing calls it yet.** Campus's gateway is Campus's own plan (NS-9), and Clavis brings the codes.
- **`message.to_number` has no privacy part.** The numbers so far are a product's people, kept for `log.message_days`. When Clavis sends to Atrium's own people, it needs one, as `email_message` has.