hot.dev/google-gmail / ::google::gmail::sync

Keep up with a mailbox: the profile (current historyId), the history feed for incremental sync, and push notifications through Google Cloud Pub/Sub (watch, stop-watch, and parsing/verifying the push request).

Sync loop

1. Once: store get-profile().historyId (or the historyId from watch).
2. Each run (schedule or push): set ::sync/sync-history(saved), handle set.added-message-ids / set.deleted-message-ids / set.history, then store set.history-id.
3. If the stored id is too old Gmail answers 404 (is-history-expired): do a full sync with ::google::gmail::messages/list-all-messages and store a fresh historyId.

Push setup (one time, in Google Cloud)

1. Create a Pub/Sub topic, e.g. projects/my-project/topics/gmail.
2. Grant gmail-api-push@system.gserviceaccount.com the Pub/Sub Publisher role on it.
3. Create a push subscription on the topic whose endpoint is your Hot webhook URL with ?token=<secret> appended; set the same secret as the google.gmail.push.token context variable.
4. Call watch("projects/my-project/topics/gmail") for each mailbox, and again at least every 7 days (Google recommends daily) from a Hot schedule: a watch expires after 7 days.

Pushes say only "the mailbox changed up to historyId N": run sync-history from the stored id. They can arrive late, twice, or out of order.

Scopes. gmail.readonly for everything here (gmail.metadata also works).

Values

MAX_HISTORY_PAGE

MAX_HISTORY_PAGE: Int 500

Functions

build-watch

fn (topic-name: Str, opts: Map): Map

The users.watch body (pure): {topicName} plus labelIds (e.g. ["INBOX"]) and labelFilterBehavior ("include" or "exclude") from opts (label-ids / label-filter-behavior, or Gmail's own names).

get-profile

fn (): Map
fn (opts: Map): Map

GET /users/me/profile - {emailAddress, messagesTotal, threadsTotal, historyId} for the authenticated user. Scope: gmail.readonly.

history-loop

fn (start: Str, opts: Map, page-token: Str?, acc: Vec, pages: Int): HistorySet

history-params

fn (start-history-id: Any, opts: Map): Map

The query parameters list-history sends (pure): startHistoryId, then your options: historyTypes (a Vec of messageAdded, messageDeleted, labelAdded, labelRemoved; sent repeated), labelId, maxResults (max 500), pageToken.

is-history-expired

fn (result: Result): Bool

True when a list-history / sync-history result is Gmail's 404 for a start id that is too old (typically older than about a week): do a full sync.

list-history

fn (start-history-id: Any): Map
fn (start-history-id: Any, opts: Map): Map

GET /users/me/history - one page of mailbox changes after start-history-id: {history: [...], historyId, nextPageToken?}. Each history record has id, messages, and messagesAdded, messagesDeleted, labelsAdded, labelsRemoved. A 404 means the id is too old (is-history-expired). Scope: gmail.readonly.

parse-push

fn (request: Map): PushNotification

Parse a Pub/Sub push request into a PushNotification (pure): the envelope {message: {data, messageId, publishTime, attributes}, subscription} with data the base64 JSON {emailAddress, historyId}. Returns an Err for a body that is not a push envelope. Does not authenticate: use verify-push.

record-ids

fn (records: Vec, key: Str): Vec

request-json

fn (request: Map): Any

stop-watch

fn (): Bool
fn (opts: Map): Bool

POST /users/me/stop - stop push notifications for this mailbox. Returns true. Scope: gmail.readonly.

summarize-history

fn (records: Vec, history-id: Str, pages: Int): HistorySet

Build a HistorySet from history records and the latest history id (pure).

sync-history

fn (start-history-id: Any): HistorySet
fn (start-history-id: Any, opts: Map): HistorySet

Read every page of history after start-history-id and return a HistorySet with the id to store next. Uses maxResults: 500 unless set. On an expired id this returns the 404 Err (check is-history-expired).

::sync ::google::gmail::sync
::messages ::google::gmail::messages

set ::sync/sync-history(saved-history-id)
cond {
  ::sync/is-history-expired(set) => { full-resync() }
  => {
    new-mail map(set.added-message-ids, (id) { ::messages/get-email(id) })
    set.history-id
  }
}

verify-push

fn (request: Map): PushNotification
fn (request: Map, expected: Str): PushNotification

Verify and parse a Gmail push: returns the PushNotification, or an Err when the ?token= secret on the push endpoint is missing or wrong. The 1-arity form checks against the google.gmail.push.token context variable and fails (halts) when it is unset, so an unconfigured endpoint never accepts traffic. Answer 2xx quickly: Pub/Sub retries anything else.

gmail-push
meta {webhook: {service: "gmail", path: "/push"}}
fn (request) {
  note ::google::gmail::sync/verify-push(request)
  cond {
    is-err(note) => { ::hot::http/HttpResponse({status: 401, body: "bad token"}) }
    => {
      send("gmail:changed", {email: note.email-address, history-id: note.history-id})
      ::hot::http/HttpResponse({status: 204})
    }
  }
}

verify-push-token

fn (request: Map, expected: Str?): Bool

True when the request's token query parameter equals expected (constant-time compare). Fails closed: an empty expected or a missing parameter is false. expected is masked in call records.

watch

fn (topic-name: Str): Map
fn (topic-name: Str, opts: Map): Map

POST /users/me/watch - publish this mailbox's changes to a Pub/Sub topic (projects/<project>/topics/<topic>; see the namespace doc for setup). Returns {historyId, expiration} (epoch ms, 7 days out): store historyId as the sync start and renew before expiration. Options: label-ids, label-filter-behavior. Scope: gmail.readonly.

renew-gmail-watch
meta {schedule: "0 3 * * *"}
fn (event) {
  ::google::gmail::sync/watch("projects/my-project/topics/gmail", {label-ids: ["INBOX"], label-filter-behavior: "include"})
}

Types

HistorySet

HistorySet type {
    history: Vec,
    history-id: Str,
    added-message-ids: Vec,
    deleted-message-ids: Vec,
    label-changed-message-ids: Vec,
    pages: Int
}

Everything that changed since a history id.

• history - the history records, in order
• history-id - the mailbox's current history id: store it for the next sync
• added-message-ids - messages added (new mail, sent mail, drafts), de-duplicated, excluding ones deleted later in the same set
• deleted-message-ids - messages deleted permanently (trashing is a labelAdded of TRASH)
• label-changed-message-ids - messages whose labels changed
• pages - how many pages were read

PushNotification

PushNotification type {
    email-address: Str?,
    history-id: Str?,
    message-id: Str?,
    publish-time: Str?,
    subscription: Str?,
    attributes: Map
}

A Gmail change notification delivered by a Pub/Sub push subscription.

• email-address - the mailbox that changed
• history-id - its history id after the change (a Str); sync from your stored id, not from this one
• message-id, publish-time, subscription, attributes - Pub/Sub envelope fields