Keep a mirror in sync: the change feed (start page token, list, walk-to-the-end), push notifications (changes.watch, files.watch, channels.stop), and parsing/verifying the webhook Google calls.

Sync loop

  1. Once: token ::changes/get-start-page-token() and store it.
  2. Each run (schedule or webhook): set ::changes/sync-changes(token), apply set.changes (each has removed, fileId, file), store set.new-start-page-token.

Push. watch-changes registers an HTTPS webhook. Notifications carry only headers (no file data): treat one as "call sync-changes now". Channels expire (at most 7 days for changes, 1 day for a file) and are not renewed automatically: renew from a Hot schedule with renew-watch. Set a channel-token and check it with verify-notification.

Scopes. drive.readonly for the feed and watches (drive.file sees only the app's files; drive.metadata.readonly works too).

Values

CHANGE_FIELDS

CHANGE_FIELDS: Str "nextPageToken,newStartPageToken,changes(changeType,time,removed,fileId,driveId,file(id,name,mimeType,size,modifiedTime,createdTime,parents,md5Checksum,sha1Checksum,sha256Checksum,driveId,webViewLink,exportLinks,trashed,version,headRevisionId,shortcutDetails),drive(id,name))"

MAX_CHANGES_CHANNEL_MS

MAX_CHANGES_CHANNEL_MS: Int 604800000

MAX_FILE_CHANNEL_MS

MAX_FILE_CHANNEL_MS: Int 86400000

WATCH_LOCAL_KEYS

WATCH_LOCAL_KEYS: Vec [
  "channel-id",
  "channel-token",
  "expiration",
  "expires-in-ms",
  "channel-params"
]

Functions

build-channel

fn (address: Str, opts: Map, now-ms: Int): Map

Build a watch channel body (pure): {id, type: "web_hook", address, token?, expiration?, params?}. Options: channel-id (default a new UUID; at most 64 chars), channel-token (echoed back as X-Goog-Channel-Token; at most 256 chars; masked in call records), expiration (epoch ms) or expires-in-ms (from now-ms), channel-params (e.g. {ttl: "3600"}). address must be HTTPS with a valid certificate.

Example

address "https://api.example.com/drive/hook"
::google::drive::changes/build-channel(address, {channel-id: "c1", expires-in-ms: 60000}, 1700000000000)
// {id: "c1", type: "web_hook", address: "https://api.example.com/drive/hook", expiration: "1700000060000"}

change-params

fn (page-token: Str, opts: Map): Map

The query parameters list-changes sends (pure): pageToken, shared-drive flags on, fields = CHANGE_FIELDS, then your Drive parameters (pageSize up to 1000, driveId, includeRemoved, restrictToMyDrive, spaces, includeCorpusRemovals, fields).

Example

::google::drive::changes/change-params("48213", {pageSize: 100, fields: null})
// {pageToken: "48213", supportsAllDrives: true, includeItemsFromAllDrives: true, pageSize: 100}

changed-files

fn (changes: Any): Vec

The file changes in a ChangeSet (or a Vec of changes) that still exist, as DocFiles, latest change per file.

Example

changes [
  {fileId: "a", file: {id: "a", name: "a.txt", mimeType: "text/plain"}},
  {fileId: "b", removed: true}
]
map(::google::drive::changes/changed-files(changes), %.name)  // ["a.txt"]

get-start-page-token

fn (): Str
fn (opts: Map): Str

GET /changes/startPageToken - the token for "changes from now on" (a Str). Pass {driveId} for one shared drive's feed.

Example

token ::google::drive::changes/get-start-page-token()

// One shared drive's feed
::google::drive::changes/get-start-page-token({driveId: "0ABCdrive"})

is-removal

fn (change: Map): Bool

True when a change means the file is gone for this user: removed (deleted or access lost) or trashed.

Example

::google::drive::changes/is-removal({removed: true})  // true
::google::drive::changes/is-removal({file: {trashed: true}})  // true
::google::drive::changes/is-removal({file: {trashed: false}})  // false

list-changes

fn (page-token: Str): Map
fn (page-token: Str, opts: Map): Map

GET /changes - one page of changes since page-token: {changes, nextPageToken?}, and on the last page newStartPageToken (save it for next time).

Example

page ::google::drive::changes/list-changes("48213", {pageSize: 100})
length(page.changes)

// Set on the last page: store it for the next run
page.newStartPageToken

parse-notification

fn (request: Map): Notification

Parse a webhook request into a Notification (pure). Header names are matched case-insensitively. Does not check the channel token: use verify-notification.

Example

note ::google::drive::changes/parse-notification({headers: {
  "X-Goog-Channel-ID": "c1",
  "X-Goog-Resource-State": "update",
  "X-Goog-Changed": "content,parents",
  "X-Goog-Message-Number": "7"
}})
note.resource-state  // "update"
note.changed  // ["content", "parents"]
note.message-number  // 7

removed-file-ids

fn (changes: Any): Vec

The file ids a ChangeSet (or Vec of changes) removed or trashed, de-duplicated. A file that came back later in the same set is not included.

Example

changes [
  {fileId: "a", file: {id: "a", name: "a.txt", mimeType: "text/plain"}},
  {fileId: "b", removed: true}
]
::google::drive::changes/removed-file-ids(changes)  // ["b"]

renew-watch

fn (old: Map?, page-token: Str, address: Str): Map
fn (old: Map?, page-token: Str, address: Str, opts: Map): Map

Replace a changes channel before it expires: open a new channel for page-token first, then stop old (a watch response or {id, resourceId}; null to skip). Notifications may briefly arrive on both. Returns the new channel. Call it from a schedule shorter than the channel lifetime.

renew-drive-watch
meta {schedule: "0 3 */6 * *"}
fn (event) {
  state ::hot::store/Map({name: "drive-watch"})
  secret ::hot::ctx/get("google.drive.channel.token")
  old ::hot::store/get(state, "channel")
  fresh ::google::drive::changes/renew-watch(old, ::hot::store/get(state, "page-token"),
    "https://api.example.com/drive/hook", {channel-token: secret, expires-in-ms: 604800000})
  ::hot::store/put(state, "channel", fresh)
}

stop-channel

fn (channel-id: Str, resource-id: Str): Bool
fn (channel-id: Str, resource-id: Str, opts: Map): Bool

POST /channels/stop - stop notifications for a channel (id and resourceId from the watch response). Only the same user and OAuth client can stop a user's channel. Returns true.

Example

channel ::google::drive::changes/watch-file("1AbCfile", "https://api.example.com/drive/hook")
::google::drive::changes/stop-channel(channel.id, channel.resourceId)

sync-changes

fn (page-token: Str): ChangeSet
fn (page-token: Str, opts: Map): ChangeSet

Read every page of changes since page-token and return a ChangeSet with the token to use next time. Uses pageSize: 1000 unless set.

::changes ::google::drive::changes

saved-token "48213"
set ::changes/sync-changes(saved-token)
upserts ::changes/changed-files(set)       // DocFiles to (re)index
deletes ::changes/removed-file-ids(set)    // ids to drop
saved-token set.new-start-page-token

verify-channel-token

fn (request: Map, expected: Str?): Bool

True when the request's X-Goog-Channel-Token equals expected (constant-time compare). Fails closed: an empty expected or a missing header is false. expected is masked in call records.

Example

// Fails closed when the header or the expected token is missing.
// Pass the expected token from ::hot::ctx/get, as in verify-notification.
::google::drive::changes/verify-channel-token({headers: {}}, null)  // false

verify-notification

fn (request: Map): Notification
fn (request: Map, expected: Str): Notification

Verify and parse a Drive webhook: returns the Notification, or an Err when the channel token is missing or wrong. The 1-arity form checks against the google.drive.channel.token context variable and fails (halts) when it is unset, so an unconfigured endpoint never accepts traffic.

drive-hook
meta {webhook: {service: "drive", path: "/drive/hook"}, secret-headers: ["x-goog-channel-token"]}
fn (request) {
  note ::google::drive::changes/verify-notification(request)
  cond {
    is-err(note) => { ::hot::http/HttpResponse({status: 401, body: "bad channel token"}) }
    note.is-sync => { ::hot::http/HttpResponse({status: 200, body: ""}) }
    => {
      send("drive:changed", {resource-state: note.resource-state})
      ::hot::http/HttpResponse({status: 200, body: ""})
    }
  }
}
Context Vars: google.drive.channel.token

watch-changes

fn (page-token: Str, address: Str): Map
fn (page-token: Str, address: Str, opts: Map): Map

POST /changes/watch - get notified at address when anything after page-token changes. Returns the channel (id, resourceId, expiration): keep id and resourceId to stop it. See build-channel for the channel options; Drive parameters (driveId, restrictToMyDrive, ...) and credentials also go in opts.

token ::google::drive::changes/get-start-page-token()
channel ::google::drive::changes/watch-changes(token, "https://api.example.com/drive/hook", {
  channel-token: ::hot::ctx/get("google.drive.channel.token"),
  expires-in-ms: 518400000   // 6 days; renew before the 7-day cap
})

watch-file

fn (file-id: Str, address: Str): Map
fn (file-id: Str, address: Str, opts: Map): Map

POST /files/{id}/watch - get notified when one file changes (states update, trash, untrash, remove). File channels last at most one day. Options as for watch-changes.

Example

channel ::google::drive::changes/watch-file("1AbCfile", "https://api.example.com/drive/hook", {
  channel-token: ::hot::ctx/get("google.drive.channel.token"),
  expires-in-ms: 3600000
})
channel.resourceId

Types

ChangeSet

ChangeSet type {
    changes: Vec,
    new-start-page-token: Str,
    pages: Int
}

Everything that changed since a page token.

  • changes - Drive change resources in order (changeType, time, removed, fileId, driveId, file, drive)
  • new-start-page-token - store it and pass it to the next sync
  • pages - how many pages were read

Notification

Notification type {
    channel-id: Str?,
    channel-token: Str?,
    channel-expiration: Str?,
    resource-id: Str?,
    resource-uri: Str?,
    resource-state: Str?,
    changed: Vec,
    message-number: Int?,
    is-sync: Bool
}

A Drive push notification, parsed from its headers (the body is empty).

  • channel-id, channel-token, channel-expiration
  • resource-id (stable), resource-uri (version specific)
  • resource-state - sync (first message on a new channel), change (changes.watch), or add, remove, update, trash, untrash (files.watch); ignore unknown values
  • changed - for update: content, properties, parents, children, permissions
  • message-number - increasing per channel (1 for sync)
  • is-sync