Migrating from Google Workspace

The generic engine (Migration) copies mail from any IMAP server. Google changes the shape of the problem rather than adding to it: with one service account there are no passwords to collect at all, for any number of mailboxes.

That is worth stating plainly, because it is the opposite of every other source. A cPanel or Plesk migration ends with somebody chasing forty mailbox passwords. A Workspace migration ends when the customer’s administrator clicks “Authorise”.

Per-user OAuth would need a redirect URI registered with Google, and every install of this product runs on the customer’s own hostname. There is no single URI we could register that would cover them all, and asking each customer to register their own OAuth client is worse than what is described here.

App-only auth has no redirect URI. It is also a better bargain for the customer:

their own administrator creates the credential and grants it named scopes, in their own console;
every call it makes appears in their audit log, attributed to it;
they revoke it in one click when the migration is finished, without changing anybody’s password.

What the service account can and cannot do


Can	List users and their aliases; list groups and members; read each mailbox’s vacation responder, filters and auto-forwarding; open every mailbox over IMAP.
Cannot	Report how much mail a mailbox holds (that is the Reports API — a separate grant, and two days stale). Read anything at all without the customer’s administrator having authorised the exact scopes first.

Nothing here writes to the source. That restraint is entirely ours and worth being honest about: Google publishes no read-only scope for mail settings, so gmail.settings.basic — which is read and write — is the only scope that can read a vacation responder or a filter list. The connector issues nothing but GETs, and there is a test asserting it, but the grant itself is wider than the use. An admin who would rather not give it can turn off Read each mailbox’s Gmail settings and lose only the autoresponder, filter and forwarding import; the mail still copies.

Setting it up

In the customer’s Google Cloud console, as somebody who can administer the Workspace:

Create a project (or use one) and enable the Admin SDK API and the Gmail API.
Create a service account, then create a JSON key for it. Keep the file; you need two fields out of it, client_email and private_key.
Note the service account’s Client ID (the long number, on its Details tab).
In the Google Admin console → Security → Access and data control → API controls → Domain-wide delegation → Add new, paste the Client ID and authorise these scopes:
```
https://www.googleapis.com/auth/admin.directory.user.readonly
https://www.googleapis.com/auth/admin.directory.group.readonly
https://www.googleapis.com/auth/admin.directory.group.member.readonly
https://mail.google.com/
https://www.googleapis.com/auth/gmail.readonly
https://www.googleapis.com/auth/gmail.settings.basic
```
The last two are only needed for Read each mailbox’s Gmail settings. https://mail.google.com/ is the one that opens mailboxes — Gmail accepts no narrower scope for IMAP.
Make sure IMAP is enabled for the users being migrated (Apps → Google Workspace → Gmail → End User Access). It is on by default, but an organisation that turned it off will fail every mailbox at login, and the connection test says so in those words.

Then, in the wizard: the Workspace domain, a super-admin in it, and the service account’s address and private key. The private key can be pasted straight out of the JSON file, \n escapes and all.

Why a super-admin? Google refuses directory calls made by a service account on its own behalf, so it has to act as somebody. That address is used for reading the user and group lists and nothing else; each mailbox’s own mail and settings are read as that mailbox’s owner.

Running one

New migration → Google Workspace, then Test with a real mailbox address. Without an address the test mints a token and runs a full discovery — the directory, groups and (when enabled) every mailbox’s Gmail settings (GoogleConnector.discover); with one it also opens that mailbox, which is the real end-to-end check of the delegation.
Add the domains on the Domains page for any that do not exist here yet.
Read the source on the job’s Discover step — every user, their aliases, groups, vacation responders, filters and forwarding.
Create and map the missing mailboxes. Each gets a new password, shown once. This source reports no quotas, so each mailbox takes the domain default, applied when the migration ends: until then it has none, so the copy cannot fail on a quota (migration.md → Quotas).
Import the forwarders, autoresponders and filters.
From here it is an ordinary migration: Migrating a domain and cutting over.

How each thing is translated

Mailboxes

Every user in the domain, with their aliases. Two are left out deliberately:

a user whose mailbox Google has not finished provisioning (isMailboxSetup: false) — there is nothing to copy yet, and listing them would produce a run that fails on every one;
Google’s own user@domain.example.google.com routing aliases, which are not addresses this server should ever serve.

A suspended or archived user is listed and marked suspended, and their mail still copies. The mark is for your information only: Create and map makes an ordinary, active mailbox for them like any other, so switch on Suspended on that mailbox’s page afterwards if they should stay locked out.

Discovery has ceilings. The directory listing stops at 5,000 users and 1,000 groups (google-api.ts) and says nothing when it does — a larger domain is silently truncated. Each group lists at most 500 members, also silently, so a larger group becomes an incomplete alias with no warning. Gmail settings (vacation, filters, forwarding) are read for the first 200 mailboxes only, and past that the rest are skipped with a warning (maxSettingsUsers, google.ts).

Sizes are not shown. Google reports mailbox usage only through the Reports API, on a different grant and about two days behind. Not worth a second scope to display a stale number; imapsync reports the real figures per mailbox once a run starts.

Folders — the one that matters

Gmail is not a folder store. Its labels appear over IMAP as [Gmail]/… pseudo-folders, and [Gmail]/All Mail contains every message a second time. Copying it verbatim would roughly double every mailbox, so it is excluded, along with Important and Starred, which are views rather than folders. Sent Mail, Drafts, Trash and Spam are renamed onto the standard names.

A message with several labels in Gmail appears over IMAP in several folders, one per label, and is copied into each — so it arrives here as one copy per label, not in one folder. imapsync’s duplicate detection works within a folder, and --skipcrossduplicates is not passed.

Groups → aliases

A Google Group is imported as an alias to its members — which is what a simple distribution list is. The wizard’s Import Google Groups as aliases switches it off. Nested groups are skipped (they need their own expansion) and suspended members are left out.

What an alias cannot be is a group with an archive: Google stores that mail in the group itself, where an alias only passes it on. The discovery warning counts the groups imported as aliases — those with at least one member that could be carried across — so nobody assumes the archives came too. A group whose members could not be read is skipped with a warning of its own. Moderation and posting rules do not exist here either.

Vacation responder → Sieve vacation

Subject, body and window come across. A responder whose window has already closed is imported switched off rather than silently reactivated on a server the sender has only just started using.

Gmail replies at most once every four days to the same sender and has no setting for it, so that is the interval used here.

Auto-forwarding → a forwarder

Gmail’s auto-forwarding becomes a forwarder on that mailbox, keeping the copy, whatever Gmail did with its own copy (archive, trash, mark read): that setting is not carried across.

Filters → filter rules

Gmail filters carry two very different kinds of instruction, and they are treated differently:

Anything that decides where a message lands, or whether it survives, must be exact, or the whole filter is refused with a note.
Gmail-only markers with no counterpart here that change no delivery are dropped, and the note says which. Gmail’s own UI sets excludeChats on every filter it creates; refusing over that would refuse nearly every filter and make nobody safer.

Where the admin reads these notes: the result under Import on the migration job’s discovery panel. A refused filter is under n not done, as filter on <address> with its reason. A filter imported with a loss is under its own list, n filters imported with something left out, as filter "<name>" on <address> with the note. It is kept apart because that rule is running. The API returns these as skipped and imperfect in MigrationImportResult. The discovery list itself only counts filters.

Translated:

Gmail	Here
From / To (a plain address or domain)	an address test, “contains”
Several addresses in one field, on its own	the same tests joined with “any”
Subject (plain text)	a Subject header test
Size larger / smaller than	a size test
Apply label	`fileinto`, keeping Gmail’s `/` nesting (not with Delete it — below)
Delete it (`TRASH`)	`fileinto "Trash"` — recoverable, as Gmail’s is
Star it	flag `\Flagged`
Mark as read	flag `\Seen`
Forward to	`redirect` keeping a copy — Gmail forwards and keeps
Mark as spam (`SPAM` added)	`fileinto "Junk"`, with a note that it no longer trains the spam filter

“Delete it” is a move, not a deletion. Gmail keeps a trashed message for 30 days, so the filter becomes fileinto "Trash". Mapping it onto Sieve’s discard would never store the message at all, quietly turning a rule the user can undo into one they cannot — the same trap, and the same answer, as Outlook’s delete in Migrating from Microsoft 365. Filters imported by 0.3.0-rc.6 and earlier were translated to discard and keep it until somebody edits them.

A filter that labels and deletes files into Trash only. Gmail keeps such a message once, in Trash, with the label attached, and it appears in no label view. Here a label is a folder, and filing into both would store two copies — one of them in a visible folder the user asked to have the mail deleted from. So when “Delete it” is among a filter’s actions, Trash is the only place the message goes: the label, “mark as spam” and an explicit Inbox are left out, and the filter’s note names each one. Starring, marking as read and forwarding still happen, as they do in Gmail. Filters imported by 0.3.0-rc.6 and earlier are affected differently: they were translated to discard, which does not undo a fileinto, so a filter that labels and deletes leaves the message in the label’s folder only — not in Trash. Replace its label and discard with a move to Trash in the filter editor, or import it again, to get the behaviour described here.

Dropped, and named in the note: excludeChats, the IMPORTANT and CHAT markers, the CATEGORY_* inbox tabs, and removing any of those or a star.

Refused, with the reason reported:

A Gmail search query (query / negatedQuery). has:attachment, list:, older_than: and the rest are search-engine features, not header tests. Treated as a literal they would match nothing at all — which is the silent failure this whole translator exists to avoid.
A search expression inside From, To or Subject — brackets, quotes, AND/OR, a leading -, or an operator such as from: or has: — for the same reason.
“Has attachment”: Sieve cannot test it without walking the body structure.
More than 20 conditions or 10 actions — the limits of one rule here — such as a From listing 21 senders. Cutting it to fit would silently stop filing mail from the senders past the 20th, so it is refused, naming the count, as the cPanel and Microsoft 365 translators refuse it.
Several addresses in one field alongside another test — a size test counts — or where one of them is not a plain address or domain. Gmail means (from a OR from b) AND subject; our rules carry one and/or for the whole filter, and there is no honest way to collapse it.
A size test that is neither “larger” nor “smaller”.
“Skip the inbox” with no label and no delete — that is “archive”, which has no equivalent here.
Removing a label — including SPAM, which is how Gmail spells “never send it to spam” — and a label the account no longer has (which would otherwise become a folder literally called Label_37).
A forward to something that is not an email address.

A filter left with no condition, or no action, that can be carried across is not imported either, and the note says which.

Gmail runs every matching filter, so imported rules are marked “do not stop”, to keep a message that matches two of them behaving the same way.

Long mailboxes and expiring tokens

Google access tokens live one hour. A 10 GB mailbox does not copy in one hour, and a token frozen into the copy process at launch would be dead long before it finished.

So the token is written to a file, and imapsync is given the path rather than the token: it re-reads that file on every reconnect. While a run is in flight the worker mints a fresh token every twenty minutes and rewrites the file underneath it.

The file is a secret on disk — random name, mode 0600, inside the worker’s own volume, and unlinked when the run ends. That is a narrower exposure than the argv passwords already accepted for every other source, and unlike argv it can be refreshed.

Errors specific to this connector

Code	Means
`source_auth` (400)	Most often domain-wide delegation was not granted for a scope, which Google reports as `invalid_grant`; the message says so. Also a super-admin address that does not exist.
`source_tls` (400)	A certificate on the way to Google did not verify — usually something intercepting TLS between this server and the internet.
`source_rate_limited` (429)	Google is throttling. Note that Google says 403 for this as well as for “not allowed” — the reason string is what separates them, and the connector reads it, so a throttle is never reported as a permissions problem.
`source_error` (502)	The user does not exist. A group that cannot be read is not an error: it is skipped with a discovery warning.
`source_unreachable` (502)	Could not reach `oauth2.googleapis.com`, `admin.googleapis.com` or `gmail.googleapis.com`.
`source_error` (502)	Anything else, with Google’s own words.

IMAP switched off for a user is not an HTTP error either. The connection test’s mailbox check reports it as auth and says in those words that IMAP is off, and a run fails that mailbox at login.

If discovery reports that Gmail settings could not be read for n mailboxes, the service account is missing gmail.settings.basic. Grant it, or turn off Read each mailbox’s Gmail settings — everything else still works.

Where it lives

Piece	Where
Schemas	`packages/core/src/migration.ts` (`GoogleSourceConfig`, `GoogleSourceSecret`)
Token minting and files	`packages/migration/src/connectors/oauth.ts`
Directory and Gmail calls	`packages/migration/src/connectors/google-api.ts`
Filter translation (pure)	`packages/migration/src/connectors/gmail-filters.ts`
The connector	`packages/migration/src/connectors/google.ts`
Token refresh mid-run	`apps/worker/src/jobs/migration.ts`
Pre-create and import	`apps/api/src/services/migration-import.ts` (shared with the panels)
UI	`apps/admin/src/routes/migrations.tsx`, `migration.tsx`