Skip to main content

Data Export & Portability — Technical Reference

← Back to Data Export & Portability

Where this lives​

Backend

Frontend

Technical implementation checklist​

  • requestDataExport — resolver wired in user-data-export.resolver.js, calls data-export.manager.js#requestDataExport; persists a pending row to the data_export table (now with real export_type/date_range_start/date_range_end/include_profile/include_messages/include_media/download_token columns, not just an opaque blob) and kicks off processDataExport in the background via setImmediate. Wired to the "Request download" button on Settings → Download data.
  • dataExportStatus — resolver wired in user-data-export.resolver.js. Fixed this session: it previously ignored the exportId argument entirely and always returned the caller's latest export, and the returned object didn't line up with the DataExportStatus schema type (missing id/exportType, so querying those non-null fields errored). It now passes exportId through to getDataExportStatus, looks up that specific export (verifying it belongs to the caller), and maps it onto the full schema shape via _toStatusShape; it returns null (a valid result for this nullable field) when the export doesn't exist or isn't the caller's. The frontend still uses myDataExports on mount (no exportId to pass until a request exists), but dataExportStatus(exportId) now works correctly for direct lookups.
  • cancelDataExport — resolver wired in user-data-export.resolver.js; only works while status is pending/processing, publishes dataExportStatusChanged. Wired to the "Cancel export" button.
  • myDataExports — resolver wired in user-data-export.resolver.js, calls data-export.manager.js#getUserDataExports. Fixed this session: it used to wrap the single most-recent export (getDataExportStatus) in a 0-or-1-item array despite the name; it now queries data_export for the caller's real history (findByUserId, newest first, capped at 50) and maps each row through the same _toStatusShape used by dataExportStatus. DownloadDataPage.tsx still only renders the first entry (data?.myDataExports?.[0]) though — the UI hasn't been updated to show a history list, so this fix isn't yet visible to users beyond correct single-export polling.
  • deleteDataExport — resolver wired in user-data-export.resolver.js, calls data-export.manager.js#deleteDataExport. Fixed this session: it used to have no independent delete logic and just called cancelDataExport under the hood, so it only worked while the export was still pending/processing (and just flipped the row to cancelled). It's now a real, distinct operation: it works on terminal exports (ready/failed/cancelled) — rejecting active ones with a "cancel first" error — hard-deletes the data_export row, and best-effort removes the locally-stored file when downloadUrl points at /uploads/exports/... (S3-backed exports rely on the bucket's own lifecycle policy instead). This matches how the UI's "Delete record" button is actually used, on ready/failed exports.
  • downloadDataExport — resolver wired in user-data-export.resolver.js, calls data-export.manager.js#downloadDataExport; checks the export is ready and not expired. Fixed this session: DataExportDownloadResponse.downloadToken is non-null in the schema, but nothing used to generate one. requestDataExport now generates and persists a downloadToken (crypto.randomBytes(32)) when the row is created, and downloadDataExport lazily generates + persists one for any older row that predates the fix, so the field always resolves.
  • processDataExport — new mutation exposing the background job (data-export.manager.js#processDataExport) directly; also invoked internally by requestDataExport. Collects posts/comments/messages/follow connections, writes the file, and flips status processing → ready (or failed). Fixed this session: now respects includeProfile/includeMessages/includeMedia and the dateRangeStart/dateRangeEnd filter (see Requesting an export below) instead of collecting everything unconditionally.
  • dataExportStatusChanged — new subscription (DATA_EXPORT_STATUS_${userId} via services/pubsub.service.js), published on every status transition (processing/ready/failed/cancelled). DownloadDataPage.tsx subscribes to it and refetches instead of polling.
  • getDataPortabilityInfo (canExport, exportFormats, retentionPeriod, lastExport) — resolver wired in user-data-export.resolver.js, but userManager.getDataPortabilityInfo just returns dataExportManager.getDataExportStatus(...), which has none of those keys. Since all but lastExport are non-null in the DataPortabilityInfo type, querying this field will fail rather than return real retention/format policy data.

Requesting an export​

requestDataExport persists a pending row to the data_export table and kicks off data-export.manager.js#processDataExport in the background (setImmediate), so the mutation returns right away. Fixed this session: the manager's response object used to be keyed estimatedTime (a fixed string, "a few minutes") instead of estimatedCompletion as the schema field is named, so estimatedCompletion always resolved to null. It now builds and returns a real estimatedCompletion DateTime (~5 minutes from the request), alongside exportId.

Of the DataExportRequestInput toggles below, all except exportType now actually gate what gets collected in collectUserData. Fixed this session: includeMessages and includeMedia used to be accepted but ignored — messages and media were hard-coded as empty regardless of the flags; they're now real data_export columns (include_messages/include_media) and are honored (messages are collected via messageAccessService.getBySender, and media is stripped from posts/messages when includeMedia is false). includeProfile is likewise now a real column and gates whether the sanitized profile blob is included. dateRangeStart/dateRangeEnd are now real columns (date_range_start/date_range_end) and are applied as a createdAt filter on posts, comments, and messages. exportType is now persisted (export_type column) but is metadata only — it doesn't change what's collected or the output format. The output format is still always json: the manager's format column defaults to json, DataExportRequestInput has no format field, and nothing in the current request flow sets a non-default value, so the manager's CSV/XML conversion exists but stays unreachable via GraphQL.

mutation RequestDataExport($input: DataExportRequestInput!) {
  requestDataExport(input: $input) {
    success
    message
    exportId
    estimatedCompletion
  }
}

DataExportRequestInput fields​

FieldTypeDescription
exportTypeStringPersisted (export_type column) as metadata; doesn't change what's collected — output format is always json
includePostsBooleanInclude the user's posts (applied; stored in the options JSONB blob)
includeCommentsBooleanInclude comments (applied; stored in the options JSONB blob)
includeMessagesBooleanInclude messages (include_messages column). Fixed this session — previously accepted but ignored
includeMediaBooleanInclude media on posts/messages (include_media column). Fixed this session — previously accepted but ignored
includeProfileBooleanInclude the sanitized profile blob (include_profile column). Fixed this session — previously accepted but unused
dateRangeStartDateTimeFilters posts/comments/messages by createdAt (date_range_start column). Fixed this session — previously accepted but unused
dateRangeEndDateTimeFilters posts/comments/messages by createdAt (date_range_end column). Fixed this session — previously accepted but unused

Tracking export progress​

myDataExports is what DownloadDataPage.tsx actually uses to load and poll the current export: it takes no arguments, so it's usable before the client has an exportId to pass to dataExportStatus. Fixed this session: it used to wrap the single most-recent export in a 0-or-1-item array despite the name; it now returns real history (newest first, capped at 50). The page itself hasn't been updated to render that history yet — it still reads data?.myDataExports?.[0], i.e. just the latest export.

dataExportStatus(exportId) also exists, for looking up a specific export by id. Fixed this session: it used to ignore the exportId argument entirely (always returning the caller's latest export) and its underlying object was missing the schema's id/exportType keys, so selecting those non-null fields errored. It now honors exportId, verifies ownership, returns the full DataExportStatus shape, and returns null if the export doesn't exist or belongs to someone else. The frontend still defaults to myDataExports for the initial load (there's no exportId yet at that point), but dataExportStatus is now safe to use for direct lookups.

subscription DataExportStatusChanged($userId: ID!) {
  dataExportStatusChanged(userId: $userId) {
    id status
  }
}

query MyExports {
  myDataExports {
    id status exportType requestedAt completedAt expiresAt
    downloadUrl fileSize progressPercentage errorMessage
  }
}

The status field cycles through: pending → processing → ready (or failed), and can also become cancelled. Once status is ready, downloadUrl is populated (S3 pre-signed URL if services/s3.service.js is configured, otherwise a local /uploads/exports/... path) and stays valid until expiresAt (30 days after the request).

Managing exports​

cancelDataExport aborts an in-progress export before it completes. Only works when status is pending or processing; it publishes dataExportStatusChanged. downloadDataExport validates the export is ready and not expired and returns the download URL/file info, including DataExportDownloadResponse.downloadToken (fixed this session — requestDataExport now generates and persists a downloadToken up front, and downloadDataExport lazily backfills one for any pre-existing row that doesn't have it yet, so the non-null field always resolves). deleteDataExport is now (fixed this session) a real, independent operation distinct from cancelDataExport: it hard-deletes the data_export row and best-effort removes the locally-stored file, and it works on terminal exports (ready/failed/cancelled) rather than only pending/processing ones — it rejects active exports and tells the caller to cancel first. This matches the "Delete record" UI, which is shown for ready/failed exports.

# Abort a pending or in-progress export
mutation CancelDataExport($exportId: ID!) {
  cancelDataExport(exportId: $exportId) { success message }
}

# Prepare a ready export for download
mutation DownloadDataExport($exportId: ID!) {
  downloadDataExport(exportId: $exportId) { success downloadUrl fileSize expiresAt fileName }
}

# Remove the export record
mutation DeleteDataExport($exportId: ID!) {
  deleteDataExport(exportId: $exportId) { success message }
}

Data portability info​

getDataPortabilityInfo is declared to return the platform's data retention and format policies (canExport, exportFormats, retentionPeriod, lastExport — the DataPortabilityInfo type lives in graphql/types/user-presence.type.js), but its resolver (userManager.getDataPortabilityInfo) just forwards to the same single-export getDataExportStatus used by dataExportStatus/myDataExports, which has none of those keys. Since canExport, exportFormats and retentionPeriod are non-null in the schema, querying this field currently fails instead of returning real policy data — this needs a dedicated implementation.

query DataPortabilityInfo {
  getDataPortabilityInfo {
    canExport
    exportFormats
    retentionPeriod   # days data is kept after account deletion
    lastExport
  }
}

Data model​

The data_export table (migration 20260719080000-create-data-export.js, plus 20260731200000-add-columns-to-data-export.js — added this session) holds one row per export request:

ColumnTypeNotes
idUUID (PK)
user_idUUID
statusSTRING(20)pending (default) → processing → ready / failed / cancelled
formatSTRING(10)defaults to json; csv/xml conversion exists in the manager but nothing in the current request flow sets a non-default format
export_typeSTRING(20)added this session; the DataExportRequestInput.exportType value (defaults to full) — metadata only, doesn't affect what's collected
optionsJSONBthe includePosts/includeComments/includeAnalytics object captured at request time (the toggles that don't have dedicated columns)
date_range_start, date_range_endDATEadded this session; DataExportRequestInput.dateRangeStart/dateRangeEnd, now actually applied as a createdAt filter on posts/comments/messages
include_profile, include_messages, include_mediaBOOLEANadded this session, all default true; now actually gate what collectUserData includes
download_tokenSTRING(64)added this session; generated with crypto.randomBytes(32) at request time (or lazily by downloadDataExport for older rows) so DataExportDownloadResponse.downloadToken always resolves
progress_percentageINTEGERadded this session; persisted at each status transition (0 → 10 on processing → 100 on ready), in addition to the live-estimated progress computed on the fly for an in-flight export
requested_at, started_at, completed_at, expires_atDATEexpires_at is set to 30 days after the request
download_url, file_path, file_sizeTEXT / BIGINTpopulated once status is ready
error_messageTEXTpopulated on failed

Indexed on (user_id, created_at).