Cloud Troubleshooting And Security

Startup Cloud failures are explicit. Every first-user failure has a stable id, a user-facing next action, and a retryability hint. Use this page when sign-in, publish, work-view selection, chat, Activity, observation append, token issue, runtime calls, hosted effects, or replay export fails.

Fast Triage

Symptom	First check
Studio Cloud buttons are disabled	Confirm Studio has a configured management URL. Users should not type this in the app; local development uses the production app default, `JACQOS_MANAGEMENT_URL`, or `JACQOS_LIVE_MANAGEMENT_URL`.
Sign-in email never arrives	Confirm the WorkOS-backed management plane is reachable and, for automated tests, that AgentMail can read the inbox configured by `JACQOS_AUTH_TESTING_EMAIL_API_KEY`.
Deployment is read-only or degraded	Run `jacqos cloud status --json` and check lifecycle, health, active package digest, evaluator digest, and last activity.
Cloud append is rejected	Check the selected cloud work view, write policy, package identity, hosted refresh state, and runtime token scope.
Chat or Activity looks stale	Check projection lag and whether the selected activation still matches the package/evaluator identity you expect.
Replay export does not verify	Treat the export as failed evidence until package, evaluator, and mapper-output digest drift is resolved.

Failure States

Error	What It Means	Fix
`cloud_configuration_missing`	Studio account actions do not have a configured WorkOS-backed management plane.	Configure the app default or set `JACQOS_MANAGEMENT_URL` or `JACQOS_LIVE_MANAGEMENT_URL`; do not use the local fixture path for normal Studio auth.
`agentmail_api_key_missing`	An automated Studio auth run needs AgentMail but the API key env var is missing.	Set `JACQOS_AUTH_TESTING_EMAIL_API_KEY` in the process environment. Do not print or commit the value.
`management_url_missing`	The code-only auth runner has no management URL.	Set `JACQOS_MANAGEMENT_URL` or `JACQOS_LIVE_MANAGEMENT_URL`.
`local_auth_fixture_enabled`	The Studio auth path is using the `local-studio-user` fixture.	Disable fixture auth for account actions. Fixture auth is only for explicitly enabled local test paths.
`verification_email_not_received`	AgentMail or the user’s inbox did not receive the WorkOS Magic Auth email before timeout.	Retry after confirming the email address, management plane, and mail delivery.
`verification_email_parse_failed`	A new email arrived but no recognizable code was found in structured fields.	Inspect provider template drift with a redacted receipt; do not log the raw email body.
`verification_code_expired`	The Magic Auth code expired.	Start sign-in again to receive a new code.
`verification_code_rejected`	WorkOS rejected the submitted code.	Re-enter the newest code or start sign-in again.
`management_route_missing`	The Studio Magic Auth route is not deployed on the management plane.	Deploy or select a management plane that includes the Studio auth routes.
`workos_identity_rejected`	WorkOS rejected the account or organization membership.	Use an invited account or ask an organization admin to grant access.
`session_persistence_failed`	Auth completed but Studio could not persist support-safe session metadata.	Retry after checking local workspace permissions and Studio API health.
`auth_state_not_authenticated`	Studio did not report `signed_in=true` after completion.	Re-run sign-in and inspect the redacted auth receipt.
`secret_material_persisted`	A receipt or state response reported that secret material was persisted.	Stop the run and treat it as a security defect.
`cloud_invite_required`	The cloud is invite-gated.	Sign in with the invite code issued to your organization.
`cloud_invite_invalid`	The invite code was not accepted.	Check the exact code or ask support for a replacement.
`cloud_signups_disabled`	New self-service signups are paused.	Wait for support to reopen signups or use an approved organization.
`cloud_device_auth_denied`	The WorkOS device authorization was denied.	Run `jacqos cloud login --wait` again and approve the device authorization.
`cloud_device_auth_expired`	The WorkOS device authorization expired before approval.	Run `jacqos cloud login --wait` again to get a fresh code.
`unauthenticated_management_request`	The management API did not receive an authenticated session.	Sign in again, then retry the management action.
`missing_management_route_scope`	The session is authenticated but lacks the required cloud role.	Ask an organization admin or support for the needed role.
`wrong_org_or_project_scope`	The session organization does not match the requested scope.	Switch WorkOS organization or select the matching project.
`duplicate_app_name`	An app with that name already exists in the project.	Choose a distinct app name or select the existing app.
`billing_handoff_not_configured`	Billing handoff is not configured for the account.	Continue with runtime setup or ask support to enable billing handoff.
`inline_package_too_large`	The deployment package exceeds the first-user handoff limit.	Remove unused assets or use a package blob handoff.
`unverified_package_publish`	Local verification failed before publish.	Run `jacqos verify`, fix fixtures or invariants, then deploy again.
`missing_package_blob`	The deployment did not include a package blob handoff.	Re-run `jacqos cloud deploy`.
`package_digest_mismatch`	The package digest does not match the verified evidence.	Regenerate the verification bundle and publish the matching package.
`stale_activation_package_identity`	Studio is trying to write through a cloud work view whose local package identity no longer matches the hosted activation.	Refresh the Cloud work view or deploy and promote the package you intend to write against.
`hosted_projection_stale`	Studio cannot prove it is writing against the current hosted observation head.	Refresh the hosted projection before appending.
`append_policy_denied`	The selected cloud write policy rejected the proposed observation.	Fix the observation body or choose a declared policy that allows this class of hosted write.
`too_many_effect_capabilities`	The deployment declares more effect capabilities than the plan allows.	Remove unused capabilities or split the app.
`hosted_effect_failure`	A hosted effect attempt failed and produced an effect failure receipt or observation.	Inspect Activity and provenance. Idempotent effects may retry; ambiguous or non-idempotent effects require explicit reconcile.
`manual_reconcile_required`	An effect outcome is ambiguous and cannot be silently retried.	Record an explicit reconcile decision before expecting new semantic consequences.
`deployment_quota_exceeded`	The organization hit the deployment quota for the current period.	Wait for reset or ask support for a plan change.
`runtime_token_issue_quota_exceeded`	The organization issued too many runtime tokens in the current period.	Reuse or rotate existing tokens, or ask support for a plan change.
`runtime_token_ttl_exceeds_limit`	The requested token expiry is too long.	Request a shorter expiry.
`runtime_token_scope_limit_exceeded`	The token request includes too many scopes.	Issue separate tokens for separate clients or operations.
`management_writes_disabled`	Support paused new management writes.	Existing runtime activations keep serving; retry after the support window.
`management_plane_unavailable`	The management API is unavailable.	Retry after the status page clears; already promoted runtime endpoints keep serving.
`runtime_cell_unavailable`	The runtime cell is unavailable.	Retry when `https://runtime.cloud.jacqos.io/healthz` is ready.
`read_only_deployment`	The deployment can be inspected and exported but cannot accept mutations.	Check lifecycle state, account readiness, token scope, and support maintenance windows.
`degraded_deployment`	Runtime or projection health is degraded.	Inspect Cloud dashboard health, last activity, lag, and degraded/error counts before appending.
`no_active_evaluator`	The runtime has no promoted evaluator for that app and environment.	Publish and promote a verified deployment.
`missing_runtime_token`	The runtime request had no bearer token.	Include `Authorization: Bearer $JACQOS_RUNTIME_TOKEN`.
`invalid_runtime_token`	The token shape or signature is invalid.	Issue a new scoped runtime token.
`revoked_runtime_token`	The token was revoked.	Rotate to a new token and update the client.
`wrong_runtime_scope`	The token does not cover the endpoint or operation.	Issue a token for the correct app, environment, and scope.
`runtime_observation_quota_exceeded`	The observation body is larger than the runtime limit.	Reduce the body or move large raw content into blob storage.
`runtime_export_quota_exceeded`	The export would exceed the runtime limit.	Export a narrower lineage slice or ask support for a plan change.
`empty_lineage_not_found`	No hosted observations exist for that lineage.	Send an observation to the scoped runtime endpoint first.
`export_digest_drift`	Local replay did not match hosted evidence.	Treat the export as failed evidence and inspect package, evaluator, and mapper-output identity.
`replay_export_mismatch`	The hosted export and clean local replay disagree on facts, intents, effects, provenance, or semantic digests.	Do not trust the export as proof until the mismatch is explained and replayed cleanly.
`idempotency_conflict`	A retry reused an idempotency key with a different body.	Reuse the original body or choose a new idempotency key.

Security Model

WorkOS authenticates users and organization membership.
Studio account actions use a configured WorkOS-backed management plane. Normal sign-in, account choice, and account creation must not fall back to the local-studio-user fixture.
Management mutations require bearer authorization. Cookie-only mutation requests are rejected before provider calls.
The management API stores projects, deployments, lifecycle receipts, token metadata, support-safe audit records, account readiness, billing summaries, and Cell Control receipts.
Runtime tokens are returned once, stored as digests, scoped by app and environment, and revocable.
Runtime cells keep observations, facts, intents, effects, provenance, and exports cell-local.
Cloud append requires explicit user acknowledgement in Studio, a declared write policy, current hosted refresh, and matching package identity.
Support surfaces redact payloads, observations, atoms, facts, intents, effects, bearer values, runtime tokens, sealed device codes, passwords, private keys, auth email bodies, AgentMail keys, and secret-like fields.
Billing and offboarding records are support-visible metadata only. Runtime token revocation and activation rollback remain explicit actions.
Provider API keys, database passwords, Railway tokens, Hetzner tokens, and operator ingress tokens are never part of the first-user flow.

Recovery Order

Run jacqos cloud readiness --json or jacqos cloud status --json to confirm whether the failure is account, deployment, activation, token, runtime, append, or replay scoped.
Fix the first failure id in the table above. Later failures often clear once the first scope issue is corrected.
Re-run the narrowest command that failed. Do not republish a package when only a runtime token, append, or export command failed.
When Activity reports hosted effect failure, distinguish safe idempotent retry from manual reconcile before expecting new observations.
When replay export reports digest drift, treat the export as failed evidence and inspect package, evaluator, and mapper-output identity before trusting the hosted state.