Internal Server Error 😢 🔥 #325
Comments
nelsonic
added
help wanted
|
Yep. dwyl/mvp#440 |
|
Suspect it has something to do with this email received from |
|
Also confirm this error on my end 😢 |
|
Lame. Thanks |
|
Thankfully we have a "Staging" instance that still works flawlessly: https://authdemo.fly.dev/ 
Just re-deploying now because the machine restart didn't work to update the environment variable. 🙃 |
|
MVP now redirects to authdemo: Works for me: 
@iteles / @LuchoTurtle please confirm you're able to access your |
|
Definitely still going to try and recover the |
|
Interesting. It's working for me on Firefox desktop! On Chrome desktop, I'm still getting an internal server error straight away (on the start page, not even getting to the login portion) even after clearing cookies and cached data 🤷♀️ But fixed for now, no point troubleshooting this further until we're back on |
|
@iteles In Chrome, if you visit https://mvp.fly.dev/ what do you see? |
|
|
Well that's mega frustrating! Thank you |
|
Works on my phone:
but doesn’t look great. Some tidy up to be done. ⏳ |
|
Works on Chrome on my phone but I can't see any of my items. |
|
Ok. So that means your |
|
Only works if I clear my cookies. I'm doing this on the web. |
|
Trying to re-deploy n@MBP auth % fly deploy --verbose
==> Verifying app config
Validating /auth/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
Waiting for remote builder fly-builder-solitary-rain-5830... 🌎 |
|
Useless! 😢 WARN Remote builder did not start in time. Check remote builder logs with `flyctl logs -a fly-builder-solitary-rain-5830`
WARN Failed to start remote builder heartbeat: remote builder app unavailable
WARN failed to create build in graphql: returned error 504 Gateway Timeout: <html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>
Error: failed to fetch an image or build from source: error connecting to docker: server returned a non-200 status code: 503 |
|
Web UI is unresponsive: https://fly.io/apps/authprod 🤷♂️ 😢 
Gotta get going to |
|
Cannot access any of 
"Something went wrong" ... indeed. 🤦♂️ |
|
Right. For the first time in a very long time I'm away from @home (family) 😢 and have a couple of hours to focus. 🧑💻 🎯 |
nelsonic
added
the
in-progress
|
Ok. Step 1: Attempt to create a backup of |
|
Following this helpful article: https://fly.io/docs/postgres/managing/backup-and-restore/ fly volumes list -a authprodb
ID STATE NAME SIZE REGION ZONE ENCRYPTED ATTACHED VM CREATED AT
vol_g2yxp4mllj6v63qd created pg_data 10GB lhr 34c3 false 2 years ago
vol_1g67340ggznrydxw created pg_data 10GB lhr 805e false 2 years agoATTACHED VM: blank. There's your problem ... 
List the snapshots: fly volumes snapshots list vol_g2yxp4mllj6v63qd
Snapshots
ID SIZE CREATED AT
vs_QGPN6pQObpLQpUma 96479939 10 hours ago
vs_za8NQRN70Dmnat9K 96479939 1 day ago
vs_8zZLpNxD3VB9Qhom 95992552 2 days ago
vs_OANN6OZkLJLNfz7 95895907 3 days ago
vs_GpqzBBJgL7DmltGK 95019181 4 days ago
vs_bjjMlxqMzXjl7sB 94680799 5 days agoDunno why it's continuing to take "dumb" snapshots if the data isn't changing. 🤷♂️ So that guide doesn't actually deliver on "backing up" the DB. And the first hit is: https://community.fly.io/t/backup-and-restore-postgresql-in-fly-io-step-by-step/11455 No formatting of the input/output and using Windows 🙄 ... But it's better than nothing. Going to follow the steps. First thing I had to do was find out what the https://community.fly.io/t/lost-postgres-password/4192/2 fly ssh console
echo $DATABASE_URL
postgres://authprod:[email protected]:5432/authprodThis also tells us that the internal TCP port used is Attempt to proxy the internal TCP port to a local one: flyctl proxy 15432:5432 -a authprodbSadly, we get: Error: authprodb.internal: host was not found in DNSNot good. fly ssh console -a authprodbGot: Error: app authprodb has no started VMs.
It may be unhealthy or not have been deployed yet.
Try the following command to verify:
fly statusTried: fly status -a authprodbgot: No machines are available on this app authprodbBut the Fly.io Web interface says it's "deployed": https://fly.io/apps/authprodb 🤷♂️
But the instance doesn't have an IP Address so it's not addressable ... 🙈 🙃 flyctl ips allocate-v4 -a authprodbPrompt: ? Looks like you're accessing a paid feature. Dedicated IPv4 addresses now cost $2/mo. Are you ok with this? (y/N)Sure, why not? If $2/month resolves this (and I will immediately remove the IP address) let's go! Appears to have worked: VERSION IP TYPE REGION CREATED AT
v4 137.66.62.95 public global 19s ago
Let's try connecting to it again. 🤞 fly ssh console -a authprodbNo started VMs: Error: app authprodb has no started VMs.
It may be unhealthy or not have been deployed yet.
Try the following command to verify:
fly statusGreat. So it allows me to attach a paid IP address to a VM that is "deployed" but not reachable. 👌 (🤦♂️) Given that it's useless to us, let's release that IPV4 address before we forget: flyctl ips release 137.66.62.95Confirmed: Released 137.66.62.95 from authprodGoing to try a different approach. Let's grab the latest snapshot and create a new Recall from above that the latest snapshot ID was Run: fly postgres create --snapshot-id vs_QGPN6pQObpLQpUmaBut got: Error: "Failed to resolve source machine from snapshot."😢Try to fork the db: fly postgres create --fork-from authprodbFails: Error: No machines associated with fork-from target. See `fly pg create --help` for more informationHelpful. 🤦♂️ Reading: https://fly.io/docs/postgres/postgres-on-nomad/#upgrading-the-postgres-app |
|
Tried: flyctl image show -a authprodbOutput: MACHINE ID REGISTRY REPOSITORY TAG VERSION DIGEST LABELS
🦗 🦗 🦗... (i.e: nothing...!)Reading this very recent forum topic: https://community.fly.io/t/postgres-crashed-cannot-restart-restore-or-fork/16186 Sadly not a very helpful comment from the OP: 🙃
Read: https://fly.io/docs/flyctl/postgres-restart/ Tried: flyctl postgres restart -a authprodbGot: Error: no active leader foundRead: https://fly.io/docs/apps/restart/ Tried restarting: fly apps restart authprodbGot: Error: postgres apps should use `fly pg restart` insteadGreat! Thanks Trying to "deploy" a machine: fly deploy -a authprodb --verboseGot the following output + error: -------
✖ release_command failed
-------
Error release_command failed running on machine 1857439c5996d8 with exit code 1.
Check its logs: here's the last 100 lines below, or run 'fly logs -i 1857439c5996d8':
Pulling container image registry.fly.io/authprodb:deployment-01HEA0YDEKTNN69C15H8MBZ7XJ
Successfully prepared image registry.fly.io/authprodb:deployment-01HEA0YDEKTNN69C15H8MBZ7XJ (4.634590051s)
Configuring firecracker
[ 0.042013] Spectre V2 : WARNING: Unprivileged eBPF is enabled with eIBRS on, data leaks possible via Spectre v2 BHB attacks!
[ 0.047890] PCI: Fatal: No config space access function found
INFO Starting init (commit: 15238e9)...
INFO Preparing to run: `/app/bin/migrate` as nobody
INFO [fly api proxy] listening at /.fly/api
2023/11/03 07:22:14 listening on [fdaa:0:42c6:a7b:1bf:a08f:f999:2]:22 (DNS: [fdaa::3]:53)
ERROR! Config provider Config.Reader failed with:
** (RuntimeError) environment variable DATABASE_URL is missing.
For example: ecto://USER:PASS@HOST/DATABASE
(stdlib 3.16.1) erl_eval.erl:685: :erl_eval.do_apply/6
(stdlib 3.16.1) erl_eval.erl:446: :erl_eval.expr/5
(stdlib 3.16.1) erl_eval.erl:123: :erl_eval.exprs/5
(elixir 1.12.3) src/elixir.erl:280: :elixir.recur_eval/3
(elixir 1.12.3) src/elixir.erl:265: :elixir.eval_forms/3
(elixir 1.12.3) lib/code.ex:656: Code.eval_string_with_error_handling/3
(elixir 1.12.3) lib/config.ex:258: Config.__eval__!/3
(elixir 1.12.3) lib/config/reader.ex:86: Config.Reader.read!/2
{"init terminating in do_boot",{#{'__exception__'=>true,'__struct__'=>'Elixir.RuntimeError',message=><<101,110,118,105,114,111,110,109,101,110,116,32,118,97,114,105,97,98,108,101,32,68,65,84,65,66,65,83,69,95,85,82,76,32,105,115,32,109,105,115,115,105,110,103,46,10,70,111,114,32,101,120,97,109,112,108,101,58,32,101,99,116,111,58,47,47,85,83,69,82,58,80,65,83,83,64,72,79,83,84,47,68,65,84,65,66,65,83,69,10>>},[{erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,685}]},{erl_eval,expr,5,[{file,"erl_eval.erl"},{line,446}]},{erl_eval,exprs,5,[{file,"erl_eval.erl"},{line,123}]},{elixir,recur_eval,3,[{file,"src/elixir.erl"},{line,280}]},{elixir,eval_forms,3,[{file,"src/elixir.erl"},{line,265}]},{'Elixir.Code',eval_string_with_error_handling,3,[{file,"lib/code.ex"},{line,656}]},{'Elixir.Config','__eval__!',3,[{file,"lib/config.ex"},{line,258}]},{'Elixir.Config.Reader','read!',2,[{file,"lib/config/reader.ex"},{line,86}]}]}}
init terminating in do_boot ({,[{erl_eval,do_apply,6,[{_},{_}]},{erl_eval,expr,5,[{_},{_}]},{erl_eval,exprs,5,[{_},{_}]},{elixir,recur_eval,3,[{_},{_}]},{elixir,eval_forms,3,[{_},{_}]},{Elixir.Code,ev
Crash dump is being written to: erl_crash.dump...done
INFO Main child exited normally with code: 1
INFO Starting clean up.
WARN hallpass exited, pid: 314, status: signal: 15 (SIGTERM)
2023/11/03 07:22:15 listening on [fdaa:0:42c6:a7b:1bf:a08f:f999:2]:22 (DNS: [fdaa::3]:53)
[ 2.293675] reboot: Restarting system
machine restart policy set to 'no', not restarting
-------
Error: release command failed - aborting deployment. error release_command machine 1857439c5996d8 exited with non-zero status of 1Dunno why it's trying to run the release command ... obviously it's going to fail!! Trying to fork from the volume: fly postgres create --fork-from authprodb:vol_g2yxp4mllj6v63qdOutput: Creating app...
Setting secrets on app authprodb2...
Provisioning 1 of 1 machines with image flyio/postgres-flex:15.3@sha256:5e5fc53decb051f69b0850f0f5d137c92343fcd1131ec413015e5260620947ac
Waiting for machine to start...
Machine 1857509c162168 is created
==> Monitoring health checks
Waiting for 1857509c162168 to become healthy (started, 3/3)
Postgres cluster authprodb2 created
Username: postgres
Password: redacted
Hostname: authprodb2.internal
Flycast: fdaa:0:42c6:0:1::6
Proxy port: 5432
Postgres port: 5433
Connection string: postgres://postgres:[email protected]:5432
Save your credentials in a secure place -- you won't be able to see them again!
Connect to postgres
Any app within the DWYL Auth organization can connect to this Postgres using the above connection string
Now that you've set up Postgres, here's what you need to understand: https://fly.io/docs/postgres/getting-started/what-you-should-know/That appears to have worked: Reading: https://fly.io/docs/postgres/getting-started/what-you-should-know/ Back to getting this working! Tried: flyctl postgres attach authprodb2 -a authprodGot: Error: consumer app "authprod" already contains a secret named DATABASE_URLDuh. Let's quickly connect to the newly created fly postgres connect -a authprodb2List the databases: \l
List of databases
Name | Owner | Encoding | Collate | Ctype | ICU Locale | Locale Provider | Access privileges
-----------+----------+----------+------------+------------+------------+-----------------+-----------------------
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | libc |
repmgr | repmgr | UTF8 | en_US.utf8 | en_US.utf8 | | libc |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | libc | =c/postgres +
| | | | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | libc | =c/postgres +
| | | | | | | postgres=CTc/postgres
(4 rows)None of these look like what we need. 🙄 Going to need to take a step back ... 💭 |
|
Connecting to the (working) fly postgres connect -a authdemo-dbWe see that the databases are: \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+------------+----------+------------+------------+---------------------------
authdemo | flypgadmin | UTF8 | en_US.utf8 | en_US.utf8 |
postgres | flypgadmin | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | flypgadmin | UTF8 | en_US.utf8 | en_US.utf8 | =c/flypgadmin +
| | | | | flypgadmin=CTc/flypgadmin
template1 | flypgadmin | UTF8 | en_US.utf8 | en_US.utf8 | =c/flypgadmin +
| | | | | flypgadmin=CTc/flypgadminInstead of the As per the postgres://authprod:[email protected]:5432/authprodWe're expecting a database called |
|
This got me scratching my head ... 🤔
Whereas the Going to contact Fly.io support. |
|
Just in case I was going crazy, I tried to deploy the fly deploy --verboseOutput: ==> Verifying app config
Validating /auth/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
Remote builder fly-builder-solitary-rain-5830 ready
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 1.8s (30/30) FINISHED
=> [internal] load build definition from Dockerfile 1.0s
=> => transferring dockerfile: 32B 1.0s
=> [internal] load .dockerignore 1.0s
=> => transferring context: 34B 1.0s
=> [internal] load metadata for docker.io/library/debian:bullseye-20210902-slim 0.3s
=> [internal] load metadata for docker.io/hexpm/elixir:1.12.3-erlang-24.1.5-debian-bullseye-20210902-slim 0.4s
=> [internal] load build context 0.3s
=> => transferring context: 29.65kB 0.3s
=> [stage-1 1/6] FROM docker.io/library/debian:bullseye-20210902-slim@sha256:e3ed4be20c22a1358020358331d177aa2860632f25b21681d79204ace20455a6 0.0s
=> [builder 1/18] FROM docker.io/hexpm/elixir:1.12.3-erlang-24.1.5-debian-bullseye-20210902-slim@sha256:a2447f82e9125abb7edb0708eb51da638d8f40e7fda4b3ed5151bee883d476be 0.0s
=> CACHED [stage-1 2/6] RUN apt-get update -y && apt-get install -y libstdc++6 openssl libncurses5 locales && apt-get clean && rm -f /var/lib/apt/lists/*_* 0.0s
=> CACHED [stage-1 3/6] RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen 0.0s
=> CACHED [stage-1 4/6] WORKDIR /app 0.0s
=> CACHED [stage-1 5/6] RUN chown nobody /app 0.0s
=> CACHED [builder 2/18] RUN apt-get update -y && apt-get install -y build-essential git && apt-get clean && rm -f /var/lib/apt/lists/*_* 0.0s
=> CACHED [builder 3/18] WORKDIR /app 0.0s
=> CACHED [builder 4/18] RUN mix local.hex --force && mix local.rebar --force 0.0s
=> CACHED [builder 5/18] COPY .env_sample ./ 0.0s
=> CACHED [builder 6/18] COPY mix.exs mix.lock ./ 0.0s
=> CACHED [builder 7/18] RUN mix deps.get --only prod 0.0s
=> CACHED [builder 8/18] RUN mkdir config 0.0s
=> CACHED [builder 9/18] COPY config/config.exs config/prod.exs config/ 0.0s
=> CACHED [builder 10/18] RUN mix deps.compile 0.0s
=> CACHED [builder 11/18] COPY priv priv 0.0s
=> CACHED [builder 12/18] COPY assets assets 0.0s
=> CACHED [builder 13/18] RUN mix assets.deploy 0.0s
=> CACHED [builder 14/18] COPY lib lib 0.0s
=> CACHED [builder 15/18] RUN mix compile 0.0s
=> CACHED [builder 16/18] COPY config/runtime.exs config/ 0.0s
=> CACHED [builder 17/18] COPY rel rel 0.0s
=> CACHED [builder 18/18] RUN mix release 0.0s
=> CACHED [stage-1 6/6] COPY --from=builder --chown=nobody:root /app/_build/prod/rel/auth ./ 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:5ba72f99464041b84ed2e1491eb709c1518dff88852706229e8d82f0f3af2d4c 0.0s
=> => naming to registry.fly.io/authprod:deployment-01HEADJTX39GT10NA7TY49HG02 0.0s
--> Building image done
==> Pushing image to fly
The push refers to repository [registry.fly.io/authprod]
7b5d131176f6: Layer already exists
4d5c3ee4879b: Layer already exists
0166b8c07570: Layer already exists
4dfd7d2f8a33: Layer already exists
11a55e4db61e: Layer already exists
ba5a5fe43301: Layer already exists
deployment-01HEADJTX39GT10NA7TY49HG02: digest: sha256:f5d586ae7234d29f8ef485ce6c06e5ec469f6ae66d77886459780fc172337147 size: 1575
--> Pushing image done
image: registry.fly.io/authprod:deployment-01HEADJTX39GT10NA7TY49HG02
image size: 120 MB
Watch your deployment at https://fly.io/apps/authprod/monitoring
Running authprod release_command: /app/bin/migrate
-------
✖ release_command failed
-------
Error release_command failed running on machine 48ed037a343368 with exit code 1.
Check its logs: here's the last 100 lines below, or run 'fly logs -i 48ed037a343368':
Pulling container image registry.fly.io/authprod:deployment-01HEADJTX39GT10NA7TY49HG02
Successfully prepared image registry.fly.io/authprod:deployment-01HEADJTX39GT10NA7TY49HG02 (5.834199334s)
[ 0.041401] Spectre V2 : WARNING: Unprivileged eBPF is enabled with eIBRS on, data leaks possible via Spectre v2 BHB attacks!
[ 0.042773] PCI: Fatal: No config space access function found
INFO Starting init (commit: 15238e9)...
INFO Preparing to run: `/app/bin/migrate` as nobody
INFO [fly api proxy] listening at /.fly/api
2023/11/03 11:02:57 listening on [fdaa:0:42c6:a7b:1be:99c2:78d9:2]:22 (DNS: [fdaa::3]:53)
11:02:58.710 [error] Postgrex.Protocol (#PID<0.136.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (top2.nearest.of.authprodb.internal:5432): non-existing domain - :nxdomain
11:02:58.710 [error] Postgrex.Protocol (#PID<0.135.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (top2.nearest.of.authprodb.internal:5432): non-existing domain - :nxdomain
11:02:59.943 [error] Postgrex.Protocol (#PID<0.136.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (top2.nearest.of.authprodb.internal:5432): non-existing domain - :nxdomain
11:03:00.660 [error] Could not create schema migrations table. This error usually happens due to the following:
* The database does not exist
* The "schema_migrations" table, which Ecto uses for managing
migrations, was defined by another library
* There is a deadlock while migrating (such as using concurrent
indexes with a migration_lock)
To fix the first issue, run "mix ecto.create".
To address the second, you can run "mix ecto.drop" followed by
"mix ecto.create". Alternatively you may configure Ecto to use
another table and/or repository for managing migrations:
config :auth, Auth.Repo,
migration_source: "some_other_table_for_schema_migrations",
migration_repo: AnotherRepoForSchemaMigrations
The full error report is shown below.
** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2975ms. This means requests are coming in and your connection pool cannot serve them fast enough. You can address this by:
1. Ensuring your database is available and that you can connect to it
2. Tracking down slow queries and making sure they are running fast enough
3. Increasing the pool_size (although this increases resource consumption)
4. Allowing requests to wait longer by increasing :queue_target and :queue_interval
See DBConnection.start_link/2 for more information
(ecto_sql 3.7.2) lib/ecto/adapters/sql.ex:760: Ecto.Adapters.SQL.raise_sql_call_error/1
(elixir 1.12.3) lib/enum.ex:1582: Enum."-map/2-lists^map/1-0-"/2
(ecto_sql 3.7.2) lib/ecto/adapters/sql.ex:852: Ecto.Adapters.SQL.execute_ddl/4
(ecto_sql 3.7.2) lib/ecto/migrator.ex:678: Ecto.Migrator.verbose_schema_migration/3
(ecto_sql 3.7.2) lib/ecto/migrator.ex:504: Ecto.Migrator.lock_for_migrations/4
(ecto_sql 3.7.2) lib/ecto/migrator.ex:419: Ecto.Migrator.run/4
(ecto_sql 3.7.2) lib/ecto/migrator.ex:146: Ecto.Migrator.with_repo/3
(auth 1.6.6) lib/auth/release.ex:12: anonymous fn/2 in Auth.Release.migrate/0
INFO Main child exited normally with code: 1
WARN Reaped child process with pid: 359 and signal: SIGUSR1, core dumped? false
INFO Starting clean up.
WARN hallpass exited, pid: 306, status: signal: 15 (SIGTERM)
2023/11/03 11:03:01 listening on [fdaa:0:42c6:a7b:1be:99c2:78d9:2]:22 (DNS: [fdaa::3]:53)
[ 5.255192] reboot: Restarting system
machine restart policy set to 'no', not restarting
-------
Error: release command failed - aborting deployment. error release_command machine 48ed037a343368 exited with non-zero status of 1
|
|
Posted Forum topic: https://community.fly.io/t/where-is-our-data-postgres-database-volume-0mb/16247 Snapshot: Full text of my forum topic in case it gets deleted: Opening this question in case anyone `else` is in the same boat (having their `Nomad`-based `Postgres` instance killed ...)
# Context
We deployed our [`auth`](https://github.com/dwyl/auth) (`Elixir/Phoenix`) app to Fly.io in 2022. :rocket:
It was a good experience and worked flawlessly until they killed it. :neutral_face:
# Fly.io App Disabled 💀
We received the following email from Fly.io:
## `Fulltext` of the email for Google indexing/findability 📧
```md
Hi. We’re in the process of removing Nomad from Fly.io. This means any apps that haven’t been upgraded to Apps V2 will no longer function.
Your app authprodb is one of those apps. Here’s what this means for you:
1. authprodb is currently disabled.
- That means the Nomad instances (virtual machines)
that ran your app have been shut down, but that the app’s configuration still exists.
2. Your app is technically now on the “machines” platform, rather than “nomad.”
- That means that any subsequent fly deploy on your app will attempt to launch Fly Machines.
- In many cases, a deploy will immediately work. In some cases, there may be some additional configuration required to get your app running on Machines.
The community forum is a great place to get help troubleshooting your deployment if you get stuck.
3. Nothing has been deleted.
- Your app’s Docker image still exists, and any Volumes that were attached to your app still exist.
Thanks for bearing with us throughout this whole migration saga. It’s not been easy, but the results should soon speak for themselves.
If you have any questions or comments, we’ve posted a similar announcement on the community forum.
Thanks, Your Friends at Fly.ioThey key point of the email: "Nothing has been deleted" ... 💭So I dived into trying to recover the database. fly deploy --verboseIt errors because the Error: release command failed - aborting deployment. error release_command machine 48ed037a343368 exited with non-zero status of 1Full log output in: #325 (comment) After a couple of hours searching and trying everything I could think of with zero progress. Question: Why is the Volume Used
|
|
Replied to the support email:
Ball is in their court. 🎾 AFK to help parents with something for the next couple of hours. 🤞 |
|
Meanwhile I've wasted my whole morning on this ... ⏳ 🔥 😢 |
|
Followed-up with |
|
Got a couple of replies on the forum: Tried creating a fork from the second volume: fly postgres create --fork-from authprodb:vol_1g67340ggznrydxwOutput: Connect: fly postgres connect -a authprodb3
I replied on the thread. |
|
Got a reply on the forum (but still nothing in email ...) Only saw it now ... didn't get a notification from the forum. We need a "Reminders" feature. Going to try the command now: fly postgres create --name new-db --vm-size shared-cpu-2x --volume-size 10 --initial-cluster-size 1 --region CHOOSE --org your-org --image-ref registry-1.docker.io/flyio/postgres:13 --stolon --snapshot-id vs_yaddayaddayaddaRe-ran the command to check the Using the latest snapshot: Creating app...
Setting secrets on app new-db...
Restoring 1 of 1 machines with image registry-1.docker.io/flyio/postgres:13
Waiting for machine to start...Gotta go be a Dad for a few hours. BRB. ⏳ |
great. going to have to run the command again ... |
|
Posted a comment on the forum topic to this effect: Not much |
|
Given the error above, I had deleted But sadly, Fly.io is not fully operational right now: https://fly.io/dashboard/dwyl-auth/status 
So the response is: Will need to revisit this in the morning. ⏳ |
|
At home now and still getting error trying to revive the DB: Creating postgres cluster in organization dwyl-auth-546
Creating app...
Setting secrets on app new-db...
Restoring 1 of 1 machines with image registry-1.docker.io/flyio/postgres:13
Waiting for machine to start...
Machine 784e679b453408 is created
==> Monitoring health checks
Waiting for 784e679b453408 to become healthy (started, 0/3)
Error: context deadline exceeded |
|
But when visiting the dashboard, we see that the Replied on the thread: https://community.fly.io/t/where-is-our-data-postgres-database-volume-0mb/16247/16?u=dwyl_auth |
|
On the advice of the Daniel https://community.fly.io/t/where-is-our-data-postgres-database-volume-0mb/16247/18 I created a new DB instance to attempt to recover the data ... https://fly.io/dashboard/dwyl-auth-546 It didn't work and the thread was automatically closed without any resolution. 😢 
The
|
Is anyone else seeing an Internal Server Error on https://auth.dwyl.com/ ??
Todo
authprodapp. 🤞The text was updated successfully, but these errors were encountered: