ERROR ImportDescriptors - Failed to import descriptor - new range must include current range

[chuegel]

After completing rescanning the accounts lss failes with following error:

• some lines were redacted

[2022/09/12 - 14:38:26]  INFO worker: Initial Block Download complete               blockHash=00000000000000000008120a2bb8275370b71382e202da863b649eba80aa43b3 blockHeight=753788
[2022/09/12 - 14:38:26]  INFO worker: Computing circulating supply...
[2022/09/12 - 14:42:54]  INFO worker: #RunTheNumbers successful                     actualSupply=19148473.42092331 BTC expectedSupply=19148681.25 BTC height=753789
[2022/09/12 - 14:42:58] ERROR ImportDescriptors - Failed to import descriptor || new range must include current range = [0,1002]-8: new range must include current range = [0,1002] descriptor=wpkh([7*******/84'/0'/0']xpub6**************/0/*)#asd***
[2022/09/12 - 14:42:58] ERROR worker: Failed while importing descriptors            error=ImportDescriptors - importdescriptor RPC failed
[2022/09/12 - 14:42:58]  INFO Shutdown server: in progress
[2022/09/12 - 14:42:58]  INFO Unloaded wallet successfully                  wallet=satstack
[2022/09/12 - 14:42:58]  INFO Shutdown server: done

Running latest satstack on MacOS
Core 23.0.0

[chuegel]

An hints on this issue?

[adrienlacombe]

no sorry, did you use a range?

[chuegel]

Thanks for your reply.
Well, everything was kept as defaults in Ledger (10 accounts to scan). When I looked into lss.jsonthe above account in the error message is right on top

[chuegel]

I managed to get it working. At least lss is starting properly.
Here is what I've done. Despite following the recommended installation method described here I went with the manual approach

I noticed that Ledger Live adds way more accounts into lss.json as the manual way, hence it takes almost two days to import the descriptors.
Unfortunately, when terminating lss and re-starting it, the whole Importing descriptors process starts all over....very frustrating

[ziggie1984]

yes, we definitely need to fix this, bc normally ledger knows about all the transactions/addresses after the first sync, which somehow should differentiate between an initial setup and a normal start of the bridge so that we only scan the blocks since we last left which is pretty fast also for more then >10 accounts

[d-rez]

Is Ledger fixing this? Same issue here on Windows. It was initially importing descriptors fine for over 4 hours (after deleting wallet file due to wrong version), then crashed with an error, now it can't resume.

[2022/12/05 - 15:00:33]  INFO worker: Computing circulating supply...
[2022/12/05 - 15:03:07]  INFO worker: #RunTheNumbers successful                     actualSupply=19224454.66594582 BTC expectedSupply=19224662.5 BTC height=765946
[2022/12/05 - 15:04:28] ERROR ImportDescriptors - Failed to import descriptor || new range must include current range = [0,1013]-8: new range must include current range = [0,1013] descriptor=wpkh([***/**'/0'/0']xpub6***/0/*)#***
[2022/12/05 - 15:04:28] ERROR worker: Failed while importing descriptors            error=ImportDescriptors - importdescriptor RPC failed
[2022/12/05 - 15:04:28] FATAL worker: Failed to send INTERRUPT signal               error=not supported by windows pid=18116
[process exited with code 1 (0x00000001)]

EDIT: Added link to another issue where I pasted the error that happened before this error started happening.

[ziggie1984]

which version of bitcoin-core are you running ?

[d-rez]

# bitcoind --version
Bitcoin Core version v24.0.0

[ziggie1984]

ok interesting, need to check with bitcoin-core 24, seems like the rpc command changed, I will give you feedback

[tonyholmes69]

+1 - also having this issue. Also running current GA bitcoind (Bitcoin Core 24.0).

[adrienlacombe]

thank you, I haven't tried with 24.0.0 yet

[d-rez]

Hey @adrienlacombe-ledger, any luck getting it to work with 24.0.0? Is downgrading the Core the only way to get it working again?

[chuegel]

I forgot to mention that the issue description applies to 23.0.0
Didn't managed to try it on 24.0.0 yet

[ziggie1984]

@d-rez tested it with 24.0.1 today, it works for me

[ziggie1984]

So I could construct your problem, it seems to be that you customized the depth variable for the accounts, which means as the error states be >1000 (check if it has a value smaller than 1000 and increase it to 1000), or you delete the wallet from your core node and rerun lss and it sshould also be fixed

For one account for example

"birthday":"2022/11/11",
 "depth": 1000

[d-rez]

Hey @ziggie1984 thanks for your comment,

This is a brand new wallet file that was created by the newest satstack release (v0.17) and Core v24. I deleted the old wallet file already because it didn't support descriptors so lss wasn't launching at all (as described in #87 (comment))

The default value for depth variable is 1000 as per the main repo README. I'm a bit confused on what to increase it to. I haven't customized any variables tbh.

I was using the default Ledger node setup flow, not the manual method

[ziggie1984]

Can you check in your lss.json which depth is shown there, or whether there is any depth variable ?

[d-rez]

I had to retry the Node config and there's no depth entry / definition in the lss.json generated by Ledger whatsoever. So it must be using whatever defaults there are.

After deleting the wallet file and going through the Node setup in Ledger, leaving values at default (10 accounts to scan), it just errors out again within a few minutes, seemingly on the first m/84'/0'/0 account that's inside lss.json.

[2022/12/19 - 00:57:04] ERROR ImportDescriptors - Failed to import descriptor || new range must include current range = [0,1013]-8: new range must include current range = [0,1013] descriptor=<first m/84'/0'/0 account from lss.json>
[2022/12/19 - 00:57:04] ERROR worker: Failed while importing descriptors            error=ImportDescriptors - importdescriptor RPC failed

[ziggie1984]

Ok understood, so it seems like that the previous ledger live version (not satstack) imported descriptors with a range 0..1013, and the new version does use default values.

Two countermeasure you could take to solve your problem:

1: go to your running bitcoin-core implementation and in the .bitcoin folder if you are running on linux and remove the wallet directory by typing rm -r satstack (this deletes the saved wallet of prior satstack usage. Then start lss or ledger live again and it should work

2. The other solution is to go to your lss.json file and add the depth of 1013 or greater to every account. then there is no need to delete the wallet dir in bitcoin-core.

Both solutions should work, because satstack always rescans the wallet from the birthday datum so it makes no difference deleting the wallet dir either.

[d-rez]

As mentioned, I've already removed the satstack wallet directory multiple times. I also made sure to delete it before trying this again today. Deleting the wallet directory does not solve the issue.

I'll try changing the depth in all those places tomorrow but this does not sound like a real solution either. I assume this number can change per account? Why 1013? I haven't done that many transactions for sure.

Or is that because my node is a full node and not a pruned node?

[ziggie1984]

no, I don't think one can import descriptors with a pruned node, bc you cannot retrieve sufficient history. You restarted bitcoind after deleting the wallet dir ? Seems weird that the wallet still seems to be there because somehow it complains about the old range which imo means there might still be a wallet dir handing around there.

Will try with ledger live today and their created lss.json, maybe I can run into your issue

[d-rez]

I restarted my node ~2 weeks ago last time when I commented in this repo's Issues, but not again. Just did now.

After restarting the node, deleting satstack wallet again and going through the process again from scratch, I waited until it finished #RunTheNumbers. Then clicked Next in the dialog so that Ledger Live would try connecting to the Node/LSS, then LSS choked after a few seconds and I got an error in CLI:

[2022/12/19 - 16:11:47] ERROR Failed to query status                        err=bitcoind unreachable: status code: 503, response: "Work queue depth exceeded"

I waited patiently and eventually it resumed and is now importing descriptors again. Live was complaining that it can't connect to Node but it's probably incomplete error handling on Ledger Live side.

[2022/12/19 - 16:15:11]  INFO worker: Importing descriptors                         duration=2m22s progress=0.64%

Last time it went there it crashed with the error from this Issue's title after around 4 hours, so I guess we'll see where it gets to. I've closed Live and will let LSS do its importing again so that Live doesn't spam the node with /blockchain/v3/explorer/status pings every 1s.

All in all... the experience of settings this up nowadays is very, very poor lol. I'll update this comment if/when importing descriptors finishes or crashes.

[ziggie1984]

Totally understand I am about to integrate this to the raspiblitz but need to add a few changes before I can release it. I release a patched version hopefully making the process a bit more smooth

Maybe already this week

[chuegel]

Wouldn't it make more sense to support Electrum/Fulcrum instead of this hacky solution? I mean, since this targets people running their own node, in 99% of the cases Electrum/Fulcrum is also running. Just a thought.

Edit: Sparrow Wallet is a good example

[adrienlacombe]

@chuegel yes, we are exploring this option.

[ziggie1984]

Ok was able to improve the loading times by 10x (when using 10 descriptors) going to release the PR today, now the amount of descriptors does not matter.

[ziggie1984]

@d-rez you could try my patch it improves startup speed enormous because it only rescan the blockchain once

[ziggie1984]

Other improvements will following which will fix the issue rescanning the wallet from the beginning all the time, maybe tomorrow

[d-rez]

Wait so after I've waited almost 24h for descriptors to import (still progressing, hasn't crashed at least yet, so there's that) it will do this all over again the next time I start LSS or reboot my PC?

[ziggie1984]

yess for now it will, but release the second PR today which will fix this, though you have to update so it makes no sense to sync now, but wait for the PR and then use the new version