Missing prev_txouts

[andreasgriffin]

Description

Syncing a wallet from scratch with kyoto for incoming tx and running

wallet.calculate_fee(tx)

gives

bdkpython.bdk.CalculateFeeError.MissingTxOut: out_points=[<bdkpython.bdk.OutPoint object at 0x716388266390>, <bdkpython.bdk.OutPoint object at 0x716388264e00>]

Steps to reproduce

On Signet, pip install bdkpython==2.0.0. Sync the following wallet with kyoto:

descriptor = "wpkh([d61bc696/84'/1'/0']tpubDCZsFxoXd76ZFhGHa9FMCJ6jf9drQeqrWcYdpb18nXfsYe2YeTkvXVB6iT2AXz5E5sSWxf1G1W3Sf7ehsMdmXEHTL8ogWZYkLA3DY8FRr98/<0;1>/*)#76zcgt8z"
scan_type = bdk.ScanType.RECOVERY(from_height=0)

After successful sync running

wallet.calculate_fee(tx)

for the incoming tx https://mempool.space/signet/tx/7792eff9b668b1ffb47779c1cec7cb7c9e929d5b066e2eebfa37432121322ef8 causes this error.

Script to reproduce issue

Download minimal_cbf_sync.py and run

python minimal_cbf_sync.py --network signet    --recovery-height 269700 "wpkh([d61bc696/84'/1'/0']tpubDCZsFxoXd76ZFhGHa9FMCJ6jf9drQeqrWcYdpb18nXfsYe2YeTkvXVB6iT2AXz5E5sSWxf1G1W3Sf7ehsMdmXEHTL8ogWZYkLA3DY8FRr98/0/*)" "wpkh([d61bc696/84'/1'/0']tpubDCZsFxoXd76ZFhGHa9FMCJ6jf9drQeqrWcYdpb18nXfsYe2YeTkvXVB6iT2AXz5E5sSWxf1G1W3Sf7ehsMdmXEHTL8ogWZYkLA3DY8FRr98/1/*)"

Tagging @ValuedMammal @thunderbiscuit @rustaceanrob

[andreasgriffin]

The missing previous tx is fundamental to CBF.
Fetching the previous txs from peers would leak wallet tx info.

[rustaceanrob]

This is a property of blocks in bitcoin. All inputs are referenced by a pointer via OutPoint. As you mentioned, fetching specific transaction for inputs reveals some data about the client. You might think, as did I, maybe we can download fake data? This also doesn't work. It is not possible to fetch historical data by OutPoint over the peer-to-peer network, which makes sense. If this was allowed, it would be a DoS vector for serving nodes.

I also want to expand on when you may not have OutPoint data available. If you are receiving a transaction, of course you do not have the data for the (all the) inputs. I would argue this doesn't matter in most cases. Why should the user care what an exchange paid in fees? In the Payjoin case though, this is a problem, as there are foreign inputs added during the negotiation process. To work around this, I think there is a add_foreign_utxo or similar in Wallet. In the sending case, the user will likely have the OutPoint data, as they are generating the transaction and thus know the corresponding UTXOs. Again Payjoins, coin joins, etc. are the limitation here.

Please leave this issue open for others to reference.