Skip to content

Bitcoin 0.32.2 upgrade - #3239

Merged
TheBlueMatt merged 12 commits into
lightningdevkit:mainfrom
arik-so:bitcoin-0.32.2-upgrade
Aug 16, 2024
Merged

TheBlueMatt merged 12 commits into
lightningdevkit:mainfrom
arik-so:bitcoin-0.32.2-upgrade

Conversation

@arik-so

@arik-so arik-so commented Aug 13, 2024

Copy link
Copy Markdown
Contributor

As the title suggests, but still unfortunately a work in progress.

@arik-so
arik-so force-pushed the bitcoin-0.32.2-upgrade branch from 2bf94c4 to db61c59 Compare August 13, 2024 23:12
@codecov

codecov Bot commented Aug 13, 2024 •

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 82.44804% with 76 lines in your changes missing coverage. Please review.

Project coverage is 90.39%. Comparing base (5ab40b2) to head (36b484a).
Report is 48 commits behind head on main.

Files Patch % Lines
lightning/src/offers/invoice.rs 57.14% 0 Missing and 9 partials ⚠️
lightning/src/io/mod.rs 77.77% 6 Missing and 2 partials ⚠️
lightning/src/routing/router.rs 0.00% 7 Missing ⚠️
lightning/src/util/sweep.rs 12.50% 6 Missing and 1 partial ⚠️
lightning-rapid-gossip-sync/src/processing.rs 25.00% 0 Missing and 6 partials ⚠️
lightning/src/util/ser.rs 87.23% 2 Missing and 4 partials ⚠️
lightning/src/chain/onchaintx.rs 50.00% 5 Missing ⚠️
lightning/src/events/bump_transaction.rs 33.33% 3 Missing and 1 partial ⚠️
lightning-invoice/src/lib.rs 84.21% 3 Missing ⚠️
lightning-persister/src/fs_store.rs 62.50% 3 Missing ⚠️
... and 12 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3239      +/-   ##
==========================================
+ Coverage   89.75%   90.39%   +0.63%     
==========================================
  Files         123      124       +1     
  Lines      102158   107249    +5091     
  Branches   102158   107249    +5091     
==========================================
+ Hits        91695    96947    +5252     
+ Misses       7773     7717      -56     
+ Partials     2690     2585     -105     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@arik-so
arik-so force-pushed the bitcoin-0.32.2-upgrade branch 12 times, most recently from 458d5cf to 9d5a386 Compare August 14, 2024 16:33
@arik-so
arik-so marked this pull request as ready for review August 14, 2024 18:03
@arik-so
arik-so force-pushed the bitcoin-0.32.2-upgrade branch from 4c829f4 to 203803c Compare August 14, 2024 19:02
Comment thread lightning-rapid-gossip-sync/src/lib.rs Resolved
Comment thread lightning/src/lib.rs Resolved
Comment thread lightning/src/lib.rs Outdated
}

pub fn read_to_end<D: io::Read>(mut d: D) -> Result<alloc::vec::Vec<u8>, io::Error> {
pub fn read_to_end<D: Read>(mut d: &mut D) -> Result<alloc::vec::Vec<u8>, io::Error> {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be dropped in favor of read_to_limit?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be, but would probably be better in a followup because it does produce some errors.

Comment thread lightning/src/util/ser.rs Resolved
Comment thread lightning/src/blinded_path/mod.rs Resolved
@TheBlueMatt

Copy link
Copy Markdown
Collaborator

We'll probably end up with a handful of followups to upstream stuff to bitcoin-io/drop our usage of Cursor/remove no-std features, but they can be in a followup.

Comment thread lightning-block-sync/src/utils.rs Resolved
@arik-so
arik-so force-pushed the bitcoin-0.32.2-upgrade branch 7 times, most recently from 48bb54e to 07e0600 Compare August 15, 2024 01:13
Comment thread fuzz/src/full_stack.rs Resolved
Comment thread lightning/src/lib.rs Resolved
Comment thread lightning/src/offers/invoice.rs Resolved
Comment thread lightning/src/offers/invoice.rs Resolved
Comment thread lightning/src/util/macro_logger.rs Resolved
Comment thread lightning-transaction-sync/Cargo.toml Resolved
@arik-so
arik-so force-pushed the bitcoin-0.32.2-upgrade branch 2 times, most recently from 215b556 to df9aa8f Compare August 15, 2024 06:57
@arik-so
arik-so force-pushed the bitcoin-0.32.2-upgrade branch 2 times, most recently from c6a34da to b72b5e9 Compare August 15, 2024 16:57
@tnull
tnull self-requested a review August 15, 2024 17:35
Comment thread lightning/src/io/mod.rs Resolved
@arik-so
arik-so force-pushed the bitcoin-0.32.2-upgrade branch 2 times, most recently from 4b824f5 to 768c040 Compare August 15, 2024 20:22
@arik-so
arik-so requested a review from TheBlueMatt August 15, 2024 21:41
@arik-so
arik-so force-pushed the bitcoin-0.32.2-upgrade branch 2 times, most recently from ebb80e6 to 7462995 Compare August 16, 2024 06:19

@tnull tnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a first pass. Already looks pretty good.

AFAICT, you can (and should) drop most to almost all (?) changes related to lightning-transaction-sync if you'd simply bump rust-esplora-client and rust-electrum-client to their latest versions which have already been upgraded to rust-bitcoin 0.32.

Comment thread lightning/src/offers/parse.rs Resolved
Comment thread lightning/src/util/ser_macros.rs Resolved
Comment thread lightning-transaction-sync/src/util.rs Resolved
Comment thread lightning-invoice/Cargo.toml Resolved
Comment thread lightning-rapid-gossip-sync/src/lib.rs Resolved
Comment thread lightning/src/util/ser.rs Resolved
In anticipation of the rust-bitcoin upgrade, which incorporates its
own `io::Read` implementation, we need to make our usage compatible
with dropping `std::io` and `core2::io`.

Notably, in version 0.32.2, `bitcoin::io`'s `Read` is no longer
implemented for `&mut R where R: Read + ?Sized`, which results in
errors anytime `&mut &mut Readable` is passed instead of
`&mut Readable`.

This commit fixes those instances.
In lieu of using `Seek` and `SeekFrom`, we will instead rely
exclusively on the `io::Cursor` type.
The rust-bitcoin upgrade will introduce `bitcoin::io` module, which
will be missing a necessary subset of traits.

To accommodate those traits' future implementations, we move the
`lightning::io` module to its own file, where we will be able to
implement the missing trait subset in the next commit.
The `bitcoin::io` Cursor will miss a bunch of functionality that we were
using that we will reimplement here, most critically `set_position`.
@arik-so
arik-so force-pushed the bitcoin-0.32.2-upgrade branch from 7462995 to e07cccb Compare August 16, 2024 16:43

@TheBlueMatt TheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically LGTM. Feel free to squash and I think we should just land this and then do fixups later.

Comment thread lightning/src/util/ser.rs
impl<'a, R: Read> Read for BufReader<'a, R> {
#[inline]
fn read(&mut self, output: &mut [u8]) -> io::Result<usize> {
let input = self.fill_buf()?;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically if we want to get fancy we can ignore the whole buffer nonsense here and actually just do a straight read, but I'm also hoping this code never makes it into a release at all (ie rust-bitcoin updates fast enough) so its not worth worrying about too much.

Taproot is not yet being actively used, and this commit suppresses
the noisy warning generation.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.
Version 0.32.2 of `rust-bitcoin` deprecates a number of methods that
are commonly used in this project, most visibly `txid()`, which is
now called `compute_txid()`. This resulted in a lot of warnings, and
this commit is part of a series that seeks to address that.
@arik-so
arik-so force-pushed the bitcoin-0.32.2-upgrade branch from e07cccb to 36b484a Compare August 16, 2024 17:32

@TheBlueMatt TheBlueMatt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. There's probably a few things to clean up here and there but we should land this to get it out of the way and can clean up in a followup.

@TheBlueMatt
TheBlueMatt merged commit 43dcf2f into lightningdevkit:main Aug 16, 2024

@tnull tnull left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Post-merge ACK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants