mtree bugs with filenames containing wildcards

Let's say I have a directory containing these files, one of which contains an asterisk:

file*
file2.txt
file.txt
file3.txt

Further assume they are different sizes and have different content.

If, in the output from mtree -c, file* is emitted before the others, the later mtree verify will apparently match all four files using the rule file* and emit errors on the other three. I observed this in the wild.

A workaround is to manually edit the emitted mtree data and move file* to be after the other non-wildcard files; then it will properly verify.

Additionally, the support for wildcards in the mtree format is not documented that I can see, and probably should be. Or perhaps it should be removed, since AFAICT, no tools are generating mtree files intentionally using this feature.

I observe a call to fnmatch in verify.c which MAY be the cause of this issue. (I am not certain).

Isn't mtree part of the base system? Could you send this as a bug report upstream please? (https://www.netbsd.org/support/send-pr.html)

Sure - pkg/56733 submitted. I'm not quite sure where to go with all these things; my apologies.

I also submitted an upstream issue (about another problem) on 2022-01-10.
Received zero response so far.

So I am not sure what is the workflow here.

@sskras

@sskras If your bug report was about a different problem, then it does not belong here.

There is no SLO on replies to bug reports.

@bsiegert

@bsiegert So I just realized something:

There is mtree in two places:

https://github.com/NetBSD/pkgsrc/tree/trunk/pkgtools/mtree/files (which is what this issue pertains to)

and

https://github.com/NetBSD/src/tree/trunk/usr.sbin/mtree

(and of course their CVS origins)

I note that these two trees aren't identical, and in fact are rather divergent. The pkgsrc version -- this one -- is portable, and in fact I maintain mtree-netbsd for Debian based on this version.

The NetBSD/src version is not portable, but appears newer (for instance, having added -F).

I did have engagement on 56733 but did not at the time realize that we were talking about two different mtrees!

So I am very much talking about the pkgsrc one, which is https://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/pkgtools/mtree/index.html . Presumably it is portable because pkgsrc itself is portable, to *BSD, Linux, and so forth.

I don't know how the NetBSD/src mtree flows into the pkgsrc mtree -- perhaps, at the moment, the answer is "it doesn't" - fair enough. All I know is that this patch fixes the issue for me:

--- verify.c.orig       2018-08-22 15:48:38.000000000 -0500
+++ verify.c    2022-03-01 09:45:04.854574448 -0600
@@ -142,9 +142,7 @@
                if (specdepth != p->fts_level)
                        goto extra;
                for (ep = level; ep; ep = ep->next)
-                       if ((ep->flags & F_MAGIC &&
-                           !fnmatch(ep->name, p->fts_name, FNM_PATHNAME)) ||
-                           !strcmp(ep->name, p->fts_name)) {
+                       if (!strcmp(ep->name, p->fts_name)) {
                                ep->flags |= F_VISIT;
                                if (compare(ep, p))
                                        rval = MISMATCHEXIT;

Noting this from an email conversation:

There are some other mtree implementations and forks from NetBSD; for instance:

https://github.com/archiecobbs/mtree-port (fork from pkgsrc; very old, doesn't even have -C)
https://github.com/archiecobbs/nmtree (more current fork from pkgsrc)

Like the pkrsrc mtree, these two seem to be outdated relative to the
usr.sbin mtree.

Other implementations:

libarchive, of course. (does not do verification)
FreeBSD has an semi-compatible mtree. I found it has issues with
filenames containing tildes, so I avoided it.
https://github.com/mratajsky/libmtree (appears to originate from
FreeBSD; apparently abandoned)
https://github.com/eric-switzer/mtree-parse (Python)
https://github.com/vbatts/go-mtree (re-implementation in Go)
https://github.com/vbatts/go-mtree (Rust parser)

The NetBSD pkgsrc tree is directly used as part of the Arch Linux
packaging format; https://aur.archlinux.org/packages/nmtree for details.
They also have a "git" mtree derived from archiecobbs/nmtree as above.

So effectively NetBSD pkgsrc tree is the ultimate upstream for a number
of other projects. It is probably reasonable to point many of those to
use archiecobbs/nmtree but I thought I ought to write down all the
options here for others that may be looking.

(geography/R-wk) Updated 0.5.0 to 0.6.0

restish: update to 0.14.0.

devel/ruby-mixlib-config: update to 3.0.27

devel/git-cliff: update to 0.9.0

git-cliff: update to 0.9.0.

Update to 9.01. From the changelog:

www/wiki-tui: update to 0.6.0

fig2dev: Update to 3.2.8b.

boost: Update to 1.81.0

archivers/par2: update to 1.1.0

libsidplayfp: update to 2.16.0.

devel/ruby-io-console: update to 0.8.2

detox: update to 3.0.1.

sysutils/pv: Update to 1.10.3

devel/ruby-rspec-its: udpate to 2.0.0

p5-XML-Parser: update to 2.48.

py-asn1: update to 0.6.3.

p5-YAML-Syck: update to 1.45.

(devel/R-rprojroot) Updated 2.0.4 to 2.1.1

p5-Template-Toolkit: update to 3.106.

bfs: update to 4.1.3. Changes:

net/ttl: update to 0.21.0

www/ruby-cgi: update to 0.5.2

Update to 0.41

Navigation Menu

Repository navigation

mtree bugs with filenames containing wildcards #101

205 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

mtree bugs with filenames containing wildcards #101

Description

Activity

bsiegert commented on Feb 28, 2022

jgoerzen commented on Mar 1, 2022

sskras commented on Mar 1, 2022

bsiegert commented on Mar 1, 2022

jgoerzen commented on Mar 1, 2022

jgoerzen commented on Mar 1, 2022

205 remaining items

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions