how to build gnu-tar with support for extended attributes

[codeskipper]

Output of brew config

HOMEBREW_VERSION: 4.0.11-128-gc516870
ORIGIN: https://github.com/Homebrew/brew
HEAD: c51687009428dd061a2dd90f46c11e0c7c1f9212
Last commit: 28 hours ago
Core tap origin: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 867d6cc6f4e4f6294fb5143139649d835004ad86
Core tap last commit: 4 days ago
Core tap branch: gnu-tar_with_xattr
Core tap JSON: 06 Apr 11:08 UTC
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 12
HOMEBREW_NO_INSTALL_FROM_API: set
Homebrew Ruby: 2.6.10 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: dodeca-core 64-bit kabylake
Clang: 14.0.0 build 1400
Git: 2.37.0 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 7.87.0 => /usr/bin/curl
macOS: 12.6.4-x86_64
CLT: 14.0.0.0.1.1661618636
Xcode: 14.2

Output of brew doctor

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: You have uncommitted modifications to Homebrew/homebrew-core.
If this is a surprise to you, then you should stash these modifications.
Stashing returns Homebrew to a pristine state but can be undone
should you later need to do so for some reason.
  cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core && git stash -u && git clean -d -f

Uncommitted files:
   M Formula/gnu-tar.rb

Warning: Some taps are not on the default git origin branch and may not receive
updates. If this is a surprise to you, check out the default branch with:
  git -C $(brew --repo homebrew/core) checkout master

Description of issue

When running the regular bottled gnu-tar with option --xattrs (to store/retrieve extended attributes inside an archive) I get a warning /usr/local/bin/gtar: XATTR support is not available.
After extracting from the resulting archive (also with with --xattrs argument, and the same warning) the extended attributes from the original file(s) are not present on the extracted file(s).

As an example, I just downloaded a Firefox installer, and I can see it has some extended attributes:

ls -@alh ~/Downloads/Firefox\ 111.0.1.dmg
-rw-r--r--@ 1 mart  staff   127M Apr  9 14:02 /Users/mart/Downloads/Firefox 111.0.1.dmg
	com.apple.macl	  72B
	com.apple.metadata:kMDItemDownloadedDate	  53B
	com.apple.metadata:kMDItemWhereFroms	 179B
	com.apple.quarantine	  57B

When running /usr/local/bin/gtar with --xattr I get the warning about the missing feature:

/usr/local/bin/gtar -cvf test.tar ~/Downloads/Firefox\ 111.0.1.dmg --xattrs
gtar: Removing leading `/' from member names
gtar: XATTR support is not available
/Users/mart/Downloads/Firefox 111.0.1.dmg

The use-case for this feature is to enable caching between Autopkg runs in a CI pipeline (GitHub Action). See actions/cache#777

In the source code for GNU tar, I can see we need a compiler flag HAVE_XATTRS. https://git.savannah.gnu.org/cgit/tar.git/tree/src/xattrs.c?h=release_1_34#n473

After some research, I tried to edit the formula to include --with-xattrs in the build arguments for configure. This runs OK, and I can see in the config.log a compiler flag HAVE_SYS_XATTR_H is now being set.
But HAVE_XATTRS is not set, and the resulting gtar still gives the same results.

From https://git.savannah.gnu.org/cgit/tar.git/tree/acinclude.m4#n49 my understanding is HAVE_XATTRS will be set only if all the listed functions are found, and I guess I'm missing the header files that declare those functions.

Any tips on how to find what dependencies I need to add to the formula to cover the developer kits that seem to be missing?

Also, I tried the gnutar package from MacPorts, and it has the same issue. I guess it could be fixed "upstream" for both, but where might that be?

[carlocab]

Use BSD tar. Trying to get it to work with GNU tar seems like a pretty deep rabbit hole. BSD tar is packaged with macOS, but also comes with the libarchive formula.

However, if you're in the mood to go spelunking, reading these might be a start:
https://stackoverflow.com/questions/42414607/tar-not-preserving-extended-attributes-of-trusted-namespace
https://bbs.archlinux.org/viewtopic.php?id=146269
https://bugzilla.redhat.com/show_bug.cgi?id=771927
https://lists.debian.org/debian-user/2012/06/msg00524.html
https://www.suse.com/support/kb/doc/?id=000018586

[codeskipper]

Thanks for the info on the issues I might run into on the other end of this rabbit hole @carlocab

I would otherwise prefer to use BSD tar as you suggest any given day. For my use case however, the existing code for GitHub Actions/cache prefers GNU tar over the built-in BSD version. This PRlists some issues that motivated the switch:
actions/cache#403
https://github.community/t/bizarre-issue-with-tar-archive-for-macos-build/145377
Cyberbeni/install-swift-tool#69

[cho-m]

The problem is macOS SDK doesn't provide l*xattr functions like:

• lgetxattr - https://www.gnu.org/software/gnulib/manual/html_node/lgetxattr.html
• lsetxattr - https://www.gnu.org/software/gnulib/manual/html_node/lsetxattr.html

EDIT: Also, some functions have different parameters like:

• getxattr - https://www.gnu.org/software/gnulib/manual/html_node/getxattr.html

  This function has extra offset and options parameters: macOS 11.1.

Since GNU tar code uses these for extended attributes support, you would need to rewrite the code in GNU tar.

---

If you want to go down this path, perhaps look at how BSD tar handles similar scenarios (e.g. libarchive has different logic under ARCHIVE_XATTR_LINUX and ARCHIVE_XATTR_DARWIN). Some examples below:

https://github.com/libarchive/libarchive/blob/master/libarchive/archive_read_disk_entry_from_file.c#L458-L461

#if ARCHIVE_XATTR_LINUX
		size = fgetxattr(fd, name, NULL, 0);
#elif ARCHIVE_XATTR_DARWIN
		size = fgetxattr(fd, name, NULL, 0, 0, 0);

https://github.com/libarchive/libarchive/blob/master/libarchive/archive_read_disk_entry_from_file.c#L466-L469

#if ARCHIVE_XATTR_LINUX
		size = lgetxattr(accpath, name, NULL, 0);
#elif ARCHIVE_XATTR_DARWIN
		size = getxattr(accpath, name, NULL, 0, 0, XATTR_NOFOLLOW);

[codeskipper]

thanks @cho-m !

That looks like a major undertaking. I'll explore a workaround for using BSD tar in actions/cache a bit more first.

[carlocab]

I also suggest reporting the actions/cache bug to the BSD tar project, if it reproduces with upstream BSD tar (and not just Apple's version of it). They might know how to fix the problem. (This should have probably been done in the first place before contemplating a switch to GNU tar.)

If it affects on Apple's bsdtar, it may still be worth reporting to Apple. (It's a lot harder to get a response to bug reports to Apple though.)