Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pad runtime to multiple of 512 bytes #602

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

harrytuttle
Copy link

If the squashfs filesystem starts at a multiple of 512bytes offset
it is possible to replace the runtime (using fdisk) or possibly
inject it (using dd) with a dos partition table at offset 446,
similarly to how the 3 AppImage magic bytes are inserted at offset 8
(the partition table is only 64bytes long, and not all of those are
actually needed)

This way the appimage can be mounted by existing tools like gnome-disk-image-mounter
which reads from the partition table the offset of the filesystem

Now if someone finds a way to fit a valid partition table inside an elf binary we could reach the holy grail of an appimage that can be both executed and easily mounted.

If the squashfs filesystem starts at a multiple of 512bytes offset
it is possible to replace the runtime (using fdisk) or possibly
inject it (using dd) with a dos partition table at offset 446,
similarly to how the 3 AppImage magic bytes are inserted at offset 8
(the partition table is only 64bytes long, and not all of those are
actually needed)

This way the appimage can be mounted by existing tools like gnome-disk-image-mounter
which reads from the partition table the offset of the filesystem
2 magic bytes are inserted at offset 510 to make the appimage into
in a dos disk image

4 bytes are then written in little endian byte order at offset 454
representing the offset of the first partition (in 512bytes sectors)

A total of only 6 bytes (or less) are changed, and these are sufficient for
gnome-disk-image-mounter to find the squashfs filesystem. The rest isn't
important: chs address, flats, partition type, lba size in sectors, etc...
can be ignored.

In case these 6 bytes impact too much the elf structure on other architectures
(particolarly on 32bit ones, where elf have different sizes for internal
data structures) it could still be possibile to inject the 4 bytes at offset
470, 486 or 502 (for the second, third and fourth primary dos partitions):
tools like gnome-disk-image-mounter will search all of those and mount only
the valid filesystems they will find.

Tested only on x86_64 for now.
@probonopd
Copy link
Member

Thanks, very clever. However the build fails with:

./runtime --appimage-offset
/AppImageKit/build/src/build-runtime.sh: line 82: ./runtime: Text file busy
/AppImageKit/build/src/build-runtime.sh: line 82: / 512 : syntax error: operand expected (error token is "/ 512 ")

https://travis-ci.org/AppImage/AppImageKit/jobs/321476534#L3023-L3025

calculate only once ./runtime --appimage-offset and store it in $APPIMAGEOFFSET
use that variable both to pad the runtime and to inject the first sector of the
first partition
otherwise the embedded runtime will lack the partition table.
@harrytuttle
Copy link
Author

I tried to fix it, the partition table is written correctly i think, at least in the x86_64 version. I don't know why the build fails on travis, when i build on my machine i get a build/out/appimagetool.AppDir/AppRun that builds working, properly padded, partitioned and mountable appimages.

@probonopd
Copy link
Member

Can you please attach such an AppImage (zipped) here please? Thanks.

@harrytuttle
Copy link
Author

I'm now trying to follow what test-appimagetool.sh is doing, it also fails on my machine, maybe i'll figure it out. Anyway here's my build result:
appimagetool-x86_64.zip

@probonopd
Copy link
Member

I can verify that this AppImage can be mounted using

$ gnome-disk-image-mounter /home/me/Downloads/appimagetool-x86_64.AppImage

I do not know how ELF-spec compliant this is. I think the proper way to do would be to have an ELF section that holds the MBR, and ensure that this ELF section always ends up at the correct offset in the binary. Not sure how to do this though.

Also, what exactly is the use case of this? Is this really much more convenient than running the AppImage with --appimage-mount or --appimage-extract?

@harrytuttle
Copy link
Author

Admittedly it's not a must have feature, i just like being able to right-click on an AppImage and open it with the already existing image mounter like the old isofs appimages.
And admittedly it may be too crude on the elf side, i don't know how to properly create some space that early in the elf file structure.
So i understand it may not be added as is, but at least the idea is here as a proof of concept, the 512 bytes padding looks safe (initially the pull request included only the padding), and if i want to build AppImages this way i can easily carry forward this 5 line patch.

@TheAssassin
Copy link
Member

if i want to build AppImages this way i can easily carry forward this 5 line patch

Well, first of all, you need to make your PR build fine. At the moment, this is a "okay, let's merge it" thing, but it has shown to fail certain tests. Please make sure your PR meets the quality we require, then we review it and reconsider merging. Thanks.

@probonopd
Copy link
Member

For merging, we'd need to properly ensure that this works on all architectures and independently of the size of the runtime. We'd also need to ensure that we still have a valid ELF file. Until then, I'd consider this as an interesting proof-of-concept experiment.

@TheAssassin
Copy link
Member

We'd also need to ensure that we still have a valid ELF file.

JFYI: qemu-arm-static can't run AppImages after the magic bytes are written (see #699 (comment)), therefore we should try to find out how much we can change until the platforms and/or qemu-* (and friends) won't be able to run AppImages any more.

@TheAssassin
Copy link
Member

Unexpected side effect of renaming of master branch. Sorry for the noise. Reopening.

@TheAssassin
Copy link
Member

Cannot reopen as the branch no longer exists. Strange, that never happened before...

@TheAssassin TheAssassin changed the base branch from appimagetool/master to master December 23, 2018 14:27
@TheAssassin
Copy link
Member

Changing the base helped. Really reopening now.

@TheAssassin TheAssassin reopened this Dec 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants