Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENG-1184, ENG-1286, ENG-1330] Rework native dependencies (+ deb fixes) #1685

Merged
merged 36 commits into from
Nov 17, 2023

Conversation

HeavenVolkoff
Copy link
Member

@HeavenVolkoff HeavenVolkoff commented Oct 27, 2023

Rework Spacedrive's native dependency build system to be a unified set of scripts that download/prepare/build/package all the required runtime native dependencies (ffmpeg libs, libheif, libpdfium) to run Spacedrive on all of its supported OSes.

Remove the old separate set of FFMpeg/libheif build scripts for macOS/Windows/Linux from the main repository

Create a new repository for the new build system: https://github.com/spacedriveapp/native-deps. This has the advantage of decoupling the native-deps from Spacedrive's release cycle, and also allows us to use releases instead of CI artifacts for distribution, which free us from the limits GitHub places on distributing artifacts (3 months expire limit, no public URL, requires using GitHub's API)

Rework preprep.mjs script to download and set up our new unified native-deps bundle for dev. Remove code for dealing with GitHub API, now that we use releases, that code is not needed anymore.

Rename macOS FFMpeg.framework to Spacedrive.framework

Remove ffmpeg libs install step from setup.sh

Change minimum supported macOS on arm to 11.0 (from 11.2)

Fix deb release issues raised by Lintian

This PR was tested on:

  • macOS arm64
    • M2 Max
    • M1 Pro
  • macOS x86
  • Linux x86
    • AMD Ryzen 9 7900
    • AMD Ryzen 7 3800x
  • Linux arm (aarch64)
    • VM running under macOS (M2 Max)
  • Windows x86
    • VM running under Linux (AMD Ryzen 9 7900)
  • Windows arm64 (Probably won't be able to test, because there are still other blockers for Spacedrive to support windows on arm)

For anyone going to test this PR, I recomend deleting your current thumbnail dev cache:
Linux: ${XDG_DATA_HOME:-${HOME}/.local/share}/spacedrive/dev/thumbnails
Windows: %appdata%\spacedrive\dev\thumbnails
macOS: ${HOME}/Library/Application\ Support/spacedrive/dev/thumbnails

Then run the normal dev setup:
./scripts/setup.sh or ./scripts/setup.ps1, pnpm i, pnpm prep, pnpm tauri dev

 - FFMpeg 6.0 uses some now removed deprecated functions
@HeavenVolkoff HeavenVolkoff self-assigned this Oct 27, 2023
@linear
Copy link

linear bot commented Oct 27, 2023

ENG-1330 Vendor FFmpeg for Linux

Create a FFmpeg build script for linux to compile our own version of ffmpeg, so we don't depend on the system version for the deb and appimage releases

@vercel
Copy link

vercel bot commented Oct 27, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
spacedrive-landing ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 17, 2023 8:22am
spacedrive-web ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 17, 2023 8:22am

@HeavenVolkoff HeavenVolkoff changed the title Rework FFMpeg build scripts [ENG-1184, ENG-1330] Rework FFMpeg build scripts Oct 27, 2023
@linear
Copy link

linear bot commented Oct 27, 2023

ENG-1184 Rework windows ffmpeg script to use zig

Zig offers a self-contained C/C++ cross compiler setup that is much easier to use than the current setup we are using for compiling ffmpeg for windows. Besides making easier to maintain the current building scripts, it also has out of the box support for windows arm64, which would remove one of the blockers to support this platform in Spacedrive

 - Update zig for windows target
…essing

 - Disable opengl and directx for libplacebo, ffmpeg only supports vulkan when using it
 - Add WIN32_LEAN_AND_MEAN to global cflags to optimize windows api usage
 - Fix 99-heif.sh incorrect bsdtar flag
…enCL build

 - Fix Dockerfile step for cleaning up the out dir
 - Improve licensing handling
 - Fix soxr failing to download due to sourcefourge
 - Only patch zimg on windows targets
 - Tell cmake to hide libheif symbols
 - Add lzo dependency
 - Publish source for the built libs
 - Add warning for missing nasm in tauri.mjs
 - Remove ffmpeg install from setup.sh
 - Add download logic for our linux ffmpeg bundle in preprep.mjs
 - Rename FFMpeg.framework to Spacedrive.framework
 - Centralize the macOS native deps build with the windows and linux one
 - Change the preprep script to only download our native deps
 - Remove old macOS ffmpeg build scripts
fogodev
fogodev previously approved these changes Nov 15, 2023
@HeavenVolkoff HeavenVolkoff linked an issue Nov 17, 2023 that may be closed by this pull request
@HeavenVolkoff HeavenVolkoff changed the title [ENG-1184, ENG-1286, ENG-1330] Rework native dependencies [ENG-1184, ENG-1286, ENG-1330] Rework native dependencies (+ deb fixes) Nov 17, 2023
Copy link
Contributor

@Brendonovich Brendonovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lfg

@HeavenVolkoff HeavenVolkoff added this pull request to the merge queue Nov 17, 2023
Merged via the queue into main with commit 2e6e00b Nov 17, 2023
11 checks passed
@HeavenVolkoff HeavenVolkoff deleted the eng-1330-1184-rework-ffmpeg branch November 17, 2023 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ENG-1286] Spacedrive fails to start after .deb install
4 participants