Transcoder Slaves

[codegrizzl]

Transcoding of videos is a process which can take a couple of minutes. This is even more iminent when ripping blurays.

Just ripping the files from Bluray or DVD to memory is much faster than the followed transcoding.

Now here comes the new feature. Beeing able to outsource transcoding to multiple machines.

Use cases:

• Outsource transcoding to an external machine without requiring to install any drives in this machine
• Transcode multiple movies at once by more than one pc automatically
• Speed up

Required:

• Shared storage
• Socket communication
• ISO must be ripped to memory before a transcode is started (only implemented for Blurays right now)

I am not sure how much sens this makes for anything other than movies/tv series.

Initial implementation idea:
Master and slave docker container. The master has to run at all time and is not much different than the current implementation. This master can connect to single/multiple slaves via a network protocol (primitive socket).
After any disk is ripped it will communicate with each slave in a configured order (set by user) if the slave is available and checks if a job is already running on this slave. If a slave is available the master sends all information to the slave. These infos contain where the raw rip is stored and which configuration should be used for the rip.
This requires all slaves and master to access the same storage location.
The slave checks if data is accessible and starts the handbrake job.
While the job is running either the slave will send progress and log information back to the master or the master will check all slaves periodicly. Second will have the advantage to check if the slave is still alive.

For configuration the master can add transcoding slaves and at which order they are used. Including the master as transcoder. Master can also be excluded.

Slaves don't have any config values except ip and port. Any other config will be transfered by the master with each transcoding job.

[microtechno9000]

Would love to be able todo this. We are working on upgrading ARM in the background, some of which would then support this.

As you noted there are quite a few changes that would be required. In its current state the ARM ripping code wouldn't support this.

Will leave this as an enhancement and put on the back burner.

[mneimeyer]

I just installed ARM with the intention of replacing the HandBrakeCLI binary with a bash script to move the raw files to the completed folder and then use the Tdarr project to convert those (distributed across my network) to the final product.

Any reason this shouldn't work?

Pro: Require no changes to the underlying ARM project.
Con: Requires additional setup and monitoring progress/status on another system.

[codegrizzl]

Was not aware of Tdarr existing. This actually would solve most of this feature.
There is no reason this should not work.

As I just flew over the features of tdarr it is mostly a automatic transcoder converting between different codecs. I don't know how good it works with raw image files/folders.
ARM currently first scans ripped ISOs and afterwards starts the transcode based on the scanned information. If you just transcode everything then this does not matter.

If Tdarr is able to do that ARM would only require a configuration to add Tdarr as a transcoding option. This would make this task easier for future users.

[mneimeyer]

Phase 1 complete. This script (without the .txt extension, purely to allow upload) lives in /usr/local/bin/

Editing arm.conf and replacing HANDBRAKE_CLI and HANDBRAKE_LOCAL with HandBrakeFake resulted in the raw mkv being copied over and ready for other programs to work on the output of ARM.

Phase 2 will be to install and configure tdarr.

HandBrakeFake.txt