Linux

Use Apple Shortcuts and Webhooks to Download Videos

Jake

Jul 9, 2024 — 5 min read

This is assuming you are using example.com as your domain and you already have DNS pointing to the server you are working on. This is also assuming you know your public IPv4 address and your IPv6 subnet.

First lock down the firewall

sudo ufw allow from any to any port 80 proto tcp
sudo ufw allow from any to any port 443 proto tcp
sudo ufw limit from YOUR_PUBLIC_IPV4/32 to any port 22 proto tcp
sudo ufw limit from YOUR_PUBLIC_IPV6_SUBNET::/64 to any port 22 proto tcp
sudo ufw enable

Install the required packages.

sudo apt update
sudo apt install python3 webhook nginx ffmpeg certbot python3-certbot-nginx

Use uuidgen to create an API key for yourself.

uuidgen

Create the webhook configuration file.

sudo nano /etc/webhook.conf

Enter this content in /etc/webhook.conf. Be smart and replace YOUR_API_KEY.

[
  {
    "id": "media",
    "execute-command": "/etc/webhook.d/media.sh",
    "response-message": "Attempting to download media.",
    "trigger-rule":
    {
      "match":
      {
        "type": "value",
        "value": "YOUR”_API_KEY",
        "parameter":
        {
          "source": "query",
          "name": "key",
        },
      },      
    },
    "pass-arguments-to-command":
    [
      {
        "source": "query",
        "name": "id",
      },
      {
        "source": "query",
        "name": "url",
      },
   ],
 },
  {
    "id": "delete",
    "execute-command": "/etc/webhook.d/delete.sh",
    "response-message": "Attempting to delete media.",
    "trigger-rule":
    {
      "match":
      {
        "type": "value",
        "value": "YOUR”_API_KEY",
        "parameter":
        {
          "source": "query",
          "name": "key",
        },
      },
    },
    "pass-arguments-to-command":
    [
      {
        "source": "query",
        "name": "id",
      },
   ],
 },
 ]

Make the directory for the Webhooks scripts

sudo mkdir /etc/webhook.d/

Create the media script.

sudo nano /etc/webhook.d/media.sh

Enter this content in the /etc/webhook.d/media.sh file.

#!/bin/bash

varFileFind=""

varFileFind=$(find /var/www/EXAMPLE.COM/media/ -type f -name ""$1".mp4")

if [ $varFileFind != "" ]
then

	varFileExists=1

	else

	varFileExists=0
fi

if [ $varFileExists = 0 ]
then
	printf "0" > /var/www/EXAMPLE.COM/media/$1.txt
	
	yt-dlp  -f 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best' --output "/var/www/EXAMPLE.COM/media/$1.mp4" $2
	
	if [ $? -eq 0 ]
	then
		printf "1" > /var/www/EXAMPLE.COM/media/$1.txt
	else
		printf "2" > /var/www/EXAMPLE.COM/media/$1.txt
	fi

fi

if [ $varFileExists = 1 ]
then
	print "1" > /var/www/EXAMPLE.COM/media/$1.txt
fi

Create the file for the delete script.

sudo nano /etc/webhook.d/delete.sh

Entire this content in the /etc/webhook.d/delete.sh file

#!/bin/bash

rm /var/www/EXAMPLE.COM/media/$1.txt
rm /var/www/EXAMPLE.COM/media/$1.mp4

Make the scripts executable.

sudo chmod +x /etc/webhook.d/delete.sh
sudo chmod +x /etc/webhook.d/media.sh

Create an nginx server block to handle the Webhooks. Remember Webhooks natively use port 9000 unencrypted so we must install nginx and use LetsEncrypt to establish a secure reverse proxy.

sudo nano /etc/nginx/sites-available/EXAMPLE.COM

Enter this into the file /etc/nginx/sites-available/EXAMPLE.COM file.

server {

	root /var/www/EXAMPLE.COM;

	index index.html index.htm;

	server_name EXAMPLE.COM;

	location / {
		try_files $uri $uri/ =404;
	}

	location /hooks/ {
		proxy_pass http://127.0.0.1:9000/hooks/;
	}

}

Link the web server configuration file to the enabled directory and disable the default.

sudo ln -s /etc/nginx/sites-available/EXAMPLE.COM /etc/nginx/sites-enabled/EXAMPLE.COM
sudo rm /etc/nginx/sites-enabled/default

Create the web directory.

sudo mkdir -p /var/www/EXAMPLE.COM/media

Correct the permissions needed to use the directory.

sudo chown -R www-data:www-data /var/www/EXAMPLE.COM/

Test the nginx configuration

sudo nginx -t

Restart nginx if it passed.

sudo systemctl restart nginx

Get a certificate for your new domain and have it re-configure your nginx server block correctly.

sudo certbot --nginx -d EXAMPLE.COM

Get a fresh copy of `yt-dlp` and make it executable. Run this again if you get watermarks on TikTok videos.

sudo wget https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -O /usr/local/bin/yt-dlp
sudo chmod a+rx /usr/local/bin/yt-dlp

How to call the webhooks

At this point you can feed it a Webhook and get it to pull a video down from the internet to the EXAMPLE.COM/media/ folder. It is what you name it.

Here’s an example

Say I want to download my TikTok in original format because I want to keep it if TikTok randomly takes it down. I’ll have a URL: https://www.tiktok.com/t/ZPREGqpeA/ I can take this and use a URL encoder to make sure it’s easier for the web server and script to handle. That will make it look like this: https%3A%2F%2Fwww.tiktok.com%2Ft%2FZPREGqpeA%2F.

Now I need to make a unique ID for the file. I do this by passing that same URL to something that will create a sha256 hash. https://www.tiktok.com/t/ZPREGqpeA/ becomes 8a97663d5ecdd7f3ea6470ff8734550bff3ff8b1ea30cb88e3debf1298f0b8c7.

Now I can take my API Key I randomly generated `7C7D4B74-ECB1-4E0B-A82C-55EE0B974793` and enter that also in the url. It’s going to look like this:

curl -X POST https://EXAMPLE.COM/hooks/media?key=7C7D4B74-ECB1-4E0B-A82C-55EE0B974793&id=8a97663d5ecdd7f3ea6470ff8734550bff3ff8b1ea30cb88e3debf1298f0b8c7&url=https%3A%2F%2Fwww.tiktok.com%2Ft%2FZPREGqpeA%2F

That will place a file named 8a97663d5ecdd7f3ea6470ff8734550bff3ff8b1ea30cb88e3debf1298f0b8c7.mp4 in /var/www/EXAMPLE.COM/media/ which I can download using https://EXAMPLE.COM/media/8a97663d5ecdd7f3ea6470ff8734550bff3ff8b1ea30cb88e3debf1298f0b8c7.mp4

Status Files

If I need to know if the download failed, it’s also going to generate a one digit status text file with the shame ID. That can be accessible at https://EXAMPLE.COM/media/8a97663d5ecdd7f3ea6470ff8734550bff3ff8b1ea30cb88e3debf1298f0b8c7.txt

A 0 is that it is still processing.

A 1 is that it succeeded.

A 2 is that it failed.

Delete Videos

If it failed or after you downloaded the file you should run the delete Webhook to free up the space. That can be run with this hook:

curl -X POST https://EXAMPLE.COM/hooks/delete?key=7C7D4B74-ECB1-4E0B-A82C-55EE0B974793&id=8a97663d5ecdd7f3ea6470ff8734550bff3ff8b1ea30cb88e3debf1298f0b8c7

Apple Shortcuts

When it comes to creating Apple Shortcuts for others, there’s no other easy way to send it to someone other than sharing an iCloud link. The best I can do is share this one here.