Download the Web.

Self-hosted tools to download audio & video resources, scrape various content types, direct download online resources and retrieve torrent file contents quickly and easily.

Build for the Raspberry Pi 4 (2GB+)

Mockup

Downloading resources from the web should be as quick and easy as possible. This is how it works:

What do you want to download?

Enter the URL or Torrent URI of the resource you want to download. Each Web DL handler will check and show if it can be used. Simply pick a handler to decide how to download and process your request.

How do you want to download it?

After selecting a handler you can configure the handler specifics (when available) to further specify how to download and process your request.

It will be handled...

Your request will be queued, and when resources are available, will download and process in the background. You'll get live updates as your request starts downloading and processing.

Get your files.

After Web DL has finished downloading and processing your request you can view it's details, download files and go deep into the timeline, raw data and logs.

Features

Here are some select features Web DL offers to make it even safer and easier to use.

User management

Create and manage different authenticated users to separate requests securely and manage access to them.

Secure

Web DL's code is publicly available to see and review and will run locally. Only you can see your requests and it's contents.

Local cloud

Requests are stored in your own cloud (on the device you run Web DL on). Client devices that interact with Web DL only receive the files you decide to download.

Lightweight

Web DL runs light! It's first iteration was created for usage on a Raspberry Pi 3 Model B. The latest version of Web DL only requires a little over 1GB of memory and can be run on any ARM device.

Locally accessible

Your requests are accessible to all devices on your local network, similar to a NAS. All you need is a browser.

Responsive

Whether you use a mobile, tablet or ultra-wide monitor, Web DL's UI can handle it all.

Many supported sites

Web DL supports many different web resources. Download from YouTube, Twitch, SoundCloud, MixCloud, Imgur, torrents and many others.

Even more features

From compressing all downloaded resource files to live status updates using a secure websocket connection, Web DL contains even more features to make it easier to use.

Want to add some more?

🛠️ along with me!

How to install

Setting up Web DL requires some manual steps, but the setup process has been designed and developed to be as easy as possible. All you need is Git and Docker.

Clone

Start by cloning/downloading the API and Website repositories from the GitHub page linked below.

Configure

Copy the following files and fill in the empty fields.

.env.dist

→

.env

Execute

Install Docker and Docker compose in order to be able to build and run the required API and website containers.

Deploy

Run the following command to build and start it all.

make start

Go and get it all.

All code is available on GitHub.

👀 On GitHub

Hello World.

I created this stack mainly for personal use. Since I often grabbed resources from the internet I created several tools/scripts to make it easier for me. What you see here is it's latest iteration (bringing it all together).