- Introduction
- Summary of required software
- Step 1: Make Ubuntu headless
- Step 2: Install a Media Streaming System:
- Step 3: Install bittorrent client and indexers:
- Step 4: Automounting USB
- Step 5: Setup unified interface using Organizr and EasyServerMonitor:
- Step 6: Users, Groups and Permissions
- Final Step: Client software
- Miscellaneous
- Extras
- Further Reading
- TODO:
In July, 2020, I setup an old laptop with a busted screen as a Linux media server to download and stream movies and TV episodes over my LAN to various wifi devices (laptops, tablets and smart TVs). Below are the notes of the various steps taken and software installed in order to achieve this.
The details of my old laptop and default setup are given below:
-
Laptop Model: SAMSUNG model: NP300E5X-U01IN
-
Output if inxi -F:
CLICK HERE
admin@MediaServer:~$ inxi -F
System: Host: MediaServer Kernel: 5.4.0-42-generic x86_64 bits: 64 Console: tty 0 Distro: Ubuntu 18.04.4 LTS
Machine: Device: laptop System: SAMSUNG product: 300E4C/300E5C/300E7C v: 0.1 serial: N/A
Mobo: SAMSUNG model: NP300E5X-U01IN v: FAB1 serial: N/A
UEFI [Legacy]: Phoenix v: P06RAC date: 10/25/2012
Battery BAT1: charge: 47.5 Wh 100.0% condition: 47.5/47.5 Wh (100%)
CPU: Dual core Intel Core i3-2310M (-MT-MCP-) cache: 3072 KB
clock speeds: max: 2100 MHz 1: 843 MHz 2: 822 MHz 3: 844 MHz 4: 805 MHz
Graphics: Card-1: Intel 2nd Generation Core Processor Family Integrated Graphics Controller
Card-2: NVIDIA GF108M [GeForce GT 620M]
Display Server: X.org 1.20.8 drivers: i915,nvidia tty size: 192x49 Advanced Data: N/A out of X
Audio: Card-1 NVIDIA GF108 High Definition Audio Controller driver: snd_hda_intel
Card-2 Intel 7 Series/C216 Family High Definition Audio Controller driver: snd_hda_intel
Sound: Advanced Linux Sound Architecture v: k5.4.0-42-generic
Network: Card-1: Qualcomm Atheros AR9485 Wireless Network Adapter driver: ath9k
IF: wlp2s0 state: up mac: 50:b7:c3:b2:80:88
Card-2: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller driver: r8169
IF: enp3s0 state: down mac: 50:b7:c3:7e:36:ef
Drives: HDD Total Size: 500.1GB (56.0% used)
ID-1: /dev/sda model: ST500LM030 size: 500.1GB
Partition: ID-1: / size: 458G used: 261G (61%) fs: ext4 dev: /dev/sda1
RAID: No RAID devices: /proc/mdstat, md_mod kernel module present
Sensors: System Temperatures: cpu: 55.0C mobo: N/A
Fan Speeds (in rpm): cpu: N/A
Info: Processes: 155 Uptime: 18:44 Memory: 2323.2/3714.4MB Init: systemd runlevel: 3
Client: Shell (bash) inxi: 2.3.56 -
Detailed hardware specs of this old laptop are hard to find. A detailed datasheet has been obtained from icecat. It is referred below:
icecat: Datasheet for Samsung NP300E5X Notebook.
In case the link rots, parts of the datasheet table (as of Aug 4, 2020), is reproduced below:
CLICK HERE
| Processor | |
| Processor manufacturer | Intel |
| Processor model | i3-2310M |
| Processor frequency | 2.1 GHz |
| Processor family | 2nd gen Intel® Core™ i3 |
| Processor cores | 2 |
| Processor threads | 4 |
| System bus rate | 5 GT/s |
| Processor cache | 3 MB |
| Processor cache type | Smart Cache |
| Processor socket | BGA 1023 |
| Processor lithography | 32 nm |
| Processor operating modes | 64-bit |
| Processor series | Intel Core i3-2300 Mobile series |
| Processor codename | Sandy Bridge |
| Bus type | DMI |
| FSB Parity | NO |
| Stepping | J1 |
| Motherboard chipset | Intel® HM75 Express |
| Thermal Design Power (TDP) | 35 W |
| Tjunction | 100 °C |
| Maximum number of PCI Express lanes | 1 |
| PCI Express slots version | 2.0 |
| PCI Express configurations | 1x16,2x8,1x8+2x4 |
| CPU multiplier (bus/core ratio) | 21 |
| ECC supported by processor | No |
| Memory | |
| Internal memory | 4 GB |
| Internal memory type | DDR3-SDRAM |
| Memory clock speed | 1333 MHz |
| Memory form factor | SO-DIMM |
| Memory layout (slots x size) | 1 x 4 GB |
| Memory slots | 2x SO-DIMM |
| Storage | |
| Total storage capacity | 500 GB |
| Storage media | HDD |
| Number of HDDs installed | 1 |
| HDD capacity | 500 GB |
| HDD interface | SATA II |
| HDD speed | 5400 RPM |
| Optical drive type | DVD Super Multi DL |
| Card reader integrated | |
| Compatible memory cards | SD,SDHC,SDXC |
| Graphics | |
| Discrete graphics adapter model | NVIDIA® GeForce® GT 620M |
| On-board graphics adapter | |
| Discrete graphics adapter | |
| On-board graphics adapter family | Intel® HD Graphics |
| On-board graphics adapter model | Intel® HD Graphics 3000 |
| On-board graphics adapter base frequency | 650 MHz |
| On-board graphics adapter dynamic frequency (max) | 1100 MHz |
| On-board graphics adapter ID | 0x116 |
| Discrete graphics memory type | GDDR3 |
| Network | |
| Wi-Fi | |
| Bluetooth | |
| Wi-Fi standards | 802.11b,802.11g,Wi-Fi 4 (802.11n) |
| Ethernet LAN | |
| Ethernet LAN data rates | 10,100,1000 Mbit/s |
| Bluetooth version | 4.0 |
| Data network | Not supported |
| Cabling technology | 10/100/1000Base-T(X) |
| 4G WiMAX | |
| Networking standards | IEEE 802.11b,IEEE 802.11g,IEEE 802.11n |
The key requirements of my home media server are as follows:
-
That it have a DLNA and web-based streaming service for hosting downloaded content on the LAN
-
That it have web based interface to indexer software that searches bittorrent indexes online for content, then download it automatically,
-
That it have a bittorrent client running that has a web based frontend for the benefit of the indexers,
-
That it have a web-based tool to configure and monitor all of the above,
-
That it have a basic SSH setup,
-
That it use hardware transcoding of audio and video using the onboard NVIDIA graphics card,
-
That it automatically mounts any USB storage devices attached to it (for manually copying content), and
-
That it automatically rip any DVD placed in the DVD drive and allow the content to be streamed.(This is now moot, as I swapped out the dvd drive for an extra HDD using a laptop hard drive caddy.
To these ends, I stripped down my Ubuntu installation from standard to barebones, so that it can run headless without starting X. Then reinstall the nvidia graphics drivers with X so that they're running at least, then setup ssh and remote-login, then setup all the abovementioned software. Note that the servers are not natted to WAN for security reasons.
This is the order in which the tasks should be done.
-
Headless Ubuntu: https://help.ubuntu.com/community/Installation/MinimalCD
-
Transmission-daemon : https://transmissionbt.com/
-
Jellyfin media server : https://jellyfin.org
-
Jackett : https://github.com/Jackett/Jackett
-
Radarr : https://radarr.video
-
Sonarr : https://sonarr.tv
-
Organizr : https://organizr.app/
-
eZServerMonitor : https://www.ezservermonitor.com/ (embedded as an iframe in organizr)
-
automount-usb : https://github.com/raamsri/automount-usb
First, I made my existing ubuntu install headless so that it runs in console mode without graphics. Strictly speaking, this is not necessary, but I believe that not running unnecessary graphics will free up some cpu and RAM for streaming, and have extra HDD space for media.
To make an existing ubuntu headless, follow the instructions in the link below:
Turn Ubuntu desktop into an headless server
For a fresh install, I suspect that the ubuntu MinimalCD is the best option. Simply boot off it and install only basic packages and internet-wifi connection stuff, no graphics or X11/wayland needed at this juncture.
Then, maintain the internet connection and auto-install the nVidia graphics drivers from the command line as per instructions given below
2 Ways to Install nVidia Driver on Ubuntu 18.04 (GUI & Command Line)
Note that the kernel headers corresponding to the RUNNING KERNEL must be installed. This can be done with the command
sudo apt install linux-headers-$(uname -a|awk '{print $3}')Install and setup the SSH server, and setup the '/etc/issue' file and ubuntu custom message for logins as per the introduction section:
See this HOWTO:
How to Enable SSH on Ubuntu 18.04
There are several media streaming servers for Linux, the most popular ones (as of 2020), being Plex and Emby. I tried both of them. However, Plex was beset with networking difficulties, and disallowed hardware transcoding unless you paid for their premium service. Furthermore, Emby is crippleware; you have to pay premium for using any client on a smart TV. Thus, I settled for the open source fork of Emby, called Jellyfin media server.
Install Jellyfin as per instructions here:
Jellyfin Quick Start: Installing on Ubuntu
If you have an nVidia graphics card that supports hardware transcoding (research your model, mine does not), then enable hardware transcoding with nVidia NVENC/NVDEC and the version of FFMPEG bundled with Jellyfin. Instructions are below.
Jellyfin Quick Start: Hardware Acceleration
Turns out that my M-series nVidia GPU does not support hardware transcoding. I learnt this the hard way by trying the config in the previous subsection and failing. To check whether your hardware transcoding is working, check the logs in "/var/log/jellyfin". There are transcode logs named "ffmpeg-transcode-XXXXXX" where "XXXXX" is some sort of hash. Go ahead and grep through them or just open them up in a text editor or whatever. See this section in the HOWTO cited in the previous subsection of this doc for details.
Nonetheless, hardware transcoding is possible in standard Intel graphics cards using the open source VAAPI libraries. For instructions on how to get Jellyfin to do it, see the HOWTO below:
Jellyfin Quick Start: Hardware Acceleration using VAAPI
Usually, once the intel graphics drivers are up and running, Jellyfish can automatically choose the codecs that are supported by the GPU once VAAPI is enabled in the "Transcoding" configuration page (see the subsection below on where to find it). In any case, if you need to know which codecs are supported for your card, then go through the HOWTO cited below:
archlinux wiki: Hardware Video Acceleration
If you have multiple graphics cards with one of them being an nVidia optimus-compatible gpu, you may install the nVidia graphics drivers the standard way and use PRIME profiles to switch them around. See the HOWTO in the link below for a simple introduction:
LinuxBabe: How To Switch Between Intel and Nvidia Graphics Card on Ubuntu
Finally, I opted for RAM transcoding, where transcoded files are written to RAM instead of disk for faster access. For background, checkout the youtube video below (this one is for Plex, but the principle is the same for all streaming servers)
To create a ramdisk and mount it on ubuntu, follow this HOWTO:
How to Create and Use a Ramdisk on Ubuntu 18.04
I chose to mount it on the path '/var/lib/jellyfin/transcodes', the default location where jellyfin stores transcoded data. If you need to use any other path, then the settings can be obtained by navigating in the Jellyfish Home page as follows: Navigation menu on the left + "Admin : Dashboard --> Playback --> Transcoding (top of the page)"
As can be seen in the screenshot above, the transcoder setting can be used to enable VAAPI transcoding as described in the Jellyfin "Quick Start" page (linked above) on Hardware Acceleration.
-
Bittorrent client: I've chosen transmission, but there are other clients with remote interfaces (see Extras). Installing transmission without a GUI (headless) and running the web interface can be done as per the instructions below:
How to set up transmission-daemon on a Raspberry Pi and control it via web interface. Note that it doesn't have to be done in a raspberry pi. The instructions will work on any computer running Debian or Ubuntu Linux.
-
Bittorrent indexers: For movies, I chose to install Radarr and for TV shows, I chose Sonarr. For details on setting them up. check out their GitHub pages:
a. Radarr: GitHub Page
b. Sonarr: GitHub Page
Note that you will have to setup supported online indexers for bittorrent. There are only a few for Radarr and Sonarr, and they don't always work well. Therefore, I also installed the Jackett proxy server which connects to thousands of online indexers and configured radarr and sonarr to use Jackett instead. For instructions, checkout the GitHub page for jackett. Also, see this section for details on how to use flaresolverr with jackett to bypass bot checks.
For further details, checkout these HOWTOs:
Indexers enabled in Jackett:
Here, I wanted to setup the media server in such as way that inserting a USB thumb drive or external hard drive into the machine would automatically mount it without manual intervention. This way, I can remotely log in and just copy the files from the drives to wherever. The simplest way is to use Ubuntu's systemd to launch the automount service at boot, the service being automount-usb. Simply checkout the github repository and follow the instructions therein
$ git checkout https://github.com/raamsri/automount-usb
$ cd automount-usb
$ sudo ./CONFIGURE.shPhew! That was a lot of installations and configs! Now, the media server has a lot of web services running in all sorts of ports, with more possibly in the future. How does one keep track of it all? Bookmarks in the browser? UGH! What about checking up on server status? Do we just keep ssh-ing periodically? How crude!
An excellent software for keeping all your media services together for easy viewing and configuration is called "Organizr". It sets up a web login using PHP on an standard web server where you can keep all your media service links in one place and see a status dashboard to boot! Head on over to their documentation and follow the installation instructions. Alternatively, see the following article (adjust for your operating system)
The Ultimate Organizr V2 Setup Guide for Windows
The above-mentioned instructions can be readily adapted to Ubuntu. I would recommend installing the latest development version, as it has more features than the stable version.
Finally, an interesting trick can be to run a simple PHP-web based system monitor and enbed the output on your organizr home page. I chose eZServerMonitor due to its simplicity and low footprint. Install it in a subdirectory of your organizr install, then open Organizr page in your browser, go to 'settings -> Tab Editor -> Homepage Items -> CUSTOMHTML-1' and input the following html markup tag
```html
<iframe src="http://192.168.1.2/apps/ezservermonitor/index.php" name="System Monitor" height="1400px" width="100%" title="System Monitor"></iframe>
```
Once this is saved and enabled, the Organizr homepage should show a nice big embedded frame with all your machine status information in it.
This is what my Organizr Homepage looks like
Finally, since all the different softwares installed above often want to communicate content data to each other in disk, I thought it best to let them. To that end, each software creates a user and group with the same name as the software (this is done automatically). Set the working directories of each of the following software to permissions where the user and group gets to read, write and execute, and assign the groups to each other. Consult standard linux documentation on how to do this. The following users/groups are important in this regard:
-
Group: admin, Users: admin, jellyfin
-
Group: debian-transmission, Users: sonarr,radarr,admin,jellyfin
-
Group: radarr, Users: debian-transmission,sonarr,admin,jellyfin
-
Group: sonarr, Users: debian-transmission,radarr,admin,jellyfin
-
Group: jackett, Users: admin
-
Group: jellyfin, Users: admin,debian-transmission,radarr,sonarr,jackett
Finally, of course, we'll need clients to connect and stream from the Jellyfin server. The clients page of the Jellyfin website has a lot of information on this:
You can also use any web browser and connect via organizr. In any case, clients exist for several operating systems and smart media devices ranging from Android TVs to Amazon Fire-TV. If none of them work, then enable the DLNA server in Jellyfish in the Media Server and use any DLNA-compatible video player in your device to detect and connect to it. Many such players exist for all sorts of operating systems. A good Open Source one is VLC, also supported on Android and Amazon Fire TV.
CLICK HERE
admin@MediaServer:~$ cat /etc/issue
.-.
.-'``(|||)
,`\\ \\ `-`. 88 88
/ \\ '``-. ` 88 88
.-. , `___: 88 88 88,888, 88 88 ,88888, 88888 88 88
(:::) : ___ 88 88 88 88 88 88 88 88 88 88 88
`-` ` , : 88 88 88 88 88 88 88 88 88 88 88
\\ / ,..-` , 88 88 88 88 88 88 88 88 88 88 88
`./ / .-.` '88888' '88888' '88888' 88 88 '8888 '88888'
`-..-( )
`-`
Welcome to \n \l
* Services : Point any Web-Browser to http://\4{wlp2s0} for details
* USB Drives : USB drives are automatically detected and mountedCLICK HERE
admin@MediaServer:~$ cat /etc/update-motd.d/01-mediaserver-info
#!/bin/bash
#
# 01-mediaserver-info - print some info about the mediaserver
#
# Authors: Analabha Roy <[email protected]>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
IP_ADDRESS=`/sbin/ifconfig | grep 'inet '| grep -v '127.0.0.1'|awk '{print $2}'`
printf "\n"
printf " * Services : Point any Web-Browser to http://"$IP_ADDRESS" for details\n"
printf " * USB Drives : USB drives are automatically detected and mounted\n"It goes without saying that this machine is running 24/7. Laptops are generally not designed for continuous running and the cpu can run fairly hot. It is important to manage this as well as possible. The best way is to install fan control software that adjusts the speed of the cpu fan to maximize cooling. There are numerous software for this on linux (google for them). The standard lm-sensors tool does not detect my cpu fan no matter what. Thus, I have chosen to try "samsung-tools", a general toolkit designed for samsung laptops using the .
Samsung-tools for linux can set the fan speed to 'overclock' mode, where the fan presumably cools more aggressively. However, the setting seems to be non-persistent, meaning it disappears after a reboot. Therefore, I've setup a user-level cron job to set it to overclock mode at bootup. This should, in principle, maximize cooling. The software can be installed on ubuntu from the PPA given below:
Simply run
$ sudo add-apt-repository ppa:voria/ppa && apt install samsung-toolsNote that samsung-tools needs D-Bus to work, so cron can't directly run it unless it has logged in as a user. An ugly hack for this is a cron job like
@reboot sleep 20 && ssh localhost samsung-tools -c overclock where ssh has been setup for login by key pairs locally. See this howto for that.
Also, I thought it a good idea to downclock the CPU in order to minimize overheating. Doesn't seem to effect transcoding or streaming. Do this with cpufrequtils as follows
$ sudo apt-get install cpufrequtils
$ echo 'GOVERNOR="powersave"' | sudo tee /etc/default/cpufrequtils
$ sudo systemctl disable ondemandAs of June 2020, a lot of torrent indexer websites are protected by services like cloudfare and others. These online services prevent bots (like Jackett, which I had recommended for automatically searching for torrents while installing indexers) from accessing these sites by putting in automatic Turing tests and/or CAPTCHAS designed for manual intervention. As it happens, the fella developing Jackett has created a workaround called flaresolverr. This is a software that runs as a service, and Jackett can connect to it, have it launch a headless chromium browser session and try to fool the bot blockers into thinking that it is a human. It's not follproof, but increasingly necessary as many torrent sites are getting bot blockers to prevent malicious hackers from crashing their sites. The github page of flaresolverr instructs how to set it up, but does not have a way to startup on boot. On Ubuntu, the following systemd script should do it. Assuming that flaresolverr has been unpacked to "/usr/local/flaresolverr", simply create the file "/etc/systemd/system/flaresolverr.service" and add:
[Unit]
Description=Systemd service script for flaresolverr
Wants=network.target
After=syslog.target network-online.target
[Service]
Type=simple
ExecStart=/usr/local/flaresolverr/flaresolverr
Restart=on-failure
RestartSec=10
KillMode=process
[Install]
WantedBy=multi-user.target
Then, run the following:
$ sudo chmod 640 /etc/systemd/system/flaresolverr.service
$ sudo systemctl daemon-reload
$ sudo systemctl enable flaresolverr
$ sudo systemctl start flaresolverrThat's it. In case any other configuration is required, then the systemd script can be modified accordingly.
If you have a DVD drive in your system, you can setup the system to automatically mount and rip any inserted DVD discs and copy the content files over to a separate folder in the Jellyfin media user space. To do that, I setup Automatic Ripping Machine (ARM). Just check out the relevant GitHub page and follow the instructions therein.
Trakt is an online platform that primarily keeps track of TV shows and movies you watch. It integrates with your media center or home theater PC to enable scrobbling, so everything is automatic. You can create a trakt account at (https://trakt.tv/), create empty lists of movies and have radarr and sonarr automatically download items from them once they get populated.
-
While trakt lists can be added directly to radarr from the "List" option in radarr's settings, this cannot (as of August 10, 2020) be done with Sonarr. Until that happens, probably best to install and run traktarr as a service that periodically scans yout trakt lists for content to add to radarr and sonarr. Check out the GitHub page linked above for setup instructions. Note that the configuration file is a json file, and needs extensive manual editing. For some background on the structure of json files (they're similar to python dictionaries), see this introduction
-
You can also scrobble your movies and shows from Jellyfin to trakt with the Trakt plugin for Jellyfin.
The web interface for the transmission bittorrent client is a bit basic. You can enhance it with a better web interface, or use a client from another machine to connect to it. Details are available in the web site of transmission itself. I've tried transmission web control.
Alternatively, you can try more sophisticated bittorrent clients with more detailed web interfaces. Examples are deluge and qbittorrent. However, transmission, coded mainly in C/C++, has the lowest resource usage of all bittorrent clients. Deluge, coded in python, consumes much more memory. Qbittorrent is written in C++, and is, by accounts, pretty lean, although I have not tried it. See this link for headless config for qbittorrent.
In addition to bittorrent, I'm experimenting with other downloaders p2p networks. The Radarr and Sonarr PVR applications only support bittorrent and usenet, as far as I know. In addition, I have tried the following independently
-
Amule: A p2p application mainly accessing the EDonkey and Kad networks:
-
Set it up as a daemon and run it as user "amule" with home directory at "/var/lib/amuled". See this howto for details. Again, it's not exclusively meant for a Raspberry Pi. See this wiki for getting started with amule.
-
You may also need the non-web remote gui for initial access and config.
-
Note that radarr, sonarr, lidarr etc do not integrate with edonkey or any other p2p network besides usenet and bittorrent. As a workaround, set the incoming folder of amule downloads to wherever (I have it set to /var/lib/amuled/.aMule/Incoming), then install the "Auto Organnize Plugin" to Jellyfin and have it monitor the amule download folder for content, which it then integrates into Jellyfin's Libraries. This way, I can manually set downloads and they, at least, get integrated into Jellyfin properly.
-
-
Youtube Downloader with a nice php-web interface. If you've got nginx setup already then simply clone the repository into your website.
The configurations above basically install a bunch of servers that have to be accessed directly using the url and port numbers. This can be a problem since firewalls will have to allow traffic into those ports, and WAN access is unsafe without encrryption. If you want WAN access to your server, then it is better to set up all services via reverse proxies in nginx. See this document for a decent guide on reverse proxying all these services.
As it turns out, TurnKey Linux has a canned media server installation CD that does a lot of what I've described above and more! Might want to check that out:
Turnkey Linux MediaServer: Simple Network Attached Media Storage It just means that it can't verify the SSl certificate you generated before. It'll still work. Just proceed to the server even if it is labelled unsafe.
-
HTPC Download Box - Same basic setup as this one, but with dockers
-
Look into WAN access via reverse proxy.
- Have not configged SSL yet. WAN forwarding is currently still disabled.
- Problem is authentication. Radarr, Sonarr, Lidarr, jackett and Transmission all have independent authentication (all are presently disabled, so exposing them to internet is risky) and needs to be maintained separately. Only Jellyfin and Organizr can have synchronized authentication.
- All Authentication can be done via LDAP + nginx reverse proxy. Organizr and jellyfin can use LDAP directly, all else can use secure LDAP via nginx. Can also add shellinabox.
-
Organizr acts like ddos when nginx is configged with fail2ban with ddos protection. Ip gets banned unless whitelisted.