Server setup and installation

[gusaus]

Resetting some of the discussion pertaining to installing LibreTime on a server like AWS. Assuming we'll want to have a guide similar to the upstream branch, what steps are necessary to get there?

[hairmare]

Thanks for moving this from the taiga to GitHub.

That's a nice guide. It contains some alsa stuff and I can't quite see how that would work on AWS (or any other hosted virtualization platform not tailored for such use).

My legacy upstream fork has some info on how to install on CentOS parts of which may be generally useful. Some of the more opinionated parts of the install script should also be moved to documentation rather than just blindly installing (ie. it doesn't make sense to tell a user to change the rabbitmq password on the Web-UI when it's too late; if we automate this we should prompt for one or generate one and use that).

[gusaus]

Sounds like there are some tasks for the community to work through. Per our previous discussion, possibly we can provide an AWS instance to test install and provide a demo site.

[hairmare]

To have everything in one space, this is my feedback from taiga:

I'm not sure if LibreTime will be lightweight enough for a demo to fit
onto the AWS free tier.

When we have the install fixed and things properly packaged for debian.
Setting up LibreTime should be relatively straightforward. A demo would
still be nice, though I'm afraid of the logistics involved in one. Once the UI
is rebranded we should at least put up screenshots.

One idea I have been thinking about is to additionally offer libretime as a
set of docker containers, a docker-compose file and maybe a kubernetes
kubelet. That could be built in a way that it's easy to do a quick demo locally.

My point being that we should make it very easy to get up and running for a demo locally.

I believe having a public demo instance would be too much of a burden at this moment.

[hairmare]

Amazon Linux seems to be a flavor of CentOS, the RPMs I plan on building should work there!

[gusaus]

@hairmare That sounds promising! Let me know when there's something to test or if there's something you need help with.

[ergonlogic]

It'd be nice to have some installation instructions documented somewhere obvious (like INSTALL.txt). Don't get me wrong, RPMs, EC2 AMIs and so forth are great. But basic manual installation instructions would be invaluable. What exists now, in the form of a long shell script, was last updated two years ago. The upstream docs refer to Ubuntu and Debian distros that either already are, or soon will be EOL.

[hairmare]

Hello @ergonlogic

Welcome to the LibreTime community. We are glad to have you on board. Help us keep LibreTime open and inclusive by following our Code of Conduct (CoC).

It'd be nice to have some installation instructions documented somewhere obvious (like INSTALL.txt).

Absolutely, what I've got so far are some CentOS docs on RaBe's opinionated fork. I plan on porting those together with the packaging setup.

From what I can tell there is also some cleanup needed for most of the distros you mention (i.e. we need silan to be well supported by debian looking forward, see debian#855319)

I'm thinking of creating a nice clean collection of the relevant docs (like INSTALL.md) in docs/ and later upgrading them to be a github pages site.

Feel free to start writing the great INSTALL.md we are lacking, LibreTime is happy to have you contributing. You can also use the wiki if that suits your needs.

Thanks again for contributing,
Lucas

P.S: I sent you a team invite so you can assign and tag issues.

[gusaus]

It sounds like there's a need for documentation? Possibly there should be a list of the types of contributions/contributors you need to develop and sustain the project?

Sorry if that's a bit off topic.

[hairmare]

CONTRIBUTING.md already has some pointers. Cleaning up and adding to the documentation are extremely worthy contributions! The C4 also helps define "Contributing" and "Maintaining".

There is not only the technical docs like INSTALL.md, we should also find a way to save the 2.5 and pro manuals. Maybe transcribing them to docs/ where we can render them as github pages is the way to go.

Documentation wrangler certainly makes your list. Does it make sense to keep such a list in the wiki?

[gusaus]

Documentation lead, development help, testing/qa.. any other key roles that are needed to fill?

[hairmare]

packaging (as in rpm and deb packages), maybe ux? all things audio/liquidsoap... possibly more. I'm not sure we need a very strict list of positions, defining responsibilities is certainly important.

A bit offtopic, but since this is about getting folks to figure out how they can help: The last few issues (#17 & #18) I created were in a fashion so they should be doable for a dev that wants to get wet feet. I'm thinking of introducing a tag for those so new contributors can find some easy low hanging fruit if they are looking.

[ergonlogic]

I filed a couple PRs to address some of the suggestions in #5 (comment).

Having now read through the installer script, I'm of the opinion that we should re-think its all-in-one nature. I think a good first step would be to split it up into separate stages. Each could still source the custom functions, etc. and calling ./install should still just run them all. However, this would allow for easier testing of failing steps as well as encourage re-use of the individual scripts in a .deb or .rpm, Ansible roles, etc.

[hairmare]

👍 For the .rpm build I need something akin to make install where I can override $PREFIX and $ETCDIR to have rpmbuild do what is needed.

I did some testing with a Makefile but realized, that all of airtime_mvc/ and vendor/ are a bit much for that to handle.

I'm not sure about others, but with .rpm I can have yum take care of upgrading and replacing or not of updated config files and running post and pre install/upgrade scripts.

I kind of like how some modern apps from redhat come with their own installer cli which you install as a package and then run separately (ie. spacewalk has spacewalk-install, freeipa has ipa-server-install and ipa-client-install). Those scripts usually also help with upgrading and to some degree management of such apps. I find the entry point of installing an installer package rather nice.

Maybe the ./install script can evolve in that direction. I'm also not quite sure if it needs to be a shell script, some python would suit the job just as well and be testable.

[gusaus]

I tried a LibreTime Vagrant install which for the most part was a success. During setup last night, I wasn't able to successfully run these commands in the terminal...

Upon doing a 'vagrant up ubuntu' just now, everything seems to be working.

As I'm a Vagrant newb, I'm not sure why it's working now... very glad I'm able to test! Great work getting 3.0.0-alpha out the door!

[hairmare]

It could have failed for numerous reasons. Did you get any output when running the commands? Maybe something went to the logs in /var/log/airtime/? Did all of them fail or just a specific one?

I've seen issues with adding a soundcard to the VM that tend to clear up after running vagrant up but I've never seen them lead to a fail during the final setup steps 😕

[gusaus]

I got a ' command not found' for example -

MacBookGus:libretime gusaus$ sudo service airtime-liquidsoap start
sudo: service: command not found

Might just be me not entering the command in the right place.

[hairmare]

You need to connect to the box with vagrant ssh ubuntu and then run the commands on the virtual box. Sorry if that wasn't clear from the docs.

After restarting the box they automatically started as is intended. They don't get started by the installer since setup needs to be completed for them to not fail.

[gusaus]

Probably would make sense to most people familiar with Vagrant. There's going to be a lot more testers and users once we're able to easily install on AWS!

[gusaus]

With regards to installing on AWS, is what's being discussed by @ergonlogic and @hairmare in the comments above part of the proposed solution? Is there anything other contributors can help with?

[Robbt]

I think that it should be fairly easy to install on a AWS virtual machine, although if you wanted to use S3 as the file storage that needs to be tested still.

[hairmare]

Absolutely, Ubuntu and Debian are both available on EC2 as an AMI that should most likely work out of the box. As Robb said S3 needs testing. We might need to bump the aws sdk version we are using, but I've used that in the past and it was never an issue.

[paddatrapper]

I will be setting LibreTime up on Debian 9 in the next week or so and will document it as I go. I'm also planning to package it for Debian, though that depends heavily on the amount of work that takes vs the amount of time I have currently

[hairmare]

@paddatrapper The debian folder originally used by legacy upstream is at https://github.com/Airtime/airtime-packaging. Maybe that can help you as a starting point.

There is also #132 where I tried to document what I think is needed.

I don't see any reason why php 7 (the default in debian 9) should not work, but it's mostly untested. Things like python-rgain, liquidsoap and silan might be a piece of work since their packages haven't been looked at for a while (ie. I did an NMU for silan at debian#855319).

[paddatrapper]

@paddatrapper The debian folder originally used by legacy upstream is at https://github.com/Airtime/airtime-packaging. Maybe that can help you as a starting point.

That will definitely help a lot. I'll only get to packaging it once I've rolled it out on the server I am currently waiting for though, so will be in a week or so. As Debian 9 Freeze is in progress, we will not be able to get LibreTime into stable until the next release, unless we use backports, but that all depends on how many other dependencies, etc we have to package.

I don't see any reason why php 7 (the default in debian 9) should not work, but it's mostly untested.

PHP 7 is fairly pliant with PHP5 code, at least in my experience, so best-case will just need testing

Things like python-rgain, liquidsoap and silan might be a piece of work since their packages haven't been looked at for a while (ie. I did an NMU for silan at debian#855319).

At least they are all still in the repos, updating their packaging will definitely be needed though

[hairmare]

I think we are getting closer to having something that works on servers like AWS. The fact that #146 has it working on an OVH VPS is promising. This is most likely replicable on AWS using the Ubuntu Server 14.04 LTS AMI, other distros are available.

Assuming we'll want to have a guide similar to the upstream branch, what steps are necessary to get there?

Now that some basics have been taken care of I'll try to list what I would consider next steps to get the ball rolling here (I'm also trying to list what is already done)

• get docs up and running Add basic Mkdocs config and initial docs. #19
• merge legacy docs Merge Airtime manuals from Booktime.pro #36
• packaging for Debian Debian Packages #132
• packaging for Ubuntu (should be easy after we have Debian working)
• packaging for CentOS (more or less ported, undergoing integration testing, unsupported (bug reports welcome) packages here)
• update generic installer docs

And the rather amazon specific things

• test, fix, and document the S3 integration (maybe upgrade it for native s3 in liquidsoap 1.3.0)
• document an automated install using cloud-init (should work for most public clouds, I'm not sure of any specifics since I haven't used any lately)

In some cases the packaging also contains getting upstream packages bumped and maintained if we need them to be.

The reason I'm putting so much focus on packaging is because I feel that it will make managing the systems so much easier. We need to have a easy way for folks to update their installs if we want to have happy users, thus having distro packages is kinda needed across the board. Most of the documentation already references a package based setup and they show a user experience that make lots of sense.

[gusaus]

Just to update this issue with elements from this related discourse thread - https://discourse.libretime.org/t/libretime-on-aws/81

This was @hairmare's response to me asking if #5 (comment) was still valid -

AFAIK the todo list is still valid. The generic points on it aren’t real blockers though. You can already install on a AWS VM but you won’t get an automatically up-gradable system until we get to providing distro packages.

The amazon specific things (cloud-init and S3) haven’t been touched at all. I have a feeling that the S3 support might be the one you really need, using local storage on a cloudy VM only makes marginal sense.

And @Robbt's response with regards to what it would take to run a LibreTime equivalent of https://www.airtime.pro/

Aside from the billing and quota management etc, it is possible to run LibreTime on AWS. I think if anyone did want to create a SaaS installable version the only thing they would need to keep in mind is the requirement of the AGPLv3 that they share all of their source code changes etc.

Finally I had some thoughts on why a hosted solution might be beneficial -

I think a hosted version would be attractive to small stations, podcasters, or individuals who don’t have the skills or resources to set up and maintain their own instance. It should also provide an ongoing funding source for the OSS project and community.

I'm curious if there are others who see the potential value of such a setup and/or would be interested in helping work through the outstanding todos #5 (comment) if there were incentives?

[paddatrapper]

I'm curious if there are others who see the potential value of such a setup and/or would be interested in helping work through the outstanding todos #5 (comment) if there were incentives?

I'd be keen to help with a hosted version of LT. I am busy with the packaging for Debian and Ubuntu, but help is always appreciated, as I have very limited time available at the moment.

[gusaus]

Thanks @paddatrapper

I am busy with the packaging for Debian and Ubuntu, but help is always appreciated

So those look like the next two items on the checklist #5 (comment)? Are there specific tasks/issues where we could direct contributors?

Might as well add @hairmare's comments regarding LibreTime docker images https://discourse.libretime.org/t/libretime-on-aws/81/7

On a side note, I’ve seen slow but steady progress with my pet project LibreTime docker images and helm charts for Kubernetes. They are still far from alpha at this point but I think I’ve got all the bits and pieces together for integrating into the ecosystem. As far as I can tell my work should be deployable on most public clouds (ie. Amazon Elastic Container Service, Azure AKS, Google GKE and others).

The thing is my personal containerization effort is really just a pet project at this point and it won’t get production ready any time soon. There is lots of grunt work to be done in making LibreTime run HA enough for all of this to make sense at scale.

Would any of that work help make LibreTime on AWS possible? If so, how/where can direct contributors to that?

[paddatrapper]

So those look like the next two items on the checklist #5 (comment)? Are there specific tasks/issues where we could direct contributors?

• Packages Required libretime-debian-packaging#1 is tracking the dependencies that require packaging
• Scripts Requiring Manpages libretime-debian-packaging#2 is tracking the manpages that need writing
• Library Changes: contextmenu Javascript #416, Library Changes: FullCalendar Javascript #417, Library Changes: ServerBrowse Javascript #418, Library Changes: tipsy Javascript #419 track modified libraries that need sorting out

[stale]

This issue has been automatically marked as stale because it has not had activity in the last 5 months. It will be closed if no activity occurs in the next month.
Please chat to us on discourse or ask for help on our chat if you have any questions or need further support with getting this issue resolved.
You may also label an issue as pinned if you would like to make sure that it does not get closed by this bot.

[stale]

This issue has been autmatically closed after is was marked as stale and did not receive any further inputs.
Feel free to let us know on discourse or ask for help on our chat if you feel this issue should not have been closed.
Thank you for your contributions.