fancy thoughts

In this post I want to point you to way technique to add some missing features to the Chicken Scheme REPL.

If you aren't too familiar with Chicken Scheme you start the REPL like this.

csi

This actually runs the chicken scheme interpreter but on user input instead of a .scm file. One thing you'll notice is that there is no command history. Essentially, what we need is readline. For a while I suffered through this until I was looking through different eggs and linenoise. This means we don't actually need readline.

Linenoise is cool because it provides a minimalist alternative to the readline library. You can add this functionality to the REPL by going through these steps.

You'll first need to install the linenoise egg.

chicken-install linenoise

Then we need to add this to our .csirc located in our $HOME directory. You'll need to create it if it doesn't exist already. Then add these lines to the file:

(import linenoise)
(current-input-port (make-linenoise-port))

Now the next time you run csi you'll be able to navigate the REPL lines and history with the arrow keys. Voila! A much tastier version of the chicken interpreter!

Happy scheming :p

Lately I've been thinking about how I only know one language, and how that limits me in terms of just being a worldly figure. A lot of people in my neighborhood speak Spanish, being better at would be useful. Also there seems to be a lot of tech action happening in Latin America. I would love to eventually be a part of that. So what I've ran into is the need for a Spanish to English dictionary. Now there are a lot of great web options. However, I wanted something offline and free. So that led me to FreeDict. Luckily, all the necessary packages are in Debian. However, I found getting started to be a little tricky. So this blog post is meant to document the basic usage.

First you want to use aptitude to search for relevant dictionaries (these will later be called databases).

aptitude search dict-freedict

This will print out a long list so I found this to be a good time to whip out grep to quickly find the Spanish dictionaries.

aptitude search dict-freedict | grep Spanish

In this case we want Spanish to English, and English to Spanish.

sudo apt install dict-freedict-spa-eng dict-freedict-eng-spa

Now in order to use these packages we have to install a freedict client and server. We do this by installing dict and dictd.

sudo apt install dict dictd

You can list all of the available dictionaries.

dict -D

When we are searching for a definition we can specify which dictionary we want to use. So if we wanted to search for the English definition of a Spanish word we can do it like so:

dict -d fd-spa-eng gato

Which outputs:

1 definition found

From Spanish-English FreeDict Dictionary ver. 0.3 [fd-spa-eng]:

  gato /ɡˈato/
  1. cat
  2. jack

If you are having trouble searching for a definition you can optionally choose to try find a word that matches.

dict -d fd-spa-eng -m gato

Which outputs a list of recommended words:

fd-spa-eng:  gato  pato  gata

Now I've found that sometimes I have hard time find words. However, for very basic vocabulary it should provide some assistance. If you notice places it can be improved, make sure to look into how to contribute to the FreeDict project!

This post will go over some thoughts that I've been mulling over about choosing the right tools for my own personal linux desktop desires.

Lately, I've been diving deeper into userspace of the OS stack. Mostly involving making my own command line utilities and desktop apps. With the ultimate goal of making my own Desktop Environment. Surprisingly, this involves making a lot of choices about tooling, libraries, and even distros. After a bit of research, I started testing some of my previous notions to see if they still stood. The one change that I have made is in the preferred programming language which I will get to.

I will break this up into 3 sections: distro, app toolkit, and programming language.

Distro

The most significant change in the Desktop Environment space is the transition to Wayland from X11. If you are going to make a DE in this day and age, you have to target Wayland, or else you are wasting your time. X11 is fun for toys, but if you want something that will stand the time for at least the next couple of decades then Wayland is the way to go.

That being said, a pure Wayland desktop experience doesn't really exist except for sway, or maybe Gnome. In order to run most apps you still have to use xwayland because the apps themselves depend on X11. If you use app toolkits like Gtk or Qt these details can be abstracted away. Where this matters the most is with the window manager. The reason XFCE will never be on Wayland is because of xfcewm. The xfcewm is literally the most core component of XFCE with most of the settings config being related to it. Sway is basically just a window manager, and there is a library called wlroots that helps you build a Wayland compatible window manager. This is still early, but I expect people to make a variety of window managers with this.

To truly make your own desktop environment you have to make your own window manager. Anything else is just a utility that doesn't necessarily depend on the specific desktop environment. If you want to make your window manger, you should do it for Wayland. Which finally brings up the point about which distribution to develop on. This is because some distros support Wayland and wlroots better than others.

Debian v Alpine

Currently, my personal use distro is Debian stable. I like it because it is FOSS, has a great history, and it just works. However, the creator of sway uses Alpine, so I thought maybe there is some logic to that. Perhaps, the Alpine ecosystem would support Wayland better. With my experiences of trying to install sway, I can assure you this is not the case. The sway Debian package actually booted up in QEMU for me. The other more important thing is that wlroots already has a package in Debian, even if it is in experimental.

People who use Alpine are digital equivalent of mountain folk if you ask me. The one thing that is nice is the use of musl over glib. I hope to see that used in Debian one day. For now, I will continue to be directing my knowledge and effort towards Debian.

App Toolkit

When looking into making your own desktop app for Linux you are faced with a few options. You can go hardcore opengl graphics or you can use an app toolkit. The options are Gtk and Qt. My concerns with these libraries go beyond the tech. What kind of license do they have, what kind of organization runs it, what is the community like?

Gtk v Qt

I chose Gtk originally because I like the GNOME foundation as an idea, and I liked XFCE which used Gtk. However, I was curious if Qt was a better option. It seems that Qt is actually more complicated than Gtk. Gtk is also more focused on Linux as opposed to cross platform which is fine with me. People also complained that Qt apps can be a bit bloated. None of this was enough for me to want to make a change. I will be continuing to put my effort into the gtk toolkit.

Sidenote: Based on my readings it seems that the direction of the gtk library is very much based on the GNOME DE. People have complained about breaking API changes, and not feeling like they have a say in the library. Which sort of supports other evidence to suggest that GNOME just cares about GNOME. I'm not super unhappy about this, but I can see why it might upset people.

Programming Language

The next thing you have to decide when building a desktop app, is what language do you want to write it in? Gtk is written in C, so it is a natural fit. C is awesome, and it is great for systems, but it stops being fun after a couple of hours. I'm looking for something that I don't mind hacking with in my free time and actually following through with my ideas. Rust is currently entering the fray, and has a lot of enthusiasm from the GNOME community. My original intentions have been to use Rust to build out my desktop apps. That was the idea until I actually tried to build some gtk apps with Rust.

Crystal v Rust

The thing with Rust is it has a really strict compiler and it makes doing mutable state programming quite difficult. In order to just get a button to click in Rust requires the use of a macro to control strong and weak pointers. The fact that I have to grok this, just to get a button to click was a real turn off for me. The whole point of not using C is so it wasn't complicated, but Rust makes it complicated. Rust is complicated, there are 10 million ways to do the same thing. This lead to a real crisis for me. My whole plan was to support Rust gtk so more people might be willing to contribute. But if the easy stuff is hard, then this whole plan goes down the drain.

Then I found Crystal. On Hacker News I ran across an entire OS implementation in mostly Crystal (some C). This blew my mind. Then after reading about Crystal more, it blew my mind again. Crystal really feels like you can have your cake and eat it too. It has seamless interop with C, it is performant, and can be compiled into an easy to install binary. And the syntax looks like Ruby but with types. After making a few apps in Crystal I have become a big fan. There is even a usable gtk binding in Crystal already. I can make a simple desktop app and I don't have to worry about strong/weak references. I can just click a button and make a thing happen.

In principle, I think the Rust approach is better. Rust pushes you to write code the correct way. The relm project is a step in this direction. I built an app with it, however, I found myself doing state bashing and it doesn't support Glade yet...

So yeah, I've made my decision to build my apps in Crystal using Gtk and to make sure that it can be Wayland compliant.

TL;DR: Source Code

Crystal has been a recent addition to my programming languages repertoire. It reminds me of my love for Ruby, but also satisfies my current concerns for performance and deployment. Crystal makes it feel like you can have your cake and eat it too. Which basically means it hits a lot of sweet spots for me. That being said it is still newish or at least the ecosystem is in its nascent stage.

While using Crystal I found myself in need of something to run a bunch of small scripts related to my project. Mostly to save on typing long commands repeatedly. I have seen a lot of people use Makefiles, and for this project I myself used a Makefile. However, I thought, wouldn't it be nice to have something like npm-scripts or like boot-clj. Unfortunately, such features aren't in the shards tool yet.

Now there is a tool I found called cake. Which is described as being like Make but for Crystal. Also, not to be confused with rake from the Ruby world. Anyways, I gave this tool a try and I had two problems with it. The first is that I found myself wishing I was just scripting with bash, and startup performance was not good. My intent was to use this for quick iterations and convenience scripts. So I felt this wouldn't fly.

After thinking about what I wanted, I came up with my own idea. This idea involves attaching small bash scripts to the shard.yml file that comes with a Crystal project. One benefit of this approach is that, it is one less file cluttering your project directory. Another advantage is that yaml is quite flexible. The command name is the key in the yaml file, with the script being the value. Thus being almost exactly like npm-scripts.

Here is an example kemal application where I have added a few convenience commands for building the project on the commands field, scripts turns out to be already taken by shards.

name: todo                                                                                                                                               
version: 0.1.0                                                                                                                                                 
                                                                                                                                                               
authors:                                                                                                                                                       
  - fancycade <fancycade@protonmail.com>                                                                                                                       
                                                                                                                                                               
targets:                                                                                                                                                       
  cityscape:                                                                                                                                                   
    main: src/cityscape.cr                                                                                                                                     
                                                                                                                                                               
dependencies:                                                                                                                                                  
  kemal:                                                                                                                                                       
    github: kemalcr/kemal                                                                                                                                      
  spec-kemal:                                                                                                                                                  
    github: kemalcr/spec-kemal                                                                                                                                 
  pg:                                                                                                                                                          
    github: will/crystal-pg                                                                                                                                    
                                                                                                                                                               
crystal: 0.32.1                                                                                                                                                
                                                                                                                                                               
license: MIT                                                                                                                                                   
        
commands:                                                                                                                                                       
  run: 
    crystal run src/cityscape.cr                                                                                                                                                                                                                                                                                         
  release: 
    crystal build --release src/cityscape.cr                                                                                                                                                                                                                                                                    

Notice the run and release keys. We can use these commands by running nrg in the project directory.

nrg run

# or

nrg release

It simply looks for a shard.yml file in the project directory, looks for that key and then runs the sting value as a system command. I have also added a few quality of life features to make it more handy. The first being the ability to run multiline commands. We could do something like this by adding the pipe character after the key to denote a yaml multiline string:

commands:
  test: |
    ./sql/setup.sh
    KEMAL_ENV=test crystal spec
    ./sql/teardown.sh

All of this is abstracted into:

nrg test

Since nrg will run each of these lines in order.

Sometimes bash scripts require passing in a parameter. An example being the psql tool requiring a database and username when executing sql scripts. I've decided to make this possible with nrg by using ? as a special character. This is similar to passing args to a bash script. I chose ? since it has the least conflicts with bash scripting. For example my setup commands can require two parameters. I can make a command like this in my shard.yml:

commands:
  setup: ./sql/setup.sh ?1 ?2

Which is used like so:

nrg setup fancycade todo

Let's take this up a notch by refactoring our test command to use our setup and teardown commands. And this accepts the username and the database name:

commands:
  setup:
    ./sql/setup.sh ?1 ?2
  teardown:
    ./sql/teardown.sh ?1 ?2
  test: |
    nrg setup ?1 ?2
    KEMAL_ENV=test crystal spec
    nrg teardown ?1 ?2

Maybe you feel this is helpful, but it certainly feels like more natural way to interact with the project. The key-value nature of the yaml file provides a built in namespace that can be leveraged to build powerful abstractions. Which I find to be very Unix-esque and down right cool.

For me, this adventure is a fun thing about getting into a new language early. There is wide open space to explore and experiment with different tooling. I can imagine similar behavior going into the shards tool itself. For now I'm going to keep hacking on nrg to add whatever features I personally find useful. And that is the beauty of programming.

In this post, I'm going to show you how to install mpsyt on a Debian machine. I haven't been this happy about an app in a long time. I spend a lot of time listening to mixes on youtube, and have always been bothered by how much CPU and RAM Firefox requires to do so. With this tool I can easily search and listen to audio from youtube while using very little RAM and CPU cycles.

install

mpsyt has a few dependencies it uses to make things work. We start by installing our debian packages these are pip3 and mplayer. pip3 being the python3 package manager, and mplayer being a comandline audio playback tool.

sudo apt install python3-pip mplayer

Once we have pip3 installed we use it to install mps and youtube-dl. youtube-dl has gone through a lot of changes since the release of Buster. pip3 will have the most up to date package.

pip3 install mps-youtube youtube-dl

Then open up mpsyt in the terminal. You can enter a search by starting the query with '/'.

/com truise fairlight

You can choose which of the results you want by entering in a number into the prompt.

Then you have it, a youtube audio streamer in the terminal that doesn't use up all of your computer's resources.

Continuing my exploration of suckless tools I wanted to try compiling a program with tcc and musl. tcc stands for Tiny C Compiler, and musl is a lightweight libc alternative to glibc. I was generally sold on the principal of these tools, but I could not for the life of me find documentation on how to get these two working together.

Turns out it is rather quite trivial if you use the right OS. In my case, I was doing this on Void Linux running inside of a QEMU vm. The reason I chose Void, was because it has a musl build, so all of the tools are built with musl instead of glibc. musl-devel is not available on the glibc version.

On Void Linux, to install the necessary dependencies:

sudo xbps-install -S tcc musl-devel

What tripped me up is that on Void's packages directory page, I had not selected x86_64-musl so I couldn't find musl-devel. Once you get these two packages installed you are in business.

Write your hello world like this:

#include <stdio.h>

int main() {
  printf("Hello, World!\n");
  return 0;
}

Then with tcc we can treat this like a script and execute it:

tcc -run hello.c

That is it! Hopefully, this saves some time for other noobs like me.

Lately, I've had a new appreciation for minimalist C applications. There is a certain Zen quality about C that I really like. Knowledge of the possible “footguns” actually makes me more focused and insistent on writing good code. It is a bit like weight training but for programmers. Coding in other languages like Python feels trivial after spending a bit of time wrangling some C. less is one of the original Unix terminal applications for viewing files. It is a good example of the Unix philosophy: “Do one thing, and do it well”. It differentiates itself from editors by not loading the entire file into ram. The wikipedia page has a very detailed explanation. If you haven't used less before, I would recommend giving it a spin before continuing with the rest of this post.

In summary what we will be making is a small terminal application that uses a file stream to view text. If you want to follow along check out the final repo here on Sourcehut. We will be using termbox for rendering and general terminal application behavior. termbox is an alternative to the popular curses library and is recommend by suckless. It is very small and easy to use.

On Debian we can install it like so:

sudo apt install libtermbox-dev

Once we have that installed we will start with a simple hello world using termbox. For compilation I am using a very basic Makefile that calls out to my default c compiler. On my machine it is gcc. I'm using c99 because people smarter than me have come to a consensus that it is best. Also notice that in the build command I am linking the termbox library. I decided to name this application bless because it is “basically less” :p.

all: build

build:
	$(CC) -o bless bless.c -Wall -W -pedantic -std=c99 -ltermbox

clean:
	rm bless

Now the actual hello world looks like this. Most of this code is basic boiler plate to get termbox up and running. We initialize termbox, render our text, and then wait for the escape key to shutdown. Easy!

#include <termbox.h>
#include <stdio.h>

void render_line(char *text) {
	int i = 0;
	while (text[i] != '\0') {
		tb_change_cell(i, 0, text[i], TB_WHITE, TB_DEFAULT);
		i++;
	}
}

int main() {

	int ret = tb_init();
	if (ret) {
		fprintf(stderr, "tb_init failed with error code %d\n", ret);
		return 1;
	}

	tb_clear();
	tb_select_output_mode(TB_OUTPUT_NORMAL);

	render_line("Hello, World!\n\0");

	tb_present();

	struct tb_event ev;
	while (tb_poll_event(&ev)) {
		switch (ev.type) {
		case TB_EVENT_KEY:
			switch (ev.key) {
			case TB_KEY_ESC:
				goto done;
				break;
			}
			break;
		}
	}

done:
	tb_shutdown();

	return 0;
}

Now we are going to step our game up by rendering a file in the limits of our terminal screen. The filename is given as user input. So lets change our main function to accept a filename and attempt to open it.

int main(int argc, char **argv) {
	(void)argc; (void)argv;
	FILE *fp = fopen(argv[1], "r");

When we shutdown termbox we also want to close the file:

done:
	tb_shutdown();
	fclose(fp);

The next step is to add the capability of rendering the file. We do this by replacing renderline with renderfile, which will accept a file pointer instead a character string.

void render_file(FILE *fp) {
	int line_count = 0;
	int x = 0;
	while (line_count < tb_height()) {
		char c = fgetc(fp);
		if (c != EOF) {
			if (c == '\n' || x == tb_width()) {
				line_count++;
				x = 0;
			} else {
				if (c == '\t') {
					x+=8;
				} else {
					tb_change_cell(x, line_count, c, TB_WHITE, TB_DEFAULT);
					x++;
				}
			}
		} else {
			break;
		}
	}
}

Since all we need to do is render a portion of the file stream we can simply loop until we reach the terminal height (tb_height) or reach the end of the file. In order to do basic line wrapping we check if our x coordinate is equivalent to the terminal width (tb_width). Another thing to note is we have to manually handle tabs or else termbox will have rendering bugs. In this case we simply skip over 8 spaces.

Now when we execute bless we pass in the name of the file we want to view.

./bless *filename*

Great we can view the file in our terminal. However, you will quickly notice we can't view parts of the file that are greater than the terminal height. Lets add some navigation features to fix this.

We will start by adding scroll down by pressing the down arrow key. To do this, we will need to add an event handler for when we press the down arrow.

case TB_KEY_ARROW_DOWN:
	cursor = scroll_down(fp, cursor);
	render(fp, cursor);
	break;

You'll notice that we are calling two new functions when the down arrow is pressed. scroll_down will set the file stream at the beginning of the next line, and return an integer. That integer is a simple variable called cursor which tracks our position in the file. Once we change position we have to render the new view.

I'll start by breaking down render. It mostly includes termbox boiler plate and we call the render_file inside of it. The reason we need this is that when we scroll the characters need to be reset or else we will see artifacts from the last scene.

void render(FILE *fp, int cursor) {
	tb_clear();
	tb_select_output_mode(TB_OUTPUT_NORMAL);
	render_file(fp);
	tb_present();
	fseek(fp, cursor, SEEK_SET);
}

After rendering the file we set the file position to the cursor offset since it changes every time we call fgetc.

The scroll down function looks like this:

int scroll_down(FILE *fp, int cursor) {
	char c;
	while (c != '\n') {
		c = fgetc(fp);
		cursor++;
	}
	fseek(fp, cursor, SEEK_SET);
	return cursor;
}

scroll_down loops from the cursor position until it finds a newline, and sets the file position at the new cursor position. Which will be at the start of the next line.

How about scrolling up? Well that is a little more tricky since we fgetc only goes down the file stream, not up. Using a combination of fseek and fgetc we can implement scrolling up in a concise manner.

The implementation of scroll up looks like this:

int scroll_up(FILE *fp, int cursor) {
	char c;
	while (c != '\n' && cursor != 0) {
		cursor--;
		fseek(fp, cursor, SEEK_SET);
		c = fgetc(fp);
	}
	return cursor;
}

And voila! Basically less with scroll down and up features in less than 100 lines of code. After messing around with this you may notice a few bugs, but for simplicity sake I think this implementation is okay. The real version of less has a lot more features, so you probably won't want to use this for personal use. I hope this post has inspired to make your own termbox app in C. At the very least it was a good learning experience for myself.

Recently I have been interested in user friendly, self hosting solutions. After a bit of web surfing, I found this was a much larger space than I had anticipated. It seems that in the last decade a variety of projects have been popping up, and with increasing frequency. Some of the buzzwords I came across where “personal cloud”, and “PaaS” or “Platform as a Service”. It is always fun to poke fun at buzzwords, but I think these terms are relevant when attempting to portray these ideas to a larger audience. This is still very niche and weird. Quite literally the best resource for a list of self hosting solutions was this kind of creepy youtube video.

The basic idea is to be able to easily deploy web services on a common platform. I'm partial to Debian based solutions, so Yunohost and FreedomBox stood out to me. This post is dedicated to Yunohost.

Overall, I like Yunohost a lot. It is fun, and easy to use. I spent a whole day installing and uninstalling apps to try things out.

What is Yunohost?

The Yunohost FAQ states:

It provides a software that aim to make it easy for people to run and administrate their own server, with minimal knowledge and time required.
YunoHost may be called a distribution or an operating system, but it's actually "just" a simple layer added over the top of Debian, which does most of the hard work for you.

FYI: The name Yunohost is pronounced, “Why You No Host?”.

Installation

The insallation instructions I followed for Debian Stretch can be found here.

I chose to run my installation on a brand new Debian cloud image with only 2G of RAM. For initial demo purposes, I noticed no serious performance issues on this machine. The first thing that got stuck me was that I was not able to install on Buster. There documentation only says Stretch, so I should've known better. You can track the status of this issue here. Besides this hiccup, the installation process was mostly a breeze. The documentation for this is very detailed. The docs are a wiki, so if you notice an issue you can actually submit patches directly from the site. I submitted a few patches myself.

To install on a Debian Stretch machine, simply run:

curl https://install.yunohost.org | bash

If you are concerned with this approach, they have a disclaimer on the docs about this. Essentially, what you are downloading is a Debian metapackage that will install the necessary dependencies. It is quite opinionated on how it does the configuration, which is why it is recommended to run this on a dedicated machine.In my opinion, having the yunohost app be an official Debian package would alleviate a lot of these concerns. It would be more intuitive for Debian users, especially when they advertise as being Debian based. I would like to be able to install the yunohost app with apt.

Out of the Box

Upon installation, Yunohost comes with a Single Sign On service, email server, xmpp server, and the admin GUI used to deploy applications.

When originally accessing the web GUI your browser will complain about the certificate. Find a way to bypass this and then you can login. Once there, you can go to Domains and set up a Let's Encrypt certificate for your domain.

The basic setup allowed for me to connect Thunderbird to the email server and send messages. These messages were sent to the Spam folders of the accounts I tested it with. With more domain hacking I should be able to fix this.

The Single Sign On Service, called SSOwat, is one of my favorite features of Yunohost. I haven't seen anything similar advertised by other projects. The implementation is a NGINX plugin written in Lua. Fine grained application access is controlled by SSOwat. Only users who are registered with the Yunohost service will be able to access these sites if the application is not made public. Easy setup private network is a great feature.

Applications

I tried a handful of applications. The official apps actually work out of the box. Others work with varying degrees of success and they try to be very transparent about this.

One thing that is confusing, is it isn't always clear how the domains should be setup for each app. Some apps are okay with a url route like my.domain.org/riot, others require a subdomain like matrix.domain.org. The yunohost app does catch these mistakes if you mess up It would be nice if this situation was more explicit though.

Now I'll go through a list of apps that I tried.

Matrix/Riot

matrix-synapse and Riot are both easy to install with basic features working. The Matrix Synapse server that is installed does not come with a GUI (that is what Riot is for) you can access it with other Matrix clients like Fractal or Riot. I wanted to give the full experience a try, so I installed Riot along with it. This was quite easy to do. You can optionally put the Riot app behind a private network as well. I will probably be using this very soon for a small group of friends.

Pleroma

My experience with manually installing Pleroma a year ago was painful, so I had high hopes for this. Those expectations were met. The package for this installation was successful and the app seemed to work out of the box. Very pleased to see this only took a couple clicks to install.

Personally, I prefer Pleroma over Mastodon for a variety of reasons. Most of them being related to the implementation differences (Phoenix vs Rails). I'm a big fan of Elixir and like that Pleroma uses less resources than Mastodon. There are also no character limits... Using less resources on something like Yunohost is crucial. These two technologies are a good fit imo.

WriteFreely

WriteFreely is a simple blogging service (the one I use for this blog). It is written in Golang with a minimalist and clean design. Unfortunately the package did not install successfully so I couldn't try this out. When this gets resolved I might migrate this blog to Yunohost.

Mobilizon

Mobilizon is a Pleroma fork for coordinating events. This is something I need, and a lot of other people need. The app was able to install, however, the email Registration did not work. This is likely a fault of my own, however I had no issue with Riot. Not sure what deal is with that.

Custom Web App

I have not tried this feature out, but I wanted to point it out because I think it is neat. The real value Yunohost provides is not having to deal with server setup and certificates. So having a tool like Yunohost to host multiple sites with is really cool. Yunohost is really helpful for a hobbyist application developer trying to get things started. It is also possible to configure a simple redirect for a domain name to a particular url, even localhost. Could be fun to write a post detailing how to do this. This is where Yunohost starts venturing into Platform as a Service with these kinds of features.

Conclusion

Yunohost does a great job at lowering the barrier to entry for deploying all of these great open source applications. Since the domain stuff is still a bit tricky, I think that the target user is still someone with a technical background. By no means do they have to be a developer, just someone who isn't too afraid of Linux, the terminal, and DNS configs. If you are tired of annoying deployment setups then I recommend trying this out. There are a lot more bits and pieces I didn't cover here, like the intuitive UI and great branding. Try it yourself, you wont regret it.

Additional Sources

This post will layout the steps for setting up an internet radio using only Debian packages. The last part of it will go over how to create a Golang proxy server to make a fun website to host your stream.

One of my other hobbies besides programming is music. I love listening to music, making music, and even curating music. One thing I've always wanted to participate in is a silent disco, either as a listener or a DJ. If you aren't familiar with a silent disco, it is simply a group of people all listening to the same music via headphones. So I thought to myself how hard might it be to make one myself? Turns out either really hard or fairly easy depending on the approach you take.

My original idea was to basically stream my laptop's audio input to a browser via a web server using Golang. What I've learned is this is actually quite difficult to build from scratch. Thus resulting in me taking a different, but much easier approach. What I ended up doing was use already available Debian packages to do all of the heavylifting. The two packages I used were icecast2 and darkice.

Icecast

Icecast is the streaming server used to send streams to listeners. It is actually really cool, has a lot of features, and has been around for a couple of decades. The neat thing about is you can host the icecast server somewhere, and use a source client to stream audio to it from anywhere in the world. Icecast + source client (darkice) = internet radio. It is really that easy. On top of that, it has a dead simple installation process on Debian.

sudo apt install icecast2

When installing, a prompt comes up to set config params (mostly passwords). For experimenting purposes I used the default settings. However, you should certainly make the passwords more secure if you are wanting to put this in production. If you need to find the config file again, it is located at /etc/icecast2/icecast.xml. You will need root access to edit it. Once the installation is complete, the icecast server will run in the background. Now all you need is a source client to stream audio to it. That is where darkice comes.

Darkice

darkice is the software used to stream audio to icecast, also known as a source client. There are a lot of other source clients out there. And I can't remember exactly why I chose darkice (might've been the cool name), but it did seem to be somewhat popular, so I was able to find links for blogs and docs to get me started. darkice is also very easy to install on Debian:

sudo apt install darkice

For testing things out, I simply installed everything on one machine. The main thing to keep in mind is that darkice is the client, and icecast is the server. darkice has a little bit more of an involved configuration, and the official website is not super helpful in this regard. Luckily, I did find this post to be very helpful in getting my configuration going. The default behavior of darkice on Debian is to look for the config file at /etc/darkice.cfg. Upon installation, this file does not exist. So you have to write it yourself. Once again make sure you have root access.

sudo nano /etc/darkice.cfg

Here is what my config file looks like:

[general]
duration      = 0       # duration of encoding, in seconds. 0 means forever
bufferSecs    = 2       # size of internal slip buffer, in seconds
reconnect     = yes     # reconnect to the server(s) if disconnected
 
# this section describes the audio input that will be streamed
[input]
device          = default # OSS DSP soundcard device for the audio input
sampleRate      = 44100   # sample rate in Hz. try 11025, 22050 or 44100
bitsPerSample   = 16      # bits per sample. try 16
channel         = 2       # channels. 1 = mono, 2 = stereo

# this section describes a streaming connection to an IceCast2 server
# there may be up to 8 of these sections, named [icecast2-0] ... [icecast2-7]
# these can be mixed with [icecast-x] and [shoutcast-x] sections
[icecast2-0]
bitrateMode     = cbr     # constant bit rate, or else vorbis doesn't work well with Android
format          = vorbis     # format of the stream: ogg vorbis
bitrate         = 128     # bitrate of the stream sent to the server
quality         = 0.4
server          = localhost # server url
         
# host name of the server
port            = 8000      # port of the IceCast2 server, usually 8000
password        = hackme # source password to the IceCast2 server
mountPoint      = stream  # mount point of this stream on the IceCast2 server, 'stream' is default
name            = Fancy Cade Radio  # name of the stream

description     = Cool radio # description of the stream
url             = http://www.myradio.net # URL related to the stream
genre           = various               # genre of the stream
public          = yes                   # advertise this

Once that is saved. All you have to do is run darkice in the terminal:

darkice

If you navigate to localhost:8000/stream you will be able to listen to your stream. If you have other devices on the same wifi they can connect as well. All you have to do is replace localhost with your machine's hostname (sometimes only the ip address works). I chose to use ogg vorbis for the audio stream as opposed to mp3 because that is how I roll, Free Software all the way!

To get hostname:

hostname

To get ip address:

hostname -I

Grab some headphones and you have a silent disco in the comfort of your own home!

Golang Proxy Server

If you want to just use the front end provided by icecast then you can ignore the rest of this post. However, if you want to use the stream provided by icecast as a resource in your own website then this next part will show you how. I chose to use Golang because it is exceptionally good at making simple, lightweight web services with no external dependencies.

You can see an example of a finished product here. The code is hosted on Sourcehut (if you haven't checked out Sourcehut, you totally should).

The key ingredient is piping the icecast stream to a route on your server. Which is done like this:

func handleAudioStream(w http.ResponseWriter, r *http.Request) {                                                                                               
        read, write := io.Pipe()                                                                                                                               
                                                                                                                                                               
        go func() {                                                                                                                                            
                defer write.Close()                                                                                                                            
                resp, err := http.Get("http://localhost:8000/stream"); if err != nil {                                                                         
                        return                                                                                                                                 
                }                                                                                                                                              
                defer resp.Body.Close()                                                                                                                        
                io.Copy(write, resp.Body)                                                                                                                      
        }()                                                                                                                                                    
                                                                                                                                                               
        io.Copy(w, read)                                                                                                                                       
} 

We request the audio stream from the icecast server, and pipe the output to the ResponseWriter.

The route looks like this:

http.HandleFunc("/audio", handleAudioStream)

On the front end we can use the audio html element to play the stream.

<audio id="audio">                                                                                                                                         
  <source src="/audio" type="audio/ogg">                                                                                                                   
</audio>

Note: This will work on everything besides iOS devices. You can thank Apple for not following modern web standards.

Nginx Forwarding

When you want to take your silent disco to the global stage. You will have to do a couple of extra steps. Specifically, you will need to run icecast2 and the silentdisco app on a server. I chose to use nginx+lets encrypt to handle HTTPS requests, because it is battle tested and well documented. The important piece in this step is that darkice needs access to icecast2. However, if you setup the reverse proxy to only redirect to silentdisco then it won't work. My solution was to simply add a route to my nginx config, that would redirect the stream request to icecast instead of the silentdisco golang app. It Add these lines to your nginx server config:

location /stream {
    proxy_pass  http://127.0.0.1:8000;
}

Restart your nginx server:

sudo systemctl restart nginx

Now listeners from around the world can listen to sweet tracks!

Conclusion

I originally had the idea of making an audio visualizer. However, it ended up making the front end code overly complicated and a bit buggy. Specifically, the MediaElementSource would fail silently for whatever reason, and take the audio stream down with it. Not cool, man! Wanted to point this out, in case anyone had a similar idea. Overall, I'm happy with my minimalist design. It loads fast and is light on resources. Which is a bigger concern for mobile listeners. We don't want the party to stop just because of low batteries!

So there you have it. A DIY internet radio with free software, Debian, and a little bit of Golang. Hopefully, this post enables you to have some good times with your friends!

This is fancycade, signing off.

In this post I wanted to go over a few CLI utilities written in Rust that I am excited about. There are a lot of different Rust CLI tools out there, but I wanted to focus on ones that replace some of the GNU Core Utilities with Rust alternatives. The tools I will be talking about are exa, ripgrep, and bat.

You may be wondering why would you need alternatives for trusted utilities that have been around for decades? Well, a few reasons. First, they are written in Rust! Which for me is reason alone to use them. Secondly, some of these tools are makeovers with pretty colors and sane defaults, and sometimes they are even faster than the original. Some of the effort behind these tools goes to support the RedoxOS project which I highly recommend checking out if you haven't.

Side note: If you want to follow along I'm assuming you are using the Debian operating system. You can also install these tools with cargo directly.

Side note 2: Unfortunately, I wasn't able to show the proper colors for these tools, because of the limitations of my blogging skills. Just keep in mind that each of these tools has great color schemes that significantly help readability.

exa

exa is a cli tool for listing files in a directory. It serves to be an alternative to the ls command. The improvements I would consider are nice colors, sane defaults, and extra goodies.

It can be installed like so:

sudo apt install exa

Besides using ls to list files you can also see how big a file is. At my job I work with a lot of large csv and json files, so this comes in handy. The only downside to this is that to list the filesizes as not bytes you have to do this incantation:

fancycade[test_dir]: ls -l --block-size=M
total 1M
-rw-r--r-- 1 shiba shiba 1M Sep  1 14:25 data.csv
-rw-r--r-- 1 fancycade fancycade 0M Sep  1 14:03 README
drwxr-xr-x 2 fancycade fancycade 1M Sep  1 14:03 src

Which only gives the file size in Megabytes. Since it rounds down or up it doesn't giving clear information as to the actual size of your file.

Whereas exa chooses the appropriate unit based on the file size without listing the unit, while also being much easier to remember.

fancycade[test_dir]: exa -l
.rw-r--r-- 786k fancycade  1 Sep 14:25 data.csv
.rw-r--r--  182 fancycade  1 Sep 14:24 README
drwxr-xr-x    - fancycade  1 Sep 14:03 src

Another feature that exa has that ls doesn't have is the ability to list the files in a directory as a tree. Normally this would be done with the tree utility, but now we don't even need to install it if we have exa. For example:

fancycade[test_dir]: exa --tree
.
├── data.csv
├── README
└── src
   └── generate_data.rb

Typical tree representation coming from the same tool. exa is pretty sweet, highly recommend giving it a try.

ripgrep (rg)

ripgrep is a Rust alternative to the well known tool grep. Who could've guessed?! Now this tool is one that I'm the least familiar with because I'm not a master grep user. However, I have found a few use cases that I find to be very helpful. The cool thing about ripgrep is that it is actually faster than the original. Which is crazy, since grep was always touted as extremely fast.

To install ripgrep you simply do this:

sudo apt install ripgrep

Now to use ripgrep we type the alias rg. Imagine a scenario where you are looking for a file in a directory that has a lot of stuff in it. You run exa and all you see is a wall of text. One clever trick is to pipe the output of exa into ripgrep to filter only the relevant filenames. So if I know I'm looking for a csv file, I can do something like this:

fancycade[test_dir]: exa | rg csv
data.csv

Instead of printing out all the contents of the current directory it only outputs files that include csv in the filename. This is only the tip of the iceberg of what you can do with this tool, so I highly suggest experimenting and trying things out.

bat

bat is a Rust alternative to the cat utility with a few extra goodies. It also combines the features of less as well, so it can be considered a cross between the two. The problem with cat is that it can be hard to read large files. Which is why less is useful. On top of that bat provides formatting for a lot of common filetypes like markdown or source code. And line numbers! Which is really helpful when looking at tabular data such as csvs.

Unfortunately, as of this post, bat isn't in Debian stable (buster). However, there is a workaround, and it will probably be available in Ubuntu fairly soon. This workaround is not typically recommended by the Debian community because it does not respect their ideas of a “stable” version. So make sure to do this with caution.

We begin the process by editing our /etc/apt/sources.list with nano (make sure you have proper permissions):

sudo nano /etc/apt/sources.list

Add this line to the file:

deb http://deb.debian.org/debian/ sid main

This allows apt to get packages in the Debian unstable (aka sid) archive.

You have to update the package lists:

sudo apt update

Now you can install bat:

sudo apt install bat

As I mentioned earlier, it is not recommended to use packages from Debian unstable so I highly suggest removing the line we added to the /etc/apt/sources.list because we don't want to accidentally grab other unstable packages later.

Once we have bat installed we can get started. We can look at a csv file (list of popular baby names) like this:

fancycade[test_dir]: bat data.csv
───────┬──────────────────────────────────────────────────────
       │ File: data.csv
───────┼──────────────────────────────────────────────────────
   1   │ Year of Birth,Gender,Ethnicity,First Name,Count,Rank
   2   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Olivia,172,1
   3   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Chloe,112,2
   4   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Sophia,104,3
   5   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Emma,99,4
   6   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Emily,99,4
   7   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Mia,79,5
   8   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Charlotte,59,6
   9   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Sarah,57,7
  10   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Isabella,56,8
  11   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Hannah,56,8
  12   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Grace,54,9
  13   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Angela,54,9
  14   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Ava,53,10
  15   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Joanna,49,11
  16   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Amelia,44,12
  17   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Evelyn,42,13
  18   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Ella,42,13
  19   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Arya,42,13
  20   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Ariana,40,14
  21   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Maya,39,15
  22   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Alina,39,15
  23   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Fiona,35,16
  24   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Ashley,34,17
  25   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Anaya,34,17
  26   │ 2016,FEMALE,ASIAN AND PACIFIC ISLANDER,Fatima,34,17
:

Notice the line numbers on the left. Like less we can navigate the file with up/down arrows and exit by pressing 'q'.

bat is really great, and has become something I use on a daily basis. It is amazing what subtle differences in tooling can do to your workflow. Hopefully, you found this post interesting, and are excited to try cli tools written in Rust like I am.

This is fancycade, signing off.