Gpodder, Rust and Domain Driven Design

Recently I’ve gotten into listening to podcasts more, notably the Severed podcast going in depth on every Severance episode. I use Kasts on my computers and AntennaPod on my phone to listen, and I looked into how I could sync my subscriptions between my devices. This search ended with Gpodder.net, a free and open-source web service that’s supported by both AntennaPod and Kasts! The server is also self-hostable, but it’s quite the behemoth written in Python, and I felt like starting another project, so I decided to implement my own version of the API instead. That project is Otter.

The Gpodder API

The API documentation for Gpodder is available on Read The Docs. The documentation is okay, but it lacks a lot of important information about how the API should behave. My development process for this project consisted of trying to implement routes via the documentation before testing it using Kasts. This took a lot of trial and error, but I ended up getting most of the routes working the way most clients expect (I hope).

Currently Otter supports all routes required for clients to synchronize subscriptions and play information for episodes. I’ve based some of my work on Opodsync, another implementation of the same idea, written in PHP.

Domain Driven Design

Usually the way I develop projects is very chaotic. I try out different things to see what sticks and often end up with very interconnected and difficult to separate components that join together into a big mess. Usually this does clean up after some time, but I still have a tendency to develop too tightly connected components. One example of this would be having ORM logic in web route handlers, or jumping through hoops to be able to use the same structs for the database operations and the web representation. For Otter, I took a step back for once.

I took some inspiration from domain driven design and clean architecture to structure my codebase. The idea (at least, the way I understand/interpret it) is to work with abstractions that cleanly separate various components. The first step was designing said abstraction. I introduced the concept of a “Gpodder repository”, which is an object that provides all the necessary methods required to function as the backend data store for a Gpodder server. This repository is defined as a collection of traits:

pub trait EpisodeActionRepository {
    /// Insert the given episode actions into the datastore.
    fn add_episode_actions(&self, user: &User, actions: Vec<EpisodeAction>)
        -> Result<i64, AuthErr>;

    /// Retrieve the list of episode actions for the given user.
    fn episode_actions_for_user(
        &self,
        user: &User,
        since: Option<i64>,
        podcast: Option<String>,
        device: Option<String>,
        aggregated: bool,
    ) -> Result<(i64, Vec<EpisodeAction>), AuthErr>;
}

For example, the EpisodeActionRepository provides the methods required to serve the Episode Actions API section of the API documentation.

async fn get_episode_actions(
    State(ctx): State<Context>,
    Path(username): Path<StringWithFormat>,
    Extension(user): Extension<gpodder::User>,
    Query(filter): Query<FilterQuery>,
) -> AppResult<Json<EpisodeActionsResponse>> {
    if username.format != Format::Json {
        return Err(AppError::NotFound);
    }

    if *username != user.username {
        return Err(AppError::BadRequest);
    }

    Ok(tokio::task::spawn_blocking(move || {
        ctx.repo.episode_actions_for_user(
            &user,
            filter.since,
            filter.podcast,
            filter.device,
            filter.aggregated,
        )
    })
    .await
    .unwrap()
    .map(|(timestamp, actions)| Json(EpisodeActionsResponse { timestamp, actions }))?)
}

In our web layer (here I’m using Axum), all that needs to be done is to call the respective function on the repository and convert it to the correct representation format, as required by the API.

This complete separation of concerns makes development much less mentally taxing in my opinion, as each component can be viewed on its own. The repository abstraction defines models and methods required for a representation layer (in this case, the web server) to access the various parts of the repository without having to know anything about the internal workings. The current implementation of the repository works using Sqlite, but thanks to the abstraction, I could add a Postgres implementation as well by simply reimplementing the abstraction. Rust’s type system adds to the strength of this design pattern by providing strong abstraction primitives via traits.

Configuration

One other fun thing I’d like to mention is how Otter handles configuration variables. In my eyes, the ideal application supports configuration as a combination of a config file, environment variables and CLI arguments, with environment variables overwriting config file variables, and CLI arguments taking precedence above all. Using Serde, Clap and Figment, this can be done in a very clean way.

First, we define our configuration as a struct. This is both the expected format of the configuration file, and the final configuration struct that we wish to end up with:

#[derive(Deserialize)]
pub struct Config {
    #[serde(default = "default_data_dir")]
    pub data_dir: PathBuf,
    #[serde(default = "default_domain")]
    pub domain: String,
    #[serde(default = "default_port")]
    pub port: u16,
}

fn default_data_dir() -> PathBuf {
    PathBuf::from("./data")
}

fn default_domain() -> String {
    "127.0.0.1".to_string()
}

fn default_port() -> u16 {
    8080
}

Important here is the Deserialize derive. The Figment library heavily relies on Serde to function, and a Deserialize implementation is required for our final configuration to be deserialized into this struct. We also define sensible defaults where applicable.

The second step is defining our CLI arguments using Clap:

#[derive(Serialize, Args, Clone)]
pub struct ClapConfig {
    #[arg(
        short,
        long = "config",
        env = "OTTER_CONFIG_FILE",
        value_name = "CONFIG_FILE"
    )]
    config_file: Option<PathBuf>,

    #[serde(skip_serializing_if = "Option::is_none")]
    #[arg(long = "data", value_name = "DATA_DIR")]
    data_dir: Option<PathBuf>,

    #[serde(skip_serializing_if = "Option::is_none")]
    #[arg(short, long, value_name = "DOMAIN")]
    domain: Option<String>,

    #[serde(skip_serializing_if = "Option::is_none")]
    #[arg(short, long, value_name = "PORT")]
    port: Option<u16>,
}

Here we see two things of note. First, the ClapConfig implements Serialize, not Deserialize. This is because we’ll be using the Serialize implementation later when combining the configuration sources. We also add a #[serde(skip_serializing_if = "Option::is_none")] attribute to each variable that can be overwritten. This tells Serde to ignore the variable altogether if its value is None as a workaround for Figment not being able to ignore None values. Important here is how the variables names in the ClapConfig struct are the same as those in the Config struct. This is required for Figment to be able to combine these configuration sources.

Finally, we combine everything using Figment:

let mut figment = Figment::new();

if let Some(config_path) = &self.config.config_file {
    figment = figment.merge(Toml::file(config_path));
}

let config: crate::config::Config = figment
    .merge(Env::prefixed("OTTER_"))
    .merge(Serialized::defaults(self.config.clone()))
    .extract().unwrap();

We start by defining an empty figment. In the Figment library, a “figment” is a combination of configuration providers that can be deserialized using Serde. A provider is an object that can provide configuration variables from some source, be it a TOML file or environment variables. Figments can be combined with providers in various ways. Here, we make use of “merge”, which overwrites any existing configuration variables with the values received from the new provider.

We merge the empty figment with the Toml provider, which reads a TOML file for configuration variables. Then, we merge again, first with the Env::prefixed("OTTER_") provider, and then with the Serialized::defaults(self.config.clone()) provider. The Env provider reads variables from environment variables, while Serialized provides variables by serializing a struct, in this case our ClapConfig struct (hence why we need a Serialize implementation). The result is a figment consisting of variables from the config file, environment variables and CLI arguments. This can be deserialized into our Config struct using extract.

In my opinion, all of this combines into a very clean configuration system that allows configuration variables from all convenient sources, without sacrificing flexibility or resorting to boilerplate code.

I hope to deploy a first build of Otter on my own servers in the near future to test it and work out any kinks. Afterwards, I’ll announce a 0.1 release!

Thanks for reading.