Ente meets Podman - A local development setup

[mnvr]

I've been using Podman for my local containers recently, and I moved my local Ente development setup to use podman instead of Docker. Some notes of how I got it to work (stream of thought) in case they're useful for other folks.

Iteration 1

There is a podman compose script, but it (apparently) is not given the same love by the Podman project as podman itself, and seems to be just for for porting over trivial docker compose files, but it doesn't guarantee full docker compose emulation. In particular, the current ente/server/compose.yaml uses a config block to pass configuration, and post_start lifetime script, neither of which is currently supported by podman compose.

Still, with minor changes and some manual setup I was able to make it work with podman compose.

Iteration 2

Next up, I started from scratch and tried using podman's native concept - pods. The great thing about these is that they are rootless (i.e. they don't require sudo to run).

1. Create a new "pod" named "ente-server" (arbitrary name), and expose ports 8080 (museum) and 3200 (minio) on the host's interfaces.

podman pod create -p 8080:8080 -p 3200:3200 ente-server

Note that at this point, this is just an empty pod, no containers are running inside it.

2. Create the Postgres container inside our pod.

podman run --detach --replace --pod ente-server --name postgres -e POSTGRES_USER=pguser -e POSTGRES_PASSWORD=pgpass -e POSTGRES_DB=ente_db docker.io/library/postgres:15

Warning

This is NOT FOR PRODUCTION USE. Do not deploy this. This is only for creating local development environment. e.g. notice how I don't bother where the volume (postgres data) goes, which obviously is not what we'd want in a real deployment. Same goes for the other things I talk of here.

3. Create the minio container inside our pod.

podman run --detach --replace --pod ente-server --name minio --replace -e MINIO_ROOT_USER=changeme -e MINIO_ROOT_PASSWORD=changeme1234 docker.io/minio/minio server /data --address ":3200" --console-address ":3201"

4. Create the bucket we'll use (I only needed one for my work since replication is not involved)

podman exec -w /data minio  mc mb -p b2-eu-cen

5. Build a museum image.

git clone https://github.com/ente-io/ente
cd ente/server
podman build --build-arg GIT_COMMIT=`git rev-parse HEAD` --tag ente-server-museum .

6. Create a museum.yaml with the following contents:

apps:
    public-albums: http://localhost:3002

db:
    host: localhost
    port: 5432
    name: ente_db
    user: pguser
    password: pgpass

s3:
  are_local_buckets: true
  b2-eu-cen:
      key: changeme
      secret: changeme1234
      endpoint: localhost:3200
      region: eu-central-2
      bucket: b2-eu-cen

internal:
    hardcoded-ott:
        local-domain-suffix: "@ente.io"
        local-domain-value: 123456
    # If this is not set, as a fallback, the first user is considered an admin.
    # admin:

7. Create a museum container

podman run --detach --replace --pod ente-server --name museum -v `pwd`/museum.yaml:/museum.yaml:Z ente-server-museum

That's it.

$ curl localhost:8080/ping
{"id":"8fa63ba9fb2d38599a0c28f334e21a5ec3a6ea67","message":"pong"}

Iteration next

This is kind of nice - no dealing with socat etc to forward ports. It is spread out across multiple commands though, but I think there will be ways to deal with that nicely (using a script or podman play).

Planning to use the current setup a bit more, and see if there is some thing we can add upstream in Ente's server directory itself to make it quick and easy.

[mnvr]

Iteration 4

The state of the running pod can be saved in a Kubernetes compatible format, and then restored with just one command:

1. Save the state (after running all the commands above):

podman generate kube ente-server > data/ente-server.yaml

2. Stop and remove the pod we manually created

podman pod rm ente-server -f

5. Recreate it

podman play kube data/ente-server.yaml

Now the next time I want to start the same cluster again, I don't need to run all the commands again, I can just podman play kube data/ente-server.yaml.

This is not saving the volumes, maybe I need to modify the commands to use named volumes so that they are preserved between runs.

[mnvr]

I added named volumes to the postgres (--volume postgres-data:/var/lib/postgresql/data) and minio (--volume minio-data:/data) so that the DB and S3 state is preserved across pod restarts.

Giving me this workflow:

1. Initial setup:

#!/bin/sh

podman pod create -p 8080:8080 -p 3200:3200 ente-server
podman run --detach --replace --pod ente-server --name postgres --volume postgres-data:/var/lib/postgresql/data -e POSTGRES_USER=pguser -e POSTGRES_PASSWORD=pgpass -e POSTGRES_DB=ente_db docker.io/library/postgres:15
podman run --detach --replace --pod ente-server --name minio --volume minio-data:/data -e MINIO_ROOT_USER=changeme -e MINIO_ROOT_PASSWORD=changeme1234 docker.io/minio/minio server /data --address ":3200" --console-address ":3201"
podman exec -w /data minio mc mb -p b2-eu-cen
podman run --detach --replace --pod ente-server --name museum -v `pwd`/museum.yaml:/museum.yaml:Z ente-server-museum 
podman generate kube ente-server > data/ente-server.yaml

2. Starting the pod: podman play kube data/ente-server.yaml

3. Stopping the pod: podman pod rm -f ente-server

Step 1 is only needed the first time around. Thereafter I can just do 2 and 3 repeatedly as desired.