I need to setup wal-e on my home server because recovery plans are good.
First I setup a new S3 bucket and IAM user for it
Then I started following the README and the following blog post.
I have modified some of the instructions to use a python virtualenv because I like clean environments.
umask u=rwx,g=rx,o= sudo mkdir -p /etc/wal-e.d/vars sudo su echo "my secret" > /etc/wal-e.d/vars/AWS_SECRET_ACCESS_KEY echo "my access key" > /etc/wal-e.d/vars/AWS_ACCESS_KEY_ID echo "us-west-1" > /etc/wal-e.d/vars/AWS_REGION echo 's3://mybucket/wal-e' > /etc/wal-e.d/vars/WALE_S3_PREFIX virtualenv env -p python3.8 source env/bin/activate pip install wal-e[aws] envdir chown -R root:postgres /etc/wal-e.d
First and foremost I want to make sure I can push a full backup via cron
sudo su postgres /etc/wal-e.d/env/bin/envdir /etc/wal-e.d/vars /etc/wal-e.d/env/bin/wal-e backup-push /virt/postgresql_data/postgresql/10/main/
I got this error, so I installed lzpop and pv
postgres@pelennor5:/etc/wal-e.d$ /etc/wal-e.d/env/bin/envdir /etc/wal-e.d/vars /etc/wal-e.d/env/bin/wal-e backup-push /virt/postgresql_data/postgresql/10/main/
wal_e.main INFO MSG: starting WAL-E DETAIL: The subcommand is "backup-push". STRUCTURED: time=2020-02-08T22:28:37.586041-00 pid=25452 wal_e.main ERROR MSG: could not run one or more external programs WAL-E depends upon DETAIL: Could not run the following programs, are they installed? lzop, pv STRUCTURED: time=2020-02-08T22:28:37.693263-00 pid=25452 postgres@pelennor5:/etc/wal-e.d$ exit root@pelennor5:/etc/wal-e.d# sudo apt install ^C root@pelennor5:/etc/wal-e.d# apt install lzop pv
Next time my cron-capable command worked from the postgres user, which suggests a few things are going well.
Now I need to edit postgresql.conf
archive_mode = on archive_command = '/etc/wal-e.d/env/bin/envdir /etc/wal-e.d/vars /etc/wal-e.d/env/bin/wal-e wal-push %p' archive_timeout = 60
sudo systemctl restart postgresql then seemed to work…
After that, I setup the base backup on crontab for postgres. All set on the backup side.
Now I need to test a restore to prove to myself this is reasonable, so I am going to setup postgres10 in docker on my laptop and try to restore from S3.
8th February 2020
I won't ever give out your email address. I don't publish comments but if you'd like to write to me then you could use this form.
I'm Issac. I live in Oakland. I make things for fun and money. I use electronics and computers and software. I manage teams and projects top to bottom. I've worked as a consultant, software engineer, hardware designer, artist, technology director and team lead. I do occasional fabrication in wood and plastic and metal. I run a boutique interactive agency with my brother Kasey and a roving cast of experts at Kelly Creative Tech. I was the Director of Technology for Nonchalance during the The Latitude Society project. I was the Lead Web Developer and then Technical Marketing Engineer at Nebula, which made an OpenStack Appliance. I've been building things on the web and in person since leaving Ohio State University's Electrical and Computer engineering program in 2007. Lots of other really dorky things happened to me before that, like dropping out of high school to go to university, getting an Eagle Scout award, and getting 6th in a state-wide algebra competition. I have an affinity for hopscotch.