Syncing the Archives via IPFS

Introduction

This article will show you how you can sync our archive of audiogames on IPFS to your node, to ensure high availability should a hosting site go down. Be sure you have enough space for the files, and are willing to dedicate the space to mirroring the files.

Procedure in Brief

These instructions are going to require a bit of elbow grease in the command line, so be prepared to get into a console or a terminal emulator. In brief, you need to:

install an IPFS daemon. For this wiki, I am assuming Kubo or go-ipfs, depending on how your package manager packages things.
Initialize your demon.
Write a user service to start IPFS, or enable it on your system if such a service is available.
Be sure that port 4001 is accessible, both by UDP and TCP.
Start your demon.
Pin the directory resolved by way of IPNS.
Optional, but really recommended: mounting the archive into MFS.
Sync the archive.

Installing the Daemon

This step will vary, depending on how your distribution packages things. Here are some of the common ones that I did a bit of research for.

Arch Linux: Available as kubo.
Void Linux: Available as go-ipfs.
Slint/Slackware and Other Distros I Hadn't Thought of: Consult the instructions at https://github.com/ipfs/kubo

Initializing your Daemon

This is pretty easy, all you need to do here is:

ipfs init

Opening the Ports

In order for your node to be able to talk to other nodes on the IPFS network, you will need to allow port 4001/tcp and 4001/udp through your firewall. In addition, you may also need to forward those ports to the box should UPNP not be working.

Start or Create a Service

For some distributions, a user service is already shipped, so starting the daemon is pretty straightforward. Consult the specifics of your init system and be sure it is running now and enabled on boot. If a service is not available, be sure that you run the following as your user, editing the user services where necessary:

ipfs daemon --enable-gc --migrate --enable-namesys-pubsub

In brief, these extra flags allow for subscribing to published events about names, allowing migrations on daemon startup, and allows for the newer pubsub system.

Sync the Archive

Now that yu have the IPFS node running, you can now get the archive and pin a copy locally by way of its IPNS key.

ipfs pin add /ipns/games-archive.stormux.org

Copy the Archive into Your MFS for Easier Perusal

If you want an easier way to parooze the archive, you can put it into your MFS (Mutable File Storage). I believe there is a way to mount this onto your filesystem by way of fuse, but I am not sure if it works, as it is an experimental feature. Regardless, you'll want to do this so that you can update the records easier (you need to keep track of the old hash for the pin to update, since ipfs pin update will delete the old hash and pin the new one). So here is how you do this:

ipfs files cp $(ipfs name resolve /ipns/games-archive.stormux.org) /games-archive

Now you can do ipfs files ls /games-archive instead of a longer path.

Syncing the Archive

To ensure you have the latest archive, run the pin on a periodic interval. Additionally, if your node is slow in updating, as mine was, you can resolve the resolve the ipns entry by hand and pipe that into pin add.

ipfs pin update $(ipfs files stat --hash /games-archive) $(ipfs name resolve --nocache /ipns/games-archive.stormux.org) && ipfs files rm -r /games-archive && ipfs files cp $(ipfs name resolve /ipns/games-archive.stormux.org) /games-archive

To automate this process, you can add it to your crontab:

crontab -e
# ipns update for audiogame-manager once a week
0 12 * * 1 ipfs pin update $(ipfs files stat --hash /games-archive) $(ipfs name resolve --nocache /ipns/games-archive.stormux.org) && ipfs files rm -r /games-archive && ipfs files cp $(ipfs name resolve /ipns/games-archive.stormux.org) /games-archive